NOTE TO USERS
This reproduction is the best copy available.
Diagnmtics for Generalized Lin-
Models
Sonia Benghiat
A Thesis in
The Department of Mathematics ancl Statistio;
Presenteci in Partial FWillment of the Requirements for the Degree of M a s e r of Science at
Concordia UniversiS. Montreal, Q u e k , Canada
Natianal Library
Bibiiotheque nationale du Canada
Acquisitions and Bibiiiraphic Services
Acquisitions et sewices bibliographiques
The author has granted a nonexclusive licence dowing the National Library of Canada to reproduce, loaa, distribute or sell copies of this thesis in microform, paper or elechmiic formats.
L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la forme de microfiche/fiIm, de reproduction sur papier ou sur format électronique.
The author retains ownership of the copyright in this thesis. Neither the thesis nor substantial extracts fiom it may be printed or otherwise reproduced without the author's permission.
L'auteur conserve la propriété du droit d'auteur qui protège cette thèse. Ni la thèse ni des extraits substantiels de celle-ci ne doivent être imprimés ou autrement reproduits sans son autorisation.
Abstract Diagndics for Generalieed Linear Models Sonia Benghiat The analysis of residuals c m capture departures from a parametrized model. In
this thesis we look at how the generalid Iinear model has become one of the m a t important developments in statistics in the las* thirty years, anci on the aciequacy of regessiion m d e l diagnostics that are meaningfiù and sienificaat in a generalimcl linear
model context. Some aymptotic pmperties are di-
and numerical examples are
providl to ilinstrate the techniques for binomial, Poimon, and g a m m a clistributed
random variables.
Résumé Des diagnostiques pour les modèles
généralisés
Sonia Benghiat L'analyse des résidia est un outil fort puhant qui nous permet de vérifier la va-
lidité d'un moc.èleparamètnquc. Dans ce mémoire, je donne un aperçu de hqmtance que les modèles linéaires généralisé; ont eu sur le déroulement des statistiques daas
les trentes dernières années. J'analyse la facilité que nous procurent de tels modèles 1orsqii'i.l s'agit des dinpostiques de régressiom. J'éxamine également les lois acc yniptatiques cmnoeniant ce8 modèles. Finalement, je présente des exemples pour des
variables aléatoires b'moniiales, Poisson, et gamnm-
Acknowledgements This thesis mdd not have k m passible without the patience and the boundless support £rom rny h u s h d .
To h I one a debt of gratitude. M y parents, niy brother
anci my sister continuously remindeci me of the importance of completing my mas%ers degree aucl to them 1 am thankfd for their peMis-tent encouragements. 1 hold a peat respect for mv supervisor Prof- Y. Chaubey. He very patiently guided the
advancenients of th thesis. To hini 1 express m y sincerest gratitude. 1 woiild also
like to thank Prof. J. Carrido who wjlhgly provided me with some usefiil materjal for the realizatiou of tbis thesis. 1 thank Prof. A. Canty for kindly acxxpting to advise me on the choice of my software application. 1 thank the graduate ~ecretariesand the proft?ssorsfrom the Mathematics and Statistics department, anct my clasmates, not least, for their insightfui help and for dering a pleasant 1eaRLing environnient altogether.
Contents 1 Introduction
1
............................. 1.1.1 V ' v of Ass~mptions . . . . . . . . . . . . . . . . . . . . . 1.1.2 a b e r Diagnmtics . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.3 Remdial Mt?i~.%ues. . . . . . . . . . . . . . . . . . . . . . . . 1.2 Outliue of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The Linear M d e l
2 The Generatized Linear Mode1
2.1 Historicril A S S A S. 2.2
............................
1 5 10
12 16 17 17
hiean and Variance Functions in au
............................
19
2.3 h n p t i ~ ~ of the Generalized Linear Mode1 . . . . . . . . . . . . . .
20
Expnential F d y 2.4
M a x i m m Likelihood Estimation for the GLM
25
2.4.1
................................ The Newton-Raphson Methoù . . . . . . . . . . . . . . . . . .
29
..................... ........... 2.4.3 lteratively Weighted Least Squares (W) The G o o h e s of M d e l Fit . . . . . . . . . . . . . . . . . . . . . . .
29
2.4.2
2.5
Fisher's Smring Methd
31 34
2.6
2.5.1
The Deviance Function
......................
35
2.5.2
The Pearson StatWtic . . . . . . . . . . . . . . . . . . . . . . .
36
2.5.3
MdualsandtheProjectionMatrix
.............. ............................
36
Alternative hlodels
3 Residual Diagnostic Measures
38 43
............................
43
3.2 Muential Observations . . . . . . . . . . . . . . . . . . . . . . . . .
50
3.3 Tésting the Chdnesof-Fit
52
3.4
....................... Testing Goodnesof-Link hc.tions . . . . . . . . . . . . . . . . . . . Software Apyliratims . . . . . . . . . . . . . . . . . . . . . . . . . . .
55
3.1 Modifieci Rtsiduals
3.5
4 Numerical Examples
GO
62
................................ Binomial Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Poisson Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gamma Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1 Intrdiiction
62
4.2
62
4.3 4.4
4.5
6'3 76
81
A Progréuns for Parameter Estimation for DifFerent Families A.l MLE program for binomial f d y . . . . . . . . . . . . . . . . . . . .
82
....................
85
MLE program for G a m m a f d y . . . . . . . . . . . . . . . . . . . .
88
.............................
91
A.2 MLE program for Poison f d y A.3
A.4 Onestep hinction
B
82
92
B.1 Output for the Herbicide data . . . . . . . . . . . . . . . . . . . . . .
92
B.2 Output for One-Step fiulction using the Herbicide data
... .. . . .
93
List of Figures 4.1 Deviame residid pmirue
4.2
for birth abnorrualities due to herbicide spray ex-
...................................
x residnals for birth abnonualities due tn herbicide spray exyosure .
4.3 Projection matrix cliagonai elements for birth abnormalities due to
herbicide spray qxmue 4.4
Standardizerl change in
..,... .... . .- .... ..... . .. for b ï ï aabormalities due tu herbicide
4.5
. .. . - . .. . . . .. . . . . . . .. . . .. . - . . . for herbicide data . . . . . . . . . . . . . . stand^^ diange iu
4.6
Devimce residiiâls for defects found on furnitlue producecl in a certain
spray expcmre
manSacturing plant 4.7
... . . . . . ..... .. .... ... . . . ..
X. residuais for defects f o d on furnitun! prociuecl in a certain nianu-
fachuhg plant
... . ... . .. ....... . .... . .... . ,.
4.8 Projection ruatrix diagonal elements for defects found on furniture prw
d u c d in a certain m1if8c.turingplant 4.9
.. .. . ....... . .. ..
Staudaràized diange in & for defecb fond on furnihue produced in
.. ........ .... ... . . .. . for furnifame damage data . . . . . . . . . for furniture damage data . . . . . . . . .
a certain manufadurhg plant 4.10 Standardid change ix&
4.11 Standardized change ir&
4-12 ~tandardiz~dcbangeïn&forfurnituredarnegedata. . . . . . . . .
b4 for furniture damage data
......... 4.14 Standarûizeci change in for furnitute cîarnage data . . . . . . . . . 4.15 Deviane residuals for lot1 of b1ooddot time . . . . . . . . . . . . . . 4-16 x resid~ialc;for lot1 of bloodclot t h e . . . . . . . . . . . . . . . . . . 4.13 Standarclizerl change in
change in
4.19 Standarclizecl change
75 75
78
79
....
79
A for lot1 of b l d c l o t time
.........
80
for lot1 of bloodclot time
..........
80
4.17 Projedon ~lliitxixdiagonal elements for lot1 of b l d c l o t time 4.18 Stand=-
74
List of Tables 2.1
Dispersion Pamrricter. Canoniaai tir&
and Viricitux Function for LLs-
................... 2-2 Dictributiorr Functiorrs un'tlr h i r Assocàatd Ltrrk.9 . . . . . . . . . . . 2.3 An Extension of the N o d - T h e o r y L h m r M& to the GLM . . . . 2.4 Dtxriaficx Furtctiorr fm Ezponctriid Famdy DiPtributions . . . . . . . . tnbutioris of the Exponer~tidFundy
3.1
Anscombe und VaMnce-SLa6iiiting R e S i d d s Ezpmssed for the
nomial. P o i ~ s o nand Gamma d%sttibutions
3.2 D ( r ~ t * uarui ~ ~ rAdjusted ~ Deviorux R d & 4.1
...............
24 41 41 42
Bi48
for the T i m e Dhtributior~~ 49
h r n b c r of birth thornuditics out of total b i r t ? ~pcr nronth for hcrli-
4.2
................................. Contingeney table for -tu= defect . . . . . . . . . . . . . . . . . .
4.3
Bloml dotting tzmw in seconds for 9 perr;entage wncentmtiong of p l a m
cidceflcct
andfor2lots
...............................
63 69
76
Chapter 1
Introduction 1.1
The Linear Mode1
Most of the generalized linear mode1 cont;epts Stern from the theory of the normal linear model. Before intrcducbg the generalized linear model, it is itsefirl to set the scene by providiy: a brief review of the nomal linear mode1 in this hrst chapter, and
hence to d e r s t a n d anti see the para1leL.s between the two types of mciclels.
The normal-theory linew m d e l is given by
where y is an n x 1 observation vector, X is a n x p known design matrix,
P is a p x 1
vector of unklluwn parameters, d e d regression parameters anri e is an n x 1 vedor
of unobserveci random Vanables w i t h zero mean and constant varianc'~2,whkh are
independently anci noroi;ùly dishibuteci. The muùel (1.1) is alternatively described
by the ruean-vector anci varianc~(YIVBfi8nceniatrix of the obrvatious y as
CHAPTER 1. INTRODUCTlON
2
The linearity of the mode1 is understoal in temis of the regrt&on parameters P. For estimation of the paramet-,
the
error~are
the maicinnun W r & d
method can be iised when
normal. LikewWe, the principle of ieast squares provicies the same
estimates of the regression parameters. However, it does not require any distributional
asmmptiou. It is d e ~ T i M klw.
Least Squares Estimation of Parameters fl
The 1-t
squares methoci estimates the regression parameters fl by minimizing the
suni of squares:
= y'y - 2CIX'y
In additiou to being i properties:
+ gx9Cp.
the least ~ qm estuiiator , (LSE) /j,
~
the foilowhg
(1) have niinini~mvariarice mong ail unbiased linear estiniatars (GaussMarkm theoreni), ( 2 ) consistent, auci
Projection Matrix and Residuals
The builcihg blocks for cletecting influentid oticsewatioris in a giveri data are generatd
by the projection matrù, M,anci r e s i d d , e which are d & d
in what follows.
Chmider the mode1 (1.1) with correspondhg fit;td values (9) and r & i d
vec%or(e)
dehed by:
The projection matrix M = (r*,) is definecl by:
is c d e d the " k t matrïxn. The projection 1~1;8trix is niast usefiil in the d . y s i s of
r e s i d d s as it spam the r & d d space,
The residU
Le.,
e memue the Merence between the obse~edanci the fitted values,
with the f o l l h g pruperties:
0
Var(e)= 02(1 - H).
An d i a s e d e s t b a t o r of d l>ased on the residual e is given by
whereby (1.8)is denoted by M S E , the nurrn s~urrriedue to c m r . Therefm,
Vm(e)= M S E (1 - H)
Theorem 1.1 The follouing am important p m p d i e . 5 &ted triz M: 1.
H and M = (1- H) are symmetric and idempotent,
2. m n l - M = r n n k ( I - H ) = t r ( M ) = t r ( I - H ) = n - p ,
and
with the pmjection nta-
CHAPTER 1- INTRODUCTION
5
2. Since (1 - H)is idempotent, r d ( I - H) = tr (1- H).Fbtherrnore, since
It can be further deducecl that
In fitting a linear rqpsion malel, the &duah e c m be uyed to j u s t e the auvmp tions about the ranciou e m m r. Since e ir iinear in y, e iu a nmd011lvariable f o U h g
a normal distribution, and hence the assumption of m d t y can be used to draw inferences about the h e a r model. TbuY, an anal*
which combines the d d u a l s
and the f i t d values will examine whether there are any departuns h m the linear
mode1 with n o d errors. The mode1 departimes to be examined are categorized as :
a non-constant variance,
non-independence,
omission of independent covariates. Graphid methoch ( s e Draper rrml Smith [7],Chapter 4), involvirig the residuak provide iisefiil tmls fur detecting s u i mode1 departures. They are describai below: 1- Plots of rwiduals agabt independent variables will detect potentid outliers,
non-constaut.&illlce,
non-linearïty of an idependent variable or the need for
niore independent variables, 2. Plots of resicliiah agains* the titted valiies wili detect non-c'onstancy of variance,
3. Plots of residuak a-t
t h (*(ifpocrsible) will de-%
non-independence anion*
errors or if the t h e effect has been omittd h m the mcdel,
4. BW-plots, n o r d probabiity piou, Half-normal plots, histograms and stem-
and-ieaf plots will check for n
o
d and ~ outliers, and
5. Plots of residuais again& other signiscant independent variables (if possible)
will detec* whether such variables rue to be included in the d e l . Formal tes* buikl statistics iavolving residualy which are uYed to test the Müdity of
the foUawbe; u o d linear regression moclel assumptions:
F-test for Adequacy of the Regmamion Mode1 Consider the Liriear regession mode1 (1.1) whereby the e
m
~i
are assumeci to be
i-id.. The aûequacy of the mode1 is interpreted in the forru of the sienificmce of the
indepeudent variables (xi} i = 1
... :p - 1. The following hypotheses are testeci:
Ho : / 3 1 = / & = . . . = & l = 0 Ha : not all p, =O; j = 1: ... , p - 1It can be shown that the likelifioocl ratio tes* for Hot V s . Ha if Hois true yieldv the following F-S-tistic:
M S E = J'(I-H)Y = - e'e
and
MSR =
yC(H - k l ' l ) ~
with the randtnu variable Fm,, having an F-distribution wïth
VI.y
desees of
frec
dom. The critical region given in (1.15) is jus%ifidby the folloaring fats:
- Xc
(A). where A = /3'Xf(H- 11') Xfit&(A)
hlSK (ü) ( p - 1) 7
denotg the non-
central &-square random variable with u degrees of f i d o m and non-centraüty parameter ( m p ) A.
(iii) AME and M S R are independent, (iv) E ( M S R ) = 2 + IrX>(H- !II') x/~/O, - 1)
1 a2 = E ( M S E ) .
The asertions (i)-(üi) are consequences of Cochmn's Theomm (see Searle [23],Chap ter 3), essentially l>y usine; the following theoreni:
Theorem 1.2 Let a (1)
-
N(0,I). Then,
d h h a 9 a ?-distribution with d ( A ) = tent;
&pzs
of jkdom, iif/. A
ip idempo-
CHAPTER 1. INTRODUCTION (2) dAz and z'Bz a m independent iff. AB = O.
where z
-
N ( O t 1) md A = (1- H ) .
S i m e A is idempotent witb rauk rz - p (Theoreni 1. l ) , it f d o w s that
MSE
(n - P) 0 ' Xn-p
and, similarly
b ~ 1)- MSR u'L
lia
a
non-central
q
u
e
=
f ( H - 1'1) y €9
distribution
with
degrees
of
5
trace (H - 11') = p - 1 and non-cxmtraliw parameter
Sk:e
HX = X, the nou-centrality paranieter simplifies to
which is 2 O and equal zero
S.Hoholds.
Independence easily follows since
The as~?rtiori in (iv) is s strict ineqU81ity if at least one of the pj # O.
freedom=
1.1.2
Other Diagnostics
Some diagnmtic bols are d to detect infiuential and outl-Mng observations in a given regression model. The Studentited ~ s i d i ds very informative in examinhg
residuals d e r a n o d mode1 skce it is stanciar-
ancl it introduces the idea of
casse deletion, where the fit for al1 o\iervations is ixnnpard to the
case.
fit witb the delet&
MW, Vkre) = ?C M.
where
The diagonal elenients m, of the pmjectzon matriz depict thœe observations with Iiigli-leverage (i-e. hi@y influentid observatioirs) since they are relatai to the distarice Msween % and S.
Giveu t h t X is of fidl r d , then
Hence, the average of diagonal elments mii is 1 - p/n and high-levwdge observations shoiild have b7n;ill values for m, as compared to 1 - p/n. A s a d e of thumb, fiom H e i n and W e W ([ll]), if
m y
5 1 - 2p/n, then the ith observation is a
hi&-leverage point. Thus, M is a uaefui diagioiltic tool for detecting iduential o b servations. Another type of ill-fitting point which a r h s in mod&fitting is an outiier. It does
not n a d y imp1y an iduential observation in a dven niodel. In fa*, an outlier may be outweighed by neighboring X-valued points. S a , the effect that an outlying
point exerts on the fit ne&
to be measurd. The smder the number of okrvations
involveci in a model, the greater the dec* of the outlier on the model- This can
be done through the diaepcstic t o d of Cook's distatux whicb meamues the &a% of delethg an outlier from the data: = (A~&xIY(A~~~).
where
A ~ B=
-
p-f, ,o.
denotuig the m a l
(1.18)
LSE of P 4 t h the tth obervation
deleted fiom the data.
It
nives
the (iistaux, h e e n the U
Y IL~ east
s q i i a r a estiruator an O. This transformation may brhg nymmetry to a skewed resyoim and reduce
the heavy tails of a distribution while still
retaïning the siniplicity of the normal iinear model. When it does not provide ag
d fit to the data, alternative a p p r d e s have to be explored. One such
a p p d is to use the genemlued liruw model (GLM), where the response is ass'u11ied to be10ng to the exponential f m y -
The assixxmptions made here are
baseci on the concept that the response depends on the preciictors through a
linear fom. Thus, the Lin-
mdeis are gendzeci through
1. a litrk hmdiou which relates the expectattion of the response to the linear
preciic-tor, and thruugh 2. an exp~nentialf d y distribution for the emrs.
This d e l will l>e descxibed in detail in Chapter 2 and i9 the highlight of this thesis.
1.2
Outline of Thesis
The next chapter introcluces the GLM,with all the relevant notatioac. It gives the properties of estimators and computational details for estimating the parameters for conuiion exponentiai fiunilieri. Tests for gOo(iflfssof-fit and incIusion/excllusion of
variables are d s o includd. The basic properties of res5duaJs in the nomai theory linear niodels are
i
h e m models in -ter
~
for d extendhg the regression diagnostics to the generalized
3. This extension is d
e possible t h u g h transfomeci
residiials, whi& is explaineci in detail in that chapter. The final b p t e r presents
numerical illustrations of the techniques cliscussecl in Chapter 3 and @ v a a handson experienco with real data through cornputer programi developed using the %Plus
software application.
Chapter 2
The Generalized Linear Mode1 2.1
Historical Aspects
The terru "generalized linear modeln w= fiFst introduced by Nelder and Wedderbuni iii 1072. The geueralized h e a r d e l ?usbeen one of the m05.t important developnients in the field of statis-tics in the last thirty y-.
Much uyed in applica-
tions to the social sciences anù medicine, these models also play an important role
in the aaalysis of sumival data. As their name imggest, these mcxiels generalize the nomial-theory hear modehi s u c h that the usual linear regression coniponent is 1
d
to desc.ribe a wider class of yrobbility distributioiis, specIfidy the exponential faru-
ily distributions. A1thoite;lig m e r m hezu modeIs have had an important impact on statistics, most introcluctory Ytativtics textbooks however, s t i l l only present n o d linear mdeis.
It was ~ e e in n Chapter 1 that an aùequate lind
m
o
n m d d y b d d inchde a
e which ellsl1it-sthe canibination of wnstancy of variance, appmbate normality
of the emfs, auri additivity of the qmtematic effects. Huwever, this d
e does not
CHAPTER 2. THE GENERALUED iimE4.R MODEL always respect all three criteria.
18
For example, if some discrete data is found to
have errors with an apprkmate Poison distriition, the systematic dects may be multiplicative, in which case log-linear models are uYually employed. The folIowbg
choices of s u s h g are obtained by t r d o m ü q 0
tm :
yL/2tO ensure apprmimate anwtancy of variance,
Generally, none of these Ycaling powibiilities combine di three criteria for an adquate h e a r regession analalysks.Alternatively, a generalized linear mode1 encornpumes sr-
ponentially djstriLmtd enors anci a variance fimc%ionwhi& depenàs on the mean in some known way, so t h t there is no neeà to d e y for nonriality of errors or
for constancy of variance. In fact, the scd.ing problem is reciuced to ensuhg that
the sys-tematic effW are aciclitive. It may be considemi to be an extension to the normal-theq lincar moclel with an exponential M XI..
~ome ddeù
modifications where the mean p of
y with resyonse variable y is
linearly related to the predictors
. . x,, by a Iuik hinction, g(p). This L describecl in detail in the sec.tions ttiat
f0Ilrn~.
2.2
Mean and Variance Fhnctions in an Exponential Family
An observation y foUows an exponential f d y distribution if its probability demi@ fiindion is givien by
where a!b. anclc are some known functions, û is the h t a o n pommeler and 4 is the
dispersion pamnxtcr. This is denoted by
When the dispersion parameter 4 is hown, 0 is the
aznonid parameter.
The mean
rind variance of y are given by U(B) and a(4)lP(B).Thuy it can be written that
is called the vu&mce finction. For example, in the case of the normal distribution, û = pt V ( p ) = 1 and a(4) = O? These may be c l e n d kom
CHAPTER 2- THE GENERALBED fiLNEAR MODEL respectively, where l is the log-iikelihod fimctïon, Note that
hence equatiou (2.6) yields
Var (y) = a(#)b"(B).
2.3
Description of the Generalized Linear Mode1
T h e okrvations belonging to a statistical mode1 can be summarid in terms of a spteniatic component and a ranàom component. In the generalizeù linear mode1
CHAPTER 2- THE GENERALIZED LZNEAR MODEL
21
(GLM)diPcussed by McCullagh and NeMa [l?],the d o m copupanent is inherent in the exponential M
y &tributionof the o h t i o n , while the systematic camp
nent assumes a linear struc.tUre in the predictor vafiaHes for a func%ionof the mean.
This fiuiction is h m as the link fwrction. When the parameter 8 is modeled as a linear function of the predictors, then the link function is known as the c a n o n i d
litut. Themfort?, for a g*en set of okrvatiorw
{yi)&
where
yi
iY wIwidered tu be
asmciated with pfeciictor values xi = (zil,... ,z*)',the GLM is expressed as:
where
6 is assumcl to depend on xi through the relation
If g is the c a n o n i d link, theri, the link function is specifieci by
h yractice, a @en
e reparauietrizec-lsuch that a = 114 and k = -#/O,
hence to get
Therefore, p = ka = -118 and mwequently, the canonid link is given by
Table 2.1 : Diqm-szon Pamnieter, Cononad Li& a d Variancc Functzon for Distrib u t i o ~of~the ~ Ezpmm~tiulF a n d y
DISTRIBUTION Notation
a(&)
9 = g(p)
Nauie
v(14
Table 2.1 gives cananical links and other components for oommon distribution faniiiicr with respect tn the exponential family gïven by equation (2.1) [17].The choice
of a proper link function that will sa*
b
the criterion of the domain of variation p is
d on: 1. how the liiik fundion will &y
interpret the paranieters in the linear predidor;
2. how the link fits to the data; and
3. the existence of a sinipie siiffiCient statistic.
cHAPTER 2- THE GENERALUED LINt3A.R MODEL
25
Pœsible link functions aasocîateci to some important members of the scponentiaî family are ated in 'Ildole 2.2.
In sunmary, gendZRd lin-
models make up a
general chus of p m h a b ' i c regression m d e h with the assumptions tbat: (1) the respnse probability distribution is a menit.m of the exponential fàtuily of distributions;
(2) the respLw ?/ii = 1: ... ? n is a set of independent raadom variables;
(3) the explanatory variables are linearly combined to srplain systematic variation in a func*tionof the mean. in a practical &ta sikiatiou, GLM fittuig involves the following: a choasing an error distribution that is relevant;
i d e u t m g the independent variables to be included in the systeniatic coniponent; and a s p e c m the link funr:tion.
The next section presents the maxixu~uiilikelihd method for estimathe; the regres sion parameters assurning that the above have been specifieci.
2.4
Maximum Likelihood Estimation for the GLM
If the probab'ility specifications of an exponential f d y mode1 are h o w u by f (y?d ) , then the h - t way to fit a generalid lin-
m d e i is by Maximum Likelihd b%i-
matiou of the parameters 13 for the data oùservd (Silverman aml Green [IO]). With
CHAPTER 2- THE GEIWMUZEI) tJNEAR MODEL
26
many desirable pmperties of maximum klihood estimatom su& as mIlSiStency, e f L ciency, diiciency and asymptotic nORnali@, it is naturd to amsider such a method for GLMs. In p e r d , the maximum lïkelihood equations which result fiom GLMs cannot
be solved expficitly and hence remunie must be made to ~ m n e r i dmethcuis.
Three meth& ore deynihl in thW section: the Newton-Fbpbn method, the Fisher Scoring methoci,
and
the
Iteratively Weightecl Leart Squares rnethoù. But k t , the
niaxinizm likelihood equations are derived. Given the raponses ylr... ,y,,where gi is üonsidered tu l x geuerated h m a menber of the exponential f d y &(Ol 4; a. 6. c ) ,
the likelikood fiuiction is written as
i=l
i= 1
Then the 1%-likeiihd is @ven by
whereby Ei is the ith mmponent to the log-likelihood and is therefore given by
The l i k e l i h d implicitly depends on the pai.anetefs pj:j = 1:. .. ? p ,h t l y t h @ the link fimction g ( p ) and se the influence that the Cth point sr&
on the d-
B through y. Then the confidence i n t e 4 displacement is measured
by the one-step approximation to &O):
where X: = r$, (2.43)-
3.3
Testing the Goodness-of-Fit
M-vriag
the goocln-f-fit
of a model can be done by calculahg the effect of
change in u on the diagnœi%icmeasmes of the deviace function D anù Pearson's &atistic X2.
In case of the deviance fimction, the maximum likelihd estimate should
m h h k D, much like the least
quates
estimate xuhimkm the resichial
SIUU
of
Yquares RSS in a nomal-theory linear model. Subeiequently, deletion of an observa-
tion d e c ~ e h D s , iïke it wodd decrease RSS in the normal-theory model. Using the observation munt ruatrix V in the l ~ l i h o o fundion d fielcis a deViance
A onestep estimate
b1(c), and a second-orderTaylor Senes
of D,(@' ( v ) ;Y)
CHAPTER 3. RESIDUAL DIAGNOSTIC MEASabout 6 appmximates the above quantity :
at
c =O :
D , . ( x ~ ~ ' (y) v ) ; is at a tnininlum of
D ( X D ; ~ )- (& + ci),
where
is
the ciiwge in the corifidence interval displaÿaiient diagno&ic i$.
The deviailice dec~easesas t. î l e rate of
O.
tfiwee of D due to perturbations is obtxhed by taking the derivative of
(3.22) with res'yec-t to r..
The change in devimce due tu deletion of the tth point is apprazaniatd by:
which are i ~ f i for d index plotting. The presence of 2 components is a feature founci in the onclstep appraxiniation, ruakine; it a useful diagnostic tool.
The Pearson's statistic is not a shaîghtforward rueasure to interpret since it doam't extend fiom the normal-theory linear model as does the deviance huiction- As o k r vations are deletecl kom a given model, the 2 meHowever, like the RSS,the 2 is the d
does not necesady decreaw.
t of the sum of squares of Merences of the
observed ikom the fitted vahies. The one-step approDcimation to the 3 due to the deletion of the tth observation is:
In extrerne cases, 2 WU in
A.2
betaO
3
c a t ("
.
.
veights=riei@tValue)$coefficientsm file="tmp.'1
.
"\nn. append=T)
# Nov execute the created comand ev~(parse(fi1e='tmpPla), local=T)
1 # Purpose of this function # etahat
