Master’s Written Examination Option: Statistics and Probability

Fall 2015

Full points may be obtained for correct answers to eight questions. Each numbered question (which may have several parts) is worth the same number of points. All answers will be graded, but the score for the examination will be the sum of the scores of your best eight solutions. Use separate answer sheets for each question. DO NOT PUT YOUR NAME ON YOUR ANSWER SHEETS. When you have finished, insert all your answer sheets into the envelope provided, then seal it.

1

Problem 1—Stat 401. (a) Suppose X has gamma distribution with parameters α and β (both α and β are pos1 α−1 −x/β itive), i.e., f (x) = Γ(α)β e , x > 0. Derive the moment generating function αx M (t) of X. (b) Utilizing the conclusion in (a), prove that (i) the distribution of X/β is still gamma distributed, where X has gamma distribution with parameters α and β and (ii) Pn i=1 Xi is also gamma distributed where X1 , . . . , Xn denote a random sample from a gamma distribution with parameters α and β. Solution to Problem 1. (a) ∞ 1 etx xα−1 e−x/β dx M (t) = E(e ) = α Γ(α)β 0 Z ∞ 1 = xα−1 e−x(1−βt)/β dx α Γ(α)β 0 tX

Z

Let y = x(1 − βt)/β. When t < β1 , we have α−1  β/(1 − βt) βy e−y dy M (t) = α Γ(α)β 1 − βt 0 Z ∞ 1 1 α−1 −y y e dy = α (1 − βt) 0 Γ(α) 1 = . (1 − βt)α Z

∞

(b) The moment generating function of X/β can be written as E(etX/β ) = M (t/β) =

1 1 = . α (1 − βt/β) (1 − t)α

Thus X/β has Gamma distribution with Pn α and β = 1. The moment generating function of i=1 Xi can be written as E(e

t

Pn

i=1

Xi

)=

n Y

E(etXi ) = (M (t))n =

i=1

Thus

Pn

i=1

1 . (1 − βt)nα

Xi has Gamma distribution with parameters nα and β.

Problem 2—Stat 401. Let X and Y be independent chi-square random variables with r1 and r2 degrees of freedom, respectively. Namely, X ∼ Γ(α = r1 /2, β = 2), X/r1 and Y ∼ Γ(α = r2 /2, β = 2). Let Z := . The distribution of Z is called an Y /r2 F −distribution. Derive the probability density function f (z) of Z by answering the following questions. Specify the support of the PDF’s in all the three questions. 2

1. Write down the joint PDF f (x, y) of X and Y . 2. Let W := Y . Find the joint PDF f (z, w) of Z and W . 3. Find the PDF of Z. Solution to Problem 2. 1. By the independence of X and Y , f (x, y) = f (x)f (y) = 2.

1 xr1 /2−1 y r2 /2−1 e−(x+y)/2 , x, y > 0. Γ(r1 /2)Γ(r2 /2)2(r1 +r2 )/2

     X = r1 · ZW  Z = r2 X r2 r1 Y =⇒  Y = W,  W = Y, So the Jacobian matrix of the transformation is  ∂x ∂x   r1 w ∂z ∂w J = ∂y = r2 ∂y 0 ∂z ∂w

r1 z r2



1

Therefore the joint PDF of Z and W is  r1 /2 f (z, w) =

r1 r2

Γ(r1 /2)Γ(r2 /2)2(r1 +r2 )/2

z

r1 /2−1

w

w (r1 +r2 )/2−1 − 2

e





r1 z+1 r2

, w, z > 0.

3. By integrating f (z, w) with respect to w, we get Z ∞ f (z) = f (z, w)dw 0

 r1 /2 =

=

r1 r2

Γ(r1 /2)Γ(r2 /2)2(r1 +r2 )/2  r1 /2 r1 r2

z

r1 /2−1

Z

∞

w

w (r1 +r2 )/2−1 − 2

e



dw

0

r1 /2−1 (r1 +r2 )/2



r1 r1 z + r2

(r1 +r2 )/2   r1 + r2 Γ 2

z 2 Γ(r1 /2)Γ(r2 /2)2(r1 +r2 )/2  r1 /2 r1  (r1 +r2 )/2   r2 r1 r1 + r2 r1 /2−1 = z Γ , Γ(r1 /2)Γ(r2 /2) r1 z + r2 2

Problem 3—Stat 411.



r1 z+1 r2

when z > 0.

Let X1 , ...Xn be a random sample from X ∼ Exp (θ) , i.e. 1 f (x, θ) = e−x/θ , x > 0, θ

where θ > 0. (1). Find the maximum likelihood estimator θˆmle for θ. (2). Find the Fisher information for θ. Is the θˆmle an efficient estimator? (3). What is the maximum likelihood estimator θˆmle for θ given that θ > 12 . 3

P

Solution to Problem 3. (1). Likelihood function L (θ) = θ−n e− equation P −n xi ∂ log L (θ) ¯ = + 2 = 0 ⇒ θˆ = X ∂θ θ θ (2). Mean and variance Z ∞ Z ∞ 2 x2 f (x) dx = 2θ2 . xf (x) dx = θ, EX = EX =

xi /θ

. Likelihood

0

0

¯ = θ. It is an unbiased estimator, E X Fisher information for θ is     2 1 1 ∂ log f (X, θ) 2X = −E 2 − 3 = 2 , I (θ) = −E 2 ∂θ θ θ θ
¯ = θ2 = [nI (θ)]−1 . It reaches the Rao-Cramer Lower variance of the estimator is V ar X n Bound, i.e. the mle is an efficient estimator. thus . as (3). Given that θ > 12 , then the maximum likelihood estimator is    ¯ ¯ < 1/2 1 X, X ¯ θˆmle = ¯ ≥ 1/2 = min X, 2 1/2, X


Problem 4—Stat 411. Let X1 , . . . , Xn denote a random sample from an exponential distribution, i.e., f (x) = 1θ e−x/θ , x > 0. Let H0 : θ = 1 and H1 : θ > 1. (a) Show that there exists a uniformly most powerful test for H0 against H1 . Determine the statistic Y upon which the test may be based, and derive the best critical region of size α. (b) Find the distribution of the statistic Y in part (a). If we want a significance level of 0.05, write an equation which can be used to determine the critical region. Let γ(θ), θ ≥ 1, be the power function of the test. Derive the expression of γ(θ). Solution to Problem 4. (a) The likelihood function can be written as L(θ) = exp(−n log θ −

n X

Xi /θ).

i=1

For θ > 1, we have n

X L(1) = exp(n log θ + (1/θ − 1) Xi ). L(θ) i=1 So L(θ) has monotone decreasing likelihood ratio in the statistics Y = the UMP level α critical region for testing H0 versus H1 is given by Y =

n X i=1

4

Xi ≥ c,

Pn

i=1

Xi . Thus

where c is chosen such that α = Pθ=1 [Y ≥ c]. (b) Since Xi isP a gamma distribution with α = 1 and β = θ, and X1 , . . . , Xn are independent, Y = ni=1 Xi follows gamma distribution with α = n and β = θ. Under H0 , Y follows gamma distribution with α = n and β = 1. Notice that 2Y follows gamma distribution with α = n and β = 2, which is also χ2 distribution with df=2n. Thus we can choose c such that P (χ2 (2n) > 2c) = α. The power function γ(θ) can be computed similarly γ(θ) = Pθ [Y ≥ c] 2c 2Y ≥ ] = Pθ [ θ θ = Pθ [χ2 (2n) ≥

2c ]. θ

You can also write γ(θ) as an integral using the density function of γ(n, θ).

Problem 5—Stat 411. P Let X1 , . . . , Xn be iid P random variables, with mean µ and n 1 1 2 2 ¯ 2 for the sample mean ¯ variance σ . Write X = n i=1 Xi and S = n−1 ni=1 (Xi − X) and sample variance, respectively. 1. Show that S 2 is an unbiased estimator of σ 2 . 2. Show that S 2 is a consistent estimator of σ 2 . 3. Consider a Poisson model, where the mean and variance both equal θ. In this ¯ and S 2 are reasonable estimators of θ, for example, both are unbiased case, both X and consistent. Is one estimator better than the other? Clearly present your case, justifying each claim. Solution to Problem 5. 1. Rewrite the sample variance as n

n

 1 X 1 X 2 2 2 ¯ ¯ S = (Xi − X) = X − nX . n − 1 i=1 n − 1 i=1 i 2

¯ 2 ) = σ 2 /n + µ2 . Putting this together, Then E(Xi )2 = σ 2 + µ2 for each i, and E(X gives n o 1 nX 2 E(S 2 ) = (σ + µ2 ) − n(σ 2 /n + µ2 ) = · · · = σ 2 , n − 1 i=1 so S 2 is an unbiased estimator of σ 2 . 2. Start with the version of the sample variance from above, i.e., n

S2 =

n

X  1 X ¯ 2= 1 ¯2 . (Xi − X) Xi2 − nX n − 1 i=1 n − 1 i=1 5

¯ → µ in probability, and the continuous The law of large numbers says that X 2 2 ¯ mapping says X → µ in probability. The law of large numbers also says Pntheorem 1 2 that n i=1 Xi → σ 2 + µ2 in probability. Writing S 2 one more time as n

1X 2 n ¯2 n · Xi − S = X , n − 1 n i=1 n−1 2

since n/(n − 1) → 1 as n → ∞, it is clear that the right-hand side converges to σ 2 + µ2 − µ2 = σ 2 in probability. Therefore, S 2 → σ 2 in probability, i.e., S 2 is a consistent estimator of σ 2 . ¯ and S 2 are both unbiased and consistent, a natural way to compare them 3. Since X is to consider the variance; that is, if there is one with uniformly smaller variance, then it’s the better estimator. In this case, it is not easy to write a formula for the variance of the sample variance, so a direct comparison is difficult. However, ¯ is a complete sufficient we know that, for the Poisson model, the sample mean X statistic. The Lehmann–Scheffe theorem that if there exists an unbiased estimator that is a function of a complete sufficient statistic, then that must be the minimum variance unbiased estimator. Since S 2 is not a function of a complete sufficient ¯ is, it must be that the variance of X ¯ is uniformly smaller than statistic, while X 2 that of S ; therefore, the sample mean is a better estimator.

Problem 6—Stat 416. A time and motion study was made in the permanent mold department at Central Foundry to determine whether there was a pattern to the variation in the time required to pour the molten metal into the die and form a casting of a 6 × 4 in. Y-shaped branch. The metallurgical engineer suspected that pouring times before lunch were shorter than pouring times after lunch on a given day. Twelve independent observations in seconds were taken throughout the day, six before lunch and six after lunch. Before Lunch: 12.6, 11.2, 11.4, 9.4, 13.2, 12.0 After Lunch: 16.4, 15.4, 14.1, 14.0, 13.4, 11.3 (a) Use Wilcoxon Rank-Sum test to find the P value for the alternative that mean pouring time before lunch is less than after lunch for the data below on pouring times in seconds. Specify your null hypothesis and alternative hypothesis. (b) It is known that “the asymptotic relative efficiency of the Wilcoxon Rank-Sum test relative to the two-sample Student’s t test is 1.50 for the double exponential distribution and 1.09 for the logistic distribution, which are both heavy-tailed distributions”. Explain the concept “asymptotic relative efficiency” and the implication of this statement. Solution to Problem 6. (a) With equal sample sizes m = n = 6, let X denote pouring time before lunch and Y denote pouring time after lunch. The null hypothesis is H0 : θ = µY − µX = 0 and the desired alternative hypothesis is H1 : θ > 0 and the appropriate P value is in the left tail for the Wilcoxon Rank-Sum test statistic WN . 6

The pooled array with X values underlined is 9.4, 11.2, 11.3, 11.4, 12.0, 12.6, 13.2, 13.4, 14.0, 14.1, 15.4, 16.4, and WN = 1+2+4+5+6+7 = 25. The P value is P (WN ≤ 25) = 0.013 from Table J for m = 6, n = 6. Thus, the null hypothesis H0 : θ = 0 is rejected in favor of the alternative H1 : θ > 0 at any significance level α ≥ 0.013 . (b) Let A and B be two consistent tests of a null hypothesis H0 and an alternative hypothesis H1 , at significance level α. The asymptotic relative efficiency (ARE) of test A relative to test B is the limiting value of the ratio nb /na , where na is the number of observations required by test A for the power of test A to equal the power of test B based on nb observations while simultaneously nb → ∞ and H1 → H0 . The statement implies that the Wilcoxon Rank-Sum test is preferable to the two-sample t test for heavy-tailed distributions since it is more powerful. Problem 7—Stat 431. In a study to compare the precision of systematic and stratified sampling for estimating the average concentration of lead in the soil. The 1 − km2 area was divided into 100-m square, and a soil sample was collected at each of the resulting 121 grid intersections. Summary statistics from this systematic sample are given below: Element n Average (mg/kg) Range (mg/kg) Standard Deviation (mg/kg) Lead 121 127 22-942 146 The investigator also poststratified the same region. Stratum A considered of framland away from roads, villages, and woodlands. Stratum B contained areas within 50m of roads, and was expected to have larger concentrations of lead. Stratum C contained the woodland, which also were expected to have larger concentration of lead because of foliage would capture airborne particles. The data on concentration of lead were not used in determining the strata. The data from the grid points falling in each stratum are in the following table: Element Lead Lead Lead

Stratum A B C

nh 82 31 8

Average (mg/kg) 71 259 189

Range (mg/kg) 22-201 36-942 88-308

Standard Deviation (mg/kg) 28 232 79

(a) Calculatea 95% for the average concentrate of lead in the area using systematic sample (Soil samples are collected at the grid intersections. We may assume the amount of soil is negligible compared with that in the region. You may also assume this sample behaves like an SRS). (b) Now use the poststratified sample, and find 95% CIs for the average concentrate of lead. How do these compare with CI in (a)? Solution to Problem 7. (a)By SRS formula, the sample mean is the estimation of population mean. An unbiased estimator of the variance of the estimation is given by r n  s2 1− . N n Since the amount of soil is negligible compared with that in the region, we can ignore the finite population correction (fpc). Thus the 95% CI is 146 127 ± 1.96 √ = [101.0, 153.0]. 121 7

(b) Using the formula for computing poststrified estimator is given by y¯post =

H X Nh h=1

N

y¯h .

Because samples are collected at the grid intersections, so we have

Nh N

=

nh . n

Thus

82 31 8 71 + 259 + 189 = 127. 121 121 121 The estimation of the variance of y¯post is given by y¯post =

H  n  X Nh s2h ˆ . V (¯ ypost ) = 1 − N h=1 N n

Based on the discussion above, we have 82 282 31 2322 8 792 Vˆ (¯ ypost ) = + + = 121.8. 121 121 121 121 121 121 Thus the 95% CI is 121.8 = [105.4, 148.6]. 127 ± 1.96 √ 121 Compared with SRS approach, poststratified approach increase the precision of the estimation. Problem 8—Stat 451. Suppose that (X, Y ) has a bivariate normal distribution with zero means, unit variances, and correlation ρ ∈ (−1, 1). The joint density function is fX,Y (x, y) ∝ e

−

1 (x2 +y 2 −2ρxy) 2(1−ρ2 )

.

1. Derive the conditional distribution of Y , given X = x. 2. Derive a Gibbs sampler for this bivariate normal. 3. Are the samples obtained by your Gibbs sampler exact bivariate normal samples, or are they approximate samples? If you think they are exact samples, then prove your claim; if you think they are approximate samples, then explain in what sense they are approximate. Solution to Problem 8. 1. Take the conditional density of Y , given X = x, is proportional to the joint density as a function of y only: fY |X (y | x) ∝ e

−

1 (x2 +y 2 −2ρxy) 2(1−ρ2 )

.

It is easy to see that the conditional density must be a normal—exponential function with quadratic in the exponent. Moreover, for general normal distributions, the quadratic term in the density has coefficient − 21 × variance and the linear term has coefficient mean/variance. So then it’s easy to check that the conditional distribution has mean ρx and variance 1 − ρ2 . 8

2. Since the conditional distribution above is symmetric in the pair (X, Y ), the full conditionals are Y | (X = x) ∼ N (ρx, 1 − ρ2 ) and X | (Y = y) ∼ N (ρy, 1 − ρ2 ). Therefore, a Gibbs sampler can be carried out by initializing X0 at, say, 0, and then iterating over t = 1, 2, . . . , as follows: Yt | Xt−1 ∼ N (ρXt−1 , 1 − ρ2 ) and Xt | Yt ∼ N (ρYt , 1 − ρ2 ). 3. The output of the Gibbs sampler is not an exact sample from the bivariate normal. For one thing, the Gibbs output is a Markov chain, so there is some dependence. Furthermore, one cannot sample exactly from a joint distribution by looking only at the conditionals, that’s not how probability works; one can, however, sample from a marginal and a conditional. Anyway, the Gibbs output is not an exact sample. It is an approximation in the sense that, in a limit along the t sequence, the Markov chain converges to its stationary distribution, with is the bivariate normal, by construction. So, various expectations related to the bivariate normal can be accurately approximated based on a long sequence of Gibbs samples.

Problem 9—Stat 461. matrix

A Markov chain X0 , X1 , ... has the transition probability  1 1 1 1  4

4

 1 1 4 4 P =  0 0 0 0

4 1 4 1 2 1 2

4 1 4 1 2 1 2

 . 

Is state 0 transient or recurrent? Justify your answer. Solution to Problem 9. A state i is recurrent if and only if ∞ X

(n)

Pii = ∞.

n=1 (n)

So we only need to compute P00 . But it is easy to see that  1  1 ∗ ∗ n n 22 22  1n 1n ∗ ∗  (n) 22 22 . P =  0 0 ∗ ∗  0 0 ∗ ∗ Hence

∞ X n=1

(n) P00

∞ X 1 = < ∞. 22n n=1

Therefore, state 0 is transient.

9

Problem 10—Stat 461. This problem is related to probability generating function. It has two subquestions. (a) Let ξ and η be independent nonnegative integer-valued random variables having the probability generating function φ(s) = Esξ ,

and ψ(s) = Esη ,

|s| ≤ 1.

Show that the generating function for the random variable ξ + η is φ(s)ψ(s). (b) Let ξ1 , ξ2 , ... be i.i.d. nonnegative integer-valued random variables with probability generating function φ(s) = Esξ1 . Let N be a nonnegative integer-valued random variable, independent of ξ1 , ξ2 , ... with probability generating function g(s) = EsN . Show that the probability generating function of the random sum X = ξ1 + · · · + ξN is EsX = g(φ(s)). Solution to Problem 10. (a)

By definition and independence of ξ and η, we have Es(ξ+η) = Eeξ eη = Esξ Esη = φ(s)ψ(s).

(b)

We have X

Es = = = = = =

∞ X

P{X + k}sk

k=0 ∞ X

∞ X

k=0 ∞ X

n=0 ∞ X

! P{X = k|N = n}P{N = n} sk ! P{ξ1 + ... + ξn = k|N = n}P{N = n} sk

n=0 k=0 ∞ ∞ XX

P{ξ1 + ... + ξn = k}P{N = n}sk

k=0 n=0 ∞ ∞ X X n=0 ∞ X

! P{ξ1 + ... + ξn = k}sk

P{N = n}

k=0

φ(s)n P{N = n}

n=0

= g(φ(s)).

10

[using (a)]

Problem 11—Stat 481. Suppose you are studying the monthly amounts that seventhand eighth-grade boys and girls spend on entertainment such as movies, music CDs, and candy. Representative samples of children within a certain school district were selected, and children were asked about their spending habits. The following results were obtained: Sample size Mean($) Standard Deviation($)

7th-grade boys 30 20.1 6.0

8th-grade boys 25 23.2 5.6

7th-grade girls 30 19.6 5.3

8th-grade girls 25 25.0 7.0

(a) Test whether the four groups differ with respect to their mean spending amounts. Use significance level α = 0.05 and F critical value F (0.05; 3, 106) = 2.69 . (b) Test whether there is significant difference in the mean spending amounts of eighthgrade boys and eighth-grade girls. Use significance level α = 0.05 and t critical value t(0.025; 24) = 2.06 . Solution to Problem 11. (a) SSError = (29)(6.0)2 + (24)(5.6)2 + (29)(5.3)2 + (24)(7.0)2 = 3787.25 with 106 degrees of freedom. y¯ = [(30)(20.1) + (25)(23.2) + (30)(19.6) + (25)(25.0)]/110 = 21.78 . SST reatment = (30)(20.1 − 21.78)2 + (25)(23.2 − 21.78)2 + (30)(19.6 − 21.78)2 + (25)(25.0 − 21.78)2 = 536.86 with 3 degrees of freedom. FT reatment =

536.86/3 = 5.01 > 2.69 3787.25/106

with p-value < 0.05 . That is, there are significant differences among expenditures. (b) We can use the 2-sample t-test with unequal variances. 25.0 − 23.2 T =q = 1.00 < 2.06 7.02 5.62 + 25 25 That is, there is no significant difference in the mean spending amounts of eighth-grade boys and eighth-grade girls at 5% level.

Problem 12—Stat 481. A company operates children portrait studios in 21 cities. The manager is considering an expansion and wishes to investigate whether the sales (Y ) in a community can be predicted from the number of persons aged 16 or younger (x1 ) and per capita disposal personal income (x2 ). Try to fit the data with a multiple linear regression model Yi = β0 + β1 xi1 + β2 xi2 + εi , i = 1, ..., n, where the i.i.d. errors εi ∼ N (0, σ 2 ) . The model can also be expressed in a matrix form Y = Xβ + ε. (1). Complete the ANOVA table below for the linear model 11

Source M odel Error T otal

S.S. 24015

DF

M.S.

F

26195

(2). State both null and alternative hypotheses you will test for the linear model. Draw your conclusion based on the ANOVA table. [Given: F(0.95,2,18)=3.55, F(0.95,2,19)=3.52, F(0.95,3,18)=3.15] (3). Find the coefficient of determination R2 and interpret   it. ˆ (4). Show that for the least square estimator β, V ar βˆ = σ 2 (X 0 X)−1 . Solution to Problem 12. (1). ANOVA Table Source M odel Error T otal

S.S. 24015 2180 26195

DF 2 18 20

M.S. 12007.5 121.1

F 99

(2). H0 : β1 = β2 = 0 vs H1 : at least one is not zero. F = 99 > F (0.95, 2, 18) = 3.55. Sales are related to the population and per capita disposal income. (3). R2 = 91.7% , i.e. 91.7% of the variation in the data can be explained by the regression model above.  (4). Least square criterion: minβ {Q (β)} = minβ (Y − Xβ)0 (Y − Xβ) The the estimator βˆ is the solution ∂Q (β) /∂β = 0,i.e. −1 X 0 Xβ = X 0 Y ⇒ βˆ = (X 0 X) X 0 Y

Thus h   i −1 V ar βˆ = V ar (X 0 X) X 0 Y −1

−1

= (X 0 X) X 0 · V ar (Y ) · X (X 0 X) −1 −1 −1 = (X 0 X) X 0 · σ 2 In · X (X 0 X) = σ 2 (X 0 X) .

12