6. Duality between confidence intervals and statistical tests

Suppose we carry out the following test regarding a single population mean at a significance level of 100α% . H0 :µ = µ0 HA :µ 6= µ0 Then we reject H0 if and only if µ0 does not belong to the 100(1 − α)% confidence interval for µ.

1 / 49

Intuition of the duality between tests and confidence intervals

Intuition: if µ0 belongs to the appropriate confidence interval, then it is a credible value for the population mean and hence we do not reject H0 . ’Significance level’ + ’confidence level’ = 100% .

2 / 49

Example 6.1 (see Example 5.2)

A sample of 100 Irish people were measured. Their mean height was 168cm and the sample standard deviation 12cm. Calculate a 95% confidence interval for the mean height of all Irish people. On the basis of this confidence interval, should we reject the hypothesis that the mean height of Irish people is 170cm? What is the significance level of such a test?

3 / 49

Solution to Example 6.1

The sample is large and 100(1 − α) = 95 ⇒ α = 0.05. The formula for the 95% confidence interval is X±

t∞,α/2 s 1.96 × 12 √ =168 ± 10 n =168 ± 2.35 = [165.65, 170.35]

Since µ0 = 170 belongs to the 95% confidence interval for µ, we do not reject H0 .

4 / 49

Applications of the concept of duality

The duality between confidence intervals and tests can easily be extended to tests for the difference of two means for 2 dependent samples. Let D denote the difference between a pair of observations. We reject the null hypothesis H0 : µD = k, at a significance level of 100α% , if and only if k does not belong to the 100(1 − α)% confidence interval for the mean difference.

5 / 49

Example 6.2 (see Example 5.4) A drug for reducing blood pressure is tested on a group of 7 patients. The maximum resting blood pressure of these patients was measured before and after treatment. Patient Blood Pressure - Before Blood Pressure - After

1 170 160

2 190 180

3 160 150

4 180 160

5 170 160

6 160 170

7 170 150

Calculate 95% and 99% confidence intervals for the mean fall in blood pressure. Can it be said at a significance level of a) 5% b) 1% , that the drug changes the mean blood pressure? 6 / 49

Solution of Example 6.2 It should be noted that this hypothesis states that µD 6= 0. This cannot be the null hypothesis. The null hypothesis is the complement of this hypothesis i.e. µD = 0. As previously, we calculate the falls in the blood pressure for each patient. Patient Before (X ) After (Y ) Difference (X − Y )

1 170 160 10

2 190 180 10

3 160 150 10

4 180 160 20

5 170 160 10

6 160 170 -10

7 170 150 20

7 / 49

Solution of Example 6.2

Now we calculate the confidence intervals for the mean difference. The sample of differences is small, hence the appropriate formulae are as follows: 95% confidence interval: D ±

sD tn−1,0.025 √ n

99% confidence interval: D ±

sD tn−1,0.005 √ n

8 / 49

Solution of Example 6.2

We calculate the mean and the standard deviation of the differences 10 + 10 + . . . + 20 = 10 7 Pn (di − D)2 sD2 = i=1 n−1 (10 − 10)2 + (10 − 10)2 + . . . + (20 − 10)2 = = 100 6 √ sD = 100 = 10. D=

9 / 49

Solution of Example 6.2

We now calculate the 95% confidence interval: D±

sD tn−1,0.025 10t6,0.025 √ √ =10 ± n 7 10 × 2.447 √ =10 ± = 10 ± 9.25 = [0.75, 19.25] 7

Since 0 does not belong to the 95% confidence interval for µD , we reject H0 at a significance level of 5% .

10 / 49

Solution of Example 6.2 We now calculate the 99% confidence interval: D±

sD tn−1,0.005 10t6,0.005 √ √ =10 ± n 7 10 × 3.707 √ =10 ± 7 =10 ± 14.01 = [−4.01, 24.01]

Since 0 belongs to the 99% confidence interval for µD , we do not reject H0 at a significance level of 1%. Hence, we have evidence that the drug affects blood pressure, although this evidence is not very strong.

11 / 49

6.1 Confidence intervals for the difference between population means based on 2 independent samples

6.1.1 For 2 large samples When both samples are large the 100(1-α)% confidence interval for the difference between 2 population means is: Confidence interval for the difference between population means X − Y ± t∞, α2 S.E .(X − Y ).

12 / 49

Confidence intervals for the difference between population means based on 2 independent samples In particular, 95% confidence interval X − Y ± t∞,0.025 S.E .(X − Y ) = X − Y ± 1.96S.E .(X − Y ). 99% confidence interval X − Y ± t∞,0.005 S.E .(X − Y ) = X − Y ± 2.576S.E .(X − Y ), where X and Y are the sample means and S.E .(X − Y ) is the standard error of using (X − Y ) to estimate µX − µY . 13 / 49

Standard error of the difference between the sample means

When both samples are large, the approximation for the standard error of the difference between the sample means is given by s sX2 s2 + Y, S.E .(X − Y ) ≈ m n where m and n are the sample sizes.

14 / 49

Example 6.3 (see Example 5.5)

The average salary of 100 male teachers is 37 600 Euro with standard deviation 12 000. The average salary of 50 female teachers is 36 900 Euro with standard deviation is 10 000. Calculate a 95% confidence interval for the mean difference in salary between male teachers and female teachers. Can it be stated at a significance level of 5% that the mean salaries of male and female teachers differ?

15 / 49

Solution to Example 6.3

First we estimate the standard error of the difference between the sample means s s2 s2 S.E .(X − Y )≈ X + Y m n r 120002 100002 = + ≈ 1855 100 50

16 / 49

Solution to Example 6.3

The 95% confidence interval for the difference between population means is X − Y ± t∞,0.025 S.E .(X − Y )=(37600 − 36900) ± 1.96 × 1855 =700 ± 3635 = [−2935, 4335]

17 / 49

Solution to Example 6.3

We have H0 :µX − µY = 0 HA :µX − µY 6= 0. Since 0 belongs to this confidence interval, we do not reject H0 at a significance level of 5%. We have no evidence that the mean salaries of male and female teachers differ.

18 / 49

6.1.2 For small samples If at least one of the samples is small, then we use the following approximation for S.E .(X − Y ). s   1 1 2 S.E .(X − Y ) ≈ sp + , m n where sp2 is the pooled variance sp2 =

(m − 1)sX2 + (n − 1)sY2 m+n−2

This formula assumes that the population variances are equal. The formula for the confidence interval assumes that if a sample is small then observations come from a normal distribution. 19 / 49

Formulae for confidence intervals for the difference between two population means (small samples) The 100(1-α)% confidence interval for the difference between 2 population means is (X − Y ) ± tm+n−2, α2 S.E .(X − Y ). (if m + n − 2 > 30, we may use the approximation tm+n−2,α/2 ≈ t∞,α/2 ). In particular, the 95% confidence interval is (X − Y ) ± tm+n−2,0.025 S.E .(X − Y ). The 99% confidence interval for the difference between 2 population means is (X − Y ) ± tm+n−2,0.005 S.E .(X − Y ). 20 / 49

Example 6.3 (see Example 5.6)

The table below gives statistics regarding the weights of Americans and Japanese. Sample size mean std. dev.

Americans 15 86 12

Japanese 10 72 10

Calculate a 99% confidence interval for the mean difference in weight between Americans and Japanese. Can it be stated at a significance level of 1% that these mean weights differ?

21 / 49

Solution to Example 6.3 First we calculate the pooled variance (m − 1)sX2 + (n − 1)sY2 m+n−2 14 × 122 + 9 × 102 = ≈ 126.78 15 + 10 − 2

sp2 =

Now we calculate the standard error s   1 1 2 S.E .(X − Y )≈ sp + m n s   √ 1 1 + ≈ 21.13 ≈ 4.597. ≈ 126.78 15 10

22 / 49

Solution to Example 6.3

Now we calculate the 99% confidence interval (X − Y ) ± tm+n−2,0.005 S.E .(X − Y )=(86 − 72) ± t23,0.005 × 4.597 =14 ± 2.807 × 4.597 ≈14 ± 12.9 = [1.1, 26.9].

23 / 49

Solution to Example 6.3 We have H0 :µX − µY = 0 HA :µX − µY 6= 0, where µX and µY are the mean wieghts of the populations of Americans and Japanese, respectively. Since 0 does not belong to the 99% confidence interval, we reject H0 at a significance level of 1%. We have ’strong evidence’ that the mean weights of Americans and Japanese differ.

24 / 49

Solution to Example 6.3

It should be noted that these calculations assume that weight has a normal distribution. Since weight has a slightly skewed distribution, the calculation of the confidence interval is not totally accurate. The evidence that the mean weights differ is thus not so strong as stated.

25 / 49

6.2 Tests for a population proportion Suppose we have a large sample. We wish to test between the following two hypotheses H0 :p = p0 HA :p 6= p0 , where p is the proportion of a population showing a given trait. The obvious estimator of this proportion is the sample proportion pˆ =

X , n

where X is the number of individuals exhibiting the given trait in a sample of n individuals. 26 / 49

Testing procedure

The test statistic for this test is Z=

pˆ − p0 , S.E .(ˆ p)

where S.E .(ˆ p ) is the standard error of the sample proportion. The p-value for the test is 2P(Z > |t|), where t is the realisation of the test statistic. At a significance level of α, we reject H0 : p = p0 if |t| > t∞, α2 .

27 / 49

Standard error of the sample proportion

The standard error of this estimate is r p(1 − p) S.E .(ˆ p) = . n Under the null hypothesis p = p0 , r S.E .(ˆ p) =

p0 (1 − p0 ) . n

This formula for the standard error is used in the standard testing procedure.

28 / 49

Confidence intervals for a population proportion

A 100(1-α)% confidence interval for a population proportion is given by pˆ ± t∞, α2 S.E .(ˆ p ). When calculating a confidence interval, the standard error is approximated using the sample proportion i.e. r pˆ(1 − pˆ) . S.E .(ˆ p) ≈ n

29 / 49

Particular confidence intervals for a population proportion

In particular, a 95% confidence interval for a population proportion is given by pˆ ± t∞,0.025 S.E .(ˆ p ) = pˆ ± 1.96S.E .(ˆ p ). A 99% confidence interval for a population proportion is given by pˆ ± t∞,0.005 S.E .(ˆ p ) = pˆ ± 2.576S.E .(ˆ p ).

30 / 49

Approximate duality between confidence intervals for population proportions and tests

It should be noted that when we are testing the hypothesis H0 : p = p0 , due to the different formulae used to calculate the standard error of pˆ, the duality between confidence intervals and hypothesis tests is only approximate (in the standard testing procedure p = p0 is used to calculate the standard error, but p ≈ pˆ is used in the calculation of the confidence interval). e.g. using a 95% confidence to test the hypothesis H0 : p = p0 , the significance level would be approximately 5% .

31 / 49

Example 6.4

183 of 300 randomly picked Irish households have a PC. i) Test the null hypothesis that 55% of all Irish households have a PC. ii) Construct 95% and 99% confidence intervals for the proportion of Irish households owning a PC.

32 / 49

Solution to Example 6.4

i) The sample proportion is pˆ =

183 = 0.61 300

The test statistic is Z=

pˆ − p0 S.E .(ˆ p)

33 / 49

Solution to Example 6.4

Under H0 , the standard error of the sample proportion is r p0 (1 − p0 ) S.E .(ˆ p )= n r 0.55 × 0.45 = ≈ 0.0287. 300 The realisation of the test statistic is t=

0.61 − 0.55 ≈ 2.09. 0.0287

34 / 49

Conclusion

Since |t| > t∞,0.025 = 1.96 and |t| < t∞,0.005 = 2.576, we reject H0 at a significance level of 5%, but not at a significance level of 1% . We have evidence that the percentage is higher than 55% .

35 / 49

Calculation of confidence interval

The approximation of the standard error is r pˆ(1 − pˆ) S.E .(ˆ p )≈ n r 0.61 × 0.39 ≈ 0.028 = 300

36 / 49

Solution to Example 6.4

The 95% confidence interval for the proportion of all Irish households owning a PC is 0.61 ± 0.028 × 1.96 ≈ 0.61 ± 0.055 = [0.555, 0.665] Since 0.55 does not belong to this confidence interval, based on the confidence interval we reject the null hypothesis that p = 0.55 at a significance level of approximately 5% .

37 / 49

Solution to Example 6.4

The 99% confidence interval for the proportion of all Irish households owning a PC is 0.61 ± 0.028 × 2.576 ≈ 0.61 ± 0.073 = [0.537, 0.683] Since 0.55 belongs to this confidence interval, we do not reject the null hypothesis that p = 0.55 at a significance level of approximately 1% . Hence, we reject H0 : p = 0.55 at the 5% level, but not at the 1% level. We have evidence that the proportion of Irish households owning PCs is greater than 55% , but this evidence is not strong.

38 / 49

6.3 Testing for a difference between two population proportions

Suppose we have two large samples. We wish to test between the following two hypotheses H0 :p1 = p2 (i.e. p1 − p2 = 0) HA :p1 6= p2

39 / 49

Test statistic The test statistic for this test is Z=

pˆ1 − pˆ2 , S.E .(ˆ p1 − pˆ2 )

where pˆi is the sample proportion for the sample from population i. Using the standard testing procedure, the standard error for the difference between two sample proportions is estimated first by calculating the pooled proportion p. p=

x1 + x2 , n1 + n2

where xi is the number of individuals in sample i with the appropriate trait and ni is the size of sample i. 40 / 49

Testing for a difference between two population proportions

This pooled proportion is simply the proportion of all the individuals observed who have the trait of interest. Under H0 the standard error of the difference between the two proportions is estimated using p S.E .(ˆ p1 − pˆ2 ) ≈ p(1 − p)(1/n1 + 1/n2 ), where ni is the size of the sample from the population i.

41 / 49

Confidence intervals for the difference between two population proportions A 100(1-α)% confidence interval for the difference between two population proportions is given by (ˆ p1 − pˆ2 ) ± t∞, α2 S.E .(ˆ p1 − pˆ2 ). In particular, a 95% confidence interval for the difference between two population proportions is given by (ˆ p1 − pˆ2 ) ± t∞,0.025 S.E .(ˆ p1 − pˆ2 ) = (ˆ p1 − pˆ2 ) ± 1.96S.E .(ˆ p1 − pˆ2 ) A 99% confidence interval for the difference between two population proportions is given by (ˆ p1 − pˆ2 ) ± t∞,0.005 S.E .(ˆ p1 − pˆ2 ) = (ˆ p1 − pˆ2 ) ± 2.576S.E .(ˆ p1 − pˆ2 ) 42 / 49

Confidence intervals for the difference between two population proportions

When calculating a confidence interval for a difference between two proportions, the standard error is estimated using s pˆ1 (1 − pˆ1 ) pˆ2 (1 − pˆ2 ) + , S.E .(ˆ p1 − pˆ2 ) ≈ n1 n2 where ni is the size of the sample from the population i. Again, the duality between confidence intervals and hypothesis tests is only approximate.

43 / 49

Example 6.5

A long term study showed that 64 of 400 smokers contracted lung cancer, while 18 of 900 non-smokers contracted the disease. i) Test the null hypothesis that there is no association between smoking and the incidence of lung cancer at a significance level of 0.1% (i.e. the proportion of individuals getting lung cancer in both groups is the same). ii) Calculate a 95% confidence interval for the difference between the proportions of smokers and non-smokers contracting the disease.

44 / 49

Solution to Example 6.5

i) The sample proportions are 64 = 0.16 400 18 pˆ2 = = 0.02 900 pˆ1 =

The pooled proportion is p=

64 + 18 ≈ 0.06308 400 + 900

45 / 49

Solution to Example 6.5

Under H0 , the estimate of the standard error is s   1 1 S.E .(X −Y ) = 0.06308 × (1 − 0.06308) + ≈ 0.0146. 400 900 The realisation of the test statistic is t=

0.16 − 0.02 ≈ 9.58. 0.0146

46 / 49

Solution to Example 6.5

Since t > t∞,0.0005 = 3.291, we reject H0 at a significance level of 0.1% . We thus have very strong evidence of an association between smoking and lung cancer. One should note that the proportion of non-smokers who get lung cancer is very small. Hence, the normal approximation used is not very accurate. However, even taking this into account, since the realisation of the test statistic is so large, we can be very confident that our conclusion is correct.

47 / 49

Solution to Example 6.5

ii) When calculating a confidence interval, the estimate of the standard error for the difference between the sample proportions is s r pˆ1 (1 − pˆ1 ) pˆ2 (1 − pˆ2 ) 0.16 × 0.84 0.02 × 0.98 + = + n1 n2 400 900 ≈0.019

48 / 49

Solution to Example 6.5 A 95% confidence interval for the difference between the population proportions is given by (ˆ p1 − pˆ2 ) ± 1.96S.E .(ˆ p1 − pˆ2 )=(0.16 − 0.02) ± 1.96 × 0.019 =0.14 ± 0.037 = [0.103, 0.177] Since 0 does not belong to this 95% confidence interval, we reject the null hypothesis at a significance level of approximately 5% . Hence, we have evidence that the population proportions are not equal. From the data it can be seen that the proportion of smokers contracting lung cancer is higher than the proportion of non-smokers.

49 / 49