Chapter 10: ANOVA: The Analysis of Variance Dr. Sharabati Purdue University
April 10, 2014
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
1 / 26
The Analysis of Variance (ANOVA)
When data are sampled from more than two numerical populations (distributions) Example: Are the mean taste ratings of Papa John’s Pizza, Domino’s Pizza, Pizza Hut’s pizza the same if you conduct a taste test?
When data are from experiments in which more than two treatments have been used Example: How the consumption of salt affects a person’s blood pressure?
The characteristic that differentiates the treatments or populations is called the factor under study. The different treatments or populations are referred to as the levels of the factor.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
2 / 26
Example 1: A random sample of 15 healthy young men are split randomly into 3 groups of 5. They receive 0, 20, and 40 mg of the drug Paxil per day for a week and their serotonin levels are then measured. Does Paxil affect serotonin levels in healthy young men? Dose of Paxil 0mg 20mg 40mg 1 48.62 58.60 68.59 2 49.85 72.52 78.28 3 64.22 66.72 82.77 4 62.81 80.12 76.53 5 62.51 68.44 72.33 Sample Size 5 5 5 Sample Mean 57.60 69.28 75.70 Sample SD 7.68 7.90 5.46
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
3 / 26
Boxplot of the data in exmaple 1 (Serotonin Level vs. Dose of Paxil)
80
75
70
65
60
55
50
0mg
Dr. Sharabati (Purdue University)
20mg
ANOVA
40mg
Spring 2014
4 / 26
Single-Factor (One-Way) ANOVA
Goal Is there at least one mean that is statistically significantly different from the others? Null hypothesis H0 : µ1 = µ2 = · · · = µI . Alternative hypothesis Ha : Not all the population means are equal (at least one is different).
Notation I = number of groups. µi = the population mean of the ith group. σi = the population standard deviation of the ith group.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
5 / 26
Idea of The Analysis of Variance (ANOVA)
The ANOVA test will compare variation due to specific sources: between treatment groups and within treatment groups.
To check if the differences between the groups’ means are small enough so that they can be explained by the variability of data observed within groups.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
6 / 26
The Analysis of Variance (ANOVA)
Assumptions of ANOVA Normality: Each of the I populations follow a normal distribution. Common standard deviation: The population standard deviations are equal across the I populations, i.e., σ1 = σ2 = · · · = σI = σ. Steps of ANOVA Calculate ANOVA table Do hypothesis test in the usual way
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
7 / 26
Notation I = number of groups. ni = then number of observations in the ith group. x = the j th observation in the ith group. ij
x ¯i• = the sample mean of the ith group, i.e., ni 1 X x ¯i• = xij ni j=1
N = n1 + n2 + · · · + ni + · · · + nI = total number of observations. x ¯•• = the grand mean all the observations, i.e., x ¯••
I ni 1 XX = xij N i=1 j=1
si = the sample standard deviation of the ith group. Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
8 / 26
Variation Within Treatment Groups Sum of Squares: SSE (Sum of Squares for Error) SSE =
I X
SSi ,
i=1
where SSi =
ni P
(xij − x ¯i• )2 = (ni − 1)s2i
j=1
Degrees of Freedom: DFE = N − I Mean Squares: MSE (Mean Squares for Error) MSE =
SSE SS1 + SS2 + · · · + SSI = DFE (n1 − 1) + (n2 − 1) + · · · + (nI − 1)
The mean squares for error is the pooled estimate of σ 2 . Pooled estimate of σ is √ spooled = MSE Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
9 / 26
Variation Between Treatment Groups
Sum of Squares: SSTr (Sum of Squares for Treatments) SSTr =
I X
ni (¯ xi• − x ¯•• )2
i=1
Degrees of Freedom: DFTr = I − 1 Mean Squares: MSTr (Mean Squares for Treatments) MSTr =
Dr. Sharabati (Purdue University)
ANOVA
SSTr DFTr
Spring 2014
10 / 26
Total Variation
Sum of Squares: SST (Sum of Squares for Total) SST =
ni I X X (xij − x ¯•• )2 = SSTr + SSE i=1 j=1
Degrees of Freedom: DFT = N − 1 = DFTr + DFE
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
11 / 26
The ANOVA Table
Results of ANOVA are commonly organized into a table as follows:
Source Treatment
DF DFTr = I − 1
SS SSTr =
I P
ni (¯ xi• − x ¯•• )2
i=1
Error Total
DFE = N − I DFT = N − 1
SSE =
ni I P P
SST =
i=1 j=1 ni I P P
(xij − x ¯i• )2
MS SSTr DFTr SSE MSE = DFE
MSTr =
(xij − x ¯•• )2
i=1 j=1
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
12 / 26
The Global F Test
1. Hypotheses: H0 : µ1 = µ2 = · · · = µI Ha : Not all µ’s are equal Rejecting H0 does not tell us which means were different. We need to do further analysis if H0 is rejected.
2. Test statistic:
MSTr MSE Under the null hypothesis, this statistic has an F −distribution with F =
DF T r = I − 1 as numerator degrees of freedom DF E = N − I as denominator degrees of freedom
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
13 / 26
3. Rejection region or p-value are computed using the F −distribution. Rejection Region α FDFTr,, DFE(α α)
P value p value
F
4. Conclusion. Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
14 / 26
Example 1 Continued - ANOVA Dose
ni x ¯i• SSi ni (¯ xi• − x ¯•• )2 Source Treatment Error Total Dr. Sharabati (Purdue University)
DF 2 12 14
0mg 48.62 49.85 64.22 62.81 62.51 5 57.60 235.87 492.64
20mg 58.60 72.52 66.72 80.12 68.44 5 69.28 249.31 15.36
SS 841.88 604.34 1446.23 ANOVA
MS 420.94 50.36
40mg 68.59 78.28 82.77 76.53 72.33 5 75.70 119.29 334.03
overall 15 67.53 604.47 842.02
F 8.36
P-value 0.0053
Spring 2014
15 / 26
Example 1 Continued - Use the global F-Test
H0 : µ1 = µ2 = µ3 ; mean serotonin levels are the same at all 3 dosage levels [or, mean serotonin levels are unaffected by Paxil dose] Ha : The mean serotonin levels of the three groups are not all equal. [or, serotonin levels are affected by Paxil dose] F = MSTr/MSE has an F distribution with df =(DFTr, DFE)=(2, 12) under H0 . Test at significance level α = 0.05. Critical value F.05,2,12 = 3.89. Observe f = 8.36 > F.05,2,12 . Reject H0 . This study provides evidence at 5% significance level that Paxil intake affects serotonin levels in young men.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
16 / 26
Multiple Comparison
When the global F -test is rejected, further analysis is needed to find out which means are different. The intuitive approach is to perform multiple tests for each pairwise difference µi − µj . However, the probability of making a type I error for the entire collection of pairwise hypothesis tests is no longer α. There is a variety of multiple comparison methods that attempt to control the so-called family-wise error rate. These include (but are not limited to): Tukey’s multiple comparison procedure, Bonferroni adjustment, Scheffe’s multiple comparison procedure.
We will only talk about Tukey procedures.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
17 / 26
Tukey’s Multiple Comparison Procedure
When the number of treatments is large and you want to perform all possible pairwise comparisons, the Tukey procedure is more powerful.
The Bonferroni’s procedure leads to conservative confidence intervals and tests in that the actual probability of making at least one error among the g pairs is usually less than α. The Bonferroni’s procedure is powerful when there are only a few comparisons.
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
18 / 26
Tukey’s Multiple Comparison Procedure The test statistic for testing H0 : µi = µj is tij = r
x ¯i• − x ¯j• MSE 1 + 2 ni
1 nj
This is the Tukey-Kramer procedure. When the sample sizes in each group are equal (to n say), this simplifies to x ¯i• − x ¯j• tij = q MSE n
In this case, it is called the Tukey procedure. If tij > qI,DFE (α), we declare that the population means µi and µj are significantly different. The critical value qI,DFE (α) is taken from the Studentized Range distribution (Appendix Table A.10) based on m = I and ν = DF E. Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
19 / 26
The 100(1 − α)% family-wise confidence intervals are given by: s 1 MSE 1 + (¯ xi• − x ¯j• ) ± qI,DFE (α) 2 ni nj
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
20 / 26
Example 1 Continued
For α = 0.05, qI,DFE (α) = q3,12 (0.05) = 3.773. The Tukey procedure has the following results: Test statistic
Confidence Interval
Conclusion
µ1 − µ2 µ1 − µ3 µ2 − µ3
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
21 / 26
SAS Code for Tukey Procedure
proc anova data=paxil; class dose; model serotonin = dose; means dose /lines tukey; run;quit;
Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
22 / 26
SAS Output Tukey’s Studentized Range (HSD) Test for serotonin Alpha 0.05 Error Degrees of Freedom 12 Error Mean Square 50.36196 Critical Value of Studentized Range 3.77278 Minimum Significant Difference 11.974 Means with the same letter are not significantly different. Tukey Grouping A A B A B B Dr. Sharabati (Purdue University)
Mean 75.700
N 5
dose 40
69.280
5
20
57.602
5
0
ANOVA
Spring 2014
23 / 26
Graphical Presentation
A common way to show the results from such pairwise comparisons is to arrange the treatments in increasing order and underline the groups of treatments that are not significantly different. For our paxil experiment we find 0 mg
20 mg
Dr. Sharabati (Purdue University)
40 mg
ANOVA
Spring 2014
24 / 26
Summary of Single-Factor Analysis of Variance (ANOVA) Goal: Is there at least one mean that is statistically significantly different from the others? If it is, further analysis is needed to find out which means are different. The Global F Test 1. Hypotheses: H0 : µ1 = µ2 = · · · = µI Ha : Not all µ’s are equal Rejecting H0 does not tell us which means were different. We need to do further analysis if H0 is rejected.
2. Test statistic:
MSTr MSE Under the null hypothesis, this statistic has an F −distribution with DF T r = I − 1 and DF E = N − I. F =
Multiple Comparison (e.g. Tukey’s procedure) when the global F -test is rejected. Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
25 / 26
The Analysis of Variance (ANOVA)
Details about the F -test Define all the µ’s State H0 in equations and words State Ha in words only Test statistic is F = MSTr/MSE Under H0 , F has an F distribution with DFTr, DFE In Table A.9 look up critical value Fcritical for DFTr, DFE, where “numerator df” = DFTr and “denominator df” = DFE Reject H0 if observed f > Fcritical Or, calculate a P-value. State scientific conclusion such as ”This study (does not) provide(s) evidence at the α significance level that there is a difference in among the groups.” Dr. Sharabati (Purdue University)
ANOVA
Spring 2014
26 / 26