Check your calendar…
If you are qualified, Fill out make-up exam form by 5pm this Friday, attach necessary documents.
3/23/12
Lecture 20
1
Interpreting the ANOVA results Chapter 9
3/23/12
Lecture 20
2
Assumptions (prior to ANOVA) • Two important assumptions for ANOVA 1. Constant variance: The variances of the k populations are the same. –
Check this with the ratio of the largest and smallest standard deviations, the ratio must be < 2
2. Each of the k populations follows a normal distribution. –
3/23/12
Check this by looking at QQplots for each group
Lecture 19
3
Quick sidenote • ANOVA is a good example of a situation where often a nonsignificant test is actually useful. – Suppose we are comparing a new drug to several standard drugs already used – Suppose also that the new drug is less expensive to produce – In this case, mostly what we’d like to show is that the new drug is at least effective as the other standard drugs used – So in this situation, a non-significant ANOVA is a great result!
• Remember: statistical significance ≠ practical significance 3/23/12
Lecture 19
4
9.3 Interpreting ANOVA results • If the results are significant in ANOVA, we’d like to know explicitly which means are different • Remark: If insignificant in ANOVA, we don’t have to try further steps… • Two benefits of ANOVA
1. Single test with single chance (α) of type I error 2. Better estimation of error among all groups •
3/23/12
By comparing all the groups simultaneously, we get a better picture of the overall error among groups. Lecture 20
5
How do we know which means are different? • We need some "Supplementary Analysis" to tell that. – One way is to do a check visually, using the "effect plots" • Scatter plot of means • Side-By-Side Boxplots
– Another way is to perform multiple comparison of means • Tukey's method • Dunnett's method • many more: LSD, Scheffe, Bonferroni, FDR, etc. Actually, We can compare the means pairwise and keep the two benefits of ANOVA, we do this by adjusting the T value we use to compare the two means. 3/23/12
Lecture 20
6
Visual Check - Boxplots
3/23/12
Lecture 20
7
Visual Check – Scatterplot of Means
3/23/12
Lecture 20
8
Multiple comparisons (General Concepts) • How far apart do two means need to be to be statistically significant? – This value can be calculated directly similar to what we did with confidence intervals • It functions like a t from a t-test
– Generally, the critical value (t for example) is modified to “correct” the inflated type I error rate to keep it at the desired α level (like 0.05) • So instead of an α error rate for each test, we get an “family” α error rate—one rate for the entire comparison 3/23/12
Lecture 20
9
Tukey’s Method Controls the type I error rate directly by modifying the T value
•
T = qα • – –
•
3/23/12
MSE 1 1 ( + ) or 2 ni n j
T = qα
MSE if n = n i j ni
qα comes from the studentized range tables Table IX on pages 577-578 Df of qα is (k, n – k) for single-factor ANOVA
If the difference in two means is greater than this critical value, we say those two means are statistically significantly different Lecture 20
10
More simply • Calculate T, calculate the differences xi − x j • If xi − x j < T, means are not significantly different • If xi − x j > T, means ARE significantly different • Remark: Tukey’s method is conservative, sometimes its conclusion will be inconsistent with that by using ANOVA test results. If we took a less stringent alpha level we might see some of the significant differences. 3/23/12
Lecture 20
11
Example—generic • Let’s say we are comparing 4 means with equal sample sizes of ni = 5 for all i. With an MSE of 10. • Looking at Table IX, we have k = 4, and Error df = n – k = 16 – qα = 4.05
• So, T = qα
MSE 10 = 4.05 = 5.73 ni 5
• Any difference of means more than 5.73 apart would be significant different 3/23/12
Lecture 20
12
Example—generic • Suppose that you have four treatment groups and the treatment means are:
• TRT 1: • TRT 2: • TRT 3: • TRT 4:
52 63 58 54
• Which pairs are significantly different?
3/23/12
Lecture 20
13
Example (from Monday’s Class) • For the cereal example, let’s use Tukey’s method using α = 0.01 • The means are (arranged in descending order):
n4 = 5, x4 = 27.2 n3 = 4, x3 = 19.5 n1 = 5, x1 = 14.6 n2 = 5, x2 = 13.4 • Note, group 3’s sample size, what effect will that have on the comparisons? 3/23/12
Lecture 20
14
Another Method: Using SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey cldiff; run;
3/23/12
Lecture 20
15
Example (cont) using SAS • Notice it stars the pairs that are significantly different. • So the only pairs that are significantly different are: 1 and 4 2 and 4
3/23/12
Lecture 20
16
Alternate SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey lines; run;
3/23/12
Lecture 20
17
Example (cont) using SAS •
•
3/23/12
Lecture 20
Same information as before, differences are: 1 and 4 2 and 4 Notice the nice “groupings” though
18
Dunnett’s Multiple Comparison
3/23/12
Lecture 20
19
3/23/12
Lecture 20
20
Multiple Comparison – Dunnett’s Method
3/23/12
Lecture 20
21
Dunnett’s Method Functions like a Tukey, just uses a different T
•
1 1 T = tα (k − 1, n − k ) MSE ( + ) ni nC •
tα comes from the Dunnett’s t table
– Table X on page 579 – Only use when one of the groups is a control group – Only interested in comparing the “other” groups to the control group
• 3/23/12
Again we take pairwise differences, Lecture 20
xi − xC 22
Example (Cereal Design) using SAS • Dunnett’s SAS code, pretend design 1 was the regular design already used proc glm data=cereal alpha=0.01; class design; model cases = design; means design / dunnett(“1”) cldiff; run; Note: lines doesn’t work with Dunnett’s 3/23/12
Lecture 20
23
Example (cont) using SAS Dunnett's t Tests for cases NOTE: This test controls the Type I experimentwise error for comparisons of all treatments against a control. Alpha Error Degrees of Freedom Error Mean Square Critical Value of Dunnett's t
0.01 15 10.54667 3.43026
• So if group1 was the control, only group 4 is significantly different
Comparisons significant at the 0.01 level are indicated by ***. Difference design Between Simultaneous 99% Comparison Means Confidence Limits 4-1 3-1 2-1 3/23/12
12.600 4.900 -1.200
5.554 19.646 *** -2.573 12.373 -8.246 5.846 Lecture 20
24
Summary about ANOVA and Multiple Comparison • ANOVA (Analysis of Variances) – Check the assumptions (constant variance/normality) – Be able to do most of ANOVA by hand or by SAS both • Lots of hand calculations • Be able to read and interpret SAS output
– For Hw, do it either way you like, but for the exam be prepared to do both!
• Multiple Comparison methods (ONLY when ANOVA result is significant) – are useful in other situations, but they all involve calculating a T value and using it to compare pairs of means – Tukey’s is approprirate if there’s no control group; try Dunnett’s if there is any control(s)
3/23/12
Lecture 20
25
After Class • Hw#8, due by 5pm next Monday • Start review of Exam 2 (Ch.7, 8 and Monday’s complete notes) – Practice Test 2 – Hw5-8, Lab 3 and 4
3/23/12
Lecture 20
26