Check your calendar…

If you are qualified, Fill out make-up exam form by 5pm this Friday, attach necessary documents.

3/23/12

Lecture 20

1

Interpreting the ANOVA results Chapter 9

3/23/12

Lecture 20

2

Assumptions (prior to ANOVA) •  Two important assumptions for ANOVA 1.  Constant variance: The variances of the k populations are the same. – 

Check this with the ratio of the largest and smallest standard deviations, the ratio must be < 2

2.  Each of the k populations follows a normal distribution. – 

3/23/12

Check this by looking at QQplots for each group

Lecture 19

3

Quick sidenote •  ANOVA is a good example of a situation where often a nonsignificant test is actually useful. –  Suppose we are comparing a new drug to several standard drugs already used –  Suppose also that the new drug is less expensive to produce –  In this case, mostly what we’d like to show is that the new drug is at least effective as the other standard drugs used –  So in this situation, a non-significant ANOVA is a great result!

•  Remember: statistical significance ≠ practical significance 3/23/12

Lecture 19

4

9.3 Interpreting ANOVA results •  If the results are significant in ANOVA, we’d like to know explicitly which means are different •  Remark: If insignificant in ANOVA, we don’t have to try further steps… •  Two benefits of ANOVA

1.  Single test with single chance (α) of type I error 2.  Better estimation of error among all groups • 

3/23/12

By comparing all the groups simultaneously, we get a better picture of the overall error among groups. Lecture 20

5

How do we know which means are different? •  We need some "Supplementary Analysis" to tell that. –  One way is to do a check visually, using the "effect plots" •  Scatter plot of means •  Side-By-Side Boxplots

–  Another way is to perform multiple comparison of means •  Tukey's method •  Dunnett's method •  many more: LSD, Scheffe, Bonferroni, FDR, etc. Actually, We can compare the means pairwise and keep the two benefits of ANOVA, we do this by adjusting the T value we use to compare the two means. 3/23/12

Lecture 20

6

Visual Check - Boxplots

3/23/12

Lecture 20

7

Visual Check – Scatterplot of Means

3/23/12

Lecture 20

8

Multiple comparisons (General Concepts) •  How far apart do two means need to be to be statistically significant? –  This value can be calculated directly similar to what we did with confidence intervals •  It functions like a t from a t-test

–  Generally, the critical value (t for example) is modified to “correct” the inflated type I error rate to keep it at the desired α level (like 0.05) •  So instead of an α error rate for each test, we get an “family” α error rate—one rate for the entire comparison 3/23/12

Lecture 20

9

Tukey’s Method Controls the type I error rate directly by modifying the T value

• 

T = qα •  –  – 

• 

3/23/12

MSE 1 1 ( + ) or 2 ni n j

T = qα

MSE if n = n i j ni

qα comes from the studentized range tables Table IX on pages 577-578 Df of qα is (k, n – k) for single-factor ANOVA

If the difference in two means is greater than this critical value, we say those two means are statistically significantly different Lecture 20

10

More simply •  Calculate T, calculate the differences xi − x j •  If xi − x j < T, means are not significantly different •  If xi − x j > T, means ARE significantly different •  Remark: Tukey’s method is conservative, sometimes its conclusion will be inconsistent with that by using ANOVA test results. If we took a less stringent alpha level we might see some of the significant differences. 3/23/12

Lecture 20

11

Example—generic •  Let’s say we are comparing 4 means with equal sample sizes of ni = 5 for all i. With an MSE of 10. •  Looking at Table IX, we have k = 4, and Error df = n – k = 16 –  qα = 4.05

•  So, T = qα

MSE 10 = 4.05 = 5.73 ni 5

•  Any difference of means more than 5.73 apart would be significant different 3/23/12

Lecture 20

12

Example—generic •  Suppose that you have four treatment groups and the treatment means are:

• TRT 1: • TRT 2: • TRT 3: • TRT 4:

52 63 58 54

•  Which pairs are significantly different?

3/23/12

Lecture 20

13

Example (from Monday’s Class) •  For the cereal example, let’s use Tukey’s method using α = 0.01 •  The means are (arranged in descending order):

n4 = 5, x4 = 27.2 n3 = 4, x3 = 19.5 n1 = 5, x1 = 14.6 n2 = 5, x2 = 13.4 •  Note, group 3’s sample size, what effect will that have on the comparisons? 3/23/12

Lecture 20

14

Another Method: Using SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey cldiff; run;

3/23/12

Lecture 20

15

Example (cont) using SAS •  Notice it stars the pairs that are significantly different. •  So the only pairs that are significantly different are: 1 and 4 2 and 4

3/23/12

Lecture 20

16

Alternate SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey lines; run;

3/23/12

Lecture 20

17

Example (cont) using SAS • 

• 

3/23/12

Lecture 20

Same information as before, differences are: 1 and 4 2 and 4 Notice the nice “groupings” though

18

Dunnett’s Multiple Comparison

3/23/12

Lecture 20

19

3/23/12

Lecture 20

20

Multiple Comparison – Dunnett’s Method

3/23/12

Lecture 20

21

Dunnett’s Method Functions like a Tukey, just uses a different T

• 

1 1 T = tα (k − 1, n − k ) MSE ( + ) ni nC • 

tα comes from the Dunnett’s t table

–  Table X on page 579 –  Only use when one of the groups is a control group –  Only interested in comparing the “other” groups to the control group

•  3/23/12

Again we take pairwise differences, Lecture 20

xi − xC 22

Example (Cereal Design) using SAS •  Dunnett’s SAS code, pretend design 1 was the regular design already used proc glm data=cereal alpha=0.01; class design; model cases = design; means design / dunnett(“1”) cldiff; run; Note: lines doesn’t work with Dunnett’s 3/23/12

Lecture 20

23

Example (cont) using SAS Dunnett's t Tests for cases NOTE: This test controls the Type I experimentwise error for comparisons of all treatments against a control. Alpha Error Degrees of Freedom Error Mean Square Critical Value of Dunnett's t

0.01 15 10.54667 3.43026

•  So if group1 was the control, only group 4 is significantly different

Comparisons significant at the 0.01 level are indicated by ***. Difference design Between Simultaneous 99% Comparison Means Confidence Limits 4-1 3-1 2-1 3/23/12

12.600 4.900 -1.200

5.554 19.646 *** -2.573 12.373 -8.246 5.846 Lecture 20

24

Summary about ANOVA and Multiple Comparison •  ANOVA (Analysis of Variances) –  Check the assumptions (constant variance/normality) –  Be able to do most of ANOVA by hand or by SAS both •  Lots of hand calculations •  Be able to read and interpret SAS output

–  For Hw, do it either way you like, but for the exam be prepared to do both!

•  Multiple Comparison methods (ONLY when ANOVA result is significant) –  are useful in other situations, but they all involve calculating a T value and using it to compare pairs of means –  Tukey’s is approprirate if there’s no control group; try Dunnett’s if there is any control(s)

3/23/12

Lecture 20

25

After Class •  Hw#8, due by 5pm next Monday •  Start review of Exam 2 (Ch.7, 8 and Monday’s complete notes) –  Practice Test 2 –  Hw5-8, Lab 3 and 4

3/23/12

Lecture 20

26