If you are qualified, Fill out make-up exam form by 5pm this Friday, attach necessary documents.

3/23/12

Lecture 20

1

Interpreting the ANOVA results Chapter 9

3/23/12

Lecture 20

2

Assumptions (prior to ANOVA) • Two important assumptions for ANOVA 1. Constant variance: The variances of the k populations are the same. –

Check this with the ratio of the largest and smallest standard deviations, the ratio must be < 2

2. Each of the k populations follows a normal distribution. –

3/23/12

Check this by looking at QQplots for each group

Lecture 19

3

Quick sidenote • ANOVA is a good example of a situation where often a nonsignificant test is actually useful. – Suppose we are comparing a new drug to several standard drugs already used – Suppose also that the new drug is less expensive to produce – In this case, mostly what we’d like to show is that the new drug is at least effective as the other standard drugs used – So in this situation, a non-significant ANOVA is a great result!

• Remember: statistical significance ≠ practical significance 3/23/12

Lecture 19

4

9.3 Interpreting ANOVA results • If the results are significant in ANOVA, we’d like to know explicitly which means are different • Remark: If insignificant in ANOVA, we don’t have to try further steps… • Two benefits of ANOVA

1. Single test with single chance (α) of type I error 2. Better estimation of error among all groups •

3/23/12

By comparing all the groups simultaneously, we get a better picture of the overall error among groups. Lecture 20

5

How do we know which means are different? • We need some "Supplementary Analysis" to tell that. – One way is to do a check visually, using the "effect plots" • Scatter plot of means • Side-By-Side Boxplots

– Another way is to perform multiple comparison of means • Tukey's method • Dunnett's method • many more: LSD, Scheffe, Bonferroni, FDR, etc. Actually, We can compare the means pairwise and keep the two benefits of ANOVA, we do this by adjusting the T value we use to compare the two means. 3/23/12

Lecture 20

6

Visual Check - Boxplots

3/23/12

Lecture 20

7

Visual Check – Scatterplot of Means

3/23/12

Lecture 20

8

Multiple comparisons (General Concepts) • How far apart do two means need to be to be statistically significant? – This value can be calculated directly similar to what we did with confidence intervals • It functions like a t from a t-test

– Generally, the critical value (t for example) is modified to “correct” the inflated type I error rate to keep it at the desired α level (like 0.05) • So instead of an α error rate for each test, we get an “family” α error rate—one rate for the entire comparison 3/23/12

Lecture 20

9

Tukey’s Method Controls the type I error rate directly by modifying the T value

•

T = qα • – –

•

3/23/12

MSE 1 1 ( + ) or 2 ni n j

T = qα

MSE if n = n i j ni

qα comes from the studentized range tables Table IX on pages 577-578 Df of qα is (k, n – k) for single-factor ANOVA

If the difference in two means is greater than this critical value, we say those two means are statistically significantly different Lecture 20

10

More simply • Calculate T, calculate the differences xi − x j • If xi − x j < T, means are not significantly different • If xi − x j > T, means ARE significantly different • Remark: Tukey’s method is conservative, sometimes its conclusion will be inconsistent with that by using ANOVA test results. If we took a less stringent alpha level we might see some of the significant differences. 3/23/12

Lecture 20

11

Example—generic • Let’s say we are comparing 4 means with equal sample sizes of ni = 5 for all i. With an MSE of 10. • Looking at Table IX, we have k = 4, and Error df = n – k = 16 – qα = 4.05

• So, T = qα

MSE 10 = 4.05 = 5.73 ni 5

• Any difference of means more than 5.73 apart would be significant different 3/23/12

Lecture 20

12

Example—generic • Suppose that you have four treatment groups and the treatment means are:

• TRT 1: • TRT 2: • TRT 3: • TRT 4:

52 63 58 54

• Which pairs are significantly different?

3/23/12

Lecture 20

13

Example (from Monday’s Class) • For the cereal example, let’s use Tukey’s method using α = 0.01 • The means are (arranged in descending order):

n4 = 5, x4 = 27.2 n3 = 4, x3 = 19.5 n1 = 5, x1 = 14.6 n2 = 5, x2 = 13.4 • Note, group 3’s sample size, what effect will that have on the comparisons? 3/23/12

Lecture 20

14

Another Method: Using SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey cldiff; run;

3/23/12

Lecture 20

15

Example (cont) using SAS • Notice it stars the pairs that are significantly different. • So the only pairs that are significantly different are: 1 and 4 2 and 4

3/23/12

Lecture 20

16

Alternate SAS code proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey lines; run;

3/23/12

Lecture 20

17

Example (cont) using SAS •

•

3/23/12

Lecture 20

Same information as before, differences are: 1 and 4 2 and 4 Notice the nice “groupings” though

18

Dunnett’s Multiple Comparison

3/23/12

Lecture 20

19

3/23/12

Lecture 20

20

Multiple Comparison – Dunnett’s Method

3/23/12

Lecture 20

21

Dunnett’s Method Functions like a Tukey, just uses a different T

•

1 1 T = tα (k − 1, n − k ) MSE ( + ) ni nC •

tα comes from the Dunnett’s t table

– Table X on page 579 – Only use when one of the groups is a control group – Only interested in comparing the “other” groups to the control group

• 3/23/12

Again we take pairwise differences, Lecture 20

xi − xC 22

Example (Cereal Design) using SAS • Dunnett’s SAS code, pretend design 1 was the regular design already used proc glm data=cereal alpha=0.01; class design; model cases = design; means design / dunnett(“1”) cldiff; run; Note: lines doesn’t work with Dunnett’s 3/23/12

Lecture 20

23

Example (cont) using SAS Dunnett's t Tests for cases NOTE: This test controls the Type I experimentwise error for comparisons of all treatments against a control. Alpha Error Degrees of Freedom Error Mean Square Critical Value of Dunnett's t

0.01 15 10.54667 3.43026

• So if group1 was the control, only group 4 is significantly different

Comparisons significant at the 0.01 level are indicated by ***. Difference design Between Simultaneous 99% Comparison Means Confidence Limits 4-1 3-1 2-1 3/23/12

12.600 4.900 -1.200

5.554 19.646 *** -2.573 12.373 -8.246 5.846 Lecture 20

24

Summary about ANOVA and Multiple Comparison • ANOVA (Analysis of Variances) – Check the assumptions (constant variance/normality) – Be able to do most of ANOVA by hand or by SAS both • Lots of hand calculations • Be able to read and interpret SAS output

– For Hw, do it either way you like, but for the exam be prepared to do both!

• Multiple Comparison methods (ONLY when ANOVA result is significant) – are useful in other situations, but they all involve calculating a T value and using it to compare pairs of means – Tukey’s is approprirate if there’s no control group; try Dunnett’s if there is any control(s)

3/23/12

Lecture 20

25

After Class • Hw#8, due by 5pm next Monday • Start review of Exam 2 (Ch.7, 8 and Monday’s complete notes) – Practice Test 2 – Hw5-8, Lab 3 and 4

3/23/12

Lecture 20

26