Review of ANOVA Computer Output Interpretation • This is from Page 104 of your text Montgomery 5th edition section 3-6. • There are many computer Statistical analysis programs available, • Popular ones include: • EXCEL Data Analysis Add in: Limited • Design Expert • Jump • Minitab • Also all programs have built in help tutorials if you forget or get lost with these tests!! ANOVA Computer Output Steve Brainerd
1
Review of ANOVA Computer Output Interpretation • Example used here:
ANOVA Computer Output Steve Brainerd
2
Review of ANOVA Computer Output Interpretation: We are asking question: Are means between treatments different? And By how much? • First attack look at graphs:from Design Expert
• • •
It appears 20, 25, 30% cotton levels are different than 15 and 35%, By how much? Is this due to chance variation? How do we assign a confidence value to any decision we make? ANOVA Computer Output Steve Brainerd
3
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert: ANOVA Single Factor Typical ANOVA Table with Sum of Squares
M for Model terms
e for error terms
ANOVA Computer Output Steve Brainerd
4
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
If there were more than one factor or source there would be A, B, C etc listed
ANOVA Computer Output Steve Brainerd
5
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor Design Expert Analysis ANOVA Terms Model: Terms estimating factor effects. For 2-level factorials: those that "fall off" the normal probability line of the effects plot. Sum of Squares: Total of the sum of squares for the terms in the model, as reported in the Effects List for factorials and on the Model screen for RSM, MIX and Crossed designs.
DF: Degrees of freedom for the model. It is the number of model terms, including the intercept, minus one. Mean Square: Estimate of the model variance, calculated by the model sum of squares divided by model degrees of freedom.
F Value: Test for comparing model variance with residual (error) variance. If the variances are close to the same, the ratio will be close to one and it is less likely that any of the factors have a significant effect on the response. Calculated by Model Mean Square divided by Residual Mean Square.
Probe > F: Probability of seeing the observed F value if the null hypothesis is true (there is no factor effect). Small probability values call for rejection of the null hypothesis. The probability equals the proportion of the area under the curve of the F-distribution that lies beyond the observed F value. The F distribution itself is determined by the degrees of freedom associated with the variances being compared.
(In "plain English", if the Probe>F value is very small (less than 0.05) then the terms in the model have a significant effect on the response.) ANOVA Computer Output Steve Brainerd
6
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
Std Dev: (Root MSe ) Square root of the residual mean square. Consider this to be an
estimate of the standard deviation associated with the experiment.
SQRT(8.06) = 2.84 Mean: Overall average of all the response data. Grand mean = 15.04 C.V.: Coefficient of Variation, the standard deviation expressed as a percentage of the mean. Calculated by dividing the Std Dev by the Mean and multiplying by 100.
CV =2.84/15.04 x 100 =18.88% ANOVA Computer Output Steve Brainerd
7
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
PRESS: Predicted Residual Error Sum of Squares – Basically a measure of how well the model from this experiment is likely to predict the response in a new experiment.. Small values are desirable The PRESS is computed by first predicting where each point should be from a model that contains all other points except the one in question. The squared residuals (difference between actual and predicted values) are then summed.
ANOVA Computer Output Steve Brainerd
8
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
R2 Factor A model (% cotton) explains 74.69% variability in response (Tensile strength) R-Squared: A measure of the amount of variation around the mean explained by the model. R2 = 1-(SSresidual / (SSmodel + SSresidual)) =SSmodel/SStotal = 475.76/636.96 = 0.7469
Adjusted R2 Adj R-Squared: A measure of the amount of variation around the mean explained by the model, adjusted for the number of terms in the model. The adjusted Rsquared decreases as the number of terms in the model increases if those additional terms don’t add value to the model.
1-((SSresidual / DFresidual) / ((SSmodel + SSresidual) / (DFmodel + DFresidual))) = 1-((161.2/20)/((475.76+161.2)/(4 + 20))) =1-((161.20/20)/(636.96/24)) = 1(8.06/26.54) = 0.6963 ANOVA Computer Output Steve Brainerd
9
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
Pred R-Squared: A measure of the amount of variation in new data explained by
the model. 1-(PRESS / (SStotal-SSblock) = 1-(251.87/(636.96 – 0) = 1-(251.87/636.96) = 1 – 0.3954 = 0.6046 = 60.46% (Remember the model we generated accounts for 74% of the observed variation in tensile strength from the % cotton factor.) The predicted r-squared and the adjusted r-squared should be within 0.20 of each other. Otherwise there may be a problem with either the data or the model. Look for outliers, consider transformations, or consider a different order polynomial. ANOVA Computer Output Steve Brainerd
10
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
Adequate Precision: Basically a measure of S/N ( signal to noise ratio), It gives you a factor by which you can judge your model to see if it “adequate” to navigate through the design space and be able to predict the response. Desire values > 4.0 Computed by : (Maximum predicted response – Minimum predicted response)/ (Average standard deviation of all predicted responses) Adequate Precision = (21.60 – 9.80)/(sqrt(MSe/5)) =11.80/(sqrt(8.06/5)) = 11.80/(sqrt(1.6119 ) = 11.80/1.2696 = 9.294 ANOVA Computer Output Steve Brainerd
11
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
Coefficient Estimate: Regression coefficient representing the expected change in response y per unit change in x when all remaining factors are held constant. In orthogonal designs, it equals one half the factorial effect.
DF: Degrees of Freedom – equal to one for testing coefficients. Standard Error: The standard deviation associated with the coefficient estimate. 95% CI High and Low: These two columns represent the range that the true coefficient should be found in 95% of the time. If this range spans 0 (one limit is positive and the other negative) then the coefficient of 0 could be true, indicating the factor has no effect.
ANOVA Computer Output Steve Brainerd
12
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics:ANOVA Single Factor
Treatment mean = estimate of the effect for each % cotton level ( treatment) Standard error = estimate of the sample standard deviation for that effect (treatment) = sqrt(MSe/n) ANOVA Computer Output Steve Brainerd
13
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: ANOVA Single Factor Differences in pairs of treatment LSD analysis Design Expert VS JUMP
ANOVA Computer Output Steve Brainerd
14
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: ANOVA Single Factor Design
ANOVA Computer Output Steve Brainerd
15
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: : ANOVA Single Factor
Leverage is the potential for a design point to influence the fit of the model coefficients, based on its position in the design space. Leverages near 1 should be avoided. Replicate the point or add more design points to reduce leverage. ANOVA Computer Output Steve Brainerd
16
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: : ANOVA Single Factor
The Student Residual is the number of standard deviations that separate the actual and predicted response values. It is the residual divided by the estimated standard deviation of the residual.
ANOVA Computer Output Steve Brainerd
17
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: : ANOVA Single Factor
The Cook's distance for this observation--not to be confused with the distance between the dining room and kitchen. It is a measure of how much the regression equation changes if this specific run is deleted. It is roughly a combination of leverage and outlier-T and can be used to help identify individual runs that may be outliers.
ANOVA Computer Output Steve Brainerd
18
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Diagnostics: ANOVA Single Factor
The outlier t test checks whether a run is consistent with the other runs, assuming the chosen model holds. The model coefficients are calculated based on all of the design points except one. A prediction of the response at this point is made. The residual is evaluated using the t-test. A value greater than 3.5 means that this point should be examined as a possible outlier. ANOVA Computer Output Steve Brainerd
19
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: Change Factor 1 to Numeric
ANOVA Computer Output Steve Brainerd
20
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: Click on Status: Model Order Cubic; Right click on Coefficient and make Model M
Click on Analysis: Note new menu options: Click on Fit Summary:
ANOVA Computer Output Steve Brainerd
21
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: FIT SUMMARY
ANOVA Computer Output Steve Brainerd
22
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: Click on Model;
Next Click on ANOVA
ANOVA Computer Output Steve Brainerd
23
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: ANOVA Screen for Cubic Model Suggested>>
ANOVA Computer Output Steve Brainerd
24
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: ANOVA Screen for Cubic Model Suggested>> Model Equations!!
ANOVA Computer Output Steve Brainerd
25
Review of ANOVA Computer Output Interpretation STAT EASE Design Expert Create a Model: Model Graphic Screen for Cubic Model Suggested
ANOVA Computer Output Steve Brainerd
26
Review of ANOVA Computer Output Interpretation JUMP Design Expert Model Fitting
ANOVA Computer Output Steve Brainerd
27
Review of ANOVA Computer Output Interpretation JUMP Design Expert Model Fitting
ANOVA Computer Output Steve Brainerd
28
Review of ANOVA Computer Output Interpretation JUMP Design Expert Model Fitting 2nd Order
ANOVA Computer Output Steve Brainerd
29
Review of ANOVA Computer Output Interpretation JUMP Design Expert Model Fitting 3rd Order
ANOVA Computer Output Steve Brainerd
30
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Journal to save as text in WORD Save as Rich Text Format .RTF
ANOVA Computer Output Steve Brainerd
31
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor JUMP output
ANOVA Computer Output Steve Brainerd
32
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump Summary of Model fit and ANOVA table Same as Design Expert!
ANOVA Computer Output Steve Brainerd
33
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump Outputs Mean confidence limits on the estimate of each treatment mean!
ANOVA Computer Output Steve Brainerd
34
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
35
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Multiple Comparisons There are a variety of methods to test differences in group means (multiple comparisons) that vary in detail about how to size the test to accommodate different kinds of multiple comparisons. Fit Y by X automatically produces the standard analysis of variance and optionally offers the following four multiple comparison tests: Each Pair, Student’s t computes individual pairwise comparisons using Student’s t tests. This test is sized for individual comparisons. If you make many pairwise tests, there is no protection across the inferences, and thus the alpha-size (Type I) error rate across the hypothesis tests is higher than that for individual tests. All Pairs, Tukey HSD gives a test that is sized for all differences among the means. This is the Tukey or Tukey-Kramer HSD (honestly significant difference) test. (Tukey 1953, Kramer 1956). This test is an exact alpha-level test if the sample sizes are the same and conservative if the sample sizes are different (Hayter 1984).
ANOVA Computer Output Steve Brainerd
36
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
Group 25 % cotton just used as a example
ANOVA Computer Output Steve Brainerd
37
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Multiple Comparisons With Best, Hsu’s MCB tests whether means are less than the unknown maximum (or greater than the unknown minimum). This is the Hsu MCB test (Hsu 1981). With Control, Dunnett’s tests whether means are different from the mean of a control group. This is Dunnett’s test (Dunnett 1955). The three multiple comparisons tests are the ones recommended by Hsu (1989) as level-5 tests for the three situations: MCA (Multiple Comparisons for All pairs), MCB (Multiple Comparisons with the Best), and MCC (Multiple Comparisons with Control). If you have specified a Block column, then the multiple comparison methods are performed on data that has been adjusted for the Block means.
ANOVA Computer Output Steve Brainerd
38
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Red Circles are the ones we are interested in and and the grays indicate they are different.
ANOVA Computer Output Steve Brainerd
39
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Diamonds: vertical 95% confidence interval Horizontal = mean
ANOVA Computer Output Steve Brainerd
40
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
41
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
42
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Example of large variation for 35% cotton and small for 15% cotton
ANOVA Computer Output Steve Brainerd
43
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Example of large mean for 35% cotton and small for 15% cotton
ANOVA Computer Output Steve Brainerd
44
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Example of large variation for 35% cotton and small for 15% cotton
ANOVA Computer Output Steve Brainerd
45
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
46
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means HSD =Honestly Significant Difference
ANOVA Computer Output Steve Brainerd
47
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Tukey-Kramer
ANOVA Computer Output Steve Brainerd
48
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
49
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Hsu’s MCB
ANOVA Computer Output Steve Brainerd
50
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Hsu’s MCB
ANOVA Computer Output Steve Brainerd
51
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor
ANOVA Computer Output Steve Brainerd
52
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means Dunnett’s
ANOVA Computer Output Steve Brainerd
53
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Means
ANOVA Computer Output Steve Brainerd
54
Review of ANOVA Computer Output Interpretation JUMP: ANOVA Single Factor Jump POST ANOVA Comparison of Variances Tests
ANOVA Computer Output Steve Brainerd
55
