Examples of applying common DOE methods using software
617
Two-way ANOVA with no replicates When experiments are conducted which involve two factors, and it is not possible to obtain repeat readings for a given set of experimental conditions, a two-way analysis of variance may be used. The following example assumes that experimental treatments are assigned at random. Note that if the factors involved are each tested at only two levels, the full factorial analysis method described below could also be used.
EXAMPLE OF TWO-WAY ANOVA WITH NO REPLICATES An experiment was conducted to evaluate the effect of different detergents and water temperatures on the cleanliness of ceramic substrates. The experimenter selected three different detergents based on their pH levels, and conducted a series of experiments at four different water temperatures. Cleanliness was quantified by measuring the contamination of a distilled water beaker after rinsing the parts cleaned using each treatment combination. The coded data are shown in Table 17.3. Table 17.3. Cleaning experiment raw data. Cold Cool Warm Hot
DETERGENT A
DETERGENT B
DETERGENT C
15 12 10 6
18 14 18 12
10 9 7 5
Part one of the Excel output (Table 17.4) provides descriptive statistics on the different treatment levels. The ANOVA table is shown in part two. Note that in the previously presented raw data table the rows represent the different temperatures and the columns the different detergents. Because there are no replicates, Excel is not able to provide an estimate of the interaction of detergent and water temperature. If you suspect that an interaction may be present, then you should try to replicate the experiment to estimate this effect. For this experiment, any P-value less than 0.05 would indicate a significant effect. The ANOVA table indicates that there are significant differences between the different detergents and the different water temperatures. To identify which differences are significant the experimenter can examine the means of the different detergents and water temperatures using t-tests. (Excel’s data analysis
618
DESIGN OF EXPERIMENTS
Table 17.4. Cleaning experiment two-way ANOVA output from Microsoft Excel. (Two-factor without replication.) SUMMARY OUTPUT
Cold water Cool water Warm water Hot water
Count 3 3 3 3
Detergent A Detergent B Detergent C
4 4 4
Sum 43 35 35 23
Average Variance 4.333333 16.33333 11.666667 6.333333 11.666667 32.33333 7.6666667 14.33333
43 62 31
10.75 15.5 7.75
14.25 9 4.916667
df 3 2 6 11
MS 22.666667 61.083333 2.75
F P-value 8.242424 0.015043179 22.21212 0.001684751
ANOVA Source of variation SS Rows 68 Columns 122.1666667 Error 16.5 Total 206.6666667
F crit 4.757055 5.143249
tools add-in includes these tests.) Be aware that the Type I error is affected by conducting multiple t-tests. If the Type I error on a single t-test is , then the overall Type I error for k such tests is 1 ð1 Þk : For example, if ¼ 0:01 and three pairs of means are examined, then the combined Type I error for all three t-tests is 1 ð1 0:01Þ3 ¼ 1 ð0:99Þ3 ¼ 0:03. Statistical methods exist that guarantee an overall level of Type I error for simultaneous comparisons (Hicks, 1973, pp. 31^38).
Two-way ANOVA with replicates If you are investigating two factors which might interact with one another, and you can obtain more than one result for each combination of experimental treatments, then two-way analysis of variance with replicates may be used for the analysis. Spreadsheets such as Microsoft Excel include functions that perform this analysis.
Examples of applying common DOE methods using software
619
EXAMPLE OF TWO-WAY ANOVA WITH REPLICATES An investigator is interested in improving a process for bonding photoresist to copper clad printed circuit boards. Two factors are to be evaluated: the pressure used to apply the photoresist material and the pre-heat temperature of the photoresist. Three different pressures and three different temperatures are to be evaluated; the number of levels need not be the same for each factor and there is no restriction on the total number of levels. Each experimental combination of variables is repeated five times. Note that while Excel requires equal numbers of replicates for each combination of treatments, most statistical analysis packages allow different sample sizes to be used. The experimenter recorded the number of photoresist defects per batch of printed wiring boards. The coded data are shown in Table 17.5. These data were analyzed using Excel’s two-way ANOVA with replicates function. The results are shown in Table 17.6. As before, part one of the Excel output provides descriptive statistics on the different treatment levels. The ANOVA table is shown in part two. Because there are now replicates, Excel is able to provide an estimate of the interaction of pressure and temperature. For this experiment, the experimenter decided that any P-value less than 0.05 would indicate a significant effect. The Table 17.5. Photoresist experiment raw data. ANOVA results. High temp
Med temp
Low temp
HIGH PRESSURE
MED PRESSURE
LOW PRESSURE
39 30 35 43 25 38 31 31 30 35 30 35 36 37 39
32 31 28 28 29 10 15 25 31 36 21 22 25 24 27
18 20 21 25 26 22 28 29 26 20 25 24 20 21 21
620
DESIGN OF EXPERIMENTS
Table 17.6. Photoresist experiment two-way ANOVA output from Microsoft Excel. (Two-factor with replication.) SUMMARY OUTPUT High pressure
Med pressure
Low pressure
Total
High temp Count Sum Average Variance
5 172 34.4 50.8
5 148 29.6 3.3
5 110 22 11.5
15 430 28.66667 46.66667
Med temp Count Sum Average Variance
5 165 33 11.5
5 117 23.4 117.3
5 125 25 15
15 407 27.13333 59.98095
Low temp Count Sum Average Variance
5 177 35.4 11.3
5 119 23.8 5.7
5 111 22.2 4.7
15 407 27.13333 43.26667
Total Count Sum Average Variance
15 514 34.26666667 22.06666667
15 384 25.6 44.68571429
15 346 23.06666667 10.92380952
ANOVA Source of variation Sample Columns Interaction Within Total
SS 23.5111111 1034.84444 139.555556 924.4 2122.31111
df 2 2 4 36 44
MS 11.7555556 517.422222 34.8888889 25.6777778
F 0.45781 20.1506 1.35872
P-value 0.6363 1.34E-06 0.267501
F crit 3.259444 3.259444 2.633534
Examples of applying common DOE methods using software
621
ANOVA table P-value of less than 0.001 indicates that there are significant differences between the different columns (pressure), but the P-value of 0.6363 indicates that there is not a significant difference between the rows (temperature). The interaction of pressure and temperature is also not significant, as indicated by the P-value of 0.267501. Since the P-value indicates that at least one difference is significant, we know that the largest difference of 34.26666667 23:06666667 ¼ 11:2 is significant. To identify which other differences are significant the experimenter can examine the means of the different pressures using t-tests. (Excel’s data analysis tools add-in includes these tests.) Be aware that the Type I error is affected by conducting multiple t-tests. If the Type I error on a single t-test is , then the overall Type I error for k such tests is 1 ð1 Þk . For example, if ¼ 0:01 and three pairs of means are examined, then the combined Type I error for all three t-tests is 1 ð1 0:01Þ3 ¼ 1 ð0:99Þ3 ¼ 0:03.
Full and fractional factorial Full factorial experiments are those where at least one observation is obtained for every possible combination of experimental variables. For example, if A has 2 levels, B has 3 levels and C has 5 levels, a full factorial experiment would have at least 2 3 5 ¼ 30 observations. Fractional factorial or fractional replicate are experiments where there are some combinations of experimental variables where observations were not obtained. Such experiments may not allow the estimation of every interaction. However, when carefully planned, the experimenter can often obtain all of the information needed at a significant saving.
ANALYZING FACTORIAL EXPERIMENTS
A simple method exists for analyzing the common 2n experiment. The method, known as the Yates method, can be easily performed with a pocket calculator or programmed into a spreadsheet. It can be used with any properly designed 2n experiment, regardless of the number of factors being studied. To use the Yates algorithm, the data are first arranged in standard order (of course, the actual running order is random). The concept of standard order is easier to understand if demonstrated. Assume that we have conducted an experiment with three factors, A, B, and C. Each of the three factors is evaluated at two levels, which we will call low and high. A factor held at a low level will be identified with a ‘‘^’’ sign, one held at a high level will be identified with a ‘‘+’’ sign. The eight possible combinations of the three factors are identified using the scheme shown in the table below.
