Minitab Notes for STAT 3503 Dept. of Statistics — CSU Hayward

Unit 10: A Nested Design 10.1. The Data A drug company wants to investigate the uniformity of one of its products. It has 2 Sites at which the drug is manufactured. From each site three Batches are obtained at random. From each batch 5 samples (pills) are taken at random. The samples are analyzed for Content. One issue is whether the two sites differ; another is whether batch-to-batch variability within the two sites is a significant source of product variability. The data are shown in the table below. Content of Drug Samples Manufactured At Two Sites (3 randomly chosen batches at each site, 5 randomly chosen pills from each of the 6 batches)

Site I II

Batch 1 2 3 1 2 3

5.03 4.64 5.10 5.05 5.46 4.90

5.10 4.73 5.15 4.96 5.15 4.95

Samples 5.25 4.82 5.20 5.12 5.18 4.86

4.98 4.95 5.08 5.12 5.18 4.86

5.05 5.06 5.14 5.05 5.11 5.07

Data from Ott/Longnecker: An Introduction to Statistical Methods and Data Analysis, 5th ed.,, page 1012, Duxbury, 2001.

Notice that there are six batches, three selected within each site. In the table above we have used different colors to show that batches 1-3 at Site 1 are different from batches 1-3 at Site II. To emphasize this distinction in our notation we might have numbered batches from 1 through 6, with batches 1-3 at Site I and batches 4-6 at Site II. Or, we could adopt some notation that links each batch number 1-3 with its corresponding site. For example, we might represent the third batch at Site II as II-3, 32 or 3(2). We shall see that the last of these choices, using parentheses, is the usual notation in statistics. Problems 10.1.1. For your convenience, the data in the table above have been listed below, reading across rows. 5.03 5.10 5.46

5.10 5.15 5.15

5.25 5.20 5.18

4.98 5.08 5.18

5.05 5.14 5.11

4.64 5.05 4.90

4.73 4.96 4.95

4.82 5.12 4.86

4.95 5.12 4.86

5.06 5.05 5.07

Cut an paste these data into c1 of a Minitab worksheet, labeled Content. Also make three subscript columns: c2 for Site, using numbers 1 and 2; c3 for Batch, using numbers 1-3 as shown in the table; and c4 with batch numbers 1-6, as discussed above, labeled Bat. When you are finished the contents of your worksheet should be as in the printout below.

Minitab Notes for STAT 3503

ROW

c1 c2 Content Site

c3 Batch

c4 Bat

ROW

c1 c2 Content Site

c3 Batch

c4 Bat

1 2 3 4 5

5.03 5.10 5.25 4.98 5.05

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

16 17 18 19 20

5.05 4.96 5.12 5.12 5.05

2 2 2 2 2

1 1 1 1 1

4 4 4 4 4

6 7 8 9 10

4.64 4.73 4.82 4.95 5.06

1 1 1 1 1

2 2 2 2 2

2 2 2 2 2

21 22 23 24 25

5.46 5.15 5.18 5.18 5.11

2 2 2 2 2

2 2 2 2 2

5 5 5 5 5

11 12 13 14 15

5.10 5.15 5.20 5.08 5.14

1 1 1 1 1

3 3 3 3 3

3 3 3 3 3

26 27 28 29 30

4.90 4.95 4.86 4.86 5.07

2 2 2 2 2

3 3 3 3 3

6 6 6 6 6

Unit 10-2

10.1.2. Make a table of the data based on Site and Batch. Also, make a summary table showing the means for each batch at each site. In what way might these tables be misleading without explanation or editing of headers?

10.2. The Nested Model This is a nested or hierarchical design. There are a = 2 sites, b = 3 batches nested within each site, and n = 5 samples (replications) for each batch. The formal model is: Yijk = µ + αi + B(α)j(i) + eijk, where i = 1, 2; j(i) = 1, 2, 3, for each i; and k = 1, ..., 5. The function notation j(i) for the nested subscript j is usually read "j within i" rather than "j of i." (It may take some getting used to: the symbol inside parentheses is not the nested subscript.) Although we have not mentioned it so far, you have seen nesting before. All replications are nested within cells of a design. Thus, to be really fussy, one should write k(j(i)), read "k within j within i." This is seldom done because replications are always nested at the lowest level of the design, and there is no need to complicate the notation to show something that is always true. The main effect, Site, is as a fixed effect. The constants αi that denote the levels of this main effect are restricted so that Σi αi = 0 and, as usual, we use the symbol θα = [Σi αi2| / (a-1) to express the overall main effect. The distributional assumption for the random Batch effect is that, for all i and all j, Bj(i) are independent, identically distributed as N(0, σB(α)2). The assumption that any variability among batches would be the same at both Sites is required for the ANOVA model, but bears checking in practice. (See Problem 10.2.2.) There is no general requirement that nested effects be random, but this is often the case in practice. As usual, we assume that the errors for replications eijk are independent, identically distributed as N(0, σ2). [See Problems 10.2.2 and 10.2.3.]

Minitab Notes for STAT 3503

Unit 10-3

Problems: 10.2.1. Why can there be no interaction term between Site and Batch? Explain this on two levels: (a) From a logical point of view, why does the concept of interaction not make sense here? (b) From a notational point of view, what subscripts would be involved in an interaction term, and why would the model not support such a term? 10.2.2. Make box plots of the data as follows: (a) Broken out first by Site. Although this graphic obscures the finer batch structure of the experiment, it gives a crude overall view that might reveal any differences between the two sites (in either location or spread). Interpret what you see. (b) Broken out by Bat (which uses values 1 through 6). Although this plot does not explicitly include Site in its display, it is easy to remember that the first three batches are from Site I and the last three from Site II. Do you see evidence of important batch-to-batch variability in Content (do the batch medians differ)? Do you see evidence of heteroscedasticity among the batches (are some batches more disperse than others)? Any skewness or outliers? (Note: you will get slightly different results here depending on whether you use character or pixel graphics box plots. The outlier rule for box plots is based on the interquartile range IQR = Q3 - Q1. Pixel box plots use the values of the lower and upper quartiles shown in the describe procedure. Character box plots use a slightly different rule for evaluating quartiles. The difference between these rules is trivial for large sample sizes, but here there are only n = 5 samples per batch.) 10.2.3. Does Bartlett's test of homogeniety show significant differences in the variability within the six batches? Levene's test? (Note: Levene's test measures variability by taking absolute deviations from group medians. Bartlett's test uses squared deviations from group means. Thus Levene's test is less sensitive to outliers and near-outliers.) In case you have forgotten, the menu path is STAT ➯ ANOVA ➯ Equal Variances. Use Bat to separate all six batches for this test. 10.2.4. Whatever you may have found in the previous two problems, suppose that you were convinced that the variability among batches for Site I is much larger than for Site II. This would violate an equal variance assumption of the model. How might you analyze the data to investigate the importance of variability among batches?

10.3 The ANOVA Table Here we use Minitab to make an ANOVA table for this nested design. The key points that must be included in performing the procedure correctly are given below: •

In specifying the model, you must indicate that Batch is nested within Site. This is done with the notation Batch(Site) either on the command line or, if using menus, in the Model box.

•

You must declare that Batch is a random effect (in the same way as in previous units).

•

For agreement with the EMS table in Ott/Longnecker, you need to specify the restricted model. (However, in this case the F-ratios are the same with either the restricted or the unrestricted model.)

•

As always, we will want to look at a normal probability plot of the residuals. to get a formal test of normality you need to store residuals. For a normal probability plot only request it under Graphs in the balanced ANOVA menu procedure.

Minitab Notes for STAT 3503

Unit 10-4

STAT ➯ ANOVA ➯ Balanced, with appropriate declarations and choices MTB > anova Content = Site Batch(Site); SUBC> random Batch; SUBC> restrict; SUBC> ems; SUBC> resid c5. Factor Type Site fixed Batch(Site) random

Levels 2 3

Values 1, 2 1, 2, 3

Analysis of Variance for Content Source Site Batch(Site) Error Total

DF 1 4 24 29

SS 0.01825 0.45401 0.29020 0.76247

S = 0.109962

R-Sq = 61.94%

Source

Variance Error component term 1 Site 2 2 Batch(Site) 0.02028 3 3 Error 0.01209

MS 0.01825 0.11350 0.01209

F 0.16 9.39

P 0.709 0.000

R-Sq(adj) = 54.01% Expected Mean Squares (using restricted model) (3) + 5(2) + 15Q[1] (3) + 5(2) (3)

The ANOVA table shows no significant Site effect. However, there is a very highly significant Batch effect, and some investigation as to how to produce more uniform batches may be in order. Notice that Site is "tested against" Batch and that Batch is tested against Error. Problems 10.3.1. Look at the normal probability plot of residuals, give the P-value of the Anderson-Darling test of normality, and interpret your findings. 10.3.2. To which of the three EMS tables in Table 17.29, page 1011, of Ott/Longnecker does the Minitab EMS table in the printout of this section correspond? Translate the Minitab EMS table into the notation of Section 2 above. 10.3.3. Suppose that the last two observations (4.86 and 5.07) for Batch 3 at Site 2 were not taken. Analyze the resulting unbalanced experiment. Are the conclusions different than for the full balanced experiment?

10.4. Three Common Errors We conclude this unit by looking at what happens upon making each of three errors commonly made by inexperienced "statisticians." The last of these incorrect analyses may be computationally useful if you are using a statistical package that cannot handle nesting. All three provide insight as to the structure of the correct nested model. Incorrectly Ignoring Batch Structure. First, let us see what happens if we ignore the finer structure of the model. We just look at 15 observations from each site, failing to recognize that both of these collections of 15 observations are made up of 5 samples from each of three different batches. (Unfortunately, this is a very common kind of error among those not trained in experimental design models.)

Minitab Notes for STAT 3503

Unit 10-5

INCORRECT ANALYSIS STAT ➯ ANOVA ➯ Balanced, include only Site in model MTB > anova Content = Site Factor Site

Type Levels fixed 2

Values 1, 2

Analysis of Variance for Content Source Site Error Total

DF 1 28 29

SS 0.01825 0.74421 0.76247

MS 0.01825 0.02658

F 0.69

P 0.414

The result is that the SSs and the DFs for the Batch(Site) and Error rows of the correct table are now combined on an incorrect "Error" row. If Batches are really different, the incorrect SS(Error) will be too big. On the other hand, the incorrect DF(Error) will also be too big. Depending on the interplay between these two errors, the incorrect P-value may be smaller or larger that the correct one, and so one might "find" a non-existent Site effect or overlook a real Site one. Incorrectly Ignoring Sites. Second, we investigate the consequences of another common mistake. Here we ignore the Sites altogether, pretending that the highest level structure of the experiment is 6 Batches. (Now we use the bogus Bat variable.) INCORRECT ANALYSIS STAT ➯ ANOVA ➯ Balanced, include only Bat in model MTB > anova Content = Bat Factor Bat

Type Levels fixed 6

Values 1, 2,

3,

4,

5,

6

Analysis of Variance for Content Source Bat Error Total

DF 5 24 29

SS 0.47227 0.29020 0.76247

MS 0.09445 0.01209

F 7.81

P 0.000

Here the Site and Batch(Site) rows of the correct ANOVA table are combined into the row for the bogus "Bat" effect (which turns out to be "significant.") The F-ratio is the same whether or not Bat is declared as random. If Bat is fixed, then a contrast can be formed that correctly compares the two sites. If Bat is random (the most common case), then it is not possible to unscramble the variation due to a possible difference between sites and that due to variability among batches within sites. Incorrectly Ignoring Nesting. Third, we make the mistake of treating Batches as crossed rather than nested within Sites. That is, we analyze this as if it were a two-way ANOVA with terms Site, Batch, and Interaction in the model. As seen in Problem 10.2.1, there can be no true interaction since no Batch has a connection with more than one site. In terms of subscripts in the true nested model, Batch(Site) has subscripts j(i). "Interaction" would also involve only subscripts i and j, and so could not be distinguished computationally from Batch(Site).

Minitab Notes for STAT 3503

Unit 10-6

INCORRECT ANALYSIS STAT ➯ ANOVA ➯ Twoway MTB > twow 'Content' 'Site' 'Batch' ANALYSIS OF VARIANCE SOURCE Site Batch INTERACTION ERROR TOTAL

DF 1 2 2 24 29

Content SS 0.0183 0.0115 0.4425 0.2902 0.7625

MS 0.0183 0.0058 0.2212 0.0121

One feature of this incorrect ANOVA table is that the correct one can be easily derived from it. Combine the DFs and the SSs on the Batch and Interaction rows of this table to obtain the correct DF and SS, respectively, for the Batch(Site) row of the proper nested ANOVA table. This is worth remembering if you ever have to analyze a nested design with a computer package that does only crossed designs. Problems: 10.4.1. Suppose that you read the results of this experiment in a report. The average Content is given for each of the three batches at each site (six averages in all), but individual observations are not given. The claim is made that there is no systematic difference in content between the sites. (a) Make a table of the six batch means, put them into a Minitab worksheet, and perform an ANOVA to test for a difference between the two sites. What conclusion do you draw? How does your F-test for the Site effect compare to F-test shown in Section 3 for Site using the fully detailed dataset? (b) Suppose it is important for you to detect a difference in content between sites that is 1/2 unit or larger. In fact you do not want the probability of a Type II error for such a difference to be larger than 10% when testing at the 10% level. How many batches should you use from each site? (Hint: Use standard formulas for sample size in a 2-sample t-test such as those on page 315 of Ott/Longnecker. Compare with the results from the procedure in the menu path STAT ➯ Power and sample size under either two-sample t-test or one-way ANOVA.) 10.4.2. Returning to the full data in Section 1, consider two alternate, fictional scenario for how the data were collected. (a) Suppose that there are exactly three production facilities at each of the two sites. One batch comes from each facility. Why would you now consider Batch (or maybe now better called "Facility") to be a fixed effect? Would this make any difference in how the F-ratios are formed? (b) At both sites suppose that Batch 1 is newly manufactured, Batch 2 has been stored for one year, and Batch 3 has been stored for two years. Is Batch (or maybe now better called "Age") a fixed or a random effect? What important difference does this change in the story make in the model? Minitab Notes for Statistics 3503 by Bruce E. Trumbo, Department of Statistics, CSU Hayward, Hayward CA, 94542, Email: [email protected]. Comments and corrections welcome. Copyright (c) 1991, 1995, 1997, 2000, 2004 by Bruce E. Trumbo. All rights reserved. These notes are intended primarily for use at CSU Hayward in courses requiring Ott/Longnecker as the text. Please contact the author for other uses. Preparation of early versions of these notes was partially supported by NSF grant USE-9150433. Modified 1/04