Six Sigma Literature: A Review and Agenda For Future Research

Six Sigma Literature: A Review and Agenda For Future Research James E. Brady Theodore T. Allen LaBarge, Inc. 1505 Maiden Lane Joplin, MO 64801 The ...
2 downloads 0 Views 305KB Size
Six Sigma Literature: A Review and Agenda For Future Research James E. Brady

Theodore T. Allen

LaBarge, Inc. 1505 Maiden Lane Joplin, MO 64801

The Ohio State University Industrial & Systems Engineering 1971 Neil Avenue, 210 Baker Systems Columbus, OH 43210-1271

Summary Like quality management in general, Six Sigma has penetrated into most sectors of today’s business world. Although Six Sigma originated in industry, it has inspired a considerable amount of academic literature. This paper reviews this literature describing the trends, sources, and findings. The paper also seeks to synthesize the literature, with an emphasis on establishing its relationship to quality management theory and topics for future research. In doing so, there is an attempt to answer the fundamental questions: (i) What is Six Sigma? (ii) What are its impacts on operational performance? and (iii) What roles can academics usefully play in relation to Six Sigma? Key words: Quality management, statistical process control, project management 1. Introduction Motorola’s Bill Smith initiated Six Sigma almost two and a half decades ago building on the philosophy, principles, and methods of Deming’s Total Quality Management.

Since then, thousands of organizations have become “Six Sigma

companies” by adopting specific training and project management practices. With Six Sigma’s industry based origins, it becomes important to assess the state of the related academic contributions now that the associated field of study is maturing.

Ahire et al.1 reviewed the broader quality management (QM) literature. Sousa and Voss2 provided synthesis and structure of that literature from the academic viewpoint. Our aim is to provide both a description of the Six Sigma literature and to provide a similar degree of synthesis and structure.

This includes establishing the

relationship of Six Sigma to QM and to business practices in general. In Section 2, we define Six Sigma, building on the definition of Linderman et. al.3 Section 3 describes the terms used to classify the 201 articles we reviewed.

The

taxonomy is itself a synthesis of those used by Zain, Dale, and Kehoe4 and Sousa and Voss2 for similar purposes. In Section 4, we use summary statistics to depict literature trends related to research interest and authorship. Also, since many of the articles on Six Sigma concern “success factors,” we present a tabulation of the factors identified by the most authors.

Section 5 relates Six Sigma to the broader literature on quality

management and Section 6 summarizes the literature regarding the impacts of Six Sigma on firm performance. Section 7 closes with a synthesis of the literature and a discussion of areas for future research. 2. Defining Six Sigma Linderman et. al.3 (p. 195) emphasized the need for a common definition of Six Sigma and proposed: “Six Sigma is an organized and systematic method for strategic process improvement and new product and service development that relies on statistical methods and the scientific method to make dramatic reductions in customer defined defect rates.”

2

Those authors further described that “the name Six Sigma suggests a goal” of less than 3.4 defects per million opportunities (DPMO) for every process. However, they did not include this principle in the definition because, “Six Sigma advocates establishing goals based on customer requirements.” One concern with the Linderman et. al.3 definition of Six Sigma as a “method” is that the definition leaves out philosophy and principles. For example, Dean and Bowen5 defined quality management to include techniques and a set of principles and practices. We suggest that emphasis on monetary gains in Harry6, Hahn et. al.7, Bisgaard and Freiesleben8, and other seminal literature warrants the following addition: “The Six Sigma method only fully commences a project after establishing adequate monetary justification.” Montgomery9 argues that it is this focus on the bottom line that keeps management interested. Virtually all popular books and training materials on Six Sigma describe statistical methods much more vocationally than standard statistical texts (Breyfogle10, Harry6, and Pande et al.11). Specifically, they omit much of the associated theory and include, in some cases, simplified versions of standard statistical methods. Further, Hahn, Doganaksoy, and Standard12 wrote that the related education goals are not to train “statistics experts” but only to give the “knowledge essential to…obtaining business results.” We therefore propose to add the following principle to the definition of Six Sigma: “Practitioners applying Six Sigma can and should benefit from applying statistical methods without the aid of statistical experts.” Another concern with the Linderman et. al.3 definition and others, is that it may be unnecessarily vague. There is an appearance in the literature that this vagueness may

3

be intentional in an attempt by advocates to avoid controversy. We submit that there is sufficient consensus within the Six Sigma literature to offer the following additional details about the Six Sigma method in its definition: The Six Sigma method for completed projects includes as its phases either Define, Measure, Analyze, Improve, and Control (DMAIC) for process improvement or Define, Measure, Analyze, Design, and Verify (DMADV) for new product and service development. Widely read books such as Harry6 and Pande et al.11 clearly imply that this refinement is part of the definition of Six Sigma. Harry6, Pande et al.11 and others also imply that multiple techniques are often used in applying Six Sigma. Therefore, the definition of Six Sigma as “a method” complicates reference to the techniques used in its application. We propose to refer to these techniques as “sub-methods” to clarify their scope relative to that of Six Sigma. The existence of “sub-methods” helps to connote the idea that Six Sigma is broader than a definition as a method might imply. Also, Six Sigma then becomes more like a “practice” than a “core method” as defined by Sousa and Voss2. Sousa and Voss2 also defined “infrastructure practices” as those that create “an environment supportive of the use of core practices”. With these definitions in mind, it becomes apparent that both of the principles above are associated with what might be called specific “Six Sigma infrastructure” (SSI) practices. It is also unmistakable from reading the most popular books on six sigma (Breyfogle11, Harry6, and Pande et al.11, and others) that there is a strong attempt to associate sub-methods with specific phases of the application of Six Sigma.

4

For

example, the application of gauge R&R would generally not be considered appropriate in the Define phase. However, to our knowledge, no specific associations have currently received sufficient consensus to become part of the definition of Six Sigma. Also, any specific set of associations could justifiably be viewed as undesirable restrictions by some portion of Six Sigma users. We define Six Sigma as a method involving either Define, Measure, Analyze, Improve, and Control (DMAIC) or Define, Measure, Analyze, Design, and Verity (DMADV) as phases. We include two principles in our definition. The first emphasizes attention to the bottom line in initiating projects. The second principle emphasizes the training of non-statisticians in the vocational use of statistical tools with minimal theory. 3. Literature Review Methods In this section, we explain the approach used to select the 201 articles covered in our review. Then, the terms used to categorize these articles are defined. Overall, it is not claimed that the list of articles is exhaustive, only that the associated database serves as a reasonably comprehensive list for understanding Six Sigma related research. 3.1 The List of Articles The list of articles was derived from a Science Citation Index (SCI) Expanded search spanning the time period from 1990 through 2003. Five descriptors were used: Six Sigma, quality systems, quality improvement, quality management, and quality metamodel. The text of each article was reviewed in order to eliminate those that were clearly not related to “Six Sigma” improvement strategies. For example, articles were removed that focused on detailed synthesis of chemicals and used the term Six Sigma in an unrelated context. Also, a small number of articles were included from magazines and 5

conference proceedings that were subjectively assessed to be academic in character. The list of journals, proceedings, and magazines that provided at least one relevant article is shown in Table 1. Table 1. List of journals or proceedings with at least one article in the study. Journal of Engineering Design Journal of Evaluation in Clinical Practice Journal of Healthcare Management Journal of Management Engineering Journal of Manufacturing Science and Engineering Transactions of the ASME Journal of Mechanical Design Journal of Operations Management Journal of Quality and Participation Journal of Quality Technology Journal of The IES Lecture Notes In Computer Science Manufacturing Engineering Milbank Quarterly Proceedings of the 2001 Winter Simulation Conference Proceedings of the 2002 Winter Simulation Conference Proceedings of the ASME Design Engineering Technical Conference Professional Engineering Quality and Reliability Engineering International Quality Digest Quality Engineering Quality Management in Health Care Quality Progress R&D Magazine Radiology Research-Technology Management Six Sigma Forum Magazine Technometrics The American Statistician The Physics Teacher Therapeutic Apheresis Total Quality Management Total Quality Management & Business Excellence Training & Development

Accreditation and Quality Assurance AIAA-2002-1471 Annual Quality Congress Transactions Annual Reliability and Maintainability Symposium Proceedings Archives of Pathology & Laboratory Medicine Assembly Automation Aviation Week and Space Technology Building Research and Information Business Management Business Month Cancer Journal Chemical Engineering Communications Chemical Engineering Progress Chemical Week Clinical Chemistry Computers & Industrial Engineering Computers In Industry Control Engineering Electronic Business Fortune Genetic Engineering News Hospitals & Health Networks Hydrocarbon Processing IEEE Engineering Management Review IEEE Software IEEE Transactions on Neural Networks IEEE Transactions on Semiconductor Manufacturing IIE Solutions Industrial Management & Data Systems International Journal of Production Research International Journal of Quality & Reliability Management International Journal of Quality Science Journal of American Geriatrics Society Journal of Applied Statistics

6

3.2 The Classification Scheme Articles were classified using the eleven descriptors in Table 2.

Authors

represented either academic institutions or industrial companies or constituted a team with representatives from both. Many articles contain definitions of the phases Define, Measure, Analyze, Improve, and Control (DMAIC), but most did not. Table 2. Descriptors used to classify articles. Descriptor Authorship Define DMAIC Topics Version 1 Topics Version 2

Source Brady and Allen Brady and Allen Oakland (1989) Sousa et al. (2002)

Industrial Sector Journal Impact Factor Mention of 3.4 ppm Research Approach

Zain et al. (2001) Science Citation Index Brady and Allen Zain et al. (2001)

Society or Area Success factors Speculative in Nature

Brady and Allen Brady and Allen Brady and Allen

Levels Industrial (I), Academic (A), or Both (I A) Yes (Y) or No (N) Systems (Sy), Tools and Techniques (To), and People (Pe) Philosophy (Ph), Practices (Pr), Tools and Techniques (To), and Other Manufacturing (M), Service (Se), or General (G) 0.13 to 4.76 Yes (Y) or No (N) Case Study (Ca), Comparative (Co), Survey (Su), Literature Review (R), or Theoretical with Application (TA) AIChe, ASME, ASQ, IIE, INFORMS, or Medical All combinations of 13 possible factors Yes (Y) or No (N)

Two schemes were used to evaluate the primary topic(s) of each article. Oakland13 divided quality issues roughly into systems, tools and technologies, and people, without providing precise definitions of these terms. We follow Zain, Dale and Kehoe4 in using this division to classify articles (version 1). Sousa and Voss2 developed a modified scheme based on philosophies, practices, and tools and techniques (version 2). Sousa and Voss2 defined “philosophy” as “an approach to management,” and practices as “an observable facet of a philosophy and it is through them that managers work to realize organizational improvements.” Those authors also described “tools and techniques” as “core elements” with examples being process control and Pareto analysis. To clarify the

7

relationship between these terms, Sousa and Voss2 wrote that the practice “process management” can be conducted using many optional core methods such as statistical process control (SPC). The next descriptor was the so-called “journal impact factor” from the Science Citation Index (SCI). This number constitutes a ratio of the citations to articles in a journal to the average number of citations to journals in that field. The impact factor can be viewed as a rough evaluation of the academic quality of the journal. Many articles made explicit reference to either the manufacturing or service sector issues, while others offered general contributions. A common feature of articles was mentioning 3.4 defects per million opportunities in relation to the definition of Six Sigma. The articles were each affiliated with one of the following sponsoring societies or areas of study: the American Institute of Chemical Engineers (AICHE), the American Society of Mechanical Engineers (ASME), the applied statistics area including publications sponsored by the American Society of Quality (ASQ) and the American Statistical Association (ASA), the Institute of Industrial Engineers (IIE), the operations research or management science (OR/MS) area including publications sponsored by the Institute for Operations Research and Management Science (INFORMS) and other related journals, or the medical area in general including the American Medical Association (AMA). Note that not all medicine related journals are included in the science citation index and, therefore, some related articles are omitted. Following Zain, Dale, and Kehoe4, articles were classified as focused on case studies, survey results, literature review, comparative analysis, or theoretical with application. Approximately 27% of the articles investigated the factors contributing to

8

the success of Six Sigma implementations. For those articles, the specific success factors mentioned were tabulated. The terminology used to describe the success factors was standardized to correspond to the dimensions of quality management practice in Sousa and Voss2. Finally, articles that recommended the usage of one or more practices without clarifying conditions in which this practice has provable properties were classified as “speculative” in nature.

This classification differentiated these articles from others

without rigorous statistics or optimization related justifications. 4. Literature Review In this section, we present a characterization of the database of articles using statistics derived from the classifiers described in Table 2.

Goals include the

identification of trends including those that relate to the authorship of articles and the subjects addressed. There is also an investigation of the interaction of authorship with research focus and a tabulation of the associated sponsoring societies or areas of study. Finally, we discuss results relating to success factors, including a tabulation of the success factors cited most often in the literature. 4.1 Literature Trends Figure 1 plots the number of articles verses the year. It suggests two findings. First, the number of articles by industrial authors peaked in 2000. We hypothesize that a subsequent declining trend was influenced by condemnations of Six Sigma in the popular press such as Clifford14 in Fortune Magazine. Second, at the same time, interest among academics grew through 2003. Over the entire search period, 69.2% of the authors had

9

industry affiliations and 30.8% had academic affiliations. These proportions have been changing to the point where 53% of the authors reviewed in 2003 were associated with a university or college. This trend in authorship from industry dominated to academic dominated is not surprising considering the industrial origins of Six Sigma.

45 40 Academia

35

Industry

Number of Articles

30 25 20 15 10 5

2003

2002

2001

2000

1999

1998

1997

1996

1995

1994

1993

1992

1991

1990

0

Year

Figure 1. The yearly number of Six Sigma related articles and their authorship.

Another trend is the diversification of research topics from primarily manufacturing focused to more general in nature as indicated by Figure 2. This trend is characterized with increased emphases on generic issues and on service related business sectors. Particularly, there is an increased emphasis on generic statistical tools such as design of experiments (DOE), probabilistic design, and statistical process control (SPC)

10

in the context of Six Sigma, e.g., Mason and Yong15, Coleman et. al.16, McCarthy and Stauffer17, Koch18, and Goh19. Figure 3 charts the percentages of articles associated with different areas. The fact that the earliest medical related publication in the database is Buck20 supports the finding that the medical area is playing an important role in increasing the topic diversity.

% of Manufacturing Articles

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1990

1992

1994

1996

1998

2000

2002

Figure 2. The percentage of articles focused on manufacturing topics.

Figure 3 also indicates that the applied statistics journals such as the Journal of Quality Technology, Quality Engineering, Quality Management Journal, Quality Progress, and Technometrics dominate scholarly publications on Six Sigma.

This

dominance is perhaps surprising considering that many people who apply Six Sigma have little or no formal training in statistics as noted by Hahn, Doganaksoy, and Hoerl12 and others.

11

IEEE OR+MS Medical ASME AICHE Applied Statistics 0%

10%

20%

30%

40%

50%

Figure 3. Percentages of articles sponsored by different societies or areas.

4.2 Research Topics and Methods Next, we examine the topics and research approaches of the reviewed articles. We begin by focusing on the topics covered and the dependence of the number of articles and scholarly impact on the authorship.

Then, we investigate the methods used in

relation to scholarly impact. Table 3 contains a cross tabulation of the topic and authorship variables. It supports three findings.

First, the majority of articles focused on either philosophy or

systems topics. The percentages on these topics were 54.7% and 66.2% respectively. In general, papers in these categories provided a general description of Six Sigma and advocated its use, e.g., Rayner21, Harry6, Snee22, and Does et. al23. We estimate that 32% of the total articles are in this category and introductory in nature. Often, these articles included definitions of six sigma in terms of 3.4 defects per million opportunities (100%) 12

and/or reviewed the Define, Measure, Analyze, Improve, and Control structure (75%). Overall, only 6% of the articles were written at the practices level which Sousa and Voss2 argued are most useful for stimulating actual organizational changes or improvements. Academic authors wrote about practices with higher frequency (10%). Second, Table 3 also shows that academics were more likely to choose topics such as tools and techniques and practitioners were more likely to present philosophical or systems level contributions.

For example, only 17% of articles by exclusively

industrial authors concerned tools and techniques, compared with 36% of the articles by exclusively academics. Table 3. Tabulations of articles by focus and authorship.

(a)

(b)

Topics Version 1 from Oakland (1989) Systems Tools and Techniques People and Systems People and Tools and Techniques Systems and Tools and Techniques

Academic 23 20 3 4 0

Industrial 83 28 18 5 5

Mixed 4 5 2 0 1

Totals 110 53 23 9 6

Topics Version 2 from Sousa and Voss (2002) Philosophy Tools and Techniques Philosophy and Tools and Techniques Philosophy and Practices Practices Practices and Tools and Techniques Totals

Academic 26 18 1 3 1 1 50

Industrial 101 25 6 3 2 2 139

Mixed 6 5 1 0 0 0 12

Totals 133 48 8 6 3 3 201

Figure 4 is a Pareto chart of the number of articles associated with the different research methods. The papers containing case studies constituted a sizable fraction of papers on all topics. For example, 40% of the papers classified as philosophy focused contained case studies and 50% of the papers exclusively on tools and techniques

13

contained a case study. The majority of all types of articles, even those on tools and techniques, contained no new techniques but rather standard techniques adapted to the context of Six Sigma projects. Of the articles with case studies, the majority contained a single case.

Number of Articles 0

10

20

30

40

50

60

70

80

Cas e Study Theoretical with Application Survey Comparitive Literature Review Other

Figure 4. Pareto chart of articles by research approach.

Journal impact factors were developed by the Science Citation Index (SCI) to provide a rough measure of journal quality or impact. Figure 5 is a box and whisker plot of the journal impact factors associated with the articles in our review. Surprisingly, academic authors exhibited only a slight tendency to publish in journals with higher scholarly impact. Figure 6 is a box and whisker plot of the journal impact factors. These factors are associated with articles pertaining to: (1) manufacturing or (2) service of business or (3) all sectors, i.e., of generic interest. The plot shows that service related publications have the highest scholarly impact.

The impacts of service related

publications can be attributed to the relatively larger audience associated with the specific

14

health care related journals Clinical Chemistry and Laboratory Medicine, the Milbank Quarterly and the Journal of the American Geriatrics Society. The associated articles covered topics classified as systems and tools and techniques.

Impact factor

3

2

1

0 A

I

IA

Figure 5. Journal impact factors of articles by type of author.

Impact factor

3

2

1

0 General

Manufacturing

Service

Figure 6. Journal impact factors of articles by business sector.

15

A sizable fraction of the articles in the most prestigious statistics journals concerned the tools that Six Sigma black belts “should know”, e.g., the discussion of Hoerl24-25 and Montgomery, Lawson, and Molnau26. These articles were classified into people combined with tools and techniques in the Oakland13 scheme and philosophy and tools using the Sousa and Voss2 scheme. These articles caused both categories to be associated with the highest median journal impact factors. Figures 7 and 8 provide box and whisker plots of the impact factors associated with research topics. Surprisingly, the topic associated with the least impact is “practices.” Sousa and Voss2 argued that this topic was the one most likely to stimulate “organizational improvement” or change. Two other important themes related to people topics. First, articles focusing on the cultural implications or management actions included Sanders and Hild27 and Wiklund28. Second, leadership and training are also popular themes (e.g., see Hahn et. al.7 and Hoerl24). Many of these articles examined success factors, as we describe next.

16

Impact factor

3

2

1

0 Pe Sy

Pe To

Sy

Sy To

To

Figure 7. Journal impact factors of articles on people (Pe), Tools and Techniques (To), Systems (Sy), or a combination of these.

Impact factor

3

2

1

0 Ph

Ph Pr

Ph To

Pr

Pr To

To

Figure 8. Journal impact factors of articles on philosophy (Ph), practices (Pr), Tools and Techniques (To), or a combination of these.

17

4.3 Success Factors In our database, 27% of the articles made reference to at least one “success factor” helpful or necessary for Six Sigma to succeed. We identified 13 distinct success factors mentioned and standardized the language around that used by Sousa and Voss2 to describe dimensions of quality management practices. Figure 9 shows the fraction of articles mentioning success factors that cited each factor. Inspection of Figure 9, suggests three findings. First, close to 50% of the articles that mentioned at least one success factor included “top management commitment”, which might be regarded as a consensus view among authors. Second, a sizable fraction of the articles mentioning success factors emphasized training programs involving adult participants from multiple disciplines, e.g., Hahn et al.12 and Snee29. Third, some of the success factors appear to be in compatible such as bottom line focus and customer focus. This highlights the heuristic nature of articles in general on the subject of success factors. For reference, a dissertation by Lee30 used survey results from practitioners to investigate success factors. Survey findings generated the following ordering of success factors from most to least important (paraphrasing): top management commitment, statistical/analytical tool usage, managerial capabilities of trained participants (i.e., black belts), managerial process, personality of black belts, six sigma training programs, previous quality programs adoption, and others. Combining results from Lee30 and from our own literature review, we find consensus in relation to top management support and the importance of established training programs. Additionally, statistical tools and “data systems” are considered important both in the literature and through the survey in Lee30.

18

0%

20%

40%

60%

80%

100%

Top management commitment Team training Data system Structured approach Forming the right team Bottom line focus Team involvement Project selection Customer focused Right project leadership Goal based approach Change management Adaptable system

Figure 9. Percentages of articles mentioning each of 13 success factors.

5. Six Sigma in the Context of Management Theory Six Sigma was developed by industry practitioners at Motorola who were not primarily interested in academic contributions. It is not surprising, then, that the role of Six Sigma in the context of management theory is obscure and, as noted in Linderman et. al.3, only a small fraction of the Six Sigma literature has been devoted to theory. In this section, we synthesize the description of Six Sigma to bring its contribution into clearer focus. Then, we suggest modifications appropriate for the evaluation of Six Sigma to the quality performance model of Garvin31.

19

5.1 The Academic Contribution of Six Sigma We begin by reviewing four facts established previously about Six Sigma. First, it can be debated whether or not the principle of establishing monetary justification for applying the Six Sigma method belongs in the definition. Yet, monetary justification of projects assuredly is associated with Six Sigma. Second, Six Sigma is relatively specific in nature in relation to the pantheon of quality management practices. This fact is established by the definition in Linderman et. al.3 of Six Sigma as a “method”. Also, 24% of articles defined the DMAIC phases and many of the most popular books on Six Sigma associate specific core statistical methods with phases (e.g., Breyfogle10, Harry6, and Pande et al.11). Third, the books and training materials associated with Six Sigma are relatively vocational in nature. For example, Hahn, Doganaksoy, and Standard12 wrote that the aim is not to train “statistical experts”. Fourth, the most important success factors associated with six sigma were believed to be (1) top management commitment and (2) multidisciplinary team training. We begin by connecting the emphasis on monetary justification with achieving top management commitment. As noted by Hahn and Hoerl32, money is the language spoken by management and key to getting projects funded (paraphrasing). Next, we connect greater specificity and relatively vocational materials with training large numbers of practitioners from multiple disciplines. Intuitively, greater specificity about what “should” be used and when it “should” be used in the context of a project, combined with the omission of complicated theory, would seem appropriate for motivating adult learners to use the methods.

20

Combining these observations, we conclude that widespread multidisciplinary usage of statistical techniques is the implied goal of Six Sigma and its main contribution to the business world. Academically, we see three related contributions. First, the bottom line and multi-phase nature of Six Sigma has likely increased the scope of research to embrace total projects and not just the portions associated with the application of a single statistical method. This explains the interest on modeling quality savings in Bisgaard and Feriesleben8 and why over one third of the articles contained case studies. Second, the relatively greater emphasis on specific core (SC) methods and specific infrastructure (SI) has spawned considerable academic discussion with greater specificity. For example, there is a substantial academic thread focused on what tools “should” be learned and used by Six Sigma trained participants or “black belts” (e.g., see Hoerl24 and the related discussion in the Journal of Quality Technology). While the discussion of training materials is not new to the quality literature, the emphasis on relatively specific references to the phases of projects is somewhat new. Third, Six Sigma has caused many people from multiple disciplines to become aware of and apply statistical methods. It is perhaps remarkable that 69% of authors of academically relevant publications had industry affiliations. While, in general, Six Sigma practitioners have learned only standard methods, they constitute a large potential market for research and, perhaps, new methods. Distinguishing features of this market include that participants: (1) are relatively practical and focused on business results, (2) need techniques for predicting the bottom line impacts of projects before they embark upon them, and (3) apply statistical methods without, in general, being experts in statistics.

21

In Section 6, we discuss the implications of these findings on the roles that academics can most usefully play in relation to Six Sigma. Next, we discuss where the specific infrastructure and core elements of Six Sigma fit into the quality performance model. 5.2 The Quality Performance Model Much research has focused on the relationship of quality management practices with the various aspects of firm performance (Sousa and Voss2). Garvin31 introduced a quality performance model to set up an empirical examination of the separate effects of management practices on internal process quality and product quality performance (QP) and their effects on operational performance (OP) and business performance (BP). In reviewing the literature on Six Sigma, we feel that it is helpful to place Six Sigma into this diagram. In our definition of Six Sigma in Section 2, we identified principles and methods associated with “Six Sigma infrastructure” (SSI) and “Six Sigma core” (SSC) quality management practices. It was argued that the method of Six Sigma is itself a quality practice while sharing some characteristics with a core method. Figure 10 shows the placement of these specific core practices and infrastructure in the Garvin31 model.

22

Internal Process Quality

Operational Performance

Quality Management Practice Six Sigma Practice Six Sigma Infrastructure

Business Performance

Six Sigma Core Product Quality Performance

Figure 10. Extended quality performance model of Garvin (1884).

6. Review of Empirical Evaluations of Six Sigma In this section, the literature relating to the performance evaluation of Six Sigma is briefly reviewed. Only a small fraction of articles in our database pertain to an empirical model or evaluation with scope greater than estimating the savings associated with a single case study. Table 4 below lists five of these articles with reference to Six Sigma core (SSC) and Six Sigma infrastructure (SSI) practices. The other acronyms used are referenced in the extended quality performance model in Figure 10. The fourth article by Gautreau, Yacout, and Hall33 does not make specific reference to Six Sigma but was included because it addresses issues related to the inclusion of specific methods in the context of quality projects. This seems relevant given Six Sigma’s emphasis on specific core methods and business outcomes. Table 4 contains a description of the roles each article plays for empirical validation in relation to Six Sigma. Goh et al.34 examined stock performance associated with announcements of Six Sigma programs and dates of quality awards. They found hints of short-lived abnormal returns but no significant evidence of short or long term 23

returns. Another data driven meta-analysis that we found was Lee30, which was based on survey data. The associated surveys indicated positive self assessments of the value of the company’ own Six Sigma efforts. Also, to our knowledge, the impacts of specific core sub-method selection on bottom line impacts has not been studied empirically, with Gautreau, Yacout, and Hall33 providing one of few relevant theory based modeling approaches.

Table 4. Literature pertinent to evaluation of Six Sigma’s effects on firm performance. Study Bisgaard and Feriesleben (2000)

Quality performance model SSI→BP

Gautreau, Yacout, and Hall (1997)

C→QP→BP

Goh et al. (2003)

SSI/SSC→BP (stocks)

Lee (2002)

SSI/SSC→BP

Linderman, Schroeder, Zaheer and Choo (2003)

SSC→QP/OP→BP

Study method Economic brake-even analysis model Partially Observed Markov Decision Process Hypothesis testing

Main findings

Fraction nonconforming and unnecessary activities can significantly influence cost and reduce profit Decision based model of process improvement activities, e.g., do nothing or inspect and separate can each be optimal depending on assumptions The majority of firms show positive returns after announcing Six Sigma programs but no statistical significance was established Survey of 106 Top management commitment, firms project selection, team leader, training and the specific tools used effect business results Goal-theoretic Types of goals effect quality model and operational performance that effect business results

Considering the emphasis on modeling profits to justify each project, it is not surprising to all attempts to provide meta-modeling tools, e.g, see Bisgaard and

24

Feriesleben8. Yet, Bisgaard and Feriesleben8 explore prediction of product value under simple, generic assumptions. As they themselves suggest, more research on related topics will likely be needed for broad applicability.

7. Synthesis and Future Research In this paper, we proposed a definition of Six Sigma and characterized the associated body of academic literature. We also clarified Six Sigma’s contributions to scholarly research and its relationship to quality management theory. Reflecting on the findings, we next return to the questions: (i) What is Six Sigma? (ii) What are its impacts on operational performance? and (iii) What roles can academics usefully play in relation to Six Sigma? 7.1 Synthesis We define Six Sigma as a method involving either Define, Measure, Analyze, Improve, and Control (DMAIC) or Define, Measure, Analyze, Design, and Verify (DMAIC) as phases.

This definition of Six Sigma as a method builds on the one

proposed by Linderman et. al.3 The inclusion of DMAIC and DMADV in the definition is supported by the fact that 75% of introductory articles on Six Sigma reference these structures. We include two principles in our definition. The first emphasizes attention to the bottom line in initiating projects. This was supported by comments of seminal writers relating to how Six Sigma differs from Total Quality Management (TQM), e.g., Harry35 and Montgomery9. Also, bottom line focus was mentioned by 24% of relevant articles as a critical success factor.

The second principle emphasized the training of non-

statisticians with minimal theory. We support this inclusion based on remarks by Hahn, Doganaksoy, and Standard12 and others about the goals of Six Sigma training. 25

In

addition, we found relatively frequent mention of multidisciplinary training as a critical Six Sigma success factor (roughly 24% of relevant articles). Finally, books on Six Sigma such as Breyfogle10, Harry6, and Pande et al.11 noticeably deemphasized theory. The span of time surveyed was form 1990 to 2003, which provided a representative sample. We feel the trends presented are still relevant as can be seen by inspecting more resent works such as the special issue of Quality & Reliability Engineering International on Six Sigma (Vol. 21, No. 3, 2005). In this issue presents nine papers on Six Sigma. Walter36, Hahn37, and Snee38 discuss definitions for Six Sigma and critical success factors relating to its implementations. Following the major theme of training, Montgomery et al.39 and Anderson-Cook et al.40 present Six Sigma training currently incorporated into two university programs at Arizona State University and Virginia Tech. Edgeman et al.41, Paterson et al.42, Neagu and Hoerl43, and Frings and Grant44 all present detailed case studies involving non-manufacturing applications continuing the trend away form the traditional application of Six Sigma. These authors represent a mix of academic and industrial backgrounds. 7.2 Agenda For Future Research Considering that methods such as Xbar & R charting and regular fractional factorials used in Six Sigma were proposed by researchers, it seems reasonable to expect researchers to continue to make useful contributions to Six Sigma. In general, Cooper and Noonan46, Linderman et al.3, and Snee47 suggest that, in general, too much research has been focused on descriptions of practice rather than on theory development that is of use to managers and scholars. As a preliminary to our proposed agenda, consider the descriptors:

26

Micro – dealing with individual statistical methods. Meso – supervisor level decision-making about method selection and timing such as which methods to apply and project budgets, objectives, and timing. Macro – related to overall quality programs including stock performance. Table 5 overviews the proposed areas for future research. The first two represent continuing on-going threads of research likely to be received gratefully by practitioners in the short run. In general, we are not aware of any research that treats Six Sigma as a product and surveys what its customers, i.e., users of the method, believe are opportunities for its improvement. Also, continued quantitative research on the value of six sigma programs will likely be of interest to stock holders and management partly because past results are somewhat inconclusive. Table 5. Overview of proposed agenda for future research Proposed Area

Level

Opinion surveys

All

Quantitative analyses of management practices

Macro

New statistics methods

Micro (new sub-methods)

Meso-analyses of project databases Optimal design of project strategies Testbed for project decisionmaking evaluation

Meso and Macro Meso Meso

Possible Outcomes Improved understanding of needs for research and improvements Improved adoption and management guidelines User friendly software offering additional method options Improved training materials and strategies, expert system software Improved training materials and strategies, expert system software Criteria for theoretical evaluation of six sigma and other strategies

Goh et al.34 and others analyzed the macro-level effects of the adoption of Six Sigma on corporate stock performance and found hints of short lived benefits while their long term analysis was largely inconclusive. Those authors also included specific caveats about the ability to connect Six Sigma programming effects at divisions with overall 27

parent company performance. Sousa and Voss2 highlighted the continuing need for empirical justifications including macro level assertions in the quality management literature.

Examples include self reported profits, the effects of success factors, and

advocacy for Six Sigma in general. Snee47-48 calls for research to help practitioners identify a robust set of what are essentially “micro-level” improvement tools to be used in conjunction with the DMAIC process. The focus in these recommendations is not so much on new techniques as on refined techniques associated with specific phases. However, we suggest that advances in computational speed and optimization heuristics provide unprecedented opportunities for new method development of all types.

For example, Allen and Bernshteyn49

proposed new fractional factorial arrays relevant for high cost experimentation. It is likely that their micro-level methods could not have been developed before the advent of modern computing.

Also, new methods can potentially dominate many or all

performance criteria relative to time-tested methods, e.g., the EIMSE optimal design of experiments (DOE) arrays from Allen, Yu, and Schmitz50 offer methods with fewer runs and lower expected prediction errors than central composite or Box Behnken designs. The definition of Six Sigma in Section 2 suggests that it is a method with a “meso-level” or project-level scope and focus. For example, DMAIC and DMADV specify the phases in a project. Also, the principle of cost justification applies at the entire projects and not individual component methods or phases. Because of Six Sigma’s focus on the meso, our proposed research agenda also focuses on meso-level analyses. Related questions include: 1. Which methods should be applied to which problems and in which phases?

28

2. Do additional investments in training make financial sense? 3. Does it make sense to terminate a given project early or apply a different method? 4. Is a Six Sigma program in statistical control or should management intervene? 5. Can the methods to forecast savings in a given operation be improved? Related to meso-level research and the above questions, Six Sigma has spawned the creation of many corporate databases containing the methods used in projects and the associated financial results.

While the majority of these databases are confidential,

practitioners at specific companies generally have access to their own organization’s database. Brady51 proposed several approaches for analyzing such databases to answer meso-level questions. The data were also analyzed using regression and Markov Decision Processes. Results included prescriptive recommendations about which core or submethods can be expected (at the related manufacturing unit) to achieve the highest profits. Additional research can consider larger databases and, possibly, achieve stronger inferences with larger scope than a single manufacturing unit. Also, a wider variety of possible analysis methods can be considered including neural nets, logistic regression, and many other techniques associated with data mining. Each of these methods might offer advantages in specific contexts and answer new types of questions. Overall, it seems the topic of analyses of project databases is largely unexplored. The definition of Six Sigma as an improvement or design method suggests that it might be useful to explore evaluation or Six Sigma and specific core method selections as if they were optimization algorithms. Implementation timing, costs, and the quality of solutions derived can be compared with alternatives using test problems. “Testbeds” that include sets of test projects can be developed and published. Possible benefits include

29

scenarios usable in training and immediate, quantitative feedback about specific and general strategies. This benefit is particularly relevant considering the difficulties of macro-level financial measurements.

Using testbeds, it might be possible to detect

whether a method that goes “beyond Six Sigma” in an important sense has been created. 7.3 Conclusions We have proposed that Six Sigma is both a method and two principles. These principles related to building and maintaining management support and to fostering usage of methods among practitioners who are not experts in statistics. Trends in the literature included an increasing academic participation and broader focus than solely on manufacturing. We found only partial consensus about the factors that make Six Sigma effective. We suggested opportunities for new research on Six Sigma including the development of more realistic project payback models, clarifying which techniques are most applicable in which situations, and possibly even for the development of new statistical methods with clear advantages for business.

Acknowledgments We would like to thank Gavin Richards and Chaitanya Joshi for their contributions to the preparation of the text. We thank the Edison Welding Institute for support of related research.

References 1. Ahire, S., Landeros, R., Golhar, D., 1995. “Total quality management: a literature review and an agenda for future research.” Production and Operations Management, pp. 277-307.

30

2. Sousa, R., Voss, C., 2002. “Quality management re-visited: a reflective review and agenda for future research.” Journal of Operations Management, vol. 20, pp. 91-109. 3. Linderman, K., Schroeder, R., Zaheer, S., Choo, A., 2003. “Six Sigma: A goaltheoretic perspective.” Journal of Operations Management, vol. 21, num. 2, Mar., p. 193-203. 4. Zain, Z., Dale, B., Kehoe, D., 2001. “Total quality management: an examination of the writings from a UK perspective.” The TQM Magazine, vol. 13, num. 2, Feb., pp. 129-137. 5. Dean, J., Bowen, D., 1994. “Managing theory and total quality: improving research and practice through theory development.” Academy of Management Review, vol. 19, num. 3, pp. 392–418. 6. Harry, M., 1998a. “Six Sigma: A Breakthrough Strategy for Profitability.” Quality Progress, vol. 31, num. 5, May, pp. 60-62. 7. Hahn, G., Hill, W., Hoerl, R., Zinkgraf, S., 1999. “The Impact of Six Sigma Improvement – A Glimpse Into the Future of Statistics.” The American Statistician, vol. 53, num. 3, Aug., pp. 208-215. 8. Bisgaard, S., Freiesleben, J., 2000. “Quality Quandaries: Economics of Six Sigma Program.” Quality Engineering, vol. 13, no. 2, Dec., pp. 325-331. 9. Montgomery, D., 2001. “Beyond Six Sigma.” Quality and Reliability Engineering International, vol. 17, num. 4, Jul.-Aug., pp. iii-iv. 10. Breyfogle, F. W. (2003), Implementing Six Sigma: Smarter Solutions Using Statistical Methods, Wiley. 11. Pande, P., Neuman, R., Cavanaugh, R., 2000. The Six Sigma Way: How GE, Motorola, and Other Top Companies are Honing Their Performance, McGrawHill, New York. 12. Hahn, G., Doganaksoy, N., Stanard, C., 2001. “Statistical tools for Six Sigma – What to emphasize and de-emphasize in training.” Quality Progress, vol. 34, num. 9, Sept., pp. 78-82. 13. Oakland, J., 1989. Total Quality Management, Butterworth-Heinemann, London. 14. Clifford, L., 2001. “Trend spotting – Why you can safely ignore Six Sigma.” Fortune, vol. 143, num. 2, Jan., pp. 140. 15. Mason, R., Young, J., 2000. “Interpretive features of a T(2) chart in multivariate SPC.” Quality Progress, vol. 33, num. 4, Apr., pp. 84-89. 16. Coleman, S., Arunakumar, G., Foldvary, F., Feltham, R., 2001. “SPC as a tool for creating a successful business measurement framework.” Journal of Applied Statistics, vol. 28, num. 3-4, Mar.-May, pp. 325-334. 17. McCarthy, B., Stauffer, R., 2001. “Enhancing Six Sigma through simulation with IGRAFX process for Six Sigma.” Proceeding of the 2001 Winter Simulation Conference, pp. 1241-1247. 18. Koch, P., 2002. “Probabilistic design: optimizing for Six Sigma quality.” Proceeds from 43rd AIAA/ASME/ASCE/AHS Structures, Structural Dynamics, and Materials Conference, 4th AIAA Non-Deterministic Approaches Forum, Denver, Colorado, AIAA-2002-1471 19. Goh, T., 2002. “A Strategic Assessment of Six Sigma.” Quality and Reliability Engineering International, vol. 18, num. 5, Sept.-Oct., pp. 403-410.

31

20. Buck, C., 1998. “Health care through a Six Sigma lens.” Milbank Quarterly, vol. 76, no. 4, pp. 749-753. 21. Rayner, B., 1990. “Market-driven Quality: IBM’s Six Sigma Crusade.” Electronic Business, vol. 1, num. 10, Oct., pp. 68-74. 22. Snee, R., 2000a. “Impact of Six Sigma on quality engineering.” Quality Engineering, vol. 12, num. 3, Mar., pp. ix-xiv. 23. Does, R., van den Heuvel, E., de Mast, J., Bisgaard, S., 2002. “Quality quandaries: comparing nonmanufacturing with traditional applications of Six Sigma.” Quality Engineering, vol. 15, num. 1, Mar., pp. 177-182. 24. Hoerl, R., 2001a. “Six Sigma Black Belts: What Do They Need to Know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 391-406. 25. Hoerl, R., 2001b. “Response - Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 432-435. 26. Montgomery, D., Lawson, C., Molnau, W., Elias, R., 2001. “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 407-409. 27. Sanders, D., Hild, C., 2000b. “Six Sigma on Business Processes: Common Organizational Issues.” Quality Engineering, vol. 12, num. 4, Jun., pp. 603-610. 28. Wiklund, H., Wiklund, P., 2002. “Widening the Six Sigma concept: An approach to improve organizational learning.” Total Quality Management, vol. 13, num. 2, Mar., pp. 233-239. 29. Snee, R., 2000b. “Six sigma improves both statistical training and processes.” Quality Progress, vol. 33, num. 10, Oct., pp. 68-72. 30. Lee, K., 2002. “Critical Success Factors of Six Sigma Implementation and the impact on Operations Performance.” Ph. D. Dissertation, Cleveland State University. 31. Garvin D. A. (1988) Managing Quality: the strategic and competitive edge, New York: Free Press 32. Hahn, G., Hoerl, R., 1998. “Key challenges for statisticians in business and industry.” Quality Progress, vol. 31, num. 8, Aug., pp. 195-200. 33. Gautreau, N., Yacout, S., Hall, R., 1997. Simulation of Partially Observed Markov Decision Process and Dynamic Quality Improvement. Computers & Industrial Engineering, vol. 32, num. 4, Dec., pp. 691-700. 34. Goh, T., Low, P., Tsui, K., Xie, M., 2003. “Impact of Six Sigma implementation on stock price performance.” Total Quality Management & Business Excellence, vol. 14 no. 7, Sept., pp. 753-763. 35. Harry, M., 2000a. “A new definition aims to connect quality with financial performance.” Quality Progress, vol., 33, num. 1, pp. 64-66. 36. Walters, L., 2005. “Six Sigma: is it Really Different?” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 221-224. 37. Hahn, G.J., 2005. “Six Sigma: 20 Key Lessons Learned” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 225-233. 38. Snee, R.D., 2005. “Leading Business Improvement: a New Role for Statisticians and Quality Professionals” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 235-242.

32

39. Montgomery, D.C., Burdick, R.K., Lawson, C.A., Molnau, W.E., Zenzen, F., Jennings, C.L., Shah, H.K., Sebert, D.M., Bowser, M.D., and Holcomb, D.R., 2005 “A University-based Six Sigma Program” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 243-248. 40. Anderson-Cook, C.M., Patterson, A., and Hoerl, R., 2005. “A Structured Problem-solving Course for Graduate Students: Exposing Students to Six Sigma as Part of their University Training” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 249-256. 41. Edgeman, R.L., Bigio, D., and Ferleman, T., 2005. “Six Sigma and Business Excellence: Strategic and Tactical Examination of IT Service Level Management at the Office of the Chief Technology Officer o Washington, DC” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 257-273. 42. Patterson, A., Bonissone, P., and Pavese, M., 2005. “Six Sigma Applied Throughout the Lifecycle of an Automated Decision System” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 275-292. 43. Neagu, R. and Hoerl, R., 2005. “A Six Sigma Approach to Predictiong Corporate Defaults” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 293309. 44. Frings, G.W., and Grant, L., 2005. “Who Moved My Sigma … Effective Implementation of Six Sigma Methodology to Hospitals” Quality & Reliability Engineering International, vol. 21, no. 3, pp. 311-328. 45. Noble, T., 2001. “Six sigma boosts the bottom line.” Chemical Engineering Progress, vol. 97, num. 4, Apr., pp. 9-11. 46. Cooper, N., Noonan, P., 2003. “Do teams and Six Sigma go together?” Quality Progress, vol. 36, num. 6, June, pp. 25-28. 47. Snee, R., 1999. “Why should statisticians pay attention to Six Sigma?” Quality Progress, vol. 32, num. 9, Sept., pp. 100-103. 48. Snee, R., 2001a. “Dealing with the Achilles’ heel of Six Sigma initiatives – Project selection is key to success.” Quality Progress, vol. 34, num. 3, Mar., p. 66. 49. Allen, T. T. and M. Bernshteyn (2003), “Supersaturated Designs that Maximize the Probability of Finding the Active Factors,” Technometrics, 45 (1), 1-8. 50. Allen, T. T., L. Yu, J. Schmitz (2003), “The Expected Integrated Mean Squared Error Experimental Design Criterion Applied to Die Casting Machine Design,” Journal of the Royal Statistical Society Series C: Applied Statistics, 52, 1, 1-15. 51. Brady, J. E. (2005), Six Sigma and the University: Teaching, Reasearch, and Meso-Analysis, Ph.D. Dissertation, The Ohio State University, Industrial & Systems Engineering.

33

Appendix A – The Article Database The descriptors used in the table below are defined in Section 3. The complete

-

0 0

0 0

G Se

0.2 NA

C C

Y N N N

-

-

-

-

-

Adaptable system

-

Customer focused

-

Structured approach

-

Goals based

-

Project leadership

Y N

Bottom line

TA

Project selection

1.5

Team involvement

G

-

Data system

0 Pe Sy Ph

-

Training

Speculative in Nature?

0

0

Change Management

Research Approach

N N N Y N N Y N N N N N Y Y Y Y N Y Y

0

Right team

Impact factor

C C C C Su C TA Co C C CR Co TA C C TA C TA TA

A

Management committed

Industrial Sector

0.7 0.7 1.5 0.3 0.3 1.5 0.3 0.2 NA NA 2.9 0.8 NA 0.2 NA 0.2 0.2 0.7 NA

2001

Success Factors

Topic(s) Version 2 Sousa and Voss (2002)

G M M M M M M M M Se Se M M Se Se G Se M Se

Define 3.4

N N

Define DMAIC

Co

Authorship

1.5

0 1 0 1 0 0 0 1 1 0 1 1 0 0 0 0 0 0 1

Ph To Ph To To To Ph Ph Ph Ph Ph Ph Ph To Ph Ph Ph Ph Ph Ph Ph Ph

G

0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1

Pe To Pe To To To Sy Sy Pe Sy Sy Sy Sy Sy To Sy Sy Sy Sy Sy Sy Sy Sy

Year

Topic(s) Version 1 Oakland (1989)

references to the articles are described below in the reference section.

-

-

-

-

-

-

-

-

-

-

-

Author(s) Abraham et.al.

Ackermann 1993 I Ackermann et.al. 1993 I Ali et.al. 1999 I Antony et.al. 2002 I Arvidsson 2003 A Bailey 2001 I Bartos 1999 I Basu 2001 I Behara et.al. 1995 I A Benedetto 2003 I Berlowitz 2003 A Binder 1997 I Bisgaard et.al. 2000 A Blakeslee 1999 I Blanton 2002 A Bossert 2003 I Breyfogle 2002 I Breyfogle et.al. 2003 I Breyfogle et.al. 2002 I Breyfogle et.al. (a) 2001 I Breyfogle et.al. 2001 I (b) Broderick et.al. 2002 A

To Sy

To Ph

34

N - - - - - N - - - - - N - - - - - N - - - - - N - - - - - N - - - - - Y - Y - - - N - - - - - N - - - - - Y Y - Y Y - Y - - - - Y N - - - - - N - - - - - Y Y - Y - Y Y N - - - - - N - - - - - N - - - - - N - - - - - N - - - - - -

-

Y -

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Buck et.al.

2001

I

1

0

Buck Buck Buggie Card

1998 2001 2000 2000

I I I I

1 1 0 0

1 1 0 0

Caulcutt Chan et.al. Chassin Chowdhury Clifford Coleman et.al. Connolly Conner Cooper Cooper Crom Dasgupta Davies Davig et.al. De Mast De Mast et.al. Dedhia DeFeo Deshpande Deshpande et.al. Does et.al. Doganaksoy et.al. Dornheim Douglas Du et.al. Duguesaoy et.al. Eid et.al. Farntz Feng et.al. Ferrin et.al. Finn Fontenot et.al. Fuller Gano Gautreau et.al. Gill Gnibus Goh (a) Goh (b) Goh Goh et.al. (a) Goh et.al. (b)

2001 I 2001 A 1998 A 2000 I 2001 I 2001 I A 2003 I 2003 I 1992 I 2003 I 2000 I 2003 A 2001 A 2003 A 2003 A 2000 A 1995 I 2000 I 1998 A 1999 I A 2002 A 2000 I A 2001 I 2000 I 2000 A 2002 I 1997 A 2001 I 1997 A 2002 I 1999 I 1994 I A 2000 I 2001 A 1997 A 1990 I 2000 I 2002 A 2002 A 2001 A 2003 A 2003 A

Sy

Ph

Se

NA

Se Se M M

1.9 NA 0.1 0.8

1 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1

1 1 1 0 1 0 0 0 1 0 0 1 0 0 0 0 0 1 0 1 0

Sy Ph Sy Ph Sy Ph To To Sy To Ph Pr To To Sy Ph Sy Ph Sy Ph To To Sy Ph Sy Ph Sy Ph Pe Sy Ph Pe Sy Ph Pe Sy Ph Sy Ph Sy Ph Sy Ph Sy Ph Sy Ph Pe Sy Ph To To Sy Ph Sy Ph

C C Su R C TA Co

M M Se G M M M M M Se G Se Se M G G Se G M G Se

0.3 0.4 1.9 0.3 NA 0.3 0.7 0.3 0.1 0.2 0.2 NA 0.8 0.2 0.2 0.2 NA NA 0.2 0.4 NA

C C R TA C Co TA C C C Su Co C R Su Co Co TA Su C C Co

N Y N Y N Y N N N N Y Y N N Y N Y N Y N N

Y N N Y Y N N N N Y Y Y N Y N N Y N N N N

Y Y Y Y Y -

-

Y Y -

-

Y - - - - - - - - - Y - - - - - - - - - - - -

-

-

-

-

-

-

-

0 0 0 0 0 0 1 0 1 1 0 0 0 0 0 0 1 1 0 1 1

0 To To 0 To To 0 Sy Ph 1 To Pr To 0 To To 0 To To 0 Sy Ph 0 To To 1 To To 0 Sy Ph 1 To To 0 Sy Ph 0 To To 0 To To 1 Sy Ph 0 To To 1 Sy Ph 0 Pe Sy Ph 0 To To 0 Sy Ph 1 To To

M M G M G M M M M M M M G M M G G G M M G

0.2 0.3 NA 0.5 0.4 0.4 0.3 0.4 NA 0.3 0.2 NA NA 0.4 NA 0.2 NA 0.2 0.3 NA NA

C TA C Su Co C TA C C Su TA Su TA C TA Su C Co R TA R TA

Y Y N Y N Y N N N Y N Y N Y N Y N N Y N Y

N N N N N N N N N N N Y N N N N N Y Y Y N N -

-

Y -

-

-

Y -

-

-

-

Y -

-

-

35

N Y Y

-

-

-

-

-

-

-

-

-

-

-

-

N N Y N

N N N N

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Gordon Grandzol et.al. Greek Gross Hahn et.al. Hahn et.al. Hahn Hahn et.al. Hahn et.al. Hammer Harrold Harrold et.al. Harry

2002 1998 2000 2001 1998 1999 2002 2000 2001 2002 1999 1999 1998

I A I I I I I I I I I I I

0 0 0 0 0 1 0 1 0 1 1 1 1

0 0 1 0 0 1 0 1 0 0 1 0 1

Harry (a) Harry (b) Harry (c) Harry (d) Harry (e) Harry (f) Henretta et.al. Hild et.al.

2000 2000 2000 2000 2000 2000 2003 2000

I I I I I I I I

0 0 0 0 0 0 1 0

0 0 0 1 0 0 1 0

Hill Hoerl Hoerl (a) Hoerl (b) Horst Howell Howell Hunter Hunter Hunter et.al. Hutchins Ingle et.al. Johnson Johnson et.al. Johnstone et.al.

2001 I 1998 I 2001 I 2001 I 1999 I 2000 I 2001 I 1999 I 2000 I 1999 I 2000 I 2001 I A 2002 I 2003 I 2002 I

0 0 0 0 0 0 0 0 0 0 0 1 0 1 0

0 0 0 0 0 0 0 0 0 0 0 1 1 1 1

Johnstone et.al. Kandebo Kane Kazmer et.al. Kazmierczak Kendall et.al. Kenett et.al. Knowles et.al. Koch Koonce et al. Krouwer

2003 I 1999 I 1998 I 2002 A 2003 A 2000 I 2003 I A 2003 A 2002 I 2003 A 2002 I

0 0 0 0 0 0 0 0 0 0 0

1 0 1 1 1 0 0 0 1 0 1

Sy Ph To Pr Sy Ph Sy Ph Pe Sy Ph Pe Sy Ph Pe Sy Ph Sy Ph To To Sy Ph Pe Sy Ph Sy Ph Pe Sy Ph Sy Ph To To To To To To Sy Ph To To To To Sy Ph Sy Ph Pe Ph To To Sy Ph Pe Sy Ph Pe Sy Ph Sy Ph Sy Ph Sy Ph Sy Ph Sy Ph Sy Ph Pe Sy Ph Pe Sy Ph Pe Sy Ph Sy Ph Sy Ph Sy To Ph Pr Sy Ph Sy Ph To To To To To To To To To To To To To To To To

G M M G M M G G G M M M M

0.2 NA 0.3 0.2 0.2 1.2 1.2 NA 0.2 NA 0.3 0.3 0.2

Co Y N Su N N Su N Y Y TA Y Y Y TA N N TA N Y TA Y N TA Y Y Y TA Y Y TA N Y C N Y TA Y N C N Y -

-

Y Y

-

- - - - - - - - - - - - Y - - Y - Y - - - - Y - Y - - - - -

-

-

-

Y -

-

-

G G G G G G Se M

0.2 0.2 0.2 0.2 0.2 0.2 0.2 NA

Su TA TA TA TA TA C Co

N Y Y Y Y Y N N

Y N N N N N N N

-

-

-

Y -

-

-

Y -

-

-

-

-

-

M G G G Se M M Se G M G M G M Se

1.5 0.2 1.5 1.5 0.3 0.3 0.3 0.3 0.3 0.3 0.2 NA 0.3 0.3 2.9

C C TA TA C Co Su Su C TA C TA Co TA Su C

N N N Y N N N N Y N Y N Y N N

N - Y - Y Y - N - N - N - Y Y N - N - Y Y N - N - N - N - N - -

-

-

-

-

-

-

Y -

-

-

-

-

Se M M M Se G M Se M M Se

1.2 0.3 0.2 NA 1.6 0.2 0.2 0.2 0.8 0.4 0.8

C TA C C R TA C C C C C

N N N Y N Y N N N Y N

Y Y N N N N N N N N N N -

-

-

-

-

-

-

-

-

-

-

-

36

-

-

Kunes Landin et.al.

I A

1 0

0 0

Leffew et.al. 2001 I A Linderman et.al. 2003 A

1 1

1 1

Lucas (a) Lucas (b) Mader Maguire (a) Magure (b)

2002 2002 2002 1999 1999

I I I I I

1 0 0 0 0

1 0 0 1 1

Mandal et.al. Mason et.al. McCarthy et.al. McFadde

1998 2000 2001 1993

A I I A

0 1 1 1

0 0 1 1

Montgomery Montgomery

2000 2001

A A

0 0

0 0

Montgomery Montgomery et al. Mukesh Munro Murugappan et.al. Nave Neuscheler et.al. Nevalainen et.al. (a) Nevalainen et.al. (b) Nielsen et.al. Noble Olexa Pearson Plotkin et.al. Pyzdek (a)

2002

A

0

0

2001 I A 2003 I 2000 I

0 1 0

2003 2002 2001

I I I

0 1 1

0 Pe Sy Ph 1 Sy Ph 1 Sy Ph Sy 0 To Ph Pr 0 Sy Ph 0 Sy Ph

2000

I

0

0

Sy

2000 1999 2001 2003 2001 1999 2001

I I I I I I I

1 0 0 0 0 0 0

1 0 1 0 0 1 0

0 0 1 1 0 0 1 0 0 0 0 0

0 1 1 0 0 0 0 0 0 0 0 0

Sy Sy Sy To To Sy Sy Pe To Sy Sy Sy Sy To Sy To Sy Sy Sy Sy

Pyzdek (b) Ramberg Rasis et.al. (a) Rasis et.al. (b) Rayner Ribardo et.al. Riley et al. Rowlands et.al. Sanders et.al. (a) Sanders et.al. (b) Sanders et.al. Sarewitz

2002 2001

2001 I 2000 A 2003 I A 2003 I A 1990 I 2003 I A 2002 A 2003 A 2000 I 2000 I 2001 I 2000 I

Sy Sy Sy To Sy Sy To Sy Sy To Sy Pe To To To Sy Pe To Sy Pe To

Ph Ph Ph To Ph

G G

0.2 0.3

TA Su

Y N N Y

-

-

-

-

-

-

-

-

-

-

Y

-

-

M G

0.4 1.5

C TA

N N Y Y

-

-

-

-

-

-

-

-

-

Y

-

-

-

Pr To Ph Ph To Ph

G G M M M

0.2 0.2 0.2 0.2 0.2

TA TA TA TA C

Y Y Y N N

Y Y N N N Y -

-

-

-

-

-

-

-

-

-

Y

-

-

Ph Pr To To Ph

G G M M

NA 0.2 NA 0.2

Su R C C TA

N N N Y

Y Y Y Y Y - - N - - N - - -

-

-

-

-

-

-

-

Y -

-

-

Ph Pr G Ph G

0.2 0.2

TA TA

Y N Y Y Y

-

-

-

-

-

-

-

-

Y

-

-

-

Ph Pr G

0.2

TA

Y N

-

-

-

-

-

-

-

-

-

-

-

-

-

G M M

1.5 0.4 0.2

TA C Co

Y N N N N N

-

-

-

-

-

-

-

-

-

-

-

-

-

Se G G

0.8 0.2 0.2

C Co TA

N N Y N Y N

-

-

-

-

-

-

-

-

-

-

-

-

-

Ph

Se

1.3

TA

Y N

-

-

-

-

-

-

-

-

-

-

-

-

-

Ph Ph Ph Pr To Ph Ph Ph To Ph Ph Ph Ph To Ph To Ph Ph Ph Ph

Se Se M M G M M

1.3 0.3 0.4 0.3 0.2 NA NA

Co N N C N N Su N N C N Y TA Y N C N N Co N N

-

-

-

-

Y -

-

-

Y -

-

-

-

-

-

G G M M G M Se M G Se G Se

1.5 NA NA NA NA 0.2 NA 0.6 NA NA NA 1.3

TA TA C C C C C C TA C TA TA

N Y N N N N N N Y Y N N N -

-

Y -

-

-

-

-

- - Y - - - - - - Y - - - -

-

-

Y -

-

37

Y Y N N N N N N Y N Y Y

Scalise Scalise Schmitt Schmitt Schmitt Sigal et al. Smith Snee Snee (a)

2001 2003 2000 2001 2002 2001 2003 1999 2000

I I I I I A I I I

0 0 0 0 0 1 0 1 0

Snee (b) Snee (a)

2000 2001

I I

0 0

Snee (b) Snee Stamatis Stein Studt Takikamalla Tang et.al. Treichler et.al. Trivedi Tylutki et.al. Vandenbrande Vaugham Velocci (a) Velocci (b) Velocci (c) Velocci Velocci Voelkel Walsh et.al. Watson Watson (a) Watson (b) Waurayniak Weinstein et.al. Westgard Wheeler Wiklund et.al. Wood Wyper et.al. Yeung et.al. Yu et.al.

2001 2003 2000 2001 2002 1994 1997 2002 2002 2002 1998 1998 1998 1998 1998 2000 2002 2002 2000 2000 2002 2002 2002 1998 2002 2002 2002 2001 2000 2003 1994

I I I I I A A I I A I A I I I I I A I I I I I A I I A I I A A

0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0

0 Sy Ph 0 Sy Ph 1 Sy Ph 1 Sy Ph 1 Sy Ph 0 Sy Ph 0 Sy Ph 1 Pe Sy Ph 1 Pe Sy Ph Pe Ph 0 To To 0 To To Pe Ph 0 To To 0 Sy Ph 0 Sy Ph 0 To To 0 Sy Ph 1 To To 0 To To 0 To Pr 0 Sy Ph 0 Sy Ph 0 To To 1 To To 1 Sy Ph 0 Sy Ph 0 Sy Ph 1 Sy Ph 0 Sy Ph 0 To To 0 Sy Ph 0 Pe Sy Ph 0 To To 0 Sy Ph 0 To To 0 Pe Sy Ph 0 To Pr To 0 Sy Ph 0 Sy Ph 0 Sy Ph 1 Pe Sy Ph 0 Sy Ph 0 Sy Ph

M Se M M G Se M G G

NA NA 0.3 0.3 0.3 NA 0.2 0.2 NA

Su N N C N N Su N N Su N N Su N N C N N C N Y Y TA Y Y R Y N -

-

-

-

-

Y -

-

Y -

-

-

Y -

-

-

G G

0.2 0.2

TA TA

Y Y Y Y Y

-

Y -

-

-

-

Y

-

-

-

-

-

-

G G M G G M M G M Se M G M M M M M G G G Se Se M G Se M Se G Se M M

1.5 0.2 NA 0.2 0.7 0.2 0.2 0.2 0.4 0.2 0.2 NA 0.3 0.3 0.3 0.3 0.3 0.2 0.3 0.2 0.2 0.5 0.3 0.2 NA 0.4 NA 0.3 NA 1.5 0.4

TA TA C Co TA TA TA Co C C C TA C Co C C C Su C Su Co C Co TA Su Su TA Co C C C Su R

Y Y Y Y Y Y Y N N N Y N N N N N N N N N N Y N N N N N N N N N

Y Y Y Y -

Y Y Y Y -

-

- - - - - - - Y Y - - - - - - - - Y - - - - - Y - - Y Y - - - - -

-

Y -

-

-

- - - - - - - - - - - - - - Y Y - - - - - - - - - - - - - - - - - Y Y - - - - - - - - - - - - - - - - - - Y - - - - - - - - - -

38

N N N N Y N N Y Y N N N N N N N Y Y N N N N Y N N Y Y Y N Y N

Y Y Y Y -

Appendix B – Article Reference Listing Abraham, B., Mackay, J., 2001. Discussion of “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, no. 4, October, pp. 410-413. Ackermann, C., 1993. “Supplier improvement via SPC application workshops.” IEEE Transactions on Semiconductor Manufacturing, vol. 6, no. 2, May, pp. 178-183. Ackermann, C., Fabia, J., 1993. “Monitoring supplier quality at ppm levels.” IEEE Transactions on Semiconductor Manufacturing, vol. 6, no. 2, May, pp.189-195. Ahire, S., Landeros, R., Golhar, D., 1995. “Total quality management: a literature review and an agenda for future research.” Production and Operations Management, pp. 277-307. Ali, O., Chen, Y., 1999. “Design quality and robustness with neural networks.” IEEE Transactions on Neural Networks, vol. 10, no. 6, Nov, pp. 1518-1527. Antony, J., Coronado, R., 2002. “Design for Six Sigma.” Manufacturing Engineer, vol. 81, no. 1, pp. 24-26. Arvidsson, M., Gremyr, I., Johansson, P., 2003. “Use and knowledge of robust design methodology: a survey of Swedish industry.” Journal of Engineering Design, vol. 14, no. 2, June, pp. 129-143. Bailey, S., 2001. “Discussion of “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, no. 4, Oct., pp. 426-431. Bartos, F., 1999. “Six sigma for complex systems.” Control Engineering, vol. 46, no. 3, Mar., pp. 90. Basu, R., 2001. “Six sigma to fit sigma.” IIE Solutions, vol. 33, no. 7, Jul., pp. 28-33. Behara, R., Fontenot, G., Gresham, A., 1995. “Customer Satisfaction Measurement and Analysis Using Six Sigma.” International Journal of Quality & Reliability Management, vol. 12, no. 3, pp. 9-18. Benedetto, A., 2003. “Adopting manufacturing-based Six Sigma methodology to the service environment of a radiology film library.” Journal of Healthcare Management vol. 48, no. 4, Jul-Aug, pp. 263-280. Berlowitz, D., 2003. “Striving for Six Sigma in pressure ulcer care.” Journal of American Geriatrics Society, vol. 51, no. 9, Sept., pp. 1320-1321. Binder, R., 1997. “Can a manufacturing quality model work for software?” IEEE Software, vol. 14, no. 5, Sep-Oct, pp. 101. Bisgaard, S., Freiesleben, J., 2000. “Quality Quandaries: Economics of Six Sigma Program.” Quality Engineering, vol. 13, no. 2, Dec., pp. 325-331. Blakeslee, J., 1999. “Implementing the Six Sigma Solution – How to achieve quantum leaps in quality and competitiveness.” Quality Progress, vol. 32, no. 7, July, pp. 7785. Blanton, P., 2002. “Quality tools in science education.” The Physics Teacher, vol. 40, Mar., pp.188-189. Bossert, J., 2003. “Lean and Six Sigma – Synergy made in heaven.” Quality Progress, vol. 36, no. 7, July, pp. 31-32. Breyfogle, F., 2002. “Golf and Six Sigma – Use Six Sigma metrics to drive proper process behavior.” Quality Progress, vol. 35, no. 11, Nov., 83-85. Breyfogle, F. W. (2003), Implementing Six Sigma: Smarter Solutions Using Statistical Methods, Wiley.

39

Breyfogle, F., Connolly, M., 2003. “Six sigma methods to ensure organizations health.” R&D Magazine, vol. 45, no. 4, Apr., pp. 28-29. Breyfogle, F., Enck, D., 2002. “Six sigma goes corporate.” Business Management, May, pp. 70. Breyfogle, F., Enck, D., Meadows, B., 2001. Discussion of “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33 no. 4, Oct., pp. 424-425. Breyfogle, F., Meadows, B., 2001. “Bottom-line success with Six Sigma – Define key process output variables and their effects on the cost of poor quality.” Quality Progress, vol. 34, no. 5, May, pp. 101-104. Broderick, L., Knuteson, H., Rankin, R., Woodward, L., 2002. “Use of Six Sigma methodology to enhance capacity management in an academic center-first year’s experience.” Radiology Society of North America, 88th Meeting, Published abstract of talk given Dec. 4, 2002. Buck, C., Miller, R., Desmarais, J., 2001. “Six Sigma – The quest for quality, Hospitals & Health Networks.” Vol. 75, no. 12, Dec., pp. 41-48. Buck, C., 1998. “Health care through a Six Sigma lens.” Milbank Quarterly, vol. 76, no. 4, pp. 749-753. Buck, C., 2001. “What Hospital leaders say about Six Sigma.” Hospitals & Health Networks, vol. 75, no. 12, Dec. pp. 43. Buggie, F., 2000. “Beyond ‘Six Sigma’.” Journal of Management Engineering, vol. 16, no. 4, Jul.-Aug., pp. 28-31. Card, D., 2000. “Sorting out Six Sigma and the CMM.” IEEE Software, vol. 17, no. 3, May-Jun., pp. 11-13. Caulcutt, R., 2001. “Why is Six Sigma so successful?” Journal of Applied Statistics, vol. 28, no. 3-4, Mar.-May, pp. 301-306. Chan, K., Spedding, T., 2001. “On-line Optimization of Quality in a Manufacturing System.” International Journal of Production Research, vol. 39, no. 6, Apr., pp. 11271145. Chassin, M., 1998. “Is healthcare ready for Six Sigma quality?” Milbank Quarterly, vol. 76, num. 4, pp. 565-591. Chowdhury, S., 2000. “Working toward Six Sigma success.” Manufacturing Engineering, vol. 127, num. 1, July, pp. 14. Clifford, L., 2001. “Trend spotting – Why you can safely ignore Six Sigma.” Fortune, vol. 143, num. 2, Jan., pp. 140. Coleman, S., Arunakumar, G., Foldvary, F., Feltham, R., 2001. “SPC as a tool for creating a successful business measurement framework.” Journal of Applied Statistics, vol. 28, num. 3-4, Mar.-May, pp. 325-334. Connolly, M., 2003. “Six Sigma deployment at DuPont.” R&D Magazine, vol. 45, num. 4, Apr., pp. 29. Connor, G., 2003. “Benefiting from Six Sigma.” Manufacturing Engineering, vol. 130, num. 2, Feb., pp. 53-59. Cooper, D., Babcock, J., Dipietro, F., 1992. “Application of 6 sigma-statistical qualitycontrol to monitoring the deposition of contaminating particles.” Journal of The IES, vol. 35 num. 5, Sept.-Oct., pp. 27-32.

40

Cooper, N., Noonan, P., 2003. “Do teams and Six Sigma go together?” Quality Progress, vol. 36, num. 6, June, pp. 25-28. Crom, S., 2000. “Implementing Six Sigma – A cross-cultural perspective.” Quality Progress, vol. 33, num. 10, Oct., pp. 73-75. Dasgupta, T., 2003. “Using the six-sigma metric to measure and improve the performance of a supply chain.” Total Quality Management & Business Excellence, vol. 14, num. 3, May, pp. 355-366. Davies, H., 2001. “Exploring the pathology of quality failings: measuring quality is not the problem – changing it is.” Journal of Evaluation in Clinical Practice, vol. 7, num. 2, May, pp. 243-251. Davig, W., Brown, S., Friel, T., Tabibzadeh, K., 2003. “Quality management in small manufacturing.” Industrial Management & Data Systems, vol. 103, num. 1-2, pp. 6877. Dean, J., Bowen, D., 1994. “Managing theory and total quality: improving research and practice through theory development.” Academy of Management Review, vol. 19, num. 3, pp. 392–418. Dedhia, N., 1995. “Survive Business challenges with the total quality management approach.” Total Quality Management, vol. 6, num. 3, July, pp. 265-272. DeFeo, J., 2000. “An ROI story.” Training & Development, vol. 54, num. 7, July, pp. 25-26. De Mast, J., 2003. “Quality improvement form the viewpoint of statistical method.” Quality and Reliability Engineering International, vol. 19: pp. 255-264. De Mast, J., Schippers, W., Does, R., van den Heuvel, E., 2000. “Steps and strategies in process improvement.” Quality and Reliability Engineering International, vol. 16, num. 4, Jul.-Aug., pp. 301-311. Deshpande, P., 1998. “Emerging technologies and Six Sigma, Hydrocarbon Processing, vol. 77, num. 4, Apr., pp. 55. Deshpande P., Makker, S., Goldstein, M., 1999. “Boost competitiveness via Six Sigma.” Chemical Engineering Progress, vol. 95, num. 9, Sept., pp. 65-70. Does, R., van den Heuvel, E., de Mast, J., Bisgaard, S., 2002. “Quality quandaries: comparing nonmanufacturing with traditional applications of Six Sigma.” Quality Engineering, vol. 15, num. 1, Mar., pp. 177-182. Doganaksoy, N., Hahn, G., Keeker, W., 2000. “Product life data analysis: A case study.” Quality Progress, vol. 33, num. 6, June, pp. 115. Dornheim, M., 2001. “Implement Six Sigma.” Aviation Week & Space Technology, vol. 155, num. 1, July, pp. 25. Douglas, P., Erwin, J., 2000. “Six sigma focus on total customer satisfaction.” Journal of Quality & Participation, vol. 23, num. 2, Mar.-Apr., pp. 45-49. Du, X., Chen, W., 2000. “Towards a better understanding of modeling feasibility robustness in engineering design.” Journal of Mechanical Design, vol. 122, num. 4, Dec., pp. 385-394. Duguesnoy, L., Berger, J., Prevot, P., Sandoz-Guermond, F., 2002. “SIMPA: A training platform in work station including computing tutors.” Lecture Notes in Computer Science, vol. 2363: pp. 507-520.

41

Eid, M., Moghrabi, C., Eldin, H., 1997. “A Simulation Approach to Evaluating Quality/Cost Decision Scenarios.” Computers & Industrial Engineering, vol. 33, num. 1-2, pp. 105-108. Frantz, K., 2001. “Apply quality to motion control.” Control Engineering, vol. 48, num. 10, Oct., pp. 8-10. Feng, C., Kusiak, A., 1997. “Robust tolerance design with the integer programming approach.” Journal of Manufacturing Science and Engineering Transactions of the ASME, vol. 119, num. 4A, Nov., pp. 603-610. Ferrin, D., Muthler, D., Miller, M., 2002. “Six Sigma and simulation, so what’s the correlation?” Proceedings of the 2002 Winter Simulation Conference, pp. 14391443. Finn, G., 1999. “Six-sigma quality for virtual products,” Manufacturing Engineering, vol. 123, num. 6, Dec., pp. 20. Fontenot, G., Behara, R., Gresham, A., 1994. “Six Sigma in customer satisfaction.” Quality Progress, vol. 27, num. 12, Dec., pp. 73-76. Fuller, H., 2000. Observations about the success and evaluation of Six Sigma at Seagate. Quality Engineering, vol. 12, num. 3, Mar., pp. 311-315. Gano, D., 2001. Effective problem solving: a new way of thinking. Annual Quality Congress Transactions, pp.110-122. Gautreau, N., Yacout, S., Hall, R., 1997. Simulation of Partially Observed Markov Decision Process and Dynamic Quality Improvement. Computers & Industrial Engineering, vol. 32, num. 4, Dec., pp. 691-700. Gill, M., 1990. Stalking Six Sigma. Business Month, Jan., pp. 42-46. Gnibus, R., 2000. Six Sigma’s Missing Link – Understanding the quality tool needed to calculate sigma ratings. Quality Progress, vol. 33, num. 11, Nov., pp. 77. Goh, T., 2001. A pragmatic approach to experimental design in industry. Journal of Applied Statistics, vol. 28, num. 3-4, Mar.-May, pp. 391-398. Goh, T., 2002. The role of statistical design of experiments in Six Sigma: perspectives of a practitioner.” Quality Engineering, vol. 14, num. 4, June, pp. 661-673. Goh, T., 2002. “A Strategic Assessment of Six Sigma.” Quality and Reliability Engineering International, vol. 18, num. 5, Sept.-Oct., pp. 403-410. Goh, T., Xie, M., 2003. “Statistical control of a Six Sigma process.” Quality Engineering, vol. 15, num. 4, June, pp. 587-592. Goh, T., Low, P., Tsui, K., Xie, M., 2003. “Impact of Six Sigma implementation on stock price performance.” Total Quality Management & Business Excellence, vol. 14 no. 7, Sept., pp. 753-763. Gordon, D., 2002. “Quality management systems vs. quality improvement.” Quality Progress, vol. 35, num. 11, Nov., pp. 86. Grandzol, J., Gershon, M., 1998. "A Survey Instrument for Standardizing TQM Modeling Research." International Journal of Quality Science, vol. 3, no. 1, pp.80105. Greek, D., 2000. “Inefficiency won’t wash.” Professional Engineering, vol. 13, num. 11, Jun. 7, p. 45. Gross, J., 2001. “A road map to Six Sigma quality.” Quality Progress, vol. 34, num. 11, Nov., pp. 24-29.

42

Hahn, G., Hoerl, R., 1998. “Key challenges for statisticians in business and industry.” Quality Progress, vol. 31, num. 8, Aug., pp. 195-200. Hahn, G., Hill, W., Hoerl, R., Zinkgraf, S., 1999. “The Impact of Six Sigma Improvement – A Glimpse Into the Future of Statistics.” The American Statistician, vol. 53, num. 3, Aug., pp. 208-215. Hahn, G., Doganaksoy, N., Hoerl, R., 2000. “The Evolution of Six Sigma.” Quality Engineering, vol. 12, num. 3, Mar., pp. 317-326. Hahn, G., Doganaksoy, N., Stanard, C., 2001. “Statistical tools for Six Sigma – What to emphasize and de-emphasize in training.” Quality Progress, vol. 34, num. 9, Sept., pp. 78-82. Hahn, G., 2002. “Deming and the proactive statistician.” The American Statistician, vol. 56, num. 4, Nov., 290-298. Hammer, M., 2002. “Process management and the future of Six Sigma.” IEEE Engineering Management Review, vol. 30, num. 4, pp. 56-63. Harrold, D., 1999. “Designing for Six Sigma capability.” Control Engineering, vol. 46, num. 1, Jan., pp. 62-70. Harrold, D., Bartos, F., 1999. “Optimize existing processes to achieve Six Sigma capability.” Control Engineering, vol. 46, num. 3, Mar., pp. 87-103. Harry, M., 1998a. “Six Sigma: A Breakthrough Strategy for Profitability.” Quality Progress, vol. 31, num. 5, May, pp. 60-62. Harry, M., 1998b. “Six sigma article inaccurate – Author’s reply.” Quality Progress, vol. 31, num. 8, Aug., pp. 10. Harry, M., 2000a. “A new definition aims to connect quality with financial performance.” Quality Progress, vol., 33, num. 1, pp. 64-66. Harry, M., 2000b. “Six Sigma leads enterprises to coordinate efforts.” Quality Progress, vol. 33, num. 3, Mar., pp. 70-72. Harry, M., 2000c. “Six sigma focuses on improvement rates.” Quality Progress, vol. 33, num. 6, Jun., pp. 76-80. Harry, M., 2000d. “Abatement of business risk is key to Six Sigma.” Quality Progress, vol. 33, num. 7, Jul., p. 72. Harry, M., 2000e. “The quality twilight zone.” Quality Progress, vol. 33, num. 2, Feb., p. 68. Harry, M., 2000f. “Quality leads, answers follow.” Quality Progress, vol. 33, num. 5, May, p. 82. Henretta, K., Walker, J., Bellafiore, L., 2003. “Applying “Six Sigma” to chromatography – Tutorial: Cutting costs through process improvements.” Genetic Engineering News, vol. 23, num. 1, Jan., 54-56. Hild, C., Sanders, D., Cooper, T., 2000. “Six Sigma On Continuous Processes: How and Why It Differs.” Quality Engineering, vol. 13, num. 1, Sept., pp.1-9. Hill, W., 2001. “Discussion - Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 421-423. Hoerl, R., 1998. “Six Sigma and the future of the quality profession.” Quality Progress, vol. 31, num. 6, June, pp. 35-42. Hoerl, R., 2001a. “Six Sigma Black Belts: What Do They Need to Know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 391-406.

43

Hoerl, R., 2001b. “Response - Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 432-435. Horst, R., 1999. “Safety and Six Sigma.” Manufacturing Engineering, vol. 122, num. 2, Feb., pp. 14. Howell, D., 2000. “The power of six.” Professional Engineering, vol. 13, num. 14, July, pp. 34-35. Howell, D., 2001. “At sixes and sevens.” Professional Engineering, May, pp. 27. Hunter, D., 1999. “Six Sigma steps.” Chemical Week, vol. 161, num. 33, Sept., p. 3. Hunter, D., Schmitt, B., 1999. “Six sigma: Benefits and approaches.” CW Conference Proceedings, Chemical Week, vol. 1661, num. 37, Oct. 6, pp. 35-36. Hutchins, G., 2000. “The branding of Six Sigma.” Quality Progress, vol. 33, num. 9, Sept., pp. 120-121. Ingle, S., Roe, W., 2001. “Six sigma black belt implementation.” The TQM Magazine, vol. 13, num. 4, pp. 273-280. Johnson, A., 2002. “Six sigma in R&D.” Research-Technology Management, vol. 45 num. 2, Mar-Apr, pp.12-16. Johnson, A., Swisher, B., 2003. “Now Six Sigma improves R&D.” ResearchTechnology Management, vol. 46, num. 2, Mar.-Apr., pp.12-15. Johnstone, P., Dernbach, A., 2002. “Six sigma quality and delivery of radiation therapy.” Cancer Journal, vol. 8, num. 6, Nov.-Dec., p. 44. Johnstone, P., Hendrickson, J., Dernbach, A., Secord, A., Parker, J., Favata, M., Puckett, M., 2003. “Ancillary services in health care industry: is Six Sigma reasonable?” Quality Management in Health Care, vol. 12, num. 1, Jan.-Mar., p. 53. Kandebo, S., 1999. “Lean, Six Sigma yield dividends for C-130J.” Aviation Week and Space Technology, vol. 151, num. 2, July, p. 59-61. Kane, L., 1998. “The quest for Six Sigma.” Hydrocarbon Processing, vol. 77, num. 2, Feb., pp. 15. Kazmer, D., Hatch, D., Zhu, L., 2002. “Investigation of variation and uncertainty in Six Sigma.” Proceedings of the ASME Design Engineering Technical Conference, vol. 3, pp.21-29. Kazmierczak, S., 2003. “Laboratory quality control: Using patient data to assess analytical performance.” Clin Chem Lab Med, vol. 41, num. 5, May, pp. 617-627. Kendall, J., Fulenwider, D., 2000. “Six sigma, e-commerce pose new challenges.” Quality Progress, vol. 33, num. 7, July, 31-37. Kenett, R., Coleman, S., Stewardson, D., 2003. “Statistical efficiency: The practical perspective.” Quality and Reliability Engineering International, vol. 19, num. 4, Jul.Aug., p. 265-272. Knowles, G., Vickers, G., Anthony, J., 2003. “Implementing evaluation of the measurement process in an automotive manufacturer: a case study.” Quality and Reliability Engineering International, vol. 19, num. 5, Sept., pp. 397-410. Koch, P., 2002. “Probabilistic design: optimizing for Six Sigma quality.” Proceeds from 43rd AIAA/ASME/ASCE/AHS Structures, Structural Dynamics, and Materials Conference, 4th AIAA Non-Deterministic Approaches Forum, Denver, Colorado, AIAA-2002-1471 Koonce, D., et. al., 2003. “A hierarchical cost estimation tool.” Computers In Industry, vol. 50, num. 3, Apr., pp. 293-302.

44

Krouwer, J., 2002. “Using a learning curve approach to reduce laboratory errors.” Accreditation and Quality Assurance, vol. 7, num. 11, Nov., pp. 461-467. Kunes, R., 2002. “Six Sigma article is misleading.” Quality Progress, vol. 35, num. 3, Mar., p. 8. Landin, A., Nilsson, C., 2001. “Do quality systems really make a difference?” Building Research and Information, vol. 29, num. 1, Jan., pp. 12-20. Lee, K., 2002. “Critical Success Factors of Six Sigma Implementation and the impact on Operations Performance.” Ph. D. Dissertation, Cleveland State University. Leffew, K., Yerrapragada, S., Deshpande, P., 2001. “6 sigma and solid-state polymerization.” Chemical Engineering Communications, vol. 188, pp. 109-114. Linderman, K., Schroeder, R., Zaheer, S., Choo, A., 2003. “Six Sigma: A goal-theoretic perspective.” Journal of Operations Management, vol. 21, num. 2, Mar., p. 193-203. Lucas, J., 2002a. “The essential Six Sigma – How successful Six Sigma implementation can improve the bottom line.” Quality Progress, num. 35, vol. 1, Jan., pp. 27-31. Lucas, J., 2002b. Response to “Six Sigma article is misleading.” Quality Progress, vol. 35, num. 3, Mar., pp. 8-9. Mader, D., 2002. “Design for Six Sigma.” Quality Progress, num. 35, vol. 7, July, pp. 82. Maguire, M., 1999a. “Six sigma saga.” Quality Progress, vol. 32, num. 10, Oct., p. 6. Magure, M., 1999b. “Cowboy Quality: Mike Harry’s riding tall in the saddle as Six Sigma makes its mark.” Quality Progress, vol. 32, num. 10, Oct., pp. 27-34. Mandal, P., Howell, A., Sohal, A., 1998. “A systemic approach to quality improvements: The interactions between the technical, human and quality systems.” Total Quality Management, vol. 9, num. 1, Feb., pp. 79-100. Mason, R., Young, J., 2000. “Interpretive features of a T(2) chart in multivariate SPC.” Quality Progress, vol. 33, num. 4, Apr., pp. 84-89. McCarthy, B., Stauffer, R., 2001. “Enhancing Six Sigma through simulation with IGRAFX process for Six Sigma.” Proceeding of the 2001 Winter Simulation Conference, pp. 1241-1247. McFadden, F., 1993. “Six sigma quality programs.” Quality Progress, vol. 26, num. 6, Jun., pp. 37-42. Montgomery, D., 2000. “The present state of industrial statistics.” Quality and Reliability Engineering International, vol. 16, num. 4, Jul.-Aug., pp. 253-254. Montgomery, D., 2001. “Beyond Six Sigma.” Quality and Reliability Engineering International, vol. 17, num. 4, Jul.-Aug., pp. iii-iv. Montgomery, D., 2002. “Changing roles for the Industrial Statistician.” Quality and Reliability Engineering International, vol. 18, num. 5, Sept.-Oct., p. iii. Montgomery, D., Lawson, C., Molnau, W., Elias, R., 2001. “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 407-409. Motyka, M., 2000. “Six sigma, QS-9000 article has one minor flaw.” Quality Progress, vol. 33, no. 8, Aug., p. 8. Mukesh, D., 2003. “Putting Six Sigma processes to work.” Chemical Engineering, vol. 110, num. 12, Nov., p. 62. Munro, R., 2000. “Linking Six Sigma with QS-9000.” Quality Progress, vol. 33, num. 5, May, pp. 47-53.

45

Murugappan, M., Keeni, G., 2003. “Blending CMM and Six Sigma to meet business goals.” IEEE Software, vol. 2, Mar.-Apr., pp. 42-48. Nave, D., 2002. “How to compare Six Sigma, lean and the theory of constraints – A framework for choosing what’s best for your organization.” Quality Progress, vol. 35, num. 3, Mar., pp. 73-78. Neuscheler, F., Norris, R., 2001. “Capturing financial benefits from Six Sigma – five lessons learned will resonate with top management.” Quality Progress, vol. 34, num. 5, May, pp. 39-44. Nevalainen, D., et. al., 2000. “Evaluating laboratory performance on quality indicators with the Six Sigma scale.” Archives of Pathology & Laboratory Medicine, vol. 124, num. 4, Apr., pp. 516-519. Nevalainen, D., Berte, L., 2000. Replies to “Evaluating laboratory performance with the Six Sigma scale.” Archives of Pathology & Laboratory Medicine, vol. 124, num. 12, Dec., p. 1748. Nielsen, K., Orshal, J., 1999. “Companies – Dow accelerates Six Sigma effort; Reports on ‘social responsibility’.” Chemical Week, vol. 161, num. 37, Oct. 6, p. 9. Noble, T., 2001. “Six sigma boosts the bottom line.” Chemical Engineering Progress, vol. 97, num. 4, Apr., pp. 9-11. Oakland, J., 1989. Total Quality Management, Butterworth-Heinemann, London. Olexa, R., 2003. “Driving quality with Six Sigma.” Manufacturing Engineering, vol. 130, num. 2, Feb., pp. 61. Pande, P., Neuman, R., Cavanaugh, R., 2000. The Six Sigma Way: How GE, Motorola, and Other Top Companies are Honing Their Performance, McGraw-Hill, New York. Pearson, T., 2001. “Measure for Six Sigma success.” Quality Progress, vol. 34, num. 2, Feb., pp. 35-40. Plotkin, C., et. al., 1999. “Panel: Advisory board – What are the successful companies doing?” Annual Reliability and Maintainability Symposium 1999 Proceedings, pp. 219-223. Pyzdek, T., 2001a. “Why Six Sigma is not TQM.” Quality Digest, Feb., p. 26. Pyzdek, T., 2001b. Discussion, “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 418-420. Ramberg, J., 2000. “Six Sigma: Fad or Fundamental?” Quality Digest, May, pp. 28-32. Rasis, D., Gitlow, H., Popovich, E., 2002a. “Paper Organizers International: A Fictitious Six Sigma Green Belt Case Study, I.” Quality Engineering, vol. 15, num. 1, Sept., pp.127-146. Rasis, D., Gitlow, H., Popovich, E., 2002b. “Paper Organizers International: A Fictitious Six Sigma Green Belt Case Study, II.” Quality Engineering, vol. 15, num. 2, Dec., pp. 259-274. Rayner, B., 1990. “Market-driven Quality: IBM’s Six Sigma Crusade.” Electronic Business, vol. 1, num. 10, Oct., pp. 68-74. Ribardo, C., Allen, T., 2003. “An Alternative Desirability Function For Achieving Six Sigma Quality.” Quality and Reliability Engineering International, vol. 19, num. 3, May-Jun., pp. 227-240. Riley, J., Justison, G., Povrzenic, D., Zabetakis, P., 2002. “Designing an integrated extracorporeal therapy service quality system.” Therapeutic Apheresis, vol. 6, num. 4, Aug., pp. 282-287.

46

Rowlands, H., Antony, F., 2003. “Application of design of experiments to a spot welding process.” Assembly Automation, vol. 23, num. 3, pp. 273-279. Sanders, D., Hild, C., 2000a. “A Discussion of Strategies for Six Sigma Implementation.” Quality Engineering, vol. 12, num. 3, Mar., pp. 303-309. Sanders, D., Hild, C., 2000b. “Six Sigma on Business Processes: Common Organizational Issues.” Quality Engineering, vol. 12, num. 4, Jun., pp. 603-610. Sanders, D., Hild, C., 2000c. “Common myths about Six Sigma.” Quality Engineering, vol. 13, num. 2, Dec., pp. 269-276. Sarewitz, S., 2000. “Evaluating laboratory performance with the Six Sigma scale.” Archives of Pathology & Laboratory Medicine, vol. 124, num. 12, Dec., p. 1748. Scalise, D., 2001. “Six sigma: the west for quality.” Hospitals & Health Networks, vol. 75, num. 12, Dec., p. 41. Scalise, D., 2003. “Six Sigma in action – Case studies in quality put theory into practice.” Hospitals & Health Networks, vol. 77, num. 5, May, p. 57. Schmitt, B., 2000. “Moving ahead with Six Sigma.” Proceedings from CW Conference, Chemical Week, vol. 162, num. 17, Apr., pp. 64-68. Schmitt, B., 2001. “Expanding Six Sigma.” Chemical Week, vol. 163, num. 8, Feb. 21, pp. 21-23. Schmitt, B., 2002. “A slow spread for Six Sigma.” Chemical Week, vol. 164, num. 6, Jan. 13, pp. 34-36. Sigal, R., Dessales-Martin, D., Ruelle, C., Kouchit, N., Guillemot, M., Klipfel, B., 2001. “Implementation of a PACS using Six Sigma methodology.” Radiology Suppl. S, vol. 221, Nov., p. 527. Smith, B., 2003. “Lean and Six Sigma – A one-two punch.” Quality Progress, vol. 36, num. 4, Apr., pp. 37-41. Snee, R., 1999. “Why should statisticians pay attention to Six Sigma?” Quality Progress, vol. 32, num. 9, Sept., pp. 100-103. Snee, R., 2000a. “Impact of Six Sigma on quality engineering.” Quality Engineering, vol. 12, num. 3, Mar., pp. ix-xiv. Snee, R., 2000b. “Six sigma improves both statistical training and processes.” Quality Progress, vol. 33, num. 10, Oct., pp. 68-72. Snee, R., 2001a. “Dealing with the Achilles’ heel of Six Sigma initiatives – Project selection is key to success.” Quality Progress, vol. 34, num. 3, Mar., p. 66. Snee,R., 2001b. Discussion of “Six Sigma Black Belts: What do they need to know?” Journal of Quality Technology, vol. 33, num. 4, Oct., pp. 414-417. Snee, R., 2003. “The Six Sigma Sweep.” Quality Progress, vol. 36, num. 9, Sept., p. 7678. Sousa, R., Voss, C., 2002. “Quality management re-visited: a reflective review and agenda for future research.” Journal of Operations Management, vol. 20, pp. 91-109. Stamatis, D., 2000. “Who needs Six Sigma, anyway?” Quality Digest, May, pp. 33-38. Stein, P., 2001. “Measurements for business.” Quality Progress, vol. 34, num. 2, Feb., p. 29. Studt, T., 2002. “Implementing Six Sigma in R&D.” R&D Magazine, vol. 44, num. 8, Aug., pp. 21-23. Tadikamalla, P., 1994. “The Confusion Over Six Sigma Quality.” Quality Progress, Nov., pp. 83-85.

47

Tang, L., Than, S., Ang, B., 1997. “A graphical approach to obtaining confidence limits of C-pk.” “Quality and Reliability Engineering International, vol. 13, num. 6, Nov.Dec., pp. 337-346. Treichler, D., Carmichael, R., Kusmanoff, A., Lewis, J., Berthiez, G., 2002. “Design for Six Sigma: 15 lessons learned – Leading corporations find out how to avoid pitfalls.” Quality Progress, vol. 35, num. 1, Jan., pp. 33-42. Trivedi, B., 2002. “Applying Six Sigma.” Chemical Engineering Progress, vol. 98, num. 7, Jul., pp. 76-81. Tylutki, T., Fox, D., 2002. “Mooooving toward Six Sigma.” Quality Progress, vol. 35, num. 2, Feb., pp. 34-41. Vandenbrande, W., 1998. “How to use FMEA to reduce the size of your quality toolbox.” Quality Progress, vol. 31, num. 11, Nov., pp. 97-100. Vaugham, T., 1998. “Defect rate estimation for ‘Six Sigma’ processes.” Production & Inventory Management Journal, vol. 39, num. 4, Oct., pp. 5-9. Velocci, A., 1998a. “Pursuit of Six Sigma emerges as industry trend.” Aviation Week and Space Technology, vol. 149, no. 20, Nov. 16, pp. 52-53. Velocci, A., 1998b. “High hopes riding on Six Sigma at Raytheon.” Aviation Week and Space Technology, vol. 149, no. 20, Nov. 16, pp. 59-60. Velocci, A., 1998c. “Six sigma takes a back seat to ‘Lean Electronics’ at Rockwell.” Aviation Week and Space Technology, vol. 149, no. 20, Nov. 16, pp. 60-62. Velocci, A., 2000. “Raytheon Six Sigma meets initial target.” Aviation Week and Space Technology, vol. 152, num. 13, Mar. 27, p. 59. Velocci, A., 2002. “Full potential of Six Sigma eludes most companies.” Aviation Week and Space Technology, vol. 157, num. 14, Sep. 30, pp. 56-60. Voelkel, J., 2002. “Something’s missing – An education in statistical methods will make employees more valuable to Six Sigma corporations.” Quality Progress, vol. 35, num. 5, May, pp. 98-101. Walsh K., Fuller J., Wood A., Moore, S., Seewald, N., Schmitt, B., 2000. “Six Sigma – Marshaling an attack on costs.” Chemical Week, vol. 162, num. 9, Mar. 1, pp. 25-27. Watson, G., 2000. “Toward a Central Tendency of Six Sigma.” Quality Progress, July, p.16. Watson, G., 2002a. “Selling Six Sigma to Upper Management.” Six Sigma Forum Magazine, vol. 1, num. 4, Aug., pp. 26-37. Watson, G., 2002b. “Breakthrough in delivering software quality: Capability maturity model and Six Sigma.” Lecture Notes In Computer Science, vol. 2349, pp. 36-41. Waurzyniak, P., 2002. “Statistics improve quality.” Manufacturing Engineering, vol. 128, num. 2, p. 39-44. Weinstein, L., Petrick, J., Saunders, P., 1998. “What higher education should be teaching about quality – but is not.” Quality Progress, April, pp. 91-96. Westgard, J., 2002. “Evaluation of cardiac troponin assay systems and validation of QC in accordance with Six-Sigma principles and ISO guidelines.” Clinical Chemistry Suppl. S, vol. 48, num. 6, Jun., C61 Part 2. Wheeler, J., 2002a. “Getting started: Six-sigma control of chemical operations.” Chemical Engineering Progress, vol. 98, num. 6, Jun., pp. 76-81. Wheeler, J., 2002b. “Getting started with six-sigma.” Chemical Engineering Progress, vol. 98, num. 9, Aug., p. 8.

48

Wiklund, H., Wiklund, P., 2002. “Widening the Six Sigma concept: An approach to improve organizational learning.” Total Quality Management, vol. 13, num. 2, Mar., pp. 233-239. Wood, A., 2001. “Management – Making Six Sigma benefits stick.” Chemical Week, vol. 163, num. 19, May, p. 40. Wyper, B., Harrison, A., 2000. “Deployment of Six Sigma methodology in human resource function: A case study.” Total Quality Management, vol. 11, num. 4-6, July, pp. 720-727. Yeung, A., Chan, L., Ledd, T., 2003. “An empirical taxonomy for quality management systems: a study of the Hong Kong electronics industry.” Journal of Operations Management, vol. 21, num. 1, Jan., pp. 45-62. Yu, B., Popplewell, K., 1994. “Meta-models in Manufacturing: a Review.” International Journal of Production Research, vol. 32, pp. 787-796. Zain, Z., Dale, B., Kehoe, D., 2001. “Total quality management: an examination of the writings from a UK perspective.” The TQM Magazine, vol. 13, num. 2, Feb., pp. 129-137. Zwass, V., 1996. “Electronic commerce: structure and issues.” International Journal of Electronic Commerce, vol. 1, num. 1, pp. 3-33.

49

Suggest Documents