Transitions to Sustainable Development New Directions in the Study of Long Term Transformative Change

John Grin, Jan Rotmans and Johan Schot In collaboration with Frank Geels and Derk Loorbach

New York

Grin 3rd.indd iii

London

T&F Proofs: Not For Distribution

12/16/2009 10:13:23 AM

First published 2010 by Routledge 270 Madison Ave, New York, NY 10016 Simultaneously published in the UK by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN

Routledge is an imprint of the Taylor & Francis Group, an informa business

© 2010 Taylor & Francis Typeset in Sabon by IBT Global. Printed and bound in the United States of America on acid-free paper by IBT Global. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging in Publication Data Grin, John. Transitions to sustainable development : new directions in the study of long term transformative change / by John Grin, Jan Rotmans and Johan Schot ; in collaboration with Frank Geels and Derk Loorbach. p. cm.—(Routledge studies in sustainability transitions) Includes bibliographical references and index. 1. Sustainable development. 2. Change. I. Rotmans, Jan, 1961– II. Schot, J. W. III. Title. HD75.6.G75 2010 338.9'27—dc22 2009035625

ISBN10: 0-415-87675-3 (hbk) ISBN10: 0-203-85659-7 (ebk) ISBN13: 978-0-415-87675-9 (hbk) ISBN13: 978-0-203-85659-8 (ebk)

Grin 3rd.indd iv

T&F Proofs: Not For Distribution

12/16/2009 10:13:24 AM

Contents

List of Figures List of Tables List of Textboxes Foreword

ix xi xiii xv

CARLOTA PEREZ

Preface

xvii

Introduction: From Persistent Problems to System Innovations and Transitions

1

JOHN GRIN, JAN ROTMANS AND JOHAN SCHOT

PART I: The Dynamics of Transitions: A Socio-Technical Perspective FRANK W. GEELS AND JOHAN SCHOT

Grin 3rd.indd v

I.1

Introduction: Exploration of the Research Topic

11

I.2

A Multi-Level Perspective on Transitions

18

I.3

Theoretical Backgrounds: Science and Technology Studies, Evolutionary Economics and Sociology

29

I.4

A Typology of Transition Pathways

54

I.5

Managing Sustainable Innovation Journeys

80

I.6

Reflections: Process Theory, Causality and Narrative Explanation

93

T&F Proofs: Not For Distribution

12/16/2009 10:13:24 AM

vi Contents

PART II: Towards a Better Understanding of Transitions and Their Governance: A Systemic and Reflexive Approach JAN ROTMANS AND DERK LOORBACH

II.1 Introduction

105

II.2 A Complex Integrated Systems Perspective

114

II.3 Conceptual Framework for Analyzing Transitions

126

II.4 Research into the Governance of Transitions: A Framework for Transition Management

140

II.5 Case Study I: Parkstad Limburg: Regional Transition Management

161

II.6 Case Study II: The Dutch Energy Transition

180

II.7 Self-Evaluation of the Development and Prospects of Transition Management

199

PART III: Understanding Transitions from a Governance Perspective JOHN GRIN

III.1 Introduction

223

III.2 Contemporary Processes of Institutional Change

237

III.3 Modernization Processes in Dutch Agriculture, 1886 to the Present

249

III.4 The Governance of Transitions: An Agency Perspective

265

III.5 Modernization as Multilevel Dynamics: Lessons from Dutch Agriculture

285

III.6 Governance of Transitions: An Analytical Perspective

315

Grin 3rd.indd vi

T&F Proofs: Not For Distribution

12/16/2009 10:13:25 AM

Contents vii Conclusion: How to Understand Transitions? How to Influence Them? Synthesis and Lessons for Further Research

320

JOHN GRIN, JAN ROTMANS AND JOHAN SCHOT

Notes References About the Authors Index

Grin 3rd.indd vii

339 345 379 383

T&F Proofs: Not For Distribution

12/16/2009 10:13:26 AM

I.6

Reflections Process Theory, Causality and Narrative Explanation

I.6.1. PROCESS THEORY AND VARIANCE THEORY In this reflection section, we address two questions: What kind of theory is the MLP? What kinds of explanations does it give? There are two general types of explanation: 1) outcomes are explained through cause-and-effect explanation, and 2) the unfolding of processes is explained by identifying patterns and underlying mechanisms. There are two ways of seeing . . . historical processes more generally. One focuses on stochastic realizations and aims to fi nd causes; the other focuses on narratives and aims to fi nd typical patterns. (Abbott, 2001: 164) These types of explanation are related to variance theory and process theory. Variance theory explains outcomes as the product of independent variables acting on dependent variables. The aim is to explain the variation/ change in outcome (dependent variable) as a result of influences from causal factors (independent variables). The process approach looks at events rather than causal variables (Abbott, 1992). Events are enacted by actors who make decisions, undertake actions and react to each other. Process theories explain outcomes as the result of temporal sequences of events and the timing and conjunctures of event-chains. On this basis they identify patterns and mechanisms. Taking the notion of path dependence seriously, process theories explain outcomes by tracing the stream of events through which a process unfolds. Figure I.6.1 and Table I.6.1 contrast both approaches. Table I.6.1 Variance and Process Approaches (based on Poole et al., 2000: 36) Variance approach

Process theory

1. Fixed entities with varying attributes. Variables do the ‘acting’. 2. Attributes have single meaning over time. 3. Time ordering among independent variables is immaterial. 4. Emphasis on immediate causation. 5. Generality depends on uniformity across contexts (search for laws).

Grin 3rd.indd 93

Entities participate in events and may change identity over time. Actors do the ‘acting’. Entities, attributes, events may change in meaning over time. Time ordering of independent variables is critical. Explanations are layered and incorporate both immediate and distant causation. Generality depends on versatility across cases (variations within overall pattern).

T&F Proofs: Not For Distribution

12/16/2009 10:14:22 AM

94

Transitions to Sustainable Development

Figure I.6.1

Two approaches to explaining processes (Langley, 1999: 693).

They differ in five ways: 1) Character of entities. In variance theory, the world is made up of fi xed entities that maintain a unitary identity through time. The entities possess a fi xed set of variable attributes that reflect significant changes in the entity. Variables thus constitute the basic terms and are assumed to do the acting. The world is thus variabilized, i.e. viewed as consisting of interrelated variables (Poole et al., 2000: 32). In process theory the world is made up of entities that participate in events and may change their identity. Central subjects are individual entities (people, groups, organizations, machines and other material artifacts). Events are what central subjects do or what happens to them. Process theories look at events rather than variables, while actors do the acting (Abbott, 1992). 2) Stability of entities. In variance theory attributes have one causal meaning throughout the process. In process theory the unit of analysis may undergo metamorphosis over time and change meaning. Entities may define themselves differently and alter identity and preferences (as a result of experiences and learning). 3) Time order. In variance theory the temporal sequence in which independent variables exert influence is not important. It employs linear combinations of independent variables to predict dependent variables (Abbott, 1988). In process theories the temporal sequence of independent variables is critical. The order in which events and causal forces come to bear is crucial and may produce different outcomes. 4) Causation and explanation. Explanations in variance theory emphasize immediate causation. A cause is perceived as a force that acts on a unit of analysis. This is a push-type causality (Poole et al., 2000: 33). It is not necessary to know the twists and turns of an entity’s history to explain it.

Grin 3rd.indd 94

T&F Proofs: Not For Distribution

12/16/2009 10:14:22 AM

Refl ections

95

In process theory explanation requires the tracing of events, twists and turns. Events may have different durations. When events or processes run longer, causal influence is more enduring. As a result, explanations should incorporate layers ranging from immediate to distant explanation. Broad structural patterns and trends may need to be incorporated in explanations. This is why process theories are “causally deep” (Abbott, 1988). 5) Generality. In variance theories the generality of explanations depends on their ability to apply uniformly across a broad range of contexts. The generality of a causal mechanism depends on statistical generalization across many cases (ideally a large-N dataset). In process theory, the generality of explanations depends on their versatility, the degree to which they can encompass a broad domain of developmental patterns without modification of their essential character. The broader its domain (the greater variety of cases, contexts, events, and patterns the theory can adapt to), the more general the explanation (Poole et al., 2000: 43). I.6.2. PROCESS THEORY AND THE MLP The MLP is a process theory instead of variance theory. Ad 1) Transitions are enacted by different social groups. Ad 2) Actors change their perceptions of interests, preferences, and identity during transitions. Ad 3) The timing of events and multi-level linkages is important, influencing the type of transition pathway. Ad 4) Explanations in the MLP are layered and involve the tracing of twists and turns and alignments of event sequences and trajectories. Ad 5) The MLP has generality because it is versatile and maintains its basic character in different case studies and transition pathways. The MLP is also a process theory because its foundational ontologies (evolution and structuration theory) are historical theories that intrinsically focus on developments over time. Variance-theory methods have limited usefulness, because transitions are a particular kind of research topic. They are macroscopic, long-term processes, which are relatively rare. It is impossible, therefore, to construct a large database that can be analyzed statistically for correlations between variables. Furthermore, transitions involve complex dynamics, which are difficult to explain as simple cause-and-effect relations. Process theories seem more appropriate, because of a growing interest . . . in complex causal relations, such as path dependence, tipping points, multiple interaction effects, selection effects, disproportionate feedback loops, equifi nality (many alternative causal paths to the same outcome), and multifi nality (many outcomes consistent with a particular value of one variable). (George and Bennett, 2004: 9–10)

Grin 3rd.indd 95

T&F Proofs: Not For Distribution

12/16/2009 10:14:24 AM

96

Transitions to Sustainable Development

Process theory is a general type of explanation. Precise explanations of transitions involve many specific mechanisms. There is no single explanatory silver bullet. Because, as we discussed above, the MLP has several theoretical roots, different kinds of processes can provide explanations (at different time and aggregation levels). 1. The global, overall explanation provided by the MLP is about alignments and linkages between different processes. Within levels this explanation follows a socio-technical logic, investigating interactions between heterogeneous elements and actors (weaving a seamless web). The focus is on co-evolutionary interactions between ongoing trajectories: developments in one trajectory (e.g. regulations) may hinder or stimulate developments in another trajectory (e.g. technology or markets). Positive and negative feedbacks play a role here. Between levels, the explanation is evolutionary, in the sense that the diffusion of niche-innovations depends on ongoing dynamics in the broader societal environment (regime and landscape). Selection is multi-dimensional, because it not only involves markets, but also regulations, cultural and social movements, infrastructure and legitimacy. So, evolution is a linkage process, which consists of making alignments between niche-variations and societal selection environments. 2. To understand individual trajectories one first needs to investigate the context (i.e. other trajectories at that level, and developments at higher or lower levels). Within this context-analysis, two further process explanations are possible. The first is to analyze trajectories as morphogenetic cycles (Chapter I.2.3), i.e. event-chains where actors a) draw on structural contexts, b) interact with each other, c) aggregate and select outcomes, and d) institutionalize outcomes in new structures (reproduction or change). The metaphor is socially embedded game playing, where actors make moves, change tangible elements, and reproduce or change the rules of the game. The explanation then comes both from the rules of the game and the moves actors make. The second explanation is evolutionary, based on the generation of variations within populations and their subsequent selection and institutionalization; this explanation is especially useful for competition and innovation dynamics in firm populations, which generate technological trajectories. 3. To understand particular events, variations or local projects, one needs to zoom in further and look at specific actors. Structuration theory can provide detailed process explanations by analyzing how actors draw upon structures in which they are embedded (Stones, 2005). Motivations, perceptions, aims, and interests of specific actors play a role here. While such detailed explanations may be useful for the analysis of local projects and niches, they are less practical for entire transitions (since it is practically impossible to study thousands of actors over 50-year periods).

Grin 3rd.indd 96

T&F Proofs: Not For Distribution

12/16/2009 10:14:24 AM

Refl ections

97

Analysts should thus mobilize different process explanations for different questions and topics, depending on time scales and aggregation levels (Zaheer et al., 1999). In general, analysts should not over-emphasize individual actions when it comes to entire transitions. Although agency is important, turning points usually depend on broader structural opportunities: Since all structures are continuously re-enacted, it will happen from time to time that several local structures under a larger one might be simultaneously disconnected and their own reproduction prevented. This leaves an opening for action, a new juncture, that might assemble their constituent parts in a new way. If some actor takes that action, the result could be a minor turning point, the larger structure going on invulnerable. But once in a while, this minor turning point may line up with other minor turning points to create an opening in the overarching master structure. Then we have a potential major turning point, in which the whole general regime can change if the proper action is taken. But just as all reproduction hinges on continuous action, so a potential turning point becomes actual only if the action is taken that makes it so. Many potential revolutions fail for want of attempt, just as many attempted revolutions fail for want of structural opportunity. . . . Only after the action has been taken that turns the key can we speak of the turning point as having occurred. It is in this dialogue of structural possibility and action that turning points are defined. (Abbott, 2001: 257–258) I.6.3. NARRATIVE EXPLANATION Instead of process theory, process-oriented scholars often also use the term “narrative explanation” (Griffi n, 1993; Calhoun, 1998; Pentland, 1999; Abell, 2004). The strength of a narrative is that it can capture complex interactions between agency and changing contexts, time, event sequences, making moves in games, and identities. Narratives are always about something or someone, who has certain aims, undertakes action, learns and adjusts. Theorizing the social process via narrative is a deep tradition in both history and sociology. If there is any one idea central to historical ways of thinking, it is that the order of things makes a difference, that reality occurs not as time-bounded snapshots within which “causes” affect one another . . . but as stories, cascades of events. And events in this sense are not single properties, or simple things, but complex conjunctures in which complex actors encounter complex structures. On this argument, there is never any level at which things are standing still. All is historical. Furthermore, there are no independent causes. Since no

Grin 3rd.indd 97

T&F Proofs: Not For Distribution

12/16/2009 10:14:24 AM

98

Transitions to Sustainable Development cause ever acts except in complex conjunctures with others, it is chimeral to imagine the world in terms of independent causal properties acting in and through independent cases. (Abbott, 2001: 227) Narrative explanation takes the form of an unfolding, open-ended story fraught with conjunctures and contingency, where what happens, an action, in fact happens because of its order and position in the story. Narrative therefore permits a form of sequential causation that allows for twisting, varied, and heterogeneous time paths to a particular outcome. (Griffi n, 1993: 1099)

Not all narratives can be seen as process theories, however. Some narratives only describe “one damn thing after another.” For this reason, Skocpol (1994: 332) has warned for an unreflexive turn to narrative: To advise people to write “narratives” is really to advise nothing. For narratives can be structured in many, many ways. It takes powerful investigative (and justificatory) methods, as well as a rich array of everrefi ned theoretical ideas to figure out what “structures” and “conjunctures” count, and which happenings are transformative as opposed to merely humdrum. (Skocpol, 1994: 332) Narrative explanations need to make explicit use of theory. Explanations in the MLP do this in two complementary ways. First, the global model of the MLP provides a framework that specifies a plot with particular elements and processes. As Pedriana explains: Narratives are not just sequences of events, but are tied together by a central theme. I argue that the contextual framework can serve as a theoretical/explanatory theme that endures throughout the analysis in ways that discipline the narrative. (Pedriana, 2005: 357) The transition pathways articulate more specific plots that guide narrative explanations. Second, the sub-processes in morphogenetic cycles provide a local, internal logic that explains connections between events. These local narrative explanations should explicate: a) How is the game structured? Who are the most important players? What are their cognitive frames, interests, resources? b) What options and possibilities do actors have? Which actions are chosen and why? How do they react to each other? c) What are the broader effects of actions? d) Are structural changes accepted and institutionalized? When global trajectories are stable, these narrative

Grin 3rd.indd 98

T&F Proofs: Not For Distribution

12/16/2009 10:14:25 AM

Refl ections

99

explanations are predictable, less important or can remain superficial. But for turning points, discontinuities, “forks in the road,” accelerations, or twists and turns, the analyst should zoom in and analyze the precise mechanisms in morphogenetic cycles. The narrative causality of the MLP is probabilistic. While the MLP specifies general forms of transitions, specific patterns and speed depend on local event sequences and conjunctures. Narrative causality does not exert deterministic influence over events (Poole et al., 2000). Even when structural alignments raise the probability of transitions, actors may or may not take advantage of windows of opportunity.

I.6.4. CASE STUDIES, PROCESS TRACING AND TYPOLOGICAL THEORY Transitions are processes that unfold over time, involving structural change and non-linearities. Investigations of this kind of phenomenon require a research method that is rich in context and tracks complex developments over time. Case studies are seen to provide such a method because they allow detailed process tracing (study of event sequences), exploration of patterns and testing of rival theories (Yin, 1994; George and Bennett, 2004). Case studies do not immediately deliver explanations. The empirical procedure of process tracing needs to be converted into more theoretical arguments. To that end, George and Bennett (2004: 210–212) distinguish four progressive steps: 1) Detailed narrative (case history) presented in the form of a chronicle. Such a narrative is specific and makes no explicit use of theory. 2) Use of hypotheses and theoretical mechanisms to explain parts of the narrative. 3) Analytic explanation: a historical narrative of a specific case is converted into an analytical explanation by identifying an overall pattern that is couched in explicit theoretical forms. 4) More general explanation about the phenomena of which the case is a case: the particular case study is used to develop theoretical arguments about a general phenomenon. This conversion works towards generalization, thus addressing a possible weakness of case studies (generalizability). The identification of patterns and mechanisms is crucial in this conversion process, as several scholars have noted: And this is where the central challenge lies: moving from a shapeless data spaghetti toward some kind of theoretical understanding that does not betray the richness, dynamism, and complexity of the data. (Langley, 1999: 694) Process methods must convert a heap of confusing data into a synthetic account in which the reader can comprehend all the data in a single act of understanding. This requires the ability to recognize recurrent

Grin 3rd.indd 99

T&F Proofs: Not For Distribution

12/16/2009 10:14:25 AM

100 Transitions to Sustainable Development patterns in event sequences, to establish necessary conditions, and to identify formal and fi nal causation. (Poole et al., 2000: 54) The case study goes beyond the case history in attempting a range of analytical purposes. Firstly, there is a search for patterns in the process and presumably some attempt to compare the shape, character and incidence of this pattern in case A compared with case B. Secondly, there is a quest to fi nd the underlying mechanisms which shape any patterning in the observed processes. . . . The teasing out of these mechanisms . . . represents one of the greatest inductive challenges for process scholars and an area of intellectual challenge. (Pettigrew, 1997) This characterizes our work well. Our cases go beyond historical descriptions, because they identify patterns and mechanisms and a typology of transition pathways. This work can be seen as a typological theory. Such a theory provides a rich and differentiated depiction of a particular phenomenon. As George and Bennett explain: Typological theory identifies both actual and potential conjunctions of variables, or sequences of events and linkages between causes and effects that may recur. In other words, it specifies generalized pathways. . . . A pathway is characterized in terms of variables, often with nominal cut off points distinguishing among types. . . . Such generalized pathways are what is distinctive about typological theory. They are abstract and theoretical even though they are closer to concrete historical explanations than are claims about causal mechanisms. (George and Bennett, 2004: 236) Typological theory is a form of configuration analysis, which acknowledges that the entities being classified are too complex to decompose into variables. They are premised on the assumption that the character of an entity emerges from the entire configuration of its properties and their interrelationships (Poole et al., 2000: 44). The construction of a clean 2x2 matrix is not possible, because too many entities and processes are involved. Instead, multiple variables are combined in configurations that have an inherent logic that binds them together (e.g. archetypes, ideal types).

I.6.5. CODA: THE SOCIOLOGICAL IMAGINATION The MLP has been criticized in recent articles for its heuristic and descriptive nature and a presumed lack of attention for politics and agency (Smith, 2005; Genus and Coles, 2008). The latter criticism, however, is too easy and

Grin 3rd.indd 100

T&F Proofs: Not For Distribution

12/16/2009 10:14:25 AM

Refl ections

101

does not acknowledge the specificity of the transitions research topic. We could easily turn the criticism around and ask what constructivist microstudies have to say about the entire process of long-term socio-technical transitions. To address this well-rehearsed micro-macro dilemma, we characterized our work on the MLP and transition pathways as a global theory that addresses the overall course of long-term change processes, but also acknowledged that this needs to be complemented by local theories which help to analyze how actors navigate, struggle and negotiate on specific alternatives. Furthermore, our new work in this volume (section I.3) discusses in some more detail the role of agency in the MLP, using insights from STS, evolutionary theory, neo-institutional theory and structuration theory. We also want to remark that in our studies that zoom in on the micro-level, agency is clearly visible (e.g. Van Driel and Schot, 2005; Raven, 2005; Geels, 2005b; Geels, 2006a). Our critics are right in pointing at the heuristic value of our work. We take this as a compliment, since we have pushed for a process theory in which theories are used as tools for the development of narrative explanations as explained above. Research of complex phenomena such as transitions cannot be reduced to the straitjacket of a variance theory, and will always contain elements of creative interpretation by the analysts. Almost 50 years ago, C. Wright Mills (1959) complained about a “general malaise of contemporary intellectual life” (p. 19), diagnosing that sociology was divided between “grand theory,” which addressed a “level of thinking so general that its practitioners cannot logically get down to observation” (p. 33), and “abstracted empiricism,” which focused on data collection and statistical correlations. As a middle way between both extremes Mills suggested “sociological imagination.” This sociological imagination is also required for the study of patterns and mechanisms in transitions. Although improved and more rigorous methods have emerged in the last decade, process theory and narrative explanation will always remain crafts (to some extent). They cannot and should not be reduced to technical procedures, with the analyst entering data and results being produced automatically. Process analysis and narrative explanation always involve pattern recognition, which to some degree entails interpretation. One can criticize this as subjective, but also appreciate that it leaves space for creativity and sociological imagination. Especially when addressing a new topic, such as transitions, these aspects are important.

Grin 3rd.indd 101

T&F Proofs: Not For Distribution

12/16/2009 10:14:25 AM