DRAFT: NOT FOR DISTRIBUTION OR CITATION

DRAFT: NOT FOR DISTRIBUTION OR CITATION PARTICIPATORY MONITORING AND EVALUATION: A PROCESS TO SUPPORT LOCAL GOVERNANCE 1. Context Participatory Moni...
Author: Sabina Stewart
0 downloads 0 Views 57KB Size
DRAFT: NOT FOR DISTRIBUTION OR CITATION

PARTICIPATORY MONITORING AND EVALUATION: A PROCESS TO SUPPORT LOCAL GOVERNANCE

1. Context Participatory Monitoring and Evaluation is a mechanism for empowerment that can increase accountability, transparency and the flow of information while improving the efficiency of development projects. Previous PME implementations have been focused on national-level processes, and geared primarily towards improving project outcomes. This concept note proposes that participatory monitoring and evaluation (PME) be considered as a process that also contributes to local governance. This builds on existing theory and practice, as documented below, and extends the theoretical framework that underlies PME. This concept note1 forms the foundation for an eventual guidance note which will contribute to and support pilot projects in Angola, the Gambia, Georgia, Senegal, Sierra Leone and Vietnam. Evidence and case-studies from these projects will form the basis for further analytical work on PME and local governance. 2. Background Local Governance Governance is a broad and nuanced concept, with multiple and competing definitions. The World Bank defines governance as the process and institutions by which authority is exercised. This is further broken down into a three parts: (1) the process by which governments are selected, held accountable, monitored and replaced; (2) the capacity of governments to manage resources efficiently, and to formulate, implement and enforce sound policies and regulations; and (3) the respect for the institutions that govern economic and social interactions.2 The Asian Development Bank varies this slightly, defining four pillars of governance – accountability, transparency, predictability and participation.3 This concept note accepts both of these definitions as valid but is focused on local governance. The World Bank and other development institutions, rooted in a critique of central planning and a belief in the advantages of local competition and information exchange,4 have recognized that improving the quality of local governance is a necessity for effective poverty reduction. From this position, support to decentralization initiatives and social accountability5 issues, with an aim to improve the quality of local governance, has become widespread and integrated across sectors. This paper is therefore focused on the relationship between governments and citizens at what the governance literature often refers to as the periphery, i.e., the local-level. The viability of local government, and its ability to effect governance can often be linked to key principles of inclusion, accountability, transparency, communication and effectiveness.6 In this context, the need for 1

The concept note is the foundation of a workprogram funded by the Trust Fund for Environmentally and Socially Sustainable Development 2 Daniel Kaufmann, Aart Kraay and Pablo Zoido-Labaton, “Governance Matters” World Bank Policy Research paper #2196, World Bank, Washington, D.C., 1999. 3 “Governance in Asia: From Crisis to Opportunity” Asian Development Bank, Manila, 1999 4 Jose Edgardo Campos and Joel S. Hellman, “Governance Gone Local: Does decentralization improve accountability” Chapter 11 in East Asia Decentralizes: Making Local Government Work, World Bank, Washington D.C., 2005 5 Accountability can be defined as the obligation of power-holders to account for or take responsibility for their actions. Social accountability is an approach towards building accountability that relies on civic engagement, i.e. in which it is ordinary citizens and/or civil society organizations who participate directly or indirectly in exacting accountability. See Malena, Forster and Singh, “Social Accountability: An introduction to the Concept and Emerging Practice”, Social Development Paper No. 76, World Bank, December 2004 6 The “White Paper on European Governance” European Commission, Brussels, July 2001, for example, defines openness, participation, accountability, effectiveness and coherence as essential to effective local government.

Page 1

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION participation has been continually re-affirmed in development theory7 and, more recently, in practice, where its value as a basis for local governance in decentralized contexts has become more explicit8. In particular, the Bank has drawn on lessons from the East Asian decentralization experience as an evidentiary basis for the importance of participation.9 The rationale behind this is rooted in an understanding of how local governments exert governance. The effective functionality of local governments can be linked to three factors related directly to participation:10 ¾ Autonomy: In a decentralized context, local government must demonstrate its capacity to make decisions and respond to local realities. In order to do so, it needs participation from citizens to be able to set mutually agreeable social and economic objectives. This can, in part, be viewed as placing a bottom-up constraint on the autonomy of the local institution, recognizing its obligation to serve citizens. ¾ Legitimacy: In order to effectively govern, a local government must gain credibility and trust from citizens. Legitimacy is built on transparency and accountability, which both require substantive participation.11 ¾ Service Delivery: The essence of contemporary governance is in the ability to assure the provision of services.12 Citizen participation not only allows for more effective planning, but through monitoring can unlock the potential for feedback loops and continuous improvement.13 Participatory Monitoring and Evaluation Participatory monitoring in support of local governance defines a process where participants – citizens, governments and service providers – at the local-level track progress toward and achievement of self-selected results, within and beyond a project context.

7

See for example, Harry Blair, “Participation and Accountability at the Periphery: Democratic Local Governance in Six Countries.” World Development, Vol. 28, No 1, pp. 21-39 (2000), and Andrew Parker and Rodrigo Serrano, “Promoting Good Local Governance through Social Funds and Decentralization” World Bank, Washington D.C., 2000 8 See in particular, Gaventa and Valderrama, “Participation, Citizenship and Local Governance”, Background note for “Strengthening participation in Local Governance”, IDS, June 1999 or Lingyah and MacGillivray, “Working from Below: Techniques fto Strengthen Local Governance in India, New Economics Foundation, London 1999 9 See in particular Susan Wong and Scott Guggenheim, “Community Driven Development: Decentralization’s Accountability Challenge.” Chapter 12 in East Asia Decentralizes: Making Local Government Work, World Bank, Washington D.C., 2005 10 For theoretical underpinnings, see Robert D. Putnam, Making Democracy Work, Princeton University Press, Princeton, 1994 or Kaufmann, Kraay and Mazstruzzi, “Governance Matters IV: Governance Indicators for 1996-2004” World Bank Institute, Washington D.C., 2005. This construct is derived from Louis Helling, “Assessing and monitoring local governance in Africa.” Tunis Workshop on Local Governance and Poverty Alleviation in Africa, 2005. but is congruent with Parker and Serrano (supra note 2) and James Manor, The Political Economy of Democratic Decentralization, World Bank, Washington D.C., 2005 11 For a more detailed discussion, see, for example, Vivi Alatas, Lant Pritchett and Anna Wetterberg, “Voice Lessons: Local Government Organizations, Social Organizations, and the Quality of Local Governance” Policy Research Working Paper 2981, World Bank, Washington, D.C., March 2003. 12 See, for example, Daniel Treisman, “Decentralization and the Quality of Government.” Paper presented at the IMF conference on Fiscal Decentralization. Treisman defines “quality of government…as the provision of public goods and services that the public demands at minimum cost in taxation and regulatory burden.” (p.1). See also Alatas et al in supra note 6. 13 For further discussion, see Ahmad, Devarajan, Khemani and Shah, “Decentralisation and Service Delivery” World Bank Policy Research Working Paper 3603, May 2005

Page 2

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION The importance of citizen participation and leadership in monitoring is widely recognized in development theory. In practice, efforts to increase participation have often been focused on social accountability movements that seek to increase the engagement between civil society and institutions at a national level and on participatory monitoring and evaluation (PME) at the local or community level. PME is not new – it is a process that has existed in various forms since the 1970’s – but the late 1990’s saw the first movement to coalesce this into a coordinated body of thought. PME evolved out of a combination of dissatisfaction with conventional M&E and a general movement towards participation in the development community.14 PME was conceived as an adaptive process, based on negotiation between parties as to what was monitored, and geared to produce learning that led to corrective actions.15 In practice, however, a wide variety of approaches have been co-opted under the rubric of PME and there is no comprehensive catalog of effective strategies. Communication between organizations working on PME to document and exchange lessons has been infrequent. The distinction between conventional M&E and PME can be summarized as follows16: Conventional Who External experts or senior managers Beneficiary Role Provide information only Questionnaire surveys, by outside evaluators,

Simple, qualitative or quantitative methods, analyzed by beneficiaries

Externally-defined, mainly quantitative indicators

Internally-defined indicators, including more qualitative judgements

To make project and staff accountable to funding

To empower stake-holders to take corrective action

How distanced from project Indicators Used

Participatory Local people, project staff, managers, and other stakeholders, often helped by a Design and adapt the methodology, collect and analyse data, share findings and link

Why agency

Method Pre-determined

Adaptive

Participatory Monitoring and Evaluation in the World Bank Within the Bank PME has tended to be realized as an upstream centered-process, providing information that feeds policy decisions or project staff and strengthened by seeking beneficiary input. PME activities have infrequently been expected to produce their own outcomes or impacts, but rather to feed into the project cycle and inform managers and implementation units. Internal analysis of PME efforts have observed, in addition to this upstream focus, a resistance to implementing participatory structures from implementing agencies, based on a lack of understanding of PME’s tangible benefits. This, in turn has been related to a general perception of some Bank Managers and Task Teams that M&E systems were a project requirement to be accommodated, rather than worthy of careful design17. 14

For a detailed discussion of the history of PME, see Marisol Estrella, “Learning from Change” in “Learning from Change: Issues and experiences in participatory monitoring and evaluation” Ed. Marisoll Estrella, International Development Research Center, Ottowa, Canada, 2000. 15 See “Participatory Monitoring and Evaluation” IDS Policy Briefing, Issue 12, November 1998 16 This table draws directly from a table in “Participatory Monitoring and Evaluation: Learning from Change.” IDS Policy Briefing 12, available at http://www.ids.ac.uk/ids/bookshop/briefs/brief12.html 17 Shankar Naryanan, “Integrating Learning in the Monitoring and Evaluation of CDD projects- a Guidebook” pp. 6-7, World Bank, Washington D.C., 2005

Page 3

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION

This experience differs slightly from non-Bank experience, which although similarly handicapped by the lack of clarity inherent in a nascent movement, has had more demonstrable success at the local level over the last few years. Experiences in Ecuador18 and Colombia19, for example, have demonstrated the ability of PME to strengthen participation, increase accountability, improve community management and alter power dynamics. Although it is likely that some Bank projects have had similar success, there has been little investment in documenting these experiences. Within the development community there remains a tendency to consider traditional M & E, when supplanted with community consultation, as sufficiently participatory.20 Recent evaluations of PME efforts across the board have supported this, noting that as a result of these limited implementations and conceptual confusion, beneficiary involvement is less substantive than desired, and stakeholder alienation, rather than inclusion, is the norm.21 Participatory Monitoring and Evaluation – a new focus Initial evidence from field experience in PME has stimulated discussion around governance impacts and PME’s potential in local development and local governance contexts. A case study of Ecuador22, for example, notes that, with investment in training, good governance of local authorities could be enhanced. As previously noted, the value of participation in governance literature is already well established. The link between PME and local governance – on an operational or a theoretical level – has not been clearly made to date. PME has primarily been implemented as contributing to ‘upward’ accountability, where national institutions or civil society movements hold governments accountable. Projects that have designed their PME to support ‘downward’ accountability, thereby recognizing PME as a mechanism for demand-driven governance, are very early-stage.23 This argument, however, that PME is either upwardly or downwardly focused is limited. The participatory monitoring process can extend beyond the project context. PME can be a mechanism that contributes directly to local governance through creating horizontal relationships between citizens and civil society, local governments and service providers. This conceptual extension requires PME to expand into an evaluative process led by and for beneficiaries, which seeks to extend beyond the project context and has its own specific and detailed governance impact. This note therefore re-affirms the principles of PME – of participation and accountability, learning and adaptability – and asserts that PME can become a local-level process, driven by and focused on horizontal support to local governance. This is an explicitly expansive concept of PME that can still be used in concert with traditional monitoring and evaluation (M&E), but focuses on the value and impact of the participatory process on broader governance outcomes. While M&E is generally a top-down, extractive procedure conducted by outside agents, PME incorporates ongoing monitoring by local actors 18 Victor Torres, “Monitoring Local Development With Communities: The SISDEL approach in Ecuador” in Ed. Estrella, supra note 11 19 Ruben Alzate, “Monitoring and Evaluating Local Development through Community Participation: The Experience of Indigenous Cabildos of Northern Cauca, Colombia” in Ed. Estrella, supra note 12 20 Supra note 14 21 Marisol Estrella and John Gaventa, “Who Counts Reality? Participatory Monitoring and Evaluation: A Literature Review” p.15, Institute of Development Studies, Sussex, England, 1998 22 Supra note 15 23 Gaventa and Blauert “Learning to Change by Learning from Change: Going to Scale with Participatory Monitoring and Evaluation” in Ed. Estrella, supra note 12

Page 4

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION within and beyond project contexts. Ex-post project evaluation can and should remain a part of PME processes – recognizing its evaluative capacities, but the focus remains on the participatory process, its monitoring structures and its governance impacts. PME as proposed should continue to provide valuable input to project managers, but that cannot remain its primary purpose. This evolution fits into the broader, society-level dialogue on social accountability. PME as conceived here should inform and support World Bank and national strategies, and integrates with social accountability programming, but is focused on entry points and horizontal relationships at the local level. 3. Linking PME and Governance This concept note proposes an iteration of PME which integrates with the broad push towards decentralization and governance. This movement provides both a demand and an opportunity space for PME at the local level to assist, transform and secure the relationships between local governments, citizens and service-providers – the parties to the modern social contract. The three factors for participation in governance, as identified earlier, can be linked to PME as follows: ¾ Autonomy: As previously explained, the need to set mutually agreeable social and economic objectives contributes to the perception of autonomy. Participatory monitoring mechanisms create fora where these expectations and their achievement can be discussed openly. As experience and understanding of abilities and obligations are built, these expectations will become more realistic, leading to improved relationships. The opportunity to draw on citizen and governmental resources to assist with the inevitable challenges of implementation further increases the ability to respond to challenges at the local level, also enhancing the sense of autonomy and self-responsibility. ¾ Legitimacy: The importance of transparency and accountability, as referred earlier, is well documented elsewhere. PME is inherently a transparency-building process; as envisaged here, citizens and local government are working together, rather than monitoring one another, building transparency and insight into operations of government and the needs of citizens. Transparency and accountability also assist in reducing the risks of elite capture and clientelism in decentralized contexts. Participatory monitoring can therefore provide an important local component of broad movements towards greater social accountability as it helps to build an engaged civil society. ¾ Service Delivery: As noted above, PME can create feedback loops that lead to improved service delivery – and thus development outcomes. In particular, PME can create fora where techniques for success and failure can be observed openly – and exchanged across communities at the local level. The ability to respond at the local level is also expected to improve the quality of service delivery through creating room for innovation. It must be stressed that participatory monitoring is only a constituent element of building effective local governance; it is, however, a factor of some importance, alongside broader issues of social accountability and civil society engagement, and the administrative, fiscal and political challenges of decentralization amongst others.

Page 5

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION This argument for participation, and hence PME’s potential for assisting local governance has, to date, mostly remained theoretical.24 As noted above, the Bank study of the East Asian experience of decentralization has begun to strengthen the evidentiary basis. This review of experience, centered on Indonesia and the Philippines, showed evidence of a risk of clientilism and elite capture, and the importance of strong local institutions to minimize this risk. In this vein, supporting local institutions through bottom up participation and developing a framework for encouraging regional-level cooperation have been prioritized. PME structures, as discussed here, link the theoretical recommendations - through “the construction of new relationships between ordinary people and the institutions – especially those of government – which affect their lives”25 - with the empirical urge to support local institutions. The link between PME and local governance is therefore enshrined in the ability of PME to stimulate effective and sustainable participation from individuals within all parties – governments, citizens and service providers. Such an activity creates a governance dividend by securing and encouraging mechanisms that support demands from citizens, governments and service providers for (i) inclusion; (ii) accountability; (iii) transparency; and (iv) improved service delivery. As argued above, participatory monitoring can also provide a framework to elicit institutional and behavioral change within all actors, driven by beneficiary and stakeholder-leadership. The participation of all actors in PME also provides an important self-correcting feature, allowing actors to find their own level of expectation and obligations, thereby minimizing the perceived threat to local governments and supporting sustainability. It is important to stress that PME, as defined here, is sector-neutral within project contexts. PME activities are flexible enough to be integrated into sectoral, community-based and local government programming, and in post-conflict or other operational environments. 4. Operational Principles. As stated, there is limited experience with PME, particularly within the Bank, which lags behind other development actors. An analysis of five case studies, Haiti, Mexico, Kenya, Tanzania, and India, and of literature in the field of participatory monitoring26 provides insight into principles for future operations: PME should incorporate and build capacity among all stakeholders – local government, citizens and service providers - as active participants, who can analyze, reflect and take action. In order to deliver real governance impacts at the local level, beneficiaries need to actively engage in developing goals and priorities for monitoring, and implementing and supervising the process in order to create learning. By supporting participation among all parties, PME promotes selfreliance in decision making and problem solving, empowering as the ability to change is demonstrated and reinforced.

24

For a more detailed discussion, see John Gaventa, “Towards Participatory Local Governance: Six propositions for discussion.” Institute of Development Studies, Sussex, 2002. Many of Gaventa’s proposals can be characterized as built around increasing participation in local mechanisms that create a direct connection between citizens and institutions. 25 Gaventa (supra note 20), p.1. 26 Estrella and Gaventa, (supra note 4), is a comprehensive literature review of participatory monitoring conducted by the Institute of Development Studies. The case studies previously referred were an internal analysis of SDV/CDD operations by Bank consultants; further information on these is available from the CDD anchor.

Page 6

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION PME processes must be representative, and must produce action and change.. The governance outcomes described here are contingent on actors being able to effect change. For PME to work, processes that reflect a belief in the ability of communities, local governments and service providers to jointly provide better development outcomes are essential. PME implementations must incorporate structures and fora for information sharing among communities. The value of using PME structures to share information has been noted above. Lessons learned show that institutional learning and results are enhanced when stakeholders with similar interests share information, and compare strategies for success and failure, through processes of benchmarking. 5. Expected Benefits PME activities, as defined above, would support local governance as follows: Inclusion: Defining inclusion as increasing the access of diverse groups to assets and opportunities, PME structures would increase inclusion by integrating beneficiaries and stakeholders, often under true beneficiary leadership, into the management process. Local government units, citizen/interest groups and service providers would also be integrated, helping to bridge existing divides. Issues of marginalization – in terms of gender, disability and other groups – can also be addressed through inclusive PME structures. Accountability: Accountability refers to the responsibility of all actors for the appropriate use of resources. Linked to the notion of autonomy, accountability constrains the extent to which parties can deviate from their obligations. PME processes support accountability by defining clear expectations for all parties, encouraging local government units, communities, and service providers to adhere to these standards, and providing fora for post-facto feedback. Again, in this context, PME indicators and structures could link to broader social accountability movements. Transparency: Transparency is often defined, in this context, as the ability of citizens to access true information about decisions. As community-created and driven open structures, entities and institutions engaged in PME are inherently transparent, providing the ability to observe essential information. The value of informing the public has been well-documented elsewhere.27 Communication and information exchange: The value of information is enhanced when that information can be used by communities.28 At a basic level, mutual expectations exchanged in a public forum breed obligations. Beyond this, PME can help parties to live up to these obligations by jointly observing problems and identifying strategies for success. These strategies – and project challenges – can be shared at regional/provincial levels, enabling benchmarking and enhancing cross-community cooperation. Results from these processes also allow communities, governments and service provides to internally identify sources of positive change, respond to their own definitions of success, and use this to improve service delivery in the future – transcending the original project context. Management benefits: PME is a continuous process of joint learning and decision making by citizens, governments, and service providers acting in concert. If implemented well, it drives the setting of realistic, locally–derived goals, and monitors progress toward their achievement. This 27

See, for example, “Making Services Work for Poor People” World Development Report 2004, Overview p.1 and 7 This separation – between ‘observing’ information and using it remains important. In low-capacity contexts, simply ensuring information is available may be an important first step with its own impacts. 28

Page 7

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION learning empowers local-level action to improve operations and effectiveness allowing PME to support management goals and operate as a results-based tool. Therefore, there is considerable scope for use of PME in support of the decentralized governance agenda; the benefits of PME extend beyond governance towards broader issues of accountability and improved service delivery as development objectives. 6. Next Steps The scope for participatory monitoring to provide essential support to the drive for local governance is clear. Within the Bank there has been much rhetoric about PME and the value of participation, but a lack of experience and evidence to understand its impacts and benefits. Demand has been strong from World Bank Staff and from client countries for support and action. To date, the response has been hampered by (i) a lack of conceptual clarity as to what PME is, and what its benefits can be; (ii) inadequate guidelines as to what constitutes good practice within PME; and (iii) insufficient funding for technical assistance to TTL’s and clients to assist in implementing good PME. At this time, the number of projects that have well-documented governance impacts is still limited, and much further work is necessary to establish a widespread understanding on effective practices and techniques. This note forms part of a TFESSD activity that seeks to address these key roadblocks to the implementation of governance-oriented PME. Pilot programs, described below, will provide the operational experience and feedback required to support broader analytical work that will inform future operations. Building on conceptual and operational principles defined in this note, further and more detailed knowledge will be disseminated and discussed over the next two years. The final output of this activity will be a compendium of case-studies that will inform and support this movement within the Bank and in the broader development community.

Short Term (next 6 months) Subsequent to the approval of this concept note, a BBL and or video-conferences with relevant Staff will disseminate these ideas. External consultants, with the appropriate academic and practitioner credentials, will develop a more in-depth Guidance Note building and developing on the ideas explored herein. A post-conflict concept paper will also be commissioned to explore the specific dynamics of PME in post-conflict contexts. This concept paper also aims to serve as a foundation for further dialogue with country and sector staff and TTL’s to determine the need for, and possible uses of PME mechanisms within their projects.

Medium and Long-Term (6 months – 2 years) This concept is being piloted either through additions to existing implementations and projects, or through support to projects, as follows:

Page 8

26 July 2005

DRAFT: NOT FOR DISTRIBUTION OR CITATION Country

Project

Unit Management

Sector/Theme

S/T

Status

Angola

Social Action Fund (II/III)

HD (Social Protection)

Education

Conflict

Existing

Sierra Leone

Institutional Reform and Capacity PREM (Public Sector) Building

Decentralization

Conflict

Existing

Senegal

Cassamances Emergency Reconstruction

Health

Conflict

Existing

The Gambia

Community Development Project ESSD (ARD)

Decentralization

Health

New

Vietnam

Poor Communes Infrastructure and Livelihood

ESSD (ARD)

Decentralization

New

Georgia

Improving Livelihood Security in the Kolkheti Lowlands

ESSD (SDV)

ENV

New

INF (Transport)

As appropriate, case-studies, BBL’s and video-conferences will be held to discuss and disseminate experiences from pilots. Case studies and evidence from these pilot projects will form a part of broader analytical work to complete this activity. A compendium of case-studies will be released at the end of two-years to Requests for information on PME in these projects or general questions should be directed to the authors, Daniel Murphy ([email protected]) and Rahul Chandran ([email protected]

Page 9

26 July 2005