GLOBAL PROGRAMME ON GOVERNANCE ASSESSMENT
LESSONS LEARNED United Nations Development Programme UNDP Oslo Governance Centre
������������������� ���������������������
���������������
Authors: Paul van Hoof and Ingvild Oia Project Coordinator: Ingvild Oia Design: QUO Bangkok Copy editors: Gert Danielsen and Tom Woodhatch UNDP Disclaimer: The views expressed in this publication are the authors’ and do not necessarily represent those of the United Nations, including UNDP, or its Member States. For further information please contact: United Nations Development Programme Bureau for Development Policy Democratic Governance Group 304 East 45th Street, 10th Fl. New York, NY 10017 Oslo Governance Centre Inkognitogata 37, 0256 Oslo, Norway www.undp.org/governance www.undp.org/oslocentre www.gaportal.org Copyright © 2012 by the United Nations Development Programme. All rights reserved.
CONTENTS Acknowledgements Introduction What is a governance assessment project?
5 6 7
1. How can think-tanks use governance assessment to advocate for reform? Chile Mexico Discussions and reflections
8 8 10 11
2. How can governance assessment widen democratic space? China Kazakhstan Viet Nam Discussions and reflections
12 12 14 15 16
3. How can governance assessments strengthen the role of parliaments? Nicaragua Paraguay Montenegro Discussions and reflections
17 17 18 19 19
4. How can governance indicators be ‘inclusive’? FYR Macedonia Tajikistan Mongolia Discussions and reflections
20 20 21 22 23
5. How can local governance assessments make institutions more responsive? Nigeria Egypt Discussions and reflections
24 24 25 25
6. How can assessments strengthen ownership of national development plans? Senegal Malawi Discussions and reflections
26 26 26 27
7. How can assessments strengthen policy planning? Bhutan Indonesia Discussions and reflections
28 28 28 29
Lessons from the ‘Oslo Principles of Democratic Governance Assessments’ Scoring exercise Overall conclusions and lessons Annex: Schedule lessons learned workshop 6 October 2011
30 31 33
ACKNOWLEDGEMENTS This report is published by the Global Programme on Governance Assessments (GAP). GAP is hosted by the Oslo Governance Centre of the Democratic Governance Group, which is attached to UNDP’s Bureau of Development Policy. GAP is grateful to the authors, Paul van Hoof and Ingvild Oia. In addition to writing this report, Paul facilitated the Lessons Learned Workshop (6 October 2011, Oslo), which informs this report. GAP is also grateful to all those who presented and took part in the Lessons Learned Workshop. The agenda with a list of names is provided in Annex I. A special thank you goes to the reviewers of UNDP governance assessment projects who have contributed to the evidence-base of this lessons learned report. This includes in alphabetical order: Helen Addison, Harry Garnett, Ian Hopwood, Gerardo Munck and Andrew Nickson. Finally, we would like to thank Asmara Achcar, Vidar Ellingsen, Danae Issa, Nina Kolybashkina, Marie Laberge, Joachim Nahem, Sujala Pant, John Samuel and Christopher Wilson. Ingvild Oia coordinated the project.
5
INTRODUCTION On 6 October 2011, for the first time since its inception, UNDP brought together practitioners working on projects supported by the Global Programme on Governance Assessments from around the world. The objective of this one-day workshop was to harness the lessons learned from their experiences. This report is a result of that workshop, as well as of the seven reviews and evaluations of UNDP’s governance assessments projects that have been conducted to date. The aim is that these lessons will be useful to other national practitioners and UNDP country staff who wish to embark on a governance assessment. Since 2008, the Global Programme has supported country-led governance assessments to enhance democratic governance, based on the principles of national ownership, capacity development and alignment. Two and a half years into programming, an increasing number of UNDP staff and national counterparts have accumulated significant knowledge. This knowledge is both substantive on the issues of governance indicators, and practical on how to manage a process of country-led governance assessments. Through this workshop, as well as the wave of reviews and evaluations that took place in 2011, the Global Programme has tapped into this reservoir of knowledge and distilled some of the lessons learned. The purpose of this report is to enhance peer-to-peer learning, thereby strengthening project design and management. It does not seek to assess or judge the performance of individual projects or of the Global Programme. The workshop served to create a space where lessons of all kinds could be discussed constructively. To strengthen the quality of the workshop, mid-term reviews (from projects in Chile and Senegal) as well as evaluations of past projects (in, for example, China, Montenegro and Paraguay) were prepared and presented by independent reviewers. The lessons in this report draw from the reflections of this group of practitioners: reviewers, UNDP staff and national counterparts. Lessons were also fed into the Programme’s mid-term review.1 A leading principle of the Governance Assessment Programme has been the principle of national ownership of governance assessments. Adherence to this principle resulted in a broad diversity of governance assessments approaches supported by the Programme, both in the methodologies used and in what actually has been assessed. Based on the specific context and local demand, some projects focus more on government performance assessment (e.g., Kazakhstan and Viet Nam), others more on the quality of democracy in general (e.g., Chile), on specific aspects of governance (e.g., corruption in Tajikistan; participation and transparency in Viet Nam) or on specific service delivery sectors (e.g., FYR Macedonia, Viet Nam and Egypt). In addition, the level of assessments also differs, from the national level (e.g., Paraguay, Chile and Senegal) to the regional level (e.g., Mexico) and down to the local level (e.g., Nigeria and Viet Nam). To make sense of such diversity, this report has grouped projects around seven thematic topics that were selected to capture some of the innovative aspects of country-led governance assessments, with a view to reflect the more strategic questions that the projects have to deal with.
1
6
Please see the Global Programme on Governance Assessments: Mid-term review, 2011
The topics are 1. How can think-tanks use governance assessments to advocate for reform? 2. How can governance assessments widen democratic space? 3. How can governance assessments strengthen the role of parliaments? 4. How can governance indicators be ‘inclusive’? 5. How can local governance assessments make institutions more responsive? 6. How can assessments strengthen ownership of national development plans? 7. How can assessments strengthen planning? The reader should be aware that most projects offer more lessons than just those relating to the themes under which they have been grouped, and that these have also been included. Sometimes, lessons from one project may even seem to contradict those from another. There has been no attempt to clear this up; rather, the reader is encouraged to use their own judgment and understanding of context-sensitivity as to when and how lessons can be applied to other projects. More information on all the projects presented here can also be found at the GAPortal, at www.gaportal.org.
What is a governance assessment project? A governance assessment project seeks to strengthen governance reform through a multi-stakeholder process. Steps include the identification of stakeholders, consultations on priorities to be measured, selection of indicators and finally the use of the data and results to inform decision-making and monitor progress. Inclusive participation and consensus building are key principles for securing a legitimate process. When conducted successfully, a nationally-owned governance assessment may serve to strengthen democratic governance at the country level. It may provide a critical accountability mechanism for government and for citizens to engage on governance issues and voice their opinions. Data produced by the assessment may aim to reflect and address citizens’ concerns. Democratic governance assessments may also offer a superior evidence base for national decision-making. Benchmarking progress provides a record of reference for planning, monitoring and evaluation.
7
1. How can think-tanks use governance assessment to advocate for reform? Producing data is just one in a series of steps in the process of conducting a governance assessment. It is often the least problematic. For assessments to have an impact on governance performance, the data need to be used by key stakeholders who are in position to inform policies. The road to policy influence varies with the kind of actors who lead the assessment. Where the assessment is led by actors external to the government, activities such as lobbying, advocacy and outreach become important in strengthening uptake of data in policy processes. In this section, lessons from assessments led by think-tanks in Chile and Mexico will be presented.
Chile 2 Led by a consortium of four think-tanks, the Chilean democracy audit, named “Governance Assessment: How Democratic is Democracy in Chile?”, aims to foster political debate and advance policy reform in political parties, elections and democratic consolidation. One of its most distinguishing features is the makeup of the consortium, which ensures that the local partners span practically the entire political spectrum. Two think tanks (CIEPLAN and Proyectamérica) are closely linked to the centre-left coalition and the other two (CEP and Libertad & Desarrollo) have strong connections with Chile’s business class and the right-wing coalition. Because of this, the project enjoys strong links with the political establishment and has substantial public visibility.
Workshop with experts and political actors as part of the consultation process
One of the most immediate ways in which the democracy audit has had an impact is through its effect on the actors that are part of the project itself. In this regard, the preparation of the democracy audit can be seen as an exercise in confidence building. The release of data from the public opinion survey in November 2010 drew extensive media attention, contributing to the project’s visibility. While the democracy audit takes the International IDEA democracy assessment framework3 as a key point of reference, it is unique in the Chilean context. The project seems to be propitiously timed and aligned with a growing political momentum that demands reform.
Lessons learned On process
• The process can be the most important product. The development of trust among think-tanks with different political affiliations and political ideologies is one of the main results of the democracy audit.
2 3
8
For more information about the case from Chile, please also see “Governance Assessment: How Democratic is Democracy in Chile? Mid-term review”, by Gerardo Munck, 2011, which this text draws on. For more on International IDEA State of Democracy Assessment, please see http://www.idea.int/sod/.
• The political weight of local partners is a key factor in making a democracy audit a success. In the •
•
Chilean experience, the choice of partners and the precedence of their joint work is probably the most notable strength. The national partners should be fully involved in every stage of the process, especially in the write-up. Involvement of party leaders may be beneficial. Deliberate involvement of party leaders in discussions on the democracy audit may increase the sense of ownership of the audit on the part of the political establishment. The Chilean experience also shows that there is an important role for UNDP in assisting countries conduct democracy audits that address the big political issues of the day, and democracy generally.
On indicators and data
• The International IDEA democracy assessment framework has costs and benefits. The Chilean experience
•
•
•
shows that it is possible to start with a ready-made framework and adapt it to reflect national priorities and contexts. But because the framework is broad and essentially organized as a long list, the write-up of the audit tends to be static. It does not lend itself to the sort of analysis needed for recommendations on how to reform. The use of survey data to capture public opinion can be a very valuable contribution. Although some data can be controversial, there is a sense among politicians that survey data convey the voice of their constituents. These data have instant appeal and much legitimacy, and such surveys are one way to effectively convince doubters of the value of a democracy audit. It is possible for a democracy audit to address the fundamental political features of the political system, that is, the broad framework used to process demands from society and make legally binding decisions. In this sense, the Chilean model stands in contrast to governance assessments that focus largely on public policies and the delivery of services. The use of comparative data could be used beneficially in the democracy audit, both because a country–one’s country–is usually understood in comparison with others and because the case for recommendations can be bolstered by using evidence that shows what happened when other countries carried out similar reforms.
On communication
• Efforts should be made to move from diagnosis to recommendations for political reforms and public • •
policies, something that necessarily has to flow from discussions among the local partners. This may increase the relevance and impact of an audit. A democracy audit report should avoid rigid organization in the way that data and analyses are presented. It should be attentive to the links between different issues areas. A more concerted development of a pedagogical strategy aimed at the citizenry for awareness-raising may be valuable.
For more information, please see gaportal.org/undp-supported/chile.
9
Mexico
Launch of findings from the study México Estatal in Oaxaca State
Led by a Mexican think tank, Centro de Investigación y Docencia Económicas (CIDE), the governance assessment project “Quality of government and accountability at the federal level” in Mexico aims to establish a database of state performance indicators to strengthen accountability and governance at the state level. In just three years, the project has managed to gather information on quality of governance, accountability and four policy areas in each of the 32 states, to produce detailed assessments in two states (with three more in process), and made the findings available in a public database. It has shed light on the strengths and weaknesses of accountability structures at the sub-national level.
At the time of writing, the project is moving beyond data collection and analysis by involving state governments, think-tanks and civil society organizations to collectively define strategies for improving democratic governance at the local level. It is progressing from monitoring to target setting and strategy development. Some state governments have started using the data for planning purposes and intend to repeat the exercise using local funds within two to three years.
Lessons learned On process
• It is important to involve relevant stakeholders from the outset. The project included leaders of civil society organisations and think-tanks, academics and representatives from government institutions in its advisory board. The Mexican experience also benefitted from the creation of synergies with existing accountability networks, engaging with thematic experts and regional partners, interacting with the National Statistical Institute and the Freedom of Information Commission, and commitment from state governors. • It is important to be mindful of political cycles. While elections may at times delay projects, they may also create opportune windows if the timing and the buy-in are right. • It may be strategic to align indicators with the state’s own planning processes. In Mexico, the use of the indicators in state government’s planning processes will start in 2012. • Follow-up activities may be needed to secure institutionalization, which is required for gains to fully materialize. On indicators and data
• It is imperative that the assessment methodology is robust and results in high quality data. In Mexico, •
10
this enhanced legitimacy of the project set the stage for better political discussions, and helped to avoid disputes about the accuracy of information. It should not be assumed that all administrative data and information exist. Data collected by the state on registrations, service delivery, case-processing and/or other issues may be relevant data sources for scoring on the indicators that have been selected as part of the governance assessment. However, even when the state is transparent, administrative data may sometimes not be collected systematically, and therefore may simply not be available.
On communication
• Timing the release of results and reports in relation to the regular political cycles can enhance effectiveness in both uptake of data and in spurring political debate. • It may also be beneficial to reach a broader audience. In Mexico, outreach activities are targeting civil society organizations and journalists within the states to increase the uptake and use of the indicators. It is expected that this will strengthen social accountability.
Discussion and reflections If think-tanks are to use governance assessments to advocate for reform, it seems imperative that much time and additional supportive activities are needed to ensure that the data are actively used, while the potential impact may easily be affected by unplanned political events beyond the control of the project. In general, governance assessments that involve a broader spectrum of stakeholders (including non-state actors) from the inception phase have a better chance of achieving impact. The likelihood of obtaining a structural improvement in governance behaviour of government (both legislative and executive) without continuous external pressure is negligible. Introducing more players with different – or even opposing – agendas creates additional challenges to coordination and reaching consensus, but will raise the quality of the exercise and the likelihood of the start of a constructive dialogue. For more information, please see gaportal.org/undp-supported/mexico.
11
2. How can governance assessments widen democratic space? Promoting democratic governance often requires a vibrant and dynamic democratic space. A public space is defined as the arena that exists between the state and the individual in which people interact to hold the state accountable, shape public debate, participate in politics and express their needs and opinions. Challenges in countries where democratic space is constrained may relate to the lack of political opportunity to engage directly with civil society organizations and/or a weak civil society. In some cases, governance assessments may provide a much needed foot in the door, and a stepping stone for broadening participation. Lessons from governance assessments in China, Kazakhstan and Viet Nam will be presented.
China 4 Led by a Chinese think-tank, the China Centre for Comparative Politics and Economics (CCCPE), the project “Support to the development of a governance assessment framework in China” aims to build a broad consensus on governance assessment. It also seeks to establish a governance assessment framework in accordance with the realities of political development in China, based on the latest international research and experience on governance assessments. China has recently embarked on a development policy that pays more attention to aspects of inclusion and equality and on improved government service delivery. The shift in focus from a growth-
“International seminar on governance and governance assessments: China and the World”, Beijing, 26 May 2011
oriented model has created a government demand for a better understanding of governance processes, which contributed to the development of a country-specific governance assessment framework. There is also rising demand from local governments for governance assessment, and increased social pressure from citizens for channels of inclusion and accountability.cadre at the central party school. Initial results look promising as governance and governance assessments for strengthening democratic processes are introduced to the political agenda. A widening community of like-minded national experts is emerging and governance has been introduced as a subject in the standard training programme of the Communist Party cadre at the central party school.
Lessons learned On process
• It is crucial to take full account of the national political and social contexts when setting expectations for a governance assessment project such as this. In China, challenges include questions on how to
4
12
For more information about this case, please also see “China Governance Assessment”, DGTTF Lessons Learned Series, by Harry Garnett, 2011, which this text also draws on.
broaden the participation of ordinary people when few formal channels for participation exist that are not controlled by the Communist Party. Furthermore, civil society formation and its activities are limited by government control. As a result, there is a lack of both civil society capacity and of a favourable institutional environment for participation. At the same time, recent years have seen new opportunities and approaches for public participation via the internet. • Broad participation in the assessment is particularly important in a politically sensitive environment. Support for something new in governance is more likely to succeed if promoted from many directions: academia, government (local and central), NGOs, and senior party officials (local and national). • Aligning the assessment with national patterns of reform processes may be an effective approach. In China, the start of governance reform lies in academia. Once academia reaches consensus, and the Communist Party approves, reform tends to move bottom-up through local government piloting before rolling out the piloting experience in the whole country. Because of this, the first part of the project sought to secure a consensus among the think-tanks in Beijing. In its second phase, the project will conduct local government pilots. If successful, in its third phase the project will be rolled out country-wide. On indicators and data
• Tools need to be designed strategically. The project recognized that a comprehensive governance framework had not only never been prepared before in China, but its very preparation would be highly politically sensitive. For this reason, it was important that the project did not go beyond relatively general definitions of the elements of good governance, and that it did not define the actual indicators before achieving more consensus on principles and approaches, which is appropriate within a society that believes in a gradual approach to social transformation. On communication
• It may be useful to create a community of like-minded national experts on country-led governance assessment using existing platforms and institutions as well as by creating new ones. In China, discussion between officials and common consensus on governance was achieved through the Central Party School. In addition, a new discussion platform among different stakeholders was launched by establishing a new journal, China Governance Review. • Revising assessment frameworks to reflect new political priorities may be worthwhile. The new concept of ‘social governance’, introduced in China in the current five-year plan, provided an opportunity to elaborate more on what this should include in the governance assessment framework, which will require some revisions. Responding to endogenous concepts may also be advantageous, such as ‘harmonious society’, (Xiaokang society) and ‘innovative state’. • It is important to establish a good model of advocacy. In China, this was achieved through workshops that stimulated communication among a group of key stakeholders covering a broad spectrum of interests: academics, government officials, NGOs and CSOs, and the media. Communication with senior CCP cadres were particularly important and was achieved face-to-face, through training, and with the publication of the framework in the CCP newsletter. Numerous academic articles on governance and governance assessment were circulated, and a book summarized the projects experiences as well as the framework itself. The media were well used in promoting the framework. For more information, please see gaportal.org/undp-supported/china.
13
Kazakhstan
Side discussion at an international conference in Astana
Led by the Administration of the President, Kazakhstan has embarked on a public administration assessment to strengthen its performance-based management. It focuses on six crosscutting themes at all levels of government, which includes six ministries, as well as provincial government bodies (oblasts). The project intends to involve NGOs actively in the assessment process. NGOs will conduct their own independent assessments that will feed into the overall reports. They will submit data on service delivery components in particular. Since NGOs in Kazakhstan have limited capacity in this field, the project introduced activities that include an NGO training component to enhance their capacities to evaluate public service provision.
As indicators were being piloted, it became clear that the assessment indicators for the proposed planning system of the government (in one of the six methodologies prepared) were not useful in monitoring changes in the effectiveness of sectoral ministries. They therefore had to be revised. Internal problems were also identified with the other five methodologies, thus stimulating the revision of the indicators system. Some of these issues arose because of a lack of objectivity emanating from self-scoring systems. The involvement of NGOs is assumed to assist with providing data from a civil society perspective, and enhance the objectivity of the assessment overall.
Lessons learned On process
• In countries with constrained political space, governance assessments may serve as important vehicles for strengthening civic engagement. In Kazakhstan the input of NGO data is viewed as beneficial for strengthening the accountability of ministries and provincial governments, which is in the interests of civil society and of the Office of the President. • Data collection processes may be quite heavy for the concerned ministries and regions. They should be streamlined or simplified where possible to ensure improved efficiency and allow the rest of the process to run smoothly. On indicators and data • To get a complete picture of government performance, it is important to integrate data from public opinion assessments as well as performance assessments. Citizens’ experiences and perceptions of public services provide an invaluable perspective that may strengthen public administration assessments. This may serve to widen democratic space. For more information, please see gaportal.org/undp-supported/kazakhstan.
14
Viet Nam 5 The Provincial Governance and Public Administration Performance Index (PAPI) in Viet Nam was made possible by an unprecedented collaboration between UNDP, the Viet Nam Fatherland Front (VFF) and the Centre for Community Support and Development Studies (CECODES). The philosophy behind PAPI is to consider citizens as “public administrative services end-users” who are capable of monitoring and assessing governance and public administration at their local levels. Based on citizens’ knowledge and experience, PAPI provides a set of objective indicators that help assess the performance of governance and public administration, and gives leverage for provinces to improve their performance over time. Respondents during a face-to-face confidential interview
Lessons learned On process
• In Viet Nam it was crucial to establish an informal research network consisting of senior international and national governance experts. Such a network produced policy papers highlighting key challenges and proposed specific and detailed recommendations for the way forward. This served to build knowledge and influence the debate on the nature and complexity of public administration reform and, more importantly, of the options and way ahead. • It was relevant to link the analysis to other research in Viet Nam that has helped to identify significant shortcomings in the reform process, in particular the lack of monitoring mechanisms to assess public administration performance, and the near absence of instruments aimed at measuring the experiences and perceptions of non-business and non-public sector actors. This opened up an opportunity to conduct surveys that capture the experiences and perceptions of citizens. • The set up of a high level national advisory board with diverse representation helped to guide and shape the process and provide a space for policy debate and dissemination of the assessment results to different stakeholders. It also added an enormous degree of legitimacy and acceptance to the process and results. On indicators and data • PAPI follows a rigorous methodology in accordance with international standards and using state-of-the-art statistical techniques to enhance the reliability and validity of the findings and indicators. • Transparency in research itself proved critical. • The PAPI obtained data from a representative selection of ordinary citizens, rather than from the heads of households. This helped to measure the perception of a cross-section of the population, including groups differentiated by gender and age.
5
For more information about this case, please see “Developing a Demand Side Governance and Public Administration Performance Index: The Viet Nam Experience”, by Jairo Acuna-Alfaro in ‘Making the State Responsive’, UNDP, 2012, which this text also draws on.
15
On communication
• Consultation and validation workshops were critical to ensure acceptance by relevant policy makers at the central and local levels • A clear communications strategy, including an interactive website and frequent interaction with media helped shape dissemination and awareness at different stakeholders. • Partnership with influential academic institutions has helped to increase understanding of factors affecting the quality of governance using PAPI data and findings.
Discussion and reflections Governance assessments offer huge potential for strengthening civic engagement and broadening democratic space in politically constrained contexts. ‘Assessments’ and ‘data’ may offer important entry points for innovative and participatory approaches to strengthen inclusion. Such entry points include the various stages of conducting the assessment, for example the stages of defining assessment priorities and indicators, collecting data, and debating the results. There are also entry points in terms of ‘indicator methodology’, such as using pro-poor and gender-sensitive indicators, and perception and incidence surveys. To maximize the potential for governance assessments to serve as a foot in the door for widening democratic space, political economy analysis may be particularly useful in designing the assessment methodology and selecting the implementing partners. Identifying the right local champions seems to be key. Targeting specific service sectors may be more politically acceptable when exploring governance assessments in countries that have a constrained political environment. Moreover, for CSOs to play an active and appreciated role in governance assessment projects it might, in some cases, be useful for the project to include a CSO capacity development component. This could focus on how they can engage with government in governance assessments. For more information, please see gaportal.org/undp-supported/vietnam.
16
3. How can governance assessments strengthen the role of parliaments? The third group of countries (Nicaragua, Paraguay and Montenegro) have used governance assessments to inform and strenghten parliament in performing its oversight function and legislative role. In addition to making evidence available to parliamentarians on the state of governance, these projects also have a clear capacity development objective.
Nicaragua The Nicaraguan project “Government indicators to strenghten public policies” focused on improving the functioning of the Public Expenditure and Monitoring Unit (PEAMU) within the Nicaraguan National Assembly. This has aimed to increase the quality of PEAMU reports to parliamentarians (including gender analysis), and to make information more available to the general public. An independent think-tank was contracted to conduct individual interviews with relevant NGOs and revise the indicators system based on their input.
Lessons learned
Visit of UNDP Paraguay to the Public Expenditure and Monitoring Unit (2010)
On process
• Governance assessment is not a technical exercise, but a highly political undertaking. If limited to a small technical unit without involving politicians from all parties, it may become difficult to achieve a meaningful impact. • Limited participation of CSOs in the project may aggravate a narrow scope of the project. CSOs can provide an important alternative perspective to the use of public funds. Even in a politically sensitive environment, it is important to find innovative ways to involve civil society as much as possible in the process. • It is important to secure a legal framework within which the technical unit can operate and to which it can refer. This will help it to clarify and establish a mandate. • Groups of politicians and of technical staff should be engaged in formulating the project. This may assist with institutionalization. On indicators and data
• It is important to introduce new approaches slowly, such as a gender perspective, to secure that there is appropriate buy-in. On communication
• It is critical to secure a close relationship between political decision-makers and technical implementers. For more information, please see gaportal.org/undp-supported/nicaragua.
17
Paraguay 6 Led by Congress, the project “Democratic governance indicators of Paraguay” aimed to support a national process of monitoring and evaluation of democratic governance in Paraguay. A Coordination Committee was established comprising representatives of national and local government, Congress, local assemblies, civil society and academic bodies, to secure an inclusive process. Congressional leadership of the assessment was assumed to be strategic to ensure broad bipartisan political consensus on accepting the evidence base for governance reform, and for strengthening the capacity of Congress in its oversight function. The National Statistical Office was used to collect the data, while government capacity was raised through the recruitment of staff.
Lessons learned
The report “Democratic Governance Indicators of Paraguay” (2009) produced by the National Congress of Paraguay and UNDP
On process
• Democratic governance assessments are eminently political activities. To ensure their sustainable impact, ‘power mapping’, political economy analysis, or stakeholder analysis should be carried out. These must be at the heart of the project design and implementation phases. • The national legislature may not be an ideal counterpart in countries with weak governance structures. This view is based on an idealized understanding of the relationship between the legislature and the executive, in particular the ‘scrutiny’ role of the former, which does not take into account the deep-rooted political culture of clientelism that is prevalent in many such countries. • It is important that the committee meets regularly, and stays actively involved throughout all stages of the project in the process of indicator selection, data analysis and the dissemination of findings. On indicators and data
• In countries with weak governance structures, technical support provided by the UNDP Global Programme on Capacity Development for Democratic Governance Assessments and Measurements should be available not just for call-down, but should become standard procedure. On communication
• Dissemination should not be viewed simply as a final add-on component of project intervention. Rather, it is essential in achieving a significant impact. For this reason, a dissemination strategy should be developed at an early stage of project design. It should identify the key stakeholders and mechanisms capable of promoting the behavioural changes required to achieve the objectives of the intervention. • Special attention should be given to planning a socially-inclusive dissemination strategy to maximize the impact of government assessment projects. A communication strategy that targets alternative means of communication (including local radio stations broadcasting in household languages) could have a greater impact on reaching out to poor and marginalized groups, who suffer most from poor governance.
6
18
For more information on the case from Paraguay, please also see “Paraguay Governance Assessment”, DGTTF Lessons Learned Series, by Andre Nickson, 2011., which this text also draws on.
• The establishment of a broad-based and representative monitoring committee, with the strong participation of civil society organizations, could be crucial in maximizing the impact of governance assessments and ensuring their sustainability. By promoting widespread dissemination, such a committee can generate a virtuous circle by creating new demand for follow-up governance assessments. For more information, please see gaportal.org/undp-supported/paraguay.
Montenegro The project “Transparency and Accountability in the Montenegrin Governance System” aimed to facilitate Montenegro’s accession to the European Union (EU) with tools to guide the creation and implementation of the laws, policies and reforms necessary for membership and effective democratic governance. Montenegro has to submit a series of reports in support of its candidacy to the EU in accordance with the Copenhagen criteria. But these reports do not specify the steps needed to achieve the required standards. This project was based on the assumption that Montenegro would benefit from a more detailed assessment of progress that would provide the Government, Parliament, civil society, and the media with a more incremental, action-oriented perspective of Montenegro’s EU accession. In partnership with the National Democratic Institute (NDI), the report ‘Transparency and Accountability in the Montenegrin System’ was produced.7 The project focused on Parliament and its relations with the executive branch and other oversight organizations, relationships that are at the heart of the effective democratization required for EU accession.
The report “Transparency and Accountability in the Montenegrin Governance System” (2009) by NDI and UNDP
Lessons learned On process
• Regional associations and accession may provide a useful incentive structure for embarking on a
governance assessment. In Montenegro, the assessment proved particularly applicable to a Parliament seeking EU accession by providing a checklist for parliamentary reform. • It may be important for the implementation of innovative projects to be particularly participatory. That is, perhaps, the most effective communications strategy for addressing issues in unfamiliar ways.
Discussion and reflections Using Parliament or its related institutions as the main counterpart for a governance assessment is no guarantee that the assessment will be effective in transforming actual governance practices. A political economy analysis is needed to determine where Parliament is the optimal implementing partner. Aligning the assessment with the core mandate of an existing standing parliamentary committee may make the assessment more relevant to decision makers than hitherto experienced, and therefore also increase effectiveness. For more information, please see gaportal.org/undp-supported/montenegro. 7
For full report, please see: www.undp.org.me/files/reports/ijr/Report%20on%20Assessment,%20Transparency%20and%20Accountability%20in%20the %20Montenegrin%20Governance%20System.pdf
19
4. How can governance indicators be ‘inclusive’? A governance assessment’s participants will influence its outcomes and impacts. Securing the inclusion of the views, experiences and opinions of various socio-demographic groups in the definition of governance indicators and the actual data collection may not only improve the quality and representativeness of the data collected, but it may also serve to strengthen the demand side for democratic governance. In addition, bringing various stakeholders together opens the opportunity for direct dialogue and policy change. Several projects aim to actively include citizens during the various stages of the assessment. In this section, the following approaches to inclusion will be presented: 1. Macedonia FYR: Integrating ‘inclusion’ as a principle in the methodology used to assess the performance of service delivery 2. Tajikistan: Making use of focus group discussions to secure an inclusive process of designing a questionnaire 3. Mongolia: Designing a satellite account to capture the views and experiences of nomad populations
FYR Macedonia Considering the fact that Macedonia is a multi-ethnic and multi-religious society, as well as the conditions for EU membership, the country’s governance assessment project strongly emphasizes social inclusion. Led by the Macedonian Centre for Research and Policy Making (CRPM), the project “People Centred Analyses” addresses two service sectors: health care and education. The project aims to track and communicate progress in achieving policy goals, provide evidence-based contribution to future policy development processes and strengthen democracy by engaging stakeholders through informed discussions. The project started with the notion that social inclusion and governance are complex issues that need to include the perception of different stakeholders (service providers and users) and requires a combination of data collection methodologies to increase its credibility and legitimacy. A wide range of stakeholders (including students and parents) were therefore included at every stage to revise the project’s methodology, provide information, review and validate The People-Centred Analyses Reports the findings, and enhance their own capacities to demand or provide services in are produced yearly since 2008 and represent an attempt to initiate a accordance with basic governance principles through training. broader discussion on social inclusion
Lessons learned On process
• Integrating ‘inclusion’ as an aspect of the technical methodology is not enough. The assessment process
also needs to be inclusive. While the indicators had a strong inclusive element in the Macedonia project, the project did not include government or civil society in its management structure. • During the design stage, it is important to anticipate how the project will ensure institutionalization of the methodology. In Macedonia, some work is still required to integrate elements of the methodology into a regular monitoring and evaluation system used by government. On indicators and data
• ‘Inclusion’ can usefully be included as a governance performance value in the assessment’s
methodology. This will allow the assessment to provide evidence on inclusive aspects within sectors.
20
On communication
• It is important to include sufficiently strong advocacy components to enable civil society to continue the project’s work and use the findings actively. To achieve lasting change, advocacy beyond a restitution workshop and a one-time training should be carried out.
For more information, please see gaportal.org/undp-supported/fyrmacedonia.
Tajikistan Led by the UNDP country office, the aim of the governance assessment project “Water Sector Integrity Vulnerability Assessment” in Tajikistan is to assess the integrity of the water sector with regard to the supply of drinking water in urban and rural areas, and to irrigation. With an annual water production of over 13,000 cubic metres per capita, Tajikistan is one of the world’s most water-rich states. Yet only 59 percent of its population has access to safe drinking water. In a regional perspective, Tajikistan has the lowest access to safe drinking water in Central Asia. Corruption is recognized as an important obstacle in scaling up the supply of water services.
Children fetching water in Tajikistan
To tackle this politically sensitive topic, the project has chosen a participatory approach. It embarked on an extensive round of focus group discussions, in both urban and rural areas, to secure input into the design of a questionnaire. The questionnaire was then used to collect relevant data from the end users on the risks, perceptions and incidence of corruption through surveys.
Lessons learned On process
• A participatory governance assessment approach may be more likely to produce practical recommendations for tackling corruption in sectors such as water, because it involves all stakeholders in the water supply chain and will collect actual evidence from the end users on corruption instances, which service providers may find difficult to ignore. • Having a large stakeholder pool in the project’s advisory group has positive and negative effects, such as long delays in feedback. Broad participation when discussing politically sensitive topics is a prerequisite for success, but can be costly and time consuming. It therefore needs to be well considered if it is to be efficient. • The same can be said for the political participation process dealing with vested interests and political sensitive topics that could backfire on government. It takes time to create sufficient political buy-in and fit the project into the ruling party’s political agenda. • To retain momentum in complex political and sensitive settings, it can be particularly useful for the project to work through a champion or a catalyst within government. They should have sufficient clout to maintain the assessment as a priority on the political agenda even in challenging political times and to ensure that the process remains efficient and inclusive. On indicators and data
• A participatory approach to survey design using focus group discussions strengthens the quality and validity of the questionnaire, and serves to provide more relevant data. In Tajikistan, focus groups identified multiple risks to corruption, which were included in the questionnaire, and which the household survey was able to quantify.
For more information, please see gaportal.org/undp-supported/tajikistan.
21
Mongolia 8 Mongolia’s MDG9 on governance and human rights has given the country significant experience in assessing governance. The project “Achieving MDG9 on Human Rights and Democratic Governance-Phase II” included the continued support to monitoring activities. Formulated as a national goal, complete with funding, targets and monitoring indicators, Mongolia means business when it talks about democratic governance and human rights. When it signed the UN Millennium Development Goals, the Mongolian Government felt the MDGs could be more ambitious. It met this need by adopting a ninth goal, unique to Mongolia, on issues of Voting in Mongolia during the 2009 Presidential Elections democratic governance and Human Rights. This project builds on previous experience in developing democratic governance indicators and MDG9 target indicators, as well as institutionalization of governance assessment processes. The project aims to develop the capacities of government agencies and civil society to report on MDG9 implementation. It also aims to collect, maintain and analyse governance-related data, as expressed in the MDG-based National Development Strategy and the Government Action Plan 2008-2012.
Lessons learned On process
• The legitimacy and authority of governance assessment outputs can be secured when the assessment process is nationally initiated and driven, and when it engages a wide range of stakeholders. International instruments can thereby be optimally tailored to the national context. However, it is important not to expect the consultative process to be quick. Designing and gaining stakeholder consensus on assessment methodologies, indicator targets and monitoring and evaluation processes requires time to allow it to be done carefully and well.
• The jurisdictional level at which governance assessment is implemented must be stable. If reforms of the mandate or authority of the jurisdiction are pending, such as of local government, the conditions for launching a full-blown initiative will not be favourable.
• Governance assessment may be most fruitful at the sectoral level. Popular and official support and understanding of governance assessment and reform may be increased by avoiding the concept of governance. Its possible lack of clarity as a concept may be an obstacle to its take-up by public officials. Reforms might be achieved more easily by referring to the specific components of governance and to concrete areas. The indicators will be more specific and intuitive and will likely be accepted more readily by public officials.
8
22
For more information, please see Evaluation Report: Support in Achieving MDG-9 on Human Rights and Democratic Governance –Phase II Project, by Helen Addison, 2011, which this text also draws on.
• The availability of evidence on governance is a precondition for improving governance policy and
practice, but it is not sufficient. The political will to undertake reforms must also be present. To avoid governance monitoring becoming an exclusively academic exercise, the determinants of political will require attention. These determinants include (1) the plurality, power and capability of civil society interest groups, (2) citizens’ demands for good governance, (3) the media’s ability to expose malfeasance in government and provide objective information on governance, and (4) leadership of the governance reform agenda at the centre of government. These are important variables in the causal link between the availability of evidence on governance and the use of that evidence to improve governance policies and practices.
On capacity development
• The individual capacities development of public officials in the methods of governance measurement
and reporting should be linked to an established human resources development system. If the institutional conditions for ensuring stability in public employment are not in place, investments in training individuals do not constitute institution building. High turnover of staff in public administration undermines the institutionalization of a national system for governance self-assessment.
• The professional quality of individuals at the heart of the initiative is a necessary and, perhaps, a
decisive factor in ensuring success. The implementation team should not just have scientific and managerial credentials, but also possess strong contacts with high-level politicians and senior officials.
• A project to introduce governance monitoring and assessment will advance its objectives by monitoring and assessing its own performance. Monitoring data and articulated records can be used to refine the implementation strategy and to demonstrate to stakeholders the value of results-based management.
Discussion and reflections As governance deals with the interaction between a state and its citizens, it is imperative to include citizens’ perceptions and experiences on the performance of government in any type of governance assessment. While the political space and opportunities for engaging with civil society varies from one country context to the next, governance assessments may offer innovative approaches, platforms and channels for such engagement. It is important that entry points for engagement are identified and maximized to enhance the democratic dividend of the assessment. While citizens’ perspectives can be brought into the assessment through a survey, or ‘inclusion’ can be investigated as a principle of governance, this will never replace the need for active and broad civic engagement throughout the various stages of the assessment process. That said, the projects demonstrated the advantages of triangulating various methodologies. The use of different qualitative and quantitative data collection methodologies not only increases the validity of the data – and therefore the credibility – of the assessment, it also increases the involvement of different stakeholders in various aspects of the assessment and, therefore, in the collective ownership of its results. On advocacy, the need for media involvement should not be underestimated or overlooked in favour of an NGO-only involvement as a means of inclusiveness and dissemination of results. Finally, the assessors need to be aware that an assessment exercise could be hijacked by one group to advance its specific agenda. So it is important to make special efforts to ensure that the inclusive participation is truly equal and does not favour one group over another. There is a possibility that inclusiveness could backfire on an assessment if one is operating in a context of ethnic diversity. For more information, please see gaportal.org/undp-supported/mongolia. 23
5. How can local governance assessments make institutions more responsive? One aim of governance assessments is to improve the supply side of governance; to identify gaps in a government’s capacity to deliver services in accordance with basic principles of democratic governance, to define benchmarks and monitor performance and to make government institutions more aware of and responsive to the needs of its citizens. In this context, two projects with a strong local governance component were presented. At the time of writing, one had almost been completed (Nigeria) and the other was just starting with the data collection (Egypt).
Nigeria The Good Urban Governance programme was conceived in response to Nigeria’s rapid urbanization and the resulting poor quality of governance at the local level. It adapted UN-HABITAT’s Urban Governance Index to fit Nigeria’s context. It also used a combination of mutually reinforcing data collection methodologies to assess the state of governance in 18 local governments and seven states in Nigeria.
Youth Leaders participating in a focus group discussion held in Abuja (photo by AMAC)
The project has produced a reliable set of data on urban governance for a selected group of local governments in Nigeria. The data are presented so that they can be readily used by local CSOs to lobby for change. Recommendations made by the governance assessment are integraed in a review of the national policy on urban governance in Nigeria.
The theory is that upstream (policy advice) and downstream engagement (involving citizens and CSOs in tackling practical issues) in urban governance has the potential to make institutions more responsive. Urban governance information resource centres are established in several urban areas. Based on the project’s outcomes, local government staff will be trained in aspects of responsive governance.
Lessons learned On process
• Implementing a governance assessment may raise awareness among service providers of the importance of adhering to democratic governance principles, and can contribute to a change in attitude of service providers. • Involvement of civil society is critical for success, because they can use the data to hold government accountable. • A competitive element whereby cities compete to attract the private sector may be a useful incentive for strengthening governance performance, which an urban governance assessment may capitalize on.
24
On indicators and data
• It is important to design a methodology that is feasible and timely. In Nigeria, delays in output delivery by national consultants meant the project had to revise the data collection methodology. • It may be more sustainable to avoid using consultants to collect administrative data, but rather to build systems for data collection inside local government. In Nigeria, this was supplemented by oversight committees composed of representatives of civil society and universities to secure objectivity. For more information, please see gaportal.org/undp-supported/Nigeria.
Egypt In Egypt, the Social Contract Centre and the Local Development Observatory work together to implement a sectoral and local governance assessment project in a rapidly changing political-economic context. As part of the on-going support to the Social Contract Centre, the objective is to conduct a baseline study on the state of governance in selected sectors at the local level, to assess the related institutional and capacity gaps and offer policy advice to government. Egypt’s recent changes means that the project has the potential to contribute to a change of attitudes of local government staff to become more responsive and service oriented. Since no formal participation mechanisms existed in Egypt, it has the potential to contribute to Participants at the Health Task Group Workshop (photo by the Social Contract Center) the design of a new mechanism of citizen participation.
Lessons learned On process
• It is critical to secure that the project is dynamic and can adapt to political transformations. Although the political context is changing rapidly in Egypt, the project is consistent in its focus on how governance assessment can be institutionalized within the emerging revised institutional setting. • As an institution facilitating citizens’ feedback on government performance, but one nevertheless linked closely to government, it is essential to find a balance between being taken seriously by the sectoral ministries involved in the assessment on the one hand and remaining impartial and flexible on the other.
Discussion and reflections An important outcome of the process of a governance assessment is to create a shift in mind set, whereby the state and citizens expect institutions to be service oriented, that access to information is a right, that citizens have a right to ask government whether it is doing a good job, and that there are mechanisms for lodging and addressing complaints. A governance assessment may serve as a direct link between citizens and service providers, where citizens’ views and experiences can be captured. However, it is important that these feedback mechanisms are linked to direct consequences, in the form of incentive structures or management decisions. Monitoring without consequences may not have positive effects. On the contrary, there are risks that citizens grow wary and develop survey fatigue. For more information, please see gaportal.org/undp-supported/egypt.
25
6. How can assessments strengthen ownership of national development plans? This section focuses on the alignment of governance assessments with ongoing national policy processes, such as national development plans. A frequent constraint to the effectiveness of these plans is lack of ownership, a key concern underscored by the Paris Declaration, Accra Agenda for Action Plan and the outcome document of the Fourth High Level Forum on Aid Effectiveness in Busan. Not only will governance assessments provide valuable information for national planning. If done properly, they will enhance the collective ownership of these plans. Two projects from Senegal and Malawi will be presented. They address the process of indicator development in line with defining targets for national development planning.
Senegal In Senegal, the project “Strengthening capacities for monitoring and evaluation of good governance” is implemented with the Ministry of Finance in support of the National Programme on Good Governance. It aims to establish a national monitoring system on governance, and involves a broad group of stakeholders. The project addresses both the demand and supply side of governance. Participants at a workshop held by the Delegation of State Reform and Technical Assistance (DREAT)
Lessons learned • Tough decisions may have to be made on trade-offs. One may choose to secure strong ownership and keep the process internal, or rely more on external expertise increasing quality but with possible consequences for ownership. One may also choose to keep the scope of the assessment manageable and measure only a few indicators, or opt for a more comprehensive assessment of the overall governance situation, which may be more complicated. One also needs to balance the time to secure institutionalization against pressure to demonstrate results quickly in order to secure continued support. • It may be important to move from indicator selection to target setting. The assessment process may be most effective if it is conceptualized as a multi-stakeholder political process that also involves setting and revising political targets. • It may also be important to move from monitoring to evaluation, and conduct more continued evaluation to enhance programmes. Monitoring may not provide sufficient information to guide policies and programmes. For more information, please see gaportal.org/undp-supported/senegal.
26
Malawi The objective of the governance assessment exercise in Malawi is to develop the national capacity to define and select governance indicators and data collection, to assist in the formulation of a five-year sector-wide strategic plan for democratic governance, and to develop a robust and evidence-based indicator and monitoring and evaluation framework to measure the performance of democratic governance in Malawi. Implemented by the Ministry of Justice, the project uses the Malawi Growth and Development Policy. In particular, it uses the Women voting in the 2009 Presidential Elections democratic governance theme as a starting point from which (USAid Photo Gallery) to formulate the indicators. This is to ensure that the outcomes feed into the Growth and Development Policy’s formulation and monitoring and evaluation system. In addition, the data are collected and analysed along the sectoral pillars in the Growth and Development Policy.
Lessons learned • Stakeholder buy-in takes time (one and a half years in the Malawi case), but is crucial for the assessment. Civil society engagement is also critical. • Policy development and assessments may benefit from moving in iterations. In Malawi, a policy framework was drafted before the baseline surveys were conducted and capacity assessments were completed. Based on the data collected the policy framework will again need to be revised. • Responding to national priorities is important. Equally, these national priorities and processes may become very politicized, enhancing the need for careful coordination with ministries and national planning processes.
Discussion and reflections Multi-stakeholder governance assessment processes have much potential to enhance the collective ownership of national development plans. Country-owned governance assessments often exemplify the lack of ownership of the framework policies in which the results need to be integrated, and could therefore make a positive contribution to the policy formulation process in general. The broad engagement may focus policy objectives and targets, and increase the political momentum and will to follow-up on stated plans and documents. A key concern is to secure follow-up beyond the formulation stage and throughout the policy-making stage, by ensuring that targets and priorities are reflected in medium-term fiscal frameworks, in annual budgets, and in monitoring and evaluation. Strengthening the attention throughout the full circle of policy making is an essential building block of national ownership. To make effective use of governance assessments, the monitoring indicators can also be used to define targets for improvement. They can also be used for future monitoring of progress towards those targets. Integration of the government assessments in national policy-making processes, as shown in the cases above, may enable such utilization of findings. However, the probability of the results of governance assessments actually being used for planning and policy-making seems to be much greater if a national good governance framework or policy is in place. For more information, please see gaportal.org/undp-supported/malawi. 27
7. How can assessments strengthen policy planning? The last two cases (Bhutan and Indonesia) focus on how governance assessment results can be integrated into national or sub-national planning processes and related capacity development, and how they can contribute indirectly to improved service delivery and enhanced responsiveness of government.
Bhutan In Bhutan, the Gross National Happiness Index tracks progress regularly – every two years. The project focused on developing policy screening tools based on the Gross National Happiness Index philosophy. The tools assess the potential impact of all proposed government policies and projects on gross national happiness by using a Gross National Happiness Impact Matrix. Development of the indicators has become more participatory with time, and the matrix has been used to assess several projects and policies. Some projects were rejected, demonstrating that the tools have real impact. Happiness in Bhutan
Lessons learned • Bureaucratic resistance to the introduction of changes and tools from the outside is likely to take place, and may be mitigated through buy-in, incentives and time.
• Screening tools may make the integration of governance visions, such as Gross National Happiness, in planning and policies much easier and more practical. For more information, please see gaportal.org/undp-supported/bhutan.
28
Indonesia The Indonesia Democracy Index is one governance assessment project that has completed the full cycle from inception to institutionalization. The index is implemented by the National Planning and Development Agency, in partnership with the Ministry of Home Affairs, the Central Statistical Agency and provincial governments. The project supports a nationally owned process for assessing and monitoring democratic governance in all of Indonesia’s 33 provinces. It aims to provide an inclusive and consultative framework for the systematic assessment and monitoring of democratic governance goals and targets expressed in Indonesia’s national and regional development plans. Since it is applied across the country, it can be used to rank and compare provinces. A utilization tool has been developed. It is intended for use at the provincial level to ensure that the principles and ideas of the index are considered during the process of local planning and policy development.
The report “Measuring Democracy in Indonesia: 2009 Indonesia Democracy Index” published by the National Planning and Development Agency (BAPPENAS) and UNDP
Lessons learned • Implementing a tool that ranks provinces is likely to result in mixed responses from provincial governments. Some will reject it, while others making a budget available for replication. Equally, competition may also be a key driver of change, as provinces strive to respond to increased public pressure, and to enjoy the benefits that a good ranking provides. • With regard to rolling out utilization tools at the provincial level it may be useful to start with pilots. Engaging civil society to make use of the findings may strengthen the tool’s policy relevance.
Discussion and reflections It is considered good practice that governance assessments not only serve to strengthen accountability after a policy has been implemented (ex-post), but also to strengthen accountability before it is implemented, namely at the policy formulation stage (ex-ante). This means that the ideas, visions and principles of governance agreed through a multi-stakeholder process should also guide the formulation of new policies, to strengthen consistent policy making. Governance assessments have potential to strengthen national policy-making processes through the development of tools, such as a policy screening tool in Bhutan, and a utilization module, as in Indonesia. Such tools are important. It is recommended that they are complemented by consultative processes, and that the voices of people directly and indirectly affected by a policy are heard during policy design. Tools can never replace consultations, but may help to highlight possible policy consequences that may contradict or support overall governance visions. For more information, please see gaportal.org/undp-supported/indonesia.
29
Lessons from the ‘Oslo Principles of Democratic Governance Assessments’ Scoring exercise The Lessons Learned workshop took place back-to-back with the Oslo Governance Forum. It adopted 11 strategic and normative principles for conducting democratic governance assessments. A scoring exercise was done at the end of workshop to assess how the governance assessment projects of the Global Programme on Governance Assessments are performing with respect to these principles. Participants directly involved in project management were asked to score their project between 0 and 10 on each of these principles. Eight responses were collected (Table 1).
Table 1 Scoring by participants directly involved in project management on the Oslo Principles of Democratic Governance Assessments Principle
Average Score
1 Promote country-ownership of governance processes and assessment.
7.1
2 Strengthen the ability of people to hold their governments to account.
5.0
3 Apply a rights based approach.
6.5
4 Strengthen governments’ capability to be responsive.
7.4
5 Strengthen accountability across government.
7.1
6 Promote and protect space for citizens to participate in governance assessments.
5.3
7 Commit to transparency and access to information.
7.3
8 Encourage a culture of evidence-based policy-making.
7.4
9 Embed the assessment in political realities.
6.1*
10 Align with national development and political vision.
7.8
11 Support democratic governance assessments at the local level.
7.5
Principle
6.22
* excluding the n/a score
The average score for each project was 6.7 and the average score per principle is 6.22. Three participants have low scores on several principles (but not necessarily on the same principles), which affect their overall scores negatively. The two principles with the lowest average score are: Strengthen the ability of people to hold their governments to account; and Promote and protect space for citizens to participate in governance assessments. The principle with the highest average score is principle 10: Align with national development and political vision. While this was only a quick and off-the-cuff exercise conducted mostly by practitioners who work on those very projects and without an external evaluator, it suggests that there is more to be done on integrating social accountability into projects: a lesson learned.
30
Overall conclusions and lessons 1.
The project presentations and the results achieved so far show that the programme’s approach is reasonably successful. a. All projects demonstrate that there is clear national ownership of the assessment process, the objective of the assessment and the methodology. There are differences in the width and depth of this ownership, though, as ownership by the national entity in charge of the project does not automatically imply that there is broader national ownership. The latter requires additional effort. b. Most projects were able to illustrate that the assessment is fine-tuned to national planning instruments and development programmes. For some projects led by non-governmental academic entities, this alignment is not systematic, even if the issues dealt with by the assessment are always linked. c. Several examples were presented on how the projects contribute to capacity development. These are mainly from steering committees and implementing agents, and less from the end users of the assessments, although some projects are in the process of developing capacities of the demand side as a second phase.
2.
During the presentations, discussion and reviews, it became clear that there are no major questions about the quality of the actual governance assessments. Some countries, however, were able to produce better results than others, depending on available information or lack of capacity/money to conduct more thorough surveys. In other cases, indicators could be even further refined to respond better to national political contexts. Despite the variety of approaches and methodologies used, it seems that most projects are well able to produce robust data and an analysis that is impartial and of good quality.
3.
During the discussions, it became clear that the actual use of collected governance data by government or civil society is a concern for most projects, as usage will not happen automatically. There is a danger that projects end with the assessment report and give insufficient attention to dissemination, publication of findings, and capacity development of the intended users to make active use of the data.
4.
It is important to define success at the right level. If projects have policy impact as a goal, the presentation of the assessment report should be considered a means or an output, but not an outcome. This will not only help to define benchmarks for evaluations of assessment projects, but will also help with decisions on possible extensions of projects into a possible second phase.
5.
A governance assessment is an instrument for strengthening democratic practices. It should therefore be clear from the start how the governance assessment will contribute to achieving its intended objectives. This requires a conceptual model, or a theory of change, that is designed by arguing backwards from the intended impact and is based on a political economy analysis of the country. It should contain all mechanisms and activities required for a governance assessment to contribute to strengthening democratic practices. That includes the preconditions and support activities necessary to ensure that the findings of the governance assessment are actually used. Such support activities need to be included in the log frame and the budget.
31
6.
During the workshop, a variety of additional activities were presented that could be applied to increase the visibility and use of the governance assessment: a. b. c. d. e. f. g. h. i.
7.
32
Active use of the governance assessments and promotion of the findings by UNDP itself; More involvement of a diverse group of stakeholders (CSOs) in validating the findings; Active media involvement and presentations to the media; Use of findings to promote public debate; Engage with research networks/academia; Draft policy recommendations and briefs at relevant moments; Hold a public campaign on good governance; Establish an easily accessible public data base; Training of party cadres/civil servants or civil society in using the data.
The consultative platforms established as part of the governance assessment, in which a wide variety of stakeholders participate, can be institutionalized. This should occur after completing the actual assessment and can promote a continuous dialogue on governance, such as the definition of realistic governance targets. It may, therefore, be useful to think of assessments as part of a larger strategy. A governance assessment may be a useful starting point for UNDP democratic governance programming at the country level. There are examples to suggest that it can be at the core of a country office’s governance support strategy. Such a strategy may help to create an enabling context to maximize impact, for example by improving access to information and state capacity to deal with governance demands.
ANNEX Schedule lessons learned workshop 6 October 2011 Time
Principle
8:30-9:00
Welcome by Joachim Nahem, Global Programme Manager Introductions by Paul van Hoof, Workshop Facilitator Explanation of Workshop objective and methodology
9:00-10:45
Session 1 How can assessments be used to promote reform? Think tank-led advocacy: Gerardo Munck, consultant: case of Chile; Guillermo Cejudo, CIDE and Antoni Diego, UNDP: case of Mexico, Zhou Houngyen, CCCPE and Qing Gu, UNDP: case of China Working with Parliaments: Laurence Klein, UNDP and Luis Bravo, National Assembly: case of Nicaragua; Andrew Nickson, consultant: case of Paraguay Chair: Paul van Hoof (consultant) Rapporteur: Alexandra Wilde (consultant)
10:45-11:00
Coffee break
11:00-12:15
Parallel Session 2 Is “inclusive participation” working? Presenters include: Marija Risteska, Centre for Research and Policy Making: case of FYR Macedonia, Alisher Karimov, UNDP: case of Tajikistan, Gani Tasmaganbetov, Presidential Office: case of Kazakhstan. Chair: Joachim Nahem, UNDP, Rapporteur: Asmara Achcar, UNDP Session 3 How can assessments make institutions more responsive? Presenters include: Falade Johnsen, UN-Habitat: case of Nigeria, Khaled Abdelhalim, Local Governance Observatory and Yasmin Khodary, Social Research Centre: case of Egypt Chair: Paul van Hoof, Rapporteur: Nina
12:30-13:30
Lunch break
13:30-13:45
Feedback in plenary
13:45-15:00
Session 4 How can assessments strengthen ownership of policy processes? Presenters include: Marie Laberge, UNDP: case of Senegal, Marius Walter, UNDP, case of Malawi, Dorji Penjore, Centre of Bhutan Studies: case of Bhutan, Irman Lanti, UNDP: case of Indonesia Chair: Gerardo Munck Rapporteur: Danae Issa, UNDP
15:00-15:15
Tea break
15:15-16:45
Session 5 Group work based on the World Café methodology, to identify some joint recommendations and lessons learned from the workshop. 3x 20 minutes groups Table hosts: Sujala Pant, Nina Kolybashkina, Asmara Aschar (all UNDP)
16:45-17:00
Conclusions/wrap up
33
United Nations Development Programme Bureau for Development Policy Democratic Governance Group 304 East 45th Street, 10th Fl. New York, NY 10017 Oslo Governance Centre Inkognitogata 37, 0256 Oslo, Norway www.undp.org/governance www.undp.org/oslocentre www.gaportal.org
