Evaluation Study on The EU Institutions & Member States’ Mechanisms for Promoting Policy Coherence for Development
Appendix XI Questionnaire Survey Report
Client: The Evaluation Services of - French Ministry of Foreign Affairs, lead agency - Germany, Belgium, the Netherlands, and the European Commission
May 2007
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007
Study Team Members James Mackie (Team Leader) Gwen Corre Marie-Laure de Bergh Niels Keijzer René Madrid
ECDPM ECDPM ECDPM ECDPM Particip GmbH
Advisory Group: Paul Engel Jean Bossuyt José Antonio Alonso Christian Freres
ECDPM ECDPM ICEI ICEI
Contact Details European Centre for Development Policy Management (ECDPM) Onze Lieve Vrouweplein 21 6211 HE Maastricht The Netherlands
[email protected] http://www.ecdpm.org
PARTICIP GmbH, Consultants for Development & Environment Headquarters: Hildastrasse 66, D 79102 Freiburg, Germany Brussels Branch: Avenue des Arts 50 (5th floor), B 1000 Bruxelles, Belgium
[email protected] http://www.particip.com/
Complutense Institute of International Studies (ICEI) Complutense University of Madrid Finca Mas Ferré, Building A Somosaguas Campus 28223 Pozuelo de Alarcón Madrid Spain
[email protected] http://www.ucm.es/info/icei
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007
Table of contents 1
Main findings........................................................................................................................................1
2
Introduction & methodology .................................................................................................................3 2.1 Link with the overall study............................................................................................................3 2.2 Focus on opinions........................................................................................................................4 2.3 Four groups of Member States and the Commission compared..................................................4 2.4 Methodology and response rate ..................................................................................................4 2.5 Limitations of the survey ..............................................................................................................5
3
Responses to survey questions ...........................................................................................................7 3.1 Question 1: ranking country mechanisms....................................................................................9 3.1.1 Introduction and overall response ........................................................................................9 3.2 Question 2: actor prioritisation ...................................................................................................11 3.2.1 Introduction and overall response ......................................................................................11 3.2.2 Disaggregated responses and cross-analysis ...................................................................11 3.3 Question 3: prioritisation of main issues for PCD ......................................................................13 3.3.1 Introduction and overall response ......................................................................................13 3.3.2 Disaggregated responses and cross-analysis ...................................................................14 3.4 Question 4: define your own PCD strategy................................................................................16 3.4.1 Introduction and overall response ......................................................................................16 3.4.2 Disaggregated responses and cross-analysis ...................................................................18 3.5 Question 5: main obstacles........................................................................................................19 3.5.1 Introduction and overall response ......................................................................................19 3.5.2 Disaggregated responses and cross-analysis ...................................................................20 3.6 Question 6: successes and challenges......................................................................................21 3.6.1 Introduction and overall response ......................................................................................21
Annex 1: Define your own PCD Strategy: selected responses and comments .........................................24
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007
1 Main findings The following four main findings can be drawn from the responses to this survey: 1: Relative consensus on the promotion of Policy Coherence for Development • •
As is also emphasised in the introduction, the responses to the survey show a relatively high degree of agreement on how intra-governmental Policy Coherence for Development can best be promoted; Despite the fact that responses were sent by officials working in sixteen different Member States and the Commission and thus represent a rich diversity of approaches to government and policy, the responses show a relatively high level of consensus on how PCD could best be promoted;
2: Policy coherence is best promoted via an inclusive approach, in which good institutional coordination is most important •
•
The responses to all the six questions stress the need for an inclusive approach to coherence, where the promotion of PCD depends on the existence and good functioning of clear policies and political support, institutional coordination, and knowledge input & assessment functions. The large majority of issues, actor groups and types of mechanisms which were listed in the survey were on average rated as ‘important’.1 This confirms the validity of earlier research during this study out of which these were drawn, in particular the conceptual framework, the literature review and the analysis of country profiles; The responses to the different survey questions also show that of the three different types of mechanisms which were proposed to the respondents at the beginning of the survey, institutional coordination mechanisms were considered most important.2 Respondents reinforced this emphasis by considering political leadership, good relations and networking between civil servants in different ministries, and institutional structures that allow this to happen to be most important of all for PCD.
3: Policy coherence for development is a relational issue and requires a lot of resources •
•
The responses to all six survey questions, but in particular to questions 1 to 4, emphasised that good relations between government officials is the most important issue for the promotion of PCD. The emphasis of the respondents on this issue suggests that PCD is best promoted gradually by improving the relations between government officials, while supported with sufficient resources, knowledge and political leadership; The responses to question six also emphasised the importance of coordination between PCD specialists and decision-making at the EU-level concerning intra-governmental policy coherence for development. This importance can be positive, for instance through sharing lessons between Member States and EU Institutions, and through deciding at the EU level of what should be done on PCD in the intra-governmental context.3 Conversely a lack of consensus at the EU level on how to move forward on PCD can also negatively affect progress in the area4;
1
In fact, all the issues that were listed were at least considered important, and of the actor groups which were mentioned only ‘other levels of local or provincial government’ were considered relatively un-significant. 2 These three types of PCD Mechanisms – Explicit Policy Statements; Institutional Coordination Mechanisms; and Knowledge & Assessment Input Mechanisms – were proposed in this study’s conceptual framework, and partly based on the 2005 Scoping Study. They have been used throughout the present study, including the desk report, country profiles and case studies. 3 For example the EU Council Decisions on PCD which were made in April and October 2006, but also the process around the DAC Peer Reviews. 4 As was noted in this study’s desk report, the sharing of lessons learned via the DAC Peer Reviews seems to suggest in some cases that the review process has contributed to action towards promoting PCD. ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 1
Draft PCD Opinion Survey Analysis
4: PCD as a process needs space, adequate conditions and appropriate flexibility, and should not become too formalised •
•
•
The responses to question 4, as well as the aforementioned general emphasis on the importance of good relations and dialogue between government officials, would seem to suggest that although respondents are convinced of the importance of a clear policy statement on PCD, they feel that this policy statement should not lead to a too strong formalisation of the PCD process and should not prescribe actions and procedures; Instead, the responses seem to suggest that respondents prefer a process where all ministries gradually learn about the importance of and their possible roles in the PCD process, while these actions are justified by a policy document setting out the overall purpose, priority and direction of policy coherence. Adequate resources in terms of time as well as human and financial resources were all considered essential for allowing this process to happen; The success stories which were sent in by the respondents indicate that progress has mostly been on creating or improving the conditions to allow PCD to be promoted. Challenges which remain include the difficulty of dealing with entrenched national interests in general, and the relationship between the EU and its Member States;
Besides the analysis of the overall responses, the respondents were divided into five different groups for the purposes of detecting different patterns of responses and to allow for cross-comparisons to be made.5 Respondents from three groups of countries were systematically cross-examined for the six different questions: 1. Whereas the Central EU and Southern EU groups mostly stressed the political nature of the PCD process, Nordic respondents emphasised the importance of cooperation and ‘persuasion’ leading to action at both the civil servant and political level. New Member States attached quite some importance to the production and dissemination of information on PCD promotion; 2. Among the three geographic groups which were cross-examined, the Nordic + group attached most importance to knowledge input and assessment mechanisms, followed by the Central EU group. The respondents from the New Member States unanimously considered inter-ministerial committees and the improvement of dialogue between civil servants to be essential for the promotion of PCD; 3. All geographic groups agreed on the relatively low importance of strong legal enshrinement and institutionalisation of PCD, given the relatively low weights attached to PCD laws and procedures. Nordic+ countries did however consider PCD action plans to be of almost essential importance; 4. Compared with the Central EU group and the New Member States, the Nordic+ countries see a more major role of their parliaments in promoting PCD. All country groups unanimously agreed on the importance of European Cooperation to advance on PCD; 5. Besides pointing at the differences in approach to policy change and government between the different country groups, perhaps the most important reason for the differences between the Nordic+ group, the Central EU group and the New Member States is that the last group has a shorter history in development cooperation. Civil servants who work on development policy still have a stronger ‘change agent’ role to play in order to create the legitimacy for their field of work, as well as for PCD.
5
More information on the composition of these groups can be found in the next section ‘Introduction and Methodology’. ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 2
2 Introduction & methodology 2.1 Link with the overall study The Heads of Evaluation for External Cooperation of the EU Member States and the European Commission have initiated a series of six evaluation studies focussing on how the Maastricht Treaty precepts of, coordination, complementarity and coherence (the ‘3Cs’) have been translated into practice, and with what impact. The current study, one of the six in the series, focuses on ‘EU Mechanisms Promoting Policy Coherence for Development (PCD)’. As part of this evaluation seven different mechanisms that promote coherence have been examined in different parts of the EU to see how they carry out this task and to what effect. This report thus covers the role of one such mechanism in promoting PCD. This evaluation of PCD mechanisms is being carried out by the European Centre for Development Policy Management (ECDPM), the Instituto Complutense de Estudios Internationales (ICEI), and PARTICIP GmbH. The evaluation analyses and assesses mechanisms for promoting intra-governmental coherence that have been introduced in the administrations of the Members States and the European institutions since the late 1990s, with the purpose of: Objectives of this evaluation: a) Judging their relevance and effectiveness, as well as the mechanisms’ efficiency, impact and sustainability, in terms of promoting PCD and within their specific contexts; b) Formulating proposals to improve the relevance and effectiveness in terms of promoting PCD of the mechanisms analysed, without neglecting their efficiency, impact and sustainability requirements in this role; c) Enabling politicians and officials in Member States and in European institutions to learn lessons from experience about effective PCD mechanisms and use these more widely. The evaluation has been commissioned and is managed by the Evaluation Service of France, with the support of a Steering Group that also includes representatives from the evaluation services of Germany, Belgium, the Netherlands and the European Commission. A ‘PCD mechanism’, as the term is used in this Study, is taken to mean a mechanism that is a clearly identifiable object of study with concrete features, such as a name and some terms of reference that include PCD as a purpose even as part of a wider scope. Clarity on who is involved, and defined operating ways, are the other elements that qualify the mechanisms. All the mechanisms have other roles as well, but this study only examines their PCD role and does not pretend to cover any other broader role they may have. Being an important element of this study’s methodology, the evaluation team set up and sent an online survey to a list of 62 PCD specialists in order to collect a spread of opinions on what works and does not work in terms of promoting PCD. 30 of these targeted respondents had already been in contact with us during earlier phases of this study, while again 29 of the targeted respondents were members of the informal EU PCD network. Eight respondents were both members of the network and had earlier contributed to our study. Over a period of two weeks, the survey gathered responses from 24 people, which represents a response rate of 35% when including indirect responses.6
6
Indirect responses are respondents which were not directly contacted by us, but who completed the survey after receiving the link to the electronic survey through their colleagues. ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 3
Draft PCD Opinion Survey Analysis
2.2 Focus on opinions We asked the participants not to share the official views of their ministries, but rather their professional opinions, experiences and ideas as PCD practitioners. The focus on opinions rather than facts ensured that the collected data would complement the evidence collected through the other methodological tools: the literature review, country profile mailing and the seven case studies. This report contains an analysis of the survey’s responses, which will later be cross checked with other information collected during the evaluation and integrated in the study’s final report. We would like to emphasise that we targeted an informed group of practitioners, most of which have at least a year of experience of working on this issue. We used the survey to look at the variations between the perceptions, experiences and opinions of this informed group. Overall, the responses to the survey results show an important level of agreement, and point towards a relative consensus on the promotion of PCD. Having noted this consistency in responses, we do find that there is still some relevant variation in responses, which will be analysed in this report while acknowledging the relative uniformity in responses.
2.3 Four groups of Member States and the Commission compared In order to identify patterns of responses we have sought to group respondents from different countries to see if the responses are similar across certain sub-regions. Ultimately, we have chosen to use the same groups as we used in the analysis of the Country Profiles in the Desk Study. This also has the methodological advantage of providing an on-going consistent thread in our analysis. In this report, disaggregated responses are thus shown for the following three groups: • The New Member States (Estonia, Latvia, Lithuania, Czech Republic, Slovenia, Slovakia, Hungary, Poland, Malta, Cyprus); • The Nordic+ group (Netherlands, UK, Ireland, Denmark, Sweden, Finland); • The Central EU group (Belgium, France, Germany, Austria, Luxembourg).7 Due to the low number of responses, it was decided not to include disaggregated data for the Southern Member States and the European Commission. However, the results for these two groups have been included in the overall analysis of the questions. More information on the responses to the survey is provided in the first section of this document. Throughout the document, individual country examples are shown in boxes to further enrich the main analysis.
2.4 Methodology and response rate Although the questionnaire was sent out during one of the busiest periods of the year, a satisfactory amount of responses was gathered by the evaluation team. Responses were received from the majority of Member States and from the European Commission, so that a sufficient geographic diversity in responses was achieved, as is detailed in the table below: Southern EU
Central EU
Nordic+
New MS
EC
No. of people contacted
7
13
18
22
2
Average contacted people per country, per group
1.75
2.2
3
2.2
2
Total 62 (67 incl. indirect responses) 2.31
7 Please note that these three groups, as well as the group of Southern European Member States and the European Commission as a separate group, do not represent any official EU geographic groupings.
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 4
Draft PCD Opinion Survey Analysis Total no. of responses (original list) Total no. of responses (incl. indirect) 8 Response rate
1
6
7
5
1
20
2
7
7
7
1
24
28.57%
63.64%
38.89%
31.80%
50%
Average per group: 42.58% Total response rate (67): 35.80%
In five cases, the questionnaire was not completed by the person we originally addressed it to, but by a colleague whom the questionnaire was forwarded to. These have been counted separately as ‘indirect responses’. Disaggregated responses per geographical group 0
10
20
30
40
50
60
70
7 Southern EU
1.75 2
Central EU
2.2
11 7
18 3
Nordic+
7
22 2.2
New MS
7
2 2
EC 1
62 Total
2.23 24 No. of people contacted
Average no. of contacted people per geographic group
No. of responses
2.5 Limitations of the survey As is shown in the table and graph, the response rate from Southern Member States was relatively low. This can partly be explained by the fact that relatively fewer persons per country were contacted than for the other groups. One respondent indicated that because of his earlier involvement in the PCD evaluation during the country profile mailing, he could no longer justify to spend more time on the study. To fully understand and appreciate the response rate, the following should thus be noted: • Of the 62 people who had been contacted during the mailing for the questionnaire, 25 persons (40%) had already contributed to the study’s desk study through commenting on the draft country profiles; 8
Indirect responses are respondents which were not directly contacted by us, but who completed the survey after receiving the link to the electronic survey through their colleagues. ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 5
Draft PCD Opinion Survey Analysis
• •
Similarly, seven of the responses to the survey were sent by persons who had been interviewed during one of the case study missions. Whereas a total of ten people in the list had been interviewed during the case studies, this can be considered more a motivating condition rather than a limiting one; Although we communicated the survey as an opinion survey and stressed that responses did not have to represent official positions, a few respondents responded that their colleague had already responded on her/his country’s behalf. Although we invited them to also respond, this only resulted in one additional response.
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 6
3 Responses to survey questions Question 1: Prioritising mechanisms 1. Institutional coordination processes were considered most important for the promotion of intragovernmental PCD. This implies that PCD is mostly considered as a matter for civil servants, and as a relational issue between government departments and ministries; 2. Most importance is attached to governmental actors in the PCD promotion process, while among the other actors CSOs are rated slightly higher than other non-state actors such as the Private sector and media. PCD has a much smaller profile at lower levels of government; Question 2 and 3: Main issues and actors for promoting PCD 3. ‘Institutional structures that bring relevant ministries together’ and ‘committed political leadership from within government’ were considered most important for promoting PCD; 4. Broad non-state actor consultation activities were considered least important of all issues, although a strong and vocal civil society was rated somewhere in the middle of the 18 issues; 5. Good knowledge management and a monitoring and learning system were considered as less crucial than other issues, but were nonetheless still considered important; 6. The respondent also suggested that – besides the issues listed by the evaluation team – training, focused awareness raising and departmental or cross-departmental goals and targets are important for PCD; Question 4: Prioritising components for a PCD strategy 7. An explicit policy statement was considered to be the most important component of any strategy towards the promotion of PCD, followed closely by inter-departmental consultation procedures for the preparation of ministerial decisions; 8. Although all listed knowledge input and assessment mechanisms were considered important, policy advisory committees were considered least important of all. Whereas the scores for this category of mechanisms were on average lower than the other two categories, this may partly be due to the supporting role which this group of mechanisms plays;9 9. The responses attached relatively low importance to a strong legal enshrinement and institutionalisation of PCD, given the low weights attached to PCD laws and procedures. Nordic+ countries did however consider PCD action plans to be of almost essential importance; 10. Manuals on procedures, monitoring systems and laws are among the mechanisms that were considered least significant, although still considered as relevant; Question 5: Main obstacles to PCD 11. A lack of dialogue between government departments was considered the biggest obstacle for promoting PCD. The second-biggest obstacle was the resistance from other ministries which had almost the same score as the third, lack of political leadership; 12. In relation to political leadership respondents were clear about the need for more human and financial resources, the need for information and back-up from a PCD unit, and the need for capacity development of civil servants who work on PCD issues. The perceived inadequacy of policies is again related to a lack of political commitment to change them; 13. Consistent with what was responded in other questions, the media and dialogue with other nonstate actors were not considered very important, which once more emphasised that respondents consider PCD promotion mostly as an ‘internal’ government issue. However, it should be emphasised that the communication and cooperation with other actors is not at all seen as illegitimate or irrelevant, but that the lack thereof is considered a ‘mild’ obstacle to promoting PCD of;
9
For example, inter-departmental consultations on ministerial decisions were considered very important, but in order to be effective can be supported by a PCD unit, a monitoring system, or external studies. ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 7
Draft PCD Opinion Survey Analysis
Question 6: Success stories and challenging cases 14. The majority of success stories focused on the creation or improvement of the conditions to allow PCD to be promoted, including the improvement or establishment of relationship and networking between different government departments and ministries; 15. It was mentioned that some European Member States are presently struggling to promote PCD in the context of their migration policies, perhaps not surprising given it’s a topical issue. Despite the progress made in some Member States in areas such as trade, in many policy areas domestic interests still prevail. 16. Although this survey focuses specifically on intra-governmental PCD, many respondents pointed at the lack of progress on the EU level which is hindering progress on PCD at the individual MS level; 17. In both types of stories, no reference is made to the role of non-state actors and the media. In addition, there is nothing mentioned on parliamentary scrutiny other than in relation to the annual reporting. Among other things, this may suggest that these actors do not immediately come to mind when government officials think about PCD; 18. Many countries have chosen to first promote PCD in areas that were relatively ‘lobby-group-free’, such as cotton, to ‘clear the path’ to promote PCD in areas with more entrenched national interests; 19. Similar as in the other questions of this survey, most responses to this question related to PCD Mechanisms in the first and second categories: explicit policy statements and institutional coordination mechanisms. Although the need to improve on the production of knowledge and information on PCD, and the sharing of information was mentioned as a success story once, no reference was made to the relevance or challenge of monitoring, evaluation and assessment;
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 8
3.1 Question 1: ranking country mechanisms ‘Identify what you think are some of the most important mechanisms to promote PCD in your country.’
3.1.1
Introduction and overall response
First of all, the respondents were asked to rank the types of mechanisms in their country, and weighing them according to their relative importance. During the analysis of the responses, these individual weights were clustered among the three different types of mechanisms which were identified for the study’s analytical framework. In a few cases, mechanisms would fit multiple categories. For example, some mechanisms combine coordinating and assessing functions. In many cases, respondents included mechanisms which mandates do not formally prescribe action towards promoting PCD, but which nonetheless play a role in the PCD process. In some cases, respondents also identified certain ‘conditions’ for PCD promotion which could not be considered as a clear mechanism or constituted a new category that had not prior been identified. In many cases, these conditions could be attributed to a certain category. For example, it was decided to count ‘political backing’ and ‘institutional status’ as a result of explicit policy statement and thus count them accordingly, and ‘informal consultations’ were counted as institutional/coordination mechanisms. In the following figure, the entries that did not fit any of the three categories were counted as ‘other’:
Weighted importance categories of mechanisms (n=24) 0
20
40
60
80
100
120
140
160
76
Explicit Policy Statement
174
Institutional/Coordination
31
Knowledge Input and Assessment
Other:
180
20
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 9
200
Draft PCD Opinion Survey Analysis
Bearing the limitations of this analysis in mind, the picture nevertheless shows that in the perception of the respondents, institutional coordination processes are most important for the promotion of intragovernmental PCD. This also implicitly shows that PCD is mostly considered as a matter for civil servants, and as a relational issue between government departments and ministries. It should be kept in mind that most respondents in fact work as civil servants, and that this may represent a bias which should be kept in mind when interpreting these results. Three respondents mentioned the importance of inter-European cooperation to further intragovernmental cooperation, mostly through the exchange of lessons learned. Some mentioned mechanisms or conditions did not fit any of the three categories. A few examples of these are: • A conference organised by an EU department; • Public Relations work and awareness-raising campaigns; • Sufficient staff members to work on PCD issues. The responses only result from the appreciation and perception of an individual, and therefore show quite some differences between different respondents of the same member state: Germany, respondent 1
1. Program of Action 2015 2. Weekly meetings of State Secretaries of the Federal Ministries 3. Interministerial Committee for Export Guarantees 4. Federal Security Council 5. Review of legislative proposals
Netherlands, respondent 1 1. Cabinet and Minister for development Cooperation with full cabinet status 2. High-level EU Coordination Committee 3. Coordinating and assessment mechanisms at working level 4. Capacity in MFA and other ministries and good working relations 5. MDG reports and other PCD progress reports
Germany, respondent 2 1. Bye-law of Federal Government requesting development policy coherence of Government action 2. administrative independence and specific mandate for Ministry (BMZ) 3. Interministerial consultations on major coherence issues 4. relevant documents by other ministries to be co-signed by BMZ Netherlands, respondent 2
Germany, respondent 1
1. Interservice Consultations 2. Interservice Meetings 3. Interservice Working Groups
1. A PCD unit in the Ministry for Dev Co. 2. Interministerial Coordination body 3. PCD position papers by the Government 4. PCD position papers by Dev Co. 5. Informal PCD networks of Member States
1. Capacity to work on PCD: PCD Unit MFA 2. Political Commitment on PCD, government wide 3. Coordination Mechanisms between ministries
Netherlands, respondent 3
ECDPM/Particip GmbH/ICEI –Appendix XI: Questionnaire Survey Report – May 2007, page 10
Draft PCD Opinion Survey Analysis
3.2 Question 2: actor prioritisation ‘Which actors do you feel are important to involve in promoting PCD so as to increase chances of success?’
3.2.1
Introduction and overall response
In this question, respondents were asked to list the actors with whom they work on PCD issues, again asking them to weigh these actors’ according on perceived importance. The following graph presents and summarises the outcome of this prioritisation for all actors: Perceived importance of PCD Actors (n=24) 4 = Essential; 3-4 = Very important; 2-3 = Important; 1-2 = Useful;
