AFRICAN DEVELOPMENT BANK GROUP
TOWARDS AN INTEGRATED SYSTEM FOR EVALUATION OF DEVELOPMENT EFFECTIVENESS – RESULTS BASED MANAGEMENT
OPERATIONS EVALUATION DEPARTMENT (OPEV)
31 July 2000
Table of Contents Page 1.
INTRODUCTION
1
2.
CONCEPTUAL FRAMEWORK FOR RESULTS BASED MANAGEMENT 2
3.
JUSTIFICATION OF A MONITORING AND EVALUATION SYSTEM
4 4.
THE MONITORING AND EVALUATION FRAMEWORK PROPOSED 6
5.
IMPLICATIONS FOR STRATEGIC PLANNING AND OPERATIONS MANAGEMENT 8
6.
CONCLUSION
ANNEXES
ANNEX 1
ANNEX 2
M634DOC.
9
Number of pages
ILLUSTRATION: STRATEGIC PROFILE OF THE PROGRAMME OF ASSISTANCE TO MALI, 1999-2001
2
RBM EXPERIENCE OF OTHER DEVELOPMENT AID AGENCIES (EXTRACT FROM DOCUMENT OF THE DAC WORKING PARTY ON AID EVALUATION)
4
1
-
INTRODUCTION
1.1. One of the main recommendations of the CODE members regarding preparation of the three-year OPEV work programmes was to make the evaluation process forward-looking by establishing linkages between the results of the appraisal and the conception of new Bank operations and policies. To that end, the 1999-2001 programme has been formulated in such a way as to conduct activities from which pertinent lessons can be drawn to contribute effectively to improving the quality of Bank assistance. This new approach consists in focussing on aspects of development effectiveness, in conducting studies of the impact of the asistance strategies and programmes at the levels of projects, sectors and countries, with a view to measuring the performances in light of the set Bank aims and objectives. 1.2 The emphasis thus laid on the quality of Bank intervention shifts from the former preoccupation with evaluation of the operational performance of projects to development efficacy in the various sectors and countries. Likewise there is need to move from evaluation of outputs to that of the impacts, in order to measure the degree of success or failure of the development aid provided. From this new standpoint, the individual project evaluation approach (project by project) no longer has the same precedence and it becomes necessary to develop an appropriate set of tools that reflect the global perspective of Bank operations. The Bank in effect needs to acquire an integrated system making it possible to increase efficiency in the implementation of the objectives of its vision and measure the results of its investments. 1.3 This system known as results based management (RBM), tracks the full evolution of the Bank’s development objectives starting from the strategic goals of the vision, which drive from the thematic and sectoral strategies that inform the country strategies, which in turn provide a framework for project formulation and implementation. The latter produce results (outputs), which yield the development effects and impacts that should aid attainment of the sectoral and thematic objectives that help achieve the strategic goals of the Bank’s vision. RBM involves introduction of a computerized monitoring-evaluation system (integrated into Project Africa) that will provide for continuous tracking of Bank operations up to the realization of the strategic objectives and goals of its vision. This system aims at overall heightening of the effectiveness of Bank aid through management of risks inherent in the attainment of objectives and makes it possible to detect, during implementation, any deviations from the projections and make the adjustments needed to put the operations back on track to meet the set objectives.
M634DOC.
2
1.4 This articulation from the Vision up to the operational objectives and the results expected at the project level is illustrated in the diagram below:
Vision
2. -
Sectoral Thematic Sectoral CONCEPTUAL MANAGEMENT
Country Strategies
Project
FRAMEWORK
Outputs
FOR
Effects
RESULTS
Impact
BASED
2.1 The Bank’s objective is to accomplish its mandate by the most efficient means possible. It therefore sought to formulate its vision in order to more clearly situate its mandate by adopting the objectives that it considers highly strategic priorities and whose attainment should have the maximum impact on development of countries in the region, especially with regard to poverty reduction. In defining its global vision, and within the framework of deliberations for the ADF-VIII Replenishment, the Bank has maintained its concern to contribute to help meet international development objectives such as adopted by the OECD Development aid Committee (DAC) as part of its «Shaping the 21st Century» Development Strategy. 2.2 Results based management, as it involves an integrated framework for monitoring/evaluating of development effectiveness, will make it possible to gauge and manage the efficacy of the attainment of the expected development results through implementation of the strategic objectives of the vision. The sectoral and thematic strategies are the strategic approaches that the Bank intends to rely on in attaining the strategic goals and targets of the Vision. These strategic orientations, like the vision, comprise global objectives that apply to the entire African continent and which, based on the analyses conducted in formulating them, cover the range of needs deemed to deserve priority in that regard. Similarly, the formulation of country strategies should reflect the needs and specific characteristics of the countries, as reflected in their own development programmes, particularly their poverty reduction strategies and programmes. The vision and strategic orientations thus serve as a guide for defining priorities and determining the resources to be provided in these countries to realize these priorities. 2.3 In sum, it is indeed a question of meeting the countries’ needs, but above all addressing these from the standpoint of the Bank’s strategic orientations, so as to enable the institution achieve the overriding development objectives that it considers most effective in terms of their impact on development and poverty reduction in the region. 2.4 The adoption of RBM by the Bank should form part of a coherent approach to operations programming in relation to the Vision and sectoral policies and priorities. As a first phase, the sectoral policy objectives should be aligned with the vision objectives and the country strategy objectives with those of the sectoral policies. It is ultimately the country strategies that will give strategic consistency
M634DOC.
3
between the policies and the selection of projects, in that it will be ensured that the statement of aims and objectives of the selected project is consistent with those of the country strategy (particularly the poverty reduction aspects) to be furthered. . 2.5 The results based management system is one that complements the evaluation function. With its monitoring-evaluation framework, it makes it possible to bolster effective attainment of the desired objectives and provide the tools needed to measure results. The evaluation function further relies on the instruments generally used for monitoring-evaluation to provide in-depth analysis of the results, their origins, their impact and their distribution. 2.6 There are several significant distinctions between this results oriented evaluation mechanism and the strict function of evaluation. RBM is a form of continuous self-assessment applied within the operations departments and essentially concerned with the extent of attainment of the results envisaged. It is also a system that supports the management function, since it provides information on the degree of achievment of the results as well as the status of the risks associated with these, and thus, using a management information system (MIS) involving balanced score cards, makes it possible to alert managers to possible slippage so that appropriate and timely steps may be taken right in the implementation stage. 2.7 The evaluation function for its part is independent of the operations and corporate management departments. It goes beyond the degree of attainment of results and seeks to determine why and how such results have been obtained, so as to assess relevance and effectiveness in implementing strategic goals and objectives for economic, social and institutional development, as well as their sustainability in the long term. Evaluation is discontinuous and selective, focussing on specific themes. It draws lessons from past experience and makes the recommendations for future action. 2.8 The following table illustrates the features of the conceptual model of RBM and evaluation.
M634DOC.
4
Conceptual Model for Results based management Strategic Levels
Strategic Framework
Criteria
Measuring results (indicators)
Strategic sectoral and Global Impact thematic Objectives
Vision
Evaluation
Efficacy (impact on development)
Level: global Sectoral and thematic objectives
Strategies and policies
Strategic and thematic Objectives
Levels: sectoral and thematic
Operations objectives
Strategies Aid programme
Strategic sectoral and Sectoral and thematic effects thematic
Level: Country
Operational objectives
Output
Strategic objectives
Programme effects
Level: Operations (projects, programmes)
Operational objectives
Achievment
3.
-
Relevance Efficacy (impact) Efficiency Sustainability
Relevance Efficacy (effect) Efficiency Institutional development Sustainability Relevance Efficacy (effect) Efficiency Institutional Dev. Sustainability
JUSTIFICATION OF A MONITORING AND EVALUATION SYSTEM
3.1 The adoption of a monitoring-evaluation framework is justified as much in terms of evaluation needs as from the standpoint of strategic programming needs. At present, the Bank does not have the capacity to gauge the results of it investments. It is unable to assess its efficiency in terms of the development outputs obtained, and thus draw lessons with regard to strategic orientations for its future programming. This situation is largely due to the fact that, until recently, the Bank concentrated on projects as development priorities in meeting the needs of regional member countries. 3.2 As regards formulation of country strategies, the Bank’s approach primarily seeks to determine the country’s priority needs, to justify the selection of areas of intervention. This method consists in identifying all the negative aspects of the socio-economic context, such as weaknesses, constraints and lapses of all kinds, in order to seek remedy for them through its aid programme which is integrated in the development programme of the Government concerned. As a consequence of this
M634DOC.
5
approach, the diagnoses thereby obtained differ very little because poor countries generally suffer the same weaknesses and selection of sectors, which is justified on the bases of the needs to be met, invariably leads to very similar assistance programmes. This approach ensures the pertinence of the aid because it directly meets the countries’ needs, however it provides no reference points with regard to how good the strategy is for effective aid to reduce poverty, and in light of the country’s development potential. 3.3 Regarding project evaluation, the Bank’s approach consists in evaluating the performance of its country operations intended to meet such needs of a country as have been identified as priorities at the time of programming. The main purpose of these evaluations is to identify and explain the factors causing project implementation delays and affecting the quality of the outputs. The lessons from these evaluations are generally well-known, though they may not be acted upon, since they are recurrent and arise from difficulty and lags in meeting loan conditions, departure from procurement regulations or lack of counterpart funds and lastly from various problems in the operational procedures, including inadequate supervision, etc. 3.4 Results based management using the framework for monitoringevaluation of development results actually obtained by the country aid programme will make it possible to appreciate these outputs in terms of the evolution of the major challenges arising in the development of the country, especially poverty reduction, good governance in the management of public resources, protection of the environment, equitable distribution of income, gender balance and regional integration. It will then be possible to draw the appropriate conclusions on the effectiveness of the strategic orientation of the assistance programme, for a more optimal programming strategy. 3.5 The following inset shows precise areas of similarity and divergence between results based management and evaluation.
M634DOC.
6
Result based management (RBM) • • • • • •
•
•
4.
Evaluation of results by managers themselves Covers all the elements relating to management and responsibility Monitoring-evaluation, permanent performance evaluation system Obligation concerning results Model based on quantitative data Management information system based on results obtained or on the degree of attainment of goals objectives, expected outputs System of alerting managers concerning any aspects that are not going as expected (System of information by balanced score card) Importance of taking remedial measures in “real” time.
-
Evaluation • • • • • • •
Independent function (or conducted by independent evaluators) In-depth analysis of results, impacts and their attribution Temporal evaluation (discontinuous) Selective evaluation on specific aspects Importance of qualitative approach to result and impact assessment Importance of the evaluation answers the questions «why » and «how »the results were obtained Importance of lessons learnt and recommendations to managers for corrective measures and follow up.
THE MONITORING AND EVALUATION FRAMEWORK PROPOSED
4.1 The monitoring-evaluation framework proposed uses the results based logframe (LF) that describes the logical relation between the strategic components (objectives) of a project or programme, the chain of expected results and the incertitude with regard to their implementation (assumptions and risk indicators). There are two fundamental aspects in which this matrix differs from the traditional logframe. Firstly as the name indicates, the results oriented LFM goes beyond specification of the aims and objectives identified, and determines the results sought, not only at project level but at all levels of the Bank’s strategy (themes, sectors, countries). Furthermore, it is no longer a static instrument serving to schematically illustrate the internal logic of the project, rather a dynamic tool being used to hedge the critical assumptions (risks) and monitor the attainment of the expected results by measuring the level of attainment using previously established performance indicators 4.2 This twelve- cell matrix comprises three rows corresponding to the three levels of the chain of results comprising the results (outputs), the effects and the impact. It also has four columns; the hierarchy of objectives (goals, specific objectives, activities and inputs), the expected results (the chain of results), performance measurement (indicators), assumptions and risk indicators (conditions required).
M634DOC.
7
LOGFRAME MATRIX BASED ON RESULTS
HIERARCHY OF OBJECTIVES
EXPECTED RESULTS
Strategic aims/ Impact objectives
Operational objectives
Inputs (Resources and activities)
Effects
Outputs (Results)
PERFORMANCE MEASURE
Impact-related indicators
ASSUMPTIONS RISK INDICATORS Conditions for causal link between effects and impact
Indicators of risk in required conditions Indicators relating to Conditions for causal link effect between outputs and effects. Indicators of risks in the required conditions Conditions for causal link between inputs and the Indicators relating to activities and outputs Outputs Risk indicators in conditions (results) required
4.3 Even though it concerns the development of the aid provided, the results based management (RBM) requires that the desired objectives and results be specified beforehand, as part of the planning strategy of the operational departments, and at all strategy levels in the form of various strategic frameworks. In principle, a strategic framework would be needed to clarify the objectives and the expected results from the standpoint of the Vision and its thematic and sectoral components, and in light of the ultimate objective of poverty reduction. A strategic framework would be needed to clearly state the objectives and expected results of each of the sectoral and thematic strategy (good governance, regional integration, the environment, gender issues, rural development, human resource development, private sector promotion etc.) 4.4.1
4.4 Lastly, at the project level, in preparing the project document, there is need to prepare a results-oriented LF linked with the strategic framework of the country programme concerned and identify all the indicators for measuring performance, (quantitative or qualitative) as well as the risk indicators of each level of expected results. The table that follows illustrates the general framework for monitoringevaluation of RBM as applied to the strategic profile of country aid programs (the case of assistance to Mali is presented in Annex 1).
M634DOC.
8
General Framework for Monitoring Evaluation Applied to Country Aid Vision
Sectoral Strategies
Country Strategies
Expected Results
Result Measuring
Agriculture Agriculture and Rural development And rural rural development sector development policy
Strategic Objectives Upgrading of human resources Objectives : strategic (education)
Strategic Objectives (health)
Economic Integration Strategic Objectives
5.
-
Objectives: Strategic Sectoral Education Policy
Indicators of impacts of effects
Objectives: Sectoral Operational Sector: education
Impact Effects
Impact Effects
Indicators impact and effect
Health Policy
Objectives: sectoral operational Sector: Health
Objectives: Strategic sectoral
Objective: Sectoral Operational
Indicators impact And effects
of
Impact Effects
Water Management policy
Sector : Public Utilities
Indicators impact And effects
of
Impact Effects
Indicators impact And effect
of
Impact Effects
Objectives: strategic sectoral
Objectives: sectoral Transport policy Objectives: strategic sectoral
Objectives: Sectoral operational Transport sector
Objectives sectoral operational
IMPLICATIONS FOR STRATEGIC PLANNING AND OPERATIONS MANAGEMENT
5.1 Results based management requires the establishment of a continuous monitoring process in the form of a management information system (MIS) involving all the parties of the project. LFM will be transposed as a computer processed balanced scorecard to be regularly updated. On a regular basis, to be determined according to the type of indicator involved (quarterly, half-yearly or annual). Information generation and updating should be predetermined by the parties concerned. As to the executing agency, it should be clearly designated in the documents relating to negotiation and establishment of project implementation units. Concerning the responsibility of the ministries of the beneficiary governments, it
M634DOC.
9
should be clearly stated in the agreements with the government. Based on clearly set out conditions, all the project stakeholders should form a network (managers from the Bank, the executing agency, the supervising ministry, etc.) and each party should be enabled to monitor and update the indicator/s under their responsibility. 5.2 Whenever the indicators show poor performance, the parties will consult to determine the remedial action required. Thus responsibility for achieving results will be collective and the partners will be evaluated in relation to the results, on the basis of the quality and accuracy of their indicators. This approach will thus eliminate any inclination to hide poor performance for fear of being blamed. 5.3 The stage of design of the RBM and the monitoring-evaluation system are shown in the inset below.
Stages of design of the Results based Management model • • • • • • • •
6.
Formulation of the global objectives of the Vision Formulation of the strategic framework of the Vision Formulation of the strategic objectives of sectoral and thematic strategy documents (policies) Formulation of the frames for sectoral and thematic strategies (policies) Definition of criteria for monitoring and measuring results and the risks inherent in these. Definition of indicators for measuring results Management Information System (MIS) to be integrated in the SAP System Adoption and utilization of a «compensation/sanctions » system in function of the results.
-
Other Ingredients of results-based Management
• • • • • • •
Making managers accountable for their results Delegation of authority and reinforcing the manager’s decision making authority «Client »-oriented participatory management Involvement of partners and stakeholders in the result management process Reform of operational policies and procedures Reform of support structures Change of cultural organization.
CONCLUSION
6.1 OPEV thus plans to include in its work programme the preparation of a conceptual document providing the tools for a results-based management system at the Bank. This document will initially stem from the experience of the evaluation study on Mali and it will be produced in collaboration with other bilateral and multilateral development institutions that are already engaged in this type of experience. 6.2 As indicated above, the RBM system will be implemented in several stages. OPEV will in the first stage, assume leadership in the formulation of the required evaluation tools, as enumerated in the conceptual document, and contribute as a
M634DOC.
10
resource and supporting mechanism for the other stages of RBM implementation. OCOD in collaboration with OESU, will assume leadership in the second stage involving the systematization of strategic planning, through strategic frameworks linked with the Vision and the sectoral and thematic policies. Lastly, the country operations departments will, during the third stage, draw up strategic frameworks for the country strategies that will link the strategic objectives of the sectoral and thematic strategies (policies) with the objectives of projects selected, and specify the objectives and expected results as well as the risk and performance indicators for each of the projected result levels. The fourth stage will consist in rendering the permanent monitoring system operational in the form of a computerized MIS incorporated in the project Africa system, which would directly or indirectly involve most of the Bank departments. 6.3 OPEV is therefore seeking the views of the members of the Operations and Development Effectiveness Committee (CODE) on the issue arising in this paper from the obligation to report to Board members on the degree of implementation of the objectives and expected results of the Vision and the need to ensure continuous follow up in a measurable manner. 6.4 OPEV would further welcome such other observations and recommendations as the CODE members may make in order to move into detailed formulation of the conceptual model described.
M634DOC.
ANNEX I Page 1 of 2 Illustration: Strategic Profile of Programme of Aid to Mali, 1999-2001
M634DOC.
SECTORS
Rural Development
SECTORAL OBJECTIVES -Limit the impact of climatic hazards
OPERATIONAL OBJECTIVES (goals)
EXPECTED RESULTS
-improve irrigation and hydroagricultural development
-
-Enhance food crop production
-
-Reinforce food security -Improve the state of the environment -Diversify crops -Reduce food imports -Increase the production of the secondary food industry
-Process agricultural products into food products -Improve the quality of cash crop production
-
-Increase agricultural experts -Support rural sector reform
Social Sector
-Raise the population’s productivity and reduce poverty
- Reduce the rural/urban poverty indicator
-
Education
-Increase the schooling and literacy rate, especially for girls and women
-Develop basic education
-
-Improve technical and vocational training
-
-Modernize the education administration
M634DOC.
-Literacy education for the rural masses
-
MEASURING THE RESULTS
Irrigation farming (for export or inter-regional trade) Food self sufficiency for food crops (millet, sorghum, ..) Industrial investments concerning processing transformation (agroindustry) showing higher yields Development of the private sector (small and medium-scale enterprises) in the rural area Agricultural income increasing Provide underprivileged populations improved access to education, health and basic infrastructure.
-
-
Access of the population living below the poverty line to education, health and basic infrastructure
Increase in crude literacy training and enrolment Technical and vocational jobs Educational failure (dropping out) reduced Gross literacy and enrolment in rural areas (gender balance)
-
Gross education rate Success rate for basic education (gender balance) Internal and external efficiency of the education system Technical and vocational jobs created
-
-
-
Ha under irrigation Cash crop exports and inter-regional trade Nutritional survey and food security Tons processed and agroindustrial value-added Establishment of SME and number of jobs Agricultural income and expenditure on food
masses -Introduce non-formal training into adult education adults
M634DOC.
-
Adult literacy reduced
-
Rural enrolment rate (gender balance) Attendance of literacy programmes by adults
ANNEX I Page 2 of 2 Illustration: Strategic profile for 1999-2001programme of aid to Mali
M634DOC.
Health
Other Sectors Public Utilities
-Improve the health of the population and children in particular
-Improve the quality, coverage and use of health services
-
-Reduce the mortality and morbidity rate
-Upgrade the district health systems by using effective resources and organizing the population
-
-Lower the prevalence of infectious, parasital and nutritional diseases
-Increased meeting of potable water supplies
-
-Contribute to the balanced development between urban and rural areas and raise living standards through greater access to energy services
-Improve the urban and rural sanitation conditions
-
- Improve availability and quality of energy services, particularly in the rural area
-
-
-
Transports
-Increase agricultural marketing -Strengthen regional integration
-Open up agricultural regions through extension, rehabilitation and maintenance of rural tracks and roads. -Promote competition between the transport modes and enterprises
-
-
Reduction of infectious diseases, pandemics Population’s access to basic health centers Appropriate health coverage
-
-
Meeting the needs with regard to safe water Access to drinking water supply networks and urban/rural networks Decrease in waterborne infectious diseases Access to electrification and renewable energy in the rural/urban areas
Consumption rate per person and per day (25-40 l/day) Rate of connection to structured, and semistructured water and sanitation networks Cases of infectious diseases Urban electrification rate and that of other energy sources (biomass, wind-driven, solar...)
-
Regular supply of markets Value added due to agricultural produce marketing Rate of utilization of feeder roads Compression of transport cost (vehicle operation costs) Agricultural income
Increased flow of inter-regional trade in agricultural products Upgrading of agricultural products Price of remunerative agricultural products
-
M634DOC.
Cases of infectious disease Number of inhabitants per health center, per doctor, paramedical staff, per bed … Health map Deterrent charges Recurrent health cost/inhabitants
ANNEX 2 Page 1 of 4
Other development aid organizations’ experience with RBM (Extract from the document of the DAC working party on Aid evaluation 1)
The donor agencies with the most experience in results based management are: USAID (United States); DFID (United Kingdom); AusAID (Australia); CIDA (Canada); DANIDA (Denmark; UNDP; and THE World Bank. The donor agencies have gained the greatest experience with establishing performance measurement systems, providing performance data and external reporting on results. Conversely, there appears to be less experience with the practical application of performance information for internal management-purposes. Results based management processes take place at three key levels within the donor agencies. The first is the project level, which has been established the longest. More recently, efforts have been underway in a few donor agencies to establish country level systems within the countries receiving development aid. As to systems establishment at the third level, which is the corporate or agency-wide level, particularly for bilateral and multilateral agencies, it is perceived as increasingly necessary in many organizations, in view of the rising public pressure requiring annual reporting by all donors and multilateral development agencies on agency-wide performance and results with regard to the development impact. Performance measurement at the Project level The data collection methods have several common features and vary according to the level of hierarchy of the project logframe. The higher the level, the more time consuming the data collection operations are and the more complex and costly. These operations in fact tend to be less frequent and generally are not conducted by the project executing agency, rather at high levels of the donor institution and/or government of the partner country.
1
Summary of the experience acquired by development co-operation agencies, with regard to results-based management (Note to the DAC Secretariat DCD/DAC/EV(2000)3 du 07/02/2000).
M634DOC.
ANNEX 2 Page 2 of 4
The data on input, processes and results is most often compiled by the project staff and based on simple information systems that are frequently updated. The data on the project impact is generally collected at regular intervals (for example, each year) using rapid and inexpensive evaluation methods, mini surveys or consultations with the beneficiaries. Assessing long-term impact generally involves conducting costly surveys based on samples, or uses the existing data sources such as national surveys, census results, local administrative systems, etc. The data on the long-term effects and impacts are generally collected at intervals of a number of years, or else at the start and at the end of the project (ex-post). Data collection at the higher levels, especially the impact level, can not easily be expected of the project implementation agency. Generally speaking, the institutions consider that, to avoid distortions, there is need to adopt as holistic an approach as possible, with information on the different result levels. For example, to gear efforts solely towards the immediate result level could lead to doing well what should not be done. On the other hand, focussing efforts solely on the higher levels, in other words the «medium term effects» and the “longterm impact” could lead to a scarcity of basic data on activity and service monitoring and mediocre project implementation. Several development agencies have developed performance assessment systems using data from the monitoring and results evaluation systems. The managers judge the performances of their projects by a grading system based on a scale (for example, highly satisfactory, satisfactory, unsatisfactory) and several criteria (for example relevance, efficacy, efficiency, institutional impact, sustainability, etc.). These ratings and self-assessments are generally submitted to the top managements of the donor agencies in a standardized form and at specific points, such as the end of the project, or in the annual stage reports. The performance rating systems have the particular advantage of allowing for pertinent comparisons between the beneficiary countries or the sectors of intervention and aggregation of data for the institution’s entire project portfolio.
M634DOC.
ANNEX 2 Page 3 of 4
Performance Management at the Country Level A few donor agencies (particularly USAID and UNDP) have developed performance measurement systems for country programmes. This is a much more comprehensive and strategic performance management approach than the project approach. It focuses on a significant development objective of the country, usually a sectoral, subsectoral, or a crosscutting objective. Thus the unit of analysis is a country programme that typically includes many activities implemented by different donor agencies and partner organizations over a relatively long period. Strategic planning and performance measurement of country programmes is conducted on the basis of a results framework, which is a graphic display of the strategies necessary and sufficient to achieve a significant strategic development effort in the country concerned. It is made up of objective tree concepts that show the logical cause-effect relationships between activity outputs, intermediate results or outcomes and the strategic development objective. This framework is also a performance measurement tool that provides a structure for the evaluating or monitoring progress made as compared to progress envisaged. The system is used to alert managers when the actual results are not meeting the targets as planned, indicating the need for adjustments in the relevant project activities. The country programme results framework puts the higher-order development objective and intermediate outcome at center stage. It is less concerned with defining the individual project means (inputs/processes) and outputs than with measuring and achieving the higher-level results. It thus falls into a more long-term time frame, free from the confines of the project life cycles. The individual project activities tend to be less well defined in this approach, allowing for more flexible design and implementation. Moreover, in some agencies, management no longer approves the projects; the operating units in the field can shift course mid-stream, if performance monitoring information indicates that certain activities are not working well and therefore not yielding the set objectives. Performance Measurement at the Agency Level All the above-mentioned development institutions have recently developed and issued policy papers or strategic plans that clearly articulate the agencies’ overall mission and the key development goals or priority areas of intervention.
M634DOC.
ANNEX 2 Page 4 of 4
These statements about agency goals serve to explain to external audiences what the overall aims of the development assistance programme are, and provide a framework or structure for gathering and reporting data on overall agency results achieved. This has been useful as an internal management tool for strategic planning and concentrating the aid portfolios and resources within areas accorded priority in the institution’s strategic Vision. In some cases, agencies have elaborated their key goals into several sub-categories, forming a multi-level framework or hierarchy of objectives. These multi-level strategic frameworks serve to clarify even further what and how the agency aims to contribute. These hierarchies serve as detailed structures for reporting on agency results at several levels. For example, a typical three-level hierarchy structure might include agency goals, sub goals, and supporting programme approaches. Data for agency-wide performance assessments and reporting is collected from two basic sources: i) existing international sources/databases that maintain country level statistics on sector development trends, and ii) from the project/programme performance measurement systems maintained by the agency’s country operating units. Some agencies also integrate findings from evaluation reports. In most cases, these data are entered and stored in computerized central agency databases that facilitate analysis and reporting and reporting to the partners. Results of these agency-wide analyses of aggregate project/programme performance and results are usually reported in annual performance reports. In conclusion, in this era of the integral approach to development and joint poverty reduction programmes, results based management does have a significant role to play. However, attainment of a strategic long-term objective will evidently depend on the activities undertaken by a large number of stakeholders – for example different donor agencies, the NGO community and, of course, the authorities of the partner countries. Some of the tools developed for performance evaluation and strategic planning of country programmes should be particularly well adapted to the new modes of development based on the sectoral programmes jointly implemented by several donors and partner countries, and which provided a framework for harmonization of investments and activities in order to have common development objectives.
M634DOC.
