ådpe

Evaluation of Nordic Climate Facility Nordic Development Fund

15. June 2013 Erlend Sigvaldsen, NCG Stein Hansen, NCG Tore Laugerud, NCG

Contact address:

Nordic Consulting Group Fridtjof Nansens plass 4 0160 Oslo T +47 24 14 01 00 Email: [email protected] Home page: http://ncg.no/

Disclaimer: The views and opinions expressed in this report are those of the authors and do not necessarily reflect the official policy or position of the client, Nordic Development Fund

NCG - Evaluation of NCF

2

PREFACE The Nordic Development Fund (NDF) commissioned an independent evaluation of the Nordic Climate Facility (NCF) in September 2012. Nordic Consulting Group Norway (NCG) won the subsequent tender, signing the contract in January 2013.

The Evaluation started with a series of meetings in Helsinki the 15th of January, 2013, and was finalized by a presentation to the NDF Board the 11th June 2013. The findings and conclusions of the Evaluation are based on interviews with a large number of stakeholders, in-depth analysis of project portfolio data, administration of a web survey to all projects included on the shortlist of the 3 calls, and a field trip to Uganda and Kenya that managed to visit 7 out of the 26 NCF projects that are currently under implementation.

The Evaluation Team consisted of the following three senior NCG staff: Erlend Sigvaldsen (Team Leader), Stein Hansen (Climate Change Expert), and Tore Laugerud (Evaluation Expert). Zozan Kaya assisted in the analysis and presentation of the questionnaire/web survey data.

All conclusions are entirely those of the NCG Team, which would like to extend sincere thanks to all involved, for their unrelenting enthusiasm, friendly and frank discussions, and great hospitality.

Oslo, 15th June 2013

Stein Hansen

NCG - Evaluation of NCF

Erlend Sigvaldsen

Tore Laugerud

3

Acronyms AECF AusAID B4D BLCF BRIC CDM CORFO COWI CPI CSO DHI ECF EEP EUR FCG FDCF GBP GEF GREENPYME HIFAB IAP IFI ILO IWEM LDCF MDG MFA mn MoE NC NCF NDF NEFCO NSC ORGUT PoA PSD PSI PSOM SCC SCCF SEK SGP Sida SME

African Enterprise Challenge Fund Australian Agency for International Development Business for Development Business Linkages Challenge Fund Brazil, Russia, India and China Clean Development Mechanism Chilean Ministry of Enterprise Development Danish consulting company Climate Policy Initiative Civil Society Organisation Danish Hydraulic Institute Enterprise Challenge Fund Energy and Environment Partnership programme Euro FCG International Ltd. (Finish consulting company) Financial Deepening Challenge Fund British pound Global Environmental Facility Innovative energy efficiency program for SMEs under the Inter-American Investment Corporation Swedish consulting company Innovations Against Poverty International Finance Institution International Labour Organisation Institute of Water and Environmental Management Least Developed Countries Fund Millennium Development Goals (under UN) Ministry of Foreign Affairs million Ministry of Environment National Coordinators (under GEF) Nordic Climate Facility Nordic Development Fund Nordic Environment Finance Corporation National Steering Committee (under GEF) Swedish consulting company Programme of Activities Private sector development Private Sector Investment Programme Cooperation Emerging Markets Swedish Cooperative Center Special Climate Change Fund Swedish kronor Small Grants Programme Swedish International Development Cooperation Agency Small and medium sized enterprises

NCG - Evaluation of NCF

4

ToR UNDP UNFCCC UNOPS USD WRMA WSFT

Terms of reference United National Development Programme United Nations Framework Convention on Climate Change United Nations Office for Project Services United States dollar Water Resources Management Authority (Kenya) Water Services Trust Fund (Kenya)

NCG - Evaluation of NCF

5

Table of Contents

1. Summary

7

2. Introduction and Background 2.1. Methodology

13 14

3. The NCF Concept - and Main Issues

15

4. The NCF Portfolio of Projects 4.1. Disbursement and Project Status 4.2. Key Characteristics 4.3. The Web Survey 4.4. Portfolio Assessment

18 18 19 22 24

5. NCF Administration 5.1. Costs 5.2. Process Assessment 5.3. Administrative Options

31 31 35 44

6. Development Considerations 6.1. International Alternatives and NCF Value Added 6.2. Adaptation and Development

46 46 48

7. Strategic Analysis: NCF and NDF 7.1. Current Strategy 7.2. Key Recommendation: Harmonisation 7.3. Thematic Specialisation

51 51 53 56

ANNEX 1

Terms of Reference

57

ANNEX 2

NCF Milestones

63

ANNEX 3

NCF Projects

65

ANNEX 4

Web Survey findings

75

ANNEX 5

People Interviewed

85

NCG - Evaluation of NCF

6

1. Summary

The Nordic Climate Facility (NCF) was established in 2009, is funded by the Nordic Development Fund (NDF), and is jointly administered with the Nordic Environment Finance Corporation (NEFCO). The purpose of NCF is to build up innovative partnerships between the Nordic countries and NDF’s partner countries in the area of climate change. NCF works through a call procedure, where proposals are invited within particular themes. There have been 3 open calls so far, for which a total amount of EUR 18 mn has been budgeted. In February 2013, there are 39 NCF projects in various stages of implementation/completion in 17 countries. NCF was initiated as a part of the process of redefining NDF, after the organization got a new mandate within climate change. It represented a new business model for NDF. After almost 4 years of NCF operation, a number of issues have emerged: 

The place of NCF within the NDF strategic framework. contribute to NDF's overall objectives?



The administration is perceived to take an increasing amount of resources, with for instance some duplication of work between NDF and NEFCO



Uncertainty about the sustainability prospects of the projects, as the 2 year project cycle is a very short time to establish viability.

How does NCF

The Evaluation discusses these and other pertinent issues. However, the point of departure is the products of NCF, namely the portfolio of projects.

Project Portfolio There have been delays in project implementation in all the three Calls. Of the 26 projects in the first and second call, 5 had finished at the end of 2012, 1 was terminated, and of the rest, 13 are assessed to face delays. Discussions for Call 3 have been lengthy, with only one contract signed at the end of 2012. Disbursement has thus been relatively slow, with only 40% of contracted amounts in Call 1 and Call 2 being disbursed by January 2013. By far the most projects (in all three calls) are in Africa (64%), with Kenya, Uganda and Ghana being involved in 5 projects each. In Asia, Vietnam and Nepal are involved in 2 projects each. In Latin America Bolivia is home to 5 projects. Of Nordic partners, the largest group is the consulting companies with 36% of the projects, then research institutes with 26%, followed by NGOs with 21%. The largest group of projects deals with mitigation alone (41%), closely followed by pure adaptation projects (36%) - the rest being combinations. The Team undertook a comprehensive web survey among all Nordic partners that made it to the shortlist in all three Calls. Of those that eventually got NCF funds, 70% of the respondents indicate that impacts from the projects have already been detected, and around 75% believe their projects would likely be sustainable. Almost half of the respondents believe that their project comprise innovative technology/techniques, with NCG - Evaluation of NCF

7

1/3 claiming the projects are new in geographical and institutional context. More than 3/4 believe the partnerships have been a success. 70% of the successful applicants claim that their projects have led to leverage/spin-offs/scaling-up. Survey respondents give high scores on the quality of the NCF Calls and on the evaluation process, even the unsuccessful applicants do so. More feedback from the evaluation process is desired, though. NCF gets credit for being professional and forthcoming, but several say that less bureaucracy and more speedy handling procedures would be preferable. The Team has made its own subjective assessment of the 26 projects in Calls 1 and 2, with regard to the main evaluation questions. Innovation: The Team found only a couple of projects comprising really innovative and adapted technology. Most were however innovative in the geographic context, related to “roll-out/replication” of already known technology. The Team thought that around 60% of the project had good elements of innovativeness. Partnership: The element of partnerships in the projects is largely found to be good, with a few being inadequate, coinciding with the web-survey results. Impact (longer-term effects): It is too early to detect impact in most projects, but an estimated majority has a reasonable chance of delivering some impact. Sustainability: The sustainability is uncertain in many projects. Lack of finances for replication of successful pilots is imminent in most countries. Having a "regular", local partner with self-interest in the project increases the likelihood of sustainability.

Administration and Costs Total costs for administering the three calls of NCF was estimated at EUR 1,152,841, or 6.4% of total committed amount of EUR 18 mn. Of this, about EUR 715,000 has been spent at the end of 2012, which is 97.4% of the allocated budget. Actual administration costs by 31st December 2012 are thus 4% of total committed NCF funds, 6.4% out of contracted, and 16.5% out of disbursed amounts. These administrative cost figures are limited to NEFCO-expenses only. The NCF administrative costs compare favourably with a number of other call based schemes. Costs between 6 and 10% of total commitments seem "normal", with so-called Challenge Funds on the high side with between 20 and 25%. While NEFCO has the main responsibility for implementation and administration of NCF, there are some cases of overlap and duplication of work between NDF and NEFCO. This is most prominent in the monitoring phase. There are three key issues: 

A monitoring structure with two levels of decision makers, sitting in different institutions with different managers, is problematic. A project should relate to one implementation unit, and one only.



The funder - i.e. NDF - should focus on their higher level role, namely being concerned about the climate and development objectives. Ideally, the funder should only be active at the top of the monitoring pyramid.



The third issue, and the one that is at the heart of the two others, is that the partnership relation between NDF and NEFCO is contractually untidy. There should be a clear outsourcing contract, with NDF being the client and funder, and NEFCO the consultant and implementer.

NCG - Evaluation of NCF

8

What makes this structure especially tangled is that NEFCO is also present in the Management Committee that is the overall governing body of NCF. This blurs the lines of accountability and responsibility.. There are bureaucratic tendencies in NCF. Finding the right administrative balance between control and flexibility is challenging, because the portfolio is so very diverse. There are three main actions that can be considered for streamlining operations: 

Harmonise projects, meaning having projects that are more similar, in similar countries and within similar themes.



Accept a greater level of performance risk, by controlling less and be more flexible with regard to implementation.



Move towards a red-flag system in supervision and monitoring.

The main cost in NCF relates to staff, and the main driver of staff time is the number of applications and projects handled. To reduce the administration cost without compromising quality, the two key actions are to i) reduce the number of applications, the number on the shortlist, and the number of projects actually given grants, and ii) harmonize the projects to realize greater professional synergies in handling them. There are several options for future administration of NCF: 1. As today, with NEFCO as implementer. In that case, it is recommended that the roles of NDF and NEFCO are made clearer. The Management Committee should ideally consist of NDF and independent experts. 2. Tender and outsource the implementation contract to an external company. This is typically what most call funds within the private sector do. However, it is not likely to be less expensive, and NCF will be removed from NDF’s core business. 3. Implement NCF in-house in NDF. This will ease the integration of NCF in NDF, and it will allow greater strategic control over NCF. The main argument against is that the process can get too internal.

Development Considerations The Team has not identified any international financial mechanism that is entirely identical to NCF. While several programmes have common characteristics such as being challenge funds and/or call-based, they do not necessarily have common objectives and targets. NCF has international added value as a rather rare mechanism combining innovation, leverage and partnership. Supporting communities to become more resilient to future climate shocks - adaptation - will become increasingly integral to international development agendas over the coming years. With its mandate combining both development and climate change, NCF is potentially well situated to face the "adaptation challenge". In fact, NCF ought to increasingly focus on adaptation in the future precisely because of this twin mandate.

NCG - Evaluation of NCF

9

Strategic Considerations When assessing the NCF in relation to the existing NDF strategy, NCF cannot be said to be in conflict with, or otherwise working contrary to, existing NDF objectives and strategies. Further, the recent Vista Evaluation of NDF gives good marks to NDF - there is no strategic imperative that calls for changes in NDF's overall operation. The greatest strategic challenge to NDF from NCF is perhaps its uniqueness - that it is so different in size, character and administration from what NDF otherwise does. Most organizations need to have a clear strategic logic behind its actions. For the type of instrument that NCF is, there are 3 reasons why it should be included in NDF's tool-box: 1. The need for innovations within climate change facilities. Perhaps self-evident, but there is need for instruments that encourage innovations. The competitive element of NCF plays a crucial role in promoting new ideas. 2. The need to leverage private sector capital within climate change. There is no possibility for public resources alone to combat the effects of climate change. 3. The need for developing broad partnerships within climate change, involving other actors than the typical government and donor sector players. There is a need for lighter, more flexible partnership facilitation measures than the heavy handed, large global types of mechanisms. NCF can be made to work even better. The key word is "harmonisation".

Recommendations Several of the recommendations are long term, in that they can only take effect from the next call and onwards. This is in particular the case with what the Team considers to be the most important recommendation: NCF should "harmonise", at three levels: 1. At the national level, as there is a need to harmonise the NCF projects more with national priorities of the host country. It is recommended that NDF consults with national authorities before it chooses themes for the NCF calls. This will make NCF more demand driven, reduce the risk of overlapping with other aidsupported projects, and increase the likelihood of developmental value added. 2. At the portfolio level, as the diversity of the portfolio needs to be reduced, in order to improve synergetic knowledge and to improve in choosing, monitoring and assisting projects. This will likely reduce operational costs, and provide a better focus for what NCF is meant to do. This harmonisation is recommended done by reducing the number of countries in each call to 4-6, and by being more restrictive in the themes selected for each call. A maximum of 10 projects should be granted money in each call. 3. At the NDF level, as NCF could be linked more directly to other potential and/or ongoing NDF projects and programmes, . While the two first harmonisations - national and portfolio - are recommended, the last is only suggested as an option as there is a risk of losing flexibility. The reduced number of projects also implies a reduction in the total amount granted per call. The current average size of about EUR 415,000 per project seems about right.

NCG - Evaluation of NCF

10

Against this background of harmonization, it is recommended that NDF starts discussing a narrowing of thematic issues to be funded under the NCF mandate. There is currently an imbalance in available international funding for adaptation projects. This could imply that more emphasis should be put on measures to adapt to climate change. Further, NDF may consider also narrowing the thematic focus within the adaptation field, e.g. on gender-related aspects since women and children are the groups most vulnerable to the adverse effects of climate change. Other recommendations relevant from the 4th Call onwards include: 

The project implementation period should be increased to at least 36 months. The 2 year life span of the projects is believed to be unrealistically short, given the nature of the projects, and the countries in which they are implemented.



It should be assessed whether a cost efficient system for feedback to applicants can be put in place as part of the selection process. It is considered good development practice to offer transparency in such selection processes.



The quality of the due diligence - or appraisal - needs to be improved: o o o

It should take place in the initial stages when projects can still be influenced. It needs more depth and analytical rigor It needs to be done by experts that know the context, as the major unknown is often how the project fits in the local institutional, cultural, social and economic framework.

There are three additional suggestions to improve the sustainability of projects: 

Ensure proper institutional commitment of the local partner, including a real interest in getting the project to succeed. Being "just" a hired implementer seldom facilitates long term sustainability of the innovation.



Demand a higher portion of self-financing, which in many cases will also indicate better foundation for continued funding after the project is finished.



Introduce a post-NCF funding facility, which can provide guarantees or other semi-commercial funding to those projects that really show promise. This would improve the chances of NCF making impact on a larger scale, as promising projects that have yet to make a footprint is given the chance to develop.

There is then long to medium term recommendations: 

NDF needs to rethink the administrative arrangement with NCF, and how it wants to work with NCF in the future. The three main administrative options are listed above: As today, outsourced, or taken over internally. NDF needs to do a proper in-depth analysis of the different options. As NEFCO today is the contracting partner with the NCF-projects, a transfer of administrative responsibilities of the three first calls might not be feasible.



If NDF decides to continue with the current arrangement with NEFCO, it is recommended NDF assesses to reorganise the Management Committee, so that mainly NDF and independent experts sit in the Management Committee. It is only NDF that should be overall accountable for NCF.

NCG - Evaluation of NCF

11



Whatever administrative option NDF may choose for NCF in the future, there should be a clear a contractual framework in place that demarcate roles and responsibilities between NDF and the implementer.

Short term recommendations, which should be looked at immediately: 

NDF should go through all the current NCF procedures, routines, agreement templates etc, with a view to reduce requirements and controls. There should not be two organisations involved in monitoring and controlling, it clearly bureaucratises NCF. Further, the Team suggests that NCF gives itself more flexibility in the application of procedures. The recommendation is to focus more on the milestones and the achievements, and less on the activities and the details of operation.



NCF ought to do an end-review of a random selection of projects. This review should not be done by staff that has worked with the project. Such an endreview is necessary to a) factually assess the project, b) verify the milestones that the project says it has met, and c) draw lessons learned.



The Team recommends a move towards a red-flag system in supervision and monitoring. Regular reporting from projects to NCF should be as today, but NDF/NEFCO should only visit projects in the field that either experience serious delays, or are faced with particular challenges. We trust NDF/NEFCO staff to have the necessary analytical discretion to identify what projects that might be. Thus, if a project faces no particular challenge or deviation, it would be visited only 1 time - at appraisal/due diligence - and then possibly by an end review.

NCG - Evaluation of NCF

12

2. Introduction and Background

The Nordic Climate Facility (NCF) is funded by the Nordic Development Fund (NDF) and is jointly administered with the Nordic Environment Finance Corporation (NEFCO). The purpose of NCF is to build up partnerships between the Nordic countries and NDF’s partner countries in the area of climate change. It was established in 2009, as a new instrument in the NDF portfolio, to provide partial grant financing for innovative climaterelated projects in low-income countries in Africa, Asia and Latin America. The NCF is not linked to multilateral banks or other former cooperation partners of NDF. The key feature is the use of competition through issuing open public calls for proposals. Projects have to include a Nordic partner as well as a partner from a low-income country, and should be innovative within the climate-related theme that is chosen for that particular call. The successful bidders would receive a grant between EUR 250,000 and EUR 500,000 for part-financing of the project. The project implementation period is set to maximum 24 months. The first NCF call for proposals was launched in October 2009. It focused on water resources and energy efficiency. Of the 138 proposals received 14 were successfully contracted by the end of 2010. The second call was launched in 2010, and covered the two themes: Urban adaptation to climate change and Renewable energy. A total of 176 proposals were received, of which 12 were selected for implementation. A third Call was launched in late 2011 with a theme of Innovative low-cost climate solutions with focus on local business development. A total of 128 proposals were received, of which 13 have been selected for contract negotiations. Most of these have yet to be finalized. Thus, in February 2013, there were 39 NCF projects in various stages of implementation/completion in 17 countries. All in all, an amount of EUR 18 mn has been budgeted for the three calls. While still in an early phase of NCF, NDF wanted an evaluation of the results so far, with the overall objective being" ..... to provide NDF with an independent assessment of the achievements made by NCF and, in particular, to what extent the program has contributed to building partnerships between the Nordic countries and NDF’s partner countries on climate change adaptation and mitigation."1 During the preliminary discussions with NDF about the evaluation, a key assessment was added as an important objective: What is the "strategic fit" of NCF, within NDF's mandate and operations? Does NCF contribute to meeting NDFs overall objectives? These are key questions with regard to the future of NCF. Even if NCF is still a young instrument, it must nevertheless be able to show a positive contribution to what NDF is expected to deliver. Most of the Team's conclusions have that as a basic premise.

1

Ref. ToR in Annex 1

NCG - Evaluation of NCF

13

2.1. Methodology The ToR specifies the basic methodology to be followed, which is further outlined in the NCG's Technical Proposal to NDF. The key elements of the methodology include: 

Review of documents and data



Interview with NDF and NEFCO staff



Interviews with NDF Board members



A web survey of all projects that were shortlisted in all three calls, altogether 103



Review of the project portfolio based on information from NEFCO and NDF



International comparison with other programmes based on "call procedures", and a scan of similar climate related programmes



Visit to Uganda and Kenya, interviewing partners and stakeholders in 7 projects.

The NCF portfolio of projects is very diverse. NCF partners consist of NGOs of “all colours” (universities, consultancies, government agencies, private companies and suppliers) engaged in projects spanning from sophisticated modelling of climate changes in 2050, to the provision of plastic containers that clean water by the use of sunlight. All of this happens in 17 different countries at different levels - from the grassroots in urban slums to policy units in government ministries. There are not two projects that can be said to be "similar". This variety is a great aspect of NCF, as it opens the door to an enormous body of diverse climate and development experiences. However, for an evaluation that is supposed to synthesise and generalise, it does pose certain challenges. It is almost impossible to define common criteria that can be used to fairly compare a so diverse mass of projects. As discovered, "innovation" and "technology transfer" have different meanings amongst the partners. Thus, it is important to stress that the assessments of the projects are mostly qualitative. The Team members are not experts in all the thematic fields covered by NCF, and did not have time (or the mandate) for a deeper analysis of the individual projects. A second caveat for the analysis is that NCF is a young instrument, starting as recently as 2009. Only a few of the projects in the first Call have been completed so far, and there are thus few tangible results delivered from finalised projects. The ToR raises several issues that are long-term in nature, like sustainability, observable impact, and leverage/spin-of effects of the NCF. In some cases it is possible to offer tentative conclusions, but for most projects it would simply be unprofessional to try. To some extent the same goes for NCF as such, as NCF itself as a funding mechanism is still on the learning curve. However, after 3 years of active operations, there is a lot of experience that offers important lessons for how NCF can and should operate. In sum, the Team believes that the methodology gives sufficient analytical foundations to deliver recommendations regarding the future of NCF, including suggestions for changes and improvements. NCG - Evaluation of NCF

14

3. The NCF Concept - and Main Issues

When as many as 138 applications were received for the first call, it was a surprise to NDF as well as NEFCO staff. NDF had hoped that perhaps 50 applicants would send a proposal. Competitions of this type are rare in the development as well as the climate change universe, and there are only a couple of comparable examples among the Nordic countries. One of these, the DemoEnv (DemoMiljö) funded by SIDA, actually provided the direct inspiration for NCF. DemoMiljö invited proposals from institutions and companies in some 40 developing countries, for testing out new environmental technology.

Competition - a new idea? Competitive calls have been seen mostly in different types of private sector development (PSD) programmes around the world. The basic idea stems from the question of how to better engage the private sector in development. Private companies cannot be given direct grants as one does with NGOs and governments under current aid regimes, but one could utilize them to initiate new ideas by playing on their competitive instinct. A key objective was to leverage increased private capital to developing countries by the way of small public grants. DFID was one of the first to introduce the concept on some scale in the early 2000s, and established what is known as "Challenge Funds". These were closed funds with thematic focus like "Financial Deepening", "Africa Enterprise" and "Business Linkages". In that last fund, businesses in target countries could apply for cost sharing grants between GBP 50,000 and GBP 1 mn on a competitive basis.2 Other call based PSD programmes of some size included the Dutch "PSOM" (Programme for Cooperation with Emerging Markets) that is now rebranded as Private Sector Investment (PSI), and the German ”develoPPP.de”. The German programme includes several public partnership instruments, but one of them provides a maximum grant of EUR 200,000 to particular innovative approaches. In 2009, the tender themes included ”renewable energy”, ”energy efficiency" and ” vocational training”. The Dutch PSI is quite typical in that payment is made against the project reaching particular performance objectives. Submitted projects are judged, among other things, by the degree of local innovation. The innovative character can consist of a new product, a new production method, or new technology for the respective country. A very recent addition comes from SIDA, and their Business for Development (B4D) strategy. In it, they have a programme called Innovations Against Poverty (IAP). Partnerships will have to be specifically oriented towards developing products, services 2

"BLCF: Assessing Achievements and Future Directions", Deloitte, 2004.

NCG - Evaluation of NCF

15

and business models that can contribute to fight poverty and climate change. There are several phases, where SIDA provides maximum EUR 200,000 in grants through two tenders per year. The idea of competing for public resources is of course not new. Research grants have long been distributed by the way of Calls for Proposals. The rationale is the same, to invite companies to compete for support in the same way that researchers apply for funds from a research foundation, to encourage good ideas for development. What makes NCF special is that it is the first and - to our knowledge - so far the only call for proposal based partnership programme exclusively related to climate change. Otherwise, NCF shares many of the characteristics of the other programmes, i.e. the intention to foster innovation and technology transfer, to develop partnerships, and to use actual results as key criteria for payments. This is part of a general trend in the aid sector, towards focusing on measurable results and on impact-value for money

The Origin of NCF - and some of the issues currently discussed. NCF was initiated as part of the re-engineering of NDF after the institution came close to being permanently shut down in 2007/08. It served well as a novel feature of the reorientation of NDF towards climate change issues. As such, it appears as an important part of the process of redefining NDF. Innovation was an oft-heard phrase, and NCF was certainly innovative - not only because it aimed at innovative projects, but also because it was a new business model for NDF. Besides the need for a new type of innovative instruments, there were probably two other motives for NCF that played a role: 

NCF provides Nordic visibility, in that "the Grantee" has to be a Nordic institution. While all other NDF activities have no geographical restrictions, NCF ensures that NDF is still seen as an interesting organization for a large part of the Nordic development environment.



Nordic institutional cooperation, with the rescue of NDF as a Nordic institution happening in a benevolent Nordic atmosphere. Implementing NCF as a partnership between NDF and NEFCO seemed to be a very good solution, where both would contribute with their respective areas of expertise.

Now, almost 4 years down the NCF road, and after 3 different calls, a number of issues have emerged as the operation has matured. This is as could be expected. NCF was designed in a period of organizational renewal, where the strategic framework for NDF was largely undecided. The key issues as of early 2013 are thus assessed to be: 

The place of NCF within the NDF strategic framework. How does NCF contribute to NDF's overall objectives? This was not clearly spelt out at the start, but has become an increasing concern for NDF stakeholders. For instance, NDF prides itself of being among the most aid-harmonised donors in the world while NCF acts more as an aid-sector libero.

NCG - Evaluation of NCF

16



The administration is perceived to take an increasing amount of resources, partly due to an organizational overlap of roles in the decision-making and the administration of NCF, and of the different institutional mandates and backgrounds. But there are also concerns over the amount of bureaucracy, both at the level of NCF and at the project level.



Uncertainty about the sustainability prospects of the projects, as a 2-year project will barely have started before it has to prepare for closure. Most projects are anyway delayed, and in need of extension. The 3rd Call - inviting businesses - tries to address that question, as it is believed that a company/institution will have better incentives to stay in business after NCF goes home.

In the following, we will try to address these and other issues of NCF in the context of an Evaluation. The point of departure, however, is the products of NCF, namely the projects. How have they performed? And how do the project promoters themselves assess NCF?

NCG - Evaluation of NCF

17

4. The NCF Portfolio of Projects

The portfolio consists of projects from three calls. Annex 2 shows the most important milestones related to the three open calls in a timeline. The portfolio will be discussed through four sections. Firstly - disbursement and general status of the portfolio; secondly - a factual description; then the key results from the web based questionnaire; and finally the Team’s assessment of the portfolio in relation to the evaluation questions.

4.1. Disbursement and Project Status The projects are scheduled to be implemented over 2 years, and this has been a major complication for most project promoters. It has taken time to get organized and getting approval of all paperwork. The due-diligence visits have been a bottleneck, and the small staff of NEFCO and NDF has not always been able to prioritize an expedient visit.

Table 1 Disbursements - Call 1 and Call 2 End of 2010 End of 2011 CALL 1 Contracted Disbursement %

5,450,842 50,000 0.9%

EUR End of 2012

5,450,842 2,012,624 36.9%

5,450,842 3,220,340 59.1%

5,254,592 0 0.0%

5,254,592 1,115,629 21.2%

10,705,434 2,012,624 18.8%

10,705,434 4,335,969 40.5%

CALL 2 Contracted Disbursement %

TOTAL Contracted Disbursement %

5,450,842 50,000 0.9%

Call 3 seems to have stalled, with only one contract signed at the end of 2012, as much as 6 months after the NCF Management Committee approved the selection.3 Disbursement has thus been relatively slow, with only 40% of contracted amounts in Call 1 and Call 2 being disbursed by January 2013. The status of the projects in the two first calls are assessed as follows:

3

NEFCO have in their comments to the draft report informed that by end of March 2013 11 contracts had been signed.

NCG - Evaluation of NCF

18

Table 2

Project Status, January 2013 Call 1 Call 2 Completed 4 1 On time 2 5 Delays 7 6 Terminated 1 0 Total 14 12 Four projects from the first call and one from the second are completed. The terminated project is in Ghana, where the local partner apparently lost interest and there were nobody to carry the project along. Most Call 2 projects are in the midst of implementation, and the on-time/delay status may change as projects advance. However, as a conservative estimate, at least half the projects experience delays. Most of these delays are due to perfectly understandable reasons, as for instance problems in getting licenses or other approvals from public authorities, staff changes in the projects, external events leading to change in designs, and simply lessons learned during project implementation. A key challenge is the 2-year life span of the projects, which is believed to be unrealistically short, given the nature of the projects, and the countries in which they are implemented.

4.2. Key Characteristics In each call, there were two rounds of assessments: i) In the first round the applicants submitted a project concept paper only; and ii) In the second round the shortlisted applicants were asked to prepare full project proposals. The number of applicants in total, and successful applicants are shown in the below table: Table 3 NCF Applications and Contracts NCF Call Total no. of Total number of concept papers shortlisted projects NCF 1 138 33 NCF 2 176 40 NCF 3 128 30 Total 442 103

Total number of contracts 14 12 13 39

The number of concept papers exceeded expectations in NDF in the first call. The number of concept papers increased in the second call, where also the tender procedures were streamlined, whereas the number was lower in the last call. The main reason for this is most likely the relatively narrow scope of the last call. Annex 3 lists all the projects that signed (or will sign) a contract under NCF. The projects are distributed on Nordic countries as follows:

NCG - Evaluation of NCF

19

Table 4 NCF Project Promoters - Origin NCF Call Denmark Finland Iceland NCF 1 3 4 0 NCF 2 3 4 1 NCF 3 5 2 0 Total 11 10 1

Norway 1 2 3 6

Sweden 6 2 3 11

Total 14 12 13 39

The average grant amount in the first call is around EUR 390,000, increasing to a little less that EUR 440,000 in the second round and EUR 425,000 in the last round. The average grant for all three calls is EUR 416,000. The average project values for the three calls were around EUR 628,000; EUR 625,000; and EUR 786,000 respectively4, with an average value of EUR 680,000 for all three calls. This is an overall average grant rate of 61%. Table 5 Regional Distribution5 NCF Call Africa Asia NCF 1 NCF 2 NCF 3 Total

12 6 7 25

0 4 4 8

Latin America 2 2 2 6

Total 14 12 13 39

By far the most projects are in Africa (64%), with Kenya, Uganda and Ghana being involved in 5 projects each. In Asia, Vietnam and Nepal are involved in 2 projects each. In Latin America Bolivia is home to 5 projects. The distribution of projects between countries show that the bulk of the NCF countries are those where the Nordic countries have had, and still have, the most extensive bilateral development cooperation, which include the English-speaking countries in East Africa. The Nordic partners of the successful applications can be grouped as follows: Table 6 Type of Nordic Partner NCF Call Consulting NGO company NCF 1 NCF 2 NCF 3 Total

6 3 5 14

4 2 2 8

Supplier

Research institute

2 0 1 3

1 5 4 10

Other private ltd company 1 2 1 4

Total

14 12 13 39

The largest group is the consulting companies having 36% of the projects (43% in the first call), with research institutes having a share of 26%, followed by NGOs with 21%. It 4

5

It is noted that one large project of EUR 1,019,300 in Call 1 and one of EUR 3.281,550 in Call 3 significantly pulls up the average. Without this latter project, the average in Call 3 would be EUR 578,000. NDF countries are: Africa: Benin, Burkina Faso, Cape Verde, Ethiopia, Ghana, Kenya, Malawi, Mozambique, Rwanda, Senegal, Tanzania, Uganda, Zambia, Zimbabwe; Asia: Bangladesh, Cambodia, Kyrgyz Republic, Lao PDR, Maldives, Mongolia, Nepal, Pakistan, Sri Lanka, Vietnam; Latin America: Bolivia, Honduras, Nicaragua.

NCG - Evaluation of NCF

20

is noted that the research institutes’ share of the projects increased significantly in the second and third call, which might be attributed to the themes in those calls. For comparison, the following statistics are valid for all the shortlisted applications in the three calls (where consulting companies and other private limited companies are lumped together in one category): Table 7 Shortlisted Nordic partners NCF Call Misc. private Suppliers companies NCF 1 20 4 NCF 2 16 7 NCF 3 9 6 Total 45 17

NGOs

Research institutes 2 11 *) 5 18

7 6 10 23

Total 33 40 30 103

*) Including one municipality

Finally, the successful projects can be grouped as follows related to the nature of the project, whether mitigation or adaptation or a combination: Table 8 Distribution on Adaptation and Mitigation NCF Call Mitigation Adaptation Combination of M. and A. NCF 1 5 6 3 NCF 2 7 5 0 NCF 3 4 3 6 Total 16 14 9

Total 14 12 13 39

(The table is based on information provided by NDF, and not based on individual project assessment by the consultant)

The largest group deals with mitigation alone (41%), closely followed by pure adaptation projects (36%). Most of the combination projects are in the last call. When looking at the total number of shortlisted applicants, the distribution of mitigation and adaptation projects in the three calls, and in the three main categories of applicants (where all private companies, consultants and suppliers alike, are lumped together in the same category), are as follows: Table 9 Type of Partner and Adaptation/Mitigation - Shortlisted Projects NCF Call Mitigation Adaptation Combination and M. and A. NCF 1 NCF 2 NCF 3 Total %

Private

NGO

Research Private

NGO

Research Private

11 13 8 32

2 3 2 7

0 6 *) 2 8

8 6 1 15

4 2 2 8

2 3 2 7

51.6

30.4

44.4

24.2

34.8

38.9

Total

NGO

Research

5 4 6 15

1 1 6 8

0 2 1 3

33 40 30 103

24.2

34.8

16.7

100

*) Including one municipality - The percentages in the bottom line refer to share of measures related to the same category of applicant (e.g. the percentages in the light blue columns add up to 100%).

NCG - Evaluation of NCF

21

The table shows that mitigation projects form the largest group, constituting 45.6% of all shortlisted applications in all the three calls combined. It is interesting that most mitigation projects are proposed by private companies. Adaptation measures constitute 29.1%, leaving combinations with 25.3%. As compared to the successful applications, a slightly lower number of mitigation projects are successful and a slightly higher percentage of adaptation, with the same percentage of combination projects.

4.3. The Web Survey The Consultant undertook a web survey amongst all successful and shortlisted unsuccessful applicants in the three calls. Annex 4 presents the main findings of the survey and a separate enclosure to this report shows all the detailed answers and comments given by the applicants in the survey. The main results from the survey could be summarized as follows: 

The web survey was sent to the contact persons of 103 projects. The overall response rate was 41%, which is reasonable. The highest response rate was amongst the successful applicants in Call 1 and Call 2 with 69%, and the lowest amongst unsuccessful applicants in these two calls with 19%. The response is in line with the actual composition of the NCF portfolio, both with regard to successful and unsuccessfully projects. Of the unsuccessful projects, distinctly more Danes than other Nordics have answered the survey. Further, 67% of all respondents amongst the unsuccessful applicants in Call 1 and Call 2 were from consulting companies.



The results of the survey must be read having the following in mind: i) Among the unsuccessful participants, many obviously declined to participate. ii) In questions asking for promoters’ self-assessment of own achievement, one might expect mostly positive answers. iii) It is difficult for respondents to be critical even with hindsight (confirming the wisdom of his/her original decision).



Project achievements, impact and sustainability: All respondents were rating the achievements/deliverables so far positively, as compared to the planned results/outputs as agreed to in the agreements (milestones), and 67% answered positively on the prospects of delivering results according to plans. All respondents rated their projects at various degrees of being “successful”. 70% of the respondents indicate that impacts from the projects have already been detected (raised awareness; improved health, monetary savings, time saved, improved production, and even climate mitigation effects and reduction of CO2 emissions), with 30% detecting no impact so far. Around 75% believed the project would likely (or highly likely) be sustainable. Amongst the risks to sustainability were mentioned: lack of political will or change of political discourse, instability in project areas, corruption, and project being “forgotten” after completion. Only 12% had prepared an exit strategy, with 41% having done so partly.



Innovation: Almost half of the respondents believe that their projects comprise innovative technology/techniques (with 24% answering “very innovative”). 35%

NCG - Evaluation of NCF

22

claimed that their project is new in geographical context and the same percentage that it is innovative in the institutional context. 

Technology transfer: 53% believed that there had been a “significant” or “good” technology transfer in the projects. The technology transfer is basically understood to be on-the-job capacity building. Dissemination of lessons learned is planned done through e.g. media articles, sharing through case studies and sharing with stakeholders.



Partnerships: Around half of the successful applicants (Call 1 and Call 2) had not had project cooperation with their partner before NCF, and in 56% of these cases the Nordic partner had taken the initiative (with almost 90% amongst the unsuccessful applicants in those calls). 76% of the successful applicants in Calls 1 and 2 believed that there had been a “high” or “very high” degree of partnerships in the project, and 65% of these respondents claimed the partnership had been successful to various degrees. Around 70% of the successful applicants in Calls 1 and 2 answered that the partnership was likely to continue post-project. Amongst the unsuccessful applicants in those calls, 44% responded that partnership continued post-rejection. NCF has obviously been an effective catalyst in promoting partnerships.



NCF management performance: The successful applicants in Calls 1 and 2 rated the quality of the call/announcement and adjacent documents largely positive (30% "very high"). Amongst unsuccessful applicants in those calls, 44% believed the quality had been “high”, with 33% rating it negatively to various degrees. Regarding the evaluation /selection process, 82% of the successful applicants rated the quality on the positive side. Several unsuccessful applicants criticized the lack of feedback on the scores and the rejection, with further criticism related to the longevity of the process. On the administrative procedures and processes in NCF almost 60% of the successful applicants in Calls 1 and 2 gave positive feedback. Comments varied from praising NCF of being professional, to claiming that the NCF procedures are bureaucratic. On the management performance NCF staff were commended for being competent and committed, with good understanding of the problems. Some respondents point to the lack of pre-financing making it difficult for smaller institutions to work with NCF.

The following figure shows how the successful respondents rated the administrative procedures.

NCG - Evaluation of NCF

23

Figure 1 NCF Administrative Procedures

Other: 56% of the unsuccessful applicants in Calls 1 and 2 claimed that their rejected application had not been financed by other sources, and the rest that it had been partly financed by others after NCF rejection. 88% of the successful applicants rated NCF more “useful” than other financial mechanisms, with similar response to the topic of NCF adding value to other mechanisms. 71% of the successful applicants in Calls 1 and 2 claimed the project had led to leverage/spin-offs, more projects, scaling-up or other synergies, beyond what was planned for.

4.4. Portfolio Assessment While showing some interesting results, the web survey findings need to be assessed in relation to two other sources: 

Reports, data and interviews from/with NDF and NEFCO staff in Helsinki.



The field visits to 7 projects in Kenya and Uganda, from all three calls (5 from Call 1, and one from Calls 2 and 3 respectively).

It should be noted that the Team only received one agreement/project proposal from Call 3 (the only one signed at the time of the Evaluation). This means that the assessment below is for Calls 1 and 2. The Team has attempted to assess the three main elements of NCF at large: Innovation, technology transfer and partnership. Additionally, the probable sustainability of the projects is elaborated upon, in addition to the expected impact of the projects. It is impossible for the Team at this point in time to objectively assess the impact of most projects - even those visited in the field. A one-day visit is not enough to verify impacts.

NCG - Evaluation of NCF

24

Innovation In the NCF guidelines, innovation is defined as: ”diffusion of technologies and practises that are new to a given context”. It is noted that in very few projects the technology is totally new, whereas in most projects there are elements that are new to the local context/specific geographic area, or climate change components are added to the local practises as new concepts. The below table summarises the Team’s assessment of innovativeness in the NCF 1 and 2 projects. The projects are characterised as having good, medium or inadequate innovativeness, with ¾ of the projects assessed through the project proposal/project reports, and ¼ assessed through field visits. The innovativeness rating of each project was based on an aggregated assessment of the elements of technology, geographical context and institutional/procedural context. Table 10: NCF Call NCF 1 NCF 2

Assessment of Innovativeness Good 7 9 Total 16

Medium 4 2 6

Inadequate 3 2 5

Total 14 13 27

The Team rates almost 60% of the projects to have good elements of innovativeness, with only around 17% as having “dubious” innovative elements. Most of the innovativeness is related to the geographical context, with projects introducing existing methodologies or technical solutions to new countries, and new settings. Some of this is truly new, as the Fuel Efficient Stoves project in Uganda, that establish a CDM Programme of Activities (PoA), that is really an umbrella within which different smaller projects (stove suppliers) can join the CDM market. The project tries to introduce this on an East African regional scale, which is clearly an innovation. The Gaia Consulting’s project in Bolivia is another such innovation in that it aims to redirect remittance flows to renewable and energy efficient appliances in a way nobody has tried before. It may not work, but it is an interesting try. Other projects, however, appear more to be roll-out of existing "infrastructure" technologies in new countries, which are perhaps not that innovative. Examples include the DHI project in northern Ghana - that is now terminated - and FCG's Nepal project. The energy audits in for instance Uganda/Rwanda and Nicaragua can also be questioned from the perspective of being real innovations. According to NEFCO, these projects were assessed to have the potential to diffuse technologies and practices that are new to a given context. For some technologies there are good reasons why they have not been introduced before. The Solvatten containers at the full price are too expensive to be affordable in Kenyan markets. It is a quite well known technology - and an excellent one - but it relies on donors to subsidise the price. There is thus in our opinion limited innovation in the project. Other technologies like the Green Resources’ charcoal plant score in our opinion well, as a new type of clean charcoal technology that has not been known in East Africa. Green Resources has also had to adapt the plant to the Ugandan context. NCG - Evaluation of NCF

25

Only some of the projects are introducing really new technological solutions, as far as the Team can see. This is contrary to the answers in the web survey and again might indicate that the project promoters are perhaps slightly too enthusiastic about the innovativeness of their own projects. In sum, the Team will score the innovativeness of NCF portfolio as medium/good. It is also a particular skill to detect the real innovators, and NCF appears to be doing better in the second call than in the first. As NCF notes in their Annual Report 2011: “As the key players tend to produce high quality, but sometimes “mainstream” applications, that typically score well. Emphasising previously demonstrated sector and country experience of the applicants in the evaluation, may to some extent conflict with the target of receiving innovative projects from applicants with new ideas". This is very relevant observation that needs to be fully taken onboard in future calls. It is not in the Team’s mandate to elaborate on semantics regarding the use of the term “innovation”. However, it is disputable whether introducing “new” (but “old”) concepts/technology in new geographical areas is real “innovation” or rather replication/roll-out of already well-proven and functioning technology and techniques. The NCF should consider distinguishing between these various types of innovativeness in future calls.

Technology Transfer The degree of technology transfer is difficult to assess by just reading the project proposals, as in several cases this is not properly outlined. Again, it is based on a subjective assessment of what has been stated in the proposals, the progress reports and what can be “read between the lines” where such descriptions are inadequate. Table 11: NCF Call NCF 1 NCF 2

Assessment of Technology Transfer Good Medium 7 5 4 8 Total 11 13

Inadequate 2 1 3

Total 14 13 27

As seen, less than half of the reviewed projects (around 43%) could substantiate a good description/strategy for technology transfer. This does not necessarily mean that the projects are not implementing adequate training and technology transfer activities. On the contrary, experience from similar projects in developing countries shows that there are always elements of capacity buildings, but that they are often under-reported and/or under-explained, as they are simply taken for granted, being implemented without saying. The mere fact that the partnerships are rated highly might indicate that transfer of technology is also good, amongst well communicating partners. Technology transfer is often primarily a question of time and the breaking of old habits of how "things are done". This is particularly the case if projects work with those that in development jargon are being "at the bottom of the pyramid". It is in most projects much too early to say whether new ideas have really diffused into practical action. The 2-year project period is considered as an impediment to effective technology transfer.

NCG - Evaluation of NCF

26

Partnership The partnership question is difficult to assess fairly without a visit and interviews with the partners in question. Most partners do not report internal difficulties unless the cooperation has gone completely sour. After the field visit and speaking to local partners, there is an impression that available reporting is perhaps too positive with regard to the partnerships. Of the 6 projects in Calls 1 and 2 visited, 3 seemed to have reasonably well working partnerships: Norges Vel and Green Resources projects in Uganda, and ORGUTs project in Kenya. In both of those 2 last projects, there is a strong local presence of the Nordic companies. The fourth, involving VI Skogen, also seemed OK, but there is little evidence of the Nordic partner - the project is run locally. The two last are assessed as more problematic, however. There is an apparent breakdown of trust between the partners in the CARE project in Uganda, and there seems to be unresolved issues between the partners in the Solvatten project. Some of the partnerships appear to suffer from: 

Lack of real interest from the Nordic partner, some of which applies for funds more as a professional routine than of particular interest to promote a new idea. The NCF Annual Report 2011 says for instance: "It seem evident that limited role and ‘ownership’ of a Nordic partner has led to delayed reporting"



Suppliers and consultants that may primarily be interested in selling their products and services, and partnerships may then be nominal in reality. It should be stressed that there are large variations, with some working hard to get projects moving, while others seem less inclined to do so.



Weak local partners, that may have overstated their true capacity to implement. Local political and contextual issues may also complicate partnerships more than anticipated.



Vulnerability to individuals, meaning that several partnerships depend crucially on particular persons. When either these persons change jobs, or when the relationships between these key persons become difficult, it can seriously impair progress.

Most of these issues are well known to any development partner, and there are seemingly not more problems with NCF partnerships than what most other similar Call programmes’ experience. About 2/3 of the respondents to the web survey rate their partnership positively. There are inbuilt tensions in any partnership, and in most cooperations these are solved by the constructive acknowledgement that both need the other to deliver the promised output. Thus, in sum, NCF rates "medium/good" in terms of partnerships, with perhaps half having laid good foundations for longer term cooperation. However, the 24-month project period is not normally sufficient for good partnerships to mature, and may instill a transitory sentiment in the whole enterprise. This is another argument for increasing the project period to at least 36 months.

NCG - Evaluation of NCF

27

Impact Each project lists likely impacts from a successful implementation, and these may well happen. The Team does not have any better estimation of impacts than what has been offered in the Annual Report 2011, and in the NCF Quarterly Reports. This does not imply that there are no impacts. There are good indications in several of the projects the Team visited that projects have delivered both results and outcomes, and many of the impacts listed in the milestone reports are plausible and credible. However, doing a reliable impact analysis requires a more comprehensive study than what this Evaluation can offer. Apart from the fact that few projects have finished, most of them miss a concrete baseline from which to assess change, and any impact analysis would have to assess the counterfactual. The net mitigation effects are not always easy to calculate, for instance where machinery is imported. Thus, impacts cannot yet be measured with any certainty for any NCF project. It is, however, easy to be charmed by the multitude of different results that the projects enthusiastically present. One of the chief positives of the NCF concept comes into focus, namely the entrepreneurship, being the overflowing of ideas and the many modes with which different organizations approach climate change and development. This is in itself a valuable result, as fresh ideas are a scare resource. NCF contributes to spreading of knowledge, capability and new-thinking within the climate area, and while this impact may be difficult to measure, it may be the most constructive result in the longer run. For mitigation projects, given that they are mostly small scale, greenhouse gas reductions are typically modest, having from a few hundreds to a few thousand tonnes of CO2/a reduction. They may be more pronounced if later scaled up. Many of the target countries generally use very little energy per capita, and have comparably low CO2 emissions in their electricity generation, so the potential for substantial reductions are for some quite limited. All mitigation projects can generate some adaptation benefits – typically by reducing deforestation. For adaptation projects, impacts are even harder to identify. It is very challenging to measure results in terms of increased preparedness for long-term climate change or infrequent extreme events. What can be looked at are intermediate outcomes, like improvements in information, capacity and reduction in exposure. Adaptation is also very context specific - challenges in the Andes and in Bangladesh are very different. NCF has a reasonable scoreboard on such intermediate outcomes. A number of projects have provided increased knowledge and contributed to improved climate change preparedness, like the COWI project in Mozambique, the Diakonia in Bolivia, and the ORGUT project in Kenya. Expected adaptation benefits are capacity building of the communities and partners, understanding of potential adaptation measures, management of climate-related disasters, and facilitation of better planning for infrastructure and better preparedness for flood disasters. There are examples of low likelihood of impacts also. For instance, NDF analysis of reporting from certain NCF projects questions whether these projects will have any discernible impact. These may be cases of simply misunderstood reporting from the

NCG - Evaluation of NCF

28

partners in question, but the concerns raised by NDF appear to the Team as very relevant. Thus, for some, impact may be less visible than planned for. Most good adaptation projects tend also to be good development projects, in that they husband and caretake natural resources in some form or other. The Call 1 projects within water would almost all be justified in a typical bilateral donor funded water resource management programme. Many of the factors that shape vulnerability and resilience to climate change have little to do with climate and more with development. These include issues of power, access to information and services, and control over resources. All are important in determining people's capacity to adapt to climate change and to engage in development. In sum, a number of impacts of NCF projects are claimed, of which many are plausible. It is noted that in the web-survey, almost 70.6% of the respondents in Category 1 (successful Calls 1 and 2) stated that some impact had been detected. While hard evidence for the whole portfolio is yet to come, the Team concludes that there are good chances that NCF will see positive development and/or climate impacts in the majority of it's projects.

Sustainability Several of the NCF projects may see their efforts sustained after closing the project. Because of the short implementation period, the main determining factor is the institutional, i.e. that there are institutions that continue and adapt the "NCF idea" after closure of the project per se. Project like the one COWI operates in Mozambique that involves local government is a case in point. NDF staff informs us that local authorities will now replicate the NCF financed GIS-tool to 4 other provinces. The organizationally complex ORGUT project in Kenya, working with the Water Authorities may not replicate a particular idea, but the studies and the strategies made under the project will be of use in their continued operation. The Demand Side Management principles that HIFAB has taught the Ethiopian Energy Authority may end up not being applied, but the knowledge remains in an existing institution to be developed. The only real private sector project in Calls 1 and 2, namely Green Resources Charcoal project, is a similar example. As long as the technology ultimately operates profitably - for which there apparently is every chance - it will be sustained. Perhaps selfevidently, but having a "regular", local institutional partner with self-interest in the project increases the likelihood of sustainability. The financial question is troubling for many of the projects, even those that continue in existing organization. How to find money to do necessary investments, or indeed, how to just fund sustained operation? Some, that have based the business idea on CDM income - there are not many - find it difficult unless they target the voluntary CDM market that pays better. The Carbon Unit in Uganda does that. Several other projects are implemented by local NGOs that work with local communities, like for instance at the Mt. Elgon in Kenya. If not alternative funds are found to continue the project, it will close. Those farmer groups that have been taught new technologies may still practice them, but that is not at all certain after only 2 years of demonstration, and the dynamics of change may be broken in the area. NCG - Evaluation of NCF

29

Other projects have never been meant to last. The recommendations from the energy audit of the tea factories in Uganda and Rwanda may perhaps in time be implemented in a sustainable and climate-beneficial manner, but internal NDF analysis raises important questions as to whether that is likely. The Solvatten containers have been mentioned above, but their key challenge is one of affordability among the target group. As far as the Team has been able to understand, the actual cost of the imported Solvatten unit is far above what it was actually sold for in the NCF funded project. Demand was slow, even at that reduced price. This example is noteworthy for one particular reason: Brilliant technology is not the same as appropriate technology. Technologies must be adaptable, and be able to survive in the markets of a developing country. Subsides may be warranted in a start-up phase, but there must be a realistic chance of it surviving on a commercial basis afterwards. Of course, sustainability might be improved if Solvatten transferred the technology and the production in full to a local producer. In general, the Team concludes that the sustainability of most projects is uncertain at this stage. The reason is that sustained efforts depend on sustained external finances and/or local subsidies, especially in Calls 1 and 2. Call 3 focuses on local business development, and the mere nature of these projects would indicate that there may be a better chance of sustainability than in the other two calls. The web-survey showed a more positive picture, where a total of 76% of the respondents answered that sustainability was highly likely or likely. Sustainability is not necessarily secured through capacity building, interest, or good communication during the project period (although the elements form a good basis). Normally, the lack of financial resources (and national priorities) in developing countries is the decisive factor. This is a concern in the comments in the web survey. The lack of financing is also closely linked to lack of political will and ownership to the projects, in an environment where all good purposes are screaming for financial support. There are four suggestions the Team will make to improve the chances of sustainability: 

Ensure proper institutional commitment of the local partner, including a real interest in getting the project to succeed. Being "just" a hired implementer seldom facilitates long-term sustainability of the innovation.



Increase the period of implementation from 24 months - few ideas are proven in just 2 years.



Demand a higher portion of self-financing, which in many cases will also indicate better foundation for continued funding after the project is finished - in particular if the project is successful. This may increase the number of private sector companies involved. Introduce a post-NCF funding facility, which can provide guarantees or other semi-commercial funding to those projects that really show promise. This would improve the chances of NCF making impact on a larger scale, as promising projects that have yet to make a footprint is given the chance to develop.

NCG - Evaluation of NCF

30

5. NCF Administration6

The initial administrational set-up was based on the premise that NCF would be a collaborative effort between NDF and NEFCO. NEFCO was to act as an implementing agency on behalf of NDF, carrying out the bulk of the implementation, administration and monitoring work, and also contribute in terms of climate-related technical input. NDF would participate directly a) in the assessments of received applications, b) in the monitoring of progress, c) by reviewing NEFCO’s recommendations, plans and reports, and d) in the Management Committee that is the overall governing body of NCF. The assumption was that NEFCO and NDF had complementary expertise, which together would realize synergies when combined. NDF knew development and the recipient countries, while NEFCO was expert in climate mitigation related issues, and with good project management experience. With the benefit of hindsight, it seems that the roles and responsibilities of each was not sufficiently spelled out and never fully clarified in the guidelines. One apparent consequence is that the two organizations have done a fair amount of double work. This is particularly pronounced in the monitoring phase, with the two organizations checking and rechecking each other's work. This is not necessarily a problem, as it can be part of a healthy set of checks and balances. However it is an issue of costs, and whether resources spent on "double work" are well spent resources. In the following analysis, descriptions of administrative routines will only be made at a cursory level, and only when it is relevant for the analysis. It is assumed that readers are familiar with the main procedures.

5.1. Costs The initial estimate was for 6.1% of the overall NCF budget to be reserved for administration, for the first year's operation alone. The budget for operating costs was later adjusted to reflect 1.3 person-years of NEFCO professional time and associated travel and meetings costs per year, which in 2011 amounted to approximately 6.2% of the approved project amounts. The actual costs in 2009/2010, 2011 and 2012 have, according to the NCF Quarterly Reports, been as follows:

6

With regard to the organisational terminology in this chapter, we use "NCF" when both NEFCO and NDF have been involved, and "NDF" and "NEFCO" when it is clearly only one of them that has been involved.

NCG - Evaluation of NCF

31

Table 12

Administrative Costs of NCF, 2009 - 2012 2009 & 2010 2011 Staff 174,900 175,576 Travel 15,438 23,332 Marketing 38,677 5,903 Consultancy 0 832 Other 379 4,890 Total 229,394 210,533

2012 223,355 28,834 9,775 12,004 712 274,680

EUR Total 573,831 67,604 54,355 12,836 5.981 714,607

The cumulative costs are 97.4% of the allocated budget, and about 4% of the total granted amount of EUR 18 mn at the end of 2012. The main cost element by far is personnel costs, with about 82% of total expenditure going to staff and consultancies. The numbers in the table do not include any costs borne by NDF. These are difficult to quantify, but there is undoubtedly considerable input from NDF: Country managers are involved in evaluation, several NDF staff participate in the working group, there are site visits, and others are involved ad hoc e.g. regarding legal concerns, newsletters, etc. In general, comparing NCF administrative efficiency over time from a cost efficiency perspective is challenging because NCF started from scratch in 2009. A substantial part of the initial effort was simply establishing routines and processes to handle the calls. What further complicates efficiency comparison over time is the change in what kind of projects the three calls invited. The third call appears to have required much more processing time and administrative effort than handling Calls 1 and 2. In short, the type and complexity vary significantly between projects and NCF calls. In August 2011 NDF submitted administrative cost estimates for the three calls of NCF to the Board. These estimates included all administrative costs for NEFCO since 2009, and estimates for assumed completion of the projects in these three calls in 2015. Total costs for this period was estimated at EUR 1,152,841. As shown in the table above, about 62% has been used at the end of 2012. To give an indication of cost effectiveness, actual costs are measured against committed (EUR 18 mn), contracted (signed contracts), and disbursed funds in the table below. Table 13

Cost Ratios 2010 - 2012 EUR End of 2010 Committed 6,000,000 Contracted 5,450,842 Disbursed 50,000

Cumul. adm. cost

End of 2011 12,000,000 10,705,434 2,062,624

End of 2012 18,000,000 11,105,434 4,335,969

229,394

439,927

714,607

3.8% 4.2% 458.8%

3.7% 4.1% 21.3%

4.0% 6.4% 16.5%

Adm. cost as % of Committed Contracted Disbursed NCG - Evaluation of NCF

32

These should be interpreted carefully, due to the caveats mentioned above. Initial establishment costs explains much of the apparent "front-loading" of costs. The ratio of "contracted" is possibly the best indicator, as this to a larger degree than the ratio for disbursed, hinges on NCF's own effort. The disbursement figure is heavily influenced by project performance, and it does signal that the portfolio experiences delays. While only 26% of the total committed amounts (minus the funds for administration) have been disbursed, NCF has used 62% of the administrative costs budgeted. Purely speculative, as we simply do not know, but adding staff costs from NDF to the tune of EUR 50,000 per year (about 5 months), the ratios would be as follows: Table 14

Cost Ratios 2010 - 2012, with NDF estimate

Cum. adm cost w. NDF

End of 2010

End of 2011

End of 2012

279,394

539,927

864,607

4.7% 5.1% 558.8%

4.5% 5.0% 26.2%

4.8% 7.8% 19.9%

Adm cost as % of Committed Contracted Disbursed

Compared to the total committed amount of EUR 18 mn, the "with NDF" scenario would calculate at about 8.1% for the full period to 2015, as compared to the 6.4% budget with only NEFCO. If NCF manages to keep the administration of the 3 calls within that budget, it compares quite favorably with a number of other call-based schemes.

International Cost Comparisons Comparisons with other funds are useful, but need to be treated with caution; the overall size of the fund, the number of countries covered, local costs and capabilities, and any differences in the objectives of the funds and their target audiences, all can have a significant impact on costs and any resulting ratios. A similar programme to NCF is the Swedish DemoEnvironment Program7 (DemoMiljö), which could be described as an attempt to reach environmental development cooperation targets by utilizing the capacity and market incentives of the private sector. By January 2010 some 400 applications have been received by the programm during six half yearly recurrent calls, and 79 projects being supported in 30 different countries. Projects are in general delayed, and disbursements are only SEK 28 mn out of in total SEK 83.6 mn. The DemoEnvironment Program displays interesting similarities to NCF. Administrative inputs over the 3-year pilot phase has been 5.8 person-years, or slightly below 2 person-years per year. It is estimated that these costs amounted to 21% of disbursed DemoMiljö funds during the 3-year period. In other words, quite comparable to the administrative performance of the NCF.

7

Anna Jennervik, Jens Andersson and Bo Adel (2010), « External review DemoEnvironment Pilot Programme 2007 – 2010”, Reference 2007-000667,

NCG - Evaluation of NCF

33

Another similar Swedish program is DemoÖst which formed the model for DemoMiljö. DemoÖst is administered by Sida. The programme experienced considerable project development costs. At the time when it was reviewed8 it had a ratio of administrative costs to disbursed funds of almost 20%, but this ratio dropped to 9% at the time of Sida’s final evaluation. Similar programmes reviewed by ILO9 have displayed significant variation between 18% and 57% in the ratio of administrative costs to disbursed funds. The “Challenge funds”10 mentioned in Chapter 3 are call-based. The basic principle has been that private enterprises compete for support to develop sustainable business concepts. Denmark, UK and Australia have such Challenge Funds in place. The most comprehensive evaluations that we found of these funds are the one of the Australian AID Enterprise Challenge Fund (ECF).11 The ECF follows a standard Challenge Fund application process, which is not that different from the NCF approach. There are three “Rounds”, which, in turn, have two stages: Concept notes and full applications for those shortlisted. A key difference from the NCF is that it is a fully independent panel that selects the final projects. The table below compares ECF with three other large regional/international challenge funds, all funded by a.o. DFID. Table 15: Comparing the Costs of Implementing Challenge Funds Challenge Fund % of Budget for Total Grant Fund Implementing the (USD) Fund ECF AECF BLCF FDCF

27% 20% 20% 22%

14,500,00 (Aus$) 40,000,000 16,600,000 32,000,000

Coverage

9 countries 54 countries International Africa and South Asia

Sources: ECF Management Statistics; Correspondence with AECF Fund Manager; BLCF and FDCF reports .

The Africa Enterprise Challenge Fund (AECF) is the largest, and the management contract is limited to 20% of the fund size. The AECF Fund Manager confirmed to the ECF review team that such a limitation is hampering their ability to deliver what they are tasked with. The GBP 18.5 mn Financial Deepening Challenge Fund (FDCF) was launched in April 2000 to extend sustainable financial services to poor people and to enterprises that employ poor people. It was completed in 2008. DFID’s Business Linkages Challenge Fund (BLCF) supported the development of viable new economic activities by linking small and large enterprises in developing and developed countries. When comparing the four challenge funds in the table above to the administrative cost ratio of NCF, NCF comes out favourable with a much lower ratio. The Dutch PSOM referred to in Chapter 2 was a similar enterprise partnership fund, based on competition, and on payment when certain milestones in the projects are met. The administrative cost was limited to 7.5% of the total fund disbursed. 8

Johansson, Jan (2006), «DemoÖst slutrappor»t, Sida Tamburn (2008), «The 2008 reader on private sector development», ILO, Geneve. 10 See Tamburn (2008), op.cit. 11 "Enterprise Challenge Fund – Mid-Term Review", November 2009, Springfield Institute 9

NCG - Evaluation of NCF

34

The Swedish Chilean Development Fund (Utvecklingsfonden Sverige Chile) was established in 2002 and lasted until 2010 to promote cooperation between Chilean and Swedish small and medium sized enterprises (SMEs). It has been administered by the Chilean Ministry for Enterprise Development (CORFO). The ratio of administrative costs to disbursed funds reached 37%, which is considered high. GEF has been mentioned as the most comparable to NDF in terms of mandate. A priori one would expect NDF, and even more so NCF, to experience a higher administrative cost per project than GEF. The reason simply being that the annual approved GEF project financing volume is 15 – 20 times larger than that of NDF, which again is several times larger than that of NCF. Project size can affect the ratio of administrative expenses to approved funding, since staff time use for preparing projects is likely to be roughly the same regardless of project size. However, the cost structures for many of the programmes under GEF management are really not comparable to NCF, as the basic programme designs tend to be different. The closest is perhaps the GEF Small Grants Programme (SGP), initiated in 1992 as a modality with many stakeholders – funding from the GEF, implementation by UNDP, execution by UNOPS, partnerships with many donor agencies. The idea was to complement the larger GEF projects by providing a window for the direct participation of local, community-based organizations. The SGP funding-per-project is less than USD 50,000 (the average grant size has been USD 25,000). The June 2007 Technical Paper of the GEF Evaluation Office demonstrated that the SGP’s management costs appear to be in line with other similar grant-making organizations. The UNOPS fee for SGP is fixed at 6% in contrast with fees to other entities of 7% plus direct charges to projects. These ratios are roughly in line with the NCF cost ratios. As a final comparison, NCG did a review of South Africa-Norway Programme on Research Co-operation that was set up to encourage cooperation between the research communities in two countries. It was administered by the Research Council of Norway and the National Research Foundation in South Africa. The programme invited call for proposals from research institutions/universities in the two countries for joint projects in this it was quite comparable to NCF in that the programme wanted to foster partnerships. For administrating the Norwegian part of the programme, the Research Council in Norway took 8.5% of total funds disbursed. Administration costs in South Africa were covered by the South Africans. In sum, NCF does not appear to be particularly more costly to administer than other, comparable programmes. This finding holds also if NDFs input is taken into account.

5.2. Process Assessment The key feature of NCF and most of the programmes assessed above is the “Call for Proposal” procedure. This inevitably requires a set of routines to handle the competition, and all programmes spend substantial resources up front. The DemoEnvironment review found that assessment, selection and approval of applications

NCG - Evaluation of NCF

35

required 50% of the administrative time. However, in NCF, some of the other processes also appear as time consuming. While processes and tools have been streamlined and adapted since the first call in 2009, the basic procedures remain roughly the same. The figure below tries to capture the most important steps in a call. For each step, the main implementers and the decision makers are listed in the right column. The main operational actors are:    

NEFCO NEFCO/NDF Working Group, with staff from both organizations NDF NCF Management Committee - consisting of both NEFCO and NDF managers

1. Prepartion of Call

NEFCO - design processes, systems and tools to handle the call. Approval from MC.

NEFCO - make first screening; NDF and NEFCO

2. Scoring and assessment 3. Selection 4. Negotiation & due dilligence 5. Monitoring, follow-up, disbursment

together prepare shortlist. WG - NDF and NEFCO score second round applications. Prepare final list.

Management Committee - approves the shortlist, and selects projects for negotiations.

NEFCO – due diligence site visit is obligatory before 1. disbursement. NDF occasionally contributes.

NEFCO

- assesses project reports, achievement of milestones, answer project enquires, makes aggregate NCF reports, supervision visits, disburses funds etc NDF - assesses reports from NEFCO and project partners, do supervision visits, approves disbursements.

Preparation There is significant work involved in the first, preparatory phase, particularly in the making of "Application Guidelines". This will be sent out to all interested firms, and must clearly communicate what type of projects the call desires, and what the key criteria for choosing the winners are. This is an area that has improved over the years, as lessons learnt from the former calls have been taken into account. However, the different themes chosen for the Calls necessitate constant adaptations. This process is handled by NEFCO.

Assessment The scoring and assessment phase has likewise improved, but is still the possibly most time-consuming job in the whole process. Initially, NCF was set up to handle maybe 4050 applications per call, and when 140 answered the first, the systems were severely taxed. Despite the improvements and lessons learned from the first call, the evaluation NCG - Evaluation of NCF

36

process during the prequalification phase was still considered somewhat laborious and time consuming by NEFCO and NDF staff in Calls 2 and 3. The scoring system at the final appraisal is now highly detailed and quite "scientific" - in this it resembles most of the other call-programmes assessed above.12 It is difficult to say whether the scoring system is too much or too little - no scoring is ever fully objective. There will always be an element of subjective assessment. A number of comments from NCF mention the issue that some applicants - in particular consultancy companies - are very good at writing proposals, while others have less skill in this form of art. They thus score better formally, but it is not at all certain that they really are that more innovative. On the other hand, a less mechanised assessment system can bias selection depending on who does the assessment, and may be more difficult to explain to stakeholders and indeed also to applicants. This is a valid argument, but it is recommended that the system allows for some assessment flexibility in cases where NCF can smell an excellent proposal buried under a less than convincing application. The key is to have experienced and knowledgeable experts doing the scoring and the assessment. Thus, scoring should still be the basic component of the assessment, but with an element of subjectivity allowed the analyst. The process itself is a combination of NDF and NEFCO work, mostly through what is called the NCF Working Group. While NEFCO does the first screening, the Working Group combined makes the selection about who to invite for a full application. The shortlisted applications are then scrutinised and scored by 2 staff each from NDF and NEFCO. The Working Group then presents a final list on nominees to the Management Committee. Most other international call processes do not include the funder in the assessment of applications. Rather, most call programmes outsource the whole management of the programme, without interfering at any stage before the selection. Indeed, some like the Challenge Funds also use an external independent panel to select the final winners. DFID management happens only at the first and the last stage of the process - through the initial guidelines for the call, and then through evaluation and reviews after the funds are disbursed. That this is still a costly process is proven by the fact that all use more than 20% of available funds for administration. Thus, there seems to be some synergies at work, which make in-house management cheaper. Interviews with involved NEFCO and NDF staff indicate that they believe this selection process works reasonably well, and that the discussions in the Working Group are informative, professional and useful. The two organisations contributed with different types of expertise.. In sum, given the initial organisational design, the working group process worked as well as could be expected. However, having two institutions involved with slightly 12

The exception is the research programs, which allow more independent judgement of proposals from professors and experts within the field in question.

NCG - Evaluation of NCF

37

different organisational agendas is not without challenges. For instance, the final scoring could have been done with possibly only 2 persons, instead of 4, with only one agency involved. In NCF 3, due to the perceived need to assess the business aspects properly, as many as 4 persons from each organisation participated. As a final note on the project appraisal, almost all respondents to the survey desired more information and feedback on why their project failed/were successful. This can be a resource-demanding exercise, particularly if applicants try to start a discussion about the verdict. On the other hand, it is considered good development practice to offer transparency in selection processes as this. Thus, the recommendation is to assess whether a cost-efficient system for feedback to applicants can be put in place.

Selection The key aspect of the selection phase is the composition of the Management Committee. While having an independent panel is perhaps not necessary, it is good governance practice that those that are involved in the assessments should not be involved in the selection. NEFCO should thus not participate in selecting projects they themselves have recommended for selection. The same can be said about NDF staff, but there is one crucial difference: It is clearly within NDF’s mandate to decide how to use their own funds. While NEFCO has substantial experience to offer, they have a different mandate that is more focussed on the mitigation aspects of the international climate agenda. However, in the Management Committee, NDF should avoid having staff that has participated in the selection process - in accordance with good organisational governance practice. If the current administrative arrangement is continued, it is recommended that mainly NDF and independent experts sit in the Management Committee.

Negotiations and Due Diligence Negotiations with the chosen projects start immediately after selection. Normally, a due diligence site visit is required before the first disbursement. In some cases these site visits lead to renewed discussions with the promoters, but most often it apparently does not. No project has been disapproved after the due diligence. The analytical content of the due diligence reports varies, with some having good discussions, while others are more of the checklist variant. An issue with the due diligence is that it can delay project progress, as a visit has to fit into the otherwise busy schedules of NEFCO and NDF staff. The Team believes that an initial due diligence is necessary, to familiarize with the project, meet the local partner, and assess the context. Meeting personally also increases the project's accountability to NCF. However, the quality of the due diligence needs to be improved: 

It should take place in the initial stages of negotiations, when projects can still be influenced. Doing it after signing of contracts risks it being just a "paper exercise". One of the lessons learned mentioned in the Annual Report for 2011 (p.19) was "...that it may be beneficial to conduct due diligence visits before the signing of the agreements."

NCG - Evaluation of NCF

38



It needs more depth and analytical rigor - most due diligence is not scheduled for more than a day or two, as an appendix to other business that NDF or NEFCO have in country. The due diligence is potentially the most effective tool NCF has in learning about the project, assessing it, and influencing it. That opportunity should be better used.



It needs to be done by country experts that know the context. The major unknown factor in most applications is precisely the country context, and how the project fits in the local institutional, cultural, social and economic framework. For such analysis to be of use, it requires experts that know the country beforehand.

To emphasize that this is a pre-project analysis, one might want to use the term "appraisal" instead of "due diligence". The term "due diligence" is used for a number of concepts, but is often understood as an investigation of a business just prior to signing a contract. The type of analysis we recommend is perhaps closer to a typical appraisal, as defined in the development aid sector. For Call 3, the process of entering into contract has taken longer time than expected. The main issue is that this Call invited business proposals, and each applicant thus sent their business plans. NCF had perhaps not expected that assessing this aspect would lead to so much discussion. NEFCO now tells the Team (22 March 2013) that most of the Call 3 contracts have been signed - which are good news. Delays of more than half a year are unfortunate. The project momentum is broken, the 2-year project durations become unfeasible, staff in projects change and initial assumptions may do likewise. This can impact promoter dedication and commitment, and on the partnership itself. Long delays should - if at all possible - be avoided. As NCF pays out only on the achievement of milestones, focussing on those milestones more than on the assumption may perhaps ease the process somewhat. However, it is acknowledged that this is a complex process, and it is worth noting that most of those international call projects that deal with private business plans - like the challenge funds - have much higher administration costs.

Monitoring and Follow-up This is apparently the area with the most duplication of work between NDF and NEFCO. While NEFCO is responsible for the day-to-day project administration and monitoring, NDF involve themselves for instance in controlling reports from projects and from NEFCO. There seems to be a fair amount of debate between NEFCO/NDF about these different reports. There have also been discussions about who should do country and project visits. Thus, NDF is apparently not always entirely satisfied with some of the administrative work. However, it is important not to exaggerate the tensions between the two organisations. The key culprit is namely a less than perfect organisational arrangement between NEFCO and NDF. There are three key issues: 

A monitoring structure with two levels of decision makers, sitting in different institutions with different managers, is asking for trouble. A project should

NCG - Evaluation of NCF

39

relate to one implementation unit, and one only, with respect to all monitoring and all other contact. This implementation unit must have full responsibility for dealing with the project. If an "outside" institution constantly overrules or interfere with the implementation on a regular basis, it may a) delay decisions in the project, b) confuse the partners, and c) reduce the commitment by the implementation unit. 

The funder - i.e. NDF - should focus on their higher level role, namely being concerned about the overall objectives, and climate and development impacts. They should not be involved in the discussion about whether an activity can be moved from one year to the next. Ideally, the funder should only be active at the top of the monitoring pyramid. The third issue, and the one that is at the heart of the two others, is that the relation between NDF and NEFCO is contractually untidy. Now, NEFCO is appointed as a "implementing and contracting agency", and sign the contracts with projects in their own name. Rather than the current partnership structure, there should be a clear outsourcing contract, with NDF being the client and funder, and NEFCO the consultant and implementer. The contract should spell out clearly what is expected from NEFCO, and the consequences in case of nonperformance. NDF might then not have the same urge to concern themselves regularly with the projects. This is the model chosen in almost all other international Call programmes.

What makes this structure especially tangled is that NEFCO is also present in the Management Committee that is the overall governing body of NCF. This blurs the lines of accountability and responsibility. It needs to be remembered that NCF is funded NDF funds, which has been provided by the Nordic governments for NDF to use. It is thus only NDF that should be responsible for its proper utilisation in relation to NDF's goals.

At this stage we find it important not to over-dramatize the issues between NDF and NEFCO. In the broader perspective, NCF operates quite well and expenditures are not out of proportion to what similar programmes costs. Most internal issues are solved, even though they take more time and resources than strictly necessary. With determined management involvement from both NDF and NEFCO, the current structure can clearly be made to work also in the future. When NCF was established the thought was that the practical details would be sorted out by the Management Committee as the NCF developed. However, most international experience indicates that organisations work better together if they have a clearly structured cooperation framework, with demarcated roles and responsibilities. Whatever option NDF may choose for NCF in the future, such a contractual framework is recommended put in place between NDF and the implementer. One additional monitoring issue is that while NCF may reduce the general supervision of projects under implementation, the organisation needs to do an end-review of at least a random selection of projects. This review should not be done by staff that has worked with the project. It would however be best if independent experts are hired, and this could be done in batches of projects. The reason is that the Team's field trip indicates that the end report from the promoter does not tell the full story. Such an end-review is NCG - Evaluation of NCF

40

necessary to a) factually assess the project, b) verify the milestones that the project says it has met, and c) draw lessons learned. For such a review to have meaning, it must be done by professionals in that particular area, and probably before the final disbursement. The NCF Draft Monitoring and Reporting Guidelines dated 20/3 -2013 presents a complex design involving both organisations at different levels at different reporting stages. Annex 1 to the Guidelines lists as many as 11 different reporting procedures for NCF . This is way too many, and way too complex.. As a more general monitoring recommendation, the Team suggests that NCF moves towards a red-flag system in supervision and monitoring. Regular project reporting from projects to NCF should be as today, but NDF/NEFCO should only visit projects in the field that either experience serious delays, or are faced with particular challenges. We trust NDF/NEFCO staff to have the necessary analytical discretion to identify what projects that should be. Thus, if a project faces no particular challenge or deviation, it would be visited only 1 time - at appraisal/due diligence, and then possibly at the end review.13 Of other regular reporting within NCF, it is perhaps not necessary to do quarterly reports - half yearly should be sufficient. The Annual Report is much too extensive and can easily be reduced. Several of the reports just repeat what has been said before, which in unnecessary for management use. In general, NCF ought to critically examine the usefulness of the current Draft Monitoring guidelines.

Bureaucracy A number of project partners report that they find NCF bureaucratic and at times slow moving. NCF is generally based on a substantial contractual and reporting framework that can be quite overwhelming at times. One comment in the questionnaire probably sums it up: NCF was perceived as “quite bureaucratic, though friendly". The current level of administrative bureaucracy can be explained by:

13



The need to have one standardized contractual template for all projects, which leads to a very comprehensive set of procedures and contracts, as the projects are all so very different. Covering all eventualities in any country with all types of partners, require a substantial contractual body. This leads to questions and requirements that some projects may feel as irrelevant.



The very real fear of financial mismanagement. A number of extra safeguards and legal paragraphs to protect the funds are thus included. In many NCF cases, these are probably good precautions and quite necessary. In others, some are perhaps less critical. For instance, in Call 2 even the smallest receipts have to be forwarded to NCF for control.



The somewhat general objectives of NCF - partnership, innovation, technology transfer, development, climate change - imply very broad ranging reporting

This is apparently also the current procedure - but the extent of field visits has still generated a fair amount of discussion.

NCG - Evaluation of NCF

41

formats. Not all feel the same need to report diligently on all of these aspects, neither do they have the expertise to do so. 

The current operational structure involving both NDF and NEFCO discussed above may contribute to delays in decision-making, in particular when one organization waits for the other.



Finally, it seems that monitoring has focused particularly at the activity level of the projects, and whether the projects have done what was in the initial plans. While this is understandable, in innovative projects, one must expect variations and constant adaptations. Indeed, it is the key feature of a successful pilot that it is able to constantly change and adjust to changing circumstances.

Finding the right balance between control and flexibility is challenging, even more so in NCF because the portfolio is so very diverse. What seems eminently sensible in one case may seem unreasonable in another. It should be stressed that NCF cannot negotiate individual contracts with each project - that would be way too costly. There has to be a standard agreement template common for all. Many of the call-based programmes use a substantial body of agreements and contracts, and as NEFCO has pointed out to the Team, the ones used in NCF are similar to those applied by other IFIs. Several donors do, however, express concern about the level of bureaucracy in these programmes, where each transaction is relatively small compared to other types of aid programmes. In a similar call-programme - the PSI - the Dutch have for instance introduced a new, lighter approach to reduce the bureaucracy and the administrative burden for its customers. This implies less official reporting on the companies' side and more informal contact between NL EVD International and its clients. 14 For NCF, there are three general areas - or actions - that can be considered for streamlining operations and reducing the bureaucracy. 

Harmonise projects, meaning having projects that are more similar, in similar countries and within similar themes.



Accept a greater level of performance risk, by controlling less and be more flexible with regard to implementation. Monitoring focus ought to be on the achievement of milestones and on the overall objectives, and less on the detailed activities. NCF is a programme that is supposed to encourage and ignite new ideas, and risks are part of that innovation process.



As mentioned above: introduce a red-flag system in supervision and monitoring.

As an overall recommendation, NDF should go through all the current NCF procedures, routines, agreement templates etc, with a view to reduce requirements and controls. Contacts with partners should be streamlined, and made as simple as possible. There should not be two organisations involved in monitoring and controlling, as it clearly bureaucratises NCF. Further, the Team will suggest that NCF gives itself more flexibility in the application of procedures. 14

"Guide to Project Administration" , Private Sector Investment Programme, NL Agency, Ministry of Foreign Affairs

NCG - Evaluation of NCF

42

As a beginning, NCF might want assess the following three practices: 

The practice of requiring a bank guarantee if a project wants an advance. Smaller advances of less than - say 5-10% of total grant amount - should be allowed even if the partner cannot offer a bank guarantee. This rule excludes a number of smaller firms, and primarily hits the local partner that is the one that starts implementation. The lack of advance may not only complicate the planning of the project, it may lead to sub-optimal implementation solutions.



The external legal opinion, that is required to ensure that the Grantee has taken all internal measures required to be bound by the Grant Agreement, is not about to go bankrupt and has obtained all relevant authorization. This is a costly procedure for NGOs, small firms and others without their own legal department. Can these assurances be gathered in a less formalistic procedure - like a selfdeclaration?



The practice where you have to send over every small receipt for any expenditure. The practice has varied between the first and the second call. NCF found it to be very challenging to follow the costs in the first call that desired receipts only for expenditure over EUR 2,000, and for NCF 2, copies of all invoices or equivalent detailed listing were required. Most bilateral donors have left that practice a long time ago, as it is very demanding to control every little expenditure. Unfortunately, that level of control does not improve financial management dramatically, as it is easy to fake receipts and costs in most of the countries that NCF is active in. One alternative option is that the projects deliver audited statements of expenditure together with their disbursement requests, without passing on every receipt.

The basic reason for the above procedures is financial control, but a) the amounts are not that substantial, and b) these are Nordic companies which in most cases tend to be both serious and worried about their reputation. One would perhaps assume that the more unscrupulous companies were weeded out in the competition itself, and that some of these additional procedures are not really necessary for those that end up as winners. These procedures are continuously assessed according to NEFCO, as experiences with the Calls are gathered. For all of the procedural questions, the recommendation is to focus more on the milestones and the achievements, and less on the activities and the details of operation.

The Two Year Implementation Period This is a straightforward recommendation: The 24 month implementation period should be increased to 36 months. All projects interviewed during the field visit find that a 2year implementation period is too short. It always takes time to start operation, and for many projects progress hinges on external actions like getting official licences etc. that is outside of their control. Some felt that when you had finally got going, you had to start planning for shutting down. At least half of the projects have experienced serious delays. There are very few new ideas that can prove sustainability and feasibility within a period of only 2 years. Most entrepreneurs require much more time to raise their project to a NCG - Evaluation of NCF

43

standard where it is anything near being replicable. Introducing for instance new technologies that are not customary in the NCF partner countries is a long term process. Changing habits among consumers and clients do not happen overnight, as seen for instance in the projects that aims at changing age-old agricultural methods, to save water. Partnerships also need time to develop, and to mature. To our knowledge, of the other international call-based programmes, only the Swedish DemoEnvironment support projects of 2 years or less - and that programme also experiences significant delays in project implementation. Most others have implementation periods of 30 - 48 months. NDF expresses concern about the cost effectiveness of implementing projects for longer than 24 months. However, the cost of implementing a project over 36 months should be roughly similar to one over 24 months, provided that the project is based on the same number of milestones. Indeed, one can hope for fewer delays in comparison to agreed plans, and thus less interference necessary for NCF.

5.3. Administrative Options The main cost in NCF is staff costs, and the main driver of staff time is the number of applications and projects handled. Assessing 140 applications certainly takes more time than doing 50. Thus, to reduce the administration cost without compromising quality, the two key actions are: 

Reduce the number of applications, the number on the shortlist, and the number of projects actually given grants.



Harmonize the projects to a larger degree to realize greater professional synergies in handling them.

The possibly easiest method is by reducing the set of thematic topics and limiting the number of countries in each call for proposals to e.g. 5-6, and all of them in the same region, and preferably in countries where NDF is already funding projects. Administration can further be pruned by going through the current practices and routines with a view to streamline and simplify, as recommended in Chapter 5.2. The overall objectives - encourage innovation, partnerships and technology transfer - should be kept in mind, and the implicit consequence that a certain level of risk is acceptable. These recommendations hold for any future administrative solution for NCF. If NCF continues into new calls, there are basically three administrative options for a continuation: 1. As today, with NEFCO as implementer. In that case, it is recommended that the roles are made clearer, and that the two parties ensure that they do not step on each other’s toes in implementation and monitoring. The Management Committee should mainly consist of NDF and independent experts. 2. Tender and outsource the implementation contract (NEFCO may of course be allowed to compete) to an external company. This is typically what most call NCG - Evaluation of NCF

44

funds within the private sector do. However, it is not likely to be less expensive, and NCF will to some extent be removed from NDF core business. 3. Implement NCF in-house in NDF, most likely with the assistance of hired consultants. This will ease the integration of NCF in NDF, and facilitate synergies with ongoing NDF operations. It will allow greater strategic control over NCF. The main argument against is that the process gets too internal, and that good checks and balances between those that select and recommend, and those that approve and fund, will not be sufficiently transparent. All three solutions can clearly be made to work. If NDF themselves run NCF it can in theory more easily assimilate NCF in their other ongoing operations. NDF has already built substantial capacity in the general assessment of climate projects, and in organising a call. Particular expertise can be bought from a consultant, if necessary. One last argument that could influence the type of administration is that NCF requires strengthening in its application of country knowledge. The Team's field trip indicated that contextual issues play a major role in explaining success or failure of the projects. It is believed that NDF has access to such expertise, or can hire it from external experts. In sum, NDF needs to rethink the administrative arrangement with NCF, and how it wants to work with NCF in the future. NDF needs to do a proper in-depth analysis of the different options listed above. Transferring the administrative responsibility for the existing calls from NEFCO to any other arrangement is not likely to be feasible. Current NCF contracts with the projects are for instance signed with NEFCO as the legal counterpart. Any transfer of administration away from NEFCO implies substantial transaction costs. Both NDF and NEFCO would thus probably benefit from finding a working administrative solution for not only the remaining administrative period of Call 1,2 and 3, but also future potential Calls However, NDF should take their time and carefully consider their options, taking due account of their future strategies, and as important, the likely development impacts from different organizational set-ups.

NCG - Evaluation of NCF

45

6. Development Considerations

In its past life, NDF was a pure aid and development oriented International Finance Institution (IFI). The loans that were then provided, and which repayments now constitute the core capital of all of NDFs activities, were founded unambiguously on development rationales. While combating and adapting to climate change is the new focus area for NDF, the organization is a provider of development and climate finance. "The objective of future NDF operations as stated by the NDF Board of Directors will be to continue to provide financial support to developing countries in the area of climate and development", according to NDF Climate Change Strategy 2012/2013. Development is thus still at the heart of NDFs objectives. The issue is then whether NCF makes sensible development contributions, in addition to fighting and adapting to climate change. There are two particular questions: Firstly, what is the value added of NCF? Does it contribute with something unique? Secondly, has it chosen the right strategy within the climate theme, to maximize development impact?

6.1. International Alternatives and NCF Value Added In short, the Team has not been able to identify any international financial mechanism that is fully similar to NCF. There are a number of programmes that fund climate projects that also focus on innovation, that generally funds small projects, that facilitate partnerships and that encourage technology transfer, and several that are call and tender based, but none that combines all of these. As the financial mechanism of the UNFCCC, the GEF manages three of its established climate funds - the Least Developed Countries Fund (LDCF), the Special Climate Change Fund (SCCF), and the Adaptation Fund - disbursing hundreds of millions of dollars every year to cover climate change mitigation and adaptation projects. With its broad scope covering both adaptation and mitigation, SCCF (established 2001) was the first GEF-managed climate change fund accessible to all developing countries. The two other UNFCCC funds, the LDCF and the Adaptation Fund, were established at the same time as the SCCF, but they are focused only on adaptation. SCCF funds projects under four different funding windows: Adaptation, technology transfer, sector-specific projects, and assistance with diversification of fuel-dependent economies. The SCCF is funded through voluntary contributions by donor countries. The largest number of SCCF projects is implemented through the United Nations Development Programme (UNDP) and the World Bank. Getting funding from SCCF is not an option for most of NCF's partners, as projects are government initiated and mostly implemented through multilateral organisations. NCG - Evaluation of NCF

46

However, the facility does work with innovations and pilots, and some of the objectives of the projects within energy and water management for instance, are quite similar to what NCF is funding. However, SCCF lacks the flexibility and the light-footedness that characterizes all of NDFs activities. The GEF Small Grants Programme (SGP) - initiated in 1992 - is a modality with many stakeholders – funding from the GEF, implementation by UNDP, execution by UNOPS, partnerships with many donor agencies. SGP responds to the demand from local communities and NGOs for grants in the GEF focal areas. The permanent presence of SGP country teams and continual access for CSOs mean that National Coordinators (NCs) and National Steering Committees (NSCs) are well apprised of community needs and understand local contexts. As soon as countries receive their annual grant allocations, the call for proposals proceeds. The programme today represents one of the GEF’s most productive investments in supporting civil society and community initiatives, and generates considerable benefits from the funds invested, as evidenced by four global evaluations (1995, 1998, 2003, 2008). With some USD 670 mn invested, GEF SGP has been able to support more than 14,500 projects in more than 125 countries. The SGP grant operations model is, however, fundamentally different from that of NCF. All grant-funding decisions is taken in-country by duly constituted NSCs. Coverage (both thematic and geographical) is broader, and the structure and organizational setup of SGP is substantially different and much more complex than that of the NCF. The private sector based challenge funds and the similar call-based programmes come closer to NCF, in that they encourage small-scale partnerships, albeit with a business objective. A number of the NCF projects within sustainable energy might for instance have qualified for several of the PSD programmes, where development and climate go hand in hand. The Africa Enterprise Challenge Fund (AECF) for instance, will open a funding window in the spring of 2013 that is targeted at business ideas based on renewable energy and adaptation to climate technologies. Several evaluations of these PSD programmes emphasize that the programmes attract quality propositions, and that they leverage considerable co-investment commitments from private firms. For AusAID, the instrument is seen to be flexible, responsive and complimentary to more established aid modalities. The Swedish DemoEnvironment programme has been mentioned already, and the basic idea of the programme is that a plant designed to fit with the local context is needed for demonstration of a viable technological solution, and that such a pioneer will need incremental support to be implemented. The programme is designed to give authorities, municipalities, institutions and companies in some forty countries an opportunity to test new environmental technology. The overall objective is environmental, but the independent review concluded that there is no monitoring of environmental indicators. Neither is a more specific assessment of environmental improvements by the demo projects performed. Predicted effects are vaguely described and in general overestimated. In conclusion, there was NCG - Evaluation of NCF

47

considerable room for effectiveness and efficiency improvements at the time of the independent review. The Energy and Environment Partnership Programme (EEP) financed by the Finnish government and NDF provides grants to promote the use of renewable energy, as well as energy efficiency and clean technologies. The programme is designed to facilitate the development of innovative ideas, approaches and concepts, into sustainable and bankable investment projects. The EEP programmes fund pre-feasibility and bankable feasibility studies as well as pilot and demonstration activities. The grants are allocated on the basis of competitive Calls for Proposals, which are launched 1-2 times per year and are open to public and private entities, research institutions, universities, and civil society organisations. EEP shares several objectives and modalities with NCF, but it is a more comprehensive and embracing programme. While this can have advantages, it loses some of the flexibility of a programme like NCF. The EEP appears more as a complementary facility to NCF - possibly funding successful NCF projects - than a competing one. While several of the programmes have common characteristics such as being challenge funds and/or call-based, they do not necessarily have common goals and targets with NCF. Except for the three GEF funds, the others tend to target a much wider range of environmental issues; some with a pure business orientation, others focusing on projects with considerable externalities. Thus, while there are similarities, there is no programme just like NCF. The tremendous funding gaps identified for both adaptation and mitigation activities in developing countries are clear indications that the NCF represent an important added value in assisting the most vulnerable countries and population groups to be better prepared to tackle the emerging climate change threats and impacts.

6.2. Adaptation and Development The most severe impacts of climate change are endured by the world’s most marginalized people, who have the least capacity to cope. The food security of an estimated 2.5 billion people dependent on agriculture in the developing world is threatened by changing climate systems. People with the least access to resources and power are the most at risk. Supporting communities to become more resilient to future climate shocks - adaptation - will become increasingly integral to international development agendas over the coming years. The UN Millennium Development Goals (MDGs) are due to expire in 2015 and debate over a new global development framework is underway. This rethink on poverty and inequality will cover food insecurity, access to resources including water, gender equality, social protection, human development, and environmental degradation. If the list is beginning to sound familiar, it is with good reason: Climate change is arguably the biggest challenge facing the international development agenda. NCG - Evaluation of NCF

48

The World Bank estimates that the “price tag” of adaptation alone will amount to between USD 70-100 billion per annum between 2010 and 2050, based on a 2 degree Celsius rise in temperature by 2050. Other studies add cross-sectoral adaptation costs, such as water supply, human health, coastal zones, infrastructure and ecosystems, leading to significantly higher estimates in the range of USD 140-210 billion by 2030, and show that even the most narrow interpretation of adaptation costs will incur sums well in excess of USD 100 billion p.a. So far, donor countries have pledged about USD 3 billion to USD 4 billion in climate adaptation finance, while actually approved spending is about USD 1 billion, and disbursed is around USD 350 million. That spending is far short of the USD30 billion in climate assistance by 2012 - balanced between adaptation and mitigation measures that donors agreed to provide at the Copenhagen climate talks in late 2009.

Adaptation Finance: Still a lot to learn. Most agree that adaptation finance should go to activities that strengthen the resilience and reduce the vulnerability of countries most susceptible to climate change. Impacts on socio-developmental issues, such as health, livelihood support, education, conflict and frequent droughts, are expected to increase. This means that adaptation encompasses a wide range of activities in many sectors (agriculture, water, energy, transport, etc.). Making sure climate change funding actually reaches the world’s most vulnerable population is a challenge that decision-makers in developing countries and donors have been struggling with for years. With its twin mandate combining both development and climate change, NCF is potentially well situated to face the "adaptation challenge". In fact, the Team would recommend that NCF in the future should increasingly focus on adaptation precisely because of this mandate. However, it is not an easy issue to address. According to the latest Climate Policy Initiative annual report “The landscape of climate finance 2012”, (CPI (2012)) “Little agreement exists on what qualifies as adaptation finance or, more narrowly, what qualifies as an adaptation intervention….. Data inconsistencies and the difficulties in defining what constitutes adaptation finance (and where the boundaries between adaptation and development finance are), hamper our understanding.” On top of this comes the inability of existing efforts to capture private formal and informal resources dedicated to such activities. Given the definitional and methodological complexity surrounding the topic, there are few estimates available so far, as few have actually tried to truly identify and assess adaptation financing. CPI (2012) op. cit. therefore reports that most institutions are not yet equipped with a proper methodology for measuring adaptation finance. However, relevant efforts to establish tracking and reporting approaches are currently underway (e.g., the MDBs’ initiative on joint adaptation finance tracking), and as a result of these new tracking initiatives, more money is beginning to be invested in adaptation activities. The CPI (2012) estimates adaptation finance within the USD 12.3–15.7 billion range.

NCG - Evaluation of NCF

49

The World Bank has developed an activity-based system, that "tracks spending on activities with adaptation co-benefits at the sub-component level only if they explicitly include climate adaptation reasoning, and directly address vulnerability or impact from climate variability and change”. This system was introduced in 2012. Developing countries have carried out national vulnerability assessments to identify their most susceptible geographic areas, ecosystems, and communities. Such assessments aim to help target activities to critical areas, but there is not yet much evidence as to if and how they drive funding decisions - both from donors and national governments. Furthermore, assessments’ ability to ensure that funding actually reaches local communities is largely untested. These are challenges faced by all adaptation finance institutions. However, the costs of proper tracking and quality assurance will be beyond the current administrative budget of NCF alone, and suggests more harmonization and coordination /cooperation between NDF and NCF to make such monitoring feasible. In order to make sure NCF provides genuine value added through its adaptation funding, it should adopt an adaptation finance accountability approach fully harmonized with what other leading MFIs are doing, see CPI (2012) op.cit. NCF in isolation does not have the resources to put such a monitoring system in place. However, in order for NCF to best support the NDF climate mandate objectives, NDF could help in making sure that NCF have at its disposal tools for the NCF-eligible developing country partners to track and monitor adaptation funding. Provided carefully designed calls for NCF are done, NCF may have an important competitive edge here, and this in itself could prove a NCF-unique value added on the adaptation finance arena fully in line with the NDF climate mandate.

NCG - Evaluation of NCF

50

7. Strategic Analysis: NCF and NDF

In the following we first assess NCF in relation to the current strategy, before we table our main recommendations, ending up with a discussion of further thematic specialization.

7.1. Current Strategy The "Climate Change Strategy 2012-2013", dated January 2012, spells out the current strategic position of NDF. According to this document, the objectives of NDFs operation are: i.

To facilitate greater investments in developing countries to address the causes and consequences of climate change;

ii.

To support development and climate change strategies in developing countries;

iii.

To maximize additionality and complementarity in relation to other available financing;

iv.

To mirror the Nordic countries' priorities in the area of climate change and development

NCF has elements that support all of these objectives: i.

ii.

iii. iv.

Facilitating investments; through partnership, leveraging other resources, and NCF may potentially provide substantial new investments in case of successful pilots. Supporting climate change strategies; as several of the projects in the portfolio specifically work with development of such in developing countries, in particular the adaptation projects. Additionality and complementarity; as there are very few other similar financing instruments within the area. Mirror Nordic priorities, by involving Nordic organizations with similar priorities and strategies.

This says little about the cost/benefit of NCF as compared to the other NDF instruments and engagements. However, as none of the objectives in NDF's strategies are quantified or given indicators by which to measure performance, such calculations cannot be done. Going back to the NDF strategy, it spells out several lessons learned that are to be taken into account in the current 2-year strategic cycle: i. ii.

Further consolidate experiences from on-going climate financing activities (types of projects, sectors, co-financing modalities, partners etc.). Focus its activities on the efficient implementation of existing commitments.

NCG - Evaluation of NCF

51

iii.

iv. v. vi.

Focus, in the selection of new financing opportunities, on activities where NDF can add value beyond the financial contribution through, e.g., synergies and leveraging effect, and on activities where NDF can promote Nordic priorities in the area of climate change and development. Further develop sector specialisation. Enhance the institutional knowledge and competence about climate issues and relevant financing mechanisms. Continue information and outreach activities to enhance the visibility of NDF.

There are five of those lessons that appear relevant for NCF: 

Consolidation of experiences - which the adaptations through 3 different calls is an example of, and which this evaluation of NCF is part of.



Value added beyond financial contribution - which is part of the objective of NCF, by leveraging resources and ideas that can potentially be scaled up.



Develop sector specialization - which NCF offers through selection of specific themes for calls.



Enhance institutional knowledge - which has partly been done through the active involvement in NCF projects, and through the joint administration of the calls. NCF can enhance knowledge even more if NCF is brought into NDF's normal operation.



Visibility of NDF - NCF is perhaps the best known instrument providing NDF visibility in the Nordic countries. There is a substantial interest and demand for new calls, from NGOs, research institutions and private companies.

On financing instruments, NDF provide co-financing together with other financing partners through three main modalities: 

Joint co-financing



Parallel co-financing



Innovative facilities: Five innovative facilities are included in NDF’s project portfolio: Nordic Climate Facility (NCF), Energy and Environment Partnership (EEP) Programme Mekong, GREENPYME, ProClimate Facility (ProCF) and EcoMicro. As of September 2011, the five facilities together stood for about 35% of NDF’s total approvals.

NCF is one of the key elements of the innovative funding modality. NDF does not signal any intention to discontinue with such more innovative financing instruments. Thus, so far, NCF cannot be said to be in conflict with, or otherwise working contrary to, existing NDF overall objectives and strategies. A key question is then to what extent the strategy is successful in meeting the stated objectives of NDF. The most current analysis of that aspect is the independent evaluation of NDF from Vista Analysis. The report was submitted in May 2012 and it concluded that:

NCG - Evaluation of NCF

52



NDF’s instruments and work fits well into the international climate financing agenda



NDF has achieved nearly all objectives and outputs set out in the 2009 Board Paper.



Project approval time is relatively short, but it is difficult to draw conclusions regarding administrative expenses compared to approval funding



Development effectiveness is good



NDF is a result-oriented and learning organization

Basically, NDF was established to work well in relation to its objectives, and had found good operational modalities to carry out its mission. Thus, current strategies are sound, and there is no immediate strategic imperative stemming from that Evaluation that calls for changes in NDF's overall operation - including NCF. However, with regard to NCF, Vista raises questions regarding the partnership with NEFCO and says "..... NEFCO’s lack of past experience in developing countries does raise questions about whether NEFCO provides sufficient complementarity to NDF on this project" (p.23). The NDF evaluation further refers to stakeholders that question NCF's cost efficiency, that wonder whether it spreads its resources too thinly, and whether the there is any climate impact at all. On those three questions we find that NCF is no worse than similar programmes with regard to cost efficiency, that it does spread thinly, and that at least half the projects are likely to have perceptible impact. In other words a reasonable scoreboard. These are all issues that can additionally be improved by reorientation of the facility. The greatest strategic challenge to NDF from NCF is perhaps its uniqueness - that it is so different in size, character and administration from what NDF otherwise does. Two of the key objectives of NCF are not fully explained as to how they are to be understood in an NDF context. Innovation and partnership, yes, but why and how do we utilise that in NDF? The Team therefore concludes that NCF is not fully NDF assimilated - or mainstreamed.

7.2. Key Recommendation: Harmonisation The NCF portfolio of projects – irrespective of their quality and development effects have developed in directions that are independent of, or disjoint with the NDF-portfolio of projects. This has meant that there are only limited – if any – economies of scale and scope from having NCF located within NDF. The fact that NEFCO has been contracted to administer and monitor the NCF portfolio is evidence to this observation. The only direct links tying the NCF operation to NDF is the limitation that the set of NCF-eligible countries must be among the NDF-eligible countries, and the part-time participation of some NDF officers along with NEFCO’s staff NCG - Evaluation of NCF

53

in the selection and monitoring process. The arrangement with NEFCO has contributed to the creation of a certain distance between NCF and NDF. There is of course no overarching rule that says there have to be synergies and links between all of an organization’s activities. There can be good reasons for stand-alone operations, like precisely the need to experiment and pilot. However, most organizations need to have a clear strategic logic behind its actions. For the type of instrument that NCF is, there are three good reasons for why it should be included in NDF's toolbox: 

There is need for innovations within climate change adaptation and mitigation. Perhaps self-evident, but there is need for instruments that encourage innovations. The competitive element of NCF plays a crucial role in promoting new ideas.



The need to leverage private sector capital within climate change. There is no possibility for public resources alone to combat the effects of climate change. Instruments to "lure" and leverage private resources are in demand, whether from NGOs or from private businesses.



The need for developing broad partnerships within climate change, involving other actors than the typical government and donor sector players. There is need for lighter, more flexible partnership facilitation measures than the heavyhanded large GEF and similar types of mechanisms.

NCF has international added value as a quite rare mechanism combining innovation, leverage and partnership. There may be similar instruments that have passed under our radar, but we believe there are very few other institutions that offer a call-based grant support scheme exclusively targeted at specified themes within climate change in developing countries. Our analysis of the NCF supported projects leads us to conclude that NCF has done reasonably well in encouraging both innovation and partnerships, and with a good chance of also having impacts. Sustainability is an issue, but the chances of viability can be increased. Also, NCF is only 3 years old and is still in the starting phase. It has not yet been allowed sufficient time to prove itself. However, we believe that the NCF can be made to work even better, more in line with stated contentions from Board members, and with even greater potentials for positive effects. The key word is "harmonisation". There are three levels of harmonisation: 1. National level 2. Portfolio level 3. NDF level At the national level, as there is a need to harmonise the NCF projects more with national priorities of the host country. NDF as an organisation gets very good scores for being harmonised with national priorities and with other donors. The same description should apply to NCF. Thus, it is recommended that NDF consults with national NCG - Evaluation of NCF

54

authorities before it chooses themes for the NCF calls. NCF does not have to follow the exact same priorities, but must be aware of what is being done in the country, where there may be need for particular attention, and where the NCF combination of innovation and partnership may be of particular relevance. This will make NCF more demand driven, reduce the risk of overlapping support projects, and increase the likely developmental value added. At the portfolio level, the diversity of the portfolio needs to be reduced. There ought to be more recognisable "red threads" between the projects, to improve synergetic knowledge and to do a better job of choosing, monitoring and assisting projects. This will likely reduce operational costs, and provide a better focus for what NCF is to do. As already indicated, this harmonisation is recommended done by reducing the number of countries in each call to 4-6, and by being more restricted in the themes for each call. A maximum of 10 projects should be granted money in each call. This also implies a reduction in the total amount available for each call. The current average size of about EUR 415,000 per project seems about right. At the NDF level, as NCF can be linked more directly to other potential and/or ongoing NDF projects. NCF could more consciously be used in combination with joint and parallel financing with the MDBs, to support particular aspects. These could be carefully designed NCF pilots ahead of the NDF components of MDB-projects, or it could be very specific in relation to sub-themes within those MDB projects like for instance gendersensitive climate considerations. The NCF can potentially be targeted at whatever NDF stakeholders find most appropriate, and that includes improving the quality of its general project portfolio by adding/adjusting/complementing NDF projects. While the two first harmonisations - national and portfolio - are recommended, the last is suggested as an option. In fact, the first assumes the second, as any harmonisation of national policies of 30 countries have little meaning. The third, closer linking with NDF portfolio, has the advantage of clearly showing the rationale of NCF in relation to NDF objectives, but may be hard to do in practice. It may restrict the innovative element and it may not always be easy to include partnerships in the call. Further, themes may become so limited that one may as well simply tender the task as a normal NDF job. Most of the initial rationale of NCF would then disappear. NDF has already a small grants facility that it uses for consultancies in relation to its projects. Still, harmonising directly with other NDF projects may make sense in particular circumstances, and should be kept as an option. With regard to administration of NCF, what was said in Chapter 5.3 still holds - a harmonised NCF can be continued “as-is” with NEFCO, it can be externally outsourced, or it can be run by NDF themselves. However, given the aim of harmonisation, to bring NCF closer to NDFs core operational logic, the last option of administering it in-house appears perhaps as worth considering . We are fully convinced that NDF has the necessary competence and expertise to do so without compromising quality. It is in this connection equally important for us to emphasise that a possible shift of administrator from NEFCO to NDF is driven primarily of strategic reasons. NEFCO can with good reasons take pride in having established a working NCF.

NCG - Evaluation of NCF

55

7.3. Thematic Specialisation Against this background of harmonization, the time may be ripe for discussing narrowing down the set of thematic issues to be funded under the NCF mandate. The current mbalance in available international funding for climate change activities shows that the shortage of funds is critical as regards needed support for adaptation projects, and supporting local capacity building to plan for and deal with inevitable impacts of climate change on poor vulnerable local people As already alluded to above, this implies that NCF should put more emphasis on measures to adapt to climate change. Adaptation to climate change has so far been relatively low on the international agenda, mainly because focus has been put on actions to limit emissions of GHG to be able to prevent climate change from taking place. Thus, adaptation is a less developed area than mitigation. Further, NDF may consider also narrowing the thematic focus within the adaptation field, e.g. on gender-related aspects since women and children are the groups most vulnerable to the adverse effects of climate change. This would present an opportunity to show continued leadership on climate justice and consolidate the record of the Nordic countries on global social responsibility. However, this will only be effective if the fund follows key principles of accountability and transparency, and focuses its efforts on supporting the most marginalized communities and individuals in the developing world who are in need of the most help. This does not mean excluding mitigation from the NCF agenda. It merely narrows the thematic focus to adaptation activities, but many such activities may include mitigation components that will strengthen the adaptive effectiveness. Technology transfer of lowcarbon innovations and other support for mitigation efforts in developing countries is an important element of the wider climate landscape. It is important to acknowledge that several types of climate change activities contain both mitigation and adaptation elements in mutually reinforcing ways. Such projects should continue to be eligible under the NCF mandate. Adaptation funding must be allocated to initiatives that directly aim to tackle the impacts of climate change on the ground. This is in line with general UNFCCC guidelines, which mandate funds for “concrete adaptation activities”. While complementary initiatives which aim to create broader enabling environments for practical climate adaptation interventions can be appropriate under the NCF mandate, NCF should consider leaving the financing of such activities to bilateral and co-funding arrangements.

NCG - Evaluation of NCF

56

ANNEX 1 Terms of Reference

NCG - Evaluation of NCF

57

NCG - Evaluation of NCF

58

NCG - Evaluation of NCF

59

NCG - Evaluation of NCF

60

NCG - Evaluation of NCF

61

Annex to ToR for Evaluation Of NCF PRIORITIES The ToR for the evaluation covers a wide range of issues and questions. To ensure proper focus of the assignment, and to assist the Team in delivering targeted recommendations in line with the key expectations of Nordic Development Fund, the following assessments will have priority: 

Assessment of achievements and performance of NCF - for each particular project and overall for the whole portfolio. Key objective themes for NCF are innovation, technology transfer and partnership. A key question is "impact": Has NCF made a real difference for beneficiaries?



Assessment of administrative procedures and arrangements for NCF with regard to efficiency and effectiveness. The selection process is a key issue. Efficiency considerations must be cross-referenced with the overall mandate of NDF - do the outputs defend the costs involved?



Assessment of the "strategic fit" of NCF, within NDF's overall mandate and operations. Does NCF contribute to meeting NDFs overall objectives?



Assessment of NCF as compared to relevant international practice, and whether NCF contributes with genuine "value added". Key themes will be mandate, governance, efficiency and impact.

These assessments constitute the analytical foundations from which the Team will A. Summarize and outline lessons learnt from the NCF experience so far, with respect to the instrument as such, and with regard to the overall lessons for NDF. B. Deliver recommendations regarding the future of NCF, including suggestions for changes and improvements.

The issues raised in the Annex 1 to the ToR are guiding questions to assist the process of addressing the above priorities. The evaluation will generally follow the methodology outlined in the Technical Proposal from NCG, dated October 2012.

One caveat is necessary, however. The ToR raises several issues that are long-term in nature, like sustainability of projects, observable impact, and leverage/spin-of effects of the NCF. It needs to be cautioned against a belief that final answers can be found after only 3 years of NCF operation. That is a very short time in both a development and climate change context. NCG - Evaluation of NCF

62

ANNEX 2 NCF Milestones

NCG - Evaluation of NCF

63

NCG - Evaluation of NCF

64

ANNEX 3 NCF Projects

NCG - Evaluation of NCF

65

Table 3.1: List of NCF projects (per 28.01.2013) NCF1: (i) Mitigation: Energy efficiency; and (ii) Adaptation: Water resources management. No .

Project title

1

Ref. No. (NDF C3) B11

Country

Nordic Partner

Local Partner

Financing (€) Tot cost NDF grant

Agreem Dura- Comments/other partners . tion signed (mths)

Benin

Naps Systems Oy

Association pour le Develop-pement Economique Social et Culturel de Kalalé

699,989

415,000

15.11.10

24

2

B12

Adapting to Climate Change in Andean Communities Depending on Tropical Glaciers

Bolivia

Diakonia (Sweden)

Agua Sustentable

856,811

496,951

18.11.10

24

3

B13

Fuel Efficient Stoves in East Africa: Reducing Emissions and Improving Livelihoods

Regional (Kenya, Tanzania, Rwanda, Uganda)

CARE (Denmark)

Uganda Carbon Bureau Ltd. (UCB) and CARE International in Uganda

707,684

353,841

27.10.10

24

4

B14

GHG Mitigation and Sustainable Development through the Promotion of Energy Efficient Cooking in Social Institutions

Ethiopia

Gaia consulting Oy

United Nations World Food Program - Ethiopia & Ethio Resource Group

253,300

212,000

04.11.10

16

5

B15

Demand Side Management for

Ethiopia

HIFAB OY

Ethiopian

454,900

407,300

25.10.10

18

Scaling Garden

NCG - Evaluation of NCF

the

Solar

Market

66

Energy

Solar Electric Light Fund (SELF, USA) and International Crops Research Institute for Semi-Arid Tropics (ICRISAT)

CARE International in Tanzania, Rwanda, and Kenya

Climate Change Adaptation for the Ethiopian Power Sector

6

B16

Initiate climate-proofed water conservation strategies in Northern Ghana

Ghana

DHI (Denmark)

Agency (EEA) and Ethiopian Society of Electrical Engineers (ESEE) Water Resources Commission (WRC)

607,500

365,625

28.10.10

24

Geohydronomics Limited (Ghana) and Centre for Human & Environmental Security (Ghana)

Addendum : 25.03.11

7

B17

Energy efficient recycling of electric and electronic scrap, e-scrap

Ghana

Raw Materials Group AB (RMG)

Environment Protection Agency and Green Advocacy Ghana

648,733

480,033

05.11.10

24

Swedish Geological AB, Ericsson AB, and Boliden AB (all from Sweden)

8

B18

Community based adaptation to climate change through environmentally sustainable water resource management in Isiolo District

Kenya

Danish Red Cross

Kenya Red Cross Society (KRCS)

685,544

395,372

15.10.10

24

Grundfos (Denmark)

9

B19

Building Adaptive Capacity to Climate Change

Kenya

ORGUT Consulting AB

Water Resource Management Authority (WRMA), Water Services Trust Fund (WSTF) and Rural Focus Ltd.

1.0193 mill.

496,750

22.12.10

24

DHI (Denmark)

10

B110

Providing Assistance for Design and Management of Appropriate Water Harvesting Technologies in Arid Lands of Kenya

Kenya

Ramboll Natura AB

Appropriate Development Consultants (ADCL)

637,500

500,000

01.10.10

24

National Agriculture and Livestock Extension Programme, NALEP & Ministry of Water and Irrigation

NCG - Evaluation of NCF

67

Ltd

Holding

A/S

11

B111

Enhancing capacity for adaptation and mitigation of climate change in Kibera, Nairobi

Kenya

Solvatten AB

Institute Environment Water (IEW) RINCOD

of and and

345,710

301,290

30.09.10

24

Addendum : 19.04.11

12

B112

Mount Elgon Integrated Watershed Management Project

Kenya

Vi-Skogen (Agroforestry) (Sweden)

Vi Agroforestry

496,200

290,000

05.11.10

24

13

B113

Strengthening Capacities on Efficiency

National Energy

Nicaragua

Motiva Services Oy

Cleaner Production Center of Nicaragua Association

478,100

386,680

27.10.10

24

14

B114

The Bukaleba Project

Charcoal

Uganda

Green Resources AS

Busoga Forest Company Limited

900,199

350,000

30.09.10

24

NCG - Evaluation of NCF

Addend.: 16.12.10

68

KARI and KEFRI

NCF2: (i) Mitigation: Renewable Energy; and (ii) Urban Adaptation to Climate Change. No Ref. . No. (NDF C3) C11 1

2

C12

Project title

Country

Nordic Partner

Local Partner

Urban and industrial waste to energy - promoting sustainable development

Bolivia

Royal Institute of Technology (KTH) Department of Energy Technology, Division of Energy and Climate Studies (Sweden)

Centro Promoción Tecnologías Sostenibles

Financing sustainable energy through remittances flows

Bolivia

Gaia Consulting OY

3

C13

Demonstrating the Feasibility of Locally Produced Ethanol for Household Cooking

Ethiopia

Stockholm Environment Institute

4

C14

Strengthening the resilience of people living in high risk urban and semi urban areas to

Malawi

Finnish Red Cross

NCG - Evaluation of NCF

Financing (€) Tot cost NDF grant

Agreem Dura- Comments/other partners . tion signed (mths)

de de

702,347

499,349

28.10.11

24

Mälardalen VafabMiljö

Fundación AMIBE CODEM/ ACOBE Asociación de Cooperación Bolivia España Gaia Association

689,550

489,550

16.12.11

24

Basel Agency for Sustainable Energy, and Arc Finance

839,783

346,059

31.10.11

24

Former Women Fuel Wood Carrier Association, and Ethiopian Environmental Protection Authority

Malawi Red Cross Society

601,091

499,500

21.09.11

24

Finnish Meteorological Institute

and

Addend.: 14.06.12

Addend.:

69

University,

weather-related disasters

18.12.12

5

C15

GIS tool for urban adaptation Mozambique to climate change and flood risk

COWI A/S

6

C16

Promoting Renewable Energy Technologies for Enhanced Rural Livelihoods

Nepal

655,535

499,236

30.09.11

15

FCG Finnish Consulting Group Ltd

Ministry for the Coordination of Environmental Action (MICOA) Centre For Rural Technology, Nepal (CRT/N)

576,804

366,410

04.11.11

24

330,000

7

C17

Enhancing sustainable energy supply for tea factories

Rwanda & Uganda

Pöyry Management Consulting OY

Rwanda Mountain Tea Ltd &Uganda Tea Development Agency Ltd

8

C18

Karisimbi Prospect

Rwanda

Reykjavik Geothermal Ltd.

Ministry of Infrastructure/ Energy, Water and Sanitation Authority (EWSA)

Geothermal

Addend.: 13.06.12

697,043

280,000

499,538

06.09.11

15

EnCatalyze Ltd, RAcell Uganda Ltd, MCI-Micro Carbon Interwood Ltd

04.11.11

24

Rwandan Energy, Water and Sanitation Authority (EWSA) has funding of € 456.776 for parallel activities

Addend.: 25.05.12

9

C19

Climate Resilient Action Plans for Coastal Urban Areas

Sri Lanka

Norwegian Institute for Water Research (NIVA)

University Moratuwa

of

605,254

455,000

19.09.11

20

UN-HABITAT Regional Office for Asia and the Pacific, and Batticaloa Municipal Council, and Negombo Municipal Council

10

C20

Sustainable renewable energy businesses

Uganda

Norges Vel

Makerere University, College of Engineering

670,680

500,000

30.09.11

24

Husk Power Systems Private Ltd, Confederation of Norwegian Enterprises, and Norwegian University of Science and Technology

NCG - Evaluation of NCF

70

11

C21

Building Technology in Urban Flood & Inundation Forecasting to be applied for Operational Early Warning System in Hanoi City

Vietnam

DHI Water and Environment (Denmark)

The National HydroMeteorological Service of Vienam (NHMS)

582,320

324,950

27.12.11

24

12

C22

Adapting Urban Construction Plans to Climate Change in Vietnam by the use of Strategic Environmental Assessment

Vietnam

The Danish Centre for Environment al Assessment (DCEA) Aalborg Uni versity

Vietnam Institute for Architecture and Urban-Rural Planning - Centre for Research and Planning on Urban and Rural Environment (CRURE)

545,000

495,000

20.12.11

20

NCG - Evaluation of NCF

71

Integra Consulting Services

NCF3: Innovative Low-cost Climate Solutions with a Focus on local Business Development No Ref. . No. (NDF C3) D11 1

Project title

Country

Nordic Partner

NAMA and Innovative Energy Bangladesh Optimisation in the steel sector in Bangladesh

Viegand Maagøe

D12

Introduction of Solarus Solar Photovoltaic-Thermal (PVT) Technology to Bolivia

Bolivia

Renewable Energy Capital Sweden AB

3

D13

Promoting cañahua in the Andean highland: a highly nutritive crop with a great market potential, adapted to extreme climatic conditions

Bolivia

University of Copenhagen

4

D14

Ecological Food Processing Unit

Burkina Faso

5

D15

Building carbon-trading business capacity for organized smallholder farmers in Cambodia

Cambodia

Danish Technologic al Institute Nordic Agency for Developmen t and

2

NCG - Evaluation of NCF

&

Local Partner

Financing (€) Tot cost NDF grant

Modern Erection Limited; Vikrampur Steel Lmtd.; Rahim Steel Trans Tech Ingeniería; Viceministerio de Electricidad y Energía Alternativa (VMEEA) Fundacion para la Promocion e Investigacion de Productos Andinos PROINPA

419,000

299,340

959,654

446,999

345,030

269,955

Isomet

480,000

393,941

Cambodian Centre for Studies in Agriculture CEDAC

484,900

386,400

72

Agreem Dura- Comments/other partners . tion signed (mths)

Grontmij AB; Airby

Ecology Finland Futures Research Centre, University of Turku C. F. Nielsen A/S

6

D16

Scaling up low carbon household water purification technologies in the Mekong Sub Region

CambodiaL ao PDR

7

D17

Ghana

8

D18

Biomass Green Briquette Fuel (GBF) Production (BidiePa) under Kitchen Efficiency Programme - Ghana Pilot Project: Efficiency Enhancement and Entrepreneurship Development in Sustainable Biomass Charcoaling in Ghana

Ghana

Pöyry Management Consulting

9

D19

Rain Water Harvesting (RWH) for resilience to climate change impact on water availability in Ghana

Ghana

SINTEF

15

15

Hydrologic Social Enterprise; TerraClear Development Co. Ltd.; Nexus Carbon for Development CookClean Ghana Limited

3.28155 mill.

495,349

700,526

494,790

African Plantations for Sustainable Development (APSD) Ghana Ltd.; Atebubu Traditional Council; Wiase Traditional Council Science and Technology Policy Research Institute (STEPRI); Water Research Institute of the Council for Scientific and Industrial Design (CSIR-WRI)

650,000

500,000

500,000

400,000

Only SINTEF Agreement for NCF3 has been signed so far.

NCG - Evaluation of NCF

73

18.12.12

10

D110

11

D111

12

D112

13

D113

Business Development Closing the Rural-Urban Nutrient and Carbon dioxide Cycles ADAPTea: Climate Change Adaptation for FAIRTRADE Tea Producers in East Africa

Developing low cost community based innovative solutions to mitigate and adapt with climate change while creating viable local business solutions From Waste to Local Business Development and Vigorous Soil

NCG - Evaluation of NCF

Kenya

Ramboll Natura AB

Umande Trust

630,000

487,880

Kenya, Tanzania, Uganda and Rwanda Nepal

Vi-Skogen

Africa Fairtrade Network also known as Fairtrade Africa

638,000

500,000

Danish Forestry Extension DFE

Wildlife Conservation Nepal (WNC); Choudhary Biosys (Nepal) Pvt. Ltd. (CBNL)

451,000

360,607

Rural Urban Development Initiatives, RUDI; Tanzania Traditional Energy Development Organization, TaTEDO

679,068

500,000

Tanzania

Norges Vel

-

74

Biosynergy A/S

ANNEX 4 Web Survey findings

There were four different categories of respondents participating in the web-survey: Category 1: Category 2: Category 3: Category 4:

Successful applicants in Call 1 and Call 2 Unsuccessful applicants in Call 1 and Call 2 Successful applicants in Call 3 Unsuccessful applicants in Call 3

The survey was divided in these categories as the projects were in different stages of implementation. Calls 1 and 2 were under implementation (with a few being completed) at the time of the survey, but projects in Call 3 had not yet started - with one exception. All applicants in Call 3 thus had much fewer questions to answer. While a few questions were open-ended, most asked the respondents to rate their answers according to scores on a scale from 1 to 6. 

A rating of 1 was “Very bad/very low/very little” and



6 was “Excellent/very high/very much”.



The middle scores would mean: 2 - bad/low/little; 3 - moderately low/moderately inefficient; 4 moderately high/moderately efficient; 5 good/high/much.

Summary of all the questions and responses in the four categories are appended as a separate enclosure to this report, for easy reference and further analysis. The enclosure also lists in separate documents the additional text comments of the respondents.

Survey Response The survey was sent out to the various applicants following a list of contact persons and email addresses provided by NDF, in total 103 projects. The overall response rate was 41% for all four questionnaires. This is a reasonable response rate on such surveys. Marketing companies estimate that for a general client satisfaction survey of medium length, response rates tend to be lower than 10 per cent when no particular "invite incentive" is used.16 Not surprisingly, the highest response rate was amongst the successful applicants in Calls 1 and 2, with 69%.

16

See, for example: “Survey response rates”, Peoples Pulse; http://www.peoplepulse.com.au/Survey-Response-Rates.htm

NCG - Evaluation of NCF

75

Table 1 Web Survey Response Rates Category No. of Total Response Answers Number rate 1. Successful applicants in Call 1 18 14+12=26 69.2% and Call 2 2. Unsuccessful applicants in Call 1 9 19+28=47 19.1% and Call 2 3. Successful applicants in Call 3 6 13 46.2% 4. Unsuccessful applicants in Call 3 5 17 29.4%

% of answers Call 1 Call 2 55.6% 44.4% 33.3% 66.7% 100% 100%

The distribution of respondents in Category 1 (successful NCF 1 and NCF 2) on Nordic countries (notably including the only one project from Iceland): Figure 1: Category 1 survey responses Country of your organisation 16,7% 38,9%

16,7%

22,2%

Denmark Finland Island Norway Sweden

5,6%

In Category 2 (unsuccessful NCF 1 and NCF 2), Denmark had by far the largest number of respondents with 55.6 % of all. Most of the respondents are (not surprisingly) involved part time in the project (88.2% and 62.5% in Categories 1 and 2 respectively, 100% in Call 3). The distribution of type of respondents in Category 1 is shown in the figure below. In Category 2, about 67% of all respondents were from consulting companies. Figure 2 Type of Respondent - Successful Projects Are you ...?

Consulting company NGO

11,1% 38,9%

16,7%

Supplier Research institute

5,6% 27,8%

NCG - Evaluation of NCF

Other private limited company

76

So far, the response is reasonably in line with the actual composition of the NCF portfolio, both with regard to successful and unsuccessfully projects. Of the unsuccessful projects, distinctly more Danes than other Nordics have answered, but this is not believed to skew the results unduly. With regard to mitigation /adaptation, there is also a reasonable balance. When crosstabulating the findings on the project areas (adaptation vs mitigation) with country origin in Category 1 (successful Calls 1 and 2), the situation looks as follows: Figure 3 Adaptation/mitigation - country of origin

How can the results from the survey be interpreted? There are a few caveats to keep in mind when reading the results: 

Those with a positive experience with NCF are likely to have responded to a higher degree than those without. Among the unsuccessful participants, many obviously declined to participate.



Several questions ask the projected promoter to self-assess his/her own achievement - naturally one can expect mostly positive answers to those questions.



A common issue in many surveys is that a person will seek to confirm the wisdom of his/her original decision - it is difficult to be critical even with hindsight.



It is individuals who answer the questions, while in many cases it is the institutional view that is really relevant.

Thus, while the survey gives important insights to the evaluation, the interpretation of its results needs to take a bias towards overly positive answers/assessments into account.

Project Achievements, Impact and Sustainability The answers were generally positive, with all rating the achievements/deliverables so far compared to the planned results/outputs as agreed to in the agreements NCG - Evaluation of NCF

77

(milestones) on the positive side of the median (16.7%-excellent, 50%-very good, 33.3%good). On the question of probability of delivering according to plans (but not yet delivered), 66.7% answered on the positive side of the median and 26.6% on the negative side (with 13.3% in the very low end). This might indicate that the planning has partly been too ambitious in the first place, or simply that problems (e.g. time delays) have materialised during implementation. All respondents rated their projects as successful to various degrees, with 27.8% as very successful and 44.4% as successful. The survey also tried to capture whether participants had detected any impact from their project yet. As much as 70.6% stated that some impact had been detected, whereas 29.4% answered that no impact had been detected so far. The impacts detected were amongst others training of workers; raised awareness for use of sustainable charcoal in the industry; climate mitigation effects; reduction of CO2 emissions; and improved production (local) and awareness of water harvest methods. Others mentioned impacts such as improved health among users; monetary savings among the users; reports on time saved for household chores; and that participation in the project lead to additional income for marginalised groups. One respondent noted that the project has had a positive influence on the communities living in an area, which reputedly is a hotspot for tribal clashes. This suggests that impacts detected include wider effects, which go beyond the agreed project outcomes. Furthermore, looking at the probability of the projects being sustainable, 23.5% responded that this is highly likely, 52.9% that it is likely, and 23.6% thought it would be somewhat (moderately) likely/somewhat (moderately) unlikely. The elements that will secure sustainability (continued operation) of the project results are diverse. For one respondent, it is the strong dialogue and coordination with relevant ministries, and the shared goal of building national guidance on the project results. Others mention that the capacity has been built up within existing institutions (community based, national and local governments), which have been highly involved in defining and carrying out the project. In another project there has been continued interest by other development players. When asked about the main identified risks to the sustainability of the project results, respondents were concerned with lack of political will or change of political discourse, instability in project areas, and corruption. Others commented that it is difficult to quantify results in the informal sector and hard to maintain sufficient working capital/continued funding. As such, the risk is that the projects may be “forgotten" after the project period. As shown in the figure below, only 11.8% have prepared an exit strategy in Category 1. 17.6% have not yet prepared a exist strategy and 41.2% have partly prepared one. The remaining 29.4% thought this issue was not relevant. That some respondents do not foresee the need for an exit strategy is interesting, and not fully compatible with the answers on the questions about sustainability.

NCG - Evaluation of NCF

78

Figure 4 Exit Strategy?

Innovation The questions on innovation refer to the extent of “innovativeness” in the projects. Here, 23.5% claimed that the projects were very innovative, 52.9% that they were innovative, and 17.6% replied moderately innovative. The individual elements that were seen as most innovative were the geographical context in which the technology/technique is used (35.3%) and the institutional/organisational context in which the technology/technique is used (35.3%). 47% believed that their project had introduced innovative/state-of-the-art technology/techniques. The project promoters’ own conception of the projects’ innovativeness on technology/techniques is interesting. It is a difficult term to define, and the Team in their review of all the project agreements, found only a handful of projects that could be said to introduce truly new technology/techniques. Many projects do geographical “innovations”, i.e. introduce an existing technology in a new country, or geographical setting. Some of those projects may actually also be truly innovative, as the practical implementation may necessitate a high degree of fresh thinking. Other projects, however, may be "just" a roll-out/replication of well known and proven technology that does not entail much risk.

Technology Transfer The survey also captured the perceived level of technology transfer in the projects. In Category 1, 53% believed that there had been a significant and good technology transfer, with only one respondent answering on the negative side of the median. Some of the elements of technology transfer listed were transfer of skills utilizing recordable energy meter technology, transfer of skills in demand-side management and transfer of skills in project management. Additionally, one listed that the project has led to “more focus on win-win situation rather than trade-off between environment protection and benefit to the farmer”. The technology transfer is basically understood to be on-the-job capacity building.

NCG - Evaluation of NCF

79

In addition, there were many ideas as to how the lessons learned from the project would be disseminated to a wider audience following project completion, e.g. through articles in newsletters and reports on the internet and social media, sharing through case studies and sharing with stakeholders, etc. One respondent noted that a stakeholder meeting already had been held, with the purpose of disseminating lessons learned from the project to other relevant stakeholders. This reflects commitment and potential for future achievements.

Partnership Questions also covered the perceptions on level of partnerships in the programme. 52.9% in Category 1 had not had any project cooperation with their partner(s) before the NCF project, while 47.1% answered “yes” to this question. On the question of who took the initiative to form the partnerships, in Category 1 56% answered the Nordic partner. It was mainly the Nordic partners who took the initiative to form a partnership also in Category 2 (88.9%). Overall, most respondents in Category 1 rated the degree of partnership (mutual dependency of partners in implementation) in the project as very high and high (76.4%), with only 17.7% on the low side of the median. Category 3 respondents’ score was similar. Category 1 respondents also rated the success of the partnership (mode of cooperation and communication) as being good (64.7% on high/very high), distributed in the figure below (only 3 out of 17 respondents answered on the low side of the median. Figure 5 Partnership Success

Success of Partnership 50,00% 40,00% 30,00% 20,00% 10,00% 0,00%

One respondent adds that the lack of success is due to a local partner leaving the project. Others mention disagreements over interpretation of the roles and responsibilities as obstacles to achieving success. The responses were mixed as to whether cooperation would probably continue between the project partners post-project. In Category 1, 70.6% expected various degrees of continued cooperation (35.3% with very high probability), while 17.7%

NCG - Evaluation of NCF

80

thought it would be a very low/low probability. In Category 3, not yet started, the expectation were, not surprisingly, somewhat higher. The Category 2 (unsuccessful Calls 1 and 2) respondents were asked whether the cooperation continued after the rejection of their application in NCF. Here, 44.4% stated that cooperation between the partners continued, but with other project ideas. 22.2% claimed that cooperation continued with the same project idea, but towards other financial institutions, and 33.3% answered that the cooperation was put on ice. In Category 4, 2 (50%) answered that cooperation continued with other project ideas, whereas one continued with the same idea towards other finance institutions. The responses seem to indicate that the NCF has been an effective catalyst in promoting partnership both for successful and unsuccessful applicants. This is a commendable achievement. For many, the element of partnership has been taken very seriously by the applicants and obviously substantial efforts have been put into setting up relevant teams to undertake the projects

NCF Management Performance All categories of applicants, successful and unsuccessful alike, were asked about the relevance of the tendering and the management of NCF. As regards the quality of the call/announcement and adjacent documents (ToR), the distribution of answers in Category 1 was largely positive, with only 3 respondents on the lower side of the median. Almost 30% rated the quality as "very high". In Category 2 the figures were 0% and 44.4% on very high and high respectively, with 33.3% on the negative side of the median. One respondent wrote: “Some of the questions in the call were overlapping and could have been reduced to a fewer number, while being more precise”, and another “Should be possible to know more about selection criteria”. As regards the quality of the evaluation/selection process, the Category 1 applicants rated the quality positively: Figure 6 Quality of Selection Process - Category 1

The figures from the unsuccessful Category 2 look, not unexpectedly, somewhat NCG - Evaluation of NCF

81

different, but still with 6 out of 9 answers on the positive side: Figure 7 Quality of Selection Process - Category 2

Several unsuccessful applicants criticized the lack of feedback on why they were not granted the funding, and the lack of providing the scores, with further criticism related to the longevity of the process. One answered that the feedback was ..: “Too little and too late. Our partners and stakeholders expected us to explain why our proposal did not receive the funding...”, another “The criteria seemed reasonable, and the selection process seemed professional. But we never got a reasoned feedback after rejection”. It is not surprising that the unsuccessful applicants rate the evaluation process lower. In fact, the Team believes that the unsuccessful scores are better than could be expected. With a relatively high percentage of unsuccessful applicants responding on the positive side, it means that NCF has to a large extent succeeded in undertaking an orderly evaluation process. Nevertheless, the Team believes that NCF should consider giving better feedback on the scores and the reasons behind the scoring. Figure 8 NCF Administrative Procedures

NCG - Evaluation of NCF

82

The responses become more diverse when rating the effectiveness and efficiency of administrative procedures and processes in NCF (including level of bureaucracy and transparency), only asked the successful applicants in Category 1. The results are shown in the figure above. Several of the comments mentioned that NCF seems to be very bureaucratic (“complicated bureaucratic procedures”, “overly bureaucratic and fixated on details”, “… a bit bureaucratic perhaps ...”, “quite bureaucratic, though friendly”). However, others praise NCF “NCF has been very professional. It has been a great pleasure to work with such a professional partner” and “Excellent and open communication”. Also on the comments related to the management performance of NCF in general, the picture was diverse. In Category 1, NCF’s contact persons were commended for being competent and committed. They were said to have good understanding of problems and potentials and manage to adapt communication with partners despite strict formal requirements. In the words of one respondent: “NCF has proven to be very engaged in the implementation process in a very professional way”. Yet, there are also some shortcomings noted: e.g. “reporting requirements have not been clear and communicated by NCF from the beginning”17. Several respondents point out that the lack of pre-financing makes it difficult for small organisations to work with NCF18. In Category 2, some other comments/issues related to the managerial process were listed. Amongst others, NCF was urged to simplify and speed up the call/evaluation process, as well as provide proper feedback upon receipt and rejection of applications (as also mentioned earlier). Comments in Category 3 (successful Call 3) included the following comments 17

NEFCO in their comments to the draft report states: “This has been a “default feature” of innovative pilot operations, i.e. all documentation, templates etc., have been developed and fine-tuned over the course of NCF implementation”. 18 This is understood to read “without bank guarantee”, as advance payment is possible against a bank guarantee, according to the grant Agreement.

NCG - Evaluation of NCF

83

worthwhile noting: “We have had very open discussion about the project with the NCF members”, “Very complicated application. Very slow process with NEFCO, much concerned with details”, and “Time consuming and expensive for the applicant”. Category 4 had one comment (Unsuccessful Call 3): “We experience the style of the NDF-administration as very 'bureaucratic'. … we prefer a donor who is less bureaucratic and engage more in dialogue with the applicants”. This comment ties in with the other comments on enhancing the transparency in the call and evaluation processes. Also, there seems to be a potential for streamlining the administrative procedures in NCF. The survey also asked if the project prepared for the NCF call was funded from other sources following the participants’ unsuccessful application to NCF. 55.6% in Category 2 answered “no” to this question, and 44.4% replied that the proposal had been partly funded from other sources. Additionally, the applicants were asked to rate the usefulness of the NCF as compared to other financing mechanisms. In Category 1, the answers varied as follows: Figure 9

NCF versus other Finance Mechanisms

The answers in Category 2 were, not surprisingly, a bit more negative with only 22.2% rating it as higher and much higher, and with 33.3% below the median. In Category 3 all answers where on better/moderately better (the middle ratings), and in Category 4, 75% was above the median and 25% (1 answer) below. The answers to the question of NCF add value to other financing mechanisms in the sector of climate change mitigation/adaptation showed a similar picture. The successful Category 1 was also asked whether the NCF-financed project had led to leverage/spin-offs, more projects, scaling-up or other synergies beyond what was directly planned for. 70.6% of the applicants answered yes, with several answers highlighting the continued cooperation within the partnership: “the project has led to interest …… to further deepen this work and we are already in talk on how to advance this” and “network increased with other partners operating in Vietnam”. NCG - Evaluation of NCF

84

ANNEX 5 People Interviewed

NDF Pasi Hellman, Managing Director Leena Klossner, Deputy Director Juhani Annanpalo, Country Programme Manager Hannu Eerola, Country Programme Manager Martina Jägerhorn, Country Programme Manager Aage Jørgensen, Country Programme Manager Linda Lundqvist, Legal Counsel and Country Programme Manager Jessica Suominen, Financial Administrator Johanna Palmberg, Country Programme Manager

Nefco Kari Hämekoski, Manager, Carbon Finance and Funds Magnus Rystedt, Managing Director Ash Sharma, Vice President, Head of Department, Carbon Finance and Funds Anja Nysten, Senior Advisor Mia Alén, Loan Administrator Maija Saijonmaa, Project Manager Aliona Fomenco, Project Officer Helle Lindegaard, Legal Advisor

Other Stakeholders/Experts Helge Semb, Former NDF Managing Director Jeremy J. Warford, Consultant Ingrid Glad, Former NDF Board Chairperson Håvard Thoresen, MoE of Norway, and NOAK Harald Rensvik, MoE of Norway and NEFCO Board

NDF Board Members Harald Tollan, MFA, Norway Lena Køvamees, SIDA, Sweden (Alternate) Christoffer Berthelsen, MFA, Denmark Eigill Heidar Gislason, MFA, Iceland (On phone) Satu Santala, MFA, Finland.

NCG - Evaluation of NCF

85

Kenya Annabell Waititu, Executive Director, IWEM Faith Chege, Financial and Administration Officer, IWEM Wangu Mutua, Dep. Programme Director, SCC Vi Eastern Africa Wangeci Gitata, SCC Vi Eastern Africa Amos Wekesa, Environment and CC Advisor, SCC Vi Eastern Africa Fred Marani, SCC Vi Eastern Africa Marion Ng'ang'a, Regional Coordinator Eastern Africa, Fairtrade Africa James G. Mwai, Executive Director, Fairtrade Africa Fred Nyoike, Ministry of Environment and Mineral Resources Jeremy Notley, ORGUT Elizabeth Diego-Lusimba, Manager Community Development, WRMA Joseph Kinuya, Technical Manager, WRMA Lawrence Thooko, WRMA Francis Edalia, WRMA Rose Nyikuri, WSFT Maria Notley, Technical Advisor, WSTF Ann Obae, Programme Officer, WSTF Michael Thomas, Rural Focus Ltd

Uganda Daphne Ayiekoh, Renewable Energy Incubator Claire Kagga, Manager, Renewable Energy Incubator Julius Butime, Makerere University Wilson Musinguzi, Makerere University Reinaart Pretorius, Norges Vel Øivind Ørbeck Sørheim, Norges Vel Åse Lekang Sørensen, Norges Vel Samuel Musoke, Private consultant Bill Farmer, Uganda Carbon Bureau Virginia Echavarria, Uganda Carbon Bureau Gloria Namazzi, Uganda Carbon Bureau Godfrey Kihika, Ministry of Agriculture Sarah Nassuna, Green Resources Isaac Kapalaga, Green Resources Ambrose Ashabahebwa, Green Resources

NCG - Evaluation of NCF

86