APPROACH PAPER – SPECIAL STUDY

Local Capital Markets Development – Interim Evaluation Regional July 2016

EBRD EVALUATION DEPARTMENT



EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 1

The Evaluation department (EvD) at the EBRD reports directly to the Board of Directors, and is independent from the Bank’s Management. This independence ensures that EvD can perform two critical functions, reinforcing institutional accountability for the achievement of results; and, providing objective analysis and relevant findings to inform operational choices and to improve performance over time. EvD evaluates the performance of the Bank’s completed projects and programmes relative to objectives. EvD’s Special Studies review and evaluate Bank activities at a thematic or sectorial level. They seek to provide an objective assessment of performance, often over time and across multiple operations, and to extract insights from experience that can contribute to improved operational outcomes and institutional performance. This report has been prepared by EvD and is circulated under the authority of the Chief Evaluator. The views expressed herein do not necessarily reflect those of EBRD Management or its Board of Directors. Responsible members of the relevant Operations teams were invited to comment on this report prior to internal publication. Any comments received will have been considered and incorporated at the discretion of EvD. Whilst EvD considers Management’s views in preparing its evaluations, it makes the final decisions about the content of its reports. Nothing in this document shall be construed as a waiver, renunciation or modification by the EBRD of any immunities, privileges and exemptions of the EBRD accorded under the Agreement Establishing the European Bank for Reconstruction for Development, international convention or any applicable law. This report was prepared by Tomasz Bartos, Senior Evaluation Manager and reviewed by Joe Eichenberger, Chief Evaluator of the EBRD Evaluation department. © European Bank for Reconstruction and Development, 2016 One Exchange Square London EC2A 2JN United Kingdom Web site: www.ebrd.com All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature.

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 2

Contents 1. 1.1 1.2 1.3 1.4 2. 2.1 2.2 2.3 3. 3.1 3.2

Introduction ...................................................................................................................... 4 Scope and nature of the evaluation Rationale for inclusion in the work programme Background to the evaluation Other relevant evaluation work Monitoring and self-assessment .................................................................................... 7 Monitoring reports Transition monitoring Self-evaluation Evaluation methodology ................................................................................................. 7 Evaluation questions Methodology

3.2.1 Strategic Relevance 3.2.2 Strategic Responsiveness 3.2.3 Strategic Results 3.3

Potential problems and limitations 4. Administrative arrangements ....................................................................................... 11 4.1 EvD evaluation team 4.2 Timetable

Annex 1: Results Framework for LC2 initiative (copied from LC2 Currency and Capital market Development – Strategic Initiative, November 2013)

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 3

1.

Introduction

1.1

Scope and nature of the evaluation

This paper outlines the proposed approach for the Evaluation Department (EvD) to prepare an interim evaluation of the European Bank for Reconstruction and Development’s (the Bank or the EBRD) outputs and outcomes designed to support the development of local capital markets (LCM) in its countries of operation (COO). The development of LCMs is a high priority for the EBRD and it was specifically identified as a core function in its founding documents1. The importance of the LCM (and local currency financing) was highlighted in the EBRD’s 2006 and 2012 Transition Reports. Initially the EBRD mainly performed this function through the Treasury Department’s borrowing and lending operations in various currencies, and the Banking Department’s issuance of corporate bonds and support of Initial Public Offerings (IPOs). The financial crisis of 2008 highlighted LCM risks arising from excessive reliance on foreign currency funding. The crisis prompted the EBRD to deepen its engagement in the development of LCMs and it launched a Local Currency and Capital Markets Development Initiative (LC2) in 2010. This initiative was designed to provide capital market participants with access to local currency financing through the development of efficient and self-sustaining LCMs. To achieve this objective, the EBRD established a dedicated team (LC2 team) in December 2012. The LC2 team coordinated the activities of many operations in this area, including Banking, Treasury, Office of General Counsel (OGC)/ Legal Transition Program (LTP) and Office of the Chief Economist (OCE). In November 2013 the Board approved the “Local Currency and Capital Markets (LC2) Development Strategic Initiative” paper (the Strategy), which outlined the EBRD’s main LCM goals and set specific objectives for the period 2013 - 2016. Reflecting these developments, the evaluation will review the four years of LC2 operations from 2012 to 20152. The evaluation will describe the evolution of the LCM development program, and how it led to the formulation of the LC2 Strategy3. It will identify the program’s links to country and sector strategies, and how LCM development efforts were implemented. The evaluation will then examine a sample of LCM development projects, identify the achievement of the planned outputs and outcomes, and to the extent possible, determine whether any broader impacts can be attributed to LCM projects implemented to date. The evaluation will identify critical issues, and draw lessons that can be used to help guide future LCM strategy development and the design of associated projects.

1.2

Rationale for inclusion in the work programme

LC2 is one of the EBRD’s three key strategic initiatives. As LC2 operations have been increasing in recent years, there is a growing interest among Board members, management and staff to learn more about the performance of its efforts to promote capital markets development. Given the importance of the initiative, the Board and management decided to include this interim evaluation in the EvD’s 2016 work programme. At the same time, it is understood the LCM program is on-going and the achievement of tangible results takes time. Therefore, this evaluation is designed as an interim report, rather than a final analysis of the overall performance of the LCM portfolio. The Agreement Establishing the European Bank for Reconstruction and Development, signed on 29 May 1990, describes in its article 2 – Functions, measures through which the Bank is to fulfil its function of fostering the transition of its countries of operation towards open market-oriented economies and the promotion of private and entrepreneurial initiative. Point (v)of this article says that the Bank will “stimulate and encourage the development of capital markets” 2 The EBRD only began to classify projects as “supportive of LCM” in the Bank’s systems in 2014. Data for 2012 to 2013 was compiled by LC2 team manually. Data for earlier LCM-supportive projects (2010-11) was difficult to compile, as the initiative was at a nascent stage and only a few LCM projects were signed at that time. Therefore the earlier (pre-2012) period will be omitted from this review. 3 The EBRD’s local currency financing operations were evaluated by EvD during 2011 to 2013 (“Local Currency Lending and Borrowing 2000 to 2010”) and these findings will help set the context for the LC2 evaluation. 1

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 4

1.3

Background to the evaluation

The LC2 strategy set out five priority interrelated themes where the EBRD would concentrate its work: 1.

Building stable and sustainable macroeconomic policy frameworks – this theme focused on promoting prerequisites for LCM development, primarily in areas such as policies, skills and tools required to stabilise inflation (e.g. inflation forecasting and development of yield curve models);

2.

Improving the legal and regulatory environment to support capital market activity – this theme targeted the improvement or development of legal frameworks and regulatory regimes for selected, more advanced financing products, primarily through Technical Cooperation (TC) projects;

3.

Developing financial market infrastructure including clearing and settlement – projects under this theme supported LCM infrastructure for processing financial transactions such as: stock exchanges (SEs), payment systems, central counterparties, central securities depositories, securities settlement systems and trade repositories;

4.

Developing the institutional investor base – this theme targeted the weak institutional investor base in the Bank’s COOs (one of the key impediments to LCM development). This constraint was addressed through a combination of TCs supporting pension reforms and several banking projects, which invested in life insurance funds; and

5.

Promoting a more efficient transaction environment and expanding the product range – most of the LCM banking projects fell into this category

The EBRD implemented this strategic program by: (i) providing assistance to develop a favourable environment for market-related activity (usually though policy dialogue and TCs), and (ii) investing (usually co-investing with private investors) in local securities (bonds and stocks of listed companies, including IPOs). During the first years of the LCM initiative the activities of the Bank (primarily LTT) and the LC2 team focused on diagnosing the legal and regulatory environment (theme 2 of the LC2) related to local capital markets and to date 16 assessments have been produced. This activity was supported by capacity building/training provided to local decision-makers to create a better understanding of the risks associated with overdependence on foreign funding and emphasise the need for LCM development. In 2012 the EBRD approved the LC2 Initiative Framework budget of €1.3 million, to support LCM development through TCs. Between 2012 and 2015 the LC2 team initiated and implemented nearly 40 TC projects related to LCM development4. The LC2’s TCs ranged from diagnostic and capacity building, to supporting integration of stock exchanges or preparation of new laws and regulatory regimes (e.g. for derivatives or covered bonds). A total of 18 projects (mostly market assessments and other smaller TCs) were completed and the balance of the TC projects are on-going. The EBRD’s TC efforts were concentrated on the Early Transition Countries (ETC) and Southern and Eastern Mediterranean (SEMED) countries. The first TC supported the preparation of the first corporate bond issue in the Kyrgyz Republic and TC projects in other ETCs followed. The EBRD signed Memorandums of Understanding (MoUs) with six ETCs on cooperation in LCM development that provided frameworks to initiate several TCs. Overall, about 20 TCs (half of the total) benefited at least one ETC or SEMED country.

4

This number includes market assessments, workshops/conferences and extensions.

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 5

Geographically, Poland accounted for over a quarter of all investment/financing projects (24), followed by Romania (14) and Turkey (13). In 2015 Greece became one of the main recipients of LCM projects with five transactions signed in that year. To date, the EBRD has signed only six LCM-supportive banking projects in the ETCs and two in SEMED countries. In terms of sectors, 70% of LCM projects (64) were Financial Institution (FI) transactions, and the balance was spread over nine other sectors. These sectors were: Manufacturing and Services (M&S), Transport and Natural Resources each having six projects; and Information and Communication Technology (ICT), Property and Tourism (P&T), Agribusiness, Power and Energy (P&E), Equity Funds and Municipal and Environmental Infrastructure (MEI) each having signed between one and three projects. Almost 50% of the banking projects (45) classified as LCM development were investments in bonds (including Eurobonds) issued in both local or foreign currencies. These projects were followed by: equity investments in listed companies (32); Swap transactions (10); Deposit insurance facilities (3); and Asset Backed Securities (2). A preliminary review of the portfolio indicates some of these transactions supported innovative financing products. However, a large majority of LCM banking projects were in either corporate bonds or listed equities (in most cases in more advanced countries). Nevertheless, these projects supported development of LCMs, as even in more advanced COOs (e.g. Poland, Slovenia) corporate bond markets were still relatively underdeveloped, while investments in listed equities in smaller countries boosted capitalisation of their stock exchanges and provided them with the EBRD’s “seal of approval”. Given these results, in recent years the LC2 team concentrated TC on supporting the development of the legal and regulatory environment (see theme 2) for new, more innovative financing products, which could lead to potential investment transactions. Two main sub-themes/products supported by the team were TC for: (i) derivatives regulatory reform, and (ii) covered bonds legal reform. The LC2 team implemented the Strategy by developing and managing certain activities itself (mainly TCs), and coordinated and advised the EBRD’s various departments on implementation of other activities (such as actual investments) related to LCM development. In particular, the LC2 team worked closely with: the Banking department (e.g Non-Banking Financial Institutions (NBFIs) on investments in stock exchanges and insurance funds; MEI department on municipal bonds; Industry, commerce and agribusiness (ICA) department on corporate bonds; Treasury on local currency borrowing and lending, and regulatory reform; and OGC/LTP on legal and regulatory reforms. Treasury department’s statutory activities (funding the Bank’s operations) were particularly conducive to supporting LCM development. It issued bonds (some in local currency) in 14 markets of the Bank’s COO (including first local currency bonds in ETCs such as Armenia and Georgia). To enable these issues, the EBRD worked with the countries’ regulatory authorities and legislative bodies to prepare new or refine existing regulations in line with best international practice. Moreover, there have been two LCY/LCM-related operations in the Treasury department’s books - the TCX Currency Exchange Fund (signed in 2007) and the Frontier Clearing Fund (signed in 2015).

1.4

Other relevant evaluation work

There have not been any previous dedicated evaluations of the EBRD’s LC2 or LCM initiatives. During 2011 to 2013 EvD evaluated the early local currency financing and funding initiatives, known as “Local Currency Lending and Borrowing 2000 to 2010”. The evaluation presented some findings and recommendations relevant to the LC2 team and Treasury. However the evaluation was focused on local currency lending and borrowing, and it did not specifically address the LCM development issues.

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 6

An evaluation of the Legal Transition Programme until 2010 was carried out from 2011 to 2012 that reviewed the legal and regulatory frameworks assessments (which related in part to the LCM assessments). This component of the Legal Transition Programme was rated highly successful. EvD prepared Operation Performance Assessment Validations (OPAVs) for two projects classified as LCMsupportive. An additional seven projects are scheduled for validation in 2016. During the course of this study, the EvD team will review relevant COO and the EBRD strategic documents, case studies and sample projects, selected articles and publications related to any evaluation of the local capital markets development projects financed by other international organisations or International Financial Institutions (IFIs), and in particular, International Finance Corporation (IFC).

2.

Monitoring and self-assessment

2.1

Monitoring reports

Most TCs on enabling environment initiatives are on-going and only a few completion reports have been prepared. They are informative but very brief. The reports for banking projects proposed as case studies and sample projects under this evaluation have been reviewed. These reports focused primarily on the clients’ financial performance and provided few references to the extent the projects supported LCM development. No PMMs have been produced yet for several of the more “landmark”/important LCM projects as they were signed recently and some of them have not yet been fully implemented.

2.2

Transition monitoring

Transition Impact Monitoring System (TIMS) reports for most LCM projects reviewed by EvD were up to date, and it will request an update for those projects which are behind schedule. Similar to portfolio management reports, TIMS have not yet been produced for several recent projects.

2.3

Self-evaluation

This evaluation will not rate specific projects. It will concentrate on identifying early results of the EBRD’s LCM projects and derive relevant lessons, rather than provide a detailed assessment of the performance of any individual project. As a result, full Operation Performance Assessments (OPAs) on the case study/sample projects will not be required. The evaluation team will rely on interviews with Operation Leaders (OLs), bankers in relevant Resident Offices (RO), and most importantly, the clients to obtain information on the current status and overall performance of the case study and sample projects.

3.

Evaluation methodology

3.1

Evaluation questions

The evaluation will seek to answer the following study questions: (i)

Were the LC2 strategy and its objectives relevant to the requirements of the COOs?

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 7

(ii)

How responsive was the EBRD implementing the LC2 strategy and what is the status of implementation?

(iii)

What have been the early results of the LCM projects?

(iv)

What have been the key issues and lessons that can be drawn from the implementation of the LCM projects so far, which could improve future project design and execution effectiveness and efficiency?

Answers to these questions will contribute to an understanding of the larger objectives of the LCM program such as the need to decrease dollarization, grow and diversify capital markets, and increase liquidity in the targeted markets. The analysis will aim to understand which projects were most effective in meeting these objectives and the critical constraints and risks impacting on program and project performance.

3.2

Methodology

The evaluation will review the EBRD’s approach to identify early results of promoting LCM development during the four years between January 2012 and December 2015. The review will consist of three parts:

3.2.1 Strategic Relevance (i) Review of the evolution of the EBRD’s approach to LCM development, particularly the differences between 2010 Local Currency Initiative and 2013 LC2 Strategy; (ii) Review of the alignment of key country strategies and sector strategies with the LC2 Strategy, identifying objectives, expected outcomes and outputs, and expected scale and timing to achieve results; and (iii) An analysis of the relevance of the Strategy (the degree to which the Strategy reflected LCM development gaps identified in the Capital Market Assessments for the Bank’s key COO).

3.2.2 Strategic Responsiveness (i) Review of organisational arrangements within the Bank to execute the Strategy. In particular the review will assess the arrangements for LC2 team’s coordination efforts, internally and externally (including the extent of Bank’s collaboration with other IFIs); (ii) Identify the level of resources allocated to LC2 over the evaluation period in terms of objectives, instruments, countries, and sectors; and (iii) Assess the evaluability of the program and projects, focussing on adequacy of definitions of outputs, outcomes, and impacts; availability of baselines and targets; and adequacy of data collection and reporting arrangements.

3.2.3 Strategic Results The study will review the status of Strategy implementation and identify early results of selected LCM projects. The two themes of building stable and sustainable macroeconomic policy framework; and developing the institutional investor base were targeted relatively infrequently and it is premature to review the performance of these two themes. The reasons why these two themes were not targeted to a greater extent will be explored to form a view on their relevance. The following three LCM themes were pursued by the EBRD and they will be the primary focus of the analysis of results:

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 8

1.

Improving the legal and regulatory environment for LCM;

2.

Developing the capacity of LCM infrastructure; and

3.

Promoting more efficient transactions environment and processes that can be used to expand LCM product volumes and range.

These themes accounted for about 85% of the EBRD’s LCM development-related projects implemented to date. In evaluating these themes, it is import to note that some LCM key projects are not yet operationally mature. For each of the three focus themes, the evaluation team will select one case study and two sample projects. A case study might be a TC project, an investment project, or both (if they are connected). The case studies will be examined in detail, primarily based on a document review, and internal and external stakeholder interviews. The case studies will be described and presented in the evaluation report’s annex. Two sample projects will be selected for each focus theme to increase the project sample size5. The sample projects will be subject to a detailed review, including internal and external stakeholder interviews. Key issues, findings and recommendations derived from the sample projects reviews will be highlighted in the report. Unlike case study projects, the sample projects will not be described in detail in the evaluation report. Only those case study and sample projects which have been completed and where sufficient evidence exists to assess their performance will be rated. The rating will reflect the delivery of the outputs and achievement of their targeted versus actual outcomes and impacts on LCM development. No other aspects of the projects’ performance will be assessed and no rating of overall project performance will be derived. The evaluation of all the LCM projects during the evaluation period is expected to generate practical findings, which could be useful for the LC2 team in the design and implementation of its programs and projects in the future. Following preliminary discussions with the LC2 team, Banking, Treasury and other teams across the EBRD, it is proposed to focus the evaluation (in terms of case studies and sample projects) on more innovative (“landmark”) projects and TCs, which are expected to bring substantial change and have a strong demonstration effect on the local capital markets, in some cases, even despite their recent vintage. Table 2 presents case study and sample projects for each focus theme, which are proposed for the study. The projects have been selected following a review of LCM-supportive TCs and investments, and extensive consultations with LC2 and banking teams. Geographic and thematic diversity was also taken into account when selecting these projects sample. A brief description of the current status of the proposed case study projects is presented in Annex 2. However, following an in-depth review and interviews for each of these projects, the evaluation team might decide to swap some of the projects’ categories, and present in the report a case study as a project, which is currently designated as a sample project. These adjustments will depend on the nature of the findings and learning potential the projects present.

5

An evaluation based on only one case study would not be sufficient to identify important issues and draw lessons

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 9

Table 2. The evaluation’s case study and sample projects

Theme

Case study projects

Sample projects

Improving legal and regulatory environment

Derivatives Law and Regulations Development TC - Ukraine

Covered Bonds Law and Regulations Development TC Poland Derivatives Law and Regulations Development TC - Morocco

Developing financial markets infrastructure

SEE Link TC – integration of smaller stock exchanges in Western Balkans (managed by Zagreb Stock Exchange, Croatia)

2 investments in stock exchanges

Promoting more efficient transaction environment and expanding product range

Muni bond

2 bond projects

TC = Technical Cooperation As part of the evaluation of the case studies and sample projects, the review team will interview representatives of the following groups (other than the EBRD staff): 

Bank clients (private and/or public – including ministries and/or central banks);



Bank’s or clients’ consultants;



Security issuers/arrangers (if different from the client);



Market regulators (security commissions);



Industry associations (where feasible);



Market participants (banks, selected institutional investors or mutual funds); and



Co-financiers and/or other IFIs active in this field in a given country.

The output of this exercise will be a Special Study report (of approximately 30 pages plus linked documents and annexes providing detailed case study analyses). A presentation and Q&A session with the Audit Committee will be held if requested.

3.3

Potential problems and limitations

The main challenge of this evaluation arises from the relatively recent nature of the EBRD’s most significant LCM development projects. Most TCs are ongoing (although in most cases their first phases have been completed) and some of the investment projects that will be reviewed were only signed in 2015. Therefore, a full evaluation of the results of these activities will not be possible and the special study is designed as an interim evaluation. More recent projects (TCs and investments) were selected as they best illustrate the EBRD’s “added value” in promoting LCM development. These projects were more innovative from an LCM development perspective than the earlier projects, which were primarily standard investments in corporate bonds and IPOs. As most of the field visits and stakeholder interviews will take place in the autumn of 2016, these projects will be more mature and it will be possible to observe early performance. Part of their impact relates to structure and innovativeness, which are largely time-neutral. It is expected there will be sufficient data to perform the evaluation, and many of the areas related to relevance, organizational arrangements, and implementation of existing operations are ready for evaluation.

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 10

Finally, the study will require a high degree of sensitivity and tact (including observation of internal “Chinese walls” and confidentiality arrangements) when reviewing projects, many of which have been classified as “highly confidential” due to their potential to influence market movements in the case of information leaks (although this risk is/was related primarily to the pre-approval stage). Any conclusions will have to take into account the wider country context and acknowledge the limitation to generalisation of findings and recommendations.

4.

Administrative arrangements

4.1

EvD evaluation team

This evaluation will be executed by Tom Bartos, Associate Director/Senior Evaluation Manager. It is envisaged that one other staff member of the Evaluation Department will be involved in this study. They will be supported by a sector consultant (Terms of Reference are attached as annex 3). Keith Leonard, Consultant or/and Barry Kolodkin, Deputy Chief Evaluator will act as peer reviewer(s). An external panel member (to be identified) may be requested to review the draft report.

4.2

Timetable

Table below presents the evaluation’s timetable.

Milestone

(2016) Date

Approach paper prepared, circulated and approved Consultant contracted Background documentation review Internal data collection and internal interviews

May – mid June 1 July April - July May-August

Field missions and report drafting

July – October

Draft report circulated to internal peer reviewer

mid November

Draft sent to External Panel Draft submitted to CE Draft circulated for Management Comments Final editing and final distribution

early December mid December January 2017 Late January 2017

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 11

Annex 1: Results Framework for LC2 initiative (copied from LC2 Currency and Capital market Development – Strategic Initiative, November 2013)

Key LC2 themes 1. Building stable and sustainable macroeconomic policy frameworks

Output - Policy dialogue/ technical assistance/ capacity building to authorities in CoO promoting stability oriented policies/improved risk management - Loans/investments in instruments providing long term local (currency) funding (see Section 1.4)

Outcome

Impact

- Lower and less volatile inflation - Smaller asset/liability mismatches

Lower loan and deposit dollarization

2. Improving the legal and regulatory environment to support capital market activity

Policy dialogue/ technical assistance/ capacity building in CoOs promoting international standards/best practices in capital market regulation/ transactions

Improved regulatory framework/ transactions environment

- Larger market size - Improved market liquidity

3. Developing financial market infrastructure including clearing and settlement

Policy dialogue/ technical assistance/ capacity building/ investments in CoOs establishing more effective financial market institutions

Improved operating environment in capital markets (faster/more secure execution, lower transaction costs)

- Larger market size - Improved market liquidity

4. Developing the institutional investor base

Policy dialogue/ technical assistance/ capacity building/ investments in CoOs establishing/ enhancing institutional investors (e.g. pension funds, insurance companies)

More assets under management at institutional investors

- Larger market size - Improved market liquidity

5. Promoting a more efficient transaction environment and expanding product range

Policy dialogue/ technical assistance/ capacity building/ investments in CoOs establishing/ enhancing financial market products/ benchmarks

- New instruments traded - New benchmarks utilised

- Larger market size - Improved market liquidity

EvD Approach Paper to Special Study: Local Capital Markets Development – Interim Evaluation 12