Assessing the Value for Money Of Technical Assistance provided by NHSSP to the Nepal Health Sector Two Case Studies Final Report

Rachel Phillipson, Independent Consultant Sumanta Neupane, Independent Consultant

18th September 2012

1

Contents Executive Summary 1. Introduction 2. Assessing Value for Money in the Nepal Health Sector 3. Case Study (1): electronic annual work plan and budget (e-AWPB) 4. Case Study (2): e-bidding for health facility contracts 5. Conclusions and Recommendations

Annexes A1. Electronic Work Planning and Budgeting: Comparisons with the Private Sector A2. Framework for ensuring VFM at the design stage in health sector interventions 
 A3. Terms of Reference A4. People Consulted A5. Documents Reviewed.

2

Executive Summary 1. The Nepal Health Sector Support Programme (NHSSP) is a DFID-funded programme of technical assistance to the Nepal Ministry of Health and Population (MoHP) and Department of Health Services (DoHS). It began in September 2010 and is managed by a consortium led by Options. Its purpose is to support a strengthened health system to increase access to sustainable, quality, essential health care services (EHCS) for , the poor, women and underserved populations.. 2. At inception, Options undertook to gather evidence of efficiencies and cost savings emerging in the programme. In August 2012, three programme areas were selected for a value for money (VfM) assessment: (i) Electronic annual work-planning and budgeting (e-AWPB); (ii) E-bidding for health facility contracts; (iii) Integrated designs for health facility construction; 3. Data were collected and interviews and field visits conducted in Nepal during July and August 2012. DFID’s Four E’s (economy, efficiency, effectiveness and equity) framework was used for the assessment. At just over 18 months into implementation, the assessments were able to find evidence of the first two Es: economy and efficiency. The results are presented here for the e-AWPB and ebidding case studies. The case study of integrated designs for health facility construction required more detailed data collection and follows at a later date. 4. On the basis of the data currently available and using only identified costs and benefits, we conclude that the TA to support development of e-AWPB and e-bidding is demonstrating value for money. Identified expenditure by NHSSP to date on eAWPB is £60,000 and on e-bidding is £54,000. Measurable results to date indicate that the implementation of e-AWPB has halved the time spent by staff on the planning cycle. Users believe that there are additional impacts on accuracy, avoidance of duplication and improved management decisions but these have not been measured. The introduction of e-bidding in 2011/12 has reduced the average price of new contracts by 12%. This is in line with international experience. It has saved MoPH NR 456 million (GB£ 3.4 million) in the first year alone. 5. Cost data are based on a range of different sources. NHSSP technical assistance (TA) costs are recorded on an activity basis by Options in London for reimbursement purposes. Since all the case study programmes were begun under previous phases of DFID-funded support we have had to locate the records of relevant expenditures under these previous programmes. These have often been estimates recalled by those involved at the time, as have GoN costs which are not usually recorded on a programme basis. It is important to have a clear picture of costs and results in the future for a more complete view of VfM. This is not just a question of calculating VfM, but of delivering a successful programme.

3

6 One of the reasons for the lack of consolidated cost information may be that sector-wide support implies wide-ranging, long-term, capacity development in activities that are continuously on-going. This presents a problem for a VFM assessment which requires costs and benefits to be isolated and attributed. There is a need to develop more of a project mode within the programme approach: more up-front definition of the objectives, budgets (both TA and GoN), timeline and targets of each key component of programme support. 7. As a VfM assessment undertaken by economists, the case studies have focussed on what can be measured and quantified in terms of costs and monetisable benefits. This is not a substitute for a technically expert review. Indeed, they should be undertaken together: a truly robust VFM assessment requires specialist technical knowledge in order to best integrate financial and economic information into judgements about feasible options and outcomes. 8. DFID’s Four Es framework is based on an audit approach and is useful for reviewing the implementation process of an intervention that has already been selected and is under way. It does not help to guide the selection and design of an intervention from scratch. Identifying and designing interventions in the health sector is a multi-staged decision process. Ensuring VFM is captured in the decision process requires a different starting point and a quite different framework. Such a framework is proposed at Annex A2.

4

1. Introduction 1.1 The Nepal Health Sector Support Programme (NHSSP) is a DFID-funded £10m programme of technical assistance to the Nepal Ministry of Health and Population (MoHP) and Department of Health Services (DoHS). Along with a pooled health sector financial aid fund of US$220m from DFID1, GAVI, AusAid and the World Bank, its aim is to support delivery of the second phase of the Nepal Health Sector Programme (NHSP- 2) over the five-year period 2010-15. The first phase, NHSP-1, was the first health sector-wide approach (SWAp) in Nepal. It began in July 2004 and ended in July 2010 and was successful in achieving significant improvements in health outcomes. NHSP-2 continues many of the programmes begun under NHSP 1. Its aim is to improve the utilisation of essential health care and other services, especially by women, the poor and excluded. 1.2 NHSSP began with an inception phase in September 2010 during which capacity assessments were undertaken. Full implementation began in January 2011. There are seven outputs for NHSSP: 

     

DOHS / regions have capacity to deliver quality and integrated EHCS, especially to women, the poor and under-served, with a focus on maternal health services (specifically, improving the functionality of CEONC services); MOHP has capacity to develop and implement an effective HRH strategy for the health sector MOHP and DOHS has systems, structures and capacity to implement the GESI Strategy MOHP & DOHS has capacity to develop and implement a transparent and sustainable supply and demand-side financing framework MOHP has capacity to strengthen and effectively use an information system to support planning and delivery of quality EHCS MOHP and MPPW have capacity to develop and implement procurement in accordance with the procurement arrangements for the health sector PPICD has a clearly defined and functional role as the focal point of the planning and policy process for the whole health sector.

1.3 The NHSSP Inception Report2 undertook to “gather case studies on value for money, cost savings and cost-effectiveness”. In July 2012, four programme case studies were selected and independent consultants appointed to undertake a value for money (VfM) assessment. Data was collected and interviews and field visits were conducted in Nepal during July and August 2012.

1 2

DFID’s contribution is £38 million (US$60 million). January 2011

5

2. Assessing Value for Money of Technical Assistance provided by NHSSP to the Nepal Health Sector A VFM Framework: the Four Es 2.1 DFID and other external development partners (EDPs) are putting increasing emphasis on achieving and demonstrating value for money (VFM) in aid spending. The aim, according to DFID, is “to develop a better understanding - and better articulation - of costs and results so that we can make more informed, evidence-based choices…and maximize the impact of each pound spent to improve poor people’s lives.”3 2.2 A common definition4 of VFM is ‘the optimal use of resources to achieve intended outcomes’. This definition highlights how demonstrating VFM involves being clear about both results and costs. Being clear about costs entails knowing what costs are involved; knowing what is driving them and knowing how they compare with other similar examples. Being clear about results is, in principle, more challenging. Results in development arise through an often long and complex linked series of processes. They are also often not evident until well into the future of a programme. 2.3 The solution is to be clear about how results are expected to emerge: how aid inputs are linked to programme outputs; how programme outputs are linked (in this case) to health systems outcomes and how these result in improved health outcomes. This is the familiar DFID ‘results chain’ which also forms the conceptual framework for assessing value for money in something as difficult as development; by breaking the results chain down in this way, it becomes easier to attribute costs and results to each ‘link’. This is the basis for DFID’s ‘Four Es’ VfM framework. Box 1: DFID’s Four Es for Value for Money 1. Economy: Getting the best value inputs. This involves good procurement practices. 2. Efficiency: Maximising the outputs from a given level of inputs. This involves good management processes. 3. Effectiveness: Knowing that the desired final outcome is achieved. This involves good evaluation. 4. Equity: Ensuring the benefits are distributed fairly. This involves good governance in delivery and good impact assessments.

3

DFID’s Approach to Value For Money. Department for International Development, July 2011 UK Nation Audit Office, quoted in DFID’s Approach to Value For Money. Department for International Development, July 2011 4

6

2.4 There is a chronology to the results chain and the Four Es, so that evidence of economy being achieved can be identified first when inputs are procured. Evidence of efficiency emerges somewhat later, when implementation is under way and is affecting processes and intermediate outputs. Effectiveness tends to be identifiable some years down the line. At this stage of NHSSP, just over 18 months into implementation, a VfM assessment can usually only expect to be able to find evidence of the first two Es: economy and efficiency. The Case Studies and Data Limitations 2.5 For a valid VFM assessment at this stage of NHSSP using the Four Es framework, the following criteria had to be met: - discernible results already emerging; - ability to plausibly and as exclusively as possible link NHSSP and other known inputs to these results; - ability to isolate all main costs. 2.6 The initial suggestions for case studies in the terms of reference (see Annex 3) did not meet these criteria. We therefore spent some time identifying alternatives. They are: (i) E-bidding for health facility contracts; (ii) Electronic annual work-planning and budgeting (e-AWPB); (iii) Integrated designs for health facility construction; 2.7 The first two of these case studies are completed and the results presented in this report. The third requires more data collection effort and is going forward in a slightly longer timeframe. 2.8 NHSSP technical assistance (TA) costs were provided by Options in London, which records them on an activity basis for reimbursement purposes. For all of the case studies, activities had begun under previous DFID-funded health programmes. We therefore also traced these costs where possible, relying on advisors’ estimates in some instances for these and also for GoN costs, which are not recorded on a programme basis. 2.9 Good data were available for contract prices before and after e-bidding but this was the only example of a clear baseline that we found. Where possible, results for impacts on staff time or other administrative outcomes were constructed using mostly qualitative information from interviews with advisors and MoHP staff. For future vfm studies, it will be important to develop good baselines.

7

A VFM Framework for the Decision Process: the Five Efficiencies 2.10 Results are presented using DFID ‘Four Es’ VFM framework. However, it is important to note there are limitations to the Four ‘E’s framework for the health sector. The Four Es are based on an audit approach. They are useful for breaking down and reviewing a project implementation plan or a project that is already underway. They do not help ensure another key element in achieving value for money: that the right project or intervention is selected in the first instance – otherwise known as ‘quality at entry’. The identification and design of a new programme in the health sector is a multi-staged decision process. Ensuring VfM is captured in that process from the start requires a quite different framework. Such a framework (named the ‘Five Efficiencies’) for guidance when planning a new health programme is outlined in Annex A3.

8

3. Case Study (1): Electronic Work Plan and Budget (e-AWPB) 3.1 The Electronic Annual Work Plan & Budget (e-AWPB) workstream of NHSSP aims to introduce IT-based financial management systems for budget preparation, execution and reporting into the Government of Nepal’s Ministry of Health and Population (MoHP) and Department of Health Services (DoHS); and to develop financial management expertise in MoHP, both centrally and in the regions.5 3.2 The e-AWPB is being implemented in phases. The first phase introduced the system for the annual planning (‘budget formulation’) cycle at the centre. A second phase - to computerize transactions and ongoing budget management (‘budget execution’) throughout all health sector spending units6 - is just beginning (July 2012). 3.3 Activities and inputs under NHSSP since August 2010 have been: - Adaptation of existing software to introduce enhanced features (eg automated coding and calculations) and reporting capability; - Access via the MoHP website enabled; - Training of about 50 MoHP planning staff in use of the improved system. Background to e-AWPB 3.4 Owing to centralisation of financial management at the Ministry of Finance (MOF), there has always been limited financial management capacity7 in the Planning and Accounts sections of MoHP and DoHS. Furthermore, control and monitoring of spending is limited by the absence of a single centralized system. Although computers are available in most MoHP/DoHS offices, there is no common accounting package, with the result that accountants use individual personal systems to record transactions and prepare financial reports. Most calculations and preparation work is done manually. With over 360 individual spending units8 and a total annual budget of NR 20 billion/£150 million9, the scope for error, abuse and loss of spending control is significant.

5

An equivalent World Bank-led programme of support to public financial management is under way at the Ministry of Finance. 6 Known as TABUCS – Transations Accounting and Budget Control System. 7 Financial management staff members are part of the Financial Comptroller Generals Office. 8 MoHP estimate. A spending unit is any part of the health system which holds a budget and is responsible for expenditure. 9 Nepal Health Budget Analysis 2011-12. NHSSP. NR 25 billion minus the 18% estimated in the Budget Analysis for salaries, which do not go through spending units’ budgets.

9

3.5 The annual work plan and budget (AWPB) cycle begins each March, running through the next 4 months to the start of the financial year in July. For the MoHP, this involves reconciling the proposed budgets and workplans of the 360 spending units with the resource envelope set by the Ministry of Finance and National Planning Commission. Almost all are still prepared manually and must be checked and consolidated first by 12 parent Divisions (eg Family Planning Division or Logistics Management Division) and then finally by the Planning Section of MoHP. It is a time-consuming process, again with considerable potential for error. Capability to do this final budget combination stage electronically is only part of the overall planning process but it represents an important first step.10 Case Study Methodology 3.6 The majority of the benefits of an improved public finance management system arises when it encompasses the many actual transactions (‘budget execution’) and extends to spending units in the districts. This has not yet begun. The case study therefore looks at the first phase/workplan cycle only. The scope of this first phase is much more limited – in this case, to the main central MoHP/DoHS institutions and their small pool of current users - but efficiency improvements (timeliness, accuracy, avoidance of duplication, staff time savings) can in principle be identified. Unfortunately, however, no baseline exists against which to measure any changes and retrospectively constructing one proved too difficult in the time available.11 3.7 We therefore measure efficiency in terms of a single indicator: reported staff time savings. We measure economy in relation to a private sector benchmark: the cost of the system relative to the size of the budget which it is intended to manage. 3.8 We do not attempt to measure effectiveness; the proposed NHSSP outcome measure for the whole eAWPB programme of ‘improved financial management directly impacting on the absorption rate of committed funds’12 could in principle be used here. However, the new system has only become fully operational in 2012 and it is too early to detect its impact on the budget absorption rate. Costs 3.9 We attempted to identify both Government of Nepal and TA costs. Costs were available from NHSSP accounts but GoN costs had to be estimated by MoHP and NHSSP staff. There was no single consolidated source of actual or estimated costs for the programme.

NHSSP Inception Report; Options January 2011 Indicators of relative efficiency improvements and effectiveness are in the process of being defined for the remainder of the e-AWPB programme. 12 NHSSP Inception Report, Options January 2011 10

11

10

Table 1. Total NHSSP Technical Assistance Costs for e-AWPB since August 2010 (GB£) Input

Budget

Actual

Long term TA Short term TA Activities Overheads Total

1,100 28,264 0 16,901 46,265

5,050 8,431 0 16,030 29,511

Source: Options NHSSP Programme Manager13

Table 2. Estimated Government of Nepal Costs for e-AWPB by Financial Year (Nepali Rupees ‘000s) Input Software Local consultant Training Workshop Production of manual MoHP staff Total

2010/11

2011/12

Total

1,000 250 300 0

700 300 350 350

1,700 550 650 350

0 50 1,600

600 50 2,350

600 100 3,950

Source: NHSSP Health Financing Advisor’s estimates

NR4m is equivalent to £30,000, so total identified spend to date is approximately £30,000+£30,000 = £60,000. 3.10 We do not have any estimates of operation and maintenance costs, e.g. of web hosting or annual maintenance, upgrading or replacement, nor of any attributable MoHP overheads. 3.11 When NHSSP began, a ‘simple, interactive database in Access 2003’14 … designed to generate analytical tables in a systematic way’ had already been ‘partly developed and implemented’ in MoHP by Research Triangle Institute (RTI) under the previous DFID-funded Health Sector Reform Support Programme dating from Budget estimate on overheads are based on total overheads for the entire project prorata’d for the current project timeframe; for actuals the overheads are calculated on total actual overheads to date, divided by 7 thematic areas and 6 broad areas of the health financing work (of which e-AWPB forms a part). Advisor costs have been calculated by OPM; activity and TA costs are actuals. 14 Microsoft's database application for desktops. 13

11

2006.15 This support is reported to have been ‘substantial’,16 and (based on advice from RTI)17 is estimated to have cost approximately £25,000. NHSSP advise that there has been substantial enough revision and renewal to ignore this previous investment. RTI disagree, however. To accomodate both points of view, we explore the implications for value for money of including the earlier phase of spending in the VfM assessment section below. Results 3.12 The original RTI software is reported to have been in partial use since 2010 at the Finance and Planning sections of the MoHP and some centres and divisions at DoHS. We were told that by 2012, however, only two out of the twelve divisions of MoHP/DoHS were using it18. This year, all twelve divisions are using the revamped system for the first time. As of July 2012, there are 55 registered users in 34 central health institutions (MoHP, DoHS, divisions, TB centres, hospitals and NPHL). 3.13 There has been no systematic recording of the impact of the new system on time saved but we were told by a sample of users in DoHS that they estimated that the new system had halved the amount of time they now spend on the planning cycle. Value for Money Assessment 3.13 Total identified costs (excluding the earlier RTI phase) are £60,000. With 55 registered users, costs per user to date are £60,000/55 = £1,091.19 The MoHP planning staff we interviewed estimated that they would normally have spent 2 of the 4 months of the planning period on planning work, and that this has now halved. This one month saved out of every twelve is an 8% efficiency improvement overall. 3.14 Assuming the 5520 registered users experience the same time saving and that they have an average monthly salary of NR 30,00021 (£200), the monetary value of the total time saved in MoHP and DHFS is £11,000 every year. Estimates for these costs are not available. Transaction Accounting and Budget Control System (TABUCS): A Concept Note. NHSSP July 2011. 17 RTI estimates (in personal communications) local direct costs of NR2.4 million: NR 750,000 for design and project management; NR 950,000 for programming; NR 700,000 for training and manual printing. For consistency, we have added 40% for RTI overheads to this figure. 18 MoHP ( personal communication). 19 We were told that the system is used most by the Divisions who do the main budget compilation work. An alternative figure would therefore be costs to date per active and most important user group (the 12 divisions) = £60,000/12 = £5,000. This is not a standard unit, however, and is not useful for comparison so we continue to use the per user statistic. 20 Although two out of the twelve divisions were reported to be using the previous system, we have assumed they have benefitted equally from the improvements. 21 Based on an under-secretary’s salary of Rs 26,000 per month plus allowances of approx. NR 4,000 per month. See ‘Commission recommends hefty civil service pay hike’. July 11, 2011 at http://archives.myrepublica.com/portal/index.php?action=news_details&news_id=7309 15 16

12

3.15 To test whether £60,000 is good value for money, we can compare the cost with the value of the time saved per user. We can also compare the total cost with the total value of resources managed. 3.16 We have estimated that the saving per user is one month’s salary per year. Assuming that to be £200 per year and the cost per user to be £1,000, this implies a payback period22 of 5 years. It seems likely that the life of the new system will be at least 5 years23, therefore on this minimum measure, the investment appears to be within the bounds of value for money. 3.17 We have calculated the identified investment cost under NHSSP to be £60,000: 0.04% of the total value of the 2011/12 MoHP budget of £150 million. Private sector experience in introducing similar ‘enterprise resource planning’ (ERP) systems suggests that large companies (with revenues of $1 billion or more) can expect a total cost equal to about 0.5% of total revenues24. ERP costs increase with the financial size of the company, so we should expect the appropriate benchmark for eAWPB to be lower than 0.5%. Even if we reduce it by a factor of ten, however, the VFM comparison looks favourable. 3.18 If we were to include the £25,000 of the earlier (RTI) phase of development of e-AWPB, the total identified costs would rise by over 40% to £85,000. The cost per user would rise to £1,545 and the payback period would rise to 8 years. The increased cost would be equal to 0.05% of the annual resources managed. These results could nevertheless still be regarded as within the bounds of value for money. 3.19 We have noted that other, equally important, impacts – timeliness, accuracy, improved management decisions – have not been quantified or monetised. The literature on ERP systems25 finds that these do not tend to be quantified in the private sector either. Instead, such impacts are usually assessed qualitatively, through surveys of users, and tracked over time. The Public Expenditure and Financial Accountability (PEFA) monitoring approach which is being explored as the basis for monitoring the next phase of e-AWPB (TABUCS) is essentially a 22

The time it takes to recover the cost of the initial investment. The life of an ERP system in a small business may be 5-10 years; for larger organisations they are likely, with modifications, to last considerably longer. 24 Quoted in Financial Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t. World Bank, 2011. See Annex. 25 ‘The literature suggests that management accountants will be less likely to do routine tasks and more likely to be involved with analysis. Similarly, the prior studies suggest that the output of management accountants will likely be more precise, more accurate and produced more frequently. However, there is no conclusive evidence to support these expectations from the research. In summary, there is confusion in the literature as to the potential for ERP systems to change management accounting and a lack of clear identification of the changes that have actually occurred.” B. Jackling & G. Sprackman. The Impact of Enterprise Resource Planning Systems on Management Accounting: an Australian Study. Deakin University, Victoria, Australia, 2005. 23

13

qualitative assessment, benchmarked and systematized so as to measure progress from a baseline. Conclusions and Recommendations 3.19 Due to data limitations, we have restricted our VFM assessment to two basic economy and efficiency measures. We have looked only at the first phase of a substantial programme of change. 3.20 Much of the work under the programme has been in-house, with limited international TA inputs. As a result, introducing/upgrading the whole package appears to have cost a modest £60,000. If the estimated costs of the previous programme of DFID-funded support were to be included, the total cost would rise to a still acceptable £85,000. 3.21 The goal of the e-AWPB programme goes far beyond the results captured by this VFM assessment to enhancing government accountability and transparency as well as improving expenditure management. A sound PFM system not only attracts foreign resources from development partners (and gives those partners confidence to channel them through the national system) but also ensures effective utilization of such resources and establishes transparency and accountability of public funds. We have captured none of this here, largely because the system has not yet been fully implemented and results such as these are still to emerge. 3.22 The time is right, as the main TABUCS phase begins, to develop measures of these impacts. Adapting the World Bank’s PEFA measures 26 as planned is a sensible first approach but they are really only various measures of improving effectiveness over time. The new TABUCS phase should also have economy and efficiency indicators: TABUCS is likely to be a much more substantial investment than the planning phase, with more scope to incur cost over-runs in implementation27; having some indicators of the range of acceptable costs and expected efficiency savings would be valuable. Producing a programme plan and budget which consolidates both NHSSP and GoN expenditure would be a good start.

26

Public Expenditure Framework Assessment (PEFA) designed by the World Bank to evaluate progress in reforming national public expenditure systems. http://www.pefa.org/ 27 Furthermore, there is enough research on the private sector experience in introducing such systems to provide some potentially useful benchmarks.

14

4. Case Study (2): e-bidding for Health Facility Construction Contracts 4.1 The construction of health facilities in Nepal is contracted out to the private sector by the Department of Urban Development and Buildings Construction (DUDBC) on behalf of the Ministry of Health. As part of a wider Government programme of e-procurement, 28 DUDBC has recently introduced electronic publication of tender notices, electronic access to tender documents, and electronic submission of tenders. 4.2 The introduction of this ‘e-bidding’ was begun under the DFID-funded Support to Safe Motherhood Programme (SSMP) in 2008,29 including software development and testing in DUDBC .30 Support has continued under NHSSP to: - Standardize bidding documents and procedures; - train DUDBC staff members at central and district levels to implement ebidding; - educate stakeholders and potential contractors on the changes and new requirements; - develop legal requirements and e-security (such as digital signatures). 4.3 E-bidding for health facility contracts went live for the first time in FY 2010/11. Background to e-bidding 4.4 There is a perception of lack of competition in the procurement of public contracts in Nepal in general. The level of competition for Government contracts is low: the average number of bidders per GoN contract is 4.5 compared, for example, to an average of 7.3 in World Bank-financed projects31. We do not have equivalent data for the health sector specifically, but anecdotal evidence suggests something similar exists. Local news reports violence and intimidation against contractors placing bids in a range of sectors. Lack of competition, with evidence of collusion 28

The Government made e-procurement mandatory in all public procurement over NRs20 million in 2011 and intends to expand it to all procurement from 2012. 29 NHSSP Infrastructure adviser: “Workshops were organized in November 2008 and January 2009 for consultation on initiation of e-bidding for works and goods in the health sector. The contract for software development and for making DUDBC ready for e-bidding was let during SSMP (7th April to 15th May 2010), with the repair and maintenance contract extending up to NHSSP period until March 2011). Training to bidders, other orientations, initiation of e-bidding initiated after NHSSP in Jan 2011. “ 30 Inception Report, NHSSP 2010 31 R. Benamghar & A Iimi. Efficiency In Public Procurement In Rural Road Projects Of Nepal World Bank Policy Research Working Paper 5736. July 2011.

15

and bid-rigging, is believed to keep contract values higher than they should be. (There are also health sector specific issues around sub-optimal design and maintenance of facilities, which are being tackled through other parts of the NHSSP programme.)32 Case Study Methodology 4.5 Reasons for introducing e-bidding usually include: - improved administrative efficiency (mainly through moving away from a paper-based process); - increased openness and transparency in government business; - increased competition and reduced prices for government contracts. 4.6 We have attempted to quantify only the last of these – increased competition and reduced prices - due to lack of data on other areas. However, there are linkages between them all: there is evidence33 that increased openness impacts on levels of competition and therefore prices of contracts. There is also evidence that intense competition in public tendering reduces the risk (and therefore the cost) of post contract amendments.34 4.7 Data for 985 different health facility contracts worth over NR 15 billion (GB£ 110 million) over the period FY 2004/05 to 2011/12 was provided by NHSSP. We selected 553 contracts for analysis, of which 416 were let and agreed in the five-year period before e-bidding began in FY 2010/11 and 137 in the one year since. 4.8 Before calling for a bid, DUDBC estimates an expected value for every contract using standard unit cost estimates35 for different components of the facility design. We calculate by what percentage the winning bid is below this estimate before and after the introduction of e-bidding. DUDBC estimates are updated annually for each district by the District Development Committee (made up of technicians, contractors and others) based on local market trends in prices for transportation, labour rates etc. It is believed that these estimates may be themselves inflated to provide cover for over-priced bids. We assume that, if and where this occurs, the percentage by 32

Eg, the introduction of integrated designs for health facilities, the subject of a separate VFM case study. 33 In Nigeria, for example, a survey of private firms on perceptions of public sector procurement reforms found that the majority had decided not to submit expressions of interest in recent public tenders because they did not trust the selection process. There was a general perception that contractors are predetermined or must pay a bribe (particularly at the subnational levels). 34 It is argued that contractors selected through intense competitive bidding are more likely to be well disciplined and therefore less likely to provoke renegotiation about costs or miss the project completion date.

16

which estimates are inflated remains constant across the six years under review. A separate comparison is done for each of five of the smaller facility types (basic obstetric unit, comprehensive obstetric unit, health post, health post with a two bed unit for maternity cases and primary health care centre). Hospitals were not included,36 nor were other contract types for which the numbers were too small for comparison. 4.9 Due to the weakness of the data, we have not calculated a discounted cash flow. Costs 4.10 Options report TA costs under NHSSP to date of £30,556 – substantially less than the £44,000 estimated at inception. In addition, costs of just over NR 2 million37 (approximately £15,500) for both the software development and new hardware for the e-bidding platform in DUDBC and TA of approximately £7,500 incurred under SSMP have been identified. Table 3: Total NHSSP, SSMP and GoN Technical Assistance Costs since August 2010 (GB£) Input Software development Hardware Long term TA Short term TA Activities Overheads Total

NHSSP Budget

NHSSP Actual

SSMP Actual

0

0

16,733 2,000 11,100 14,487 44,320

15,680 666 470 13,740 30,556

5,370 10,185 3,000 1,444 1,452 1,666 23,117

Source: Options NHSSP Programme Manager38

The total identified costs to date are: £30,556 + £23,117 = £53,673.

36

A few, exceptionally large, observations such as hospital construction costs could skew the results. NR725,000 for software development and NR 1,375,000 new hardware. Records provided by NHSSP Infrastructure adviser. 38 For NHSSP, overheads are based on total prorata’d for budget estimates and project to date for actual, divided by 7 thematic areas and 7 broad areas of procurement and infrastructure work; advisor costs are based on total costs of advisor programme to date divided by number of key areas on which the advisor is working; activity and TA costs are actuals. 37

17

4.11 We have not been able to isolate GoN staff costs of introducing e-procurement, nor identify any subsequent possible cost efficiencies since there is no programmebased reporting of Government expenditures or savings. Results 4.12 We found that for the five years preceding e-bidding, the winning bid price across the five types of facility averaged 9% below the DUDBC estimate. In the one year following the introduction of e-bidding, the average difference between winning bid prices and DUDBC estimates averaged 21%. The results are set out in Table 4 below. Table 4: Selected Health Facility Estimated and Actual Contract Prices 2004/05-2011/12

A. Type of Facility

B. Number of Projects

C. MoHP Estimated Total NR 000'

D. Agreed contract Total NR 000'

Difference between Estimate and Agreed Contract (C-D) Total NR 000'

Difference between Estimate and Agreed Contract (C-D) Percentage (%)

Without E-bids

With E-bids

Without E-bids

With E-bids

Without E-bids

With E-bids

Without E-bids

With E-bids

Without E-bids

With E-bids

BEOC

27

11

211,948

136,009

194,021

113,431

17,927

22,577

8

17

CEOC

21

3

189,109

395,73.61

169,473

30,504

19,636

9,069

10

23

HP HP_2mat

190

79

2,461,788

1,607,804

2,276,045

1,281,736

185,743

326,068

8

20

39

84

28

280,971

129,559

243,415

93,505

37,556

36,059

13

28

PHCC

94

16

2315,113

931225

211,0530

718,684

204,582

212,541

9

23

Total

416

137

5,458,929

2,844,170

4,993,485

2,237,861

465,444

606,310

9

21

Value for Money Assessment 4.13 A cost saving of 12% on the annual MoHP public works construction budget in the first year has been achieved. With a budget for new build and renovation of NR 3.8 billion in 2011/1240, a 12% saving amounts to NR 456 million (£3.4 million). 4.14 Against a total identified expenditure (TA and GoN capital spend) of under £54,000, we have found first year savings of £3.4 million. This level of saving can be expected to continue into the future and may even increase as more potential

39 40

Health Post with two additional beds for maternity cases. Nepal Health Budget Analysis 2011-12. NHSSP, 2012.

18

contractors are encouraged to bid and this, as evidence suggests, continues to reduce expected prices. 4.15 Our assumption that the margin of error in DUDBC construction cost estimates remains constant across the six years under review ignores the possibility that estimates in the year e-bidding was introduced may have anticipated the impact of the new system. This would most likely act to reduce the over-inflation element in the estimate, leading us to under-estimate the impact of e-bidding. However, the 12% saving on contract prices is consistent with international experience. In Norway, the ‘e-handel’ platform is reported to be delivering contract price savings in the region of 2-10%. The UK has reported savings ‘frequently exceeding 10%’ through the use of e-auctions. A comparison of public hospitals procured in 2009 (using paper based systems) and 2010 (using e-Procurement) in Portugal concluded that a cost reduction of 18% had been achieved due to the increase in competition generated by e-procurement. 4.16 We were not able to identify the impact (costs or benefits) on administrative efficiency as a result of the introduction of e-bidding. While computerization may be expected to reduce the time spent on many related activities, full realization of such efficiencies often depends on new ‘back-office’ processes and organizational reform. There are some consequences of e-bidding which may actually increase the administrative load; in the road sector in Nepal, the time required to evaluate bids and award a contract was found to increase with the number of bids received. The World Bank study estimated that one additional bid could add 12 days on average to the evaluation period. 41 Conclusions and Recommendations 4.17 We have found that the introduction of e-bidding has generated contract price savings in the first year of 12% of the MoHP construction budget, worth £3.4 million. On the basis of the costs identified, this generates a first year benefit-cost ratio of 70:1. We have not identified all costs due to lack of data, especially on the GoN side. However, unidentified costs are very unlikely to be high enough to significantly undermine this conclusion. The level of savings is consistent with international experience. 4.18 We can infer from the price savings achieved that competition in bidding has increased. This conclusion could be confirmed with evidence that the average number of bidders per contract has risen and/or that the difference between the winning bid price and the second-best losing bid price (an indicator that bid-rigging

41

R. Benamghar & A Iimi. Efficiency In Public Procurement In Rural Road Projects Of Nepal World Bank Policy Research Working Paper 5736. July 2011.

19

through ‘cover pricing’42 is occurring) is reduced. We understand these data exist in DUDBC but we have not yet been able to access them. 4.19 While we can conclude that the introduction of e-bidding has been good value for money, there is evidence that additional, low or no cost, actions can be useful in increasing competition. The World Bank study of rural roads suggests several actions that could be replicated in health facility procurement: - a longer bid preparation period can reduce the entry barriers for potential contractors, particularly inexperienced firms. In addition, having a standard minimum bid preparation time would help combat corruption since one of the indicators of possible corruption or collusion is that an invitation to bid is published just a few days or a few weeks before bids have to be submitted. - reducing the cost of bidding documents also reduces the costs of entry, especially to small firms. While Government would lose the revenue arising from the charge, the impact on the number of bidders and average prices could be significant. - advertising in national rather than local newspapers The World Bank study estimated that if the media is changed from local to national newspapers, bidders for roads contracts would increase by about 20%, which would contribute to decreasing the procurement costs by 6-7%. 4.20 In conclusion, e-bidding for health facilities appears to have been developed with good economy, although many costs remain unidentified. E-bidding has had an immediate impact on the efficiency of the procurement process, in that average contract prices have fallen. Other potentially important efficiencies in related administrative processes have still to be demonstrated.

42

Cover pricing is when colluding companies agree to submit seriously under-priced bids, in order that one of their number is guaranteed to win. Also known as ‘low-balling’.

20

5. Conclusions and Recommendations 5.1 On the basis of the limited data and using only identified costs and benefits, we conclude that the e-AWPB and e-bidding programmes [and integrated designs for facility construction?] are showing value for money. 5.2 The e-AWPB programme has halved the time spent by staff on the planning cycle. Based on the salary cost of time saved by planning staff only, the investment to date would be paid back in 5 years. Using the simple rule of thumb that, for any investment, the pay-back period should be less than the life of that investment, this phase of AWPB passes a minimum VfM test43. This ‘monetised’ result would appear more favourable if Nepal civil service salaries were not amongst some of the lowest in the world.44 Other impacts on accuracy, avoidance of duplication or improved management decisions are not included. They were not measured but are felt by users to be significant. 5.3 The introduction of e-bidding in 2011/12 has reduced the average price of new contracts by 12%. This is in line with international experience. It has saved MoHP NR 456 million (£ 3.4 million) in the first year alone. 5.4 Identified expenditure under NHSSP on e-AWPB is £60,000 and on ebidding is £54,000. Both therefore appear to be relatively low cost, with limited international TA inputs. 5.5 We were not able to identify all GoN financial costs or administrative impacts, whether positive or negative. The introduction of e-procurement in particular can increase as well as reduce administrative costs and processes. It is important to have a clearer picture of these in the future for a more complete view of VfM. This is not just a question of calculating VfM, but of delivering a successful programme. 5.6 The lack of information itself is likely to reduce VfM; its absence hampers the identification of emerging problems or successes and reduces the speed of response. 5.7 One of the reasons for the lack of consolidated cost information may be that sector-wide support implies wide-ranging, long-term, capacity development in activities that are continuously on-going. There are few clear beginning and ends to activities or spend. This presents a problem for a VFM assessment which requires costs and benefits to be isolated and attributed. There is a need to develop more of a project mode within the programme approach: more up-front definition 43

See footnote 19.

44 Madhu Nidhi Tiwari Governance Reform in Political Transition: The Case of Nepal’s Civil

Reform. Nepalese Journal of Public Policy and Governance, Vol. xxiv, No.1, July, 2009

21

Service

of the objectives, budgets (both TA and GoN), timeline and targets of each key component of programme support. 5.8 A value for money assessment does not demand every output or outcome to be monetisable. This will often not be possible. However, when outputs are definable, they are usually quantifiable, or progress against a baseline is measurable. There are often equivalent commercial or civil society sectors which provide examples and benchmarks. 5.9 As a VfM assessment undertaken by economists, these case studies have focussed on what can be measured and quantified in terms of costs and monetisable benefits. This is not a substitute for a technically expert review. Indeed, they should be undertaken together: a truly sound and robust VFM assessment requires a specialist knowledge of the technical dimensions in order to best integrate financial and economic information into judgements about feasible options and outcomes. 5.10 DFID’s Four Es framework is based on an audit approach and is useful for reviewing the implementation process of an intervention that has already been selected and is under way. It does not help to guide the selection and design of an intervention from scratch. Identifying and designing interventions in the health sector is a multi-staged decision process. Ensuring VFM is captured in that process requires a different starting point and a quite different framework. A proposed VFM framework (named the ‘Five Efficiencies’) for guidance when planning new health interventions is at Annex A2.

22

Annex A1 Electronic Work Planning and Budgeting: Comparisons with the Private Sector
 Taken From: Financial Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t. World Bank, 2011. Financial Management Information Systems (FMIS) solutions used in public sector projects were originally designed to address similar needs in the private sector. Hence, an overview of the implementation of similar information systems in the private sector is presented below to highlight some of the common patterns of success and failure. 
 Information systems similar to FMIS are called Enterprise Resource Planning (ERP) systems in the private sector. 
 ERP solutions are designed to integrate all departments and functions across a company into a single computer system that can serve different departments’ particular needs. An ERP functional module generally includes Finance, HR, Manufacturing, and the Warehouse. In general, private firms initiate ERP implementation by installing Finance and HR modules first, and expanding gradually to have the full package for domestic and international business needs. 
 As in public sector projects, commercial ERP packages require a substantial and costly change in the ways companies do business. In most cases, it takes several years to implement such changes. ERP vendors usually promise to implement the core modules in three to six months. This may be true for small firms, in which case the ERP system is nothing more than a very expensive accounting system. For a comprehensive ERP implementation, business processes need to be changed and staff must be trained in the new rules and procedures. Such complex transformational ERP projects usually run between one and three years, on average, despite high levels of ICT literacy in private firms and the large number of qualifed consultants involved. Transformations enabled by the Internet, such as cloud computing or software as a service (SaaS) may reduce the implementation time of ERP packages. Nevertheless, changing business processes and company culture remain lengthy processes. Moreover, storage of confidential or business critical information in remotely hosted “trusted” servers should be handled carefully. Many companies are also benefitting from mobile services and advanced Web applications (Web 2.0) to improve the efficiency and reduce the cost of operations. The manifestation of these new trends in the public sector are not visible yet, due to the special arrangements needed to meet specific data storage and information security needs of PFM information systems, as well as the size of the problem (a relatively large number of concurrent users and connected nodes). One of the most often-cited studies of the total cost of ownership of ERP, completed by Meta Group in 2002 (acquired by Gartner in 2005) taking into 23

consideration the hardware, software, professional services and internal staff costs, plus two year maintenance required, revealed that among the 63 companies surveyed— including small, medium and large companies in a range of industries—the average total cost of ownership was $15 million (the highest was $300 million and lowest was $400,000). Results from a 2007 Aberdeen Group survey of more than 1,680 manufacturing companies of all sizes found a correlation between the size (users and nodes) of an ERP deployment and the total costs. According to the report, “as a company grows, the number of users goes up, along with the total cost of software and services.” For example, large companies with more than $1 billion in revenues can expect to pay, on average, nearly $6 million in total ERP costs. This is comparable to implementation within the public sector. Interestingly, internal resistance to change is often mentioned as one of the key factors for failure in private sector ERP projects. Another failure factor is the high level of customizations which make the ERP software more unstable and harder to maintain when it is finally activated. Not surprisingly, these are similar to the patterns of failure visible in the public sector as well.

24

Annex A2 A Framework for ensuring VFM in health programmes at the design stage: ‘The Five Efficiencies’ The notion of value for money is intuitively simple. In practice, however, the measurement of VfM is challenging. A start can be made by seeing VfM as a ‘process of successfully transforming financial resources into valued health system outputs.’45 There are several stages to that transformation, each of which can be measured with different degrees of accuracy and ease. The framework proposes five different stages (called here ‘efficiencies’) which are common to all health interventions. Each stage or efficiency can be assessed separately, generating a set of VfM indicators for the intervention under review. This should also enable an element of diagnosis in the process, allowing the assessor to isolate where, amongst the five efficiencies, particular problems or potential for improvement lie. The five efficiencies are as follows. They are summarised in a matrix in the Annex, which includes suggestions on how they may be measured. 1. Allocative Efficiency – This is the extent to which the limited funds of the health system are directed towards the ‘right’ health issues – ie the most serious health burdens. Does the financial size of the programme under review reflect the scale of the problem? Is the relative size of the programme in the total health budget commensurate with its relative importance as a health issue nationwide? International guides to the health budget allocation decision, such as the health-related Millenium Development Goals or Essential Medicine Lists - as well as other micro-level planning tools for national prioritisation, such as WHO’s Lives Saved Tool (LiST) - are aimed at improving the allocative efficiency of existing intervention mixes. Theoretically, the allocation decision is the start of the sector planning process. In practice, however, the process never begins by scratch; how the health budget is actually allocated is heavily determined by pre-existing spending patterns. Changing spending patterns to improve allocative efficiency is therefore likely to be a slow and incremental process. As we have said, cost–effectiveness is not the only consideration when deciding on an optimal mix of interventions. In cases where fairness, equity or basic decency are at issue, the social value of a particular health intervention may differ from the value of the health benefits it produces. 2. Technical Efficiency - Technical efficiency refers to the extent to which resources may be being wasted on inappropriate or poorly implemented 45

Peter C Smith. Measuring value for money in healthcare: concepts and tools 
 Centre for Health Economics University of York. September 2009.

25

interventions. Health systems should spend money on a chosen intervention only if it is effective in a particular setting and if the same results could not be obtained in another way for lower cost. In this framework, technical efficiency is intended to refer specifically to medical/scientific evidence and choices - before questions around delivery are considered. It can also refer to other types of ‘pure technology’ questions, however, such as the evidence for the chosen standardized integrated design for health buildings in the infrastructure programme. In all cases, there is potential for efficiency gains from (i) a better mix of interventions or (ii) better (ie less wasteful) implementation of the chosen intervention. The former depends on the availability of (and evidence for) interventions that can substitute to some degree for one another. 3. Delivery Efficiency - Programme managers must choose how to deliver the selected intervention to the targeted communities, households, and individuals. For example, bednets can be distributed in conjunction with immunization campaigns, sold commercially with or without subsidy, or delivered by community health workers. Delivery efficiency also refers to the choice between public and private sector provision, the contracting of NGOs, or the balance between the three. These delivery models may differ in effectiveness or cost, and as a result there may be efficiency gains from switching. 4. Administrative Efficiency – While delivery efficiency (above) will include consideration of delivery-specific administrative costs, this fourth category aims to capture the role of the wider health system in the cost-effectiveness of the particular programme being assessed. Administrative efficiency is achieved when those elements of the wider health system necessary for the particular programme are present, functioning and operating at low cost. It is not enough to consider the share of total expenditure going to overheads, however defined: although reducing administrative expenses might free resources to be spent on prevention and treatment, spending more on management and training might in some cases improve planning and implementation and thus increase overall efficiency. For this reason it is difficult to determine the “right” amount to spend on programme management, though cross-country comparsions and broad benchmarks do exist. Administrative efficiency also involves institutional incentives and accountability. The WHO list of major causes of inefficiency (Box 1, above) suggests outright fraud and corruption also may be important in some places. 5. Aid Efficiency - Forty percent of health funding in Nepal comes from donors. It is therefore reasonable to assume that the way these donor resources are provided, eg in vertical programmes, multi-donor trust funds or via resultsbased aid (RBA) arrangements, has important implications for value for money.

26

Prospective and Retrospective Assessment and the VfM Cycle In the matrix we make the distinction between prospective and retrospective VfM assessment. This is because different programmes are always going to be at different stages of implementation, with different types and quantity of associated information available for the assessor.46 Different types of indicators of VfM, with different levels of certainty and robustness, will result. The VfM assessment framework needs to be able to accommodate this.

Box 2: DFID’s VfM Cycle

Including an option for either a Source: DFID’s Approach to VfM, July 2011 prospective or retrospective VfM assessment reflects the ‘transformation process’ of turning resources into health outcomes referred to earlier (para 2.1), which has elsewhere been called the ‘VfM cycle’: achieving VfM is the planning and corporate management processes of identifying need, appraising options, commissioning and monitoring delivery, evaluating outcomes and feeding the learning from that back into the start of the process. The existence of an effective VfM cycle at a health system-wide level is critical for achieving VfM in individual programmes. It is also important when capturing VfM in a single measure is elusive: the second-best alternative for demonstrating VfM is always to ensure that at a minimum, the right processes are in place. This is what we are attempting to capture and isolate in the fourth ‘administrative efficiency’.

46

Prospective VfM assessment is equivalent to DFID’s appraisal stage of a programme, while retrospective assessment is similar to evaluation. In VfM assessments, however, in each case there is more emphasis in linking costs to (expected or actual) outputs and outcomes - and benchmarking them.

27

Conceptual Framework for VFM Assessment of Health in Nepal: ‘The Five Efficiencies’ Definition

Prospective Indicators (appraisal)

Retrospective Indicators (evaluation)

DFID’s four Es

Sources of info for each case study

Allocative efficiency is achieved when the current Is the programme directed set of interventions maximizes health given the to tackling a proven high budget constraint. priority?

Global health burden

Actual cost per QALY added comparison

Effectiveness

Health sector strategy and background papers.

2 Technical Efficiency

Research & evidence

1 Allocative Efficiency

Is the right medicaltechnical option being used?

3 Delivery Efficiency Is the service being delivered to the user through the most costeffective channel(s)?

Technical efficiency is achieved when as much as possible is produced with the inputs employed, and at minimum cost.

Delivery efficiency is achieved when no alternative channel can be demonstrated to reach more people or reach the same number at lower cost.

Distribution of burden Total health budget

Technical evaluations Modifications for context

Distribution of impacts

Health sector budget

Budget savings Actual unit costs achieved compared to i) expectations; ii) international comparisons iii) rejected options.

Results chain

Indirect benefits/costs generated

Evidence and evaluations of feasible public, private, NGO, joint implementation options.

Actual unit costs compared to i) expectations; ii) rejected options.

Evidence of potential cost-saving features.

Equity

Evidence of cost-savings achieved. Measures of patient

28

Efficiency

Project proposal and forecast budgets. Budget outturns. Annual reviews.

Efficiency

Options appraisals

Economy

Project monitoring reports Cost summaries User Surveys

4 Administrative Efficiency

Administrative efficiency is achieved when those elements of the wider health system necessary Are all back-office for the particular functions necessary for the VfM cycle in place and programme are present, functioning and operating at are related overhead low cost. costs proportionate?

Unit cost implied by programme budget & outputs falls within a max-min.

experience (promptness, dignity, empowerment)

Systems for monitoring costs & outputs

Evidence of improvements in VfM cycle in response to needs of programme

Processes for controlling abuse & corruption Expected distribution of impacts is understood Costs for the above attributable to programme

5 Aid Efficiency Does the way donors provide funding keep transactions costs low and foster domestic health institutions?

Aid efficiency is achieved when no other aid instrument could deliver funds at lower cost or greater benefit for health system strengthening.

Efficiency

Annual budgets Annual Reviews

Actual distribution of impacts identified Costs of the above Evidence of savings

Aid Effectiveness assessments …

….

29

Annex A3 Terms of Reference 1)

Background

The Government of Nepal (GoN) is committed to improving the health status of Nepali citizens and has made impressive health gains despite conflict and other difficulties. The Nepal Health Sector Programme-1 (NHSP-1), the first health Sector-Wide Approach (SWAp) in Nepal, began in July 2004, and ended in midJuly 2010. NHSP-1 was a highly successful programme in achieving improvements in health outcomes. Building on its successes, the MOHP along with External Development Partners have designed the second phase of the Nepal Health Sector Programme named as NHSP-2, a 5 year programme, which will be implemented from mid-July 2010. The goal of NHSP-2 is to improve the health status of the people of Nepal, especially women, the poor and excluded. The purpose is to improve utilisation of essential health care and other services, especially by women, the poor and excluded. Options Consultancy Services Ltd (Options) and partners are providing technical support to the GoN to implement NHSP-2. Specific background Our programme of support to the GoN (the Nepal Health Sector Support Programme – NHSSP) is based upon a framework of capacity enhancement (CE) to the Ministry of Health and Population (MOHP), Department Of Health Services (DOHS) and Regional Health Directorates (RHDs). The National Audit Office defines VfM as being ‘the optimal use of resources to achieve intended outcomes.’ A recent paper defines DFID’s approach in aid programmes as ‘maximising the impact of each pound spent to improve poor people’s lives.’47 Rationale As part of NHSSP ongoing commitment to support the MoHP, DoHS and RHDs achieve value for money (VfM) in the delivery of NHSP-2 it is proposed that 4 case studies are developed which demonstrate the VfM of specific activities or interventions undertaken through the collaboration between government, NHSSP and other partners. The findings will be useful to guide the policy and management decisions needed to continue and /or modify such activities in order to ensure improvement in and sustainability of results. The VFM case studies, where possible, will provide indications of monetisable savings. However, the studies will not be limited to saving money, and should encompass a range of areas including more speculative areas where savings could be attributed, and finally areas where no attribution was possible, i.e. the ‘value’ could be other non-financial values. 47

DFID’s Approach to Value for Money (VfM). DFID, July 2011.

30

The areas identified for the case studies are: Safe Motherhood: guidelines for Dip. Of Gyn/Obs, SBA training review, blood transfusion guidelines and training, CEONC strengthening 1. Gender Equality and Social Inclusion: Social audit 2. Gender Equality and Social Inclusion: One-Stop Crisis Management Centres (OCMCs) 3. Infrastructure: standardized designs and strengthening of the Health Infrastructure Information System (HIIS) 2)

Assignment Objectives

Overall Purpose The overall purpose is to review documentation and carry out interviews and consultations with key stakeholders in order to produce 4 case studies of VfM in activities carried out by government and NHSSP. The main objective of the assignment is to develop 4 VfM case studies which can be used by the MoHP, DoHS, DFID and NHSSP in demonstrating the quantifiable and non-quantifiable results from specific changes resulting from the collaboration between government, NHSSP and partners, . Specific objectives: The specific objectives are 1. to describe, analyse and assess in terms of Vfm the four case studies, including the following aspects:  Objectives of the projects/activities  Practice: o History of key events o Structures o Activities  Performance/results , including analysis of key strengths and weaknesses  Costs  Contextual factors affecting process and results/analysis of why events and results occurred as they did  Analysis of VfM: economy, efficiency, effectiveness, cost-effectiveness  Potential for sustainability and scale-up 2. To make recommendations which will assist in maintaining and improving the results of each of the 4 cases studied (and in guiding scaleup?) 3. To analyse the 4 cases in order to draw more generalised conclusions about the lessons learned in achieving or not VfM.

31

4. To prepare 4 case studies of VfM (and an overview report of the consultancy). 3) Activities The International consultant will: o Hold initial briefings with the NHSSP Advisors, MoHP / DoHS counterparts and local consultants. o Carry out desk-based review of current information on the 4 areas identified as VfM case studies o Develop a workplan and methodology with relevant NHSSP Advisers and the local consultants o Carry out field visits and interview key informants and stakeholders – service providers, beneficiaries, local and international partners, government officials and others- to obtain required information o As necessary, train interviewers to collect data, and arrange for transcription of interviews and translation into English o Develop case studies based on Economy, Efficiency, Effectiveness, and Cost Effectiveness o Prepare a draft report o Facilitate a workshop to review the draft report with key stakeholders and revise it based on feedback received. The report should follow the outline of: o Executive Summary o Introduction o 4 case studies, including objectives and research questions, methodology, results, conclusions and recommendations for each case o Overall conclusions and recommendations re VfM o References o Annexes 4)

Outputs o Development of an analytical framework for measuring and improving VFM performance in health service delivery (with possible relevance for broader service delivery) o Recommendations on how we could use the analytical framework and case study findings for future programme development such as the selection of aid instruments o Analysis on the use and appropriateness of results based aid approaches o Draft report of four case studies that can be utilised by various actors, in particular DFID, to demonstrate VFM. o Final report of case studies o A consultancy process report which provides an overview of the consultancy, an executive summary of the 4 case studies, the methodology used to prepare the case studies (unless this is in the case studies themselves), the list of interviewees and documents reviewed.

32

Soft copies of both the draft and final draft reports should be provided in word doc. format. Three hard copies of both reports should also be submitted. 5)

Timing and duration

The assignment will take place between 8th June and 31st July 2012 with the international consultant providing in-county inputs between the above dates.

33

Annex A4

People Consulted Name

Position

Organisation

Alison Dembo Rath

NHSSP Technical Director

Options Ltd

Anil Maharjan

Senior Technical Officer

Astrid Thygesen Chhaya Jha

Senior Procurement Advisor GESI Advisor, NHSSP

Nepal Central Blood Transfusion Service NHSSP

Damodhar Adhikari

Health Financing Advisor

David Hepburn

NHSSP

Deborah Thomas

Senior Procurement Advisor Gender/Equity Mentor

Duncan Overfield

Value for Money Advisor

DFID Nepal

Dr. Ganga Shakya

Maternal Health Advisor

NHSSP

Greg Whiteside

NHSSP

Hom Nath Subedi

NHSSP Senior Quality Assurance Advisor EAP Advisor

Jabinder Panday

Planning Officer

Kirstan Hawkins

NHSSP Technical Director

Department of Health Services Options Ltd

Krishna Bohara

Under Secretary

Nepal Ministry of Finance

Matthew Gordon

DFID Nepal

Maureen Dariang

Human Development Team Leader EHCS Advisor

Dr. Mukunda Sharma

Pathologist

Nancy Gerein

NHSSP International Lead

National Public Health Laboratories NHSSP

Rajan Adhikari

Planning Officer

Ramchandra Man Singh

Sitaram Prasai

Health Systems & Governance Adviser NHSSP Programme Manager GESI Advisor

Sudhira Acharya

Planning Officer

Sarah Hepworth

34

Nepal Ministry of Health & Population RTI Ltd

Options Ltd

NHSSP

NHSSP

Nepal Ministry of Health & Population NHSSP Options Ltd NHSSP Department of Health

Services NHSSP

Sunil Khadka

Infrastructure Advisor

Shree Krishna Bhatta

M&E Director

Suresh Tiwari

Health Financing Advisor

Nepal Ministry of Health & Population NHSSP

Tomas Lievens

NHSSP Technical Director

OPM Ltd

35

Annex A5 Documents Reviewed Annual Review of the Nepal Health Sector Support Programme. January – December 2011. NHSSP 2012. DFID’s Approach to Value For Money. Department for International Development, July 2011. Efficiency In Public Procurement In Rural Road Projects Of Nepal. R. Benamghar & A Iimi. World Bank Policy Research Working Paper 5736. July 2011. Financial Management Information Systems: 25 Years of World Bank Experience on What Works and What Doesn’t. World Bank, 2011. Measuring value for money in healthcare: concepts and tools.
 P. C Smith. Centre for Health Economics University of York. September 2009. Nepal Health Budget Analysis 2011-12. NHSSP, 2012. Nepal Health Sector Support Programme Inception Report Options Ltd, January 2011 Public Expenditure and Financial Accountability Programme (PEFA). http://www.pefa.org/ The Impact of Enterprise Resource Planning Systems on Management Accounting: an Australian Study. B. Jackling & G. Sprackman, Deakin University, Victoria, Australia, 2005. Transaction Accounting and Budget Control System (TABUCS): A Concept Note. NHSSP July 2011.

36