How to write your own value for Money case study

Accountability in Tanzania Programme

How to Write your own Value for Money Case Study

To be used together with the main report ‘Value for Money of AcT Partners Results’ accessible at http://www.accountability.or.tz/?attachment_id=1035

1

How to write your own value for Money case study

Background

As the case studies were being shared for discussion with other partners, it also became clear that partners would benefit from a much more detailed example of how the metrics and explanations for the different VFM elements were derived.

This short note provides clarity on the basic information that is needed to demonstrate value for money (VFM), along with a step-bystep sample calculation of VFM, based on Case study 1 of the main VFM Report. It shows how the different VFM elements were established as a guide for others who wish to use the approach to document their own case studies.

Why you might want a case study

In 2012 AcT intensified efforts to work with partner organisations to strengthen their capacity to document all the necessary details needed to establish VFM of their interventions. Such capacity building has been organized by way of general and tailored training and has also been in-built in various reporting requirements.

How to establish the different VFM elements

In addition, AcT commissioned a study to examine a sample of partner results with a view to develop a set of case studies which demonstrate the value of what is being achieved by partner organisations. The intention was also to develop an approach for establishing VFM and to determine how it can be adopted for application by partners more widely. To that effect, the study identified specific metrics and methods for calculating and explaining whether or not a programme represents VFM. The exercise documented five case studies from different partner organisations. From that experience, we have found that the exercise has the potential to throw some useful light on the work Civil Society Organizations (CSOs) and donors do. However, we also recognise that there are costs involved in the dedication of time and energy to this process. An important prerequisite to the documentation of the five case studies was the availability of clear information about the results that are being achieved and the processes leading to their achievements. This includes both the qualitative and quantitative aspects of the results and the mix of strategies employed. After all, VFM is not just about numbers and quantitative measures as sometimes the most important results do not have monetizable or quantifiable value. 2

To demonstrate the cost effectiveness of what your organisation is doing, to demonstrate the value of governance work to other programmes, as well as to compare two kinds of similar projects within your own organisation.

STEP 1: Produce an overview of the results chain When this overview is comprehensive enough it provides an important starting point for accurate estimation of the costs and benefits. The overview needs to provide details of strategies employed and justification for their choice because if the reader does not understand the programme well enough or the working context it can be too easy to rush to incorrect judgment about the choices made. What to do:  Conceptualize and document your results

chain from inputs to the specific higher level result  Ensure that your overview includes a summary of the time taken, the costs and benefits at each stage, together with key aspects of the strategies and possible alternative strategies that could be used in delivering interventions. STEP 2: Estimate the cost of the inputs to produce the result This step is where most of the work lies, especially when cost estimates are not easily produced (e.g. in the case of shared staff and activities). Clarity on how these costs are estimated is important because it will facilitate making comparisons between programmes.

How to write your own value for Money case study What to do:  Identify the quantities and costs of each

input identified in your results chain. This may include staff time (for all programme and support staff ), materials, communication and all logistics involved in undertaking related activities by (a) your organisation and (b) others e.g. community members’ inputs in form of time, money and material. If the costs that are directly attributable to the intervention are not easily determined, then identify what is a reasonable basis for calculating the costs. For example, is it reasonable to attribute a proportion of ‘back office’ time and costs? If so, what basis will be used to attribute a reasonable proportion? Similarly, when results are achieved by the AcT partner working in partnership or in a sub-contracting relationship with other organisations, is there a justifiable way of apportioning costs to get a full view of the total inputs that contributed to achieving the results in question?

STEP 3: Collect data on the monetary, quantifiable and qualitative value of the result (outputs and outcomes) What to do:  Collect accurate data on numbers of direct

and indirect beneficiaries (include clear documentation of the assumptions made)

3

 Clarify the basis for the estimates (whether

actual counts from programme offices or estimates on the basis of a normally accepted multiplier e.g. of 5 people per household, or 250 people per water point as defined in service standards).

 Where the results cannot be monetized or

quantified, explain why. It is important to keep records of the qualitative aspects of the intervention in terms of the qualitative outcomes such as empowerment and rule of law, because these have value even where it can’t be quantified.

STEP 4 (Optional): Where feasible extrapolate from the qualitative aspects to derive estimates of value by using valuation methods. What to do: For the qualitative outcomes in Step 3 above, determine if there are appropriate valuation methods to estimate monetary value e.g. contingent valuation methods. Good examples of how you can achieve that are available from www.sroi-africa.com

How to write your own value for Money case study

How to translate the VFM elements into a VFM analysis???

Based on Case Study 1: Oxfam’s Chukua Hatua programme in Mwime Village Data Area

Specific Items

Overview of the Result Chain

Mwime Village is adjascent to the Barrick Gold Mine Site in Shinyanga. For some time the village has been working to set an appropriate mechanism to receive agreed sums of money from the mine. After receiving Oxfam training as an animator, Maimuna, used the skills to inform other villagers to organise and demand for action from village goverment. Villagers used their political representatives and other allies to get the engagement of all parties and eventually the govenment and the mining company responded and money was paid to the Village. For detailed information See page 4 of the main report ..Background to Oxfams Intervention in Mwime Village

Quantity and cost of inputs

Value

Description

Programme Costs

A.  One Training session for the animators

Tsh 6 Million

The entire training costed Tsh 6 million for 30 animators. Cost per animator is therefore Tsh 200,000/= (A1)

B.  Two Mentoring sessions for the animator in two different districts

Tsh 10 Million

Two mentoring sessions were organised in two different districts each costing Tsh 5 million. 10 animators attended & therefore cost per animator was Tsh 1,000,000/= (B1) for both sessions

C. Ongoing costs borne by the animator and community members

Not Valued

Time and resources spent in various meetings and follow-ups and mobilising other community members. Unfortunately there was no proper recording of this and hence it is difficult to estimate the costs accurately

D.  Cost borne by the Tsh 500,000/= parliament committee to support community members efforts

Given as a gift to support opening of the village account

Back office costs E.   Oxfam’s staff costs

Quantity and value of outputs and outcomes

E.   Operational & office costs

Tsh 4.8 Million Oxfam estimates that admin costs including staff costs is approx. 30% of programme costs (i.e 30% of 16 Million invested for training and mentoring of animators). For a single animator, the admin costs will be 30% (A1 + B1) = Tsh 360,000/= (E1)

F.  Initial payout to the village account

Tsh 180 Million

G.  Additional annual payout amount for subsequent years

Tsh 60 Million

The company will continue to pay the Village Tsh 60 Million annnualy for the duration of the contract

H.  Number of Direct beneficiaries

3480 (1710 M,1770 F)

The funds are managed by the village government for the benefit of the entire population of Mwime Village . It is unlikely that the funds will be distributed among villagers. Collective decisions on what the money will be invested in will be made using normal planning and budgeting procedures at the village level.

I.  Indirect Beneficiaries / population of entire Kahama District

1.3 million

Population of Kahama District, where Mwime Village is located. These are considered indirect beneficiaries because the expereince gained by Mwime Village and the services or goods purchased by the funds is likely to spill over to other adjascent villages

J.  Value of infrastructure from the funds received

This refers to the value of benefits that the villagers are receiving as a result of the investment in above. Information not yet available at the time of the study.

K. Ongoing costs borne by the animator and community members

This refers to what the received funds were used for. This information was not available at the time of the study

Other qualitative L. The intervention empowered the animators and eventually villagers in Mwime, strengthened aspects/outcomes informaiton flow and networking between villagers and their elected reprsentatives and also of the intervention strengthened the rule of law where provisions in the mining contracts were followed up and effected.

4

How to write your own value for Money case study

Table 2: Estimating the VFM elements for the Mwime Case based on data on Table 1 above VFM Category

Link to

Economy

Inputs

Key Questions

3. Is the result monetisable? What is the cash value of the gain?

 

3.2 Over five years

F+(G × 4yrs)

 

4. What is the ratio of input: output? Towards cost per unit output

(A+B+E)/F

 

Yes

 

3.3 Over 10 years

5. Can number of beneficiaries be quantified? 5.1 direct beneficiaries

5.1.1 cost per direct beneficiary 5.1.2 value of gain per direct beneficiary 5.2 indirect beneficiaries

5.2.1 cost per indirect beneficiary 5.2.2 value of gain per indirect beneficiary 6. Can the outcomes be monetised? (eg putting a cash value on access to clean water) 7. Can the outcomes be quantified?

Effectiveness Outcomes

F

F+(G × 9yrs)

H.

(A1+B1+E1)/H

   

 

unit costs as only one of the trained animators worked in Mwime 

(F/H) for yr 1 + (G/H) yr 2 onwards   I.

(A1+B1+E1)/I.

   

(F/I) for yr 1 + (G/I) yr 2 onwards   No –( J and K)

 

No ( J and K)

 

7.1 Over one year

 

7.2 Over five years

 

7.3 Over 10 years

 

8. Are the outcomes qualitative? 8.1 What are they?

5

Judgment

Background to the intervention and comparison with other programmes; challenge to find direct comparability

F

3.1 Over one year

Outputs

Basis for answer and evidence

1. (a) What did it cost? -Total (a) A+ B+E=Tsh 20.8m programme costs (b) Cost per unit input? - Cost (b) for the one village animator A1+B1+E1=Tsh1.56m  

2. Could the same input have been generated for less?

Efficiency

Answer

Yes L

   

How to write your own value for Money case study

VFM FAQs 1. Where can I get learning materials on Value for Money? AcT produced some for explaining the concept to partner organisations in the programme. They are available at: www.accountability.or.tz/learningprogramme-2/ - including a presentation and some case study material 2. Producing a case study like this looks like a lot of work! How could I use it?  Internally within your organisation (both the executive and the Board) you could use it to compare different programmes and different ways to achieve your results. Sometimes organisations carry on doing the same thing year after year just because they always have done. It can be a very refreshing way to look again at what you do, and to assess which of the things you do are most efficient and effective. Hence it provides good evidence from which to make planning and budgeting decisions.  VFM is an increasing concern of donors at the moment. Having a well documented case study of this sort can be a very good way of showing them that you are not only thinking about how best to manage the costs of your programme (always a good management practice that they will be pleased to see!) but also that you are looking further into issues of the effectiveness of your work and being able to justify investments made in terms of longer term results. If you are able to take this even further into being able to quantify some of the difficult concepts like empowerment, you would really be getting ahead of the game, including internationally! (see for example, Jeremy Holland, ed (2013) Who counts? The Power of Participatory Statistics, Practical Action Publishing, Rugby UK. )

3. How do I go about calculating core costs? Core costs are difficult to attribute because they are, by definition, costs that persist regardless of the projects that an entity chooses to undertake. Examples of core costs include rent for office space, communications & IT, the cost of an annual audit and costs of support staff. These costs benefit the organization as a whole but their contribution to specific projects is indirect. Even though these costs are indirect, they must be factored into VfM calculations because they are a real cost of implementation. 6

There are quite a number of legitimate methods for allocating core costs and the selection of an appropriate method is an internal decision that is influenced by any number of unique factors, including the stage of growth that your organization has reached. In terms of VFM, the credibility of your calculations will depend on your organization’s ability to demonstrate a fair and realistic allocation of core costs across projects. It is also important to remember that the decisions you make about apportioning core costs will affect your VfM either positively or negatively. Therefore, it is important to look for the appropriate methods to estimate true cost of an initiative without overstating or understating the core costs that are attributable to it. 4. How do I assign or estimate monetary values for qualitative aspects of programme results? – Arguably this is one of the most challenging aspects of establishing VFM but one that must be taken care of especially for governance related programmes since the expected results will include a good number that are qualitative in nature e.g. Empowerment or strengthened rule of law and human rights. Some qualitative results will be easier to assign monetary value to than others; ones where there are already clear established standards. For example, if the result is an observable general improvement of older persons health as a result of advocating for their rights to free medical services, one could adopt the use of Disability Adjusted Life Years (DALY), an approach that was developed in 1993 by Harvard and the World Bank and adopted by WHO. It is possible to attach a monetary value to DALYs and there are standard values developed for different countries and for different conditions available in the WHO website. Because DALYs are non-financial metrics you need to attach a monetary value to it and Lvovsky et al (2000) is a good reference for approaches on how to attach a monetary value to established DALYs.

How to write your own value for Money case study What is required is to determine the most appropriate valuation method to estimate a monetary value. Good examples of how other programmes have done this are available from www.sroi-africa.com. From the examples it will be possible to determine which of the approaches is most suitable for your own case. 5.  Doesn’t this force me to claim my organisation caused a result on its own, when we know what we did was make a contribution to a process in which others were involved? It has to be admitted that this is a risk. Two important steps are (a) to make very clear in the write up to your case study that you value the contributions from others that have been involved - that the result could not have been achieved

7

without them, and that you are not setting out to ‘over claim’ what you have done. (b) wherever possible try to calculate the value of those contributions. Particularly important is to put a value on ‘community contributions’, as these are often assumed in development projects but not made explicit. For example, the time community members spend collecting data for community scorecards can be assessed. The very minimum would be at the rate of casual labour in that area, but actually the work is more technical and demanding than that, and requires a level of education to do well. One way would be to ask those doing the work what is the ‘opportunity cost’ of getting involved – i.e. what would they have been doing if they hadn’t been participating in the activity and what could they have earned doing that other work?

How to write your own value for Money case study

For further informantion: Programme Director

Accountability in Tanzania (AcT) 11th Floor, PPF Tower | Garden Avenue/ Ohio Street | P.O. Box 1160 Dar es Salaam, TANZANIA Phone: +255 22 2122003 | Fax: +255 22 2113343 Website: http://www.accountability.or.tz