PNPM-Urban Supervision and Monitoring – 2012 Progress Report Summary Information Status Task Team Leader(s) Executing Agency(ies) Start date to Closing Date Geographic Coverage Approved Budget

Closed (extension to December 2014 proposed to the JMC) George Soraya / [email protected] Ministry of Public Works (MPW) November 2009 to December 2012 National: 10,924 urban wards in 268 cities/districts (2012) USD 800,000

1) Project Rationale and Overview As in the case of PNPM-Rural, while the expansion of the PNPM-Urban to over 11,000 urban wards has been successful, the process has strained the program’s management and governance frameworks. Issues with the quality of higher level project management, particularly delays in the procurement of consultants, have hampered implementation progress. Capacity across all levels of government and of communities to implement the program remains limited. Special support is also needed to strengthen the GOI’s coordination capacity and ensure consistency among donor funding to PNPM-Urban and ensure the same quality of supervision and implementation in all areas. 1 Since 2009, PSF has provided critically needed supervision and monitoring support to the Ministry of Public Works (MPW) to help ensure full supervisory coverage of PNPM-Urban. MPW and PSF have prioritized five areas of strengthening: (1) project governance, (2) quality of monitoring and evaluations (M&E), (3) Management Information System (MIS) and Complaints Handling System (CHS), (4) development of the neighborhood upgrading scheme and Poverty Alleviation Partnership Grant (PAPG) and; (5) overall capacity building support. As importantly, this project provides special advice for the synchronization of various urban poverty programs with PNPM and support further innovation and development under PNPM-Urban. PSF’s continued involvement in the program has allowed the GOI to benefit from technical expertise and global knowledge in areas such as urban upgrading and service provision, CDD, and disaster risk management (DRM); an effective, proven mechanism to facilitate high quality management and oversight of the program; strong local governance controls and fiduciary oversight mechanisms; and design improvements based on lessons learned during implementation and findings of evaluations and special studies. 2) Progress in Previous Years Some of the key areas of focus of PSF implementation support in previous years have included: technical advice to overhaul the CHS and upgrade the MIS and program website; hiring of 1

PNPM Urban currently receives funding from three donors (Islamic Development Bank, USAID and the World Bank).

1

evaluation experts to enhance MPW’s capacity to assess program results; advice for annual updates of the program’s Operations Manual; and supervision of the implementation of the training program for over 7,000 facilitators. Intensive implementation support proved critical to address various issues, particularly related to the design of projects, evaluation and MIS. In 2011, continuous improvements to the CHS supported the resolution of 95% of cases. Improvements to the MIS for the Revolving Loan Funds (RLF) also ensured that the RLF data for all areas were 90% complete. Special attention went to Eastern Indonesia (Maluku, Papua, Papua Barat) where travel is difficult and human resources are limited. PSF support proved critical to foster the innovation and flexibility that are required to enhance quality in these areas. The project, for example, supported MPW in developing a customized capacity-building strategy to improve the capacity of consultants, facilitators and Local Government and improve communication flows from the central level to the oversight consultants. 3) Progress in 2012 In 2012, PSF continued to provide hands-on implementation and institutional support to PNPMUrban. Two formal supervision missions helped review implementation progress of PNPMUrban and associated special programs and the quality of project governance at all levels. The missions noted that several guidelines, training modules and SOPs had been revised to enhance capacity building and project implementation, incorporating the findings and recommendations from PNPM-Urban studies and evaluations. Capacity building at community level, however, continued to require further improvements to increase communities’ understanding and participation in the program. Mission recommendations included to use ICTs for socialization and to systematically involve RT/RW in community trainings. The missions also emphasized that the program suffers from significant pressure to disburse of the funds due to the target of disbursements set at the national level for APBN. This is a classical problem for community empowerment programs. This pressure for disbursement should be accompanied with a system that would allow for community preparation to take place at its own organic pace. The Missions also observed that the bidding processes continued to require significant improvements, in particular as regards their documentation.2 To inform continued improvements in project design and implementation, PSF also support the design of ToRs and completion of draft reports for the quantitative and qualitative evaluation of urban poverty projects (UPP2 and UPP3). A series of meetings were also organized with the PMU and Local Governments to follow-up on the findings and recommendations from the PNPM-Urban process evaluation (See box 1).

2

Many of the procurement delays occurred because of weak documentation that needed clarification. The process of Oversight Service Provider is new, however, and the bidding committee will need some time to become familiar with it. The PMU has not been able to recruit an advisor with international experience to support them.

2

Box 1. Findings of the PNPM-Urban Process Evaluation The process evaluation found that overall, the program is doing well in terms of achieving its objectives of community participation, capacity building and governance improvements (such as the occurrence of regular, well-attended community meetings, physical audits and external monitoring). The evaluation also noted the good quality of small-scale infrastructure on which approximately 80% of the block grant funding is spent. Most respondents responded favorably to the PNPM-Urban financed infrastructure. 3 The evaluation indicated the need for continuous improvements in a number of areas, however. For example, facilitation does not effectively support active participation by the poorest and women; the PNPM-Urban planning processes could be aligned more systematically with local government budgeting processes (Musrenbang); and the pilot Neighborhood Development program could be better focused on the poor. Interviews also highlighted suggest that PNPM-Urban should place more focus on the socioeconomic priorities of communities during project identification, through improved facilitation. These findings are being addressed through changes in project design and management.

Even though supervision/studies continue to show good levels of accountability in the project, continuous improvements to the MIS, complaint handling and verification of payments made to firms are needed. The provision of technical assistance to support improvements to the program’s MIS, CHS and website has been a continued focus in 2012: •

•

There have been some improvements as regards the usage of MIS as a tool which can monitor the progress and performance of all participating PNPM-Urban kelurahans, including on community participation, disbursement and utilization of funding. Three improvement plans for the grand design have been developed in response to the MIS evaluation.4 The completeness of MIS data (now at 90% 5) and their quality has greatly improved. A key strength of PNPM-Urban remains its capacity to resolve complaints effectively. Since 2003 up to 31 August 2012, about 88,000 complaints were reported, of which about 99% have been resolved. Complaints about misuse of funds are 528 cases with the total amount of US$1.4 million. The majority of perpetrators came from UPK, BKM & KSM. As of 31 August 2012, 474 cases have been resolved with 34 cases being processed in courts and by the police. About US$760,000 equivalent (or 53% of the total) had been recovered, although this percentage is expected to increase as the Government is currently taking steps to recover the remaining funds. About 80% of this amount was detected in 2009 or after, and most funds misused from 2003-2008 have been recovered. It is also worth noting that average misuse of funds remained below US$5,000 in all years except in 2003, the first year of implementation of PNPM-Urban.

3

Other evaluations assessing the cost-effectiveness of projects by MPW and Bappenas have found that community self-help construction under the UPP program was estimated to be 30-66% cheaper than traditional contracting methods, depending on the type of construction. 4

(i) Time Series Glossary and Reports at all levels (Kelurahan-Kota-Province-National); (ii) Project series report, also at all levels, and (iii) development of keluharan profiles. 5 The completeness of RLF data has reached 95% in WB funded areas and 89% in IDB funded areas.

3

•

Supervision suggests, however, that the city coordinators and facilitators’ understanding of the CHS still needs improvements. Socialization of the SMS-based complaints also should be continuously carried out to communities. NMC should make sure all facilitators have received the guidelines and have been trained on the complaint handling system, including the system available on the website. The number of visitors to the program’s website has increased to an average of 9,700 per day during Jan-Oct 2012, but its role as the center of project information and communication needs to be further strengthened, especially for the Local Governments and community. Some recommendations from past missions are still pending such as improvements to the search engine to enable easier information finding on kelurahan, the posting of all contracts with firms on the website, quarterly reviews of the MIS, Quick Status application (QS), and improvement of Executive Information System (EIS).

The FY12 audit 6 noted internal control weaknesses within the projects related to the control environment, risk assessment, control activities, information and communication, and monitoring. The audit findings reported consisted of excessive payments, inadequate work volume, constructions built not in accordance to specification, inaccurate block grant targets, weak project management and misuse of RLF. To address some of these issues, the PSF team continued to provide extensive technical inputs to MPW on key implementation documents, including: facilitator training modules; SOPs for community trainings, housing construction, and social activities; the Neighborhood Development (ND) technical guidelines and more detailed guidelines (Petunjuk Teknis) on the ND project cycle and community mapping. The project is also supporting collaboration initiatives with local governments to improve RLF performance. There are some early indications that these are gaining momentum. The major of Kota Surabaya, for example, committed to assign a special unit from the concerned dinases. In Merapi-affected areas, the project provided three technical experts in community development, spatial planning, land use and land acquisition management, coordination and program management to advise recovery efforts and Disaster Risk Reduction (DRR) activities and assisted the incorporation of PNPM-Urban to the BNPB budget allocation in 2012, including through an adaption of the manuals for the purpose of domestic funding. About 75% of population has agreed to move from the danger zone, of which about 50% will have their houses and settlement completed in the end of December 2012. Almost all infrastructures in 88 villages had completed construction by December 2012. The remainder will be completed in 2013 due to the complex nature of the work.

6

FY 2012 field audit for PNPM Urban was carried out from March to May 2012. The block grant audit for FY 2012 by BPKP has been done by involving local inspectorates covering 162 districts/cities spread in 32 provinces. The audit reports were received on time with unqualified opinion.

4

4) Key Results

Result Indicators

Unit of Measure

Baseline

Total Target

Targets (cumulative)

Delivered in 2012 (only)

Cumulative Progress as of Dec 31, 2012

Difference Comments on with progress Before 2011 2012 cumulative 2011 target for 2012 PDO: To ensure full supervisory coverage of Urban PNPM-Mandiri programs and provide special advice for the synchronization of various urban poverty program with PNPM and further innovation and development under PNPM Urban. Reliable MIS % of data which are 80% 90% 80% 80% 85% 90% 90% +5% System complete Maintain Reliable % of complaints N/A 95% 90% 95% 95% 98% 98% +3% Complaint which are resolved Resolution Support the 2 5 7 On target # of revised 0 7 5 capacity building of manuals developed MPW and with PSF inputs community institutions in program management, planning, fiduciary, and oversight Strengthened field # of kelurahan 50 75 50 75 97 97 +22 supervision visited/year Increased # of joint # of missions/year 0 2 2 2 2 2 On target supervision mission for high risk areas and eastern Indonesia More interactive # of visitors / hit 3,000 5,000 4000 4000 4,500 9,700 9,700 +5,200 websites, with an per day increased number of visitors Improvements to # of new/revised 0 3 1 3 3 +2 the Poor manual produced

5

Result Indicators

Neighborhood Upgrading program Improved collaboration with local governments, based on existing good practices. Strengthened Disaster Risk Reduction component under PNPM-Urban

Improvements in the quality of evaluations and implementations of key recommendations from the recent PNPM Urban process evaluation

Unit of Measure

and applied with PSF support # of LG initiatives initiated

% of Merapi cash for work activities financed by the Disaster Management Support Project completed satisfactorily # of DRR manuals/modules included in PNPMUrban training program # of experts recruited to support MPW Number of studies disseminated through workshops

Baseline

Total Target

Targets (cumulative) 2012

Delivered in 2012 (only)

Cumulative Progress as of Dec 31, 2012

Difference with cumulative target for 2012

Before 2011

2011

0

3

-

-

1

2

2

+1

0

N/A

-

-

70%

100%

100%

+30%

0

N/A

N/A

2

2

n/a

0

N/A

N/A

3

3

n/a

0

N/A

2

1

1

(1)

Comments on progress

Urban poverty study will be disseminated in early 2013

6

5) Lessons learned •

While the MIS is robust, it could produce more meaningful data for evaluation purposes, especially for the local government and community. A key improvement would be to automatically generate data on the achievement of all key performance indicators. Many of key indicators have already been simplified for 2012-2014. The improved MIS is expected to track results, provide feedback to PNPM stakeholders in a timely manner, and capture citizens’ views on PNPM. The project will support GOI in conducting regular reviews and implementing improvements as well as continuous awareness raising and enforcement of MIS to foster changes in mindset and behavior of the operators.

•

Since 2001, PNPM Urban (and its predecessor UPP) has conducted a number of evaluations that have helped to improve the quality of project design and implementation. The project will continue assist the GOI to improve the quality of the analytical work that will be conducted in 2013-2014, including an impact evaluation of urban slum upgrading, several thematic evaluations and quick studies on satisfaction and efficiency and an assessment of improved access to social and infrastructure services.

•

The current Complaints Handling System has successfully captured a large numbers of complaints. The CHS has been widely used by management to improve the quality of implementation, particularly the performance of consultants and facilitators. However, the system has been overloaded with over more than 60,000 complaints over the last few years. Data organization needs to be systematized to allow for the prioritization of cases. The capacity to resolve complaints also needs to be enhanced by involving the local governments. The PMU also needs to enhance its own capacity to review and analyze CHS data in order to identify risk trends, possible problem areas in the program, trends in enforcement, etc.

•

In Merapi-affected areas, one the key lessons learned has been the importance of establishing a consultation platform where central, provincial and local governments, community and affected people could discuss and solve various options for relocation, which involve complex issues related to land tenure and scarcity. The development of various discussion forums (formal and informal) has fostered innovation and allowed for the generation of practical solutions on the ground. Such platform has enabled the transfer of land owned by traditional authorities to the community and establishment of new settlements.

6. Future plans The GOI has decided to continue PNPM Urban in 11,000 villages (about 1,000 sub-districts) in 2012-2015. PSF has proposed an additional allocation of US$ 550,000 to this project to the JMC, which is being reviewed, to extend the implementation support over the same period. As explained above, despite good progress in key areas, a number challenges remain to improve 7

program design and effectiveness, to consolidate its governance arrangements and to institutionalize program mechanisms at the local level. In Merapi-affected areas, the implementation support should be continued until the closing of the Disaster Support project in December 2013. Support would be focused on assisting the completion of additional houses funded by PSF and BNPB domestic funding using the Rekompak model, and the development of institutional arrangements for a nationwide application of the post-disaster settlement reconstruction in collaboration with PNPM regular programs. The adoption of Rekompak-PSF model by BNPB is a major step to ensure the sustainability of the use of CDD in disaster recovery. Some of the specific deliverables over the next couple of years include: • Increased coverage of detailed implementation reviews to 100 urban wards/ year (from 75 wards/year). • Thematic mission on the Neighborhood Development program to support improvements to its technical guidelines, update training materials and dissemination plan based on findings and recommendations from the latest studies. • Enhancement of the social and economic program in pilot areas through a collaboration with Local Governments (including technical report). • Further technical assistance to support improvements to the MIS and CHS. • Annual report on progress on the governance action plan and consolidation of Local Auditors' reports. • Completion of impact evaluation of the Neighborhood Development program in 2014. • Support the drafting and signing of a MoU between MPW and BNPB about the use of PNPM and Rekompak structures and mechanisms to support disaster response and settlement reconstruction. 7.

Disbursement Performance

The approved budget of US$800k was almost fully expended at project closure in December 2012 and balance returned to the PSF main fund. The disbursement projection for 2012

8

included funds from an expected additional financing (US$550k) for a two year extension of the project, which has not yet been approved by the JMC. Approval is expected for January 2013.

9