LESSON 2: POLICY ANALYSIS Cristina Puentes-Markides
KEY THEMES Basic concepts of policy analysis Types and models Policy analysis
1. 2. 3. i.
Problem analysis
ii.
Solution analysis
iii.
Communicating results
CPM-2012
RATIONALE Governments face today health problems of increased complexity. The influence of social determinants tends to evanesce sectorial boundaries, which makes it difficult to circumscribe public problems and solutions to a single sector or group. The multiplicity of global actors with diverse interests, motivations and beliefs generates powerful forces that shape the possible solution alternatives and politicize the process. Often, governments embark in policy implementation through programs or projects that lack the necessary backing of a cautious process of priority setting, analysis and of a more or less systematic design. CPM-2012
4
“Policy analysis is client oriented advice
relevant to public, or private, decisions and informed by social values.” Weiner & Vining, p. 27
Not all public problems are amenable to policy analysis; when they are, the analysis can support decision making • Assisting decision makers in identifying complex problems and choosing a preferred course of action from complex alternatives and under uncertain conditions. • Provide policy relevant information a language and format that is understandable to others. • Prepare written policy documents (memos, regulatory impact assessments, policy issue papers, research reports) on potential solutions to problems • Oral briefings, meetings, conversations and conferences communicate the contents of policy documents • Heightening the quality of decisions, contributing to the political debate and providing a bargaining chip.
KEY MESSAGES Policy analysis is not a panacea or a substitute for inadequate policymaking processes, defects of public decisions, bad judgment on the part of analysts or policymakers Policy analysis is not infallible, an exact science or a tool for advocacy by the analyst for his/her own views. Policy analysis is not a substitute defective policymaking processes or bad judgment on the part of the analyst or decision makers. A good analysis does not guarantee that the optimal solution will be selected and/or implemented. CPM-2012
TYPES AND MODELS OF POLICY ANALYSIS
TYPES OF POLICY ANALYSIS WHEN
PURPOSE Predictive (To anticipate likely effects that might result from the adoption of a particular policy)
Before selection, Ex ante (a priori, recommendat pre hoc, Prescriptive (recommendations to ion and prospective) achieve a particular result), to implementati inform process of decision making on and deliberation Ex post (a posteriori, post hoc, retrospective); also known as program evaluation.
Descriptive (also interpretative), to describe the effects of a policy already implemented
After policy adoption, and implementati Evaluative (were the policy objectives achieved?) Assess on has whether policy satisfies began. CPM-2012established policy outputs and outcomes
8
9
We will look at the ex-ante type of analysis
Alternative Policy Analysis Models (according to different authors)
CPMCPM-2012
10
ONE WAY OF LOOKING AT THE ANALYSIS:: The Problem and The Solution
The Policy Analysis Process Source: Weimer and Vining, pp. 257
II. Solution Analysis 1. Choosing evaluation criteria 2. Specifying policy alternatives 3. Predicting impacts of alternatives & valuing them in terms of criteria 4. Recommending actions
I. Problem Analysis 1. Understanding the problem (Receiving, framing, structuring,
III. Communication Conveying useful advice to clients
modeling)
2. Choosing & explaining relevant goals & constraints 3. Selecting a Solution method
Information Gathering Identifying and organizing relevant data, theories, and facts for assessing problem and predicting consequences of current and alternative policies
12
RECOMMENDED METHODS
PROBLEM ANALYSIS
Epidemiological analysis, statistical methods; simple observation, focus groups, brainstorming, experts meetings, consensus conferences; quick decision analysis; lit. reviews, bibliographic searches; stakeholder, political & media analyses, map the policy, create operational definitions; construct a logic model; forecasting. Technical, economic/financial, political, administrative feasibility analyses. Ask the decision maker, consult with experts, determined by the analyst.
SOLUTION ANALYSIS
Lit. reviews & analysis (reports, journals, evaluations, experiences dealing with similar problems in akin population groups, laws, regulations, science & technology); brainstorming, interviews, quick surveys of relevant stakeholders, experts, focus groups, interviews, Delphi); typologies, analogies / metaphors; comparison with real experiences, with an ideal, modification of existing solutions; models, simulations; analyze effects of maintaining the status quo. Forecasts and projections, extrapolations, scenarios; CBA, CEA, discount techniques, sensitivity analysis; political feasibility analysis; implementation analysis; matrices (to display assessed policy alternatives); paired Comparisons, satisficing, grading method, lexicographic ordering, non-dominated alternatives.
Paired comparisons; strengths & weaknesses; scenarios (best/worst case, no change); matrices, charts, tables, timetables, arguments, diagrams for COMMUNICA presentation. TE RESULTS 13 Source: Patton andBefore Sawicki & after comparisons; case control studies; experimental & quasi experimental models
I. PROBLEM ANALYSIS
PROBLEM ANALYSIS
• Understanding the problem (Receiving, framing, structuring, modeling) • Choosing & explaining relevant goals & constraints • Selecting a Solution method
PROBLEM DEFINITION DEPENDS ON: How issues are framed, portrayed. Problems are defined not only by their objective dimensions but also by their social constructs Endless ways to look at a particular problem, yet limited time, money and energy. Highly complex issues are simplified Focus on certain aspects happens at the expense of other aspects/elements. How it is measured.
Policy solutions are shaped according to how the problem is defined 1. Constant and increasing rates of obesity among young adults and among certain population groups can be structured as: • The failure of the individual to modify his/her eating habits. • Neglecting obesity is a form of class bias. • The inability or incapacity of health care providers to offer appropriate and timely information • The obsession with obesity stigmatizes fat bodies, neglects factors related to dangerous body weight and appropriate care.. 2. The increase of lung cancer rates among certain population groups can be structured as: • Certain individuals / groups lack the necessary will to quit smoking, • Health care providers fail to provide timely and appropriate information or intervene. • The government’s failure to regulate the use of tobacco,
PITFALLS IN PUBLIC POLICY PROBLEM DEFINITION 1. 2. 3.
4.
5.
Accepting the client's definition of the problem Looking only for the simple and obvious Thinking that any and all problems need a public solution Confusing the need for short- versus long-term solutions Confusing the values of individuals versus collectivities Source: Patton & Sawicki
II. SOLUTION ANALYSIS
SOLUTION ANALYSIS
1. Choosing evaluation criteria 2. Specifying policy alternatives 3. Predicting impacts of alternatives & valuing them in terms of criteria 4. Recommending actions
SELECTION OF CRITERIA Before defining options to solve the policy problem, ex-ante evaluation criteria related to the desired goals need to be defined. Criteria connote measurement; yard stick by which to assess the efficacy of policy alternatives. (efficiency, equity, political feasibility, effectiveness.) CPM-2012
IN POLICY ANALYSIS, EX-ANTE EVALUATION CRITERIA Represent evaluative standards that guide decision-making. Can be measures, rules, standards, and all those attributes considered relevant by the decision maker (individual or group) in a given situation. Are measurable dimensions that serve to compare and rank the policy alternatives in order of preference. They are used to judge the outcomes of implementing the alternatives. CPM-2012
Commonly used evaluative criteria
CPM-2012
21
Commonly used evaluative criteria
22
IN PUBLIC HEALTH, ADDITIONAL CRITERIA MAY INCLUDE: Economic and social costs of a disease/injury; loss of productivity, medical costs, etc. Need to act quickly (to mitigate a disaster, to control the spread of a disease). Burden of disease (measured by the relative number of people dead –mortality-; measured by the relative number of people affected – morbidity, other). Effect of the problem on special populations (lower income, women, children, the disabled, aged, etc.) or on the physical environment. Safety Privacy Etc. CPM-2012
SOME USEFUL CONCEPTS DEFINITION
EXAMPLE
formal, broad statement about desired long-run achievements
reduce poverty; reduce population-level consumption of salt; reduce populationlevel consumption of saturated fats, increase levels of physical activity, etc.
OBJECTIVES
focused, concrete statement about end states
reduce % pop on welfare; reduce % of people on dialysis; etc.
CRITERIA
dimensions of the objectives used to assess alternative policies
Equity, effectiveness, political viability; cost, etc.
tangible operational definitions of criteria
cost per 1% reduction, number of people that suffer from a particular disease; number of deaths caused by the disease.
GOALS
MEASURES
CPM-2012
Before defining options to solve the policy problem, ex-ante evaluation criteria related to the established goals need to be def ined.
Policy goals are established.
These goals express what the policy should achieve in terms of the solution of the problem. (outcome)
These goals are translated into objectives, more concrete statements about the future results that are desired.
CPM-2012
A good criterion should be • • •
Clear Consistent Supported by measures
CPM-2012
EXAMPLE: PRIORIT-SETTING CRITERIA FOR RESEARCH AT THE NATIONAL INSTITUTES OF HEALTH, USA
Public health needs Scientific quality of research Potential for scientific progress (e.g. existence of promising pathways and qualified researchers) Diversification of the portfolio expand research Adequate infrastructure support (e.g. human capital, equipment, instruments, facilities).
CPM-2012
EXAMPLE (CONT.)
Each criteria is operationalized, see example for public health needs Public health needs Scientific quality of research Potential for scientific progress. Diversification of the portfolio to expand research Adequate infrastructure support.
28
• Number of people that suffer from a particular disease. • Number of deaths caused by the disease. • Degree of disability caused by the disease. • Degree in which the disease shortens a normal, productive and comfortable life • Economic and social costs of the disease. • Need to act quickly to control the spread of the disease.
CPM-2012
CONSTRUCTION OF OPTIONS/ALTERNATIVES The development of feasible alternatives to solve or mitigate the problem is a frequently underestimated and creative phase. Alternatives should include policy options that key political actors are proposing or seem to have on their mind, but should also propose or identify additional ones that may be superior to those under political discussion.
CPM-2012
POLICY OPTIONS MAY ALSO BE OF DIFFERENT NATURE • Technical (add or expand services, facilities) • Managerial (modify the current operation of the system or service) • Economic/Financial (change the prices charged, modify taxes, etc.) • Regulatory (various degrees of coercion) CPM-2012
HOW TO DEVELOP POLICY OPTIONS
no policy exists, exists,
•begin from generic alternatives.
•use it as a base & gradually modify components (add, If there is a reduce, substitute, combine current or split, eliminate, reorient, policy switch components, approaches, resources, modify timing, try different forms of financing, etc.) Develop typologies, use analogies and/or metaphors. Compare the various alternatives with real experiences or with an ideal situation. Consult with experts and stakeholders (brainstorm, debate, surveys, public hearings, focus groups, formal and informal meetings, etc.).
Keep the current system (do nothing) Modify the current system (tweak or embark on more profound changes) Use an existing design (maybe that works in another part of the country for a similar problem and population) Create a new design Combinations? 31
32
A FINAL THOUGHT….
“The definition of the alternatives is the supreme instrument of power…because the definition of alternatives is the choice of conflicts, and the choice of conflicts allocates power.” --Schattschneider, E.E. (1975). The Semisovereign People: A realist's view of democracy in America. Hinsdale, Illinois:The Dryden Press.
CHALLENGES AND TRADE-OFFS (EXAMPLE)
Events, circumstances and/or the action of stakeholders pushing their own 33 agendas may pose additional or different trade-offs for the decision maker.
DESIRABLE CHARACTERISTICS OF GOOD POLICY OPTIONS CHARACTERISTIC Invulnerability Flexibility Degree of Risk Communicability Merit Ease of Implementation Compatibility
DESCRIPTION Can function if one or more parts fails? Can be used for more than one objective or purpose? Does it have a high probability to fail? Is it easily understood by those involved and affected? Does it actually address the problem at hand? Is the option relatively easy to implement? Is it compatible with existing norms and procedures?
Evaluability
Are policy outcomes measurable, would they allow monitoring and evaluation?
Reversibility
How difficult is it to return to previous conditions should the option fail?
Stability
Would it work even when conditions change?
appropriate, Warren, RAND is the Europe option ethical? EthicalAdapted from IfWalker
34
Example of policies that address obesity.
ACTIVE LIVING
HEALTHY EATING
Make roads and paths safe for ACTIVE bicyclists and pedestrians TRANSPORT Expand bicycle lanes and trail ATION connections Re-evaluate urban design and comprehensive land use plans to LAND USE improve active living FOR ACTIVE Improve community design LIVING features to encourage physical activity OPEN Increase access to recreation SPACES, facilities and open spaces, PARKS AND including parks and community RECREATION gardens Offer at least 30 minutes of quality physical activity daily
Ensure that students have appealing, healthy food and beverage choices in schools QUALITY Support farm-to-school and school NUTRITION IN garden programs SCHOOLS Implement a standards-based health education program taught by teachers certified in health education
QUALITY PHYSICAL ACTIVITY IN AND NEAR SCHOOLS
Consider requiring standardsbased physical education classes taught by certified PE teachers Support walk to school and safe routes to school (SRTS) programs
Attract grocery stores that provide SUPERMARK high-quality, healthy affordable foods to lower-income neighborhoods ETS AND HEALTHY Encourage convenience stores and FOOD bodegas to offer healthier food VENDORS Establish healthy mobile markets Support farmers' markets FARM-FRESH Support community gardens LOCAL Support the procurement of locally FOODS grown food Encourage restaurants to offer RESTAURANT reasonably sized portions and low-fat and low-calorie menus S
Facilitate joint-use agreements SAFETY AND Keep communities safe and free CRIME from crime to encourage outdoor PREVENTION activity
Encourage restaurant menu labeling FOOD AND BEVERAGE MARKETING
Regulate the marketing of unhealthy food in or near schools and other youth facilities
EXAMPLE OF “BEST BUY” INTERVENTIONS CONDITION
INTERVENTIONS
Tobacco use
Tax increases; smoke-free indoor workplaces and public spaces; health information/warnings; advertising/ promotion bans
Harmful alcohol use
Tax increases; restrict retail access; advertising bans
Unhealthy diet and physical inactivity
Reduced salt intake; replacement of trans fat with polyunsaturated fat; public awareness about diet and physical activity
Cardiovascular disease and diabetes
Counseling and multi-drug therapy (including glycemic control for diabetes) for people with >30 percent cardiovascular risk (including those with cardiovascular disease); treatment of heart attacks with aspirin
Cancer
Hepatitis B immunization to prevent liver cancer; screening and treatment of precancerous lesions to prevent cervical cancer
FURTHERMORE Don’t wait to find the perfect option. Compare preferred policy with those already in existence. Estimate all the alternatives in terms of the established goals. Ensure that the alternatives are mutually exclusive. Avoid alternatives that are too broad or lack specificity. Try to ensure that the options are consistent with the available resources. CPM-2012
OPTIONS AND ISSUES
However, each policy option presents different issues that might enhance or diminish its ranking or feasibility. There are trade-offs to be made (equity vs. effectiveness, technical feasibility vs. political viability, legality vs. acceptability, etc.
Example of policies that address obesity. HEALTHY EATING Ensure that students have appealing, healthy food and beverage choices in schools QUALITY Support farm-to-school and school garden NUTRITION programs IN SCHOOLS Implement a standards-based health education program taught by teachers certified in health education Attract grocery stores that provide highquality, healthy affordable foods to lowerSUPERMAR income neighborhoods KETS AND Encourage convenience stores and bodegas HEALTHY to offer healthier food FOOD VENDORS Establish healthy mobile markets Support farmers' markets Support community gardens Support the procurement of locally grown food Encourage restaurants to offer reasonably sized portions and low-fat and low-calorie RESTAURAN menus TS Encourage restaurant menu labeling
EXAMPLES OF POSSIBLE ISSUES Proven cost effective, needs to be accompanied by behavioral change strategies. Effective but long term, may involve high costs, new legislation. Effective but long term and may involve high costs, new legislation, opposition from teachers’ unions. Long-term, may require additional resources, and grocery owners may resist. Effective, but may require new regulations and initial costs. Behavior change may consume a great deal of financial resources; results may be too long term and effectiveness not always proven.
FARMFRESH LOCAL FOODS
FOOD AND BEVERAGE
Regulate the marketing of unhealthy food in
Possible strong opposition from restaurant owners (loss of revenues). May require new legislation. Possible strong opposition from restaurant owners (additional costs) and industry. May require new legislation. Possible strong opposition from the industry
COMPARISON OF THE ALTERNATIVES An analysis of possible policy alternatives based on their satisfaction of the various criteria, and regarding the projected outcomes or impacts of each alternative. The recommendation should be treated as a useful but not overriding element in the analysis.
Source: Weimer and Vining
THE GOALS/ALTERNATIVE MATRIX (A PRIORITY SETTING EXERCISE) Matrix illustrates the comparison of alternatives according to the criteria established once the relative weights have been defined. Policy consequences are rarely straight-forward (the results of a program may generate additional issues in the same or other areas or sectors). Alternatives need to be compared in terms of the desired outcomes (or goals), & also in terms of the possible spillovers/externalities that its implementation may produce. Recognize that technically sound options may not have the necessary political and/or social support. Some good policies may not be easily implemented. Source: Weimer and Vining
Let’s assume these are the criteria agreed upon to rank alternatives
CPM-2012
EXAMPLE OF A GOALS/ALTERNATIVE POLICY MATRIX CRITERIA
OPTION A
OPTION B
OPTION C
EFFECTIVENESS (Degree to which the option ( addresses or responds to the problem)
Very Positive
Positive
No impact
ECONOMIC EFFICIENCY: EFFICIENCY (Maximization of satisfaction by society, assess economic cost benefit, environmental cost benefit)
Very Positive
Positive
Positive
EQUITY (Degree of disproportionate impact on ( some social groups or regions).
Positive
Positive
Negative
POLITICAL VIABILITY (Consistency with ( government priorities, public perception, effect on tax payers, impact on the private sector, etc.)
High
Medium
Low
SOCIAL/CULTURAL FEASIBILITY (Degree of ( consistency with national/local traditions, policies and institutions; acceptable for the local population.
High
Medium
Low
Short
Medium
Long
ADMINISTRATIVE FEASIBILITY (Complexity and timing of implementation, fiscal impact on government, legal, jurisdictional or commercial restrictions).
CPM-2012
COST
High
43
Medium
Low
YOU MAY WANT TO EXPLORE THIS EX-ANTE TOOL FOR SELECTING ALTERNATIVES PRODUCED BY THE FAO/EASYPOL 44
http://www.fao.org/easypol/output/search_by_type.asp?lang=&id=&id_elem=&latestrecords=types&first=y&id_cat=&id_topic=&display=&type_ pub=&abstr_type=&dddate_from=&mmdate_from=&yydate_from=&dddate_to=&mmdate_to=&yydate_to=&lchain=&search=&text=&ascdes=& ordenAnt=4&orden=6&Direccion=5
45
YOU STILL NEED TO DO THE ANALYSIS…
III. COMMUNICATING RESULTS
WRITING FOR RESEARCHERS Context Questions/Hypothesis Methodology/Data Sources Results Discussion Significance/Implications
WRITING FOR POLICYMAKERS Results Questions/Hypothesis Context Discussion Significance/Implications Methodology/Data Sources 47
Source: Center for Studying Health System Change, USA
TYPE OF POLICY PAPER AREAS OF DIFFERENCE
Policy Study (Issue Paper)
Policy Brief
Policy Memo
Targets other policy specialists (may not have been requested by a decision-maker)
Targets decisionmakers
Targets a broad audience of stakeholders
Focus
Issue driven: broad recommendations and analysis of policy issues
Audience driven Specific policy message designed to convince key stakeholders.
Audience Driven: Specific policy message designed to convince key stakeholders
Context of Use
Dissemination and debate on results of policy research, informs the policy brief.
Used for advocating and lobbying purposes.
Used for advocacy, lobbying & to encourage stakeholders to read further
Methodology
Can include much primary research.
Rarely includes primary research.
Rarely includes primary research.
Ideas/ Language Used
Can be quite discipline specific/technical
Must be very clear and simple.
Must be very clear and simple.
Length
Up to 60 pages.
Between 6-15 pages.
Up to four pages.
Audience
48
THE POLICY MEMO Offers analysis and recommendations in reference to a specific situation/problem. It is short, concise, well organized, well written, convincing! (Decision-makers don’t have time to read long, convoluted texts)) It responds to the needs of a “client” who seeks advise on how to address a situation/problem that will need a policy decision The client could be a president, a prime minister, a member of cabinet, a legislator, YOUR BOSS!!) CPM-2012
Date From
To
Subject 1. Executive Summary
Example: Structure of a Policy Memo
2. 3.
Introduction The Problem
4. 5.
Background of the issue Description of the policy arena/landscape Policy Options
6. 7. 8.
Analysis of trade-offs Recommendation of a course of action Conclusion
9. 10. Bibliography of Sources CPM-2012
50
CRITERIA FOR ASSESSING POLICY MEMOS AND OTHER DOCUMENTS (MEMOS, BRIEFS, ISSUE PAPERS)
Economy of style
Clarity
Directness
Understandability
Organization
Attention-Getting
Low costs to reader
Source: William Dunn
DO Remember the client! Keep in mind that your task is to provide useful advice. Set priorities! Organize your information carefully. (essential material in the text, supporting material in appendices). Decompose your analysis into component parts. Use headings that tell a story; avoid abstract headings such as “Market Failure”. Be balanced! Give appropriate coverage to problem analysis and solution analysis. Acknowledge uncertainty but provide your resolution of it. Support your resolution with sensitivity analysis where appropriate. Be credible by documenting as extensively as possible. Be succinct, avoid jargon and clearly explain any technical terms. Be value overt. Make explicit arguments for the importance of goals.
Don’t Write an essay! The difference between an essay and a wellstructured policy analysis should be clear . Tell the client everything that you know as it comes into your head. It’s fine to think nonlinearly, but write linearly. Write a mystery! Instead, state your important conclusions up front in an executive summary.
Write crisp text. Favor short and direct sentences; use the active voice.
Source: Weimer and Vining
53
THANK YOU!!
Extra: Tips for Conducting Policy Analysis
CPM-2012
54
