Outline z basic policy analysis cycle {verifying, defining and detailing the problem {establishing evaluation criteria y g stakeholder p perspectives p {identifying {identifying alternatives g alternatives {evaluating {display and distinguish among alternative policies p p policy y {monitor implemented

Pharm 532 Methods of Pharmaceutical Policy y Analysis y

Policy Cycle

Tom Hazlet, Pharm.D., Dr.P.H. Pharmaceutical Outcomes Research & Policy Program Department of Pharmacy [email protected] 206.616.2732 1

2

A Basic Policy Analysis Process from Patton & Sawicki Basic Methods of Policy Analysis and Planning, 2nd Edition,1993

Policy Analysis Cycle Verify, Define, and Detail the Problem

Monitor the Implemented Policy

Evaluate Stakeholder Perspectives

Bardach Eightfold Path problem define the p assemble some evidence construct the alternatives select l t th the criteria it i

Establish Evaluation Criteria

project the outcomes Display and Distinguish among Alternative Policies "Recommendations" Recommendations

confront the tradeoffs decide t ll your story tell t

Identify Alternative P li i Policies Evaluate Alternative Alt ti Policies

3

Patton & Sawacki verify, y, define and detail problem p establish evaluation criteria identify alternative policies evaluate l t alternative lt ti policies li i display & distinguish among alternatives monitor and evaluate p p policyy implemented

4

Problem Definition

Evaluation Criteria

zdescribe the problem zdelineate the boundaries of the problem zdevelop a fact base zdescribe goals and objectives for resolving the problem; metrics zidentify the policy envelope zcosts and benefits of resolving the problem p

zfree market model {costs & benefits zstanding zexternalities zelasticity g analysis y zmarginal zequity

5

6

Evaluation Criteria 2 …

Evaluation Criteria 2a

ztechnical feasibility zeconomic and financial possibility zpolitical viability zadministrative operability

ztechnical feasibility zeconomic and financial possibility zpolitical viability zadministrative operability

7

8

technical feasibility

Evaluation Criteria 2b

z A criterion i i for f evaluating l i alternatives; l i measures whether h h the h alternative actually produces the desired result -- meeting the major objectives. { To what degree does the proposed action accomplish the objectives set forth? { Can changes in the real world be traced back to the program, or they are the result of other factors (sometimes called “secular trends”)? { Is the impact direct (it addresses the stated objective) or indirect (creates an impact not associated with the program)? { Will change (if any) be short term (for 1-2 years) or long term (>2 years)? [discount rate]

ztechnical feasibility zeconomic and financial possibility zpolitical viability M Measures the th costt off th the alternative(s) lt ti ( ) and d the th benefits b fit zadministrative operability it (they) will produce

9

10

political viability

Evaluation Criteria 2c

z Measures whether h h the h alternative l i is i acceptable bl or can be b made d acceptable to relevant groups.

ztechnical feasibility zeconomic and financial possibility zpolitical viability zadministrative operability

• Proposed policy changes must survive the political test: if a policy will not be supported by decision makers, officials or voters it has little chance of being adopted or implemented voters, implemented. • What alternatives are available? p to various groups? g p • What will be acceptable • What concessions will have to be made to gain support for each option? • Do D you h have trade-offs t d ff iin order d tto secure agreementt on an alternative?

11

12

administrative operability p y

Evaluation Criteria 2d

z Measures how possible the alternative will be to implement. z Is the existing administrative system (e (e.g., g the FDA or Department of Health and Human Services) capable of delivering the policy or program?

ztechnical feasibility zeconomic and financial possibility zpolitical viability zadministrative operability

• How much control does the administrative system have? • What other groups/individuals must be relied upon. • Are you aware of administrative bottlenecks in the existing system? • Are there organizational limitations? • Specifically, will the administrative system have the authority to implement the policy? That is, have you crafted the statutes/regulations correctly? • Is there institutional commitment? That is, is the administrative system willing to back your program? 13

14

Evaluating Alternatives; Projecting the Outcomes

Identifying Alternatives z researched analysis & experimentation z no-action analysis z quick surveys z literature review z comparison of real realworld experiences

• D Does th the existing i ti (or ( proposed) d) administrative d i i t ti system t have h the th capability to implement you program?

z passive collection and classification z development of y g typologies z brainstorming z comparison with ideal

15

z Forecasting {extrapolation {modeling {intuitive forecasting

z Evaluation methods {discounting {efficiency z net present value z benefit-cost f ratio z internal rate of return

z sensitivity iti it analysis l i z allocation formulae

16

Project the Outcomes - 2

Selecting Among Alternatives

zIs modeling possible?

Evaluation Criteria

{magnitude g estimates?

Altternatives

zworst case / best case for sensitivity analysis {minimum acceptable performance given costs {likelihood estimates q of success / failure {collateral consequences

Do nothing

17

18

Stakeholders

Monitoring

Groups, individuals, and businesses, etc. – folks that will be impacted p by y yyour proposal z who will support or oppose your proposal? zhow you will assess the degree of political influence of each stakeholder group? zhow you will assess the position of the stakeholders? t k h ld ?

zEx-ante policy analysis zpolicy maintenance zpolicy monitoring zex-post policy evaluation

19

20

Monitoring 2

Monitoring 3

zBefore-after comparisons zwith / without comparisons zactual / planned performance zexperimental (controlled) models zquasi-experimental models zcost-oriented approaches

Soumerai SB et al. A critical analysis of studies of state drug reimbursement policies: research in need of discipline The Milbank Quarterly 1993;71(2):217

21

22

Monitoring 4 zStrength of research design {randomized controlled trial (“experimental”) ( p ) {quasi-experimental

Analysis Cycle Round 2

zwell controlled (time series with comparison group) zwell-controlled zpartially-controlled • time series without comparison p • pre-post without comparison • post only (cross-sectional)

zinadequately controlled Cook TC and Campbell DT. Quasi-Experimentation: Design and Analysis Issues 23 for Field Settings. Houghton-Mifflin, Boston, 1979.

24

Menu

Measurement Issues

zMeasurement issues zStakeholders zAlternatives zAssessment of Best Alternative(s)

zAdministrative data (& Faust) zHypotheses: … didn’t didn t hurt anyone {Cost saving (silo) {C t saving {Cost i ((overall) ll) {Non-cost Outcomes {Unintended consequences

25

26

Measurement Issues 2

Measurement Issues 3

zObservational vs. experimental studies

z“sufficient data” before and after intervention zNo co-interventions

{What makes for “experimental” p studies? {Policy analysis and experiments zInformed consent zLaw zEthics zWhy/why not studies to inform formulary decisions • Selective introduction

27

28

Measurement Issues 4 6000

z“Policy” vs. “clinical” models

1400000 1200000 1000000 800000 600000 400000 200000 0

5000 4000 3000 2000 1000 -33 -29 -25 -21 -17 -13 -9 -5 -1 3 7 11 15 19 23 27

0

ERVIS

ADMITS

OFFVIS [R] Schneeweiss et al Health Policy 2001:55;97 29

Measurement Issues 5

30

Schneeweiss et al al., policy model

zBiases “adherers”

{Policyy Model zNet effect zMisbehavior Æ underestimation when noncompliance [sic., non-adherence] is high • Necessary to separate high and low adherence

{Clinical Model zSelection bias zSurvivor cohort zReverse causation

“Non-adherers“ & avoiders DCS = differential cost sharing 31

32

Stakeholders -- Analysis

Identifying Your Stakeholders

z The Th b benefits fit off using i a stakeholder-based t k h ld b d approach are that: {You can use the {Y th opinions i i off th the mostt powerful f l stakeholders to shape your projects at an early stage. {Gaining support from powerful stakeholders can help you to win more resources {By communicating with stakeholders early and f frequently, tl you can ensure that th t they th fully f ll understand d t d what you are doing and understand the benefits of your y policy {You can anticipate what people's reaction to your project may be, and build into your plan the actions that will win people's people s support. support 33

Adapted from http://www.mindtools.com/pages/article/newPPM_07.htm

34

Prioritize Your Stakeholders

Understanding your key stakeholders

z Hi High h power, interested i t t d people: l these are the people you must fully engage and make the greatest H i efforts to satisfy. y g z High power, less interested h people: put enough work in with these people to keep them satisfied, but not so much that they become bored with your message. P o z Low power, interested people: w keep p these p people p adequately q y e informed, and talk to them to ensure r that no major issues are arising. These people can often be very helpful with the detail of your L o project. w z Low power, less interested people: again, monitor these people, but do not bore them with excessive communication.

z What financial or emotional interest do they have in the outcome of your work? z What motivates them most of all? z What information do they want from you? z What is their current opinion of your proposed policy? Is it based on good information? z Who influences their opinions generally? Do some of these influencers therefore become important stakeholders in their own right? z If they are not likely to be positive, what will win them around to support your project? z If you don't think you will be able to win them around, how will you manage their opposition? z Who else might be influenced f by their opinions? ? Do these 36 people become stakeholders in their own right?

Keep Satisfied

Manage Closely

Monitor (Minimum Effort)

Keep Infomred

Low

Interest

High

35

Alternatives

Alternatives 2

z… there’s always “do nothing”*

zAlternatives are not necessarily mutually exclusive

{Consider p projecting j g the “do nothing” g alternative into the future as a basis for comparison

{May facilitate or mitigate some problem with basic policy {… “with 3 you get egg roll”

zBasics vs vs. variants {Regulatory, subsidy, tax (dis-)incentive

zFunding if someone has to do something

*recall Bardach’s Bardach s admonition 37

Questions

39

38