Review article HEALTH ECONOMICS AND ORTHOPAEDICS N. Maniadakis, A. Gray From the Institute of Health Sciences, Oxford, England

t is becoming widely accepted that research which considers only the outcome and not the costs associated with new technologies in health care, is of limited value in making decisions about the use of scarce resources. Economic evaluation is becoming a standard feature of clinical research but many published economic evaluations fall short of best practice in their methodology. We have described the essential features of economic evaluation, using published studies in orthopaedics, in order to try to improve the ability of orthopaedic surgeons to read, understand and appraise such studies critically, and to encourage them to consider including economic evaluation in future investigations.

I

J Bone Joint Surg [Br] 2000;82-B:2-8. Received 20 October 1998; Accepted after revision 23 July 1999

Over the last three decades the demand for health care has increased rapidly. Since resources are scarce and cannot meet all demands, it is important to ensure that they are allocated in ways which maximise overall social benefit. The decision on which procedures to adopt depends upon their cost-effectiveness. Thus, economic evaluation of interventions in health care is now becoming common. It is concerned with the systematic comparison of the costs of the resource and the outcomes of alternative interventions, including drugs, devices, equipment and supplies, clinical procedures and organisational and managerial systems. In orthopaedics new technologies are emerging rapidly and these are often more expensive than existing treatments. Considerations of cost-effectiveness are therefore unavoidable. Economic evaluation in orthopaedics has been less common than in other areas of medicine, such as cardiology

N. Maniadakis, PhD, Manager Searle Division of Monsanto, Global Health Outcomes, PO Box 53, High Wycombe, Buckinghamshire HP12 4HL, UK. A. Gray, PhD, Director Health Economics Research Centre, Institute of Health Sciences, University of Oxford, Old Road, Headington, Oxford OX3 7LF, UK. Correspondence should be sent to Dr A. Gray. ©2000 British Editorial Society of Bone and Joint Surgery 0301-620X/00/19575 $2.00 2

or oncology, but this is almost certain to change. In this review we outline how economic evaluation has been used so far in orthopaedics, and highlight the key points which should be considered in pursuing such studies. We wish to try to improve the ability of orthopaedic surgeons to read, understand and appraise published economic evaluations critically, but we also hope to encourage them to consider using such information in their future investigations.

What is economic evaluation? Economic evaluation is concerned with efficiency. Its purpose is to help to select those interventions which maximise the benefit to health from given resources. It compares the costs and outcomes of each available option. If the aim is to find whether a new treatment for back pain is preferable to one which is currently in common practice, the ‘comparator’, four situations may arise when comparing the costs and effects of these treatments. These are represented graphically by the quadrants of the cost-effectiveness plane 1,2 as shown in Figure 1. In this diagram the horizontal axis represents the difference in effect between the new procedure and the relevant comparator and the vertical axis the difference in costs. In quadrant II of the plane the new treatment ‘dominates’ the comparator since it is less costly and more effective. The opposite happens in quadrant IV in which the comparator dominates. The dominant therapy should always be preferred. The decision is not as straightforward in quadrant I, however, in which the new treatment is more effective and more costly, or in quadrant III, in which it is less effective and less costly. There is then a trade-off between costs and effects, and a decision on which treatment to use will depend on how much we are prepared to pay for a gain in effectiveness. This means placing a ceiling on the acceptable cost per unit of effect, as shown graphically in Figure 1 by the dotted line which passes through the origin.

Outcome measures Depending on the outcome measure used, economic evaluation can be classified into three main groups, namely analysis of cost-effectiveness, cost-utility and cost-benefit. In analysis of cost-effectiveness a single outcome is measTHE JOURNAL OF BONE AND JOINT SURGERY

REVIEW ARTICLE

3

Fig. 1 Diagram showing the cost-effectiveness plane.

ured in terms of physical or natural units, such as the 3 number of complications avoided in surgical operations, 4 the mobility score or the number of life years saved as a 5 result of treatment. If the effectiveness of the treatments considered is equal, or almost equal, the objective is to find that which minimises costs. This is described as cost6 minimisation analysis. Studies reporting costs and multiple outcomes separately, leaving the reader to decide what weight to attach to each, are sometimes referred to as cost7,8 consequence analyses. It may be desirable to measure several dimensions of the outcome of an intervention such as the quality of life and survival. These composite measurements of health are referred to by health economists as utility states, since they reflect the valuation which an individual places on a particular state of health and the well-being derived from it. Utility may be measured on an index with a scale from 0 to 1 (or 100) in which 0 relates to the worst and 1 (100) to the best imaginable state of health. A year in a particular state of health can then be termed a Quality Adjusted Life Year (QALY). The instruments and methods available for evaluat9-12 Because quality ing health states are discussed elsewhere. of life is important and this type of analysis facilitates broad comparisons between health-care interventions, such cost13,14 utility analyses are used increasingly in orthopaedics. Cost-benefit analysis attempts to report all costs and effects associated with an intervention in monetary terms, for example, by estimating a value of life. Various approaches can be used to convert health effects into monetary terms. Traditionally, in the human capital approach, average earnings were attached to working time gained as a result of an VOL. 82-B, NO. 1, JANUARY 2000

intervention and the total value was then measured as the present value of all future earnings. There are examples of cost-benefit studies in orthopaedics using the human capital 15-18 approach, but it has severe disadvantages in that it places very low valuation on those not in paid employment. There are alternative methods, but all present difficulties; consequently, cost-benefit analyses are not commonly used at present. It is important to choose the outcome and design of the study which will best assess the materials. In general, analysis of cost-effectiveness is more appropriate when the effect of the treatments considered can be captured by a single measure. If quality of life and survival are important outcomes, analysis of cost-utility is preferable. Utility measures (preferably QALY) should be always included in 19 the analysis.

The perspective A health intervention may affect patients and their families, the health-care sector, other fields and society as a whole. In theory, economic evaluation should consider all the effects and resource consequences of the treatments assessed. In practice, however, these often depend on the purpose of the evaluation, which may be undertaken from 17,18 the hospital secthe perspective of particular agencies, 13 20 tor or society in general. The results may affect these various agents quite differently and therefore the perspective adopted should be clearly defined. For example, a study comparing surgery with conservative treatment for the management of a herniated

4

REVIEW ARTICLE

Table I. Cost (US$1989) of surgery v conservative treatment for the management of a herniated lumbar intervertebral disc (adopted from 21 Shvartzman et al ) Treatment

Cost of treatment

Compensation cost

Total cost

Surgery Conservative

26 643 16 572

29 411 37 066

56 054 53 638

21

lumbar intervertebral disc found no significant differences in cost or effect and thus neither treatment was preferable to the other. Table I shows that the total cost comprised that of the treatment together with the compensation related to absenteeism. Since the outcome was the same, the hospital would consider rehabilitation to be more cost-effective, whereas the insurance scheme, which pays the compensation, would prefer surgery. To date, most economic evaluations have measured only costs occurring within the health sector, and often include only those of the hospital. Many interventions have much broader implications, however, and such narrow perspectives do not give the full picture. In determining policy, generally measurement of the overall effect on society is to be preferred, so that the observations can be arranged into 19 multiple viewpoints depending on the use of the study.

Effectiveness Studies on cost-effectiveness rely on good information. The most reliable study for obtaining such data is the randomised double-blind controlled trial, with other study designs 22 ranked thereafter. In orthopaedics there have been very few economic evaluations carried out in association with randomised controlled trials, as has been shown in system23,24 Most of such evaluations atic reviews of the literature. 3 20 have focused on drugs or conservative treatments rather 25 than on surgical interventions, which are more difficult to assess. Randomised controlled trials of surgical procedures are now progressing, however, as in the Spine Stabilisation Trial and the Knee Arthroplasty Trial funded recently by the Medical Research Council and the National Health Service Research and Development Programme, respectively, in the UK. In spite of their advantages such trials can sometimes be unnecessary, inappropriate, impossible, inadequate or very 26 costly. Observational studies (e.g., cohort, case-control) are then needed to evaluate effectiveness, and these are the most common form of investigation undertaken so far in orthopaedics. Such studies, however, are more vulnerable to bias and errors in inference which increases the uncertainty concerning their results, especially when they involve small samples. In some cases evidence on effec6 tiveness comes from a review of the literature, or from a meta-analysis which synthesises data from a variety of 5,27,28 primary clinical studies. By pooling a number of such studies, meta-analyses achieve large sample sizes and thus make it possible to detect small differences in effects, but it is crucial to ensure that the treatments are comparable, the

original studies have been conducted properly, the bias to which they are subject has been minimised, and that differ22,29,30 ences in setting are acknowledged.

Costing Costing involves measuring the quantities of resources consumed to obtain a cost per patient. Collecting full details of all resources used in an intervention can be timeconsuming, but information such as the length of stay, numbers of consultations or time in the operating theatre can be obtained from the clinical notes or from hospital information systems. Costings based on microdata are likely to be more accurate and reliable than the average costs or prices often quoted in national statistics or hospital reports. Theoretically, the correct price for a resource is its opportunity cost, namely the value of the opportunities forgone by not using the resources in the best way, but in practice market prices are often used instead. Usually, these are easily obtainable, but there are difficulties in measuring the value of resources such as volunteer, family, or leisure 31 time. It is also not easy to assess the costs of capital outlay in buildings and equipment, and the overhead costs of resources such as administration, which are not directly related to the treatment. Alternative ways of treating such 10,12,31 costs have been discussed. Most evaluations in orthopaedics have been confined only to costs strictly related to the hospital treatment provided. Very few studies have attempted to measure costs incurred outside the hospital. A treatment, however, may transfer costs from the hospital sector to the patients and their families, and to other organisations, and these effects should also be considered. Most published studies focus on the methods used to derive estimates of effectiveness, and fail to give detailed information as to the estimation of the overall costs of the treatments considered. Costing should be done cautiously, the data should be as precise as possible and the methods used to derive the estimates should be explained thoroughly.

Analytical horizon and discounting The study should be long enough to capture all the major resource implications and health effects associated with the treatments assessed and the cut-off points should be clearly defined. To show how the time assessed matters, consider again a study on the management of a herniated lumbar 21 intervertebral disc. The length of this study was five years, and the total costs per year are given in Table II. The initial costs of surgery were higher, but subsequent expense was substantially lower. Hence a study with a limited period of follow-up may reach a misleading conclusion. Unfortunately, it is not always easy to determine the appropriate cutoff point in advance, as witnessed by studies of similar 8,13,20 conditions adopting different periods of follow-up. THE JOURNAL OF BONE AND JOINT SURGERY

REVIEW ARTICLE

5

Table II. Average yearly total cost (US$1989) of the treatment for the management of a herniated 21 lumbar intervertebral disc (adapted from Shvartzman et al ) Year Treatment

1

2

3

4

5

Total

Surgical Conservative

36 761 25 510

8350 14 546

5560 5412

3650 4170

1733 4000

56 054 53 638

In orthopaedics interventions are often associated with costs and effects which occur in the long term, and hence assessment may extend far beyond the period for which primary information is available. In such cases mathematical models may be used to extrapolate from the inter5,13,23,24,30 Extrapolation mediate to the final endpoints. beyond the period observed in a clinical study is not always straightforward and the assumptions used should be justified and thoroughly tested. Moreover, since comparisons are made in the present, measurements have to be adjusted for timing. This is because individuals have a positive rate of time preference. They prefer the desirable consequences of health improvements to occur earlier and the undesirable features, such as costs, later. Future effects and costs have therefore to be discounted to the present. At present the consensus among UK economists is that both costs and outcomes should be discounted at the rate recommended by the Treasury of 6% per annum. By 9 contrast, in the USA the rate is 3%. Thus, for comparative purposes analyses using a range of rates should be performed.

Treatment comparator and incremental analysis Economic evaluation is based on comparisons between different courses of action. The cost-effectiveness of a treatment should always be established relative to an appropriate comparator, which may be standard practice, the least costly or more effective alternative or simply doing nothing. There are many examples of orthopaedic studies 3,8,14-18,20,21 using the above comparators. Economic evaluations should always report incremental results, that is the additional costs which a treatment imposes relative to a comparator over the additional benefits it 9-12,32-34 delivers. There are many examples in the literature 10,33,34 of average cost-effectiveness ratios, but they can be misleading. Table III gives the costs and effects associated with treatments for hip replacement in the case of undis14 placed subcapital fracture. The incremental analysis indi-

cates that quality-adjusted life gains come at much higher cost than those indicated by the average analysis. This is because the option to do nothing still yields some QALYs and therefore attributing all the health outcome to the intervention overstates its effect.

Modelling When the costs and effects of an intervention take many years to occur, modelling can be used to extrapolate the findings. It is also useful when diseases are characterised by multiple stages, when data and results need to be moved from one setting to another, or when research needs to compare two treatments which have previously been individually assessed against a common option, such as a placebo. Modelling can be used early in a study to investigate whether it is worth proceeding with the evaluation, to identify the key variables which should attract attention and to estimate the sample sizes needed to detect significant differences in cost-effectiveness. Some commentators therefore argue that the use of models will become un35 avoidable in economic evaluation. There are very few studies in orthopaedics in which modelling has been 5,23,24,30,36 used. This is likely to change, but it should be kept in mind that models also have shortcomings. Those who develop and use them should pay particular attention to sources of information, underlying assumptions, and 29,37 their overall validity.

Uncertainty Economic evaluations are often based on data from various sources which may be imprecise, subject to variation, or involve guesses and assumptions. The samples used may also be very small, increasing the uncertainty concerning any results obtained. A simple way of testing the robustness of the results is a one-way sensitivity analysis in which each component is varied over a certain range to assess the impact on the results. More sophisticated investigation 14

Table III. Cost-effectiveness in the treatment of undisplaced subcapital fractures (adapted from Parker et al )

Treatment A Do-nothing B Conservative C Operative BvA CvA CvB VOL. 82-B, NO. 1, JANUARY 2000

Life expectancy (yr)

Survival rate

Utility score

8.6 8.6 8.6

0.6 0.7 0.7

0.85 0.97 0.98

QALYs

Cost (£1988/9)

Average costeffectiveness

4.902 5.839 5.899

0 2500 2800

428 475

Incremental costeffectiveness

2667 2807 4983

6

REVIEW ARTICLE

Fig. 2 Confidence region for the cost-effectiveness ratio.

Fig. 3 Sensitivity analysis in neonatal screening for hip dislocation.

includes changing two or more variables simultaneously. Uncertainty may be countered by employing stastistical methods to calculate confidence intervals around costs, effects and the estimated incremental ratio of cost-effectiveness. This is illustrated in Figure 2 in which the horizontal and vertical ‘I’ bars show the 95% confidence intervals around the effect difference and cost difference, respectively. The line which connects the intersection of the two ‘I’ bars with the origin of the plane has a slope equal to the estimated cost-effectiveness ratio, and the spherical area gives an approximate idea of the variance around this. There are methods available which measure the confidence inter-

38-41

val around the cost-effectiveness ratio more precisely. Orthopaedic studies have commonly used one-way sensitivity analysis but have failed to deal with uncertainty in 5,13,16,28,30 a more rigorous manner, but this is a general 42 problem with economic evaluations. A good example of how simple but useful sensitivity analysis can be is a study 15 on neonatal screening for dislocation of the hip. As shown in Figure 3, the authors calculated the net benefit per 1000 infants screened for a range of false-negative 43,44 rates used in different studies in order to demonstrate the conditions under which a screening programme would be cost-effective. THE JOURNAL OF BONE AND JOINT SURGERY

REVIEW ARTICLE

7

Table IV. Summary of guidelines proposed by the US Panel on Cost-Effectiveness The framework

The background of the problem being addressed and the general design of the programme under investigation, including the target population, should be stated The type of analysis to be performed should be clearly defined and justified The comparator programme should be described The perspective and the time horizon of the study should be stated

Data and methods

All resources of interest in the analysis should be clearly identified, measured and valued All outcomes of interest should be clearly identified, measured and valued Methods of obtaining estimates of effectiveness, costs and quality-of-life valuations and the sources of information should be given

Results

The base case results in terms of costs, effectiveness and incremental cost-effectiveness ratios should be clearly set out The sensitivity of the results and the assumptions and uncertainties should be reported

Discussion

The results should be placed in the context of other relevant economic evaluations The relevance of the study and policy questions and any ethical or distributive implications should be discussed

Conclusions Economic evaluations of interventions in health care are becoming common and unavoidable. Orthopaedics has seen only a limited number so far, but this will change in the future. It is important that these studies should be standardised and comply with certain underlying principles which are reflected in a number of guidelines recently published on how to perform, report and referee economic valua12,19 The US Panel on Cost-Effectiveness has protions. posed a set of guidelines which are of assistance both to those wishing to appraise existing studies critically and those planning further investigations. An adapted summary of this checklist is given in Table IV. This article is based on work carried out while the authors were involved in the Spine Stabilisation Trial (SST) funded by the Medical Research Council (MRC) in the UK. The opinions expressed here do not necessarily reflect those of the SST collaborators or the MRC. We are grateful to Andrew Briggs and Jeremy Fairbank for valuable comments. Any remaining errors are our own responsibility. No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.

References 1. Anderson JP, Bush JW, Chen M, Dolenc D. Policy space areas and properties of benefit-cost/utility analysis. JAMA 1986;255:794-5. 2. Black WC. The CE plane: a graphic representation of cost-effectiveness. Med Decis Making 1990;10:212-4. 3. Borghi B, Lazzaro C. Indobufen in the prevention of complications in orthopaedic surgery: a pharmacoeconomic assessment. Br J Med Econ 1996;10:129-44. 4. Jonsson B, Larsson SE. Functional improvement and costs of hip and knee arthroplasty in destructive rheumatoid arthritis. Scand J Rheumatol 1991;20:351-7. 5. O’Brien BJ, Anderson DR, Goeree R. Cost-effectiveness of enoxaparin versus warfarin prophylaxis against deep-vein thrombosis after total hip replacement. CMAJ 1994;150:1083-90. 6. Grimer RJ, Carter SR, Pynsent PB. The cost-effectiveness of limb salvage for bone tumours. J Bone Joint Surg [Br] 1997;79-B:558-61. 7. Powell ET, Krengel WF, King HA, Lagrone MO. Comparison of same-day sequential anterior and posterior spinal fusion with delayed two-stage anterior and posterior spinal fusion. Spine 1994;19:1256-9. 8. Javid MJ. Chemonucleolysis versus laminectomy: a cohort comparison of effectiveness and charges. Spine 1995;20:2016-22. 9. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996. 10. Johannesson M. Theory and methods of economic evaluation of health care. Dordrecht: Kluwer, 1996. VOL. 82-B, NO. 1, JANUARY 2000

11. Sloan F. Valuing health care: costs, benefits, and effectiveness of pharmaceuticals and other medical technologies. New York: Cambridge University Press, 1995. 12. Drummond MF, O’Brien BJ, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Second Edition. New York: Oxford University Press, 1997. 13. James M, St-Leger S, Rowsell KV. Prioritising elective care: a cost utility analysis of orthopaedics in the north west of England. J Epidemiol Community Health 1996;50:182-9. 14. Parker MJ, Myles JW, Anand JK, Drewett R. Cost-benefit analysis of hip fracture treatment. J Bone Joint Surg [Br] 1992;74-B:261-6. 15. Tredwell SJ. Economic evaluation of neonatal screening for congenital dislocation of the hip. J Pediatr Orthop 1990;10:327-30. 16. Tunturi T, Niemela P, Laurinkari J, Patiala H, Rokkanen P. Costbenefit analysis of posterior fusion of the lumbosacral spine. Acta Orthop Scand 1979;50:427-32. 17. Brown KC, Sirles AT, Hilyer JC, Thomas MJ. Cost-effectiveness of a back school intervention for municipal employees. Spine 1992;17: 1224-8. 18. Versloot JM, Rozeman A, van Son AM, van Akkerveeken PF. The cost-effectiveness of a back school programme in industry: a longitudinal controlled field study. Spine 1992;17:22-7. 19. Canadian Co-ordinating Office for Health Technology Assessment (CCOHTA). Guidelines for economic evaluation of pharmaceuticals. Ottawa:CCOHTA, 1997. 20. Goossens M, Rutten-Van Molken MP, Kole-Snijders AM, et al. Health economic assessment of behavioural rehabilitation in chronic low back pain: a randomised clinical trial. Health Econ 1998;7:39-51. 21. Shvartzman L, Weingarten E, Sherry H, Levin S, Persaud A. Costeffectiveness analysis of extended conservative therapy versus surgical intervention in the management of herniated lumbar intervertebral disc. Spine 1992;17:176-82. 22. Sheldon TA, Song F, Smith GD. Critical appraisal of the medical literature: how to assess whether health-care interventions do more good than harm. In: Drummond MF, Maynard A, eds. Purchasing and providing cost-effective health care. London: Churchill Livingstone, 1993:31-48. 23. Fitzpatrick R, Shortall E, Sculpher M, et al. Primary total hip replacement surgery: a systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses. Health Technol Assess 1998;2:1-64. 24. Faulkner A, Kennedy LG, Baxter K, et al. Effectiveness of hip prostheses primary total hip replacement: a critical review of evidence and an economic model. Health Technol Assess 1998;2:1-134. 25. Bourne RB, Rorabeck CH, Laupacis A, et al. A randomised clinical trial comparing cemented to cementless total hip replacement in 250 osteoarthritic patients: the impact on health related quality of life and cost effectiveness. Iowa Orthop J 1994;14:108-14. 26. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215-8. 27. Cutler RB, Fishbain DA, Rosomoff HL, et al. Does nonsurgical pain center treatment of chronic pain return patients to work? a review and meta-analysis of the literature. Spine 1994;19:643-52. 28. Borris LC, Lassen MR. Thromboprophylaxis with low molecular weight heparin after major orthopaedic surgery is cost effective. Drugs 1996;52:42-6.

8

REVIEW ARTICLE

29. Sheldon TA. Problems of using modelling in the economic evaluation of health care. Health Econ 1996;5:1-11. 30. Gillespie WJ, Pekarsky B, O’Connell DL. Evaluation of new technologies for total hip replacement: economic modelling and clinical trials. J Bone Joint Surg [Br] 1995;77-B:528-33. 31. Posnett J, Jan S. Indirect cost in economic evaluation: the opportunity cost of unpaid inputs. Health Econ 1996;5:12-23. 32. Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 1977;296:716-21. 33. Karlsson G, Johannesson M. The decision rules of cost-effectiveness analysis. Pharmacoeconomics 1996;9:113-20. 34. Briggs A, Fenn P. Trying to do better than average: a commentary on ‘statistical inference for cost-effectiveness ratios’. Health Economics 1997;6:491-5. 35. Buxton MJ, Drummon MF, van Hout BA, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ 1997;6: 217-22. 36. Chang RW, Pellisier JM, Hazen GB. A cost-effectiveness analysis of total hip arthroplasty for osteoarthritis of the hip. JAMA 1996;275: 858-65.

37. O’Brien B. Economic evaluation of pharmaceuticals: Frankenstein’s monster or vampire of trials? Medical Care 1996;34:DS99-108. 38. O’Brien BJ, Drummond MF, Labelle RJ, Willam A. In search of power and significance: issues in the design and analysis of stochastic cost-effectiveness studies in health care. Med Care 1994;32:150-63. 39. Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ 1997;6:327-40. 40. Van Hout BA, Al MJ, Gordon GS, Rutten FF. Costs, effects and C/E ratios alongside a clinical trial. Health Econ 1994;3:309-19. 41. Chaudhary MA, Stearns SC. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomised trial. Stat Med 1996;15:1447-58. 42. Briggs A, Sculpher MJ. Sensitivity analysis in economic evaluation: a review of published studies. Health Economics 1995;4:355-71. 43. Tredwell SJ, Bell HM. Efficacy of neonatal hip examination. J Pediatr Orthop 1981;1:61-5. 44. Jones D. An assessment of the value of examination of the hip in the newborn. J Bone Joint Surg [Br] 1977;59-B:318-22.

THE JOURNAL OF BONE AND JOINT SURGERY