Case Study #3:
Screening for Colon Cancer: A Technology Assessment David M. Eddy, M. D., Ph. D. Professor Program for the Analysis of Clinical Policies Department of Engineering”Economic Systems Stanford University Stanford, Calif.
INTRODUCTION A good case can be made that cancer of the colon is the most important malignant disease in the United States. Although cancer of the colon is second in frequency to cancer of the skin, second to cancer of the lung as a cause of death in men, and the third most common cause of cancer death in women, it is overaII the most common of the “lethal” cancers. About half a million persons currently alive carry the diagnosis of colon cancer (22). This year about 114,000 people will be discovered to have the disease (I), and over half of them will eventually die of it (22).
Colon cancer incidence and mortality rates have been fairly stable over the past decades, and there is little evidence that attempts to improve therapy have helped anyone live longer. Whether one- looks at 3-, 5-, or 10-year statistics, males or females, or cancers diagnosed in local or regional stages, survivaI rates have
moved up but a few percentage points over the last 30 years. Patients with local cancers have a far better prognosis than those with regional disease (with 5-year survival rates of 73 percent as opposed to 47 percent), but the proportion of cancers detected in a local stage has been constant at 41 percent for three decades (2). The desire to screen is based on the finding that screening tends to detect cancers in early stages. Better case-survival rates with cancer detected in earlier stages impIy that patients detected through screening will have a better prognosis than others, and mortality will be reduced. While there are subtle problems with this line of reasoning, it motivates the search for an effective screening program. This case study examines the available technologies used to screen for cancer of the colon: their development, evaluation, cost effectiveness, and use.
DESCRIPTION OF SCREENING TECHNOLOGIES There are three main ways to detect a malignant lesion in the colon1 before a patient will seek care for signs or symptoms: 1) digital rectal examination, 2) sigmoidoscopy, and 3) stool occult blood test. 1 We shall consider both cancer of the colon and cancer of the rectum together, Unless a specific region of the colon is identified, the term “cancer of the colon” refers to any malignant lesion in the large bowel, including the rectum.
Digital Examination The rectal examination is the simplest method and was used by the Egyptians and Greeks. About one-sixth of all colon cancers are within reach of the exploring finger. Though some lesions may be missed because they are too smaIl or blend in with mucus or stool, the test is obviously quite quick, inexpensive, and safe, and 3
4 ● Background paper #2: Case Studies of Medical Technologies
its place in the routine physical examination is firmly entrenched.
Sigmoidoscopy The rigid sigmoidoscope was introduced in the 19th century. In theory, this instrument enables one to examine the entire terminal 25 cm of the colon, where about one-half to two-thirds of all cancers and adenomatous polyps (which may be premalignant) develop. The main advantage of the instrument is that it permits direct visualization and biopsy, or even removal, of a suspicious lesion. The main disadvantages are discomfort to the patient and the possibility of bowel perforation, the latter being reported to occur as frequently as about 1 out of 100 (21) and as infrequently as 1 out of 50,000 examinations (29). The charge for this examination varies greatly from physician to physician, but on the average, patients have to pay about $35. At present, almost all sigmoidoscopies are performed by physicians. In an attempt to reduce the discomfort and perforation rate, flexible fiberoptic sigmoidoscopes have been developed in lengths up to 65 cm. Although used enthusiastically in Europe, at present in the United States the flexible sigmoidoscope is generally considered too expensive and specialized to be used for screening. If the cost can be reduced and if physicians or paraprofessionals without special training in endoscopy can learn to use it safely and effectively (which seems quite likely), the greater length and greater patient comfort provided by the flexible sigmoidoscope could significantly change the cost effectiveness of colon cancer screening programs. @2
Hemoccult Test The third main screening test is to search for occult blood in the stool. Cancers and adenomatous polyps can bleed, but depending on the @ Registered trademark of SmithKline Diagnostics.
2 The guaiac-impregnated slide marketed in the United States by SmithKline Diagnostics under the trade name “Hemoccult” is virtually the only guaiac-impregnated slide used in the United States and comprises about 90 percent of all types of occult blood tests worldwide. Since the available data on the costs and effectiveness of occult blood tests pertain specifically to the Hemoccult, we refer to this trade name in the remainder of this paper.
location and amount, the blood may be invisible to the naked eye. This occult blood can be detected, however, by smearing a small amount of stool on a piece of filter paper, adding a drop or two of a chemical (most often guaiac, orthotolidine, or benzidine), and then adding hydrogen peroxide. In the presence of blood, this combination produces a blue or blue-green reaction. Because small amounts of blood might be mixed unevenly in a large stool sample, and because bleeding can be intermittent, current practice is to take two samples from one stool specimen each day on 3 consecutive days (for a total of six tests). One problem with the stool occult blood test is that substances other than the hemoglobin in blood can produce a blue reaction when exposed to the chemical, thus incorrectly implying that blood is present. Since myoglobin from meat is one substance that can give a false-positive result, individuals being screened may be requested to follow a special diet before collecting the stool samples. A procedure for impregnating guaiac directly into filter paper, thus improving the sensitivity and specificity of the stool occult blood test and making it quite inexpensive and easy to perform, was developed about 20 years ago. As currently marketed under the trade name “Hemoccult” each piece of guaiac-impregnated filter paper (or “slide”) contains spaces for two stool samples. A package of three slides (which are needed for 3 days of screening) costs the physician about $0.75, and the patient might be charged about $4. Collecting stool and smearing it on a piece of filter paper obviously carries no risk. If a patient needs an otherwise unnecessary diagnostic workup because of a false-positive result, however, the risks and additional costs can be large. If a Hemoccult test is positive, the location of the bleeding source must be identified with other diagnostic procedures; These usually include a sigmoidoscopy (if the patient has not already had one), a barium enema, and when available, a colonoscopy. Colonoscopy involves passing a long, flexible fiberoptic scope several feet up the rectum to the cecum. This procedure can be quite uncomfortable and carries some risk of
Case Study #3: Screening for Colon Cancer: A Technology Assessment
perforating the bowel. Perforation has been reported to occur from between once every 87 examinations to once every 458 examinations (3,8,28,30). With improvements in techniques, the perforation rate today is undoubtedly closer to 1 out of 1,000. If these tests do not reveal the
●
5
source of the bleeding, it may be necessary to study the upper gastrointestinal tract with Xrays and other procedures. The workup of a patient with a positive Hemoccult can require up to 5 days and cost $100 to $2,000, depending on the protocol followed.
UTILIZATION OF SCREENING TECHNOLOGIES It is difficult to determine precisely how frequently colon cancer screening tests or examinations are performed in the United States, although some rough estimates can be made on the basis of data from public surveys and marketing reports. Digital Examination.—It has been reported, for example, that in 1977 there were slightly over 1 billion physician visits in the United States, about 25 million of them for general checkups of persons over 45 (24). Twenty-five million represents about one-third of the population in that age group. If all physicians follow the policy of performing a rectal examination as part of every general checkup, and if we believe these statistics, then about 25 million digital rectal examinations are performed each year, and about one-third of the population over the age of 45 receives an annual digital examination. A similar estimate is obtained by examining a 1978 poll of 1,553 men and women over the age of 18 conducted for the American Cancer Society (ACS) by Lieberman Research: Thirty-four percent of respondents reported having “regular” digital examinations (23). Sigmoidoscopy. –The frequency with which sigmoidoscopies are performed can also be estimated from public surveys. The Lieberman poll indicated that 14 percent of men and women over the age of 18 had a sigmoidoscopic exam-
ination at some time in their past, and 4 percent receive one “regularly” (23). A Gallup poll conducted for ACS indicated that in 1976, 9 percent of people over 18 had a sigmoidoscopic examination, although not all of these examinations would have been for screening (7). An informal survey of a dozen physicians in the San Francisco area suggests that about 3 percent include one in a general checkup. These figures imply that about 1 out of 33 or about 750,000 persons over the age of 45 receive sigmoidoscopies each year. Hemoccult. —Current sales of the Hemoccult in the United States in 1979 were about 30 million slides (20). This implies that abouts million
people were screened with the test last year. The Lieberman poll found that in 1978, 16 percent of adults over the age of 18 had heard of the Hemoccult and 3 percent had used it. On the other hand, 8 percent of respondents to the Gallup poll indicated that they used the HemoccuIt in 1976. A sense of the potential use of the test can be gained from the fact that 59 percent of the adults interviewed indicated they would use it if it were easily available (7). The estimates based on polls probably overestimate actual utilization since responders to polls are more likely to overstate than understate their use of preventive medical services.
THE DEVELOPMENT AND DIFFUSION OF THE HEMOCCULT TEST The rectal examination and sigmoidoscope have been used for so long and have evolved so slowly that it is difficult to extract many lessons. Unlike the development of these technologies,
however, the development of the Hemoccult test has been both sudden and recent. Since its story displays nicely how decisions by individual physicians, private industry, and the Govern-
ment intertwine to set policies that affect millions of patients, it will be told in some detail. Prior to the development of the Hemoccult, several chemical tests were used to detect hidden blood in the stool. The three most common, all
introduced shortly after the turn of the century, were the “bench guaiac” (as distinguished from the guaiac-impregnated slide), benzidine, and orthotolidine. These procedures were considered unsatisfactory for large-scale screening, and only marginally satisfactory for hospital use. To be screened by these methods, a patient had to save several samples of stool in dixie cups or jars in the refrigerator for several days, and then physically carry them to the physician’s office or hospital. Needless to say, this was unpleasant for both the patient and the technician who opened the jar. A second problem was that guaiac is sensitive to light and heat. A fresh solution must be prepared at least once a day and even then the sensitivity of the test is not guaranteed. Perhaps more serious, the tests are quite nonspecific. For example, even when the bench guaiac test is used with a meat-free diet to minimize falsepositive results, the false-positive rate (the proportion of cancer-free people who have positive test results) is 32 percent; with an unrestricted diet, the rate is 56 percent. The orthotolidine has a false-positive rate of about 23 percent with a meat-free diet and 32 percent with an unrestricted diet (25,26). Because of these problems, the bench guaiac, benzidine, and orthotolidine tests were rarely used for general screening.
A relatively simple innovation in 1958, however, led to dramatic changes in the stool occult blood test’s value and visibility. Searching for a way to make the test simpler, more stable, and more specific, Dr. Eric H. Mueller decided to impregnate the guaiac resin directly in the filter paper. The patient no longer had to save and store the stool, but could apply it directly to the filter paper and mail it to the physician’s office or laboratory. Since the activity of the guaiac resin is not challenged until it is developed (with hydrogen peroxide and alcohol), its sensitivity and specificity are stable, and in practice the test has a shelf life of at least 3 years. Furthermore, the sensitivity and specificity of the test can be
controlled precisely by altering the amount of guaiac in the paper, and the false-positive rate can be brought to between 1 and 2 percent. Mueller was a medical advisor to Schieffeiin and Co. of New York City. This company copyrighted the trademark “Hernoccult ” and introduced the test commercially in 1958. It did not market the Hemoccult aggressively, however, and sales were negligible. In the n~id1960’s, Schieffelin and Co. sold the Hemoccult trademark to the Laboratory Diagnostics Co. (LDC) in Roselle, N.J. LDC developed a semi-
automated procedure for impregnating the filter paper with the guaiac (which had been done by hand at Schieffelin and Co. ) and packaged it as a single piece of filter paper (or “slide”) contained in a sheath open at both ends. Like Schieffelin and Co., LDC did not invest much to market its product, and sales remained small. The Hemoccult did not begin to receive widespread attention in the profession until the late 1960’s and early 1970’s. The early work was
done by Dr. David Greegor, a private physician from Columbus, Ohio. Greegor reported his use of the Hemoccult in his clinical practice in an article published in 1967 in the Journal of the American Medical Association and described seven cases of colon cancer found among 2,OOO persons given “routine examinations” (14). All seven had positive tests for occult blood. In his
article, Greegor specifically identified the value of searching for stool occult blood, described the now common practice of obtaining two slides a day for 3 days, and by trade name stressed the role of the Hemoccult in screening for colon cancer. He published similar papers recommending the Hemoccult test in 1969, 1971, and 1972 (13,15,16). Although Greegor had a close working relationship with LDC and suggested several design changes for the product, LDC’S support of his work was apparently
limited to a free supply of Hemoccult slides. In 1969, SmithKline acquired the rights to the Hemoccult trademark by chance. In the early 1960’s, Smith Kline & French Laboratories saw that it had few new pharmaceutical products on the horizon, and that its 500-person sales force would be relatively idle unless new product lines were developed. The search for new products
Case Study #3: Screening for Colon Cancer: A Technology Assessment
led the company to LDC, which had a unique system for culturing specimens in a physician’s office, the “Clinicult” test. Although SmithKline’s primary interest was in the Clinicult, and its purchase of the Hemoccult was incidental, it decided in the fall of 1970 to study the Hemoccult’s clinical usefulness and market potential. A market research firm was contracted to learn more about the product and its potential market from about 200 physicians. Favorable results led SmithKline to introduce the Hemoccult in 1970 in two test areas in Ohio. For 6 months, two counties were subjected to a massive promotional campaign in which the Hemoccult was advertised in journals, through direct mail, and at medical meetings. The primary focus of the campaign was to make physicians aware of the published data and recommendations (such as Greegor’s) on the use and effectiveness of the Hemoccult, and then to assess the receptivity of physicians and potential for sales. At the same time, SmithKline improved the method for impregnating filter paper with guaiac and repackaged the test to function like a matchbook. With the new Hemoccult packaging, the patient needed merely to lift the lid, smear stool on the exposed filter paper, close the lid, and put the package in the mail. On the basis of the field tests, SmithKline decided in the spring of 1971 to market the Hemoccult nationally. The poor experience of Schieffelin and LDC made it apparent that this test would not be used widely without a large marketing effort. In 1973, therefore, a separate company, SmithKline Diagnostics, was created within Smith Kline & French Laboratories for the sole purpose of marketing the Clinicult (renamed the “Isocult”) and the Hemoccult tests purchased from LDC. Since 1973, SmithKline Diagnostics has grown to become a multimillion dollar company. In 1978, about 35 percent of the net sales value of the slides, or about $1. s million, was spent on marketing the Hemoccult in the United States (20). Although the structure of the marketing network is complicated and has undergone a num-
●
7
ber of changes over the years, the marketing strategy is conceptually quite simple. SmithKline Diagnostics sees its role as an educational one. The company systematically surveys the literature and conferences to identify all clinical studies that employ the Hemoccult, extracts any data and recommendations, collates and packages the pertinent findings, and disseminates the results to physicians through direct maiI campaigns, advertisements, and more conferences. In practice, this marketing strategy greatly amplifies the impact of anyone who advocates a particular technology. Fewer than a dozen physicians have been featured in Hemoccult advertisements, yet their voices are heard way beyond the journals in which they publish. It is instructive to compare the different standards placed on the medical profession, public organizations, and private industry. Basically, a physician can say anything he/she wants. In 1971, Dr. Greegor could describe 12 cases of silent colon cancer of whom 11 had positive tests for occult blood (with the Hemoccult) and write that “We believe that every adult should have this screening test annually” (15). Dr. Gnauck of West Germany could state that screening with the Hemoccult has been shown to be cost effective (12). Dr. Helfrich could recommend that “In view of the current public colorectal cancer mortality, we would urge the promotion of screening while prolonged analysis of ongoing studies is being conducted” (17). The Journal of the American Medical Association could write a headline that “Regular screening would reduce cancer of colon and rectum toll” and report an ACS task force contention that “nearly three of every four of [colon cancer] patients might be saved by early diagnosis and prompt treatment” (27). There is no formal system to ensure the quality of the reasoning behind physicians’ statements. No substantiation is required before such statements can be published, and at best their “truth” is established only slowly through an informal and often fallible process of consensus. A company like SmithKline Diagnostics can quote these statements, but can never make similar claims by itself. The Food and Drug Ad-
ministration, competitors, and consumers can all challenge any unsubstantiated recommendation. Thus, all product claims have to be approved by virtually everyone who participates in the research, development, and marketing of the Hemoccult, including specialists in legal and regulatory affairs. In general, SmithKline Diagnostics only advertises the Hemoccult as a test to detect hidden blood in the stool. It does not claim that the test is specific for colon cancer (if it is positive), that it guarantees that one does not have cancer (if it is negative), that it detects all colon cancers, or that it reduces mortality from the disease. Armed with data and recommendations from the profession, SmithKline Diagnostics aims most of its marketing effort at practicing physicians. But the company also has an obvious interest in legislators, administrators, and other “opinion leaders. ” The company’s lobbying is low pressure, especially in the United States, but can nevertheless have a profound impact. For example, SmithKline Diagnostics’ representative in West Germany, Rohm Pharma, successfully lobbied to have the West German Government include the Hemoccult in its national screening program. Specifically, all persons over the age of 45 in West Germany are offered a set of guaiac-impregnated slides “with the method according to Greegor. ” This qualifying phrase in effect specifies the Hemoccult test, and the implications have been huge; in the first year the Hemoccult was included in the national screening program, 5.5 million West Germans were screened with over 16 million slides. Very little research was needed to develop the Hemoccult for the market. The basic step of im-
pregnating the filter paper was worked out by Dr. Mueller and Schieffelin and Co. in 1958. Since then, product development has focused mainly on market research (it cost SmithKline about $150,000 to conduct the surveys and field test in Ohio), on improving production meth-
ods, and on packaging. Although this is not inexpensive (a change in packaging can cost hundreds of thousands of dollars), the R&D commitment to this product has been far less than that required for a drug. SmithKline Diagnostics has and may continue to support clinical research and mass screening programs if it is felt they will contribute significant knowledge about the effectiveness, safety, or acceptability of the product, but the company’s budget for such investigations is comparatively small. About a dozen projects, each costing $5,OOO to $10,OOO are supported each year. No special animal or clinical tests of the Hemoccult were required by the Medical Device Amendments of 1976, because it is a product that has been available for decades. Thus, the main continuing “research” expense is for market research, which continues to be indispensable in defining the existing market, discovering new ones, and testing various strategies (20). The manufacturer’s effort with the Hemoccult appears to have been worthwhile. Before the entry of SmithKline Diagnostics in 1973, sales of the product were virtually nonexistent. Since then, sales have grown at an annual rate of 30 to 35 percent, and in 1979 the net sales in the United States was $6 million, which represents
about 30 million slides (20). Although some of the manufacturer’s sales success is attributable to refinements in the function of the product (e.g., sensitivity, specificity, and stability), the real difference has been made by marketing. The case of the Hemoccult dramatically displays the influence on the dissemination of a medical technology that private industry can have. The test has to be good, and some physicians in the medical community have to recommend it, but by increasing the transfer of information, a company or any other organization can pull a product from the shadows and make it the most visible and important new development in the prevention of colon cancer.
Case Study #3: Screening for Colon Cancer: A Technology Assessment
●
9
CLINICAL STUDIES OF THE EFFECTIVENESS OF SCREENING TECHNOLOGIES Digital Examination. —The digital examination is so old and “time honored” that few formal studies of effectiveness have been performed. Its costs are so small and it is so safe that it has not been thought necessary to document its benefits. Only one study might indicate the value of the digital examination. In the controlled trial of multiphasic screening conducted by the Oakland Kaiser Health Plan, about 5,000 persons selected by patient record number were offered a large package of tests that included a questionnaire for signs and symptoms of disease, a digital rectal examination, blood studies for anemia, and a sigmoidoscopy. There was no test for occult blood. A control group of approximately the same size was not offered the tests but was allowed to receive whatever routine care they chose as provided by the Kaiser program. While only about a third of those in the experimental group had sigmoidoscopies, virtually all had their rectums examined. After 11 years of followup, the experimental group had a lower mortality rate from colon cancer than the control group, statistically significant at the p < 0,05 level (5). This study was not designed to determine which tests caused the decrease in mortality, but it is possible that the digital examination played a role.
Sigmoidoscopy.—There is more evidence about the effectiveness of the sigmoidoscope than about the effectiveness of digital examination. One study is the Kaiser trial of multiphasic screening just mentioned. The decrease in mortality in the experimental group possibly is due to the fact that about a third of those screened had sigmoidoscopies. Another study was begun at the University of Minnesota in 1948. By 1979, 18,158 patients over the age of 45 had received more than 100,000 sigmoidoscopies and general physical examinations. Any adenomatous polyps discovered were removed. During the study, 13 can-
cers were found—15 percent of the number anticipated from projections of the cancer incidence rates in the State, which suggests that many of the cancers were to have come from polyps and that a major benefit of the sigmoidoscope is to detect and remove polyps. Beyond
this, all of the cancers were localized at the time of discovery, and not a single one of the 13 patients has died of colon cancer as of 1979 (10,11). A third study was similar in design. From 1946 to 1954, 26,126 men and women over the age of 45 were offered sigmoidoscopies and occult blood tests annually at the Strang Clinic in New York. About 90 percent of the patients had no symptoms of colon cancer, and cancers were discovered in 58 of the 26,126 men and women screened. Of the 50 cancer patients followed over 15 years, the survival rate was close to 90 percent (18,19). Both the Minnesota and New York studies are uncontrolled and could be affected by some of the biases that wilI be mentioned below, but the duration of followup and the survival rates are so marked that some of the observed effect is quite likely due to screening.
Hemoccult.—The Hemoccult has been the subject of many large programs. At a recent meeting of the International Symposium for Colorectal Cancer (supported by an educational grant from SmithKline Diagnostics), investigator after investigator reported the experience in his or her community. Over 10 million persons have been screened in West Germany; 8,000 in an Austrian town; 4,000 in Tampa, Fla.; 35,000 by the Italian Society for Prevention and Detection for Cancer; 69,000 at the Portes Clinic in Chicago; 20,000 in Washington, D. C., and so forth. The great majority of these programs were uncontrolled—designed more to test the feasibility of mass screening and gather preliminary data on the yield of cancers and the false-positive rates than to test the effectiveness of the Hemoccult rigorously.
10 ● Background paper #2: Case Studies of Medical
Technologies
Two controlled studies are in progress, but they are in early stages. About 12,000 people who came to the Strang Clinic voluntarily dur-
ing a certain period of time have been offered an annual sigmoidoscopy and Hemoccult test. About 7,OOO people who came to the Clinic during another period serve as controls. Thus, the study is controlled, but not randomized, and tests the impact of adding the Hemoccult to an annual sigmoidoscopy. Only preliminary results are available at this time and no firm comparisons between the control and experimental groups can yet be made (31,32). A second study of the Hemoccult is underway at the University of Minnesota. Forty-eight thousand people between the ages of 5O and 80 were divided randomly into three groups: one to
receive a Hemoccult annually, another to receive a Hemoccult biennially, and a third to receive no special care (and serve as controls). This study is also in a preliminary stage, but as of June 30, 1979, 94 colon cancers had been detected through screening, 77 percent in a local stage (Dukes’ A or B). The mortality rates at present are similar among the three groups, but the numbers are still too small to be statistically meaningful. Further, there is reason to suspect a delay in the reporting of the deaths in the control group. (There is about an 80-percent chance that, if the Hemoccult test can in fact reduce colorectal cancer mortality by 25 percent, this effect would not appear at this stage in the study.) When completed, this study should provide excellent data on the value of the Hemoccult as a single screening test for colon cancer (9, 10).
METHODOLOGICAL DIFFICULTIES IN EVALUATING THE COST EFFECTIVENESS OF SCREENING PROGRAMS To determine the cost effectiveness of colon cancer screening technology, several questions must be answered. The most important concerns methodology: How does one estimate the costs, risks, and benefits of screening programs? There is no simple process for determining the “cost effectiveness” of a screening procedure. The following are all reasonable measures of effectiveness: change in life expectancy; total mortality; the probability of “curing” a patient (which can be defined mathematically); the 5-, 10-, 15-year cause-specific mortality rate; decrease in morbidity; improvement in quality of life; decrease in earnings lost as the result of premature death; decrease in disability; reduction of anxiety; and so forth. There is much overlap among these measures, but no single measure dominates or contains all the others. Nor do all the measures move in the same direction. In addition, it should be noted that a screening procedure has many types of costs and risks. These include direct financial costs, the risks of a perforation, the morbidity of a diagnostic workup for a positive test, the radiation of
barium enema, lost time from work, the psychic costs of a biopsy, and so forth. Estimating all these outcomes is extremely difficult. The most common sources of information and insights are randomized controlled clinical trials, uncontrolled clinical studies, and quantitative or statistical analyses. Each of these presents formidable methodological obstacles.
Randomized Controlled Trials Ideally, one would like to base a cost-effectiveness analysis (CEA) on the results of randomized controlled trials (RCTS) that measure all the important outcomes of all the important screening options. Unfortunately, RCTS, like other evaluation techniques, cannot provide all the answers needed to assess cancer screening programs. Because they are quite expensive and time consuming (it takes 10 years to observe 10-year survival rates), RCTS can only be used to evaluate one or two options. For example, two controlled studies are in progress: one assessing the marginal value of an annual Hemoccult in addi-
tion to an annual sigmoidoscopy in a voluntary screening program, the other assessing annual and biennial Hemoccults. These studies will not measure the value of an annual Hemoccult and triennial sigmoidoscopies, or the use of a Hemoccult only on persons who have negative sigmoidoscopies, or any of the other possible options. To use RCTS to determine the costs and benefits of screening with every combination of tests applied every 1, 2, 3, . . . years, to males and females, who are high risk, average risk, low risk, starting at age 40, 45, 50, . . ., in cities and in the country . . . is impossible. Another problem is that clinical trials of cancer screening tend to focus on only one outcome— mortality. Although there is nothing inherent in the methodology that requires this, RCTS usually ignore costs, risks, and morbidity. A third problem is that the results of RCTS are vulnerable to technological change. It is quite likely that the technology being assessed will change in the 10 to 15 years required to complete the RCT. Thus, the results of the trial may be invalid the day they are available. The sensitivity of the Hemoccult test first used in the Minnesota study, for example, was temporarily changed to increase both its sensitivity and false-positive rate by dehydrating the slides before they were read. For these and other reasons, RCTS cannot provide all the information needed to make decisions about screening tests.
Uncontrolled Clinical Studies The next thought is to turn to uncontrolled studies for information that might be used in CEA. Several additional methodological problems, however, some of which are peculiar to screening programs, complicate the use of information from this source.
Lead-Time Bias.—It is frequently noticed that cancers detected by screening are in earlier stages and have better case-survival rates than cancers detected in the interval between screening examinations or in control populations. It is tempting to infer from this that screening is beneficial, The facts that a screening test can detect a condition before it is detectable by other
means, that it tends to detect cancers in earlier stages, and even that it delivers higher case-survival rates, however, do not necessarily mean that the test will increase the chance for a “cure” or prolong a patient’s life. It is possible that the time of detection (and the stage at the time of detection) has no effect on the course of the disease and that earlier detection only moves forward the time of a patient’s diagnosis, without moving back the time of death.
Patient Self-Selection Bias.—Another problem is that persons who elect to receive screening tests may be different from those who do not, in ways that could affect their survival from a disease like cancer. For example, they could be more health conscious; more likeIy to control risk factors such as smoking, diet, and sexual habits; more astute to the presence of signs and symptoms of disease; or more compliant to treatment. Any of these factors could produce a longer survival from cancer in a way that is independent of the screening program. The observance of better survival in screened patients as compared with the general population, therefore, could be due more to the selection of patients than to the effect of the screening test. In this case, the assessment of benefits from screening would be biased.
Length Bias.—A third problem that biases the estimation of screening benefits is that cancers detected through the act of screening in a periodic screening program tend to have longer preclinical intervals than average. (Conversely, the interval cases tend to be cancers with shorter preclinical intervals. ) The preclinical interval is the interval between the time a screening test could detect a cancer and the time a patient would seek care on his or her own initiative for signs and symptoms that appear in the absence of screening. The duration of this interval is related to the growth rate and other biological characteristics of the tumor, and to the awareness of the patient to cancer signs and symptoms. Both of these factors can influence how long a patient survives from the time of diagnosis. For example, it may be that tumors that have long preclinical intervals have slower growth rates and are less malignant.
12 Backround Paper
●
#2: Case Studies of Medical Technolologies
Thus, compared to control or unscreened patients, patients detected through screening have longer preclinical intervals—which may imply slower growth rates, less malignancy, and longer survival. Patients with cancers missed at screening but self-detected between examinations have shorter preclinical intervals, which may mean faster growth rates and lower survival. It can also work the other way, however. The preclinical interval can be long if a patient delays a long time before seeking care. This delay would lengthen the preclinical interval by postponing its end point, without changing the cancer’s rate of growth.
peutic procedures (e.g., the perforation rate of a sigmoidoscopy); 9) the sensitivity and
specificities of the Hemoccult, digital rectal examination, and sigmoidoscopy; 10) the effectiveness of therapy; 11) the proportion of malignant lesions that come from pol-
●
Formal Cost= Effectiveness Analysis Because of the difficulties of measuring in actual clinical practice the costs and benefits of different colon cancer screening programs, it is useful to supplement the results of clinical research with insights gained from formal CEAS
which combine the results of clinical research and clinical judgment. Unfortunately, quantitative analyses of the cost effectiveness of colon cancer screening programs are also difficult to perform. First, CEAS of colon cancer screening programs share most of the methodological problems that plague all medical CEAS. It is no easier to measure the intangible outcomes of cancer screening programs than it is to measure those of any other medical program. Nor is it any more obvious in screen-
●
●
ing for cancer whether $1,000 should be spent to give a person 10 more days of life expectancy. Second, colon cancer screening programs have special features that make them especially difficult to evaluate. To be useful, quantitative analyses of colon cancer screening programs should take the following into account. ●
Over a dozen factors have to be analyzed. These include: 1) patient age and sex; 2) risk factors; 3) schedule of digital rectal examinations; 4) schedule of Hemoccult tests; 5) schedule of sigmoidoscopies; 6) age- and sex-specific incidence rates of colon cancer; 7) age- and sex-specific mortality rates of colon cancer and other causes of death; 8 ) complication rates of diagnostic and thera-
●
yps; 12) the proportion of polyps that will eventually become malignant; and 13) the proportion of malignant lesions that develop in the three regions of the colon. There are three screening tests, each with a different cost, a different ability to detect cancers, and a different false-positive rate. Thus, there are four ways that a malignant lesion can come to clinical attention in a screening program: through any one of the three tests at a screenin g s e s s i o n , o r through self-referral by the patient for symptoms in the interval between screening examinations. The proportions and survival of patients in each group will vary with the structure of the screening program, and all must be analyzed in unison. The “accuracy” of any particular screening test is not constant but varies continuously, from virtually zero when the cancer is one cell large to virtually one when the cancer is 14 cm in size. Thus, it is quite unrealistic to assume that the true-positive rate of any particular test is a fixed percent. There are at least three different sources of colon cancer that must be analyzed separately. Some cancers come from villous adenomas, some from adenomatous polyps, and some arise de novo from the mucosa. What proportion of invasive cancers come from each source is not certain. Not all villous adenomas or polyps become malignant. In addition, the ability to detect each source is different for each screening test. Cancers from each source develop at different frequencies in different regions of the colon. This is important because each screening test searches a different region: The finger can explore the distal 10 cm; the sigmoidoscope can explore the distal 25 cm; and the Hemoccult can detect bleeding from anywhere, including the upper gastrointestinal tract.
Case Study #3 Screening for CO1 ON Cancer; A Technology Assessment
●
●
The three screening tests can be given in any order and in any frequency. The question is not, “Is screening for colon cancer cost effective?” One must try to estimate the costs and effectiveness of a virtually limitless number of ways that screening programs can be constructed. The whole problem changes with time. Sequential screening for cancer is not like using a test for differential diagnosis, or screening for a disease such as sickle-cell
●
●
13
anemia or Tay-Sachs that a patient either “has” or does not have. Patients age; at any moment they may develop the disease; cancer grows over time; the sensitivity of the tests vary as the cancer grows; and so forth. The gathering and interpretation of clinical data for use in quantitative CEAS of screening programs is complicated by the existence of various biases (e. g., lead-time bias, patient self-selection bias, length bias).
combinations and frequencies of screening tests. The mathematical model employed in this instance can be used to estimate a wide variety of outcomes. 3 A few of the most important are displayed in table 1. The eight screening policies examined in this example were chosen to display the effect of different frequencies of sigmoidoscopic examinations. In the absence of a screening program, probability that a 50-year-old woman will diagnosed as having cancer sometime in the of her life is about 5 percent. Table 1 shows
the get rest that
3 The mathematical theory, other assumptions, and the method of estimating parameters are described elsewhere (6). For this ex-
ample, the effectiveness of the tests ere estimated from data col-
lected in the clinical trials already discussed. To generate the results shown in table 1, it was assumed that the false-positive rate of the Hemoccult test is 1 percent, that 75 percent of cancers come from adenomatous polyps, and that 67 percent of cancers and polyps are within reach of a sigmoidoscope. The effect of changing each one of these and many other assumptions has been explored.
14 ● Background Paper #,2: Case Studies of Medical Technologies
a major benefit of screening is to discover ade-
nomatous polyps before they become malignant; if, as assumed in this example, 75 percent of all colon cancers come from such polyps, all programs with a sigmoidoscope show a major decrease in morbidity from cancer (column 2). The increase in life expectancy (column 3) applies to all women who are screened. Since most women (about 95 percent) will never get cancer, this number is small. The increase in life expectancy for a woman who will eventually gel colon cancer is much greater, on the order of 4 years. The precise increase will depend on the woman’s age at the time of diagnosis. The savings in lost earnings are given in column 4 . Through postponement of the time of death, those screened have a longer productive life, reflected by a decrease in the amount of earnings lost due to premature death. The present value of the financial costs of the screening program (discounted at 3 percent) is shown in column 5. These costs include the immediate costs of screening, working up all patients who have positive tests, initial therapy, terminal care for colon cancer, and terminal care for other causes of death. In figure 1, the increases in life expectancy are compared to the present value of the financial Figure l.–Colon Cancer Screening: Effect on Life Expectancy and Program Costs of Different Combinations and Frequency of Tests
costs of the screening program. With an annual Hemoccult test and a sigmoidoscopy every 5 years, about 89 percent of the increase in life expectancy is obtained at about 13 percent of the cost, compared with an annual Hemoccult test and a sigmoidoscopy every year.
Costs and Benefits of Screening at Different Ages To decide the age at which screening with a sigmoidoscope should begin, the problem can be viewed from the standpoint of a 45-year-old woman who is receiving an annual Hemoccult test and who must decide whether to begin receiving sigmoidoscopies now or wait to begin receiving this examination when she is so or 55 years old. For a program consisting of an annual Hemoccult test and a sigmoidoscopy every 5 years, the impact on some of the costs and benefits of starting sigmoidoscopy at various ages is shown in table 2. This example assumes that the woman is receiving a Hemoccult test annually from age 45 on. The effect of starting this test at different ages could also be examined.
Sensitivity Analysis Many factors affect the structure of a colon cancer screening program, and it is important to explore the impact of different assumptions. Consider two examples: 1) the false-positive rate of the occult blood test, and 2) the proportion of malignant lesions that come from polyps.
50-year-old, average-risk woman
100 80
0 0
100
200
300
400
500
Increase in costs (dollars)
600
700
The false-positive rate of the Hemoccult has its main effect on the risks, pain, and costs of screening. For example, it can be very expensive to track down a hidden source of bleeding. Estimates of the proportion of people who do not have cancer or adenomatous polyps, but who do have positive tests, range from less than 1 to 10 percent. The importance of a low falsepositive rate is shown in table 3, which gives the life expectancy and present value of screening costs for a 50-year-old, average-risk woman under different assumptions about the proportion of noncancer patients who have positive Hemoccult tests. The screening program in this example includes an annual Hemoccult test and a sigmoidoscopy every 5 years.
Case Study #3 Screening
for Colon
Cancer A Technology Assessment
●
15
Table 2.—Age To Initiate Sigmoidoscopy: Cost-Effectiveness Measures, 45-Year-Old, Average-Risk Woman Age to start sigmoidoscopy 45 50 55
Decrease in probability get cancer 2.34% 1.92 1.82
Increase in life expectancy (days) 81,80 75.76 67.96
Present value of program costs
Decrease in lost earnings $194.58 166.72 137.10
SOURCE. D M Eddy, Screening for Cancer. Theory, Analysis and Design (Englewood Cliffs, N.J.
$109,40 86.08 71.73 Prentice Hall, 1980)
Table 3.—Effect of Different Assumptions About the Hemoccult False-Positive Rate: Cost-Effectiveness Measures, 50-Year-Old, Average-Risk Woman False-positive rate 1% 5 10 SOURCE D. M Eddy,
Life expectancy (days) 74.54 74.41 74.25
Increase in cost of differential diagnosis $66.81 330.11 659.23
Screening for Cancer Theory, Analysis and Design
The importance of the false-positive rate is obvious from this table. In a national screening program with perfect compliance, a shift of the false-positive rate of 1 percent will increase the costs of screening by about one-half billion dollars each year. Another important factor affecting screening is the role of adenomatous polyps in the genesis of malignant lesions. It is not known how many cancers arise from adenomatous polyps, the estimates ranging from none to all. The results given in tables 1, 2, and 3 were based on the assumption that 75 percent of malignant lesions come from adenomatous polyps. If only 25 percent of malignant lesions come from adenom-
Increase in present value of screening costs $ 74.09 337.44 666.63
(Englewood Cliffs, N J. Prentice Hall, 1980)
atous polyps and 75 percent come directly and rapidly from the mucosa, we would expect to observe costs and benefits such as those in table 4. The changes in life expectancy and changes in program costs associated with these different assumptions about the Hemoccult false-positive rate and the role of polyps are shown in figures 2 and 3, respectively. The points marked by squares in figure 2 represent different falsepositive rates (FP) of the Hemoccults. The points marked by triangles in figure 3 are based on the assumption that most malignant lesions (75 percent) arise de novo from the mucosa.
Table 4.—Effect of Different Polyp/Cancer Assumptions: Cost” Effectiveness Measures, 50-Year-Old, Average-Risk Woman Frequency of tests (e.g., every X years) Hemoccult Scope
1
1
1
5
Decrease in probability get cancer 1 .190/0 0.59
Increase in life expectancy (days) 72.84 48.15
Decrease in lost earnings $153.09 103.01
Present value of program costs — $627.27 166,79 —
SOURCE. D M Eddy, Screening for Cancer Theory, Analysis and Design (Englewood Cllffs, N J Prentice Hall, 1980)
16 . Background Paper #,?, Case Studies of Medical Technologies
Figure 2.—Changes in Life Expectancy and Program Costs That Result From Using Different Assumptions About the Hemoccult FaIse-Positive Rate (FP) 50-year-old, average-risk woman
Figure 3.—Changes in Life Expectancy and Program Costs That Result From Using Different Assumptions About Polyps 50-year-old, average-risk woman
1
1
0 0
100
200
300
400
500
600 700 800
100
200
300
400
500
600 700 800
Increase in costs (dollars)
Increase in costs (dollars)
CURRENT POLICIES Most medical students are taught to perform a digital examination with every routine patient workup. The CEA behind this policy was very informal and performed long ago. The costs and risks are small, and although the benefits are unknown, they are almost certainly positive. The sigmoidoscope involves greater costs, discomfort, and risks, but its benefits are also perceived to be greater. Even before the Minnesota and New York studies of the sigmoidoscope were completed, there was a general consensus that it was a valuable screening test. Medical students were taught to include it in their routine annual examinations of persons over age 40, and a complete “executive” screening examination would have to offer it. Like that of the digital examination, the CEA of the sigmoidoscope was informal and never written down. For example, a workshop sponsored by the National Cancer Institute (NCI) concluded (4): The National Cancer Institute and the American Cancer Society should recommend to all physicians that all patients over 40 should have a proctosigmoidoscopy at least every three years by a qualified physician or proctotechnologist.
The report of the workshop did not contain any estimates of costs, risks, or benefits. It is also significant that at three recent meetings convened to assess the evidence on the effectiveness of screening for colon cancer, d most of the time was spent discussing the Hemoccult test; the sigmoidoscope was virtually ignored, perhaps because its value was taken for granted. Policies on the Hemoccult test are still evolving, however, and it is instructive to examine some of the forces that are currently shaping them. The Hemoccult has been the subject of intensive scrutin y in the last 2 years. One of the most important decisions was made at NCI in 1972, when a request was sent out for proposals to conduct an RCT. It is well known that the evaluation of screening programs is extremely complicated and that many biases can confuse the interpretation of uncontrolled clinical data. RCTS correct for sampling, selection, lead-time, ‘These meetings were: 1) “The State of the Art Conference: Screening and Early Detection of Colorectal Cancer, ” sponsored by the National Cancer Institute, June 26-28, 1978; 2) the 1979 workshop on “Approaches to Prevention and Treatment of Large Bowel Cancer, ” sponsored by the National Large Bowel Cancer Project, Jan. 31-Feb. 2, 1979; and 3) the International Symposium on Colorectal Cancer, Mar. 6-7, 1979.
and length biases, and greatly simplify the job of constructing a screening policy. There is little doubt that policy decisions made when the NCIsponsored trial is nearing completion 5 years from now will be dominated by its results. Unfortunately, in addition to providing invaluable information about the effectiveness of the test, the existence of a complete RCT may introduce some problems. One is that of external validity. It may be that special aspects of the program or population (other than the use of the Hemoccult) will determine the outcome, and different results would have been obtained in a different setting. Second, once the trial is completed it may be tempting to think that the information base is complete. For example, the costs and risks may be ignored. Or it may be that the test could be used more efficiently every 6 months or every 3 years, or should begin at age 40 or 55, but there would be no comparable “scientific evidence” of this and it is unlikely that such a policy would be adopted.
A third problem is that any clinical study, RCTS included, can give “false-negative” results. This could happen in several ways. The sample size of the Minnesota RCT of the Hemoccult is such that if there is a true effect of screening (i. e., if it actually reduces mortality by 25 percent), there is about an 80-percent chance that the trial will fail to detect it. Other false-negative results could occur as follows. If an annual Hemoccult fails to reduce mortality in Minnesota, some might feel (incorrectly) that all screening for colon cancer is ineffective, generalizing from the Hemoccult to condemn the rigid sigmoidoscope, 65-cm flexible sigmoidoscope, immunological tests for occult blood, and other tests. If the New York study using the Hemoccult in addition to an annual sigmoidoscope fails to show an effect, some might conclude that the Hemoccult, or sigmoidoscope, or both are ineffective, when in fact the Hemoccult alone or sigmoidoscope alone might well be effective, and it is just the incremental effect of the Hemoccult that was too small to be detected in the trial. These are some of the possible policy implications of the RCTS currently in progress. In the meantime, decisions must be made with less
complete information. There is no doubt that the Hemoccult can detect occult cancers in asymptomatic people, nor that cancers detected in asymptomatic people tend to be detected in earlier stages than those found after signs or symptoms have developed. There is suggestive evidence that screening with the Hemoccult is effective in reducing mortality. The controlled studies designed to correct for the biases are still not far enough along to permit final conclusions. In addition there are costs and risks, especially associated with the workup of patients who have false-positive screening tests. With uncertain effectiveness and documented costs and risks, at least three organizations have concIuded that mass screening with the Hemoccult cannot be recommended at this times The International Symposium and ACS went on to address the use of the Hemoccult in private practice. Given the evidence from one controlled and two uncontrolled studies that early detection of colon cancer does reduce overall mortality, these two groups have recommended that the Hemoccult can be used by individual physicians, provided the evidence of benefit, the risks, and the costs are carefully explained. Recognizing that some group screening is currently being performed with the Hemoccult, ACS defined conditions that should be satisfied. Specifically: While mass screening is not recommended, if screening is to be conducted other than in a physician’s office, for example as a demonstration project, the following criteria must be met: 1. The test should be delivered in accordance with the recommended ACS guidelines for early detection. 2. There should be provisions for tailoring the guidelines to the specific needs of the persons to be screened. 3. The persons to be screened are adequately informed face-to-face (one-on-one or in a group setting) of the expected benefits, the risks, and the fallibility of the test.
‘The three were: 1) “The State of the Art Conference: Screening and Early Detection of Colorectal Cancer, ” sponsored by the National Cancer Institute, June 26-28, 1978; 2) the International Symposium on Colorectal Cancer, Geneva, Switzerland, Jan. 28-30, 1980; and 3) The American Cancer Society Report on the CancerRelated Health Checkup, February 1980.
