Assessing Interim Data Using Figures William Coar, Axio Research, Seattle, WA ABSTRACT Data Monitoring Committees (DMCs) are now standard for many types of clinical studies, and have a primary role to protect patient safety. This is done through interim assessment of data through the course of a clinical trial. Whether the interim reviews are with respect to a formal interim analysis or strictly a review for safety concerns, integrity of the data is of vital importance. Assessment of data quality is often in the form of determining the degree of completeness of the data as well as identifying anomalous or missing values. Two graphical approaches to better understand patient follow-up and completeness of data will be presented. The first approach is based upon standard Kaplan Meier type step plots and is a natural extension of customary time-to-event figures. The second approach entails more custom combination of horizontal bar charts. Both methods can rely on Graph Template Language (GTL). Because of the detailed nature of GTL, only higher-level overviews will be presented. Poor data quality greatly increases the complexity of reporting and interpreting results based on interim data. The approaches presented begin to allow DMCs, clinical data management, and even study teams better understand and improve patient follow-up and data quality.

INTRODUCTION Many randomized clinical trials, whether double blind or open label, can often take years to enroll and observe patients long enough to assess the primary outcome measure. Because of the many inherent uncertainties in conducting clinical trials particularly with patient safety, it is becoming more common to have independent oversight of a clinical trial. Data Monitoring Committees (DMCs) play this vital role. DMCs often consist of medical experts in the disease area under study as well as at least one statistician. The DMCs primary roles are to protect patient safety and assure quality of study conduct. See Ellenburg, et al., [2002] for a detailed explanation of DMCs. The DMC meets periodically to review data from the ongoing clinical trial. Although there is an emphasis on patient safety, there is also a desire to assess some form of efficacy to truly assess the risk/benefit profile of a drug under study. For example, a DMC may need to determine if a potential 2 month increase in a cancer patient’s life may be worth the risk of some potentially harmful side effect. Furthermore, DMCs also review data with respect to study conduct as discussed by Kerr, et al. [2011]. An understanding of the status of the database at the time of the review assists in the decision making process. At the conclusion of each review, the DMC provides a recommendation to the sponsor company conducting the trial. Such recommendations may be: continue without modification, continue with modification, or stop the study. DMC recommendations have a tremendous overall impact on the current and future development of drugs under study. We propose the use of graphical methods to assist with the DMCs assessment of the quality of interim data. Specifically, we present graphics that begin to address the following questions: 1.

How current is the data under review?

2.

Where are patients within the treatment phase, and when are they likely to discontinue treatment?

3.

Are there potential problems with patients choosing to completely withdraw consent, not allowing any more information to be collected?

4.

How much CRF data is missing that one might expect to have at this point in the study?

We consider two approaches: standard Kaplan-Meier methods to analyze time-to-event data and a more custom figure based on basic horizontal bar charts. These approaches presented were developed in a Windows environment using 9.2.

KAPLAN-MEIER METHODS Kaplan-Meier, also known as product limit estimator, measures the fraction of patients without an event at a certain time t. For example, t might be some specific amount of time after treatment in a Clinical trial setting. KM methods also consider censoring, which allows us to account for various follow-up times associated with each patient. Some patients may withdrawal before the final outcome is measured, or a patient simply isn’t in the study long enough for us to observe the event. The various types of censoring are outside the scope of this paper.

1

Assessing Interim Data Using Figures, continued

Figure 1 displays a fairly standard KM plot for the time to disease progression in an oncology trial. Using the eye-ball approach, the plot suggests the median time until the disease progresses is approximately 10 months after treatment is initiated regardless of treatment received in the study. We can also estimate the probably of a patient experiencing disease progression after 5 months to be approximately 0.85.

Figure 1 Sample KM Plot for Time-to-Disease Progression Although an internally developed macro is used to produce the remaining KM plots, the SAS code to produce the example in Figure 1 is fairly simplistic: proc lifetest data=patient method=pl plots=survival(test atrisk=0 to 25 by 5); ods select SurvivalPlot; time ttpmnth*ttpstat(0); strata rtrtgrp / test=Logrank; run; The data going into this procedure consists of one record per patient, each patients randomized treatment group, a flag to identify patients that experienced an event, and a time variable measuring the either the time it took to experience an event, or the duration a patient was followed if no event occurred.

LAST VISIT TO DATA CUTOFF Consider measuring the time from a patient’s last visit entered into a database to a data cutoff where data are extracted for interim review. This lag should be small relative to the expected duration between visits. This method of assessment is more useful for studies with regularly schedule visits, such as monthly. Large lags imply that a subject’s data is not current. This could be due to a patient not actually attending the visits on schedule, or possibly that the data just isn’t entered yet. Regardless, the missing data is of concern. All patients are considered as having an event. Patients that discontinue study (for any reason) are considered current, since we are not expecting any additional data. In this case, a duration of 1 is imputed. A sample is provided in Figure 2.

2

Assessing Interim Data Using Figures, continued

Figure 2 Last Visit to Data Cutoff Given that subjects are to be followed on a 28 day cycle in the above example, we expect the lag from last visit to data cutoff to be around 28 days or less. Figure 2 suggests there are only a few patients whose data are current. A consequence of this may be a delayed DMC recommendation, or a recommendation to a sponsor to increase site monitoring and encourage investigators to enter data in a more timely manner. This figure can be further updated to see if patient follow-up changes when subjects discontinue treatment yet remain in the study to collect long term information. This would allow the DMC to make sure patients were still being following as scheduled, even after discontinuing treatment, which may help protect the integrity of long-term data.

TIME TO TREATMENT DISCONTINUATION It may be of interest to see how long before subjects discontinue study treatment. Complicated dosing regimens or adverse experiences may lead subjects to discontinue treatment earlier than expected. In this case, only subjects that discontinue treatment have an event. All others are considered censored at the time determined by their duration of treatment.

Figure 3 Time to Treatment Discontinuation

3

Assessing Interim Data Using Figures, continued

The above example suggests that subjects in Arm B tend to discontinue earlier that those in Arm A. The DMC may then request further information to determine why such as an aggressive dosing regimen. In extreme cases, the DMC may recommend a protocol modification that might allow the trial continue and still yield meaningful results.

TIME TO WITHDRAW OF CONSENT During the course of the study, patients have the right to withdraw consent for any further participation. Unfortunately, this reason for termination of study is often misunderstood by patients and physicians, and therefore is often misused. It is not unusual for a patient to elect to discontinue receiving protocol specified treatment. What may not be clear is that it can be important for these patients to remain on study to capture (long term) follow-up information. Many feel withdraw of consent should be initiated by the patient, and means that the sponsor is not authorized to contact them or collect any more information.

Figure 4 Time to Withdraw of Consent Figure 4 provides information on when patients are withdrawing consent from participation in the study. In some cases, patients withdraw consent because they feel the drug may be ineffective or that it is not worth the side effects they are experiencing. This would be considered informative missing data and can result in biased estimates at the conclusion of a study. Should the current trends continue, the DMC may request further information in efforts to understand the reasons for this. A potential consequence may be a recommendation to provide additional training to sites (and subsequently patients) to emphasize the importance of remaining on study to capture long term follow-up.

CUSTOM HORIZONTAL BAR CHARTS Yet another way to assess completeness of data is to identify subjects that hypothetically should have data but do not. The anticipated number of subjects for each visit is estimated by considering the duration from first dose administration (or randomization if first dose date is missing) and the data cutoff. Knowledge of this duration allows estimation of hypothetical visits each subject should have. Cross referencing this list with the actual data allows us to identify subjects that are: 1.

Anticipated to have data and do in fact have data (light blue)

2.

Anticipated to have data but discontinued study at a prior visit (medium blue)

3.

Anticipated to have data but no data are found (dark blue)

Figure 5 is a sample of a figure displaying information on Anticipated versus Actual Data by Visit. The left side of the figure provides the anticipated number of subjects at Day 1 of each cycle of study drug administration. The right side of the figure provides information about actual follow-up.

4

Assessing Interim Data Using Figures, continued

Figure 5 Anticipated vs Actual Data by Visit At each visit, the percentage of anticipated subjects in each classification is displayed. A large proportion in the first classification is indicative of good patient follow-up as it implies data is being captured as scheduled. The proportion in the second classification allows the reader to assess when/if subjects are discontinuing study. The percentage of subjects in this category is less concerning since we are not expecting to receive any more information for these subjects. The last classification is the most problematic as it provides a measure of the proportion of subjects that are lacking data that we might expect to have. This could be a function of subject’s data not being entered in a timely fashion. It could also be a function of subjects not following the protocol defined schedule. Regardless of the reason, the darkest blue section of each horizontal bar describes the proportion of patients with missing data at each visit. Figure 5 suggests that the data capture is very timely for this interim data. In general, vital signs data would be used as it is usually captured every time a patient visits a physician. However, this type of presentation can be repeated for individual CRF pages, especially those that may be associated with a primary endpoint for an interim analysis. Repeating Figure 5 by investigator can also help identify sites that my benefit from addition training or monitoring. In one last extension, Figure 5 may also be repeated to assess dosing at each cycle where the right hand side may summarize patients that receive full dose, partial dose, or no dose.

CONCLUSION In short, important decisions are made during conduct of the study by a DMC. Questions such as Are there safety signals that warrant stopping the study? or Is the conduct of the study appropriate? are of great interest to DMCs, sponsor companies and current/future patients. In making decisions on interim data, there needs to be a balance between clean data and current data. The missing data problems that programmers and statisticians face at the conclusion of a trial are inherently worse for a DMC. An understanding of the status of the database at the time of the review assists in the decision making process. Both graphical approaches can easily be modified to suit specific needs with the use of Graph Template Language (GTL). Although straightforward Proc Lifetest procedures can be used to generate information in the above examples, other internal tools that use GTL may be adaptable allowing additional assessment of data quality without much additional investment. Although the concepts discussed throughout this report are in the context of DMCs, they clearly may be useful to site monitors and data management as well. The approaches presented begin to allow DMCs andstudy teams better understand and improve patient follow-up and data quality.

REFERENCES Data Monitoring Committees in Clinical Trials Ellenburg SS, Fleming TR, DeMets DL (2002) Kerr, D. and Grant, S. 2012, Creating a Better, Shorter DMC Report: A Stack of Needles, Not a Needle in a Haystack,

5

Assessing Interim Data Using Figures, continued

Joint Statistical Meetings, San Diego, CA SAS Help: Statistical Graphics using SAS, Modifying the Layout and Adding a New Inset Table (Example 49.2) SAS/GRAPH 9.2 Graph Template Language, Second edition SAS® 9.2 Output Delivery System User’s Guide

ACKNOWLEDGMENTS The author would like to acknowledge the many programmers publishing information on the internet, and apologizes to those not properly referenced above, citing the inherent nature of internet searching makes it difficult to track so many individual ideas.

CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: William Coar, PhD Project Director/Biostatistician Axio Research, LLC 11001 West 120th Avenue, Suite 400 Broomfield, CO 80021 Email: [email protected] SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies.

6