Reliability of CAD CAM Technology in Assessing Crown Preparations in a Preclinical Dental School Environment Richard S. Callan, D.M.D., Ed.S.; John S. Blalock, D.M.D., Ed.S.; Jeril R. Cooper, D.M.D.; John F. Coleman, D.M.D.; Stephen W. Looney, Ph.D. Abstract: In order to use CAD CAM (Computer Aided Design, Computer Aided Manufacturing) technology as an assessment tool when evaluating the preclinical performance of dental students, it is imperative that one has confidence in the reliability of the process. In this study, a variety of alignment methods were compared to determine both the consistency and accuracy of each method. Although the “Tooth Dots Diagonal” method exhibited the best precision (coefficient of variation=5.4 percent), it also represented the least accurate method when compared to the other methods tested. Using “Small Dots Diagonal” on the gingiva appears to be the best option, exhibiting an acceptable coefficient of variation (17.6 percent) and a high degree of accuracy in terms of tolerance (mean±standard deviation=0.163±0.029). Based on the results of this study, further investigation of CAD CAM technology for the purpose of assessment and education of dental students is recommended. Dr. Callan is Associate Professor and Chair, Department of General Dentistry, College of Dental Medicine, Georgia Regents University; Dr. Blalock is Professor, Department of General Dentistry, College of Dental Medicine, Georgia Regents University; Dr. Cooper is Associate Professor, Department of General Dentistry, College of Dental Medicine, Georgia Regents University; Dr. Coleman is Assistant Professor and Assistant AEGD Program Director, Department of Oral Rehabilitation, College of Dental Medicine, Georgia Regents University; and Dr. Looney is Professor, Biostatistics, College of Graduate Studies, Georgia Regents University. Direct correspondence and reprint requests to Dr. Richard S. Callan, College of Dental Medicine, Georgia Regents University, Room GC 3080, 1430 John Wesley Gilbert Drive, Augusta, GA 30912-1290; 706-721-3881; [email protected]. Keywords: CAD CAM, e-learning, dental students, accuracy and precision, assessment, dental education, preclinical education Submitted for publication 11/29/12; accepted 4/1/13

T

his study is the first in a series designed to highlight the potential of using CAD CAM (Computer Aided Design, Computer Aided Manufacturing) technology as an adjunct to the clinical and preclinical education of dental students. In dentistry, CAD CAM technology was developed primarily for clinical application: the design and manufacturing of crowns, bridges, copings, and frameworks. The use of this technology for the purpose of educating dental students is a relatively new concept and has been embraced by only a few manufacturers. The technology and software used for this study (E4D Dentist and E4D Compare) were developed by D4D Technologies (Richardson, TX, USA). The hardware consists of a cart containing a wand (camera), a monitor and keyboard, and a computer. Desktop models (E4D Labworks, E4D Studio) are also available for non-intraoral procedures. Digital models are created through the “stitching together” of numerous images captured through lasers located at the end of the wand. The software allows for further manipulation of the image to align the model, identify the margin of the preparation, and design the restoration. Once the design of the restoration is completed, 40

it is sent to a mill for fabrication. The E4D Compare software uses the images from the design software to enable the individual to visually compare the two preparations and to obtain measurements that quantify various differences between the preparations. There has been an increase in reports on the use of simulation and computer-assisted simulation in the training of future dental health care providers.1-12 Technologies that were originally designed and implemented in other enterprises have been adapted for this purpose. The rationale for the use of simulation in the training of health care providers includes safety, effectiveness, efficiency, and economics.1-6 CAD CAM technology can enable students to visualize the difference(s) between their tooth preparations and an ideal tooth preparation and to quantify the differences. It can also be used to compare the student’s preparation to an unprepared tooth. The concept of subjective versus objective assessment immediately comes to mind, not to mention the model of self-guided learning. These topics are, and will continue to be, the center of numerous studies as newer and improved technologies are developed. When we consider the implementation of a new technology, technique, or material, it is imperative Journal of Dental Education  ■  Volume 78, Number 1

to determine its accuracy, precision, reliability, effectiveness, and feasibility.8,13-21 Accuracy refers to whether or not the technology gives the “correct” answer, to within a reasonable tolerance. Precision refers to degree of variability that is present in the results produced by the technology. Reliability refers to whether the technology can repeatedly produce the results one is expecting: that is, can the technology, technique, or material be depended upon to predictably produce the intended results? Effectiveness refers to the ability of the technology, technique, or material to produce the desired effect; that is, is it fulfilling its intended purpose? Feasibility refers to the barriers that prohibit the utilization of the technology, technique, or material, even if it is reliable and effective. Feasibility questions include: does the cost outweigh the benefit; is it difficult to acquire, maintain, or service; is it impractical; and will students and faculty utilize it? The purpose of this study was to investigate the accuracy and precision of CAD CAM technology. By comparing the accuracy of various methods of overlaying identical scans of the same tooth preparation, a determination of the appropriateness of this technology for the purpose of student preclinical assessment can be made. Our hypothesis was that the most accurate and precise alignment of models will be obtained using the method prescribed in the

manufacturer’s user manual, the Anatomical method (A). Subsequent studies are under way to address the issues of reliability, effectiveness, and feasibility.

Methods Utilizing a Kilgore dentiform (Kilgore International, Inc., Coldwater, MI, USA) made specifically for the Georgia Regents University School of Dental Medicine, tooth #30 was prepared by a faculty member to represent what our students are taught as the ideal preparation for a full gold crown. The same types of dentiform and teeth are utilized by dental students to prepare tooth #30 to receive a full gold crown. The dentiform and teeth adjacent to the tooth to be compared were modified to facilitate a comparison of alignment methods. Small dots were placed on both the dentiform and the adjacent teeth. Large dots were placed on the dentiform (Figure 1).

E4D Dentist Design Center The primary purpose of this study was to investigate the accuracy and precision of the various processes that can be used with E4D Compare for alignment of two models. Using the E4D Dentist, the ideal tooth preparation was scanned twice; the first scan was labeled “ideal prep,” and the second

Figure 1. Location of pinpoints and margin in the ideal preparation Anatomical landmark (A), Big Dots (B), Small Dots on Gingiva (C), Small Dots on Teeth (D), and location of Margin (E)

January 2014  ■  Journal of Dental Education

41

scan was labeled “ideal master.” No additional scans were made. In order to eliminate operator effects, the primary investigator was the only individual to do the scanning. Following the prescription dictated by E4D Dentist, a crown was selected as the restoration of choice, tooth #30 was indicated as the target tooth, the Library was “Clone,” the Material selected was “Bob (Acrylic),” and the shade was “Clear.” Once acquired, the model of the preparation was oriented, the area of interest was selected, and the margins of both preparations (ideal and master) were identified. When fabricating an indirect restoration, the actual margin of the preparation is identified prior to the design and fabrication of the restoration. In contrast, in order to be able to use the technology to evaluate the cavo-surface angle of a preparation, the margins of the ideal and master preparations were identified as the juncture of the tooth and the gingiva (Figure 1), not the actual margin of the preparation.

E4D Compare The E4D Compare toolbar progressively walks the operator through the requisite steps necessary for the comparison of models. Each step must be completed before the program will allow the operator to progress to the next. This is indicated by the white highlighting of that next step on the toolbar. While in the “Setup” mode, the models are imported into the program. The models are then aligned using the “Pinpoint” option. Once aligned, the “Margins” mode is made accessible and the margins are identified. The “Compare” mode can now be selected, which will enable a number of options by which models can be evaluated and compared. Digital impressions obtained using the E4D Dentist can be imported into the E4D Compare program simply by selecting them from the patient file in which they are stored. The “ideal tooth prep 30” was imported as the Sample, and the “ideal tooth master 30” was imported as the Master. Manual alignment of the two models is accomplished by pinpointing four corresponding spots on the two models. The position of the dots on the teeth was guided by the example illustrated in the E4D Compare User Manual. The positioning of the small dots on the gingiva facilitated the determination of the dependence of horizontal (SDH) vs. diagonal (SDD) dot placement on the accuracy of the overlay. The margins of both models were identified using E4D Dentist, so the “Compare” mode automatically became accessible. In order to compare the preparation teeth and not 42

the entire model, the “Trim Model” icon for both the Sample Model and the Master Model was selected. Selecting the “View Overlap” icon on the Sample screen created an image of the Master model (orange) overlaying the Sample model (yellow). Selecting the “View Overlap” icon on the Master screen created an image of the Sample model (orange) overlaying the Master model (yellow). To utilize the “Difference Map” (percent), one must first decide on a “Tolerance” (mm) by which to compare the two models. After indicating a desired tolerance, the “Show Gradient” icon is then selected. The “Show Gradient” feature gives a visual color differential and a percentage difference between what is underreduced (blue), overreduced (red), and ideal (green) of the Sample preparation when compared with the Master (Figure 2).

Comparing Sample to Master Six different methods of alignment of the models were tested: Anatomical (A) (Figure 3), Teeth Dots Diagonal (TDD) (Figure 4), Teeth Dots Occlusal (TDO) (Figure 5), Big Dots (BD) (Figure 6), Small Dots Horizontal (SDH) (Figure 7), and Small Dots Diagonal (SDD) (Figure 8). Each method was replicated twenty times. The tests consisted of adjusting the Tolerance levels (mm) until the Difference Map (percent) changed from 100 percent to 99.9 percent. For instance, if the Difference Map was 100 percent at 0.22 mm tolerance and 99.9 percent at 0.21 mm tolerance, the value recorded would be 0.21 mm (Figure 9).

Statistical Analysis The tolerance values for each method (in mm) were summarized using mean, standard deviation (SD), and coefficient of variation (CV). The CV, which is equal to the standard deviation divided by the mean, is the preferred summary statistic for comparing various methods in terms of precision (i.e., variability) when the mean values differ. The paired t-test was used to compare each method with all of the others in terms of mean tolerance. The likelihood ratio test21 was used to compare each method with all of the others in terms of the CV. The Shapiro-Wilk test was used to determine if the data were normally distributed; if violations of the normality assumption were found, rank-based methods were used instead. All statistical analyses were performed using NCSS 8 statistical software, 2012 (J. Hintze, NCSS LLC, Kaysville, UT, USA) and online software for statisti-

Journal of Dental Education  ■  Volume 78, Number 1

Figure 2. The Show Gradient feature Note: The Show Gradient feature gives a visual color differential and a percentage difference between what is underreduced (blue), overreduced (red), and ideal (green) of the Sample preparation when compared with the Master.

cal inference for the coefficient of variation (available at www1.fpl.fs.fed.us/covnorm.dcd.html). All statistical tests were performed using a significance level of 0.05. Because of the exploratory nature of this study, no adjustment was made for multiple testing. By applying the paired t-test to pilot data with n=5, we obtained estimates of the standard deviations (SDs) of the differences between each of the methods to be compared in this study, with the exception of Tooth Dots Occlusal. The Anatomical vs. Big Dots comparison had the largest estimated SD of the paired differences: 0.055. A power calculation based on this estimated SD indicated that a sample size of n=20 would yield 80 percent power to detect a difference between any two methods of 0.037 or larger using a significance level of 0.05. In the pretest, all of the pairwise comparisons between methods had a paired difference at least this large, with the ‒ exception of Anatomical vs. Big Dots (d = 0.008) and Small Dots Diagonal vs. Small Dots Horizontal January 2014  ■  Journal of Dental Education

‒ (d = 0.000). Thus, a sample size of twenty was used for our study.

Results Tolerance (mm) is summarized for each method in Table 1. The Shapiro-Wilk test indicated that the differences in tolerance between each pair of methods were all normally distributed (p>0.05 for all pairs). Thus, the paired t-test was used to compare the methods in terms of mean tolerance. The results of the paired t-test comparisons of each method with all of the others are shown in Table 2. In terms of mean tolerance, all methods were significantly different from all other methods, with the exception of the SDD and SDH methods. In order of decreasing mean tolerance (and hence increasing accuracy), the methods were arranged as follows: TDD > A > TDO > BD > SDD = SDH.

43

Figure 3. Anatomical (A) alignment Note: Alignment of the models is accomplished by placing four pinpoints (yellow, blue, red, and green) on corresponding locations on the two models. This screenshot illustrates the placement of a yellow pinpoint and a blue pinpoint on two Anatomical landmarks. Two additional pinpoints (red and green) are required to complete alignment.

Figure 4. Teeth Dots Diagonal (TDD) alignment Note: This screenshot illustrates the placement of a yellow pinpoint and a blue pinpoint on two small dots on the teeth. Two additional pinpoints (red and green) are placed on the teeth to complete alignment.

44

Journal of Dental Education  ■  Volume 78, Number 1

Figure 5. Teeth Dots Occlusal (TDO) alignment Note: This screenshot illustrates the placement of a yellow pinpoint, a blue pinpoint, a red pinpoint, and a green pinpoint on the occlusal surface of the two adjacent teeth to complete alignment.

Figure 6. Big Dots (BD) alignment Note: This screenshot illustrates the placement of a yellow pinpoint and a blue pinpoint on two Big Dots located on the gingiva. Two additional pinpoints (red and green) are placed on the lingual to complete alignment.

January 2014  ■  Journal of Dental Education

45

Figure 7. Small Dots Horizontal (SDH) alignment Note: This screenshot illustrates the placement of a yellow pinpoint and a blue pinpoint on two Small Dots located Horizontally on the gingiva. Two additional pinpoints (red and green) are placed on the lingual to complete alignment.

Figure 8. Small Dots Diagonal (SDD) alignment Note: This screenshot illustrates the placement of a yellow pinpoint and a blue pinpoint on two Small Dots located Diagonally on the gingiva. Two additional pinpoints (red and green) are placed on the lingual to complete alignment.

46

Journal of Dental Education  ■  Volume 78, Number 1

Figure 9. The Difference Map Note: The accuracy of the alignment of the two models is 99.9 percent with a tolerance of 0.21 mm.

When the six methods were compared in terms of CV (Table 1), there were no significant differences among the A, BD, and SDH methods. The SDD method has a slightly lower CV than the SDH method; as a result, SDD had a significantly lower CV than BD (p