Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

1164

Novelty and Retention for Two Augmented Reality Learning Systems Brady Patzer, Dustin C. Smith, & Joseph R. Keebler Wichita State University

Studies were conducted to measure novelty and learning retention while utilizing augmented reality (AR) in two learning systems. The first taught participants the basics of guitar and either a melody or scale using an AR guitar with an LED-embedded fret board. The guitar provided digital representations of learning patterns that users would otherwise need to visualize during the learning process. Results of three studies indicate that participants using the AR learning tool were able to perform more of the melody or scale after two-weeks. The second taught participants the basic functioning and anatomy of the heart, using either an AR model or a fiberglass model. Learning and technology acceptance were measured. Results indicated that the AR learning tool was as effective for participant learning when compared to the conventional fiberglass model learning tool. Furthermore, the AR learning tool was rated more enjoyable, curiosity inducing, and easier-to-use than the fiberglass model.

Copyright 2014 Human Factors and Ergonomics Society. DOI 10.1177/1541931214581243

INTRODUCTION Augmented reality (AR) can be defined as the seamless overlay of digital information on the real world environment, which can be visualized either directly or indirectly (Azuma, 1997; Ellis et al., 2012). AR has been increasingly integrated into learning systems (Radu, 2014). With the advent of consumer priced augmented reality head-mounted displays (e.g., Google Glass, Space Glasses) to the marketplace, AR’s integration into modern learning systems may continue to increase. The implementation of augmented reality into learning systems has the potential to provide several benefits. In fact, a meta-analysis conducted by Santos and colleagues (2013) examined whether augmented reality learning experiences are as effective as a complementary tool in educational settings. The authors found a moderate effect for the use of AR effecting learner performance (Santos et al., 2013). In a youth learning setting, grades five through seven, a study was conducted to compare the use of augmented reality versus no augmented reality for teaching the scientific concepts of magnets and magnetic fields. The study’s results indicated that the students who used the augmented reality spent more time interacting with the learning material (i.e., magnets) and displayed more frequent teamwork behaviors than those who did not use the augmented reality learning tool (Yoon & Wang, 2014). Although learning outcomes were not directly measured, the results suggest that the augmented reality afforded a better learning environment. The augmented reality learning tool held the student’s attention longer, and led to more interaction with their peers. However, there are several major challenges involved with the use of AR in systems. For example, in order to improve learning systems with AR, the digital

information presented needs to invoke the perceptions of the material that the system was designed to deliver. Moreover, augmented reality needs to be integrated into the system in a way that fits within the overall learning workflow. Therefore, emerging learning system designs have aimed to induce the appropriate learning material, while integrating seamlessly into the overall learning system. In fact, a recent review conducted by Radu (2014), was used to develop a heuristic questionnaire that aims to evaluate the educational potential of augmented reality learning applications. Radu recommends that an augmented reality application be evaluated on a scale from 1 (Strong Disagree) to 5 (Strong Agree) for the following five statements (Radu, 2014). First, “the application transforms the problem representation such that difficult concepts are easier to understand.” Second, “the application presents relevant educational information at the appropriate time and place, providing easy access to information and/or reducing extraneous learner tasks.” Third, “the application directs learner attention to important aspects of the educational experience.” Fourth, “the application enables learners to physically enact, or to feel physically immersed in, the educational concepts.” Finally, “the application permits students to interact with spatially challenging phenomena (Radu, 2014).” Although this tool still needs more rigorous evaluation, it may serve as a worthwhile heuristic for evaluating augmented reality learning tools. In this paper, we will discuss studies that have been conducted using two augmented reality learning systems. First, we will discuss an augmented reality guitar. Next, we will discuss an augmented reality anatomy training tool. Finally, we will conclude with an evaluation of the educational potential of these systems using Radu’s (2014) heuristic questionnaire.

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

1165

AUGMENTED REALITY GUITAR Learning to play an instrument can be a challenging experience. This may be due to the internalized (i.e., use of one’s knowledge skills and abilities) and externalized (i.e., environmental resources) modes of learning accentuated by studying an instrument (McDermott et al., 2013). That is, internalized modes of learning typically create more initial difficulties but foster greater long-term retention. Because these initial difficulties can be frustrating and hard to overcome, one attempting to learn an instrument may quit (McDermott et al., 2013). While the guitar is popular across the world, many people attempt to learn this instrument and quit after experiencing these initial difficulties. The Optek Fretlight® guitar is an emergent technology that integrates an augmented reality interface for learning the guitar. LEDs embedded in the fret board show the user where to place their finders for scales, or dynamic melodies (See Figure 1). We describe this particular technology as augmented reality because it digitally represents patterns that a user would otherwise need to visualize. As a result the process of learning to play the guitar becomes more externalized and more embodied. That is, users do not need to repeatedly glance at separate charts or diagrams (externalized modes of learning) to generate mental schemas (internalized modes of learning) of where to place their fingers. This enables increased interaction with a device that has embodied and embedded material for learning.

Figure 1. Optek Fretlight® guitar with augmented reality LED embedded fret board. Two studies have been conducted to compare the use of the Fretlight® augmented fret board and a traditional guitar with using tablature diagrams for learning and performing the A minor Pentatonic scale (See Figure 2; Keebler et al., 2013; Keebler, Wiltshire, et al., Under Review).

Figure 2. Fret board diagram of the A minor pentatonic scale. A more recent study was also conducted where learning and performing of the melody smoke on the water was compared across the Fretlight® augmented fret board and tablature diagram conditions (See Figure 3).

Figure 3. Tablature diagram for the “Smoke on the Water” riff. For all three experiments, undergraduate volunteers were randomly assigned to one of two conditions; either training using the augmented fret board or a traditional guitar using a tablature diagram. Participants learned the basics of guitar through a guided training. For the first two experiments, participants practiced the A minor pentatonic scale for 30 trials. Subsequently, note accuracy was measured and examined immediately after the training and for the second experiment two weeks later. In the third experiment, participants practiced the melody “Smoke on the Water” for 30 trials. Again note accuracy was measured during two measurement periods: immediately after and exactly two weeks later. There was a significant main effect across the training and two testing phases for both the riff experiment and the scale experiments. There was also a significant interaction effect (see Keebler, Wiltshire, et al., Under Review). That is, participants in the Fretlight condition were significantly better at playing the melody or scale after a two-week waiting period. With that, it appears that training an individual to play a scale or melody with an embedded learning tool can improve long-term retention. While a tool such as a tablature diagram has been well established in the music community, it is distributed. That is, the diagram is represented separately from the instrument. With these

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

distributed learning tools, users must first decipher the meaning of the diagram and then translate that meaning through space to their instrument. With an augmented reality guitar, the user immediately knows where to play. By decreasing the need to decipher and translate a coded pattern through space, augmented reality provides a learning experience that is more demand on actually learning the instrument. These studies describe an augmented reality technology that improves a person’s ability to visualize and retain a pattern. Next we will introduce an augmented reality that, through its novelty, engages users during training. To conclude, we will discuss future augmented reality research implications.

1166

within the human body would be enormous. Conversely, AR can offer a relatively cost-efficient and novel tool, allowing for a virtually unlimited storage of digital models and direct annotations to the augmented scene (See Figure 4 for an example of an annotated AR anatomy model). Consequently, augmented reality learning tools can offer an effective complement to the conventional anatomy training tool repertoire.

AUGMENTED REALITY ANATOMY Current research of augmented reality learning tools is progressively growing. Increasingly, augmented reality has been used within the medical setting to train and teach medical knowledge. Moreover, studies have begun to test the utility of AR as a supplemental tool in education. One such study was conducted to determine whether mobile augmented reality learning was comparable to textbook learning for medical students (Albrecht et al., 2013). The results indicated that those participants who were assigned to the augmented reality tool learned more than the participants that used the textbook. The authors suggest that a portion of this increase in learning may be attributed to the novelty of AR tools versus conventional learning tools, such as textbooks. However, the sample size of the Albrecht and colleagues study limits our ability to understand the effect of AR learning tools. Jan and colleagues (2012) proposed that augmented reality within the medical learning setting is important because it allows the user to “…immerse themselves in the [learning] scenario, being active learners and also becoming learning objects at the same time, interacting within a learning scene in the real world” (Jan et al., 2012). The use of augmented reality learning tools in the medical setting has the potential to mitigate ethical concerns shrouding some traditional methods. For example, cadavers, which have been used as an anatomy training tool for decades, have several problems (Winkelmann, 2007). Not only are there a variety of issues with the logistics of using cadavers (e.g., storage, ventilation, disposability), many students have ethical struggles with the use of cadavers to teach anatomy in medical school (Lempp, 2005). Supplemental learning tools exist, such as replica fiberglass anatomy models. However, the cost and storage space of the models required to replicate all of the anatomical diversity

Figure 4. Example of an annotated augmented reality heart model. A study was conducted to compare the use of digital AR models with directly labeled structural anatomy cues, AR models without direct structural anatomy labels, using a supplementary textual reference taken from an anatomy textbook, and physical fiberglass models without direct structural anatomy labels, using a supplementary anatomy reference sheet (see Keebler, Lazzara, et al., Under Review). Undergraduate volunteers were randomly assigned to be in one training condition (see Figure 5 for a summary of the training conditions used in this study); either training using a (1) labeled augmented reality heart, (2) unlabeled augmented reality heart with a textual reference, or an (3) unlabeled fiberglass model of the heart with a textual reference. Then, they were trained on the basic functional and structural anatomy of the heart. Training for the functional anatomy of the heart refers to the way in which the heart functions as an organ within the body. For example, participants were trained on the functional anatomy of the heart by explaining how the heart acts as a pump to propel blood from the heart’s chambers, through the major vessels, the lungs, and back throughout the body. Participants were trained on the structural anatomy of the heart by pointing out the major components of the heart during the training period.

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

1167

Table 1 Radu’s (2014) Heuristic Questionnaire Rating Question Item 1. The application transforms the problem representation such that difficult concepts are easier to understand.

Figure 5. Training conditions examined in this study An experimenter presented verbal anatomy training, lasting approximately fifteen to thirty minutes. As the training progressed, participants interacted with the training material by identifying the major structures of the heart when prompted by the experimenter. The heart training and declarative knowledge metrics were developed in collaboration with a subject matter expert (i.e., an anatomy professor) at the University of Central Florida. Declarative knowledge of the heart was assessed prior to training (i.e., the baseline) and after the training (i.e., post-training). Learning was defined as the change in declarative knowledge test scores between the baseline and post-training scores. Learning scores were calculated by subtracting the post-training scores from the baseline scores. Participants also took a survey to measure their technology acceptance after training. Results suggest that learning the anatomy of the heart with an augmented reality training tool presents a more enjoyable, curiosity inducing, and easier to use learning tool than the fiberglass model. Furthermore, the addition of labels directly onto the AR model significantly boosted learning when compared to an unlabeled AR model. EDUCATIONAL POTENTIAL Using the heuristic questionnaire proposed by Radu (2014), the authors evaluated these existing learning systems. See Table 1 for the author’s ratings of the learning systems discussed. From these ratings, it appears that both systems have a relatively high educational potential. However, the AR guitar learning system has a higher total rating. Future research on AR systems should allow participants to evaluate the learning system’s educational potential using Radu’s questionnaire.

2. The application presents relevant educational information at the appropriate time and place, providing easy access to information and/or reducing extraneous learner tasks. 3. The application directs learner attention to important aspects of the educational experience. 4. The application enables learners to physically enact, or to feel physically immersed in, the educational concepts. 5. The application permits students to interact with spatially challenging phenomena. Total Rating

AR Guitar

AR Heart

5 (Strongly Agree)

3 (Neither Agree nor Disagree)

5 (Strongly Agree)

5 (Strongly Agree)

5 (Strongly Agree)

4 (Agree)

5 (Strongly Agree)

3 (Neither Agree nor Disagree)

5 (Strongly Agree)

5 (Strongly Agree)

25

20

CONCLUSION Thus far, studies conducted by the authors on augmented reality learning systems have suggested that, when designed with the aforementioned challenges in mind, augmented reality tools represent an intuitive and motivating approach to learning. While additional inhouse studies will be conducted exploring the benefits of augmented reality, there are numerous researchers examining the same trends (Biocca et al., 2007; Furmanski et al., 2002; Kim & Dey, 2009; Katz et al., 2012; Klatzky et al., 2008; Nagy, 2011; Schall et al., 2013; Slijepcevic, 2013; Tang et al., 2003). Therefore, the benefits of augmented reality applications will continue to emerge as these lines of research grow.

Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting - 2014

REFERENCES Albrecht, U., Folta-Schoofs, K., Behrends, M., & Jan, U.V. (2013). Effects of mobile augmented reality learning compared to textbook learning on medical students: Randomized controlled pilot study. Journal of Medical Internet Research, 15(8), e182. Azuma, R.T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355-385. Biocca, F., Owen, C., Tang, A., & Bohil, C. (2007). Attention issues in spatial information systems: Directing mobile users’ visual attention using augmented reality. Journal of Management Information Systems, 23(4), 163-184. Ellis, S.R., Havig, P., Hale, K.S., & Hollands, J.G. (2012). Augmented reality: Implications towards virtual reality, human perception and performance. Proceedings of the Human Factors and Ergonomics Society 56th annual meeting. Boston, MA. Furmanski, C., Azuma, R., & Daily, M. (2002). Augmented-reality visualizations guided by cognition: Perceptual heuristics for combining visible and obscured information. Proceedings of the International Symposium on Mixed and Augmented Reality ISMAR ’02. Jan, U.V., Noll, C., Behrends, M., & Albrecht, U. (2012). mARble: Augmented reality in medical education. Biomedical Technology, 57, 67-70. Katz, B.F.G., Dramas, F., Parseihian, G., … Jouffrais, C. (2012). NAVIG: Guidance system for the visually impaired using virtual augmented reality. Technology and Disability, 24, 163-178. Keebler, J.R., Wiltshire, T.J., Smith, D.C., & Fiore, S.M. (2013). Picking up STEAM: Educational implication for teaching with an augmented reality guitar learning system. Proceedings of the International Conference of Human Computer Interaction. Las Vegas, NV. Keebler, J.R., Wiltshire, T.J., Smith, D.C., Fiore, S.M., & Bedwell, J.S. (Under review). Shifting the paradigm of music instruction: Implications of embodiment stemming from an augmented reality guitar learning system. Submitted to Frontiers. Keebler, J.R., Lazzara, E. H., Patzer, B., Benishek, L., & Salas, E. (Under Review). Learning the heart: Effects of an augmented reality simulation-based anatomy training. Submitted to Human Factors. Kim, S.J., & Dey, A.K. (2009). Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 133-142). USA.

1168

Klatzky, R.L., Wu, B., Shelton, D., & Stetten, G. (2008). Effectiveness of augmented reality visualization versus cognitive mediation for learning actions in near space. ACM Transactions on Applied Perception (TAP), 5(1) 1-23. Lempp, H. K. (2005). Perceptions of dissection by students in one medical school: Beyond learning about anatomy: A qualitative study. Medical Education, 39, 318–325. doi: 10.1111/j.13652929.2005.02095.x McDermott, J., Gifford, T., Bouwer, A., & Wagy, M. (2013). Should music interaction be easy? Music and Human-Computer Interaction (pp. 29-47). London: Springer. Nagy, I.K. (2011). Cognitive aspects of augmented reality applications. Proceedings of the 2nd International Conference on Cognitive Information Communications (pp. 1-3). Radu, I. (2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing, 1-11. Santos, M. E.C., Chen, A., Taketomi, T., Yamamoto, G., Miyazaki, J., & Kato, H. (2013). Augmented reality learning experiences: Survey of prototype design and evaluation. IEEE Transactions on Learning Technologies. IEEE Computer Society. Schall, M.C., Rusch, M.L., Lee, J.D., Dawson, J.D., Thomas, G., Aksan, N., & Rizzo, M. (2013). Augmented reality cues and elderly driver hazard perception. Human Factors: The Journal of the Human Factors and Ergonomics Society, 55(3), 643-658. Slijepcevic, N. (2013). The effect of augmented reality treatment on learning, cognitive load, and spatial visualization abilities. (Doctoral dissertation). Retrieved from http://uknowledge.uky.edu/cgi/viewcontent.cgi?artic ar=1003&context=edc_etds Tang, A., Owens, C., Biocca, F., & Mou, W. (2003). Comparative effectiveness of augmented reality in object assembly. New Techniques for Presenting Instructions and Transcripts, 5(1), 73-80. Winkelmann, A. (2007). Anatomical dissection as a teaching method in medical school: A review of the evidence. Medical Education, 41, 15–22. Yoon, S. A., & Wang, J. (2014). Making the invisible visible in science museums through augmented reality devices. Tech Trends: Linking Research & Practice to Improve Learning, 58(1), 49-55.