Mobile Learning for the NHS: Research Report

Dr Naomi Norman 11 March 2011

1

Contents Acknowledgements ............................................................................................. 4 Executive summary ............................................................................................. 5 Foreword: comments from Dr Chris Davies and Professor John Traxler ................ 6 1. Background ................................................................................................... 7 2. Research aims and objectives ........................................................................ 8 2.1 The aims.................................................................................................................. 8 2.2 The objectives ......................................................................................................... 8

3. Research methodology .................................................................................. 9 3.1 Data collection ........................................................................................................ 9 3.1.1 Desk research ......................................................................................................................... 9 3.1.2 Surveys ................................................................................................................................. 10 3.1.3 Focus groups ........................................................................................................................ 12 3.1.4 Telephone interviews........................................................................................................... 13

3.2 Data analysis......................................................................................................... 14

4. Findings....................................................................................................... 15 4.1 Introduction to findings ......................................................................................... 15 4.2 Key findings........................................................................................................... 19 4.2.1 Benefits of mobile learning .................................................................................................. 19 4.2.2 Challenges of mobile learning .............................................................................................. 23 4.2.3 How and where mobile learning can contribute to training and assessment ..................... 28 4.2.4 How and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in Statutory and Mandatory Training Framework .................... 31 4.2.5 The kinds of mobile devices available to NHS South Central staff ...................................... 33 4.2.6 The integration of mobile devices with existing NHS IT infrastructures.............................. 35 2

5. Recommendations in relation to project objectives ..................................... 37 5.1 Mobile learning recommendations......................................................................................... 37 5.2 Mobile technology recommendations .................................................................................... 41

Final comments by Professor John Traxler, Professor of Mobile Learning........... 43 Appendix A: Research timetable ........................................................................ 44 Appendix B: Findings from desk research ........................................................... 47 Appendix C: Survey ............................................................................................ 59 Appendix D: Survey communications ................................................................. 72 Appendix E: Focus group plan ............................................................................ 76 Appendix F: Telephone interview schedules ...................................................... 79 Appendix G: Mobile projects ............................................................................. 83

3

Acknowledgements The author of this research report would like to thank the following people: Alison Potter and Helen Bingham of NHS South Central Strategic Health Authority for the opportunity to undertake this important and interesting piece of work. The many research participants (of which there were over 170), all who generously gave their time to offer their thoughts and opinions on mobile learning and assessment, be it through written questionnaire, attending a focus group, or telephone interview. Berkshire Healthcare Foundation Trust, Buckinghamshire Healthcare Trust and South Central Ambulance Service, for their co-operation in helping find research participants. Dr Chris Davies and Professor John Traxler for their thorough peer review and insights on the study. Marcus Boyes, Head of Mobile Learning at Epic, for his professional contribution.

4

Executive summary NHS South Central Strategic Health Authority commissioned Epic to undertake a research study to seek to understand the potential benefits of mobile learning to meet the needs and preferences of the healthcare workforce. In particular, they were interested in where mobile learning could contribute to the delivery of statutory and mandatory training and assessment. They also wanted to understand the challenges of mobile learning, and how they may be overcome, including the feasibility of implementing it within current NHS IT infrastructures. The aim was to form recommendations and actions that would enable the NHS to take the necessary steps to being ‘m-learning ready’, both now and in the future. This report is the final output from the study, which took place over four months from November 2010 to March 2011. It describes the research objectives, methodology, findings and subsequent recommendations, and is supported by appendices that include the data collection tools and background descriptions of papers that emerged from a literature review. More than 170 NHS staff participated in the research, including 130 of 270 randomly sampled NHS practitioner and managerial staff, from across three Trusts and six distinct job roles, who were invited to complete an online survey. The response rate to the online survey (at 48.1%) was much higher than anticipated, perhaps indicating a keen interest in mobile learning. Other research participants shared their views through two large focus groups involving National and Strategic Health Authority E-Learning Leads; or through telephone interviews, which were held with key stakeholders, technical experts, and NHS staff who had been involved in the few existing or previous small-scale mobile projects. In addition, desk research resulted in twelve of the most relevant journal papers forming a literature review. The findings set out 27 different benefits to mobile learning for the NHS workforce. Among them were some obvious ones, such as convenience and flexibility, but also some less obvious ones, such as the immediacy of feedback leading to speedier remediation, and the ability to frequently reference material supporting decision making, both of which have the potential to contribute to improved patient care. The study also identified ways in which mobile learning may contribute to training and assessment, in particular for statutory and mandatory skills. The findings also set out 22 different challenges, ranging from that of screen size to the potential for dishonesty if individuals are undertaking assessments using a mobile device and the possible problems of interfacing with NHS IT systems. All these challenges are addressed in the final recommendations. The research was peer reviewed by Dr Chris Davies, Head of the E-learning Research Group at Oxford University, and Professor John Traxler, the UK’s only Professor of Mobile Learning, to ensure quality and rigour. Professor Traxler acknowledged the ‘enormous possibilities but complex challenges’ presented by mobile learning, and expressed confidence that the findings and recommendations of this research study ‘take us considerably further than before within their particular professional and organisational context’. 5

Foreword: comments from Dr Chris Davies and Professor John Traxler Comments on the overall study and its methodology Dr Chris Davies, University of Oxford Department of Education I think this could prove to be an important study – there is no question in my mind that this is an issue which is finally of real relevance, because of the massive increase in the usability and uptake of mobile devices such as smart phones and new generation tablets in the last few years. Overall, the research plan seemed well developed, with excellent coverage of logistical issues. The data analysis seemed well formulated and appropriate too. Initially, it was not clear how staff would respond to questions about mobile learning possibilities without a fairly clear exposition of what this means and how it might work, but a good job was done of exemplifying the e-learning possibilities through the use of illustrations, which were excellent and helpful. It all seemed to me to be very carefully and well thought through, with my comments offered as thoughts for possible refinement, rather than signifying any problems with the research methodology.

Comments on the overall study and its findings and recommendations Professor John Traxler, University of Wolverhampton I have read this report with considerable interest. I think it is a very thorough, focussed and comprehensive review and a model of its kind. It captures, analyses and contextualises a wealth of relevant experiences and outcomes from across the current pilots, programmes and initiatives being undertaken in mobile learning. I think the report does an impressive job in both its findings and recommendations in exposing the complexity of large-scale mobile learning; the findings are understandably complex and sometimes counter-intuitive but wholly in line with what in general terms I would anticipate.

6

1. Background NHS South Central Strategic Health Authority has developed a Statutory and Mandatory Training Framework. It defines learning outcomes and refresher intervals in nine essential skills areas, which have to be covered by all staff.1 These refresher intervals have been incorporated into Electronic Staff Records (ESRs), so that if staff move across Trusts their training requirements move with them. Other Strategic Health Authorities have similar frameworks and training in place. NHS South Central SHA is now focussing on creating online assessments for staff to undertake before their refresher training. There is a desire to ensure its quality and consistency, and to be able to deliver it to those who need it, when they need it, in flexible and innovative ways, in line with local workplace needs. This is also true of other training and assessment. E-learning offers a cost-effective solution, especially given the size of the NHS South Central workforce. However, there are many NHS staff working in community settings, away from desktop computers, as well as staff that work in single healthcare settings whose jobs require them to be constantly on the move, for example from ward to ward in a hospital. For these staff, mobile learning, i.e. accessible through a mobile device, may be the answer. Therefore, at the end of October 2010, NHS South Central SHA commissioned Epic to undertake a research study to provide them with sound research-based and empirical evidence to guide next steps and inform any investment and development decisions for mobile learning. This report is the culmination of that four-month research study and details the objectives, the methodology, the key findings that emerged and recommendations based on those findings. (A timetable for the research can be found in Appendix A.) It is hoped that this work will be relevant to other Strategic Health Authorities too, and to NHS England nationally, especially in the context of the Department of Health’s proposed new Technology Enabled Learning Strategy. It may also be of interest to other large organisations seeking to put in place a mobile learning strategy, or to get the most from mobile devices for training and assessment.

1

The nine areas are: Conflict Resolution; Equality, Diversity and Human Rights; Fire Safety; Health and Safety; Infection Prevention and Control; Moving and Handling; Resuscitation; Safeguarding Children; Safeguarding Vulnerable Adults

7

2. Research aims and objectives 2.1 The aims The research study investigated three key areas: 1. Mobile learning and success for the learner 2. Mobile learning and technology 3. Mobile learning readiness for the organisation

2.2 The objectives This led to four research objectives to meet aims 1. and 2. above, and two project objectives to meet aim 3., as follows:

Mobile learning and success for the learner Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework

Mobile learning and technology Research Objective 3: Establish the kinds of mobile devices currently available to NHS South Central staff Research Objective 4: Determine how mobile learning can integrate with existing NHS IT infrastructures

Mobile learning readiness for the organisation Project Objective A: Make mobile learning recommendations based on the outcomes of research objectives 1 and 2 Project Objective B: Make mobile technology recommendations based on the outcomes of research objectives 3 and 4

Note: For the purposes of this research study, mobile learning and assessment is defined as any training done on a device that is handheld, and so is portable and can be easily carried for long periods of time while undertaking work. For some that may mean a mobile device that is carried in a pocket so that it is available at any time, for example a mobile phone, but when not in use can be quickly and easily put away, leaving both hands free to undertake healthcare work. For others, a tablet PC or an electronic notebook (i.e. a small laptop computer) may suffice. 8

3. Research methodology The methodology employed both quantitative and qualitative data collection, incorporating desk research, surveys, focus groups and telephone interviews, to meet research objectives 1 to 4. Then, the analysis of the data resulted in key findings that led to recommendations to meet project objectives 5 and 6.

3.1 Data collection 3.1.1 Desk research To meet research objectives 1 and 2: Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework The desk research sought to identify the most relevant journal papers, using a range of databases to ensure the broadest possible coverage. 2 Search terms included, for example, ‘mobile learning’, ‘mobile assessment’, ‘mobile education’ and ‘mobile devices’ for medical databases; and ‘mobile learning health’ and ‘mobile assessment healthcare’ for education databases. A large number of the references that resulted from the searches offered examples of mobile technology for clinical use, or the management of particular conditions. For instance, the use of mobile phones for skin tumour screening3, or designing mobile support for glycemic control in patients with diabetes4. Other references explored the potential health risks to mobile phone use, such as biological responses to mobile phone radiation exposure5. Then there were references that used the term ‘mobile’ to refer to the mobility of people rather than to mobile technology. All such references were rejected for their lack of applicability, or only partial relevance, to the research objectives the desk research set out to meet. The remaining references were added to a list of potentially relevant papers supplied to this project by Professor John Traxler, Professor of Mobile Learning. Then, a cut-off date was applied, because 2

For medical references, some examples of databases used were: BioMed Central Journals, Medline, PubMed, Web of Science and SCOPUS; for education references, some examples of databases used were: ASSIA, British Education Index, ERIC, JSTOR and SAGE Journals Online. 3 Massone, C., Hofmann-Wellenhof, R., Ahlgrimm-Siess, V. et al (2007) Melanoma Screening with Cellular Phones, [Online], Available: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0000483 [13 Jan 2011] 4 Harris, L. T., Tufano, J., Le, T. et al (2010) 'Designing mobile support for glycemic control in patients with diabetes', Journal of Biomedical Informatics, vol. 43, pp.37 – 40. 5 Karinen, A., Heinavaara, S., Nylund, R. and Leszczynski, D. (2008) Mobile phone radiation might alter protein expression in human skin, [Online], Available: http://www.biomedcentral.com/1471-2164/9/77 [13 Jan 2011]

9

where references were dated prior to 2008, many of the challenges the papers described had been overcome by technological developments or the wider proliferation of mobile devices. A list of the resulting twelve papers for review, along with relevant commentary that contributes to the research findings, can be found in Appendix B of this research report.

3.1.2 Surveys To meet research objectives 1, 2 and 3: Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework Research Objective 3: Establish the kinds of mobile devices available to NHS South Central staff Two hundred and seventy of the NHS South Central SHA workforce were randomly sampled from six specific NHS job roles defined by technology remoteness (that is, two roles where the jobs involved working with no access to desktop computers; two roles where the jobs involved some access; and two roles where the jobs involved continuous access). This sampling ensured a representative view across those who may or may not have the most to gain from mobile learning. Data also was triangulated across three (diverse geographically and demographically) Primary Care Trusts (specifically, Berkshire Healthcare Foundation Trust, Buckinghamshire Healthcare Trust and South Central Ambulance Service), and across seniority of role, thus involving both practitioners and managers. Therefore, the randomly sampled survey participants can be classified as follows: 1. 30 staff (20 practitioners and 10 managers) from across two specific job roles that involve working remotely, with no access to desktop computers, to include: a. 10 community nurses and 5 managers of community nurses b. 10 ambulance workers and 5 managers of ambulance workers 2. 30 staff (20 practitioners and 10 managers) from across two specific job roles that have some access to technology, but not necessarily always connected, to include: a. 10 junior doctors and 5 managers of junior doctors b. 10 community psychiatric nurses and 5 managers of community psychiatric nurses 3. 30 staff (20 practitioners and 10 managers) from across two specific job roles that have constant access to technology, to include: a. 10 administrators and 5 managers of administrators b. 10 radiographers and 5 managers of radiographers

10

In summary: Across 3 Trusts: Practitioners Managers Totals

Working remotely, no access 60 30 90

Some access 60 30 90

Always connected 60 30 90

Totals 180 90 270

The anticipated response rate was approximately 20%, resulting in 54 responses. However, the actual response rate was significantly higher at 48.1%, as follows: Across 3 Trusts: Practitioners Managers Totals

Working remotely, no access 24 18 42

Some access 24 6 30

Always connected 19 7 26

Other

Totals

18 14 34

85 45 130

Note: There were 118 respondents who completed every question. That amounts to a response rate for completed surveys of 43.7%. Others answered only some questions. The success of this response rate was deemed to be due to a ‘marketing campaign’ put in place to encourage participants to complete the survey. That entailed four emails sent by a senior person in each of the three Trusts. These emails were as follows: Email 1 trailed the survey, highlighting its significance and informing participants that it was on the way. It also encouraged them to immediately block out 20 minutes in their diaries before 20th December 2010 to complete the survey. Email 2 included the online link to the survey and confirmation that the survey would be anonymous, thus the inclusion of a research participant number. (This number enabled participant names to remain unknown to the researcher undertaking data analysis and the writing of this report.) Email 3 included a ‘countdown’ prompt for those participants who had yet to complete the survey, reminding them they only had a few days left to do so. Email 4 recognised that participants may have been busy in the run-up to Christmas and was sent out at the beginning of 2011, extending the deadline for completion of the survey to the end of the first week of January. Those staff working remotely with no access to desktop computers (and so unlikely to receive email) were also sent a letter and a printed survey with pre-addressed return envelope. In addition, any staff randomly sampled whose email addresses could not be confirmed (for example, where the email bounced), were also sent paper communications. Surveys included compulsory multiple-choice and Likert-scale questions with options to support answers with a written justification if desired. Questions sought to understand, for example, what 11

had worked well, or not so well, for staff training and assessment in the past; learning preferences; mobile devices available to staff; and the potential acceptance for mobile learning, including assessment, especially for the Statutory and Mandatory training in the nine essential skill areas. Manager and staff surveys were mostly the same, but inevitably some questions were unique to practice or managerial responsibilities. Survey questions, along with the email communications to encourage participation, can be found in Appendices C and D of this document respectively. These were peer reviewed by Dr Chris Davies.

3.1.3 Focus groups To meet research objectives 1 and 2: Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework Two focus groups were held. The first one, held on 15 December 2010, was a 40-minute focus group with 14 National and Strategic Health Authority E-learning Leads and some additional stakeholders, including two staff leading the Skills for Health programme, an Electronic Staff Records manager and the E-learning for Healthcare Programme Director. The second one, held on 14 January 2011, was a one-hour focus group with 13 NHS South Central SHA Trust E-learning Leads. These staff are responsible for learning and development, so e-learning and mobile learning sits under their remit. Each focus group session explored beliefs regarding the benefits and challenges to mobile learning for the NHS workforce. A little time was also spent looking at some existing mobile content to stimulate creative thinking, then identifying if and where mobile learning may play a part in the training and assessment for some of the nine essential statutory and mandatory skills areas. During focus groups, questions were posed and small groups given time to discuss their responses, before feeding back to the whole group. A detailed focus group plan can be found in Appendix E of this document. This plan was peer reviewed Dr Chris Davies.

12

3.1.4 Telephone interviews To meet research objectives 1, 2 and 3: Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework Research Objective 3: Establish the kinds of mobile devices available to NHS South Central staff 30-minute telephone interviews were conducted with one healthcare professional from each of four different existing or completed projects in NHS South Central where staff are making use of mobile devices for clinical or information purposes. The four were identified by Alison Potter, E-learning Programme Manager for South Central Strategic Health Authority, in consultation with some Elearning Leads. During telephone interviews, participants were asked to describe the project they had been involved with, what had worked well, and not so well, and what kind of mobile technology was employed. They were also asked about lessons learnt that may apply to training and assessment for the nine essential skill areas in the Statutory and Mandatory Training Framework. 40-minute telephone interviews were conducted with five key National Leads who would be potential stakeholders in taking forward recommendations that emerge from this research study. Job titles included: ‘Associate Director of Workforce and Education’, ‘Head of Educational Development’, ‘National Programme Director for the Department of Health E-learning’, ‘Senior Clinical Adviser, Department of Health’ and ‘Knowledge Services Programme Manager’. These interviews sought to uncover the benefits and challenges to mobile learning in the NHS from a senior management high-level perspective. To meet research objective 4: Research Objective 4: Determine how mobile learning can integrate with existing NHS IT infrastructures 40-minute telephone interviews were conducted with five key technical staff, including those with national roles (for example, from the central ESR National LMS team, or a national e-learning content provider), those with regional roles (for example, the SHA Chief Information Officer) and those with local Trust roles (for example, Trust Heads of IT). During these interviews, technical questioning sought to uncover a better understanding of the interoperability of mobile devices with the NHS’s National Learning Management Systems and the possibilities for tracking learner activity using mobile devices.

13

All telephone interviews were semi-structured. Therefore, while the sequence and wording of questions were determined in advance, the interviewer had the freedom to delve deeper where responses required further clarification. For example, where an interviewee described an experience as ‘good’ the interviewer asked ‘what, in particular, made it good?’, or an interviewer sought more information from a technical response in order to gain a fuller understanding of the NHS’s technical architecture. As much of the data as possible was transcribed during the interview, although responses were also tape recorded as back-up to enable later transcription as required, for example where an interviewee spoke quickly. All interview schedules, along with interviewer notes, can be found in Appendix F of this document. These were peer reviewed by Dr Chris Davies.

3.2 Data analysis To meet project objectives A and B: Project Objective A: Make mobile learning recommendations based on the outcomes of research objectives 1 and 2 Project Objective B: Make mobile technology recommendations based on the outcomes of research objectives 3 and 4 Quantitative data collected through surveys (described in 3.1.2) offered frequencies and central tendencies. Qualitative data, some collected through surveys, but largely collected through focus groups and telephone interviews, were coded according to common motifs, and then preliminary themes were drawn from the data. Finally, supporting quantitative data, and relevant references that emerged from the desk research, were inserted under the appropriate themes to bring all the data together. Themes that emerged are presented in this report as key findings (see Section 4: Findings), that led to the recommendations (see Section 5). All findings and recommendations have been peer reviewed by Professor John Traxler.

14

4. Findings 4.1 Introduction to findings Key findings are represented as themes that emerged independently from more than one data source, for example, from a desk research paper and a telephone interview, or from more than one survey respondents’ comments, or from both focus groups. They are described in Section 4.2 of this research study, beneath relevant objectives 1-4. Each finding is numbered and then coded to indicate the data collection tool used, as follows:

Desk research findings are coded as follows: *DR for desk research+ *Paper’s reference number+.*Paper’s key finding number+, e.g. DR1.3 Note: See Appendix B for the outcome of the desk research in its entirety. Papers are listed in alphabetical order and then assigned a paper reference number. Each paper reference is accompanied by a summary of the project it describes and key findings of relevance to this research.

Survey findings are coded as follows: [S for survey] [P for practitioner], i.e. SP [S for survey] [M for manager], i.e. SM Note: There were 130 survey respondents (85 practitioners and 45 managers), with a reasonable spread across Trusts and job roles: Job role

Berks

Community nurse Community nurse manager Ambulance worker Ambulance worker manager Junior doctor Junior doctor manager Community psychiatric nurse Community psychiatric nurse manager Administrator Administrator manager Radiographer Radiographer manager Other (practising)

2 2 1 0 14 3 9 3 3 1 0 1 12

Trusts Bucks S. Cen. Amb. 4 0 5 0 1 16 0 11 0 0 0 0 0 1 0 0 0 2 0 1 14 0 4 0 3 3

Other (manager)

8

2

4

14

Totals

59

33

38

130

Totals 6 7 18 11 14 3 10 3 5 2 14 5 18

24 practitioners working remotely, with no access to desktop computers; 18 managing these remote workers. 24 practitioners working with some access to technology, and so only sometimes connected; 6 managing these sometimesconnected workers 19 practitioners working with constant access to technology; 7 managing these constantly connected workers 4 practitioners working remotely; 12 practitioners sometimes connected; 2 practitioners constantly connected. 2 managing those working remotely; 8 managing those sometimes connected; 4 managing those constantly connected.

Those who selected ‘Other’ worked in a variety of roles, most popularly as consultant psychiatrists (9 survey respondents) and health visitors (3 survey respondents). Then, there were also one-off roles mentioned, such as multi-skilled control assistant or project manager. 15

116 surveys were submitted online and 14 were returned as print-outs completed in pen. Paperbased surveys came from across the three Trusts and were spread across job roles. The majority (9) came from practitioners and their managers working in settings with some access to technology, so presumably with a choice of how to engage with the survey: either online or with pen and paper. Then, two came from those working remotely, with no access to desktop computers, and one came from a manager of practitioners working with constant access to technology. 118 of the total 130 survey respondents completed all questions in the survey (77 practitioners and 41 managers). The remaining 12 respondents abandoned the survey at various stages.

16

The statistical diagrams below offer background on survey respondents. 125 survey respondents in total answered these questions: 81 practitioners and 44 managers. Survey respondents’ ages (practitioners)

Survey respondents’ ages (managers)

Survey respondents’ gender (practitioners)

Survey respondents’ gender (managers)

17

Survey respondents’ regular use of a computer/laptop (practitioners)

Survey respondents’ regular of use of a computer/laptop (managers)

Survey respondents’ membership of online communities (practitioners)

Survey respondents’ membership of online communities (managers)

Of the 41 practitioners who answered ‘Yes’, 39 used Facebook (2 also used Twitter and 1 also used LinkedIn). One survey respondent was only on MSN and another would not divulge their usage.

Of the 17 managers who answered ‘Yes’, 15 used Facebook (1 also used Twitter and 1 also used MSN). One survey respondent was on LinkedIn only and another would not divulge their usage. 18

Most survey respondents were reasonably regular users of computers or laptops (using them twice or more per week). This, along with access to online communities, was not generally dependent on age, gender or job role. Nonetheless, the data did reveal that there may remain a very small percentage of NHS workers for whom technology may not be a regular part of their lives. In particular, there were 3 survey respondents (2.4% of the 125 survey respondents) who stated that they used a computer or laptop less often than monthly or never at all: an aged 44-55 female radiographer, an over-55 female radiographer manager and an over-55 community psychiatric nurse manager. This perhaps suggests older female workers, in general, may be less technologically savvy.

Focus group findings are coded as follows: [FG for focus group], i.e. FG

Telephone interviews are coded as follows: [T for telephone interview] [MP for mobile project], i.e. TMP [T for telephone interview] [S for Stakeholder], i.e. TS [T for telephone interview] [T for Technical], i.e. TT Note: See Appendix G for descriptions of mobile project from TMP and TS.

4.2 Key findings Research Objective 1: Identify the benefits of mobile learning for NHS South Central staff, along with any challenges Sections 4.2.1 and 4.2.2 below address research objective 1

4.2.1 Benefits of mobile learning 4.2.1.1 Convenience (learning location): Of 80 practitioner survey respondents, 75.1% believed that a mobile device for learning would make it more convenient for them to undertake training, because they would not have to leave the workplace.6 80.5% (of the 44 manager survey respondents) shared this opinion.7 [SP/SM] Typical supporting comments were: ‘The potential is endless in terms of convenience’ [TS], ‘No need to go elsewhere for learning events’ [FG] and ‘It will avoid having to leave a clinical area to attend training.’ [TS] The desk research revealed similar findings in other projects [DR5.1; DR7.1]. For example, where student midwives were using iPods to access small video files to support them in newborn infant physical examinations, they spoke of the convenience of having the resources available to them in clinical areas, or indeed on their way to work, on a train or a bus. [DR2.1] 4.2.1.2 Convenience (timely learning): Mobile learning was described by many as ‘just-in-time learning’ or ‘right time learning’. [FG] Supporting comments included: ‘There's a real opportunity for access to knowledge or learning on a time-needed basis’ and ‘Mobile devices offer good flexibility/access to learning at times that are most convenient.’ [SM] The desk research also highlighted medical students benefiting

6 7

17.5% strongly agreed; 38.8% agreed and 18.8% slightly agreed. 19.5% strongly agreed; 51.2% agreed and 9.8% slightly agreed.

19

from the ‘more timely and relevant feedback’ they received from tutors when using PDA mobile devices for communication. [DR3.5] 4.2.1.3 Reassurance: There was evidence that the timely access to information on a mobile device offered clinicians extra reassurance in their professional judgements. [DR2.5; DR7.8] 4.2.1.4 Flexibility: The term ‘flexibility’ arose on a number of occasions. Typical written comments were ‘Flexible – timely’; ‘Flexibility – quick and easy to use’ and ‘Mobile workforce needs access to learning in a flexible way’. [FG] In addition, ‘flexibility’ was mentioned in the desk research, where 'the flexibility of where the learning could take place’ was credited with enhancing student midwives’ acquisition of skills. [DR2.3] 4.2.1.5 Relevance: Mobile learning enables training to be ‘situated rather than simulated’ with the learning ‘at the point of need’. [FG] 77.6% (of 80) practitioner survey respondents agreed, believing that a mobile device for learning would make training more relevant, as it could take place in the work setting where the learning is put into practice.8 82.9% (of the 44 manager survey respondents) shared this view too.9 [SP/SM] A literature search of mobile learning for remote work environments also identified mobile devices as a means to offer access to relevant instruction or knowledge. [DR6.1] 4.2.1.6 Control of learning: The use of mobile devices allows learners to control the pace at which they learn. [FG] A supporting comment for the use of a mobile device for learning by reading (words and pictures) was: ‘I always prefer learning by reading because I control the speed of it.’ [SP] There was evidence from the desk research too that mobile devices ‘empowered’ learners [DR2.2] and encouraged them to take the initiative to direct their own learning activities. [DR7.9] 4.2.1.7 Quick and easy access to information: Both the phrase ‘quick and easy’ and the individual terms ‘quick’ and ‘easy’ were used a number of times with regard to access to information, for example, ‘Easy for fact checking, e.g. dosages’ and ‘Access to quick medical information straight away’. [FG] Furthermore, HIV/AIDS healthcare workers in Peru (as reported in the desk research) liked the ‘easy’ access mobile learning offered them. [DR12.7] 4.2.1.8 Quick and easy communication: Of 80 practitioner survey respondents, 83.9% believed that they would benefit from a mobile device for learning, because they would be able to use it to quickly and easily communicate with peers or with experts to get timely advice.10 92.7% (of the 44 manager survey respondents) shared this opinion.11 [SP/SM] Certainly, a literature review on the impact of mobile devices revealed that inter-professional communication was improved by the use of mobile devices instead of pagers [DR5.2], and a study of anaesthesiologists revealed fewer incidents of medical error due to the easy communication between colleagues that mobile phones enabled. [DR7.12] In addition, mobile devices were perceived by many as a good way to communicate certain messages to staff, for example ‘Can text to remind coming out of compliance period’. [FG] 4.2.1.9 Easy knowledge sharing: Of 80 practitioner survey respondents, 75.1% believed that they would benefit from a mobile device for learning, because they would be able to more easily share their

8

15.0% strongly agreed; 46.3% agreed and 16.3% slightly agreed. 14.6% strongly agreed; 51.2% agreed and 17.1% slightly agreed. 10 16.3% strongly agreed; 48.8% agreed and 18.8% slightly agreed. 11 19.5% strongly agreed; 48.8% agreed and 24.4% slightly agreed. 9

20

knowledge and experiences with peers and managers.12 92.7% (of the 44 manager survey respondents) shared this opinion.13 [SP/SM] Student nurses also credited mobile devices with enabling easy knowledge sharing and partly contributing to better learning during their clinical placements. [DR11.5] 4.2.1.10 Improved learner confidence: The ease of access to short nuggets of learning offered by mobile devices, just prior to or during clinics with patients, was shown to improve practitioners’ confidence in their own clinical skills. [DR1.1; DR2.4] 4.2.1.11 Supported decision making: The ability to reference material more frequently with the use of a mobile device was found to support clinical reasoning and decision making. [DR3.3; DR5.3; DR7.7] 4.2.1.12 Improved care: There were some who believed that mobile learning could lead to improved clinical care. Supporting comments included: ‘It has potential to improve patient care by providing quick, accessible information’ [SP] and ‘It would increase clinical time potentially at the bedside, because people may be able to learn in ten minutes what it might otherwise take physically two to three hours to do [if undertaking training elsewhere+.’ [TS] The use of mobile devices was also shown to improve communication between professionals and, in doing so, lead to improved coordination of patient care. [DR5.2] 4.2.1.13 Fits many learning styles: The many different ways that learning may be accessed via a mobile device means that there are opportunities for it to appeal to many different learning styles.14 [FG] 4.2.1.14 Choice: Offering another way to access learning and assessment was deemed very positive in ensuring more people engage. [FG] Certainly, the desk research revealed that individuals with dyslexia more effectively engaged with assessments via mobile devices compared with assessments that were paper based. [DR8-10.18] 4.2.1.15 Good use of ‘dead time’: Of 80 practitioner survey respondents, 75.1% would be likely to use a mobile device for learning during ‘dead time’, for example when waiting for a clinic to begin, or an emergency call.15 75.6% (of the 44 manager survey respondents) shared this opinion.16 [SP/SM] Typical supporting comments were: ‘Definitely would be useful to utilise dead time as finding a computer to log on to can sometimes be challenging’ [SP] and ‘Fits in with peaks and troughs of working day, so encourages best use of time.’ [FG] This benefit was acknowledged in the desk research too, whereby podcasts on mobile devices were deemed useful for physicians to access during time that would otherwise be unused. [DR7.11] However, some survey respondents wrote of ‘dead time’ in the NHS being ‘non-existent’. [SP/SM] 4.2.1.16 Lends itself to informal learning: The portability of mobile devices means that they can be available all times, and so they lend themselves to learning that is learner-driven at a moment and opportunity to suit them. [FG] Those researching the HIV/AIDS healthcare workers in Peru (as reported in the desk research) also reported mobile devices bridging the gap between formal and more informal experiential learning [DR12.8], and enabling the freedom to plan learning activities according to personal requirements. [DR12.6] 12

11.3% strongly agreed; 43.8% agreed and 20.0% slightly agreed. 19.5% strongly agreed; 43.9% agreed and 29.3% slightly agreed. 14 Examples can be found in Section 4.3.2: How and where mobile learning can contribute to training and assessment 15 26.3% strongly agreed; 40.0% agreed and 8.8% slightly agreed. 16 19.5% strongly agreed; 36.6% agreed and 19.5% slightly agreed. 13

21

4.2.1.17 Convenience (assessment location): ‘Mobile devices can support work-based assessment’. [FG] Of 77 practitioner survey respondents, 76.7% believed that a mobile device for assessment of knowledge and skills would be more convenient, because they could undertake formal assessments in their place of work and send them off there and then via the mobile device.17 80.5% (of the 41 manager survey respondents) shared this view.18 [SP/SM] 4.2.1.18 Convenience (timely assessment): Of 77 practitioner survey respondents, 78% believed that a mobile device for assessment of knowledge and skills would be more convenient because they would have greater choice regarding the timing of assessment tasks.19 78.1% (of the 41 manager survey respondents) shared this opinion.20 [SP/SM] 4.2.1.19 Easy evidence collection: The portability of a mobile device means that it is likely to be available when wishing to collect evidence for portfolios. A typical comment was: ‘Can be more reactive and capture evidence instantly.’ [FG] Those reporting on the Assessment and Learning in Practice Settings (ALPS) project, whereby students on a range of health and social care degree courses used mobile devices for assessment, also acknowledged that they enabled events in clinical settings to be ‘easily captured in the moment’. [DR810.2] 4.2.1.20 Immediacy of assessment feedback: Assessments on mobile devices offer ‘opportunity for instant feedback’. [FG] Of 77 practitioner survey respondents, 66.3% would prefer to get assessment feedback and advice on next steps via a mobile device, so that they could use mobile learning to immediately address their needs.21 73.2% (of the 41 manager survey respondents) shared this opinion.22 [SP/SM] Rapid feedback, followed by speedier remediation, was a benefit recognised in the literature too. [DR7.10] 4.2.1.21 Easy reference to previous assessments: Of 77 practitioner survey respondents, 72.8% would prefer to get assessment feedback and advice on next steps via the mobile device, so that they could refer to it with ease again at a later date.23 68.3% (of the 41 manager survey respondents) shared this opinion.24 [SP/SM] This view was also supported by one of the projects reported in the desk research, which commented on student medics appreciating being able to later reflect on clinical experiences they had recorded at the bedside for assessment purposes. [DR3.1] 4.2.1.22 Easy repeating self-assessment: Of 77 practitioner survey respondents, 72.8% would be likely to use a mobile device for doing quick self-assessments to work out for themselves where they needed to brush up on knowledge and skills.25 78% (of the 41 manager survey respondents) also believed this to be the case.26 [SP/SM] A typical comment was: ‘It is iterative – can keep testing oneself to improve.’ [FG] 4.2.1.23 Easy contact with mentors: Of 77 practitioner survey respondents, 68.9% would be likely to use a mobile device to seek advice from a mentor if they had identified a gap in their knowledge that they

17

11.7% strongly agreed; 41.6% agreed and 23.4% slightly agreed. 24.4% strongly agreed; 29.3% agreed and 26.8% slightly agreed. 19 15.6% strongly agreed; 39.0% agreed and 23.4% slightly agreed. 20 14.6% strongly agreed; 41.5% agreed and 22.0% slightly agreed. 21 14.3% strongly agreed; 29.9% agreed and 22.1% slightly agreed. 22 12.2% strongly agreed; 39.0% agreed and 22.0% slightly agreed. 23 16.9% strongly agreed; 37.7% agreed and 18.2% slightly agreed. 24 9.8% strongly agreed; 31.7% agreed and 26.8% slightly agreed. 25 19.5% strongly agreed; 44.2% agreed and 9.1% slightly agreed. 26 14.6% strongly agreed; 31.7% agreed and 31.7% slightly agreed. 18

22

wanted to quickly address.27 78% (of the 41 manager survey respondents) shared this view.28 [SP/SM] The desk research also revealed appreciation for the easy contact with mentors that mobile devices offer. [DR12.4] 4.2.1.24 Consistency: The quality of training experiences can be variable. Mobile devices were viewed as an opportunity to make such experiences more consistent. [FG] This, in turn, could contribute to achieving consistency across practitioners too, especially in areas of healthcare where there is little time for training or a high turnover of staff. [DR6.5] 4.2.1.25 Efficiency and cost savings: The desk research suggested that mobile learning that directly supports performance and clinical procedures enables clinicians to work with greater efficiency. [DR6.2] Others also believed that mobile learning had the potential to ‘save on costs’. [TS] Typical comments were: ‘I think, as a large corporate organisation, mobile e-learning is the right way forward and should be more cost effective’; ‘...not having to leave the workplace is better efficiency saving’ [SM], ‘It’s cheaper than giving everyone laptops and can be done on existing mobile phones’ and ‘Could save time and numbers of rooms needed.’ [FG] 4.2.1.26 Wide reach: While mobile learning may be designed to be delivered via the mobile devices offered by the NHS to its staff, there is no reason why staff may not also access it on personal devices. [FG] At least one of the projects reported in the desk research made use of personal mobile devices. [DR3.9] 4.2.1.27 Appeal across the board: One might think that NHS staff working more remotely with no access, or infrequent access to technology may have the greatest desire for mobile learning. However, the data did not reveal this to be the case. In fact, there was no evidence that positive or negative attitudes towards mobile learning related in any way to job title. Nonetheless, there was an assumption from some that certain groups of staff may have the most to gain from mobile learning. Example comments included: ‘Great for PCT / community staff (district nurses, health visitors etc.)’, ‘Good for assessment in labs and clinical areas where IT equipment is not available’ [FG] and ‘Community-based roles, lone workers... mobile technology could be really useful for them.’ For others, there was a belief that ‘it is all staff that would benefit’, right across the workforce’. [TS] With regard to age, there were three distinctive categories. Frequently, 100% of the under-25s survey respondents gave positive answers to survey questions about mobile learning. This was followed by the 26-45 year olds (who were split into age ranges: 26-35 and 36-45). Typically, 75% to 90% of these age groups gave positive answers to survey questions about mobile learning (with no particular pattern to which of them, 26-35 or 36-45, were the more positive). And finally, the 46-55 year olds and over- 55s were the least positive, although still 55% to 75% gave positive answers to survey questions about mobile learning.

4.2.2 Challenges of mobile learning 4.2.2.1 Little or no experience of mobile learning: Of 125 survey respondents, 70 stated they had little or no experience of mobile learning. See statistical diagrams below.

27 28

16.9% strongly agreed; 28.6% agreed and 23.4% slightly agreed. 14.6% strongly agreed; 39.0% agreed and 24.4% slightly agreed.

23

Practitioners:

Managers:

[SP/SM] This lack of familiarity was deemed a key challenge for mobile projects: ‘If staff are unfamiliar with the technology then that is a major barrier for them.’ [TMP] 4.2.2.2 Reluctance to accept mobile learning: This was a theme that emerged from two perspectives: firstly from staff, and secondly from patients. It was thought that among staff there may be ‘some individuals not attracted to this method of e-learning’ [TS], perhaps due to unfamiliarity with the technology [FG/DR4.4], or concern about a potential addition to their workload. For example, for one project (reported in the desk research), where nurses were using mobile devices with cancer patients, there was widespread concern about the additional potential burden brought by technology, although far fewer expressed the same concern having used the devices for some weeks. [DR4.6] By then, they were ‘clear on how it *the mobile learning+ added value’. ‘*It is+ this that leads to motivation’ and users ultimately embracing it. [TMP] Indeed, the more time users spent with the mobile devices the more relaxed and accepting they were of them. [DR8-10.13] One suggestion made by the Assessment and Learning in Practice Settings project (as reported in the desk research) was the need to identify champions to challenge the less enthusiastic learners. This was done by raising awareness through publishing leaflets that clearly explained the purpose of the devices, and regular meetings with a wide range of stakeholders. [DR8-10.16] In addition, it was considered essential that it was communicated to patients and families of patients how the mobile devices were being used, to avoid any false impressions or negative reaction. [DR2.6; DR3.12; DR8-10.19] 4.2.2.3 Poor perception of e-learning transferred to mobile learning: There is a perception among some learners that e-learning lacks value. For example, one survey respondent with little experience wrote: ‘I never get time to do e-learning, I’m too busy.’ Others remain loyal to face-to-face learning only. Typical comments were: ‘I'm a firm believer in teaching 'the old- fashioned way'...with a tutor and pupils’ and ‘It would be a sad day if I couldn't learn by peer group discussion or in an undisturbed setting with a trainer.’ [SP/SM] In addition, there was a concern that when used for work-placed learning, ‘participants don't value the approach with technology in the same way that they might if used for social purposes’. [TS] 4.2.2.4 Misconception that mobile learning will be introduced for all training: Concern that mobile learning will be introduced to replace all types of training (even though this was never implied in any of the 24

data collection tools). Typical comments were: ‘I would miss group discussion and networking with other agencies on training days’; ‘I can see the benefits, but this would be seen as a substitute to classroombased training [SP/SM]; ‘Some topics are simply not suitable to deliver electronically’ and ‘Self-assessment is not possible for clinical skills where observation and sign-off is required.’ [FG] 4.2.2.5 Solitude of learning on a mobile device: There was a perception that mobile devices would eliminate opportunities for discussion or questioning while learning. Example comments included: ‘I always find it much more helpful to ask questions as the training unfolds. If I was just doing it alone on a phone I don't think I would get a good understanding’ and ‘I find learning in groups much more helpful.’ [SP] The iPad was deemed to be one mobile device that had the potential to be ‘good for group work’. [FG] 4.2.2.6 Preference for laptops/PCs over mobile devices: Despite more than 75% of practitioners being positive about the benefits of mobile devices for learning for all kinds of reasons (for example, see findings 4.1.2.1, 4.1.2.5, 4.1.2.8, 4.1.2.9, 4.1.2.15, 4.1.2.17 and 4.1.2.18), only 25.1% (of 80 practitioner survey respondents) opted for a mobile device over a laptop or a PC when explicitly asked which they would prefer to learn on. [SP] Two key reasons were given, as addressed below in 4.2.2.7 and 4.2.2.8. 4.2.2.7 Screen size: Typical comments were: ‘I would struggle with a small screen for this sort of learning’ [SP] and ‘Due to screen size some staff would prefer to work from a laptop/computer’ [SM]. Concerns regarding screen size were also expressed by focus group attendees [FG] and appeared throughout the desk research. [DR6.10; DR8-10.5; DR12.2] However, there was recognition that the slightly bigger screen offered by tablets may make screen size more agreeable: ‘The iPad-style device would seem like an ideal half-way house’ [SP]; ‘The user experience and interaction with any kind of learning is going to be far more positive with a 9-inch screen rather than a small screen.’ [TS] In addition, the desk research pointed to some devices having better resolution and screen size than others (for example, the iPhone being more acceptable than the Nokia N95) [DR12.5], as well as acceptance of screen size being dependent on ability to zoom and the colour quality and ambient light available to users. [DR6.8] 4.2.2.8 Lack of confidence in the reliability of mobile technology: Typical comments were: ‘Mobile networks are often slow or *there is+ bad reception which limits them’, ‘What happens when the server goes down? Or the apparatus malfunctions?’ [SP/SM] ‘Data may be lost or not saved’, ‘System/network is not always reliable’ [FG] and ‘A lot of new hospitals have steel walls so the wireless doesn't always work from room to room.' [TS] 4.2.2.9 Appropriateness when capturing evidence: Consent and confidentiality were concerns with regard to capturing evidence for assessment purposes, particularly when that evidence is photographic or video. Typical comments were: ‘Confidentiality would have to be thought about carefully’ and ‘Capturing inappropriate evidence – there are consent issues here.’ [SP/SM/FG] This concern was echoed in the desk research where there was a reminder of the need for clear ethical guidelines when collecting assessment evidence (of the kind provided by the Royal College of General Practitioners: www.rcgp.co.uk) [DR8-10.14] 4.2.2.10 Potential for dishonesty: There was concern that there may be abuse when undertaking assessments using a mobile device: someone other than the learner being assessed may answer questions, or the learner may look up answers. [SM/FG] 4.2.2.11 Lack of skill with mobile technology: Staff’s lack of confidence with technology, and more specifically their lack of skills with mobile devices, was considered a key challenge that would require addressing. [FG] Typical comments made by those who had run mobile projects included: ‘There was more 25

requirement for one-to-one with lower IT literate staff... [even though] the majority use a mobile phone in one way or another... *For example+, they had to be aware of being able to touch the screen. It’s just about them being confident with that technology.’ [TMP] Of 80 practitioner survey respondents, 35 (43.8%) believed they were proficient enough with technology to make use of a mobile device without training. That left 45 (56.3%) who identified a need for some initial training. In most cases, job role made no difference. For instance, nine ambulance workers expressed a requirement for training, while seven did not; eight radiographers expressed a requirement for training, while six did not. The least need was among junior doctors (four expressed a requirement for training; nine did not), and the most need was among administrators (all five who answered the question identified a need for training). This view was supported by the majority of the 41 manager survey respondents, across all job roles, with 24 (68.3%) identifying a training need for some and ten (24.4%) identifying a training need for all staff. [SP/SM] Even where staff were comfortable with using technology, training was still deemed essential, where ‘it entailed a new way of working’. For example, midwives had training on how to use their digital pens and Blackberries where this technology was employed for their visits to mothers, as well as training on a new way of taking notes. Moreover, ‘a dedicated resource’ to monitor the use of the mobile devices from the early stages ensured smooth processes and quick follow-up where issues emerged. [TMP] The desk research findings also supported a need for training [DR3.11; DR5.7], with one project putting in place a cascading model for training those who could not attend the classroom sessions. [DR4.2] In the Assessment and Learning in Practice Settings project, it was deemed that the training worked best where it involved face to face, with encompassing documentation offered as mobile versions to encourage use and familiarity. [DR8-10.8] Moreover, no assumptions were made about learners’ capabilities with mobile devices: the same training was made available to all. [DR8-10.7] Clearly, there is already some good practice with regard to IT training within the NHS, as when asked to identify the most valuable NHS training received, two practitioner survey respondents chose some IT training. One wrote, ‘The most valuable training would be for the software we use for my job role. It took around three weeks, was delivered mainly using PowerPoint presentations, and without it I would not be able to use the software I use every day’; another wrote, ‘My one-to-one on computer skills. I had no previous knowledge of how to use a computer and now regard myself as totally literate. The training was four sessions of one hour's duration.’ [SP/SM] 4.2.2.12 Inequality of access: Although some people have a mobile device supplied by the NHS, and others have their own mobile device (see findings 4.2.5.1, 4.2.5.4 and 4.2.5.5), there was a concern about access, in particular ‘inequality’ of access between individuals and potentially between Trusts too. [TS] 4.2.2.13 Technical support requirement: It was considered essential that staff are offered ‘IT support / troubleshooting 24/7’. [FG] This was also mentioned in many of the projects reported in the desk research. [DR1.4; DR3.8; DR8-10.9; DR12.9] However, there was concern about the ability to fulfil this obligation, especially if there were a number of different types of mobile device being used. A typical comment was: ‘There's a capacity issue in terms of available expertise and skills, and then there is the more variance you put into an infrastructure, the greater the risk.’ [TS] This kind of support was deemed to work best when it was offered in collaboration between the clinical area the staff work in, the learning and development or training department, and the IT unit. [DR3.8] 26

4.2.2.14 Input limitations: One survey respondent wrote in relation to using a mobile device for assessment by rating skills and then commenting on them: ‘Would tend to use a PC to enable full-size keyboard and speedier inputting.’ Certainly, one project examined in the desk research reported evidence that entry via a stylus was slower and more erroneous and less satisfactory than entering data via a QWERTY keyboard, even on a mobile device [DR5.8] A focus group attendee expressed concern about ‘manual dexterity with touch screens’ too. [FG] A similar concern was found in the desk research, with regard to the possible requirement to offer alternative means of input where a user had ‘limited hand mobility’. [DR8-10.17] 4.2.2.15 Easy to lose or steal: The portability of devices naturally means they are small, which led to concern about mobile devices or components (such as chargers and styluses) being easily mislaid or stolen. [FG] One project described in the desk research made responsibility for the mobile equipment a key condition under which it was lent out. [DR8-10.15] 4.2.2.16 Conversion of existing e-learning to mobile: There was concern about converting existing e-learning content, where it may not be compatible for mobile devices. [FG] Certainly, the desk research revealed that good design of content or task, taking account of the size of a mobile screen, was very important. [DR6.8; DR12.2] It was also deemed important to remember that mobile devices are often used while undertaking other tasks, and so one-handed operations were desirable. [DR6.9] 4.2.2.17 Device agnosticism: Any newly developed learning and assessment resources for mobile devices should be device-neutral and appropriate for different platforms, and, as far as possible, future proof, so that they would not become redundant when a device is updated with a newer model. [FG] This was supported by one of the desk research projects, which recommended designing for solid platforms that were not dependent on new fashions and trends and would ensure interoperability into the future. [DR3.10] 4.2.2.18 Security: The storing and transfer of personal data needs to be secure and respect data protection laws. [FG] In addition, patient confidentiality must be preserved and protected, and any integration with NHS systems must not be allowed to corrupt information already being held on a server. [TS] While these security issues were endorsed by key stakeholders, there was an indication of a risk-adverse culture with regard to IT. Typical comments included: ‘There's a very risk-adverse approach to IT development because of the fears around security... [and] I think organisations’ willingness to explore potential mobile technologies will be affected. [TS] People worry they will lose control of the security of the network if they give people Wi-fi access. [TT] ‘Personally, I think our IT department are a bit too fastidious about our security on the basis they almost want there to be no risk within the system. But if you look at the level of risk that we work at in medicine most days of the week, then the risk levels of not being able to access certain things is much greater than the risk of people who shouldn’t access it, accessing it... The data is predominantly protected by the fact the devices won’t work if you are outside the area. Plus you need a pin code to log on to the system.’ [TMP] Data security was also mentioned in the desk research, though it was considered to be surmountable by engaging early with IT departments to negotiate potential solutions [DR8-10.11], including putting in place good security practices like password protection, data encryption, virus protection software, identification and frequent backing up of data. [DR3.6] 4.2.2.19 Interference with other equipment: The question was raised about whether there would be problems with mobile devices interfering with vital medical equipment, in particular on wards. [FG] However, the desk research suggested that many healthcare settings overestimated this risk 27

[DR7.14; DR8-10.10] and, in fact, it may only cause a problem in certain areas of hospitals, such as cardiac and intensive-care units. [DR3.13] 4.2.2.20 Costs: Hardware costs and app development costs were deemed a challenge, especially in current times of budget cutting. [FG] In addition, service fees to operators, and the need for an IT help desk to solve technical problems were raised in the desk research as having additional cost implications. [DR12.9] 4.2.2.21 HR policy: A question was raised about whether the introduction of mobile devices for learning would require a re-examination of HR policies and procedures, for example in relation to protected learning time and out of hours work. *FG+ ‘It is the same problems we currently have with e-learning, because if we say people can do their e-learning out of work in a more flexible way, then there is a human resource aspect to that, in terms of do they get time back or are they just expected to do it in their own time?’ [TS] 4.2.2.22 Interfacing with NHS IT systems: There were many challenges noted with regard to mobile devices interfacing with the NHS’s IT systems. These are described and explored in Section 4.2.6 The integration of mobile devices with existing NHS IT infrastructures. Research Objective 2: Identify how and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in the Statutory and Mandatory Training Framework Sections 4.2.3 and 4.2.4 below address research objective 2

4.2.3 How and where mobile learning can contribute to training and assessment 4.2.3.1 Mobile learning by reading (words and pictures): Of 80 practitioner survey respondents, 85% would use a mobile device for learning by reading (words and pictures).29 87.8% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for learning in this way.30 [SP/SM] Learning by reading was also identified in the desk research, in particular with reference to anaesthesiologists accessing mobile reference tools such as the Anaesthesia Drug Handbook and the fiveminute Clinical Consultant, and patients accessing self-care information via mobile devices. [DR4.7; DR7.5] 4.2.3.2 Mobile learning by listening to audio: Podcasts were mentioned in the desk research as a resource distributed via mobile devices to students as part of their problem-based learning medical curriculum [DR3.4], as well as for physicians to get an overview of latest research, or to listen to medical lectures. [DR7.11] Of course, they can be listened to, ‘in a cafeteria, or on the way to work, or whatever moment in the day works best’ too. [TS] 4.2.3.3 Mobile learning by watching a short video: Of 80 practitioner survey respondents, 87.5% would use a mobile device for learning by watching a short video.31 Video was widely mentioned among practitioners when asked to describe their most valuable NHS training experience. For example, one survey respondent wrote: ‘Moving and handling, video showing how to get patients down the stairs using a slide. Good visual demonstration.’ 85.4% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for learning in this way.32 [SP/SM] 29

27.5% strongly agreed; 37.5% agreed and 20.0% slightly agreed. 12.2% strongly agreed; 58.5% agreed and 17.1% slightly agreed. 31 22.5% strongly agreed; 47.5% agreed and 17.5% slightly agreed. 32 14.6% strongly agreed; 53.7% agreed and 17.1% slightly agreed. 30

28

4.2.3.4 Mobile learning by watching an animation: Of 80 practitioner survey respondents, 66.3% would use a mobile device for learning by watching an animation.33 65.9% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for learning in this way.34 [SP/SM] 4.2.3.5 Mobile learning by working through decision tress: Decision trees, or network diagrams, not only enable a large amount of information to be represented visually, but also for inter-connectedness to be clearly displayed. [DR11.3] Of 80 practitioner survey respondents, 66.3% would use a mobile device for learning by working through decision trees.35 73.1% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for learning in this way.36 [SP/SM] 4.2.3.6 Mobile learning by sending messages and discussing with peers and/or experts: Of 80 practitioner survey respondents, 72.6% would use a mobile device for learning by sending messages and discussing with peers and/or experts.37 65.9% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for learning in this way.38 [SP/SM] 4.2.3.7 Mobile learning by researching on the internet: When given the opportunity to describe any other ways mobile devices may or may not support learning, there were many mentions of accessing websites to look up information instantly. [SP/SM] The ability to use a wireless connection to go online for required information was also mentioned in the desk research. [DR7.2] 4.2.3.8 Mobile learning that integrates with clinical practice: Where PDAs have been used to collect patient data at the bedside, it was found that they were also educating staff and improving their clinical practice: ‘For example, it will tell you, you put in a value for this person’s blood pressure that is very low, did you mean to do that? The answer may be yes, but it may be that you made an error. It also teaches me that this particular oxygen mask needs to have this particular flow rate... as part of the process it does educate people.’ [TMP] 4.2.3.9 Intelligent pre-assessment: There was a desire for less ‘blanket’ training and more pre-assessments to gauge learning needs. Example comments were: ‘Send me the quiz on the key learning points I need to know about fire, then if I know them... I do not need to attend the lecture. You have two things: one is evidence that you know that I have what I need to know, and two, you have saved me time and effort doing the training’; ‘There is potential use of mobile for pre-assessment. So if someone has completed an assessment successfully on a given topic and they meet a required threshold, then they shouldn't have to attend a training session in order to have it recorded.’ [TS] 4.2.3.10 Assessment by multiple choice questions: Of 77 practitioner survey respondents, 81.9% would use a mobile device for assessments that involve answering multiple choice questions to test their knowledge.39 85.4% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for assessments of this kind.40 [SP/SM] 33

12.5% strongly agreed; 32.5% agreed and 21.3% slightly agreed. 9.8% strongly agreed; 31.7% agreed and 24.4% slightly agreed. 35 8.8% strongly agreed; 35.0% agreed and 22.5% slightly agreed. 36 7.3% strongly agreed; 26.8% agreed and 39.0% slightly agreed. 37 11.3% strongly agreed; 38.8% agreed and 22.5% slightly agreed. 38 7.3% strongly agreed; 29.3% agreed and 29.3% slightly agreed. 39 24.7% strongly agreed; 48.1% agreed and 9.1% slightly agreed. 40 24.4% strongly agreed; 48.8% agreed and 12.2% slightly agreed. 34

29

4.2.3.11 Assessment by choosing the correct photograph: Of 77 practitioner survey respondents, 83.2% would use a mobile device for assessments that involve choosing the correct photograph, for example ‘to demonstrate that I have the knowledge necessary to identify equipment or danger.’41 78.1% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for assessments of this kind.42 [SP/SM] 4.2.3.12 Assessment by rating skills and commenting on them: The desk research unveiled a project where mobile devices were used for practitioners to rate their skills and proficiencies and add comments, which then outputted a personalised action plan. [DR11.2] Of 77 practitioner survey respondents, 67.6% would use a mobile device for assessments that involve rating their skills and then commenting on them.43 65.8% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for assessments of this kind. 44 However, one issue raised in response to this kind of assessment was that of the ease of input into the mobile device: see Finding 4.2.2.14 [SP/SM] 4.2.3.13 Assessment by collecting photographic/video evidence: Of 77 practitioner survey respondents, 62.4% would use a mobile device for assessments that involve describing or taking photographs/video to collect evidence of their skills.45 Only 46.3% (of the 41 manager survey respondents) supported this view, believing their staff would use a mobile device for assessments of this kind. Many respondents expressed concern about confidentiality and consent if using mobile devices in this way: see Finding 4.2.2.9 [SP/SM] 4.2.3.14 Assessment through improved access to e-Portfolios: There were comments made about staff struggling to get access to their e-Portfolios on computer terminals on wards, and the download/upload speeds being very slow. ‘Mobile devices would bring assessment closer to the doctor-patient interaction, improve accessibility and ease of access.’ [TS] 4.3.3.15 Induction: A number of respondents mentioned induction in relation to mobile learning: ‘...could be on their own phones, or could be a phone that is loaned to them for a period of a month.’ [TS] 4.3.4.16 Statutory and mandatory training: With organisations reportedly finding it challenging to meet the demands of statutory and mandatory training, typical comments with regard to mobile learning were: ‘The obvious place where people should start is compliance training – from an organisational point of view that will be the priority’ and ‘The areas of health and safety, fire training, moving and handling, food hygiene... if those were undertaken remotely by these sorts of [mobile] devices, using interesting and innovative technology, I suspect then the benefits in uptake would outweigh the disadvantages of failing to comply. [TS] 4.2.3.17 A need for a mobile learning strategy: There was a belief that e-learning strategies, at least at a local level, may make some small reference to mobile learning, and the use of mobile phones is covered by information governance rules, ensuring staff do not put patient confidentiality at risk. However, there is currently no NHS mobile learning strategy that sets out, for example, best practice in terms of the use of mobile devices for learning and assessment [TS] and advises on such issues as appropriate templates or technology. [DR8-10.3; DR8-10.4] This may be partly, though not wholly, addressed by the Department of Health’s new 'Technology-Enhanced Learning Strategy', due to be published in June 2011, which will call for greater adoption of learning technologies, including mobile. [TS] 41

19.5% strongly agreed; 48.1% agreed and 15.6% slightly agreed. 22.0% strongly agreed; 34.1% agreed and 22.0% slightly agreed. 43 6.5% strongly agreed; 33.8% agreed and 27.3% slightly agreed. 44 7.3% strongly agreed; 24.4% agreed and 34.1% slightly agreed. 45 6.5% strongly agreed; 33.8% agreed and 22.1% slightly agreed. 42

30

4.2.3.18 A need for stakeholder involvement in developing a mobile learning strategy: Currently, there are no dedicated leads around this agenda, rather it is one where different people might have an interest. Some of the key people to involve would include: the NHS Board, Department of Health (procurement), Postgraduate deaneries, Trust Directors of Medical Education and Development, Trust Directors of Ops, Trust IT managers, Trust HR Directors, Learning and Development Managers, E-portfolio developers and providers, E-learning leads, Medical Royal Colleges, the BMA and other representative bodies and H.E. institutions. [TS] 4.2.3.19 Clinical staff first: While there was acknowledgement that many groups of staff could benefit from using mobile devices for learning and assessment (see Finding 4.2.1.27), there was a belief that ‘it is not one size fits all - it must be targeted’, with clinical staff coming first. Typical comments were: ‘Area to target first would be those engaged in delivery of clinical services: nurses, doctors, healthcare professionals’, ‘It is going to be more applicable to mobile staff... the more clinically based staff, a lot of whom do a lot of education outside their work hours, that's where I think the distinct advantage could lie’, and ‘For assessment... any staff member who has to provide evidence for the purposes of training or revalidation/relicensing: students nurses, medical students. [TS] 4.2.3.20 A need for pilot projects: Small-scale pilot studies were mentioned both by those telephone interviewees who had been involved in mobile projects before, as well as those projects described in the desk research. Such pilots were used as a means to co-design learning on mobile devices and get feedback on prototypes [DR11.7; TS], thus securing engagement [DR8-10.21] and also as a means to undertake live trials to check the supporting IT infrastructure. [DR8-10.22] 4.2.3.21 The future of mobile: There was some excitement about the potential for mobile devices for learning and assessment in the future. Augmented reality, where the camera in a mobile device superimposes relevant information on to a real work scene, was mentioned in both the desk research [DR6.11] and by one of the stakeholder telephone interviewees, who stated: ‘In a health environment with things like augmented reality, for anatomy and physiology... we want to capture it and run with it.’ [TS] There was also mention of games-based learning and virtual reality, and a desire expressed to ‘catch up’: ‘We need to keep a-pace with people’s expectations and what they are doing in everyday lives and embed that into the healthcare setting.’ [TS]

4.2.4 How and where mobile learning can contribute to training and assessment, in particular for the nine essential skill areas in Statutory and Mandatory Training Framework Note: The nine essential skill areas in the Statutory and Mandatory Training Framework are: Conflict Resolution; Equality, Diversity and Human Rights; Fire Safety; Health and Safety; Infection Prevention and Control; Moving and Handling; Resuscitation; Safeguarding Children; Safeguarding Vulnerable Adults. Research participants were asked about a selection of these skill areas. 4.2.4.1 For theory-based topics: Mobile learning was considered ideal for theory-based learning, for example for Equality, Diversity and Human Rights. [FG] 4.2.4.2 For legislation updates: Mobile devices would enable learners to stay up-to-date with any changing legislation, for example for Health and Safety. [FG]

31

4.2.4.3 For easy reference: Mobile devices would mean that learners can always have information available for quick and easy reference, for example policy and protocols for Equality, Diversity and Human Rights or Conflict Resolution. [FG] 4.2.4.4 For recording reflections: Learners would be able to record how experiences made them feel and engage in timely reflection with the use of a mobile device, for example for Equality, Diversity and Human Rights or Conflict Resolution. [FG] 4.2.4.5 For video story-telling: Video/simulation story-telling would work well on mobile devices for exploring scenarios, for example for Equality, Diversity and Human Rights or Conflict Resolution. [FG] 4.2.4.6 For visual learning: Photographs on mobile devices were deemed very appropriate for certain topics, such as body language for Conflict Resolution. In addition, a practitioner survey respondent commented on photographs: ‘this could be used to assess fire training’. [SP] 4.2.4.7 For video assessment: Mobile devices provide the opportunity to video practice and then use the video clip as evidence of competency in a skill, for example for Moving and Handling. [FG] 4.2.4.8 For ‘safe’ practice: Mobile learning offers a ‘safe’, non-judgemental environment in which to practice, for example to reflect on a response to scenarios for Conflict Resolution. [FG] 4.2.4.9 For Skype mentoring: Discussion via Skype calls (with or without video, so the callers can see each other) would enable mentoring to happen regardless of location. Hence, discussion could still take place, for example for Conflict Resolution. [FG] 4.2.4.10 For part of a blend: Mobile learning was deemed to have a role to play for all nine essential skill areas, but only suitable as part of a blend, not as the whole solution. For example, it would be difficult to test people on soft skills, such as behaviours for Equality Diversity and Human Rights, or on the practice essential for Resuscitation, or on the discussion that is necessary for Conflict Resolution, or on the ‘handson’ experience required for Moving and Handling. [FG] It was deemed that the ‘best approach would be to learn the principals and do as much as you can electronically, but then you have the flexibility within organisations to deliver anything locally appropriate and relevant’. [TS] 4.2.4.11 For assessing essential knowledge through quizzes: Of 77 practitioner survey respondents, 76.7% would be happy to use a mobile device for assessments on the essential knowledge required for subjects such as Fire Safety and Infection Prevention and Control (for example, by answering a series of word or picture questions in a quiz).46 83% (of the 41 manager survey respondents) shared this opinion.47 [SP/SM] 4.2.4.12 For assessing essential skills by collecting evidence: Of 77 practitioner survey respondents, 70.2% would be happy to use a mobile device for assessments on some of the essential skills required for subjects such as Moving and Handling and Health and Safety (for example, by providing written, photographic or video evidence.)48 75.6% (of the 41 manager survey respondents) shared this opinion.49 [SP/SM] 4.2.4.13 Time saving for refresher training: Survey respondents commented on the time that would be saved by using mobile learning rather than face-to-face courses for refresher training. Typical comments were: ‘It would be really useful to do yearly updates on essential skills rather than taking time to attend 46

20.8% strongly agreed; 44.2% agreed and 11.7% slightly agreed. 24.4% strongly agreed; 36.6% agreed and 22.0% slightly agreed. 48 19.5% strongly agreed; 40.3% agreed and 10.4% slightly agreed. 49 24.4% strongly agreed; 31.7% agreed and 19.5% slightly agreed. 47

32

lectures’ and ‘E-learning, as a refresher course, was best NHS training. Didn't waste time going to courses.' [SP] 4.2.4.14 Impact: In the current climate, the focus is on learning that makes a difference now, rather than long-term education. This means education that can be tracked, so that the NHS can see that everyone in a Trust has had specific training. [TS] Research Objective 3: Establish the kinds of mobile devices available to NHS South Central staff Section 4.2.5 below addresses research objective 3

4.2.5 The kinds of mobile devices available to NHS South Central staff 4.2.5.1 Job roles supplied with a mobile phone by the NHS: More survey respondents had a mobile device supplied by the NHS than did not. In fact, of 125 survey respondents, 78 (that is 62.4%) answered ‘yes’ to being provided a mobile device by the NHS (of which 77 were referring to a mobile phone and one was referring to a laptop). In some cases, the provision, or not, of a mobile phone appeared to depend on job role. For example, see community psychiatric nurses and radiologists respectively in the statistical diagrams below: Practitioners:

Managers:

[SP/SM] In addition, it was estimated that about 500 mental-health workers are mobile users as part of their job function. [TT]

33

4.2.5.2 Blackberry, Nokia and HTC (Android) were the most common makes of mobile phone supplied by the NHS: There is no mandate that dictates the devices supplied by the NHS as long as they are information governance compliant. [TT] Nonetheless, practitioners tended to have Blackberries, Nokia or HTC (Android), while managers only stated Blackberry. [SP/SM] Key technical staff and key stakeholders also spoke of the use of Blackberries (in particular in mental health), and of some staff being given HTC handsets. [TT/TS] 4.2.5.3 Windows mobile and HP tablet devices are being explored by the NHS: One of the key technical staff interviewed spoke of hopes for Windows mobile, due to ‘everything over the past ten years in the NHS has been Microsoft’. The same interviewee spoke of trials on HP tablet devices. (However, this handheld tablet runs WebOS not Windows.) [TT] 4.2.5.4 Apple, Blackberry, Android phones and Nintendo DS are the most common mobile devices available to NHS staff inside and outside work: While Blackberry, Nokia and HTC (Android) were mentioned with regard to mobile devices supplied by the NHS, responses were different when practitioners and managers were questioned on the mobile devices they used outside work too. Of 125 survey respondents, 27 stated they used an Apple product (either iPhone (15), iPod Touch (10) or iPad (2)); 23 stated they used a Blackberry, and six stated they used an Android/Google smart phone. The Nintendo DS was also popular, with 18 survey respondents stating they made use of this mobile device outside work. The table below shows all responses. (Note that survey respondents could select more than one device.) [SP/SM] Do you use any of the following mobile devices either in or outside work? Response Count (practitioners)

Response Count (Managers)

Totals

Mobile phone (with no internet access) Apple iPhone Apple iPod Touch Apple iPad Blackberry Android/Google smart phone Symbian/Nokia smart phone Windows mobile smart phone Other tablet Nintendo DS

54 10 7 1 10 4 1 0 0 13

29 5 3 1 13 2 1 0 0 5

83 15 10 2 23 6 2 0 0 18

Mobile phone (with internet access, but I do not know what kind) Other (please specify)

14

10

24

3

0

3

Answer Options

(Others specified: Samsung Wave; 3G RAS token; laptop with toggle for mobile internet access) 4.2.5.5 Learners access via their own personal devices: Key technical staff spoke of personal iPads and iPhones being used by NHS staff [TT], and there was anecdotal evidence that ‘a lot of people are already using iPhones to access their e-Portfolio using Safari’, with NHS Scotland being of the opinion that iPhones, iPods and iPads are ‘used a lot to access online material even though they are not supplied by the NHS’.

34

[TS] One survey respondent wrote, ‘I'd rather learn at home not at work, so it would be good to be able to use my mobile device’. [SP] 4.2.5.6 Lending learners’ devices: It was deemed essential that learners were lent devices, where they are not already supplied with a device by the NHS, nor have a device of their own [DR8-10.15]: ‘The minute you start to mandate any type of learning from a Trust's point of view, you have an obligation to make sure people have the ability to access that, which in turn is an obligation to provide the devices.’ Other supporting comments included: ‘*If+ people already have the devices themselves, or we could provide the devices, then I think the potential is vast’ and ‘You are always going to have the issue that a huge amount of people won't have devices like iPad, and short of those being available on a loan basis, I don't know how you would overcome that.’ [TS] Research Objective 4: Determine how mobile learning can integrate with existing NHS IT infrastructures Section 4.2.6 below addresses research objective 4

4.2.6 The integration of mobile devices with existing NHS IT infrastructures 4.2.6.1 The NHS IT infrastructure: It is essential to understand the IT systems with which mobile devices for learning and assessment must interact. [DR8-10.23] There was some concern about connecting to NHS IT networks, and in particular the e-learning system. [FG] This is because the NHS has a National Learning Management System (NLMS) which uses OLM (Oracle Learning Management) to deliver e-learning to all those with a staff record on ESR (Electronic Staff Record). Therefore, as learners progress through e-learning courses they are tracked, and test results and other data are recorded in ESR. Any mobile learning or assessment would need to integrate with these existing systems. [TT] 4.2.6.2 Tracking learners on NHS IT infrastructure: E-learning sits on a separate domain to enable it to be remotely accessed via the internet without compromise to the wider system. The SCORM API (which forwards the user’s tracking information to ESR) is written in JAVA. This is because it needs to communicate across different secure domains within the NHS. Therefore, Oracle’s JAVA SCORM adapter has to be used to update ESR. Unfortunately, JAVA has yet to be written for ARM devices, which is the hardware used by a large number of smart phones . This makes it difficult to have SCORM-compliant mobile content that saves directly back to the LMS. [TT] 4.2.6.3 Feeding information back to NLMS: The easiest way to feed information back to NLMS would be to use Oracle’s JAVA API, but that would only work while the connection is live and, as outlined in 4.2.6.2, will only work on certain mobile devices. In addition, if the learner loses connectivity, then all information would be lost. Another solution would be to create a mobile API app that would interface in some way to ESR, but this may be costly and would need a lot of development. [TT] Besides, system changes have to be from a national perspective, which makes rolling out any kind of change to the NLMS to allow for mobile tracking a lengthy undertaking. A further complication is that each Trust sets up their network and infrastructure in a slightly different way as there is no consistent national policy. [TS] 4.2.6.4 NHS IT may inhibit mobile learning design: There was some disappointment expressed that use of technology in home life, for example for playing games, is: ‘far from the very simplistic health e-learning... the platforms used in the NHS are not conducive to forging ahead with technical advances in e-learning.’ [TS] 35

4.2.6.5 Flash content may not work on all mobile devices: There were concerns about running Flash on Apple products. One desk research project mentioned the constraint imposed by using iPods because of their inability to run ‘flash’- based learning objects [DR2.7], and the iPad was also mentioned as not being able to accommodate Flash, although there was recognition of a workaround with HTML5. [TS] 4.2.6.6 Wireless connectivity: There was much evidence from the desk research that wireless was considered to be a necessity, with wireless transfer of data far preferable and more convenient for users over cable or docking device. [DR3.7; DR6.7; DR7.13; DR8-10.6] It was estimated that at the present time about 50% of NHS sites have Wi-fi, with all staff at those sites allowed to connect, and, in some cases, visitors too. One example where there is Wi-fi across all sites is in the ambulance service. Certainly, it was deemed preferable for there to be wireless connectivity for mobile devices, although it was acknowledged that ‘Wi-fi isn’t perfect’. Where this wasn’t possible, then one solution suggested was to connect via a Wi-fi hotspot or at home and then synchronise later, so pushing data via SSL outside the organisation’s network. For the blood tracking mobile project, it was essential to achieve widespread and reliable wireless connectivity to ensure true traceability and avoid having any ‘lost’ units of blood. And for the collecting patient data at the bedside mobile project they installed enough wireless points to achieve 98% of the data being uploaded within one second. [TMP] Currently, South Central SHA is seeking to procure a network layer that can communicate with all the current structure securely, which would include Wi-fi and 3G. At the time of this research study, South Central SHA was awaiting the go-ahead for this. [TS] 4.2.6.7 Other ways to connect: 3G has a different, and potentially high, costing structure compared to Wifi. Access to the data also could be much more difficult, as giving access to all systems via this extra layer requires greater security management. This means that while the Ambulance Service, for example, currently has wireless access on sites, they don’t have wide access on the road. However, there is a desire in the South to provide constant connections for ambulances, but this will involve introducing over 600 devices with limited 3G bandwidth and synching with Wi-fi when they return to base. Previously, content has been put on to tablet devices which then have been plugged into the network to synchronise and upload on to the LMS server. Other solutions have entailed leveraging the GPRS, the WAP and 3G directly from the device to get an internet connection, and then direct that back to the NHS. Therefore, Wi-fi was considered a luxury, not a necessity. [TT] 4.2.6.8 Content across mobile platforms: There was acknowledgement that staff would not wish to have many devices for different tasks, and this would mean offering content that works on multi-platforms: ‘Today I mention iPhones, but tomorrow will we be talking about Androids? We need software that can be used for both systems.’ *TS+ ‘Whatever we do, we need to do it in a uniform way – we need to have one device to access everything.’ [TMP] 4.2.6.9 Identifying new apps: There was much enthusiasm for different kinds of apps. For example, one interviewee asked, ‘Should we be developing an app to simplify and facilitate easier mobile access to learning and e-portfolios?’ [TS]

36

5. Recommendations in relation to project objectives Project Objective A: Make mobile learning recommendations based on the outcomes of research objectives 1 and 2 Section 5.1 below addresses project objective A

5.1 Mobile learning recommendations Recommendation 1: An NHS mobile learning strategy is put in place to set out best practice in identifying and meeting learning or assessment needs through the use of mobile devices. The following recommendations should contribute to building that strategy. Ref: findings 4.2.3.17 A need for a mobile learning strategy and 4.2.3.18 A need for stakeholder involvement in developing a mobile learning strategy

Recommendation 2: Consider where mobile learning may benefit all NHS staff roles, not solely those that entail working remotely. After all, there are many benefits to mobile learning that apply across all jobs, such as the convenience of not having to leave the workplace for training, the relevance of the learning being offered ‘in situ’ rather than simulated, and the ease of evidence collection for assessment purposes. Ref: findings 4.2.1.27 Appeal across the board; 4.2.1.1 Convenience (learning location); 4.2.1.17 Convenience (assessment location); 4.2.1.5 Relevance and 4.2.1.19 Easy evidence collection

Recommendation 3: Offer some induction training via mobile device. This would have several benefits: it would be cost effective, as it would not use other staff time; enable a consistent induction experience across all new staff; contextualise the learning, as it could take place in the actual workplace; and ‘set the tone’ for delivery of training in the organisation. Ref: findings 4.3.3.14 Induction; 4.2.1.24 Consistency and 4.2.1.25 Efficiency and cost savings

Recommendation 4: Recognise the additional benefits of mobile learning to clinical staff. For example, the benefit of being able to very easily double-check a decision, the additional reassurance this offers in terms of confirming professional judgements, and so the potential this has to lead to improved care for patients. Therefore, if having to prioritise, consider supporting these job roles with mobile learning first. Ref: findings 4.2.3.19 Clinical staff first; 4.2.1.11 Supported decision making; 4.2.1.3 Reassurance and 4.2.1.12 Improved care

Recommendation 5: Where there is a need for performance support (i.e. learning in the moment), consider offering easy-to-reference materials on mobile devices. These may be searchable databases of static information to access knowledge, or short video clips that demonstrate a skill. Ref: findings 4.2.1.2 Convenience (timely learning) and 4.2.1.7 Quick and easy access to information

Recommendation 6: Where there is a need for sustainable learning (i.e. learning to be applied over time), consider offering small distinct nuggets that can sit alone, but together form part of a coherent whole. This enables control over the duration of the learning experience, making it easy to fit in during

37

‘dead’ time in the working day. It also enables the learner to very specifically revisit particular content, thus empowering them to direct their own learning activities. Ref: findings 4.2.1.4 Flexibility; 4.2.1.6 Control of learning; 4.2.1.15 Good use of ‘dead time’ and 4.2.1.16 Lends itself to informal learning

Recommendation 7: Employ a range of mobile learning approaches to fit with a variety of learning styles and preferences. Many learners seem to prefer to read text or to watch short video vignettes or stories; some may prefer to engage with visual representations of information like decision trees; or to listen to podcasts, perhaps while undertaking other tasks; or to perhaps watch short animations. All these means can be designed with interactions so that learners can actively engage with the content as they work their way through it. Many are likely to at some point wish to extend or confirm their learning by being able to research for themselves on the internet. Ref: findings 4.2.1.13 Fits many learning styles; 4.2.1.14 Choice; 4.2.3.1 Mobile learning by reading (words and pictures); 4.2.3.2 Mobile learning by listening to audio; 4.2.3.3 Mobile learning by watching a short video; 4.2.4.5 For video story-telling; 4.2.4.6 For visual learning; 4.2.3.4 Mobile learning by watching an animation; 4.2.3.5 Mobile learning by working through decision trees and 4.2.3.7 Mobile learning by researching on the internet

Recommendation 8: Seek opportunities to integrate learning into mobile resources that primarily support clinical practice. The growing use of mobile devices for clinical tasks, such as recording patient data at the bedside, provides an opportunity for learning alongside. For example, small performance support nuggets may be made accessible to remind clinicians of the limits of what would be considered normal for, say, blood pressure, temperature, heart rate and so on, or to provide feedback on data they have entered. Ref: finding 4.2.3.8 Mobile learning that integrates with clinical practice

Recommendation 9: Stimulate social learning (i.e. communicating with peers and experts) by offering both written and voice communication options. SMS texting reminders, knowledge sharing forums, ‘ask a question’ forms and the use of telephony are all means to enable interaction between staff, and will address the perception by some that mobile learning means learning in solitude. Ref: findings 4.2.1.8 Quick and easy communication; 4.2.1.9 Easy knowledge sharing; 4.2.3.6 Mobile learning by sending messages and discussing with peers and/or experts; 4.2.4.9 For Skype mentoring and 4.2.2.5 Solitude of learning on a mobile device

Recommendation 10: Offer mobile assessments to identify learning needs just prior to the required refresher intervals for Statutory and Mandatory knowledge and skills. This will enable learners’ knowledge and skills to be tracked, and training offered only where required, rather than having to attend a whole course regardless. This will have cost-savings for the NHS and time-savings for the individual. In addition, some of that training may be offered via mobile device, such as theory-based topics, or legislation updates, although inevitably some may be better suited to face-to-face, such as the practical skill of resuscitation. Ref: findings 4.2.4.16 Statutory and mandatory training; 4.2.3.9 Intelligent pre-assessment; 4.2.4.13 Time saving for refresher training; 4.2.4.14 Impact; 4.2.4.1 For theory-based topics; 4.2.4.2 For legislation updates and 4.2.4.10 For part of a blend

Recommendation 11: Offer quick-fire practice mobile assessments/quizzes in between the required refresher intervals for Statutory and Mandatory knowledge and skills. This may take the form of multiple38

choice text or image questions. These would not only enable learners to maintain their knowledge and skill in between training, but also allow them to stay up-to-date with any new policy or protocols, as well as repeat self-assessments, aim to improve scores and track their own level. This would make it less burdensome to undertake the full assessment at the appropriate refresher interval and is likely to reduce training needs at that time. Ref: findings 4.2.1.22 Easy repeating self-assessment; 4.2.4.3 For easy reference and 4.2.4.11 For assessing essential knowledge through quizzes

Recommendation 12: Improve access to e-Portfolios through a well-designed mobile interface. This would address the complaint from some staff that they struggle to gain access via computer terminals on wards, and then when they do gain access the download/upload speeds can be very slow. The interface would need to be checked and possibly updated to display in a mobile-friendly way. Ref: findings 4.2.3.14 Improved access to e-Portfolios and 4.2.6.9 Identifying new apps

Recommendation 13: Use mobile devices to validate learner identity and avoid cheating on important assessments. For mobile devices with an in-built camera, photographs can be taken at random intervals, to ensure it is indeed the correct staff member undertaking the assessment. The photograph could be matched against the one on record for staff security badges. This would, of course, require the staff member being made aware that their identity was being validated in this way, and recommended to undertake the assessment in a reasonably lit space. Where anyone objected to this means of assessment, they should be offered a traditional classroom-based assessment with trainer present. Ref: finding 4.2.2.10 Potential for dishonesty

Recommendation 14: Ensure equality of access by offering some mobile devices ‘on loan’. This would be essential not only for those staff without mobile devices, but also for those staff who have mobile devices without the appropriate functionality. For example, where a member of staff does not have a mobile device with a camera, but would like to collect photographic or video evidence for assessment. Ref: findings 4.2.2.12 Inequality of access and 4.2.5.6 Lending learners devices

Recommendation 15: Have clear policies with regard to mobile learning. This relates to ethics when collecting data on mobile devices, in particular where photographic and video evidence is being collected for assessment purposes, or written ‘stories’ of patients are being shared. (There are already good examples of ethical guidelines having been put in place, for example by the Royal College of General Practitioners.) It also entails updating current HR policy to encompass mention of mobile learning with regard to flexible working and learning that is done outside contracted hours of work. Ref: findings 4.2.2.9 Appropriateness when capturing evidence; 4.2.3.13 Assessment by collecting photographic/video evidence; 4.2.4.12 [For Statutory and Mandatory training]: For assessing essential skills by collecting evidence; 4.2.4.7 For video assessment and 4.2.2.21 HR policy

Recommendation 16: Change the delivery mechanism for trainer support. Instead of trainers’ expertise being used to offer solely face-to-face feedback to individuals, they should also offer timely qualitative feedback via mobile devices to mobile assessments, thus enabling rapid remediation. Ref: findings 4.2.1.10 Immediacy of assessment feedback and 4.2.1.23 Easy contact with mentors

39

Recommendation 17: Offer face-to-face training in the use of the mobile devices, and technical support thereafter. There was strong opinion that face-to-face training in the use of mobile devices for learning and assessment was essential for all staff and supported by documentation on the mobile devices themselves to encourage use and familiarity. Then, there should be ongoing technical support. Of course, this may become a challenge where there are many different mobile devices in use. However, the training and technical support could be easily restricted to only those mobile devices supplied by the NHS (i.e. the Blackberry, HTC and Nokia) and the most popular devices known to be personally owned by staff (e.g. Apple iPhone, iPod touch and certain Google/Android smart phones). Ref: findings 4.2.1.11 Lack of skill with mobile technology and 4.2.2.13 Technical support requirement

Recommendation 18: Design specifically for the small screen. Obviously, small screens limit the amount of information that can be viewed at one time. This does not mean that content cannot be carefully treated to offer easy navigation and access to a lot of information, nor take advantage of some of the pedagogical techniques that can be successfully employed through multi-media to aid conceptual understanding and learning. Concerns about learning on small screens will only be overcome by learners being offered good design for mobile devices, be it only for display on a mobile device, or for display on laptop, but formatted for mobile-friendly access too. Ref: findings 4.2.2.6 Preference for laptops/PCs over mobile; 4.2.2.7 Screen size and 4.2.2.16 Conversion of existing e-learning to mobile

Recommendation 19: Carefully consider requirements to input. Where asking learners to interact, they will need to input in some way. If this is simply touching the screen with finger or stylus to select, perhaps a rating for a skill, then there is no problem. However, if it requires entering text or commenting, then one must be mindful of the limitations of using a keyboard on many mobile devices and undertake user testing (specific to each common mobile device) to determine where the tolerable limit lies. Of course, there are other means for inputting data other than text, such as recording reflections. Ref: findings 4.2.2.14 Input limitations; 4.2.3.12 Assessment by rating skills and commenting on them and 4.2.4.4 For recording reflections

Recommendation 20: Always undertake small-scale pilot studies that entail opportunities for users to give feedback on prototypes and engage in co-design, and also enable any connectivity or other technological glitches to be identified and resolved. Ref: finding 4.2.3.20 A need for pilot projects

Recommendation 21: A communications campaign is put in place to raise awareness among NHS staff and key stakeholders of the multitude of benefits to mobile learning. Not only will this overcome some of the challenges identified, such as a lack of understanding of its value (due to the relative little experience with mobile learning to date), and limit the poor perception of e-learning transferred to mobile learning, but also it may encourage those who already own mobile devices to explore using existing mobile resources to support their work-placed learning. Ref: findings 4.2.1 Benefits of mobile learning; 4.2.1.26 Wide reach; 4.2.2.1 Little or no experience of mobile learning and 4.2.2.2 Reluctance to accept mobile learning

Recommendation 22: Host a bi-annual m-Health symposium with key people working at the forefront of research and delivery of mobile health, and who attend mobile learning conferences across the world, to 40

ensure the NHS is able to exploit mobile learning to benefit staff professional development, both in current times and into the future. Ref: findings 4.2.3.21 The future of mobile and 4.2.6.4 NHS IT may inhibit mobile learning design

Project Objective B: Make mobile technology recommendations based on the outcomes of research objectives 3 and 4 Section 5.2 below addresses project objective A

5.2 Mobile technology recommendations Recommendation 23: Commission mobile learning and assessment that works across devices, and at the very least on the most popular ones distributed by the NHS, and personally owned by its staff. This will lessen barriers to access. Therefore, at present, this would include Blackberry, Nokia, HTC (Android) and Apple devices at a minimum. Note that the main cost should be the initial design and development of the mobile content for a single platform, with a small additional cost for publication to each subsequent platform. Ref: findings 4.2.5.1 Job roles supplied with a mobile phone by the NHS; 4.2.5.2 Blackberry, Nokia and HTC (Android); 4.2.5.4 Apple, Blackberry, Android phones and Nintendo DS are the most common mobile devices available to NHS staff inside and outside work; 4.2.5.5 Learners access via their own personal devices; 4.2.6.8 Content across mobile platforms and 4.2.2.20 Costs

Recommendation 24: Ensure all mobile learning and assessment is also accessible through a PC. This gives users maximum flexibility with regard to how they access the learning. It is also easier to design and develop for the mobile to work on a PC rather than to develop for a PC and convert to mobile. Ref: finding 4.2.2.17 Device agnosticism

Recommendation 25: Design all mobile learning and assessment with an offline capability that doesn’t mean tracking data is lost if connectivity is lost. This means that users tracking information must be stored on the mobile device and then synchronised with the LMS when a connection becomes available, thus overcoming any problem with connectivity via 3G or wireless. Ref: findings 4.2.2.8 Lack of confidence in the reliability of mobile technology; 4.2.2.22 Interfacing with IT systems; 4.2.6.1 The NHS IT infrastructure; 4.2.6.2 Tracking learners on NHS IT infrastructure; 4.2.6.3 Feeding information back to NLMS; 4.2.6.6 Wireless connectivity and 4.2.6.7 Other ways to connect

Recommendation 26: Create a mobile learning app, where all the assets for the training are loaded on to the mobile device, rather than use the mobile web browser. This has five key benefits: i. It overcomes the problem of the LMS having Java-based SCORM APIs, which will not work on mobile web browsers. Ref: findings 4.2.2.22 Interfacing with IT systems; 4.2.6.1 The NHS IT infrastructure; 4.2.6.2 Tracking learners on NHS IT infrastructure and 4.2.6.3 Feeding information back to NLMS

ii. It overcomes the problem of tracking data going astray, which is what would happen if learning and assessment was delivered through a mobile web browser and the network connectivity was lost. Ref: findings 4.2.2.22 Interfacing with IT systems; 4.2.6.6 Wireless connectivity and 4.2.6.7 Other ways to connect

41

iii. The performance of the app on the mobile device will give a richer experience, because it can include larger amounts of animation and media than it can through a mobile web browser. iv. If the app were uploaded to an App store (for example Google Market, Apple App Store or Blackberry App World), then access could be limited to NHS staff only by including some basic security mechanism in the app. In addition, there would be no download limitations within the NHS due to server load. This means that larger amounts of video, audio or animation could be embedded in NHS apps without having to adjust the existing learning delivery server infrastructure to cope with the increase in bandwidth. v. If all asset files are stored on the mobile device, then only the tracking would need to synchronise with the servers when connected to the network, so the load on NHS servers would be limited (estimated to be a few kilobytes per user). Some may consider it a drawback that this solution does not enable existing e-learning to be delivered to mobile devices without some development work. However, it is not considered good practice, nor recommended, to simply take existing e-learning and offer it through a mobile web browser on a mobile device, anyway. (See Recommendation 18: Design specifically for the small screen and Recommendation 19: Carefully consider requirements to input.) Ref: finding 4.2.2.16 Conversion of existing e-learning to mobile

Recommendation 27: Consider recouping costs of some mobile learning by uploading two versions of the same app to the app stores: one that requires a password and is accessible for free by NHS staff, and another paid-for version that is available for download by the general public or those working in healthcare, but outside the NHS or UK. Ref: finding 4.2.2.20 Costs

Recommendation 28: Maintain network security by allowing only approved devices to connect to the NHS internal networks and in some instances only when on an NHS site. This draws on good practice across the NHS and other organisations. E-learning content, which sits on a separate domain to the main NHS IT systems, is available over the internet anyway, and is considered low risk to be accessed from anywhere. The same would be true of any NHS content on an app store. However, where data is of a more sensitive nature, such as scores for important assessments, then it could be fed back in a secure way to the system only from NHS sites, and would sit on the mobile device until the learner is able to connect to the network. Users should be encouraged to use a pass code on their mobile devices to protect that data in case of loss or theft. Ref: finding 4.2.2.18 Security

Recommendation 29: Have clear policies with regard to the use of mobile devices. The policy should be clear on those specific areas of NHS sites where mobile devices prove a real risk by interfering with other electronic equipment. However, this should not apply universally across NHS sites regardless. Ref: findings 4.2.2.19 Interference with other equipment; 4.2.3.17 A need for a mobile learning strategy

42

Final comments by Professor John Traxler, Professor of Mobile Learning The findings and recommendations are laid out with considerable clarity and seem sound, robust and sensible; in fact they take us considerably further than before within their particular professional and organisational context. They do however represent only a snap-shot of a very rapidly evolving and expanding discipline. Mobile learning itself is changing rapidly and many of the challenges and hurdles recognised in this report are being addressed and resolved. The report as whole is a snap-shot taken at a time characterised by rapid technical and social change, as powerful and highly functional mobile phones become ubiquitous and pervasive. Consequently, reading the trends and the generalities, and the interactions between them is a continuing challenge. Moreover I think we are near a critical point in the evolution of mobile learning. Until very recently mobile learning represented one of the leading edges of e-learning within organisations such as universities, colleges and schools and was driven and deployed by these organisations within the organisations. Increasingly however mobile phone ownership, access and familiarity are nearly universal across the whole population and learners’ attitudes and expectations of mobile learning will increasingly be driven not by those inside their organisation but by those of the wider social world, from the family, the community, the peer group and the sub-cultures. For any organisation this issue will become increasingly important, as the context changes from deploying a specialist dedicated technology and pedagogy from within the organisation to attempting to appropriate and exploit a commonplace social phenomenon. This opens up enormous possibilities but complex challenges. Looking at the same issue from a different perspective, all of our experience warns us against treating mobile learning as merely a continuation or extension of e-learning; this may have been its genesis and its history but it will not be its future, nor will it realise its full potential. The report is right to explore and question the relationships between mobile learning and e-learning. I would advocate a clear conceptual distinction between mobile learning and e-learning, not just for the objective differences in such areas as procurement, technical support, infrastructure or usability but for the less tangible differences in perceptions of ownership, control, attitude, social significance and patterns of use. Learners will come to mobile learning with clear expectations understandably imported from e-learning but also with a range of expectations coming from their experience of the social, recreational, informal and personal use of mobile devices. The report is also valuable in drawing attention to the evolving continuum between mobile learning, that is enhancing and developing skills and knowledge before an activity, and mobile performance support, that is enhancing skills, knowledge and decision-making in the course of an activity.

43

Appendix A: Research timetable Dates

Activity

Responsibilities

Research methodology planning By Mon 8 Nov 2010

Draft research plan written and submitted to NHS South Central

Dr Naomi Norman, Director of Learning, Epic

By Wed 10 Nov 2010

Research plan comments/clarifications to be fed back to Epic

Alison Potter, E-learning Programme Manager; Helen Bingham, Libraries & E-Learning Lead, NHS South Central

By Fri 12 Nov 2010

Research plan finalised and submitted to NHS South Central

Dr Naomi Norman, Epic

Desk research: literature review undertaken

Dr Naomi Norman, Epic

By Mon 29 Nov 2010

Draft survey questions written and submitted to NSH South Central

Dr Naomi Norman, Epic

By Mon 29 Nov 2010

Draft survey emails (and paper-based communication written for those without email addresses) submitted to NHS South Central

Dr Naomi Norman, Epic

By Mon 29 Nov 2010

Sample of 270 survey participants identified

Alison Potter and Helen Bingham, NHS South Central

By Wed 1 Dec 2010

Survey questions and emails finalised

Alison Potter and Helen Bingham, NHS South Central

By Fri 3 Dec 2010

Survey questions peer reviewed by Oxford University

Dr Chris Davies, Lecturer in Educational Studies, Oxford University

On Fri 3 Dec 2010, am

Email/Letter 1 to be sent to survey participants

Alison Potter and Helen Bingham, NHS South Central

Desk research Mon 15 – Mon 22 Nov 2010

Professor John Traxler, Wolverhampton University

Data collection

(Note any participants whose emails bounced should be sent letters) 44

On Mon 6 Dec 2010

Amends (highlighted by peer review) are made

Dr Naomi Norman, Epic (with agreement by NHS South Central)

On Tue 7 Dec 2010

Email/Letter 2 to be sent to survey participants

Alison Potter and Helen Bingham, NHS South Central

By Wed 8 Dec 2010

Draft Focus Group plan submitted to NHS South Central

Dr Naomi Norman, Epic

By Thu 9 Dec 2010

Focus Group plan finalised

Alison Potter and Helen Bingham, NHS South Central

By Fri 10 Dec 2010

Focus Group plan peer reviewed by Oxford University

Dr Chris Davies, Oxford University

By Fri 10 Dec 2010

Telephone interviewees identified

Alison Potter and Helen Bingham, NHS South Central

On Mon 13 Dec 2010

Amends (highlighted by peer review) are made

Dr Naomi Norman, Epic (with agreement by NHS South Central)

On Wed 15 Dec 2010 2pm – 2.40pm

Focus Group – National and SHA ELearning Leads

Arranging: Alison Potter and Helen Bingham, NHS South Central Planning: Dr Naomi Norman, Epic Running: Dr Naomi Norman, Epic (Lists of participants and their job roles to be provided to Epic prior to 15 December)

On Wed 15 Dec 2010

Email/Letter 3 to be sent to survey participants

By Fri 17 Dec 2010

All telephone interviews scheduled for Alison Potter and Helen Bingham, week of Thu 13 January or week of 17 -2 NHS South Central 1 January

By Fri 17 Dec 2010

Draft telephone interview schedules submitted to NHS South Central

Dr Naomi Norman, Epic

By Tues 21 Dec 2010

Telephone interview schedules finalised (for Thurs 13, or Mon 17 – Fri 21 January)

Alison Potter and Helen Bingham, NHS South Central

On Wed 5 Jan 2010

Email/Letter 4 to be sent to survey participants

Alison Potter and Helen Bingham, NHS South Central

On Thu 6 Jan 2010

Telephone interviews confirmed (as reminders for participants)

Alison Potter and Helen Bingham, NHS South Central

45

Alison Potter and Helen Bingham, NHS South Central

On Fri 7 Jan 2011

Survey closes

Dr Naomi Norman, Epic

By Wed 12 Jan 2010

Telephone interview schedules peer reviewed

Dr Chris Davies, Oxford University

By Fri 14 Jan 2010

Amends (highlighted by peer review) are made

Dr Naomi Norman, Epic (with agreement by NHS South Central)

On Fri 14 Jan 2011 4pm – 5pm

Focus Group –NHS South Central Trust E-learning Leads.

Arranging: Alison Potter and Helen Bingham, NHS South Central Planning: Dr Naomi Norman, Epic Running: Dr Naomi Norman, Epic (Lists of participants and their job roles to be provided to Epic prior to 14 January)

Thurs 13 and Mon 17 – Fri 21 Jan 2011

Telephone interviews take place

Dr Naomi Norman, Epic Marcus Boyes, Head of Mobile, Epic

Data analysis By Mon 14 Feb 2011

Data analysis

Dr Naomi Norman, Epic

By Mon 28 Feb 2011

Write up

Dr Naomi Norman, Epic

By Fri 4 Mar 2011

Findings/recommendations peer review

Professor John Traxler

Report submitted to NHS South Central

Dr Naomi Norman, Epic

Report writing

Delivery By Fri 11 Mar 2011

46

Appendix B: Findings from desk research Reference 1: Andrews, T., Smyth, R. (2010) ‘Utilizing Students’ Own Mobile Devices and Rich Media: Two Case Studies from the Health Sciences’, 2010 Second International Conference on Mobile, Hybrid and On-Line Learning – Conference Proceedings, pp. 71-76. Paper summary: This paper describes two case studies that explore University of Queensland (Australia) Health Science students’ use of technology to support their learning. The first of these case studies involves a clinical educator who recorded short vodcasts on voice therapy techniques. These were uploaded to the course Blackboard site for students to download to their mobile devices. Students were encouraged to use their mobile devices not only to view the vodcasts, but also to record their clinical practice sessions and their own reflections. Key findings: 1.

The viewing of the vodcasts, particularly just before clinical practice activities, improved students’ confidence in dealing with patients. (p.74)

2.

Using the mobile devices to make voice recordings enhanced students’ reflections and enabled better reviewing of activities. (p.74)

3.

There were some issues with downloading the vodcasts from Blackboard site to mobile devices, due to file formats and sizes. (p.74)

4.

The provision of technical support was essential to overcoming difficulties with the technology. (p.74)

Reference 2: Clay, C. (2010) ‘Exploring the use of mobile technologies for the acquisition of clinical skills’, Nurse Education Today, 8 October 2010 [Epub ahead of print]. Paper summary: This paper describes a small-scale research study in the Faculty of Health and Life Sciences at Coventry University. The study explored the use of iPods to assist the teaching and learning of the Newborn Infant Physical Examination (NIPE). Small video files were loaded on to the iPods, each of them outlining an aspect of the physical examination, and then they were given to eight student midwives. The midwives were encouraged to take them into the clinical area with them and, if necessary, use the device synchronously whilst performing the examination. Key findings: 1.

The student midwives found it easy to access the iPods in the clinical areas, or on their way to work, on a train, or a bus, or at any time convenient to them. (p.4))

2.

The student midwives liked the control they had over their learning and reported that they felt 'empowered to learn'. (p.4) 47

3.

The study found that 'the flexibility of where the learning could take place enhanced their *the student midwives’+ acquisition of... skills required for NIPE'. (p.4)

4.

Where midwives used the iPods in the clinical settings, they reported benefiting from being able to watch the short video file on a specific competence and then perform the skill for themselves, subsequently resulting in an increase in confidence. (p.4)

5.

The midwives reported that following a newborn examination they gained immediate reassurance in their professional judgements, as they could access the video clips on the mobile device immediately. (p.4)

6.

Parents were receptive to the mobile devices being used synchronously whilst performing the NIPE. (p. 4)

7.

An identified constraint was the inability to run 'flash'- based learning objects on the iPod. (p.4)

Reference 3: Luanrattana, R., Than Win, K., Fulcher, J. and Iverson, D. (2010) ‘Mobile technology use in medical education’, Journal of Medical Systems, 13 April 2010 [Epub ahead of print]. Paper summary: This paper explores the use of PDA mobile devices for a problem-based learning medical curriculum at the Graduate School of Medicine, University of Wollongong in Australia. The curriculum involves students learning by solving 93 different clinical problems during their four years of medical study. The PDAs were used by students for clinical logs, references, communications and personal organisation (such as for contact lists, reminders and so on), while on clinical practice placements. Key findings: 1.

The ability for students to use the PDAs to record clinical experiences at the bedside, which they could later reflect upon to identify any learning gaps, and use for assessment purposes, encouraged a good habit – not just for medical study, but also for life-long learning in their future professional careers. (p.4)

2.

The upload and transfer of clinical logs enabled the medical school to regularly monitor students’ progress and offer supervision where required. (p.4)

3.

The ability to reference material at the bedside was found to support students’ clinical reasoning and decision making. (p.5)

4.

Internet access was useful for online courseware, audio- stream lectures, clinical guidelines and policies, library resources, PDA-based instructional tools, technological simulations for medical and clinical studies, and online journals. (p.6)

5.

The PDAs facilitated interpersonal communication among peers, for example via online discussion groups, emails and text messages. They were also found to instigate more timely and relevant feedback from tutors. (p.6) 48

6.

Data security entailed putting in place password protection, data encryption, virus protection software, identification and frequent backing up of data. (p.7)

7.

Wi-fi connectivity was not available in some areas of hospitals or clinical placement. In these cases, a PDA docking station was used to upload data. (p.7)

8.

Maintenance and support was essential, and worked best when it involved collaboration between the Graduate School of Medicine, the University's IT unit and the educational technology unit. (p.7)

9.

Interoperability ensured students with existing PDAs or other mobile devices (such as smart phone or tablet PC) would not need to acquire a new or additional device. And for those who did not own a PDA, it offered the flexibility for them to select a PDA which best suited their needs and budget. (p.8)

10. It was considered important to use a solid platform, not dependent on new fashions and trends that would ensure interoperability in the future. (p.8) 11. Pre-education and training regarding PDA use was offered at an early stage. This not only included the use of the technology, but also the aims, objectives and benefits for students using PDAs in their medical study. (p.8) 12. The community were educated regarding the purposes of using PDAs, to avoid any negative reaction to them. (p.8) 13. There was concern that PDAs may generate electromagnetic interference with medical equipment. However, this was only considered to be a problem for certain areas of hospitals, such as cardiac and intensive care units, where local policies and regulations were already in place to prohibit their use. (p.8)

Reference 4: Maguire, R., McCann, L., Miller M. and Kearney, N. (2008) ‘Nurses’ perceptions and experiences of using a mobile-phone-based Advanced Symptom Management System to monitor and manage chemotherapy-related toxicity’, European Journal of Oncology Nursing, vol. 12. pp.380-386. Paper summary: This paper reports on the perceptions of 35 nurses who participated in a UK-wide trial that assessed the use of mobile phones for home monitoring and symptom management of cancer patients receiving chemotherapy. Patients completed questionnaires on a mobile phone twice a day and at any time they felt unwell. The information was sent immediately via a secured GPRS connection to a nurse, who carried a dedicated 24-hour pager system. Nurses received an amber alert on their pagers if a patient questionnaire reported symptoms in which early intervention may be advisable and a red alert where a patient questionnaire reported symptoms that were severe or life-threatening. Patients also were able to use the mobile devices to access self-care advice, such as how to cope with nausea. Key findings : 1.

While new technologies offer solutions to many of the obstacles faced by current healthcare systems, the perceptions of the key users are vital to their development and successful implementation within clinical practice. (p.381; p.385) 49

2.

Nurses involved in the study attended a designated training day on the use of the system. Anyone unable to attend received ‘in-house’ training from staff members who had attended the day, but this cascading of training was deemed unsatisfactory. (p.381; p.383)

3.

Beforehand, the majority of nurses felt that the mobile devices would result in better symptom management. This view was heightened post roll-out. Only four of the 35 nurses felt that the mobile devices had had no impact on symptom management. (p.382-383)

4.

Prior to using the mobile devices, the majority of nurses had an expectation that their use would be challenging, yet valuable and educational. (p.382-383)

5.

The majority of nurses felt that the use of the mobile phones to instigate alerts were a worthwhile feature and had been successful in identifying where early detection of symptoms would prevent subsequent hospitalisation. (p.383-384)

6.

Beforehand, most of the nurses expressed concern that the use of mobile devices would impact on their workload. Fewer expressed the same concern having used the devices. (p.383)

7.

Both nurses and patients were positive about the self-care information on the mobile devices, stating that ‘It’s been very good and easy to read.’ (p.384)

Reference 5: Prgoment, M., Georgious, A. and Westbrook, J. I. (2009) ‘The Impact of Mobile Handheld Technology on Hospital Physicians’ Work Practices and Patient Care: A Systematic Review’ Journal of the American Medical Informatics Association, vol. 16, no. 6, Nov-Dec, pp.792-801. Paper summary: This paper offers a systematic literature review on the impact of mobile devices. The authors chose solely quantitative studies conducted between years 2000 and 2007 and involving only hospital physicians. Applying some further minor criteria, such as comparative group methodology and the inclusion of usage surveys, the search for studies was whittled down to 13 for inclusion in the paper: six were from the US and seven were from the UK, Australia, Canada, Denmark, Finland, Germany and Hong Kong. Key findings: 1.

The authors acknowledged that as physicians move between patients, wards, clinics, laboratories, operating theatres and offices, they require easy access to data and resources. While a computer may offer the access, it does not necessarily support the mobility, and while paper offers the mobility, it does not support simultaneous and seamless access by multiple users. Mobile technology addresses the shortfall of both computers and paper. (p.792)

2.

Inter-professional communication and coordination of patient care was improved by the use of mobile devices, over the use of pagers. (p.795)

3.

Physicians more frequently accessed information to support decision-making when the information was available on a mobile device compared with when it was offered to them on paper. (p.795) 50

4.

Input on to mobile devices was found to be of greater accuracy than with pen and paper, where illegibility was sometimes a problem. (p.795-796)

5.

Prompts to ensure all mandatory data were entered meant the information on mobile devices was more complete. (p.795-796)

6.

Data entry at the point of care, often at the bedside, with mobile devices, was also found to lead to more complete documentation. (p.797)

7.

Some form of user training was provided in seven of the 13 studies. (p.797)

8.

There was evidence that entry via a stylus was slower, more erroneous and less satisfactory for users than entering data via a QWERTY keyboard. (p.797)

9.

The benefits of handheld devices to hospital physicians were found to be greater when connected to the internet and to the hospital’s electronic information systems. (p.799)

Reference 6: Ruffner, J. W. and Deibler, N.P. (2010) ‘Ensuring Mobile Devices Deliver Mobile Support: Are We There Yet?’, The 6th International Scientific Conference E-learning and Software for Education: Bucharest, April 15-16, 2010 Paper summary: This paper explores the literature on the use of mobile devices for task-relevant instruction or knowledge in remote work environments, where desktop or even laptop computers are impractical. In particular, it focuses on the critical issues and implications for delivering content via mobile devices for learning (m-learning) and for performance support (m-support). Key findings: 1.

Mobile workers frequently require quick, accurate and timely access to relevant instruction or knowledge to complete tasks in their work environments. This was where mobile learning and mobile performance support were deemed to add value. (p.2)

2.

Mobile performance support can deliver up-to-date support or step-by-step procedures to mobile devices, thus enabling already trained workers to perform faster and more efficiently. This obviously has cost-saving implications. (p.4)

3.

Mobile performance support can enable non-experts to perform closer to the level of experts and limit error rates; it can also enhance the competence of an employee beyond the level of his or her training. (p.4)

4.

Mobile performance support was deemed particularly beneficial in complex, infrequent or changing situations. (p.4)

5.

Mobile performance support was found to offer consistency across practitioners where there is little time for training or frequent staff turnover. (p.4)

51

6.

Characteristics of mobile devices to consider included: display dimensions and resolution, user interface features (e.g. touch screen, hard/soft keyboard, tiltsensing mechanism), processor speed, connection protocols and speed, photo capture, video capture and playback capability and battery life. (p.6)

7.

Good network connectivity was considered essential. (p.7)

8.

A screen at least the size of a business card was considered appropriate for learning and performance support, depending on the nature of the task, and whether the user could zoom, along with the characteristics of the user's environment, for example the intensity and colour quality of the ambient light. (p.7)

9.

Given the ubiquitous nature of mobile devices, users were often observed engaged in other tasks while operating the devices, like writing notes on a piece of paper. This led to the suggestion that one-handed use is a desirable option. (p.8)

10. The limited screen size means only a small amount of content can be presented at one time. Therefore, there must be one finger touch scrolling and paging and two finger touch zooming to improve access to additional viewable content. (p.8) 11. Augmented reality, that uses the camera in a mobile device to superimpose relevant information on to a real-world scene, was considered a technology ‘to watch’ in future as a means of enhancing m-learning and m-support. (p.10)

Reference 7: Ruskin, J. K. (2010) ‘Mobile Technologies for Teaching and Learning’ International Anaesthesiology Clinics, vol. 48, no. 3, pp. 53-60. Paper summary: This paper explores the benefits of mobile learning for anaesthesiologists who work in a dynamic environment, in which information critical to patient care must be quickly and accurately exchanged. Key findings: 1.

Mobile devices could provide anaesthesiologists with rapid access to vital information from any location. (p.53)

2.

Mobile devices with wireless connectivity enabled users to go online for the information they required, while offline they could still use a medical calculator or charting tool, as well as store information for upload when connected. (p.53)

3.

Physicians in training have already widely adopted mobile devices to access commercial databases, medical texts, their appointments and calendars. (p.54)

4.

Physicians using mobile devices to access vital information at the bedside were shown to be more likely to correct a patient’s diagnosis or treatment. (p.54)

52

5.

The most popular mobile reference tools for anaesthesiologists include Stoelting’s Anaesthesia and Co-Existing Disease, the 5-Minute Clinical Consult and the Anaesthesia Drug Handbook. These were respected information sources because they are updated regularly, are easy to search and they enable drug interaction checks. (p.54)

6.

There was some concern about variability between interaction screening programmes and boxed drug interaction warnings. (p.55)

7.

Mobile devices were shown to influence clinical decision making. (p.55)

8.

Evaluations on mobile devices enabled learners to determine in real time whether their educational needs had been met. (p.55)

9.

Using mobile devices to create real-time evaluations was found to encourage learners to take the initiative to direct their own learning and evaluations. (p.55)

10. Real-time electronic evaluation enabled rapid feedback and speedier remediation, and so improved performance. (p.55) 11. Podcasts allowed physicians to get an overview of the latest research, or review a block technique, or listen to a lecture during time that would otherwise be unused. These podcasts may be audio only, or perhaps be accompanied by slides, graphics or video. (p.56) 12. The results of at least one study reported on mobile telephones decreasing the incidence of medical errors in operating rooms, because they made it easy to communicate with peers and other experts. (p.57) 13. Wi-fi access points can be used to provide a core communication infrastructure. (p.58) 14. Many healthcare institutions were found to have policies prohibiting the use of mobile communication devices because of the possibility that they may interfere with medical equipment. However, these fears were deemed largely unfounded, because modern medical equipment is designed to be compatible with radio-frequency energy, and modern communication devices use very low-power transmitters. (p.58) 15. A common cause for complaint was battery life, though it was recognised that this was getting better with newer devices. (p.58)

53

Reference 8: Dearnley, C., Haigh, J. and Fairhall, J. (2008) ‘Using mobile technologies for assessment and learning in practice settings: A case study’, Nurse Education in Practice, vol. 8, no.3, pp. 197-204.

Reference 9: Sandars, J. and Dearnley, C. (2009) ‘Twelve tips for the use of mobile technologies for work based assessment’, Medical Teacher, vol. 31, no.1, pp. 18-21.

Reference 10: Coulby, C., Davies, N. and Dearnley, C. et al (2010) ‘Assessment and Learning in Practice Settings (ALPS) – Implementing a large scale mobile learning programme: A Report. Paper summary: These papers report on Assessment and Learning in Practice Settings (ALPS): a collaborative programme between five Higher Education Institutions – University of Bradford, University of Huddersfield, University of Leeds, Leeds Metropolitan University and York St John University. The programme was built on the broad educational premise that assessment, whether formative or summative, helps build learners’ understanding and knowledge of their performance, as well as encouraging reflective practice. The focus of this particular programme was student assessments using mobile devices when undertaking work placements for a range of health and social care degree courses. Key findings : 1.

Mobile learning is a fast-growing area, and assumptions cannot be made that it will remain a novelty for long. (Ref 10: p.12)

2.

Mobile technologies offered opportunities for the collection of evidence for assessment that would be otherwise easily missed. The nature of healthcare work means that events can change quickly, particularly in clinical practice, but the constant availability of mobile devices meant such events could be easily captured in the moment. (Ref 9: p.18)

3.

Mobile devices should be chosen so that they do not in any way limit the assessment strategy. In fact, the authors believed they should go beyond the current assessment strategy, to take account of those users building confidence over time, and wishing to explore new ways for collecting assessment evidence. (Ref 9: p.19)

4.

Assessment approaches should be developed before adopting templates. For example, even if questions required a choice of only two answers (yes/no or true/false), templates were built to offer multiple responses that could in future cover a range of learner misconceptions or typical errors. (Ref 9: p.19)

5.

The limitations of collecting free comments were recognised, in particular with regard to the usability of the device keyboard and the size of its screen. Where free text was essential, users seemed to prefer traditional QWERTY keyboards on screen, rather than any kind of stylus input (Ref 8: p. 201; Ref 9: p.19) 54

6.

Wireless transfer was found to be far preferable and more convenient for users over cable or docking device. (Ref 9: p.19)

7.

No assumptions were made about users’ capabilities with mobile devices, and so training was offered to all. The authors found that just because a user was young, or adept with one mobile device, this did not mean they were able to apply their technical knowledge to a different mobile device supplied for assessment. (Ref 9: p.19)

8.

Training in mobile devices worked best when it was a mixture of face to face and online, with face to face encompassing lots of mobile use, and all documentation offered as mobile versions to encourage use and familiarity. (Ref 9: p.19) This initial training was considered crucial. (Ref 10: p.22)

9.

A ‘one-stop’ support service was provided to ensure users could easily get help. This was very much appreciated by users. (Ref 9: p.20)

10. The authors found that many healthcare settings overestimated the risk from mobile technologies interfering with vital electronic medical equipment. Best practice guidelines were developed with healthcare providers, and led to the banning of mobile devices only in those areas perceived as high risk, like critical care units. (Ref 9: p.19) 11. It was important to engage early with IT departments to identify, negotiate and overcome potential problems, such as firewalls that may impede data transfer, or out-of-date policies that may prohibit the use of mobile devices in certain environments. (Ref 9: p.20) 12. The authors suggested user acceptability testing as part of a pilot before widespread roll out, seeking feedback on device size, compactness, battery life, screen size, colour quality, method of data entry and data transfer. (Ref 9: p.20) 13. The more time users spent with the devices, the more relaxed and accepting they were of them. Therefore, users’ time with the devices should not be limited, even if they are ultimately for a single assessment or short learning module. (Ref 9: p.20) 14. It was important to make users fully aware of ethical guidelines when collecting photographs or video recordings for assessment evidence (such as provided by the Royal College of General Practitioners. (Ref 9: p.20) 15. Ensure any staff that are ‘loaned’ mobile equipment understand their responsibility to take care of it, and the conditions under which it is lent. (Ref 10: p.67) 16. There needs to be a balance between identifying staff champions to lead and promote the use of technology and keeping less enthusiastic staff involved to ensure future embedding. (Ref 10: p.41)

55

17. It was necessary to be aware of the possible requirement for alternative means of assessment, if a user had a physical disability such as reduced vision or limited hand mobility. (Ref 9: p.20) 18. Individuals with dyslexia were found to more effectively engage with the assessments via mobile devices than those that were paper based. (Ref 9: p.20) 19. The authors raised awareness of the use of mobile devices for assessment and so minimised any false impressions about their use. This was done by publishing leaflets that clearly explained the purpose of the devices, along with regular meetings with a wide range of stakeholders, including assessors, IT technical staff, administrative staff, clinical staff, patients and carers. (Ref 9: p.20-21) 20. Assessment should always be closely aligned with teaching to ensure effective learning. One way that this could be achieved is by offering an ‘assessment sandwich’ that includes some revision or learning as pre-assessment and a review for reinforcement as post-assessment. (Ref 9: p.21) 21. Pilot projects should be undertaken at a local level and intensively supported to secure engagement and the best possible roll-out. (Ref 10: p.22) 22. Mobile learning as part of IT infrastructure should be piloted and tested prior to live trials. (Ref 10: p.12) 23. It is essential to understand and recognise the external systems with which the mobile learning solution will need to interact. (Ref 10: p.33)

Reference 11: Zhang, P., Millard, D. E., Wills, G. B., Howard, Y., Faulds, S. J., Gilbert, L. And Sparks, D. (2008) ‘A Mobile Toolkit for Placement Learning’, Advanced Learning Technologies 2008. ICALT 2008 Eighth IEEE International Conference on Advanced Learning Technologies, Santander, Cantabria, 1-5 July 2008, pp. 92-96. Paper summary: The Nursing and Midwifery departments at the University of Southampton, Thames Valley University and the Bournemouth and Poole College created a Mobile Placement Learning and Assessment Toolkit (mPLAT) on PDA phones (old versions of smart phones) to connect students’ competencies in practice while in work placements, with the competency network of skills and proficiencies they are assessed against. It also was intended to enable mentors to easily access and monitor students’ progress. Key findings: 1. A co-design process involved users developing personas and scenarios that highlighted their problems with existing paper-based systems. (p.93) 2. The mobile tool was not designed to simply mimic the paper-based version it replaced, but rather to offer personalised functionality that enhanced its use. For example, students could use the mobile tool to rate their skills and proficiencies and add comments, outputting a personalised action plan. (p.94) 56

3. The mobile tool enabled large amounts of information to be represented in a visual diagram. For example, a network diagram showed students’ strengths and weaknesses and how they inter-connected as part of the entire competency framework. (p.95) 4. Mentors could use web tools to get a quick and easy snapshot of students’ profiles and their progress, even when students were not present. (p.95) 5. Students did not only use the mobile phone for the mobile toolkit. They also made use of it to access internet resources, download documents, share notes and files, as well as keep in contact with their fellow students on placements elsewhere, all of which the authors believed contributed to better learning during the placement. (p.96)

Reference 12: Zolfo, M., Iglesias, D., Kiyan, C., Echevarria, J., Fucay, L., Llacsahuanga, E., de Waard, Inge, Suarez, V., Llaque, W. C. and Lynen, L. (2010) ‘Mobile learning for HIV/AIDS healthcare worker training in resource-limited settings, AIDS Research and Therapy, 2010 vol. 7, no. 35. [Available at: http://www.aidsrestherapy.com/content/pdf/1742-6405-7-35.pdf]. Paper summary: The Institute of Tropical Medicine Alexander von Humboldt in Lima, Peru and the Institute of Tropical Medicine in Antwerp (ITM) used mobile devices to deliver a three-month education program from November 2009 to January 2010. They provided 20 HIV/AIDs healthcare workers in Peru with smart phones (ten Nokia N95s and ten iPhones) equipped with solar chargers. The workers ranged in age from 35 to 55 and most had no prior mobile learning experience. Each phone used wireless to offer five learning modules and quizzes involving multiple choice questions to track progress. The phones also enabled access to Skype, to discuss advice with HIV specialists, and Google documents for document sharing. Key findings: 1. The project was set up based on the belief that mobile devices ‘play a major role’ in offering access to the latest medical information and in facilitating exchange of ideas and expertise on how to treat difficult clinical cases. This is especially true of developing countries. (p.1-2) 2. Existing learning modules on drugs and treatment for HIV/AIDs was re-designed in a mobile-accessible format. This adaptation of content for small screen size and for performance on different operating systems was considered to be essential. (p.2; p6) 3. Healthcare workers were taught in a relatively short timeframe (half a day) to use the mobile devices, to search, upload and download information. (p.4) 4. Mobile phones were liked by the healthcare workers for the ease with which they could access learning modules, and also for the ease of connecting with others, be it for making contact with a mentor, receiving immediate feedback, or building a network with colleagues. (p.4)

57

5. iPhones were found to be more user friendly than Nokia N95s in every respect: screen size, keyboard size, quality of image, touch screen versus digit buttons, and also ease of finding and accessing programmes. For example, in the feedback, 66.7% of iPhone users versus 22.2% of Nokia users found Skype ‘easy to access’. (p.3-4) 6. 86.6% of users believed mobile devices added value because they enabled the freedom to plan their own educational activities according to their individual personal learning requirements. (p.3) 7. 94.4% liked the portability and easy access offered by mobile devices compared with computers. (p.3) 8. The authors noted that the portability of the mobile devices meant they were always at hand around the clinics, bridging the gap between formal and more informal experiential learning. (p. 5) 9. Some unavoidable cost limitations were identified, particularly in relation to the purchasing of many mobile phones, service fees to operators and the need for an IT help desk to solve technical problems. (p.5)

58

Appendix C: Survey Notes

Questions Section 1: About you, your job, and your use of technology *1. The email/letter you received about this survey included a research participant number. Please enter it below.

Note: throughout, star * denotes required question. (For those who complete the survey online, they will not be able to skip these questions).

*2 Which of these Trusts do you work for? O Berkshire Healthcare NHS Foundation Trust O Buckinghamshire Healthcare NHS Foundation Trust O South Central Ambulance Service

Ref: Q2 and Q3 The research participant number should give us this information, but: 1. It is always a good idea to cross reference. 2. Collecting this information means we can guarantee participant anonymity – the independent researcher responsible for data analysis will not need to access the NHS list of names for data analysis, and the NHS will not have access to which particular participant number answered in a particular way.

*3 Which of these jobs best describes your role? O Community nurse O Community nurse manager O Ambulance worker O Ambulance worker manager O Junior doctor O Junior doctor manager O Community psychiatric nurse O Community psychiatric nurse manager O Administrator O Administrator manager O Radiographer O Radiographer manager O Other (please specify) ____________________

59

Ref Q4 and Q5 These may give useful data on the age and gender of those most likely to accept mobile learning, and so the kind of work that may need to be done in preparation for any roll out.

*4 What is your age range? O Under 25 O 26 – 35 O 36 – 45 O 46 – 55 O Over 55 *5 What is your gender? O Male O Female

Ref Q6 and Q7 This should give some indication of technical familiarity and knowhow.

*6 How often do you use a computer/laptop for work or for personal use? O Every day, seven days per week O Every weekday (Monday to Friday) O About twice per week O About once per week O About once per fortnight O About once per month O Less often than monthly O Never Please comment, if required.

*7 Are you a member of any online communities, such as Facebook, Linked-in or Twitter? Do you participate in chat rooms or write blogs etc? O No O Yes (please list which ones)

*8 Does the NHS provide you with a mobile device, such as a mobile phone? O No O Yes (please specify)

60

Ref Q9 I avoided asking ‘do you own...’ as I thought some people may be reluctant to answer honestly, fearful that they may be asked to use their personal device for work use.

*9 Do you use any of the following mobile devices either in or outside work? Mobile phone (with no internet access) Android/Google smart phone Apple iPhone Apple iPod Touch Blackberry Symbian/Nokia smart phone Windows mobile smart phone Apple iPad Other tablet Nintendo DS Mobile phone (with internet access, but I do not know what kind) Other (please specify) ______________ *10 Mobile technologies can be used to deliver information to you in the workplace or to assess your knowledge and skills, wherever you happen to work. How would you describe your experience with mobile learning until now? O Lots of experience O Some experience O A little experience O No experience Please comment, if required.

61

Section 2: Your thoughts and opinions on mobile learning (practitioners)

*11 Assume the NHS has provided you with a mobile device. Which of these statements would be true for you? I am good with technology and I am sure I would find the mobile device intuitive. I would not require training in its use. I would like some initial training in how to use the mobile device.

Section 2: Your thoughts and opinions on mobile learning (managers) *11 Assume the NHS has provided your staff with mobile devices. Which of these statements would be true for them? All my staff are good with technology and I am sure they would find the mobile devices intuitive and not require training in their use. Some of my staff are good with technology and I am sure they would find the mobile devices intuitive, but others would require training in their use.

Note: left-most column are questions for practitioners, and the column next to it are questions for managers. (Determined by answers to question 2 above).

I would like all my staff to have some initial training in how to use the mobile devices. This is an introduction to questions 12 – 16.

Please note: images used in this survey are examples only. They do not indicate the exact mobile device that may be used, nor the exact content employed for NHS training. 62

For the following questions, assume the NHS has provided you with some kind of mobile device for workplace learning, and offered you training to use it.

For the following questions, assume the NHS has provided your staff with some kind of mobile device for workplace learning, and offered them training in their use.

*12

*12

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for learning by reading (words and pictures).

I believe my staff would use a mobile device for learning by reading (words and pictures).

Please comment on your answer (optional).

Please comment on your answer (optional).

63

Ref: Q12 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

*13

*13

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for learning by watching a short video.

I believe my staff would use a mobile device for learning by watching a short video.

Please comment on your answer (optional). *14

Ref: Q13 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

Please comment on your answer (optional). *14

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for learning by watching an animation.

I believe my staff would use a mobile device for learning by watching an animation.

Please comment on your answer (optional). Please comment on your answer (optional).

64

Ref: Q14 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

*15

*15

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for learning by working through decision trees.

I believe my staff would use a mobile device for learning by working through decision trees.

Please comment on your answer (optional).

Please comment on your answer (optional).

*16

*16

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for learning by sending messages and discussing with peers and/or experts.

I believe my staff would use a mobile device for learning by sending messages and discussing with peers and/or experts.

Please comment on your answer (optional).

Please comment on your answer (optional).

65

Ref: Q15 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

Ref: Q16 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

17 Please describe any other ways mobile devices may or may not support you to learn (optional).

17 Please describe any other ways you believe mobile devices may or may not support your staff to learn (optional).

*18 Indicate the extent to which you agree or disagree with the statements below.

*18 Indicate the extent to which you agree or disagree with the statements below.

If I had a mobile device for learning:

If my staff had mobile devices for learning:

It would be more convenient for me to undertake training, because I would not have to leave my workplace.

It would be more convenient for them to undertake training, because they would not have to leave the workplace.

It would be more relevant for me because I could learn in the place where I have to put my training into practice.

It would be more relevant for them because they could learn in the place where they have to put their training into practice.

I would use it for learning during ‘dead time’, for example when waiting for a clinic to begin, or an emergency call.

I believe they would use it for learning during ‘dead time’, for example when waiting for a clinic to begin, or an emergency call.

I would rather use a computer/laptop than the mobile device if I was asked to learn using technology.

I believe they would rather use a computer/laptop than the mobile device if asked to learn using technology.

I would benefit from being able to use it to quickly and easily communicate with peers or with experts to get timely advice.

They would benefit from being able to quickly and easily communicate with peers or with experts to get timely advice.

I would benefit from being able to use it to more easily share my knowledge and experiences with peers and my manager.

They would benefit from being more easily able to share their knowledge and experiences with peers and management.

Please comment on your answers (optional) and/or describe any other advantages or disadvantages that you believe mobile learning may offer you.

Please comment on your answers (optional) and/or describe any other advantages or disadvantages that you believe mobile learning may offer your staff.

66

Ref: Q17 Note: I have not made this a required question. Ref: Q18 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’ to each of the statements. In the statements I have tried to address issues, such as portability, situatedness (i.e. learning in the workplace) context sensitivity (i.e. relevance to the context), learning preferences, connectivity and collaborative learning. I have deliberately inserted one negatively worded statement (bullet 4) to introduce balance and signal that negative responses are acceptable.

This is an introduction to questions 19 – 22.

For the following questions, assume the NHS has provided you with a mobile device for assessment, and offered you training to use it. *19

For the following questions, assume the NHS has provided staff with mobile devices for assessment, and offered them training in their use. *19

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for assessments that involve answering multiple choice questions to test my knowledge.

I believe my staff would use a mobile device for assessments that involve answering multiple choice questions to test their knowledge.

Please comment on your answer (optional).

Please comment on your answer (optional). 67

Ref: Q19 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

*20

*20

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for assessments that involve choosing the correct photograph, for example to demonstrate that I have the knowledge necessary to identify equipment or danger.

I believe my staff would use a mobile device for assessments that involve choosing the correct photograph, for example to demonstrate they have the knowledge necessary to identify equipment or danger.

Please comment on your answer (optional).

Please comment on your answer (optional).

*21

*21

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for assessments that involve rating my skills and then commenting on them.

I believe my staff would use a mobile device for assessments that involve rating their skills and then commenting on them.

Please comment on your answer (optional).

Please comment on your answer (optional).

68

Ref: Q20 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

Ref: Q21 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

*22

*22

Indicate the extent to which you agree or disagree with this statement.

Indicate the extent to which you agree or disagree with this statement.

I would use a mobile device for assessments that involve describing or taking photographs/video to collect evidence of my skills.

I believe my staff would use a mobile device for assessments that involve describing or taking photographs/video to collect evidence of their skills.

Please comment on your answer (optional).

Ref: Q22 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’.

Please comment on your answer (optional). 23 Please describe any other ways mobile devices may or may not support you in assessment (optional).

23 Please describe any other ways mobile devices may or may not support your staff in assessment (optional).

*24 Indicate the extent to which you agree or disagree with the statements below.

*24 Indicate the extent to which you agree or disagree with the statements below.

If I had a mobile device for assessment of my knowledge and skills:

If my staff had mobile devices for assessment of their knowledge and skills:

It would be more convenient because I would have greater choice regarding the timing of assessment tasks. It would be more convenient because I could undertake formal assessments in my place of work and send them off there and then via the mobile device.

It would be more convenient because they would have greater choice regarding the timing of assessment tasks. It would be more convenient because they could undertake formal assessments in their place of work and send them off there and then via the mobile device.

69

Ref: Q23 Note: I have not made this a required question. Ref: Q24 Participants would have the option to select: ‘Strongly agree’, ‘Agree’, ‘Slightly agree’, ‘Slightly disagree’, ‘Disagree’, ‘Strongly disagree’ for each of the five areas.

I would prefer to get assessment feedback and advice on next steps via the mobile device, so that I could use mobile learning to immediately address my needs.

I believe they would prefer to get assessment feedback and advice on next steps via the mobile device, so that they could use mobile learning to immediately address their needs.

I would prefer to get assessment feedback and advice on next steps via the mobile device, so that I could refer to it with ease again, at a later date.

I believe they would prefer to get assessment feedback and advice on next steps via the mobile device, so that they could refer to it with ease again, at a later date.

I would be likely to use it for doing quick self-assessments to work out for myself where I need to brush up on knowledge and skills.

I believe they would be likely to use it for doing quick self-assessments to work out for themselves where they need to brush up on knowledge and skills.

I would be likely to use it to seek advice from a mentor if I identified a gap in my knowledge that I wanted to quickly address. I would be happy to use it for assessments on the essential knowledge I require for subjects such as Fire Safety and Infection Prevention and Control (for example, by answering a series of word or picture questions in a quiz). I would be happy to use it for assessments on some of the essential skills I require for subjects such as Moving and Handling and Health and Safety (for example, by providing written, photographic of video evidence.) Please comment on your answers (optional) and/or describe any other advantages or disadvantages that you believe mobile assessment may offer you.

I believe they would be likely to use it to seek advice from a mentor if they identified a gap in their knowledge that they wanted to quickly address. I believe they would be happy to use it for assessments on the essential knowledge they require for subjects such as Fire Safety and Infection Prevention and Control (for example, by answering a series of word or picture questions in a quiz). I believe they would be happy to use it for assessments on some of the essential skills they require for subjects such as Moving and Handling and Health and Safety (for example, by providing written, photographic of video evidence.) Please comment on your answers (optional) and/or describe any other advantages or disadvantages that you believe mobile assessment may offer your staff.

70

Section 3: Your previous experience of NHS training (practitioners) *25 Describe in no more than 3 sentences the most valuable training (with or without the use of technology) you have received in the NHS. Include the subject, the length, how the training was delivered, and two things that made it particularly valuable.

*26 Describe in no more than 3 sentences the least valuable training (with or without the use of technology) you have received in the NHS. Include the subject, the length, how the training was delivered, and two things that made it least valuable.

Section 3: Your staffs’ previous experience of NHS training (managers) *25 Describe in no more than 3 sentences the most valuable training (with or without the use of technology) you believe your staff have received in the NHS. Include the subject, the length, how the training was delivered, and two things that made it particularly valuable.

Ref: Q25 This may offer some common opinions on what works for training from a practitioner/manager point of view, which could then offer insights for mobile training design.

*26 Describe in no more than 3 sentences the least valuable training (with or without the use of technology) you believe your staff have received in the NHS. Include the subject, the length, how the training was delivered, and two things that made it least valuable.

Ref: Q26 This may offer some common opinions on what does not work so well for training from a practitioner/manager point of view, which could then offer insights for mobile training design.

71

Appendix D: Survey communications Email/Letter 1: Trailing the survey For all 270 identified survey participants, to be sent on 3 December Subject: An important request for your opinion Dear [NAME] We need your help in advancing training within the NHS. As you know, many NHS staff work in community settings, or in jobs that require them to be on the move. Learning and assessments on mobile devices (supplied by the NHS, such as mobile phones) may be the answer to providing a consistent training across all staff. We need you to tell us if you believe that this is indeed the case. Next week we will be sending a short survey. It is divided into three sections: section 1 is about you, your job, and your existing comfort with technology; section 2 provides some very brief descriptions of mobile learning and asks for your opinions and thoughts on them; and section 3 asks you to comment on the NHS training that you value most and least. The survey is designed to take you no more than 20 minutes and needs to be completed and submitted by 20 December. Your insights are critically important to ensuring we make the best decisions for NHS training in the future. Therefore, your co-operation is very much appreciated. Thank you. Yours sincerely, ------PS All you need to do is remember to put aside 20 minutes by the 20th! Please block out a 20 minute slot now in your diary for completing and submitting the survey before 20th December.

Email 2: Research participant number and survey For all identified survey participants with email access, to be sent on 7 December Subject: An important request for your opinion Dear [NAME] I wrote to you recently about some important research into the future of training at the NHS, and asked if you would invest just 20 minutes of your time to complete a survey about your thoughts and opinions. Your research participant number is XXX – you will need this at the start of the survey, so please jot it down now so you can enter it when requested. When you are ready, please click here to access the survey: INSERT LINK (S) 72

Your co-operation is much appreciated. Yours sincerely, ------PS Please note, the survey is exploring the potential of mobile devices for learning and assessment, and it is not a test of you. Your answers will be anonymous and received by independent researchers, who will not be supplied with your name. The NHS will only see aggregated data. Please complete the survey by 20th December.

Letter 2: Research participant number and survey For all identified survey participants without email access, to be sent on 7 December Dear [NAME] I wrote to you recently about some important research into the future of training at the NHS, and asked if you would invest just 20 minutes of your time to complete a survey about your thoughts and opinions. Your research participant number is XXX – you will need this at the start of the survey. When you are ready, please complete the survey and return it in the envelope enclosed. Your co-operation is much appreciated. Yours sincerely, ------PS Please note, the survey is exploring the potential of mobile devices for learning and assessment, and it is not a test of you. Your answers will be anonymous and received by independent researchers, who will not be supplied with your name. The NHS will only see aggregated data. Please complete the survey by 20th December.

Email 3: Countdown prompt to end of survey For all identified survey participants with email access, to be sent on 15 December NOTE: This should be a forwarded email 2 above (thus still including the link to the survey), as follows: Subject: FW An important request for your opinion Dear [NAME] You received the message below from me on 7th December, requesting your help in better understanding if and how mobile devices could be used for learning and assessments within the NHS. If you have already responded to the survey, then thank you very much for your participation. If not, then I would appreciate it greatly if you would spare just 20 minutes today to complete the survey. Your cooperation is very much appreciated. Yours sincerely, ------73

Letter 3: Countdown prompt to end of survey For all identified survey participants without email access, to be sent on 15 December Dear [NAME] You received a letter, dated 7th December, requesting your help in better understanding if and how mobile devices could be used for learning and assessments within the NHS. If you have already completed and returned the survey enclosed with that letter, then thank you very much for your participation. If not, then please take 20 minutes as soon as possible to complete and return it. The deadline is 20th December. Your cooperation is very much appreciated. Yours sincerely, -------

Email 4: Deadline extension For all identified survey participants with email access who have not responded, to be sent on 5 January Subject: An important request Dear [NAME] Last month, you received a request from me asking for your help in advancing training within the NHS. Specifically, I asked you to invest just 20 minutes of your time to give your thoughts and opinions on a new way forward. I understand that it may have been difficult for you to fit this in during the run up to Christmas. Therefore, I ask that you start the New Year by setting aside 20 minutes before the end of this week to participate. Your insights are critically important to ensuring we make the best decisions for NHS training in the future. Your research participant number is XXX – you will need this at the start of the survey, so please jot it down now so you can enter it when requested. When you are ready, please click here to access the survey: INSERT LINK (S) Your co-operation is much appreciated. Yours sincerely, ------PS Please note, the survey is exploring the potential of mobile devices for learning and assessment, and it is not a test of you. Your answers will be anonymous and received by independent researchers, who will not be supplied with your name. The NHS will only see aggregated data. Please complete this survey by 7th January. It is important.

74

Letter 4: Deadline extension (to be sent on 5 January) For all identified survey participants without email access who have not responded, to be sent on 5 January (Naomi Norman of Epic to provide all relevant research participant numbers) Dear [NAME] Last month, you received a request from me asking for your help in advancing training within the NHS. Specifically, I asked you to invest just 20 minutes of your time to give your thoughts and opinions on a new way forward. I understand that it may have been difficult for you to fit this in during the run up to Christmas. Therefore, I ask that you start the New Year by setting aside 20 minutes before the end of this week to participate. I enclose the survey once again and ask that you return it in the envelope provided. Your insights are critically important to ensuring we make the best decisions for NHS training in the future. Your research participant number is XXX – you will need this at the start of the survey. Your co-operation is much appreciated. Yours sincerely, ------PS Please note, the survey is exploring the potential of mobile devices for learning and assessment, and it is not a test of you. Your answers will be anonymous and received by independent researchers, who will not be supplied with your name. The NHS will only see aggregated data. Please complete and return this survey by 7th January. It is important.

75

Appendix E: Focus group plan Introduction to the focus group (to be sent out prior to the session) Mobile learning in the NHS: A research project As you know, many NHS staff work in community settings, or in jobs that require them to be on the move. Learning and assessments on mobile devices (supplied by the NHS, such as mobile phones) may be the answer to providing consistent training across all staff. We need you to tell us if you believe that this is indeed the case. Therefore, at your meeting on [Wednesday 15 December/Friday 14 January], approximately [40 minutes/1 hour] will be dedicated to collecting data for a research project into the potential of mobile learning and assessment for the NHS. You do not need to do any work in preparation. All that we ask is that you approach the session with an open-mind and willingness to contribute your opinions and ideas. The agenda for the session is as follows: •

Brief explanation of what we mean by mobile learning

•

Opportunity for hands on – iPhone, iPad and Nintendo DS (No previous experience necessary)

•

Small group discussion: what would be the benefits/challenges of NHS staff learning in this way?

•

Brief explanation of what we mean by mobile assessment

•

Small group discussion: what would be the benefits/challenges of NHS staff being assessed in this way?

•

Small group discussion: what would be the benefits/challenges of NHS staff learning/being assessed in this way for some of the nine key essential areas set out in the Statutory and Mandatory Training Framework?

•

Whole group discussion of key themes that emerged.

Please note: some of the session may be tape recorded to aid data analysis by independent researchers. However, please rest assured that all contributions to focus group discussions will be reported anonymously, so will be non-attributable, in the final research report.

76

Focus group activities

1. Introduction to the focus group (to be distributed with an agenda before session) Brief background to the purpose of the study and the significance of the focus group discussion How the focus group session will work, including mention of tape recording, confidentiality, and how the data will be used and analysed

2. Explanation: What do we mean by mobile learning? (2 mins) •

Brief explanation of mobile learning, including examples such as learning from words and pictures, learning from video etc. Hand out: graphical representations of examples of mobile learning (as used in survey, questions 12 - 16)

3. Hands on (4 mins) Small groups to be given some real examples – for example, on iPhone, iPad and Nintendo DS – to support them in their discussions. These will include examples of learning and assessment. Note: groups will be reminded of the short time available and therefore encouraged to stay focussed and capture as much as possible during the following discussions.

4. Discussion: What would be the benefits/challenges of NHS staff learning in this way? (8 mins) • Benefits and challenges to be discussed in small groups, and to be captured on post-its: benefits on yellow post-its and stuck on an A1 sheet of paper headed ‘Learning Benefits’; challenges on green postits and stuck on A1 sheet of paper headed ‘Learning Challenges’.

5. Explanation: What do we mean by mobile assessment? (2 mins) • Brief explanation of mobile assessment, including examples such as answering multiple choice questions to test knowledge, or rating skills and then commenting on them etc. Hand out: graphical representations of examples of mobile assessment (as used in survey, questions 19 22)

6. Discussion: What would be the benefits/challenges of NHS staff being assessed in this way? (8 mins) • Benefits and challenges to be discussed in small groups, and to be captured on post-its: benefits on yellow post-its and stuck on an A1 sheet of paper headed ‘Assessment Benefits’; challenges on green post-its and stuck on A1 sheet of paper headed ‘Assessment Challenges’. Hand out: copies of the Statutory and Mandatory Training Framework (published: May 2010)

77

7. Discussion: What would be the benefits/challenges of NHS staff learning/assessment for some of the nine essential skills areas in the Statutory and Mandatory Training Framework? (8 mins)50 • Small groups to be given two of the nine essential areas to focus on – one that is more knowledgebased (for example, health and safety); and one that is more skills-based (for example, moving and handling). Benefits and challenges of mobile learning and assessment for these particular areas to be discussed in the small groups, and to be captured on post-its: benefits on yellow post-its and stuck on an A1 sheet of paper headed ‘Benefits for *essential area+’; challenges on green post-its and stuck on A1 sheet of paper headed ‘Challenges for *essential area+’.

8. Whole group discussion of key themes that emerged (8 mins)

Note: for the 1 hour session, activities 4, 6, 7 and 8 above will be extended to 10 minutes and 6 minutes will be dedicated to whole group feedback /discussion after sessions 4 and 6.

50

Please note that while the framework is specific to NHS South Central, other regions are using/developing similar frameworks, and all those attending the National focus group on 15 December acknowledge that statutory & mandatory training is a key driver for adopting learning technology solutions.

78

Appendix F: Telephone interview schedules Mobile projects: interview questions 1. 2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.

17. 18.

What is your job title? Please very briefly describe what your job role involves. I understand that you have been involved in a project that has made use of mobile devices in the NHS. Were you involved in managing the project, were you a user of the mobile devices, or were you both? What were the mobile devices (makes and models)? How many of them were distributed? What were the job roles of those using the devices? What was the intended use of the mobile devices? Was their actual use any different from their intended use, and if so, in what ways? In what ways, if any, did people change how they used the devices as time went by? In what ways, if any, do you think users benefitted from using the devices? Were there challenges to the use of the mobile devices, and if so, what were they? Were any of these challenges overcome, and, if so, how? Were users required to be involved in the mobile project, or did they volunteer? Please describe any support or training that was given on the use of the mobile devices themselves or the content. What could be done better if this project was rolled out again? What lessons do you think we can learn from this project if we wanted to use mobile learning and assessment for training in essential areas, like Health and Safety, Moving and Handling or Resuscitation? Has the project now ended? If so, how did people feel about the devices being taken away from them? Is there anything else you would like to add?

Potential stake-holders: interview questions 1. 2. 3. 4. 5. 6. 7. 8. 9.

What is your job title? Please very briefly describe what your job role involves. Are you aware of mobile devices being used, or having been used, for any learning and assessment within the NHS? (If yes: please could you describe how they have been used) Are you aware of any mobile learning strategy or policy within the NHS? (If yes: please describe its content). In what main ways, if any, do you think NHS staff could benefit from using a mobile device for their learning and assessment? What are the key challenges, if any, for the use of mobile devices for learning and assessment among NHS staff? How might the challenges you identified be overcome? Who are the key people you believe would need to be involved in making mobile learning and assessment work for the NHS? What specific staff roles do you feel would be best supported by mobile learning and assessment? 79

10. 11.

12. 13. 14. 15.

Are there any specific content areas you feel would be best supported through mobile learning and assessment? Now I am going to read out some content areas. Please rate how appropriate you believe they would be for mobile assessment specifically. 1 is highly appropriate; 5 is not at all appropriate. Conflict resolution Fire Safety Infection Prevention and Control Moving and Handling Resuscitation Please would you comment on each of your ratings. Why did you rate [content area] as a [number]? Which, if any, mobile devices are provided or supported for use by staff within the NHS? (Please give details of makes and models.) Are you aware of any existing or future plans for mobile learning strategy or policy within the NHS? (If yes: please describe its content). Do you have any other thoughts or views on the use of mobile devices for learning and assessment within the NHS that you would like to add?

Technical: interview questions 1. 2.

3. 4. 5. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15.

What is your job title? Please very briefly describe what your job role involves. The next few questions I would like to ask you regard getting mobile devices on to a controlled secure network within the NHS. What percentage of NHS sites, if any, do you estimate have wireless access? What kind of job roles would have access? Where there is wireless access, are all NHS staff allowed to use it? Does the NHS have a wireless policy, and if so, would you briefly tell me about it? What challenges, if any, do you believe there would be in putting wireless into NHS sites that are currently without it? Where wireless is not available, what do you believe would be the main challenges of getting access to the intranet via 3G? To make any kind of mobile assessment or training effective we would need to save the user’s data back the your Oracle Learning Management system. [NOTE TO INTERVIEWER: Questions 9 – 12 are for the Senior Development Adviser for ESR and National LMS Implementation Officer only. For other interviewees please go to question 13.] Is OLM available over the internet or only on NHS intranets? What are the differences between the NLMS (National Learning Management System) and OLM from a security perspective? Does the NLMS currently work from a mobile device? If so, how? Your Electronic Staff Records is another database that has been discussed. Is this tied directly into either the NLMS or the OLM, or would it need to be updated separately? What do you think would be the simplest way for a mobile device to send back user data to your network infrastructure? Please would you describe any other mobile projects, or similar instances, where data has had to be sent back to your network infrastructure? Which, if any, mobile devices are provided or supported for use by staff within the NHS? 80

16. 17.

18.

19. 20.

(Please give detail of recommended makes and models) In your experience, which roles within the NHS are provided with mobile devices as part of their job function? In your experience, who decides what devices NHS staff should use? There have been some trials run in certain areas of the country that have used mobile devices such as Blackberries to record patient information. Are you aware of any other projects currently running in the NHS, where staff are supplied with mobile devices for other purposes, and where their value for training also could be explored? (If yes, please describe and comment on any lessons learnt from such projects) Are you aware of any existing or future plans for mobile learning strategy or policy within the NHS? (If yes: please describe its content). Is there anything else you would like to add that you think may be useful to me in determining how mobile learning can integrate with existing NHS IT infrastructures?

Interviewer notes You will need:

 A room with: o

speaker telephone

o

computer with internet access

 Name and telephone number of the interviewee  Recording equipment Pre-interview:

 Set up recording equipment next to the telephone  Open the relevant NHS mobile telephone interview data collection tool *Please double-check you have opened the correct one.

 Enter the interviewee’s name.  Have your calendar available in case the timing of the interview is no longer convenient The interview:

 Check you are speaking to the correct person  Read the following introduction: I am ringing regarding the telephone interview set up by Alison Potter (nee Wright), the E-learning Programme Manager at South Central Strategic Health Authority. Are you still ok for me to ask you a few questions?

81

*If the interviewee says ‘no’, then schedule another time before the end of January that is more convenient) The interview is about the potential of mobile learning for the NHS, and will play a vital role in decisions on e-learning in the NHS going forward. Answers will be aggregated with responses from other participants, and so will not be attributable to you. Therefore, all I ask is that you approach the interview with an open-mind and a willingness to contribute your honest opinions and ideas. [For technical interviews only: I also ask that if you are unsure about an answer, then you indicate that you are unsure, or state you do not know, so that I can be certain I am collecting accurate information.] Finally, please be aware that I will be writing down your responses, but I am also tape recording this call for data collection purposes only, so that I can be sure I capture everything that is said. The independent researcher, outside the NHS, who will be analysing the data is the only person who will have access to this recording.

 Now work through the questionnaire, entering data as you go, and clicking the next button. As you do so: o

Encourage the interviewee to give examples from their own experience

o

If the interviewee can't answer a question, move on to the next

o

If the interviewee gives an ambiguous response, seek clarification, for example if they state something ‘worked well', it would be useful if you could ask them in what way.

o

If the interviewee goes off the subject, politely say you need to move on.

o

Don't be afraid to ask people to repeat or elaborate.

o

If the interviewee is interrupted and cannot continue the call, make a note of their name and click through the questionnaire until you get to Done, so you can save the data you managed to collect. Arrange a time to call back later. On the second call, enter the respondent’s name again into the survey and click through the questions you've already answered. Then start entering data from that point forward.

 When you get to the end, thank the interviewee for their time, tell them they've been very helpful, and then click the Done button.

82

Appendix G: Mobile projects Previous and current NHS mobile projects were described in mobile project telephone interviews [TMP] and there was some mention of other mobile projects in the Stakeholder telephone interviews [TS]. Summaries of projects appear below in no particular order.

Project 1 [TMP]: Laptops in ambulances (in planning stages - January 2011) Transcript extract: “The project I am working on at the moment is to put a mobile device *laptop+ into ambulances, so that they can collect information about patients rather than having to write on pieces of paper. And then you can connect that up to other systems (both upstream and downstream) so you get a richer picture of information and you can make better decisions... All the information will be beamed back to a data centre and could be viewed in the control room, an A&E department or a coronary care unit.... you have the ability to stream information like ECGs and you could actually start to move some of the care out of A&E and into the ambulance [so that it happens sooner]. You can associate the application with other applications, such as decision support modules and workflow tools, so that you can be more consistent in the way you treat patients. And obviously [it can be used]... to improve handover and to do an audit on what you did, for example. There are opportunities for communications [as well]: you can have video conferencing [and] web chats. It will be configured primarily to be a platform that hosts patient records applications, but there is no reason why you couldn't host other applications on it. And also it will have web access. The Ambulance Trust want to use it for other applications such as e-learning, so that when people are on the road and not doing anything, sitting in a lay-by, they can do some training... like on NHS Pathways, triage etc.”

Project 2 [TMP]: Blood tracking on PDAs (at end of eight-month pilot phase and moving into implementation – January 2011) Transcript extract: “Handheld PDA units with barcode scanner built into them... *that+ trace the movement of blood *from blood bank to hospital fridge to patient to start / end of transfusion and to lab]... An application records the details of the user who is undertaking the transaction, the details of the blood, the patient, start time and end time of blood transfusions. [It involves a range of staff], everyone from nurses, transfusion people, cancer care, lab staff. So, on a ward it is every one of the nursing staff and every member of the portering staff, [as well as] staff in labs. [For the pilot, there have been] 20 [devices] across several wards and labs. At implementation, it will be 100s of them that we roll out - there will be one per ward area. But they will go anywhere there is a high volume of blood taking and transfusions. 83

[This offers a] massive patient safety boost, [as there is] 100% traceability of the blood. Blood has a validity period out of a fridge. From the Trust perspective, someone would collect blood for an area, they'd take it away and then it would turn out that it was not required. At that point there was no accurate record of when the blood came out of the fridge, how long it had been in transition, the storage in the meantime - in a fridge or on a trolley. So there is savings, in terms of waste, because we can identify if blood is still feasible to be used or has to be destroyed. [And] staff using the devices are finding the benefit of actually seeing where blood is, rather than having to phone people, or to randomly walk around fridges, [now] they can see by looking at a PC... They can also use their PDA to request blood, rather than have to phone lots of people to chase it up. This saves them time.”

Project 3 [TMP]: Collecting patient data at the bedside (currently being piloted on under 100 wards at Queen Alexandra Hospital, Portsmouth – January 2011) Transcript extract: “We’ve removed all the paper charts at the end of the bed to record heart rate, blood pressure etc, and we collect all our data using handheld devices... at the bedside (Hewlett Packard PDAs, and a range of tablets - the C4 device which is a healthcare tablet, but more recently we’ve been using the Panasonic tough books)... [We] aggregate the data in hospital computer servers and produce patient charts from that. [This has been done before] in intensive care units and critical care units around the world, but very few people have tried it on a general ward, so that’s what we have done. There are 6 PDAs per ward and 3 tablets per ward.... Nursing staff uses the PDAs predominantly; medical staff uses the tablets predominantly. That’s just the nature of their work... The nurse is moving between beds doing observations on a range of patients, so the handheld device was thought to be the perfect thing for them. The idea that every time you update a record, you have to return to a desktop PC would really disrupt the workflow. So we put the device in the nurses’ hand and that’s why a small device like a PDA works perfectly. And for a doctor’s ward round where you might have five, six, seven people on the round, everyone gathering round one PDA wouldn’t work, so that’s why they have a tablet because it is much easier for a group of people to look at the information. ...This originally started out as a way of recording vital signs like blood pressure, temperature and heart rate and things, but very soon we put on surveillance for MRSA screening, so screening against infections when people come into hospital. So there are a whole range of other things that now the staff are asking for us to put on to the device. So it is now seen as a solution to some of the problems people encounter within their daily work. ...There is an application we are hoping to put in which is part of the system which would allow in the first instance people with iPhones to directly access the data and it will probably be available for some of the android phones in the future...This is part of our development plan. And anyone in the hospital who needs to see this data can also access it on their desktop PC as well, so they can pull up a chart of any patient on their desktop PC, providing they’ve got the appropriate access rights. The part about the iPhones, which is perhaps the next application we put in, the purpose there is to directly communicate with the medical staff when there is 84

a problem they should know about. So if you are in a different part of the hospital and one of your patients deteriorates, then the system would allow the doctor to be automatically alerted to the problem, and they could pull up on their mobile phone all of the information that would be available to them at the bedside. Now if you try and put the paper back you’ll find there’s a lot of resistance. In 2010 we won two patient safety awards for this, one from the BUPA Foundation and another one.”

Project 4 [TMP]: Digital Pen and Blackberry project within Portsmouth maternity services (currently 130 pens and Blackberries – January 2011) Transcript extract: “The mother’s diary is normal paper, with an overlay of unique dotted paper... *The midwife uses+ a big biro-like pen with a built-in camera, [which] records the dots written over, and once completed ticks a box to send the data. It takes what the midwife has written in the mother’s diary... *and+ links to a Blackberry device that carries over normal network coverage... [and then] feeds it into multiple systems.... , [for example] into maternity system and GraphNet (EPR electronic patient record system). It converts it to readable format (PDF),... [so] you can see the original midwifes handwriting... There is no missing information - you know when, which midwife, and what she wrote. If the mother loses their notes they are able to print another set of notes which are exactly the same... [And they can use the Blackberries] to book meetings... [access] medical dictionaries, sat nav to route them to an address for a visit. The original method used lots of duplication [and] you have travel time, time delay in getting information back to hospital.... They do not need to keep going back and forth between the mother and hospital now. [Then, there used to be+ carrying of patient confidential data in the midwife’s own diary. Annual recordings in a midwives diary will have over 100 patients’ information, so if it was to go missing it would cost money. Midwifes wanted something small, compact and easy to use, a laptop’s battery would only last three hours but a smart phone lasts two to three days. The blackberry is used due to being totally secure. If the midwife was to lose the device, the whole blackberry can be remotely wiped.”

85

Other projects: mentioned in Stakeholder interviews [TS] Transcript extracts: “Some of the Royal Colleges are piloting the use of mobile devices.” “NHS Education Scotland is very interested to see if mobile could be used to support assessment and other data gathering as part of Foundation Training.” “Handheld decision aids [PDAs], for example with the Oxford Handbook of Clinical Medicine loaded on to them, have been used in Leicester, but ‘probably five or six years old’; and also in Wales, ‘two to three years ago’.” “Some organisations have used audio mp3 players for induction and telling new entrants about their organisation through audio.” “There are tele-medicine and tele-activities involving sending x-rays or scans to remote organisations for interpretation, which also have been used for teaching as well as stimulation activities.” “PDAs have been used as a means of assessing ‘Skills for life’ baseline. They were pre-loaded with an electronic assessment that tests knowledge of the skills. When completed, participants handed the devices back and they were connected to a dock for download.” “Recently, there has been some experimenting with lending out iPads, but previously it's been done with other windows-based tablets. Mostly they were for the content staff have to do, so more related to content that would improve safety, and less so clinical content.” “There’s a ‘near me’ app. You put in your postcode and it will tell you where your local health services are. Also, NHS local have run a campaign around obesity and they have an app where you can work out your BMI and how many calories you have consumed.”

86