Leverhulme Trust Outline Application For Research Project Grant

Principal Applicant (1.1) General Details

Title First Name(s) Surname

Dr

Gender

Leslie

Date of Birth

Ball

Male 07/02/1959

Honours

(1.2) Contact Details

Institution Department Position In Department Address

University of Abertay Dundee School of Computing and Creative Technologies Lecturer University of Abertay Dundee Bell St Dundee DD11HG

Telephone No.

01382 308251

Fax No.

01382 308627

Email Address

[email protected]

(1.3) Other Details

How is your post currently funded? Date Appointed To Current Post

Government 01/08/2000

(1.4) Proposal Details

Long Title

An SQL e-Tutor Fusing Intelligent Agents with Game-Based Learning

Short Title

An SQL e-Tutor Using ITS and GBL

Main Field of Study Sub Field of Study Start Date Duration

Applied sciences (including architecture] Computing 01/10/2009

Total Budget Figure £

73911

24 Total time to be spent on the project by Principal Applicant

7

Proposal (2.1) Abstract

The Structured Query Language (SQL) is the top skill sought by the Information Technology sector. However, students often fail to master the skill beyond the basic level of competence. This proposal will determine the efficacy of teaching SQL via an Intelligent Tutoring System (ITS) embedded within a computer game environment. Whilst the case-based reasoning technique will be investigated to deliver instructions, monitor student progress and provide appropriate feedback, a suitable gaming interface will be examined to provide the necessary motivation and reinforcement critical to learning. The main aim and novelty is therefore to fuse these two aspects effectively. (2.2) Why the Leverhulme Trust?

The primary reason for applying to the Trust is that the proposal crosses the boundaries of several traditional and contemporary disciplines, requiring a fusion of skills from Educationalists, Artificial Intelligence (AI), Game-Based Learning (GBL) and Psychology. This blend fits the originality and responsive mode criteria well, matches strongly with the “problem itself must set the rules” spirit of the Trust and is outside the remit of any specialised funding agency. Moreover, the contemporary issue of GBL is particularly aligned with the “changes to fit the times” view and culture established by the Trust. SQL technology is pervasive through Computer Science degrees and other derivative programmes. Any proven successes in the proposed methods could therefore have far reaching consequences. SQL is also core to the teaching of the Principal Investigator (PI), who has first hand experience of the learning difficulties encountered by students using traditional teaching pedagogies. The proposal’s research links to teaching are thus self-evident. The inherent risk is the departure from established working patterns for the PI by integrating the work across a multi-disciplinary research framework involving pedagogy, game playing, computing and psychology. The impact of the research is expected to be in enhancing student learning by increasing motivation, performance and employability, as well as in ultimate student retention. Priority would be to disseminate any successful outcomes into the field of Computing in Higher Education. This will fundamentally address the overarching and critical criteria of the Trust in that the research should be of benefit to a wider community. (2.3) Other Research Commitments

The applicant is currently collaborating on a one year Carnegie Trust Large Grant award in the field of biometrics to analyse user keystroke dynamics for verification and identification purposes, involving a collaboration of Scottish Institutes. The project is due to end on the 1st August 2009.

Rationale The Structured Query Language (SQL) remains the single most important computing skill sought by the Information Technology sector1. The teaching of this skill consequently pervades university computing programmes and their derivatives. Acquisition of the skill, however, remains a problem for many students, possibly due to its declarative structure compared to the more standard procedural programming languages2,3. The associated pedagogical challenge for academics is to design a strategy that implements material delivery in a way that facilitates effective knowledge transfer to the learner4. Where conventional class sizes are large, the problem of knowledge transfer can be exacerbated further because one-to-one tuition is unattainable. With advancements in technology, Intelligent Tutoring Systems (ITS) have been developed to assist the teacher to transfer domain knowledge to the student by offering expertise in the form of an etutor5. Problem and Aim To be effective, any ITS needs to interface with three inter-dependent components: the instruction (lessons and feedback), the student response, and the domain knowledge. There have been several attempts to implement the teaching of SQL through virtual media using this framework2,6,7,8,9. None of these, however, embed the learning process within a case-based reasoning (CBR) artificial intelligence framework, nor within a motivational and immersive interface offered by the gaming environment, which has been itself shown to support learning10. This presents the opportunity and the main aim of the proposal, to determine the efficacy of embedding a case-based teaching and learning paradigm for SQL within an immersive gaming environment. To achieve the aim the following key objectives are established: 1. To establish a system architecture that supports both the incremental SQL learning and the purpose of the game. 2. To investigate the use of CBR for the delivery of effective teaching and learning components. 3. To design an appropriate gaming interface to motivate and integrate the student with the SQL domain knowledge. 4. To evaluate psychologically the effectiveness of the proposed approach compared to more conventional methods. Research Context and Methods The research questions rooted in the four objectives are focused on how to implement an effective online teaching paradigm that not only monitors and adapts to student learning behaviour through computational analysis, but also examines whether the acquisition of the SQL skill is enhanced by incorporating game playing. SQL queries can vary from basic (e.g. SELECT … FROM …) to aggregation and complex (e.g. multiple nested) structures. The learning is thus incremental and builds on previous knowledge, lending itself to a constructivist type of learning paradigm. This scenario would fit a corresponding incremental framework for game design too11, whereby players attain higher levels of game playing in alignment with their level of SQL expertise. A teaching framework needs to consider material delivery, knowledge design and knowledge transfer4, and this must be carefully fused within the gaming strategy. Key to knowledge

transfer is the ‘intelligence’ of the tutoring system in terms of decision-making. For example, techniques based on chatbots that understand mathematical notation12, and use of Bayesian networks, which reason with uncertainty in the teaching of computer programming13, have been implemented. For this scenario, CBR14 is proposed as a technique which could map very strongly to this teaching paradigm of SQL in that each set of queries at any level could be represented as case knowledge. That is, a specified query set by the e-tutor would have associated with it the student responses and solution sets to represent matching, monitoring and feedback. Cased-based curricula have become more popular since the 1980’s15, but have not been implemented in the specific teaching of SQL. Rather, the implementation has been based in constraint-based modelling8, an evaluation function2 and an automatic grading system with peer review7. Certainly, none of these have incorporated the motivational aspect of game playing. The unique combination of a technique such as CBR to support the teaching and learning process and a suitable gaming framework forms the heart and novelty in the research proposal. ‘Serious Games’ is a term coined around the renewal of interest in the use of games for imparting knowledge and skills, for example, in computer-based learning16 and language learning10. The term refers to an interactive computer application that 1) has a challenging goal 2) is fun to play and/or engaging 3) incorporates some scoring and 4) imparts to the user a skill, knowledge or attitude17. In this proposal the SQL skill clearly relates to the fourth component, while the ITS will manage the third and the game playing the second. The ITS and the game playing together will manage the learning and gaming goals of the first component respectively. The emergent tool will thus require a system architecture that caters for the integration of the SQL, ITS and gaming components. Ultimately, the tool will require rigorous testing to demonstrate its efficacy. This will be addressed by using a combination of standard teaching tools and the proposed gaming tool in an Experimental Psychology framework, rather than in a teaching and learning context as part of an ongoing course, for ethical reasons. To test the efficacy of games-based teaching, we shall compare the performance of students who experience the standard method only (control group) with that of students who experience the standard method in addition to the game-based tool (experimental condition) in a defined performance context. Suitable course-based related performance tasks (e.g., accuracy and time to complete a particular task) will be given to both control and experimental groups in a study setting. A statistical comparison of the performance of both groups will allow a quantitative assessment of the efficacy of the new gaming tool in combination with standard teaching methods. In addition, questionnaires will gauge student experience of the different methods, providing a more subjective, qualitative measure of the teaching and learning experience.

Outcomes and Benefits Establishing a successful tool holds both specific and generic value for knowledge transfer. Specifically, the omnipresence of SQL across university Computing Programmes and the industry sector demands its effective learning for enhanced employability. Generically, any case-based curriculum such as computer programming or management decision-making could benefit from the technique.

Thomson, R. (2009) Uk’s Top Most Wanted IT Skills For Permanent Staff. http://www.computerweekly.com/Articles/2009/01/22/234353/uks-top-20-most-wanted-it-skills-forpermanent-staff.htm [last access 21/5/09]. 1

Sadiq, S., Orlowska, M., Sadiq, W. and Lin, J. (2004) SQLator – An Online SQL Learning Workbench, ITiCSE ’04, Leeds, UK. 2

Barrow, D., Mitrovic, A., Ahlosson, S. and Grimley, M. Assessing the Impact of Positive Feedback in Constraint-Based Tutors, In: Intelligent Tutoring Systems (eds. Woolf, B., Aimeur, E., Nkambou, R. and Lajoie, S.), 9th International Conference, Montreal, Canada, June (2008). 3

Renaud, K. and van Biljon, J. Teaching SQL – Which Pedagogical Horse for this Course? In: Proceedings 21st British National Conference on Databases, BNCOD. Lecture Notes in Computer Science, TLAD Workshop, July (2004), vol. 3112, pp244-256. 4

Woolf, B., Beck, J., Ekiot, C. and Stern, M. Growth and Maturity of Intelligent Tutoring Systems, In: Smart Machines in Education (eds. Forbus, K. and Feltovich, P.), MIT Press (2001), pp99-144. 5

Kearns, R., Shead, S. and Fekete, A. A Teaching System for SQL. In: Proceedings of the 2nd Australian Conf. on CSE, Melbourne, Australia (1997) pp-224-231. 6

De Raadt, M., Dekeyser, S. and Lee, T. Do Students SQLify? Improving Learning Outcomes with Peer Review and Enhanced Computer Assisted Assessment of Querying Skills, ACM Int. Conf. Proc. Series , Vol. 276, 6th Baltic Sea Conference on Computing Education Research, Koli Calling (2006), pp101-108. 7

Mitrovic, A. (2003) An Intelligent SQL Tutor on the Web, International Journal of Artificial Intelligence in Education, 13, pp171–195 . 8

Bhagat, S., Bhagat, L., Kavalan, J. and Sasikumar, M. (2002) Acharya: An Intelligent Tutoring Environment for Learning SQL. Proceedings of the Vidyakash 2002 International Conference on Online Learning, Mumbai, India. 9

Johnson, W. and Wu, S. (2008) Assessing Aptitude for Learning with a Serious Game for Foreign Language and Culture, In: Intelligent Tutoring Systems (eds. Woolf, B., Aimeur, E. et al.), LNCS, 9th Int. Conf., ITS, pp520-529. 10

Mills, C. and Dalgarno, B. (2007) A Conceptual Model for Game-based Intelligent Tutoring Systems. ICT: Providing choices for learners and learning. Proceedings Ascilite Singapore 2007. 11

Knill, O., Carlsson, J., Chi, A. and Lezama, M. (2004) An Artificial Intelligence Experiment in College Math Education. http://www.math.harvard.edu/~knill/preprints/sofia.pdf 12

Butz, C., Hua, S. and Maguir, R. (2004) A Web-based Intelligent Tutoring System for Computer Programming, Proceedings of the IEEE/WIC/ACM Int. Conf. on Web Intelligence (WI’04), pp159165. 13

Williamson, S. (1992) Putting Case-Based Instruction into Context: Examples from Legal, Business, and Medical Education, Journal of the Learning Sciences, 2, pp367-427. 14

Regan, P. and Slater, B. (2002) Case-based Tutoring in Virtual Education Environments, Proceedings of the 4th International Conference on Collaborative Virtual Environments, Bonn, Germany, pp2-9. 15

Fordis, M., King, J. et al. (2005) Comparison of the Instructional Efficacy of Internet-Based CME with Live Interactive CME Workshops: A Randomized Controlled Trial, JAMA, pp1043-51. 16

17

Bergeron, B. (2006) Developing Serious Games, Charles River Media Inc., ppxvii.

Publications (4.1) Publication 1

Ball, L. and Leimich, P. (2008) Contemporay Database Topics: Learning by Teaching, TLAD 08, Cardiff. (4.1) Publication 2

Leimich, P. and Ball, L. (2003) On-line Fun with Databases. LTSN Workshop.

Co-Applicants (5.1) Co-Applicant 1

Title First Name(s) Surname Institution Department Address

Dr Suheyl Ozveren

Gender Date of Birth

Male 23/10/1954

Honours

University of Abertay Dundee School of Computing and Creative Technologies University of Abertay Dundee Bell Street Dundee DD11HG

Telephone No.

01382 308260

Fax No.

01382 308627

Email Address

[email protected]

(5.1) Co-Applicant 2

Title First Name(s) Surname Institution Department Address

Mr Matthew Bett

Gender Date of Birth

Male 21/10/1980

Honours

University of Abertay Dundee School of Computing and Creative Technologies University of Abertay Dundee Bell Street Dundee DD11HG

Telephone No.

01382 308352

Fax No.

01382 308627

Email Address

[email protected]

(5.1) Co-Applicant 3

Title First Name(s) Surname Institution Department Address

Dr Andrea Szymkowiak

Gender Date of Birth

Female 09/10/1964

Honours

University of Abertay Dundee Psychology University of Abertay Dundee Bell Street Dundee DD11HG

Telephone No.

01382 308596

Fax No.

01382 308749

Email Address

[email protected]

Referees (6.1) Referee 1

Title Position Institution Department Address

Dr Senior Lecturer

First Name(s) Surname

Karen Renaud

University of Glasgow Computer Science Department of Computing Science, University of Glasgow, 17 Lilybank Gardens, Glasgow, G12 8RZ, UNITED KINGDOM

Telephone No.

0141 330 2052

Fax No.

0141 330 1937

Email Address

[email protected]

(6.1) Referee 2

Title Position Institution Department Address

Professor Chair in Interactive Multimedia Systems

First Name(s) Surname

Bob Stone

University of Birmingham School of Engineering Electrical and Computer Engineering Edgbaston Birmingham B15 2TT UK

Telephone No.

0121 414 7395

Fax No.

0121 414 4291

Email Address

[email protected]

Previous and Current Applications (7.2) Previous / Current Application To Other Funding Bodies

Title Date Status Reason for Rejection if known

Funding Body

Salary Budget Researcher 1 (8.1) General Details

Type

Research Assistants

First Name(s)

Title

Surname

Date of Birth Current Employment Position Held Title of Thesis % of time spent on the project

Date Thesis Awarded

100

(8.2) Figures

Year 1

Year 2

Total

23

24

24152

24877

49029

0

0

0

National Insurance

1649

1715

3364

Superannuation

4468

4602

9070

30269

31194

61463

Scale Point Basic Salary Local Allowance

Total

Associated Costs Cost 1 (9.1) General Details

Type Justification

Conference Year 2: The International Games Development Conference in the USA (GDC) will be a major point of dissemination for the second year and the maturity of the tool(s). 2 delegates are scheduled.

(9.2) Figures

Year 1 Cost £

0

Year 2

4000

Total

4000

Cost 2 (9.1) General Details

Type Justification

Computing The XPS Gaming PC has the appropriate configuration and specification to support the development of an ITS within the graphics intensive environment of game playing.

(9.2) Figures

Year 1 Cost £

1099

Year 2

0

Total

1099

Cost 3 (9.1) General Details

Type Justification

Computing A standard laptop will be required for transportability of tool development for conference and networking purposes.

(9.2) Figures

Year 1 Cost £

1049

Year 2

0

Total

1049

Cost 4 (9.1) General Details

Type Justification

Computing A free-standing hard disk drive facility is essential for incremental backup and recovery during the development cycle.

(9.2) Figures

Year 1 Cost £

300

Year 2

0

Total

300

Cost 5 (9.1) General Details

Type Justification

Meetings It is envisaged that some networking (estimated 2 to 3 meetings per year) will be necessary with other academic institutions (e.g. University of Birmingham for Serious Games) and the games

industry both locally and UK-wide. (9.2) Figures

Year 1 Cost £

500

Year 2

500

Total

1000

Cost 6 (9.1) General Details

Type Justification

Conference In Year 1: 2 attendees will participate in the European Conference on Games-Based Learning in October 12-13 2009, Graz Austria to make European links with both educationalists and gamebased theorists and developers (cost £4,000). Also, 2 attendees will attend the annual UK GDTW (Games Design Technology Workshop) at John Moores University, Liverpool in 2010 to enhance links during the early stages of design (estimated cost £1000).

(9.2) Figures

Year 1 Cost £

5000

Year 2

0

Total

5000

Budget Summary Year 1

Year 2

Total

Research Assistants

£30269

£31194

£61463

Researcher Total

£30269

£31194

£61463

Other Salary Total

£0

£0

£0

Salary Grand Total

£30269

£31194

£61463 ( 83% )

Conference

£0

£4000

£4000

Computing

£1099

£0

£1099

Computing

£1049

£0

£1049

Computing

£300

£0

£300

Meetings

£500

£500

£1000

Conference

£5000

£0

£5000

Associated Costs Grand Total

£7948

£4500

£12448 ( 17% )

Grand Total

£38217

£35694

£73911

Researcher

Other Salary

Associated Costs