College of Medicine and Veterinary Medicine Assessment These pages provide an opportunity to share our views and information about our strategy across the MBChB programme in assessment. Key information can be found at the below links and please contact those listed for more information not covered on these and the associated pages. 1-Purpose of Asssessment 2-Assessment staff contacts 3-Assessment glossary 4-MBChB Assessment url 5-Principles and Precepts of Assessment in the MBChB PURPOSE OF ASSESSMENT Assessment is the attempt to measure the extent to which students have mastered the learning objectives of a course. Assessment is important to a number of people and for a number of different reasons:







students: like to know how well they are performing during the course and look forward to receiving a qualification at the end of the course teachers: use assessment to confirm that students have assimilated what has been taught doctors: require a level of competence on which to build further teaching the public: have confidence in the qualifications of the profession the General Medical Council (GMC): need a mechanism for validating the competence student and the standard set by the medical school institutional "watchdogs": observe issues relating to quality assurance and benchmarking

Since medicine embraces a very wide range of knowledge and human experience assessment in medical education is necessarily a complex issue. Everything that is taught in the Edinburgh MBChB course has the potential for assessment. Assessing the full range of skills, attitudes and knowledge in a way that is fair, valid and reliable is by necessity a process that makes demands of both staff and students. It is often said that assessment drives learning and that the nature of assessment determines the outcome of the learning. In defining the assessment programme it is acknowledged that a clear statement about the nature and purpose of assessment is of fundamental importance for students who wish to achieve the highest level of performance as medical practitioners. Phillip Evans, Andrew Calder, Sandy Reid

Assessment Contacts Dr Sandy Reid, Chair MBChB Curriculum Executive: [email protected] Lindsay Dalziel, MBChB Assessment Officer: [email protected]

Assessment Glossary GLOSSARY OF EDUCATIONAL TERMS Aim Angoff

A statement of what those who have devised a course are attempting to achieve. Usually a course has a number of aims. Aims are more general than objectives. A method of setting standards. A group of experts discuss and agree on

Assessment

Common marking scheme Core Deep approach to learning

Dry Practical Evaluation

Facilitator

Feedback Formative Assessment Holist In course assessment Learning approach Learning Style Learning objective (outcome) Mark Modified Essay Question (MEQ) Module

the raw scores that correspond to various cut scores, such as pass, excellent. Then the raw scores are converted into marks. See further details in EEMeC Assessment page on standard setting. The group of procedures by which student's knowledge, skills and attitudes are determined, although assessment of attitudes is somewhat controversial. Most assessment is by examination but some is undertaken during the course by written work completed by the student. This is termed 'in-course assessment'. Assessment can be formative or summative and may imply various techniques, such as multiple choice questions, structured case questions, etc. Assessment should be distinguished from evaluation. The set of marks with boundaries decreed as standard by the University. Those prescribed for use in Medicine are different from those used by the wider University because of the perceived need to test for competence. The knowledge, skills and attitudes that are regarded as essential for the aspiring doctor. This is in contrast to student selected components, formerly called options. This occurs when the student processes information by thinking about it and relating it to other information already learned. A deep approach to learning occurs particularly when the student is engaged in a dialogue with other students or members of staff or with a computer. It can also incur when the student questions his or her own learning. An important tenet of education is that it should provide an environment that encourages a deep rather than a surface approach to learning. A practical class in which the students inspect objects rather than do experiments. In the UK this refers to the process of determining how effective a teaching or learning course has been. It does not refer to examination procedures, which are termed assessment. Evaluation may be undertaken by course questionnaires, interviewing students, etc. Someone who is present with a small group of students undertaking a learning session but whose job is not to impart knowledge to the students but to assist them in the process of learning for themselves. The facilitator is particularly important in the small groups who undertake problem based learning. There is controversy on whether the facilitator should be an expert in the subject matter under discussion or not. The consensus is that the facilitator should not be an expert, as this reduces the likelihood of the learning session turning into a mini lecture. Information received back, usually in answer to questions, as part of an evaluation. Also, information given to students about their performance as part of formative assessment. An examination or other assessment procedure which is undertaken part of the way through a course or module, the main aim being to give the student guidance into how he or she is progressing. Someone who learns by viewing the subject matter in a broad context, that is as a whole, then closes in on the details. Many people have a basically holistic learning style. A piece of work constructed in own time and submitted by the student during the course of an academic year. It is then marked and returned to the student, with comments. The approach a learner takes to learning and studying: deep, surface or strategic. The general approach to acquiring knowledge, skills and attitudes of a given individual. This may be holist or serialist. Some people are mainly holists or serialists, although many use one approach at different times. A statement of what the learner will be able to do at the end of a learning experience. The result of converting the raw score into another number according to some standard setting procedure. A style of assessment in which the student is asked to supply short answers to a series of questions which form a story or a clinical case summary. Unit of education, mainly self-contained. The length is variable but may

be from 1 or 2 weeks up to a term. A statement of what the learner will be able to do at the end of a learning experience. A type of clinical examination in which the candidate performs a number of defined tasks in a defined time, rotating round different stations. The tasks involve interacting with patients, actors or examiners in a structured way and demonstrating practical and interpretative skills as well as core knowledge. Assessment follows a structured objective marking scheme. Portfolio Range of items, in this context constituting a means of assessment. Usually the items are disparate so that marks cannot be added together meaningfully. A group of case reports may constitute a portfolio. A style of learning based around discussion of a problem. The problem Problem Based Learning (PBL) may or may not be a clinical case based around a patient. It should be discursive in nature, not just a series of questions expecting short answers. The idea is not that the students should solve the problem but that in addressing the issues which it raises they will experience deep learning about the subject matter. PBL is used in certain medical schools, notably McMaster in Ontario , Canada , Maastrich in The Netherlands and Newcastle , New South Wales , Australia . An important aspect of problem based learning is that the students take charge of their own education. Much research has been undertaken on PBL; broadly it appears to be successful in achieving its aims but some students and staff have difficulty adapting to this style of learning. Score The raw number accorded by a marker to a piece of assessed work. Student selected A piece of study that the student selects for further study as part of the whole course. A substantial part of the medical course is designated for component this by the General Medical Council. Serialist Someone with a learning style who learns details first of all and then later acquires an overall picture of the whole. This is in contrast to the holist. Strategic approach This term implies that the learner pays particular attention to gaining high to studying grades or marks. This may well be combined with either a deep or surface approach to learning. An examination or other assessment procedure at the end of a course or module in which the main aim is to determine the student's state of Summative knowledge, etc. Usually summative assessment provides some mark Assessment which counts towards the final assessment at the end of that year or course. Surface approach This implies that the student learns, for example, by reading without to learning seriously digesting the information. This may readily happen when a student reads from a textbook or receives didactic teaching in a lecture. A surface approach to learning is particularly prone to happen when a timetable is crowded so that a student faces excessive quantities of information. The result is rote memorisation of factual detail. Wet Practical A practical class entailing experimental work with reagents. Objective (outcome) Objective Structured Clinical Examination (OSCE)

MBChB Assessment url https://www.eemec.med.ed.ac.uk/node.asp?id=as000000

Principles and Precepts of Assessment in the MBChB (with short bibliography) Background Since 1998, the Coordinating Assessment Board [CAB] and latterly the Curriculum Executive [CE] have been responsible for overseeing the development of the assessment strategy within the MBChB programme. It has operated within the University Assessment Principles and the Assessment Regulations but has not had a statement of its own Principles or Precepts. It was felt that such a document would provide useful information particularly for new staff involved in teaching and assessment and would be particularly helpful to those considering refining their own course assessments. It should also help clarify and publicise some of the assessment principles that are apparently overlooked e.g. that students must be assessed against the standard expected at that stage of learning. This document has therefore been written to describe the principles and fundamental rules the CAB and CE have invoked in guiding their assessment strategy and draws on current best practice and evidence from the literature. A short bibliography in support of the document is given below. 1. The assessment strategy is designed, coordinated, supervised and reviewed by the Curriculum Executive within the University Assessment Principles http://www.aaps.ed.ac.uk/regulations/AsstPrinciples.htm and Assessment Regulations http://www.aaps.ed.ac.uk/regulations/exam.htm and taking cognisance of the Code of Practice of the Quality Assurance Agency for Assessment of Students: http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section6/COP_AOS. pdf The Curriculum Executive reports to the Board of Studies and the College Undergraduate Studies Committee. Individual Year Committees may pass proposals to the Curriculum Executive for discussion and approval. 2. The assessment strategy and practices should be informed by evidence, both from the education literature and from local evaluation including feedback from students past and present, academic teaching staff, Foundation Years educational supervisors and tutors, external examiners, and audit of the assessment data. 3. Assessment should be designed as an integral part of learning and consideration must be given to its usefulness or value. Several factors contribute to the value of an assessment including its validity (whether the assessment measures what it claims to measure), reliability (the degree to which the measurement is accurate and reproducible), impact on future learning and practice, acceptability to learners and faculty, and costs1. 4. The purposes of assessment, its processes, criteria and regulations should be explicit and communicated clearly for the benefit of students, staff and examiners. 5. Staff should be offered training in assessment matters to ensure they can competently create, apply and review assessments: staff are encouraged to seek out such training. 6. Timing of assessments will depend on their purpose and their intended influence on students’ private study and approach to learning. Thus examinations are often delivered at the end of a block of teaching making assessment hurdles relatively small but frequent to encourage continuous effort by students. There is also a place for examinations at the end of a year to assess the synthesis of knowledge and skills from the modules and curriculum themes and the ability to apply that learning.

7. Since formative feedback aids learning, students should be apprised of their performance whenever possible and due regard given to frequent informal feedback since dialogue helps students interpret the feedback and apply it. Efforts should be made to develop efficient systems that improve formative feedback to students. 8. Opportunities for academic feedback should be designed into the curriculum and include: interactive teaching and practical sessions, problem-based learning and other tutorials, in-course clinical tests, practice tests, online tests with or without computer-assisted learning, formal examinations, portfolio assignments, and end of module review of performance. 9. Feedback may take many forms: verbal critique to individuals or groups; structured written review; commentary to individuals, groups or a class on EEMeC (the virtual learning environment); review of test material with a class; exam results with distribution data; and appraisal discussions. 10. Assignments designed purely for formative assessment are to be encouraged. 11. Completing formative assessments may be stated as a requirement for a course, to emphasise to students their importance as a learning opportunity. 12. Self and peer assessments are to be encouraged as a means of helping students benchmark and improve their own performance. This may be achieved by engaging with marking criteria and learning from others’ approaches to assessment tasks. 13. Formative feedback should be provided following summative assessments wherever possible and is essential when a student faces a resit. 14. Student performance in assessments should inform developments in the curriculum, including teaching methods, future assessments and standards. 15. A programme of integrated assessments around physiological systems and clinical specialties fits best with the curriculum design. 16. A system of partial compensation allows some conjunctive assessments (i.e. individual hurdles without compensation) yet recognises the compensatory nature of some of the knowledge and skills in this integrated programme. 17. Students must pass the specified hurdles to progress to the following year or to graduate. If a specified hurdle is failed there is normally one opportunity to resit, unless there are mitigating circumstances. 18. Assessments will comprise examinations, time-limited or otherwise, and in-course assessment using a variety of tasks and media to match the attitudes, behaviour, knowledge, cognitive skills and competencies required of a medical undergraduate and the range of situations in which they will be applied. This approach also acknowledges that different students favour different media to demonstrate their achievements. 19. Assessments will address published learning objectives and the format will aim to encourage students to adopt forms of learning that match the real-life competences under test and that can be sustained throughout their careers. 20. Assessment of students must make reasonable adjustments, as defined under the Disability Discrimination Act (DDA), for those with a disability and a learning profile, determined in consultation with the Disability Office and the College Academic Adjustments Officer. 21. The format for assessment must be made accessible to all students unless the format itself is integral to assessing a relevant competence standard as defined

under the Amendment to the DDA 2006. Care should be taken to familiarise students with the format before summative assessments. 22. Students must demonstrate appropriate professional behaviour (also termed Personal Professional Development in the curriculum) as well as academic competence to graduate and/or to provisionally register with the General Medical Council. Concerns raised are reported to the relevant Year Director and Board of Examiners who make decisions to support students and protect patients and others. 23. Difficulties within personal professional development are reported to the Director of Studies whose role is to offer students immediate guidance and support. The Director of Studies, Year Director or Board of Examiners may refer students with complex or multiple difficulties to the Professional Development Committee for further pastoral support. 24. Assessments in personal professional development will be collated throughout the 5-year programme to ensure difficulties are addressed and resolved and appropriate ongoing help is offered to students who need it. 25. In cases where a student has persistent or severe problems that might potentially affect their ability to demonstrate ‘Good Medical Practice’ as described by the General Medical Council, the Board of Examiners or Year Director will refer the student to the College Fitness to Practise Committee. 26. Judgements on performance in assessments, outcomes for students and their progress should be made against criteria that are clearly communicated to students, staff and examiners. Standards set must be realistic and students’ performance will be judged against the standard expected at their stage of studies and not against that of a graduating student, foundation doctor or subject expert. 27. Students’ work may be marked using a numeric scale (a score) that is later translated to the Common Marking Scheme using a recognised standard-setting procedure. 28. Assessments in Years 1 and 2 will refer in detail to the material of the immediately preceding semester but will also recall important applications of knowledge acquired in previous semesters. 29. Comprehensive double-marking of assignments or exams should be used judiciously: it is important where a single mark or outcome has a significant effect on the student’s progress e.g. when a portfolio item has been awarded a fail mark. Otherwise it may be more effective, in terms of reliability and generalisability of the results, to increase the number of singly marked items. Double-marking of a sample of assessments will be undertaken by internal and external examiners as a quality assurance measure and systems will be in place to deal fairly with discrepancies. 30. Assessments must be fair to all and have clearly described processes that deal with special circumstances and appeals in a sensitive and equitable manner. Approved by Curriculum Executive 2 October 2007

1. Van der Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ 1996;1:41-67

A Short Bibliography in Support of the Principles and Precepts Schuwirth L. Van Der Vleuten CPM. How to design a useful test: the principles of assessment. Edinburgh: Association for the Study of Medical Education; 2006 Heywood J. Assessment in Higher Education. 3rd Edition. London: Jessica Kingsley Publishers; 2000 Brown G, Bull J. Assessing Student Learning in Higher Education. London and New York: Routledge; 1997 Wood D. Formative Assessment. Edinburgh: Association for The Study of Medical Education; 2006 Norcini JJ. Workplace-based assessment in clinical training. Edinburgh: Association for the Study of Medical Education; 2006 Cantillon P. Hutchison L. Wood D. ABC of Teaching and Learning in Medicine. London: BMJ Publishing Group; 2003.