DAAD Quality Assurance in Higher Education and Bologna Reforms Workshop: Quality Assurance at Programme Level – the case Engineering Tbilisi, Sep 29, 2010
From educational objectives to learning outcomes
www.asiin.de
Content
1. Principles and definitions 2. Reference frameworks for learning outcomes 3. From educational objectives to learning outcomes 4. Exercise 1: Defining programme learning outcomes 5. Exercise 2: Defining module learning outcomes
2
“Quality Assurance at Programme Level”
>What is Quality? •Measurability •Certain objectives are met, >What is Quality Assurance? •The process of making sure that the objectives defined correspond to the (learning) outcome of the educational process
Learning outcomes: Definition
Learning outcomes are statements of what a student should know, understand or be able to do at the end of a learning activity. The learning activity could be a lecture, a lab, a seminar, a module (combining different forms of teaching and learning) or an entire programme. Learning outcomes must be simply and clearly described. Learning outcomes must reflect the level of capability as well as the range. Learning outcomes must be capable of being validly assessed. 4
Learning outcomes: Definition
There is a distinction between intended learning outcomes (ILOs), i.e. written statements in a course/programe syllabus, and achieved learning outcomes (ALOs), i.e. those results that students actually have (and can be demonstrated to have) achieved.
5
Quality: Definition
In our context, Quality means: •Intended Learning Outcomes (ILO) are met, •i.e. Actual Learning Outcomes correspond to the ILO
Correspondence of ILOs and ALOs
ILOs
Input of HEI
ALOs
learning outcomes / competence profile knowledge, skills, competencies job / career profiles occupational area(s) work environment(s)
support processes e.g. student services, staff resources, infrastructure, programme structure, curriculum, didactic concept, quality assurance
correspondence of educational objectives and learning outcomes results of outcomes assessment and internal/external evaluation
Assessment of the process: coherence of goals, input + outcomes 7
Content
1. Principles and definitions 2. Reference frameworks for learning outcomes 3. From educational objectives to learning outcomes 4. Exercise 1: Defining programme learning outcomes 5. Exercise 2: Defining module learning outcomes
8
Learning outcomes: Reference Frameworks Learning outcomes for degree programmes and modules/courses should refer to established qualification frameworks to ensure compatibility, for example: European Qualification framework for Higher Education (“Dublin Descriptors”). EUR-ACE Framework Standards for Engineering Higher Education (EUR-ACE/ENAEE). ASIIN e. V. qualification framework for degree programmes in engineering, informatics, the natural sciences and mathematics. …
9
• Bologna: System of qualification frameworksLearning outcomes - European level EQF/Dublin Descriptors Learning outcomes - National level National qualification descriptors e. g.: Knowledge and understanding Skills and competences
Learning outcomes - Programme level
LO:s COURSE/MODULE LEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
10
• Bologna: System of qualification frameworksLearning outcomes - European level EQF/Dublin Descriptors Learning outcomes - National level National qualification descriptors e. g.: Knowledge and understanding Skills and competences
Learning outcomes - Programme level
LO:s COURSE/MODULE LEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
11
• Bologna: System of qualification frameworksLearning outcomes - European level Learning outcomes - European level EQF/Dublin Descriptors
Translation into sectoral Frameworks e.g. EUR-ACE for Engineering
Euro-Inf for Informatics
Eurobachelor /master for Chemistry
12
• Bologna: System of qualification frameworks
Learning outcomes - European level
Translation into sectoral Frameworks e.g. EUR-ACE for Engineering
•
Euro-Inf for Informatics
Eurobachelor /master for Chemistry
Eg. “First Cycle graduates should have the ability to apply their knowledge and understanding to analyse engineering products, processes and methods” 13
Bologna: System of qualification frameworks
Learning outcomes - European level
Translation into sectoral Frameworks e.g. EUR-ACE for Engineering
•
Euro-Inf for Informatics
Eurobachelor /master for Chemistry
Eg. “Graduates having completed a first cycle degree should have demonstrated: understanding the complexity of information problems and the feasibility of their solution” 14
Bologna: System of qualification frameworks
Learning outcomes - European level
Translation into sectoral Frameworks e.g. EUR-ACE for Engineering
•
Euro-Inf for Informatics
Eurobachelor /master for Chemistry
Eg. “Graduates having completed a first cycle degree are expected to be able to conduct risk assessments concerning the use of chemical substances and laboratory procedures.” 15
Bologna: System of qualification frameworks
Learning outcomes - European level
Translation into sectoral Frameworks e.g. EUR-ACE for Engineering
Euro-Inf for Informatics
Eurobachelor /master for Chemistry
16
Bologna: System of qualification frameworks Learning outcomes - European level + sectoral Frameworks e.g. EUR-ACE for Engineering
Euro-Inf for Informatics
Eurobachelor /-master for Chemistry
Learning outcomes - National level National qualification descriptors Learning outcomes - Programme level
LO:s COURSE/MODULE LEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
17
Bologna: System of qualification frameworksLearning outcomes - European level EQF/Dublin Descriptors Learning outcomes - National level National qualification descriptors e. g.: Knowledge and understanding Skills and competences
Learning outcomes - Programme level
LO:s COURSE/MODULE LEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
18
Bologna: System of qualification frameworksLearning outcomes - European level EQF/Dublin Descriptors Learning outcomes - National level National qualification descriptors e. g.: Knowledge and understanding Skills and competences
Learning outcomes - Programme level
LO:s COURSE/MODULE LEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
LO:s COURSE/MODULE LEVEL
LO:s TASKLEVEL
LO:s TASKLEVEL
19
Professional and academic mobility – A European approach Completion of the EHEA until 2020: choice of a Promotion of comparability and compatibility of Paris Completion of the European Higher Education Area until and compatibility of degrees: wide variety of high quality courses; benefits of Promotion of comparability degrees: (1998) 2020, in which citizens can choose from a wide and Two cycle study system, ECTS, adoption of a system of transparent order of high quality courses and benefit smooth recognition procedures easily readable and comparable degrees, Diploma from smooth recognition procedures Three cycle study system, ECTS, Supplement, permeability of study structures Diploma Bologna Goals: European knowledge society characterized by European knowledge society Goals: Supplements, Permeability of study Promotion of the European Dimension in Higherstructures(1999) high mobility and permeability, lifelong learning Education characterized by high mobility and permeability, Prag (2001) lifelong learning
Academic Mobility
Recognition of degrees
Transnational cooperation in QA
Berlin (2003) Bergen (2005)
Common European Market The European Directive on recognition of London The European Directive on the the recognition of Common European Market Goals: freedom of movement of workers, (2007) professional qualifications : professional qualifications : right of establishment, freedom of ofservices. Goal: „ A clear, secure and quick system for the Goals: freedom of movement of workers, right Leuven recognition of qualifications in the field of the regulated establishment, freedom of services. Goal: „ A professions clear, secure and system for the (2009) is required to ensurequick free movement.“ Declaration ofof Lisbon, Lisbon, Barcelona: „to Declaration Barcelona: „to make Europe the make recognition of qualifications inshould thefacilitate field of the The EC institutions and member states most competitive and dynamic knowledge-based Europe the most competitive employment and the provision of services through economy in the world.“ and dynamic regulated wholesale professions isofrequired to ensure free consolidation the existing regimes of knowledge-based economy in the world.“ professional recognition in the regulated professions. movement.“
Professional Mobility
20
ASIIN e. V. qualification framework: Definitions “Qualification”: Formal outcome of an assessment and validation process. “Learning outcomes”: Statements of what a learner knows, understands and is able to do on completion of a learning process, defined in terms of knowledge, skills and competences. “Knowledge”: Outcome of the assimilation of information through learning (theoretical and / or factual knowledge) . “Skills”: Ability to apply knowledge to complete tasks and solve problems (cognitive skills such as logical, intuitive and creative thinking and practical skills such as manual dexterity and the use of methods, materials, tools and instruments). “Competence”: Proven ability to use knowledge, skills and personal, social and / or methodological abilities, in work or study situations and in professional and / or personal development. Based on the European Qualification Framework 21
ASIIN e. V. QF: Bachelor graduates...
22
ASIIN e. V. QF: Bachelor graduates...
23
ASIIN e. V. QF: Master graduates have... have...
24
ASIIN e. V. QF: Master graduates have... have...
25
Benefits of learning outcomes
Help to explain more clearly to students what is expected of them and thus help to guide them in their studies. Help teachers to focus more clearly on what exactly they want students to achieve in terms of knowledge and skills. Help teachers to define the assessment criteria more effectively. Help to provide guidance to employers about the knowledge and understanding possessed by graduates of programmes. Important instruments in achieving comparability and transparency of qualifications and facilitating mobility (of students and graduates/professionals). 26
Potential problems with learning outcomes
Could limit learning if learning outcomes written within a very narrow framework – lack of intellectual challenge to learners. Danger of assessment-driven curriculum if learning outcomes are too confined. Could give rise to confusion among students and staff if guidelines not adhered to when drawing up learning outcomes, etc. Must reflect established national and cultural standards and practices in order to be accepted and useful.
27
Content
1. Principles and definitions 2. Reference frameworks for learning outcomes 3. From educational objectives to learning outcomes 4. Exercise 1: Defining programme learning outcomes 5. Exercise 2: Defining module learning outcomes
28
Student--centred approach Student Learning outcomes place an emphasis… on the learner, and on the learner’s ability to do something.
Traditional approach: focus on teaching – description of aims and objectives and use of terms like “know”, “understand”, “be familiar with”.
Learning outcomes approach: Focus on what you want the student to be able to – use of terms like “define”, “list”, “name”, “recall”, “analyse”, “calculate”, “design”, etc.
29
From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes.
What is characteristic for graduates, what will they do in which environment?
How do the objectives translate into a detailed competency profile of the graduates?
How does the curriculum support learning outcomes and programme objectives? 30 30
From programme objectives... The overall objective of the undergraduate programme “Chemical and Metallurgical Process” is to educate professionals who are able to plan, control and monitor processes as well as to develop and operate chemical and metallurgical processes. In particular they should supervise processes, design and develop laboratory tests, identify and analyse problems and create and implement solutions. They should manage resources effectively and work in teams in an effective and creative manner. Students should be committed to quality, environmental protection and safety as well as to ethics.
31
...to programme learning outcomes (1) Graduates should be able to operate and control chemical and metallurgical processes based on their knowledge of chemistry and of unit operations. They also should be able to apply their knowledge of chemistry, physics, mathematics and technology in chemical and metallurgical processes, to perform chemical analyses and metallurgical tests, and interpret results in order to improve processes. They should also be capable of designing chemical and metallurgical processes and of identifying, analysing and solving technical problems. 32
...to programme learning outcomes (2) Furthermore, graduates should be able to work in teams, to know contemporary aspects of their profession and be prepared for lifelong learning. They should also apply environmental protection standards and demonstrate an ethical awareness towards their work. Furthermore, they should be able to manage both material and human resources effectively.
33
From Educational Objectives to Learning Outcomes: Example Educational Objective: Graduates are team oriented
Programme Learning Outcome: Graduates are able to function effectively as member of a team that may be composed of different genders or cultural backgrounds
Module Learning Outcome (e.g. for a module “Team Project”) : Knowledge on: Methods and instruments of information research supported by electronic media; presentation and moderation techniques and their application; project planning tools. Skills: They are able to… - gather information on a specific problem relevant for their project team and - communicate this information effectively; - assume the designed function within a team and independently fulfil the responsibilities associated with it; - accept the function and the personalities of other team members; - actively share their knowledge with other team members; - listen to other team members. 34
From Educational Objectives to Learning Outcomes: Example Educational Objective: Graduates take legal and ethical considerations into account when taking decisions
Programme Learning Outcome: Graduates are aware of the social and ethical responsibilities that underpin their actions, and of the professional ethical principles and standards that apply to their chosen discipline, and able to apply them.
Module Learning Outcome (e.g. as part of a module “Software Engineering”): Knowledge on: Codes of ethics, laws, standards and guidelines relevant to software engineering; positions in the current discussion on software engineering and its impact on society. Skills: They are able to… - make ethical decisions when faced with ethical dilemmas, with reference to general principles of ethics as well as codes of ethics for software engineering; - describe and apply the laws that affect software engineers, including laws regarding copyright, patents, and other intellectual property; - describe the effect of software engineering decisions on society, the economy, the environment, their customers, their management, their peers, and themselves; - understand the role of standards and standards-making bodies in software engineering. 35
The task of writing learning outcomes Learning outcomes should be clearly written so that they are understood by students, colleagues and other relevant stakeholders. When writing learning outcomes it may be helpful to focus on what you expect students to be able to demonstrate upon completion of the module or programme. Focus on “specialist skills” as well as on “social skills”. Avoid complicated sentences. If necessary use more than one sentence to ensure clarity.
36
The task of writing learning outcomes List the learning outcomes using a phrase like “On successful completion of this module, students should be able to: … [list of learning outcomes].” General recommendation: 5 to 8 learning outcomes per module. Avoid certain words………. The key word is DO and the key need in drafting learning outcomes is to use active verbs. Avoid ambiguous verbs like “understand”, “be familiar with”, “be exposed to“, “be aware”, “appreciate”… 37
Content
1. Principles and definitions 2. Reference frameworks for learning outcomes 3. From educational objectives to learning outcomes 4. Exercise 1: Defining programme learning outcomes 5. Exercise 2: Defining module learning outcomes
38
From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes.
What is characteristic for graduates, what will they do in which environment?
How do the objectives translate into a detailed competency profile of the graduates?
How does the curriculum support learning outcomes and programme objectives? 39 39
Exercise 1: Programme learning outcomes a) Group: Chose one of your department’s undergraduate programmes and write down the educational objectives. (10 minutes). b) Individual: Formulate 4 key learning outcomes for the selected programme, based on its educational objectives (adhesive labels). (10 minutes) c) Group: The group members collect their proposals on a joint sheet (5 minutes). d) Group: Jointly sort the proposals and group them by content/topic (10 minutes). e) Group: Formulate one programme learning outcome for each group of proposals and thus create a joint list (10 minutes). Please be prepared to briefly present your result to the plenary. Time available:
45 minutes group work 40
Exercise 1: Programme learning outcomes Objectives: • You have formulated programme learning outcomes based on its educational objectives. • You have experienced a typical situation of a programme coordinator who practices outcomeoriented curriculum development.
41
- Break -
42
Content
1. Principles and definitions 2. Reference frameworks for learning outcomes 3. From educational objectives to learning outcomes 4. Exercise 1: Defining programme learning outcomes 5. Exercise 2: Defining module learning outcomes
43
From Educational Objectives to Learning Outcomes Correspondence of educational objectives, programme learning outcomes and module learning outcomes.
What is characteristic for graduates, what will they do in which environment?
How do the objectives translate into a detailed competency profile of the graduates?
How does the curriculum support learning outcomes and programme objectives? 44 44
Exercise 2: Module/course learning outcomes a) Group: Chose one of the programme learning outcomes you have just formulated and follow the instructions b) to e) b) Individual: Formulate knowledge and skills that describe the chosen learning outcome in more detail (adhesive labels). Question: What does one have to be able to do to demonstrate that the learning outcome has been achieved? (10 minutes) c) Group: The group members collect their proposals on a joint sheet (5 minutes). d) Group: Jointly sort the proposals and group them by content/topic (10 minutes). e) Group: Formulate/sum up one skill for each group of proposals and thus create a joint list of skills (15 minutes). Please be prepared to briefly present your result to the plenary. Time available:
40 minutes group work 45
Exercise 2: Module/course learning outcomes Objectives: • You have transformed a learning outcome at the programme level into concrete and measurable learning outcomes at the level of a specific teaching module. • You have experienced a typical situation of a programme coordinator who practices outcomeoriented curriculum development.
46
Exercise 2: Module/course learning outcomes
Did you find these exercises challenging? In how far?
47
