history: from those who wrote it The process of problem-based learning: what works and why Henk G Schmidt,1 Jerome I Rotgans2 & Elaine HJ Yew3

OBJECTIVES In this review, we portray the process of problem-based learning (PBL) as a cognitive endeavour whereby the learner constructs mental models relevant to problems. Two hypotheses are proposed to explain how learning is driven in PBL; an activation– elaboration hypothesis and a situational interest hypothesis. METHODS Research relevant to these hypotheses is discussed. In addition, research studying the effects of various support strategies used in PBL is reviewed. Finally, we summarise a number of recent studies in which a new ‘microanalytical’ methodology was used to trace the process of PBL in the natural classroom setting. CONCLUSIONS We conclude that there is considerable support for the idea that PBL works because it encourages the activation of prior knowledge in the small-group setting and provides opportunities for elaboration on

that knowledge. These activities facilitate the comprehension of new information related to the problem and enhance its long-term memorability. In addition, there is evidence that problems arouse situational interest that drives learning. Flexible scaffolding provided by cognitively and socially congruent tutors also seems to be reasonably effective, as opposed to ‘hard’ scaffolding represented by, for instance, worksheets or questions added to problems. Small-group work protects against dropout and encourages students to study regularly. Initially, students do not study much beyond the learning issues generated; the development of personal agency in selfstudy needs time to develop. The extent of learning in PBL results from neither group collaboration only (the social constructivist point of view) nor individual knowledge acquisition only; both activities contribute equally to learning in PBL.

Medical Education 2011: 45: 792–806 doi:10.1111/j.1365-2923.2011.04035.x 1 Department of Psychology, Erasmus University, Rotterdam, the Netherlands 2 Centre for Research in Pedagogy and Practice, National Institute of Education, Nanyang Technological University, Singapore, Singapore 3 Centre for Educational Development, Republic Polytechnic, Singapore, Singapore

792

Correspondence: Professor Henk G Schmidt, Department of Psychology, Erasmus University, Burgemeester Oudlaan 50, Rotterdam 3062 PA, the Netherlands. Tel: 00 31 10 408 1749; Fax: 00 31 10 408 9009; E-mail: [email protected]

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning

INTRODUCTION

In this article we review current knowledge about what works in problem-based learning (PBL) and why it works. We will not attempt here to review the entire literature on the topic; the database PubMed alone presents over 5000 articles that use the term ‘problem-based learning’ in their titles or abstracts. Nor will we make another attempt to review outcomes of PBL schools in comparison with those using conventional curricula. Curriculum comparison studies have been reviewed extensively over the past 20 years.1–6 Here, we will focus on reviewing studies that have examined what happens to the learner in PBL in order to elucidate the process that unfolds when students try to learn new material through this approach to learning. However, we must first establish which learning process we refer to. In an earlier paper, we suggested that there are at least three different perspectives on PBL in the literature: PBL as a ‘process of inquiry’; PBL as ‘learning to learn’, and PBL as a ‘cognitive constructivist’ approach, which defines the purpose of PBL as helping students to construct mental models of the world.7 Fortunately, all three perspectives concur on the following PBL-defining characteristics: (i) problems are used as a trigger for learning; (ii) students collaborate in small groups for part of the time; (iii) learning takes place under the guidance of a tutor; (iv) the curriculum includes a limited number of lectures; (v) learning is student-initiated, and (vi) the curriculum includes ample time for self-study.8–12 As only the cognitive constructivist approach has led to a sizable volume of research, we will confine our review to this interpretation. In this paper we will first briefly describe the process of PBL, emphasising that it is a special way of acquiring knowledge of a domain. Secondly, we will interpret learning in this approach in terms of two hypotheses derived from cognitive psychology – the activation–elaboration hypothesis and the situational interest hypothesis – and review the evidence supporting these. Thirdly, we will look at research into the educational aids intended to support learning based on problems, including: the problems; the tutorial group; the tutor; scaffolds, and the self-directed learning activities of the students. Finally, we will discuss a number of recently conducted studies in which attempts were made to chart the learning process in PBL in the natural classroom setting using a micro-analytical methodology.

THE PROCESS OF PBL FROM A COGNITIVE CONSTRUCTIVIST POINT OF VIEW

The following description of the process of PBL will be brief because its constituent elements from the perspective of the cognitive constructivist framework have been sketched previously.7 In PBL, learners are presented with a problem in order to activate their prior knowledge. This prior knowledge is then built upon further as the learners collaborate in small groups to construct a theory or proposed mental model to explain the problem in terms of its underlying causal structure. As learners continue to study related resources, their initial mental model is further modified and refined. Moreover, as the learners’ preconceptions are activated, they become more easily able to identify gaps in their prior knowledge, thus enabling better learning to take place (the activation–elaboration hypothesis). Motivational processes support these cognitive changes. Situational interest is aroused by the enigmatic nature of the problem and acts as the motivating force that drives the learner to engage with the literature and to continue to seek relevant information until his or her hunger for new information related to the problem is satisfied (the situational interest hypothesis). These two hypotheses have been tested in a number of studies that will be reviewed in the next section.

EMPIRICAL EVIDENCE RELEVANT TO THE TWO THEORETICAL CLAIMS

In this section, we will summarise the evidence relevant to determining the mechanism by which PBL affects learning. Firstly, we will present studies relevant to the activation–elaboration framework. Secondly, we will review studies that relate to the emergence of situational interest in PBL classrooms. The activation–elaboration hypothesis The literature on the constructive nature of learning and the role of elaboration through self-explanation,13 discussion with peers,14 practising15 or responding to questions16 is extensive. What needs to be demonstrated here is that these processes are also vital to the process of PBL. This involves the review of a number of experiments designed to answer the following questions: 1

Does the initial discussion of a problem lead to the activation of previously acquired knowledge?

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

793

H G Schmidt et al 2 3

Is there evidence that having to explain the problem leads to elaboration? Do these activities facilitate the comprehension and retention of relevant new information?

In a series of early studies that aimed to elucidate the role of prior knowledge activation and elaboration in the initial discussion of a problem, Schmidt and colleagues17–19 presented students who had studied the biological process of osmosis during high school with either a problem unrelated to osmosis or with the following problem: ‘A red blood cell is put in pure water under a microscope. The cell swells and eventually bursts. Another blood cell is added to a solution of salt in water. It shrinks. Explain these phenomena.’ When subsequently asked to recall whatever they remembered about osmosis, the students who had discussed the blood cell problem produced almost twice as many ideas about osmosis as the students involved in an unrelated task. This finding suggests that the initial discussion of a problem does indeed have a considerable effect on the activation and elaboration of previously learned knowledge. The authors were, however, unable to distinguish the influence of activation from the effect of elaboration (the students were prompted to explain) and nor were they able to exclude the possibility that the students simply learned from one another in the course of providing their explanations and thereby gained more knowledge. To resolve these issues, De Grave et al.20 compared the effects of small-group discussion of a problem with those of individual problem analysis and direct prompting for knowledge about osmosis. They discovered that small-group discussion had a larger positive effect than individual problem analysis; direct prompting for knowledge had the smallest effect. The assignment to explain leads students to further elaborate on their prior knowledge even in the absence of other students, but group discussion has a considerable additional effect, suggesting that elaboration on prior knowledge and learning from one another (even before new information is acquired) are potent means of facilitating understanding of problem-relevant information. The text comprehension studies discussed so far had difficulty in studying effects of elaboration (through the explanatory activities required of the students) in isolation. To deal with this problem, van Blankenstein et al.21 introduced a new research paradigm for studying the effects of elaboration in small groups on the learning of individual students. They taped a group discussion about a problem and presented this video to individual students. These students either only

794

watched the video and listened to the discussion, or were prompted to actively provide explanations themselves a number of times while watching the discussion. In this way, the investigators were able to keep the information provided by other students constant over the conditions of the experiment (all participants saw the same video). Subsequently, both groups studied the same problem-related text for the same amount of time and were tested for knowledge twice: immediately and after 1 month. The authors found that immediately after studying the text, the elaboration group had a 28% higher score. After a month this difference had increased to 30%.21 It seems that elaboration in a small group not only facilitates the processing of a study text, but also adds to its longerterm memorability. This view is reinforced by other reports of the long-term effects of PBL.22–24 The situational interest hypothesis The situational interest hypothesis underlying PBL states that problems or puzzles create a desire in students to find out more about the topic, which leads to increased concentration, focused attention and a willingness to learn.25 Situational interest, as the term implies, is not a stable or dispositional form of interest, but is situationally aroused by an intriguing or captivating puzzle or problem. The psychological processes underlying this desire to learn can best be explained in terms of the incongruity hypothesis proposed by epistemic curiosity researchers (see Loewenstein26 for a review). Humans seem to have a natural tendency to wish to make sense of the world: when they encounter something they do not understand or something that violates their expectations, they experience situational interest because this state of affairs makes them aware of a knowledge gap between what they know and what they want to know. This phenomenon has been referred to as a cognitively induced experience of (knowledge) deprivation.27,28 This experience of deprivation initiates information-seeking behaviour intended to close the knowledge gap (i.e. learning occurs).29 As the knowledge gap closes, through the assimilation of new information into existing knowledge structures, situational interest decreases until the knowledge equilibrium is re-established.30 Studies testing the situational interest hypothesis in the context of PBL are limited because situational interest has only become the focus of attention recently (but see31,32). The central assumption of this hypothesis is that a problem presented in the context of learning arouses situational interest because it confronts students more extensively with what they

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning still do not understand. In turn, this perception of ignorance acts as a driving force to engage in learning. To test this idea, Rotgans and associates conducted a number of studies using a short rating scale that mentions the topic to be studied and consists of items such as ‘I want to know more about this topic’ and ‘I think this topic is interesting’.30,33 These studies tested the level of situational interest of students at several time-points, including on their arrival in the classroom, after a relevant problem had been presented and after some initial discussion about the problem. They demonstrated that the presentation of the problem significantly increased the level of situational interest in the students and this increase was maintained during the small-group discussion.30,33

RESEARCH ON PBL SUPPORT STRATEGIES

In this section, we will review research focusing on the means through which learning in PBL is fostered, including: the use of problems; small-group collaboration; tutors; scaffolds, and self-directed learning (SDL). Roles of problems in PBL Most of the research on problems in PBL revolves around the issue of what makes a good problem in the views of tutors and students.34–37 The general message from these studies is that a problem should: (i) be authentic; (ii) be adapted to the students’ level of prior knowledge; (iii) engage students in discussion; (iv) lead to the identification of appropriate learning issues; (v) stimulate SDL, and (vi) be interesting. Of course, a key question concerns how these attributes influence student learning in PBL, such as how problems lead to the identification of appropriate learning issues. Dolmans et al.38 investigated the effectiveness of problems in a course by comparing the learning issues identified by students with those intended by faculty members. The assumption behind this investigation was that an effective problem should lead students to identify the learning issues referred to by the problem designers when they developed the problem. The results of this study revealed that, on average, 64% of the learning issues identified by students matched those intended by faculty, with large differences among individual problems. Interestingly, many of the non-intended learning issues generated by students were found to be considered relevant in hindsight by the problem designers.38 Similar findings were reported by Mpofu et al.39

Soppe et al.40 investigated whether problem familiarity had an influence on student learning. In an experimental set-up, students worked with either a ‘familiar’ or an ‘unfamiliar’ version of the same problem. A measure of perceived problem quality was administered and outcome measures, such as the number of explanations for the problem put forward by the students, the quality of the learning issues derived from the discussion, the amount of time spent on selfstudy and the amount of knowledge gained as measured by a test were also obtained. The results showed that participants in the familiar-problem condition perceived the problem to be of higher quality than participants in the unfamiliar-problem condition. However, no significant differences in learning were found. The authors40 proposed that problems might be improved by making them more relevant to the everyday experience of students, an important consideration also stressed by others.41 A number of studies have taken a more holistic perspective on the role of problems in PBL.42–45 These studies sought to determine the problem’s influence on learning in the context of other variables influencing learning. To this end, they tested causal models of PBL using structural equation modelling in which all key elements of PBL were included. They were able to demonstrate that problem quality (defined in terms of how clear the problem appears to students, its ability to stimulate interest and to trigger group discussion, etc.) as perceived by students is a major source of influence in PBL. Problem quality influences not only the quality of small-group discussion, but also time spent on self-study and interest in the subject matter. Importance of small-group collaboration The cognitive benefits of small-group cooperation have been discussed extensively by various authors46,47 and need no further elaboration here. However, we would like to focus on three other aspects of the small-group tutorial. Firstly, the small tutorial group provides a platform for the development of friendships among students. Secondly, it enables closer contacts between students and teachers compared with those possible in a larger class. Thirdly, the regularity of small-group tutorials in the PBL environment generates peer pressure that is useful in motivating students to be diligent in their self-study and to meet the deadlines for work agreed by the group. These two non-cognitive side-effects of small-group collaboration have been found to be advantageous in preventing dropout from school48,49 and may be a reason why students in PBL

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

795

H G Schmidt et al curricula tend to graduate faster than students at conventional schools.50–52 Role of the tutor The tutor’s role in a PBL tutorial differs from that in a conventional tutorial. In PBL, tutors are expected to facilitate or activate student learning and to promote effective group functioning by encouraging the active participation of all members, monitoring the quality of learning and intervening when necessary.53–56 Tutors also play active roles in the scaffolding of student learning by providing a framework that students can use to construct knowledge on their own.57 Because it encourages students to think more deeply and offers some modelling of the types of questions students should be asking themselves during problem solving, the tutor–student relationship can be viewed as supporting a type of cognitive apprenticeship.10,58 In view of this shift in the tutor’s role in the studentcentred PBL process, many studies have sought to better understand how tutors contribute to student learning in PBL. One subject of considerable debate has centred around the importance of the tutor’s subject matter expertise: is it sufficient for tutors to have good facilitation skills or do they also need substantial knowledge of the subject matter? Results from several studies demonstrate that tutor expertise has a significant effect on student learning outcomes,59 whereas other studies indicate that it has no noticeable effects.60,61 One hypothesis that may explain this contradiction in findings is that the subject matter expertise of the tutor impacts on student learning more significantly when the cues and scaffolds within the problems and resources are insufficient to guide students in the process of identifying what is important to study. In such situations, students are more likely to depend on their tutor for guidance and thus a tutor who is more knowledgeable in the subject matter is of more benefit.62 Another group of studies investigated the influence of tutors’ content expertise on the tutorial process. Silver and Wilkerson63 found that tutors with subject matter expertise were more inclined to play a directive role in the tutoring process, to speak more often and for longer periods, to supply more direct answers to questions posed by students, and to suggest more points for discussion. They concluded that these tutor behaviours might have a negative impact on the development of students’ skills in active SDL and also in collaborative learning. By contrast, Eagle et al.64 found that students with tutors who were content

796

experts were able to generate more than twice as many learning issues for SDL and that these learning issues were almost three times more congruent with case objectives compared with those generated by students who were guided by non-experts. Moreover, compared with the latter group, students with contentexpert tutors put in double the amount of time in selfstudy. Similarly, Davis et al.59 found no significant differences between expert-led and non-expert-led groups in terms of teacher-directed and studentinitiated interactions, but instead demonstrated increased student achievement and satisfaction in groups led by experts. Both groups of authors59,64 suggest that the expert tutors, by virtue of their subject knowledge expertise, were better at posing questions at critical moments, thus positively influencing student learning. However, although the proficiency of content experts in using their subject matter expertise to direct student discussion has positive effects on student learning, their knowledge of when and how to use this expertise to facilitate learning is more beneficial. Thus, ideally, a tutor should be expert in both the respective subject matter and in facilitating student learning processes. One theory of the effective tutor merges these two qualities.65 A key idea in this theory is the concept of ‘cognitive congruence’, which is defined as the tutor’s ability to express him or herself in the language of the students, using the concepts they use and explaining things in ways they can easily grasp. Schmidt and Moust66 suggested that both subject matter expertise and social congruence were necessary conditions for cognitive congruence to emerge. In this context, social congruence refers to interpersonal qualities such as the ability to communicate informally and empathically with students, and hence the ability to create a learning environment that encourages the open exchange of ideas. Thus, it was hypothesised that a tutor who is more socially congruent and better able to use subject matter expertise would be more cognitively congruent. Using structural equation modelling, the authors66 demonstrated that both social congruence and subject expertise influenced cognitive congruence, which, in turn, influenced tutorial group functioning and thus indirectly affected the level of student achievement by increasing the time spent on selfstudy. Social congruence directly influenced group functioning during the problem-solving process and the subject matter expertise of the tutor had a slight direct positive impact on student achievement. Hence, this study showed that effective tutoring that results in better student achievement requires the tutor to have both content knowledge and the ability

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning to interact with students on a personal level, as well as to utilise language students easily understand.66 Chng et al.67 hypothesised that tutors who exhibit more cognitively congruent behaviours would influence knowledge construction and acquisition at each learning phase in the PBL process. Thus, students who are under the tutelage of such tutors would be more extensively involved in the construction of knowledge (i.e. they would recall more relevant concepts in each PBL phase) and would ultimately achieve better results at the end of the learning process. Tutor behaviours were assessed by students and the tutors were subsequently divided into three groups according to their levels of subject matter expertise, cognitive congruence and social congruence. Student achievement at the end of the PBL cycle was measured using an essay test. An analysis of covariance (ANCOVA) using students’ pre-existing grade point average (GPA) scores as the covariate found that, contrary to the authors’ original hypothesis, only the social congruence of tutors had a significant influence on the students’ learning process.67 This effect of social congruence was evident in the total number of concepts recalled at the end of the PBL phases of problem analysis, SDL and reporting. No significant effects were found for the subject expertise and cognitive congruence of the tutor on any of the learning phases in the PBL process.67 (However, in line with the findings of Schmidt and Moust,66 all three tutor behaviours had a significant effect on final student achievement.) The results indicate that the social congruence of the tutor influences the learning process more substantially than cognitive congruence and subject matter expertise, at least in this educational context. The willingness of a tutor to establish an informal relationship with his or her students and to display an attitude of genuine interest therefore has significant impact. This is possibly because socially congruent tutors are able to create a non-threatening learning environment and to develop strong tutor–student relationships that support the open exchange of ideas and questions and promote student engagement in the learning process. Some studies on tutors move away from investigating the tutor’s role in influencing student achievement alone and instead focus on the student learning process and issues such as how and when a tutor should intervene in the event of difficult incidents in tutorial groups.68,69 Hmelo-Silver and Barrows70 describe in detail how ‘collaborative knowledge building’ is facilitated. By analysing the discourse that took place between five medical students and their

facilitator as they worked on a problem over 5 hours in two sessions, these authors demonstrate how the facilitator can support knowledge building through the use of open-ended questions that serve as scaffolds for students.70 The use of scaffolds in PBL From the descriptions above, it is clear that the role of the tutor as a scaffold to facilitate meaningful learning is one that is generally agreed upon. Although Kirschner et al.71 suggest that PBL is a minimally guided approach and is therefore less effective and efficient in helping students learn, others argue that PBL provides extensive guidance and scaffolding to help students learn.72,73 The metaphor of scaffolding refers to the temporary support provided for learners to help them complete a task they would otherwise not be able to complete on their own.74 Although scaffolds can take multiple forms, Saye and Brush75 suggest that most scaffolds can be classified as either ‘soft’ or ‘hard’. Soft scaffolds are dynamic and refer to tutor actions that support specific learner needs, as described in the section on the role of the tutor in PBL.75–77 By contrast, hard scaffolds are static supports that can be developed in advance based on typically expected learner difficulties associated with a task.75 Such scaffolds can take the form of computer- or paper-based cognitive tools, such as worksheets.78 In line with the metaphor, scaffolds should gradually be withdrawn or fade from use as the learner becomes increasingly responsible for his or her own learning.79 For example, although novices in a PBL environment may be supported initially with some resources that scaffold their learning, as their expertise increases, these students should be provided with fewer resources. In this way, students develop as independent learners through a form of flexible scaffolding.73 Studies investigating the use and impact of hard scaffolds in the PBL context are rather limited. A study by Simons and Klein80 examined the impact of hard scaffolds on the learning outcomes of middleschool learners during the implementation of a PBL unit. Students were subjected to one of three experimental conditions, including one in which no scaffolding was provided, one in which optional scaffolding was provided, and one in which students were required to use all the scaffolds provided. Results showed that students in the optionaland compulsory-scaffold conditions performed significantly better than students in the no-scaffold condition. In addition, those in the compulsory-

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

797

H G Schmidt et al scaffold condition produced better organised project notebooks containing a higher percentage of relevant entries. The PBL environment described in this study was one with very little consistent teacher support (soft scaffolding), in which students generally performed poorly. Simons and Klein80 therefore concluded that hard scaffolds seem to enhance student performance, especially under circumstances of limited teacher support. Another study investigating the effect of worksheets as a scaffolding tool on student achievement in a PBL environment was carried out by Choo et al.81 Fortyeight teams (241 students) were randomly assigned to one of two experimental groups; students in one group were provided with a worksheet and students in the other group were not. No statistically significant difference emerged between the two groups in terms of their post-lesson concept recall tests. The findings of Choo et al.81 therefore suggest that scaffolds such as worksheets may not play a significant role in enhancing student learning within this PBL context. Furthermore, results from a survey administered to the students in this study indicated that the strongest factor to impact on their learning was the tutor, followed by the tutorial group; the worksheet was rated as having the lowest influence.81 Two other experiments have examined the effect of hard scaffolds in the form of questions provided as guidelines for self-study in addition to the problem.82,83 In one of these,82 the presence or absence of the scaffolds did not matter; in the other,83 scaffolding was even detrimental to achievement.

cluded that the learning issues produced during group discussion may not represent the only factors on which students base their decisions of what to study during self-study.86 Other factors may be involved, such as tutor suggestions, content covered in previous courses, literature found during self-study and the nature of the learning resources available. Interestingly, first-year students tended to focus on the learning issues agreed upon more than older students.86 It appears that students become more selfdirected as the years of study progress. A similar finding was reported by van den Hurk et al.,87 who found that students in their first year of study adhered strictly to the learning issues, whereas in later years students pursued their personal learning interests to a larger extent. In addition, students who tended to study beyond the agreed learning issues spent more time on individual study and performed better on achievement tests. As SDL is considered to be a key element in PBL, helping students develop the ability to regulate their own learning is an important priority. Some studies have demonstrated outcomes that can be considered to result from the emphasis on SDL in PBL. Students in a PBL curriculum have, for instance, been shown to borrow more books from the library than students in conventional curriculum schools.88–90 These findings suggest that PBL students are more independent learners and take more personal responsibility for their learning.

CHARTING PBL IN THE CLASSROOM

In conclusion, the effects of scaffolding in PBL are rather inconclusive and more research is needed here. Problem-based learning was originally developed in medical schools for use with relatively mature and motivated learners, a context in which hard scaffolds may have been of limited value. However, in view of the increasing implementation of PBL by educators of students at different levels of education and in various disciplines,84,85 research to shed further light on the role of scaffolding to support student performance in PBL is necessary. Self-directed learning Dolmans et al.86 examined the relationship between student-generated learning issues and self-study. As in an earlier study,38 they found considerable overlap between the learning issues identified by students and the faculty-intended objectives, but increasing this match by improving problem quality did not result in more self-study time. Therefore, they con-

798

A micro-analytical approach to studying learning processes in PBL Any attempt to study PBL as it occurs in its natural habitat – the PBL classroom – or to survey the underlying mechanisms of learning during PBL must seek new approaches to measurement. Conventionally, motivation and learning data are collected at the end of a course. These responses provide an ‘averaged’ impression of how motivated students were during the course or how much they learned. Although such a measurement approach is useful when the focus of interest is students’ overall motivation or learning, this type of measurement may hide important contextual variations during learning. The alternative approach involves carrying out detailed classroom observation: a learning event is videotaped and subsequently transcribed. Although this measurement approach tends to provide more detail on what is happening in a classroom, the

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning resulting qualitative dataset usually poses considerable technical and interpretational challenges. In the subsequent sections, we present a series of studies on motivation and learning in PBL using an approach we refer to as micro-analytical measurement.33,91 In this micro-analytical measurement approach, a short questionnaire or a short knowledge test is re-administered several times at critical points during the course of a learning event. These repeated measurements are subsequently related to one another and to final outcomes of learning. Administering a measure several times over a learning event may provide insights into what actually happens during learning. From this, inferences can be made about the mechanisms underlying PBL. Interest is a driving force in PBL but is consumed over the course of learning In a study by Rotgans and Schmidt,33 a microanalytical measurement approach was applied to investigate how students’ situational interest develops during PBL. Five short measures of situational interest were administered over the course of a 1-day PBL session. The first two measures were administered, respectively, immediately before and after the presentation of the problem to determine whether the problem would trigger students’ interest. The third measurement was conducted after the discussion of the problem and the generation of learning issues. The fourth administration took place after self-study to determine whether SDL activities had an effect on situational interest. The fifth and last measurement took place after the final discussion. The results of the study showed that once the problem had been presented, students’ situational interest increased significantly. However, as the learning event progressed, situational interest decreased: it seemed as if the initial increase in situational interest created by the problem was slowly consumed over the course of the learning event. The authors used the situational interest hypothesis to explain this phenomenon. The experience of being confronted with a problem that contains unknowns that need to be known triggers awareness of knowledge deprivation in the form of a knowledge gap that must be bridged by the finding of information about the unknowns. Knowledge acquired during self-study closes this gap. As situational interest is an indicator of the existence of such a gap, its decrease over time provides empirical support for this hypothesis. This study also showed that situational interest predicted students’ academic achievement with considerable accuracy, demonstrating that it drives learning.33

In a follow-up study, Rotgans et al. (unpublished data) applied the same micro-analytical measurement approach to determine students’ situational interest in both PBL and direct instruction (DI). In a quasiexperiment study involving several classes of primary school pupils, the authors ensured that all factors were kept constant, except for the instructional approach. For instance, situational interest measures were administered at the same time-points to both groups, the same teacher was present, the same learning resources were available and the same amount of time was allocated in both groups. The results of this study revealed that the PBL group was consistently more interested than the DI group on all but two measurement occasions (Rotgans et al., unpublished data). Fig. 1 shows an overview. The first of these differences in interest occurred at the start of the session, just before the PBL group read the problem and the DI group was presented with some examples to demonstrate the application of the topic to a real-life context. This was to be expected because this time-point gave a measure of students’ situational interest before the treatment took place. Students in both conditions showed ‘equally little interest in the topic’. However, when students in the PBL group were presented with the problem, their situational interest increased significantly. After the problem had been presented, the level of situational interest in the PBL group decreased significantly, as in the study by Rotgans and Schmidt.33 In the DI group, situational interest decreased significantly from the start when the group was presented with some examples (instead of the problem). This outcome demonstrates that the DI group did not experience the positive surge in situational interest generated by the problem and the corresponding feeling of knowledge deprivation. Their interest decreased immediately from the onset. However, the DI group showed one significant increase during self-study, which occurred when this group was given the opportunity to engage in independent group discussion to discuss some reallife applications of the topic. At this time-point there was no difference in situational interest between the PBL and DI groups (Rotgans et al., unpublished data). Overall, applying a micro-analytical approach to the study of student interest in PBL revealed the underlying situational interest processes, which would have been difficult to achieve with conventional survey measures or classroom observations. In the following study the micro-analytical measurement approach was applied to measure how students’

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

799

H G Schmidt et al DI group

4.00

Level of situational interest

Presentation of the problem

Identification of learning 3.80 goals

3.72

3.52 3.50 Students were informed about the topic

Elaboration 3.67 Presentation of findings 3.63 of what was learned 3.54

3.28

List of learning objectives and group activity

3.17 3.00

Self-study

PBL group

Presentation of examples

2.95

Lesson 1 (1 hour)

Assignment Elaboration of what was

2.86 Discussion

assignment

Lesson 2 (1.5 hours)

2.79 learned

2.82

Lesson 3 (1 hour)

2.50 SI1

SI2 SI3 SI4 SI5 Points at which situational interest was measured

SI6

Figure 1 Mean levels of situational interest (SI) over the learning event in the direct instruction (DI) and problem-based learning (PBL) groups and description of learning activities. (Based on Fig. 1 in Rotgans et al., unpublished data)

knowledge construction develops during the learning process. Cumulative process of learning in PBL

in pre- and post-tests, were analysed using structural equation modelling. Results demonstrated that only the hypothesised model had a good fit with the data obtained.92 Fig. 2 shows an overview.

The idea that learning is cumulative and that new learning depends on what has been learned previously is generally accepted by most educationalists. However, most of the evidence for this has come from experiments in psychology laboratories, particularly in the field of text comprehension. Yew and Schmidt92 argue that learning outcomes in PBL can only reflect collaborative processes because social constructivism suggests that knowledge is constructed through collaborative interactions and co-constructions.93,94 By contrast, it is also possible to infer that the individual SDL phase more significantly influences student learning in PBL as studies on selfregulated learning have demonstrated that the use of self-regulated learning strategies strongly impacts on students’ academic outcomes.95

This model showed that students’ prior knowledge significantly influenced learning in the first PBL phase (problem analysis), which then strongly influenced learning at the end of the SDL phase. Similarly, learning at the end of the SDL phase significantly impacted on learning at the end of the reporting phase. Finally, students’ learning achievements were significantly influenced by learning at the end of the reporting phase. By contrast, results from the alternative hypotheses tested indicated that collaborative small-group learning or self-directed study alone were insufficient to account for learning in PBL. These findings also demonstrate that the sequential nature of learning in PBL (from the problem analysis phase to SDL to the reporting phase) is essential in influencing student learning outcomes.92

Yew and Schmidt92 therefore tested the hypothesis that PBL is cumulative and that learning in PBL involves both collaborative and self-directed learning. They also tested three alternative hypotheses: (i) only the collaborative learning phases influence learning in PBL; (ii) only the self-directed study phase influences learning in PBL, and (iii) both collaborative learning and self-directed study influence learning in PBL, but not in a cumulative process. The units of analysis used as an indicator of student learning at the end of each PBL phase were the relevant concepts recalled by 218 students at the end of each learning phase. These concepts, together with students’ results

800

DISCUSSION

We have sketched a picture of PBL that emphasises its cognitive constructive nature. We have reviewed empirical evidence supporting six propositions. The first is that the initial discussion of a problem in a small group of students leads to the activation of prior knowledge. This prior knowledge is elaborated upon to collaboratively construct a tentative theory explaining the phenomena described in the problem. The cognitive constructions that result from this exercise in the minds of these students subsequently

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning 0.33

Prior knowledge

0.45

Concepts recalled 0.75 after problem analysis phase

Presentation of the problem

Concepts recalled 0.77 after self-directed learning phase

Selfdirected learning

Concepts recalled 0.28 after reporting phase

Achievement

Reporting of findings

Figure 2 Path model for the hypothetical model of relationships among different phases of problem-based learning. Numbers indicate standardised regression weights. (Based on Fig. 1 in Yew & Schmidt92; reproduced here with permission)

facilitate the comprehension of new information and its long-term survival.96 Next, problems drive learning by generating situational interest. We have demonstrated that the introduction of a problem in the learning situation leads to an increase in situational interest in the topic at hand, and that this situational interest is largely maintained over the course of learning (although to some extent it seems to be ‘satisfied’ by the new knowledge acquired and diminishes over time). A higher level of situational interest, in turn, relates to higher levels of achievement.33 Thirdly, as well as providing cognitive benefits, smallgroup tutorials contribute to students’ feelings of being ‘at home’ in their class, both socially and academically, and these protect against dropout. In addition, as these groups are focused on interaction in which students are expected to explain subject matter to one another, free riders are discouraged and students are encouraged to study regularly. This may be a reason why PBL curricula tend to have higher graduation rates.97 Fourthly, tutors’ subject matter knowledge, ability to relate to students and ability to be cognitively congruent with students all contribute to learning in PBL. Findings in this area suggest that good tutors provide flexible scaffolding by supporting student learning on a ‘just-in-time’ basis. Experiences with so-called ‘hard’ scaffolds, such as those generated when worksheets or additional questions are supplied, were more equivocal; the few studies conducted in this area suggest limited effects and one even showed hard scaffolds to be detrimental to learning.81,83 Fifthly, students in the PBL classroom have been shown to be more ardent users of library resources

than students in conventional programmes, which suggests that these students are more self-directed in their learning. However, students need time to develop this propensity. One study showed that students at the beginning of PBL do not study much beyond the learning issues generated in the small group.87 Real personal agency appears to require time to develop, but when it does, it has surplus value over learning driven solely by external stimuli. Finally, the extent of learning in PBL does not result from either group collaboration (the social constructivist point of view), or individual knowledge acquisition in isolation: both activities contribute equally to learning in PBL. In summary, PBL seems to have fairly strong effects on learning and achievement compared with conditions in which learning is not driven by the presentation of problems. The studies reviewed generally showed learning gains in PBL students that extended beyond those in students in control conditions in which problems were not the focus of attention or in which students were not encouraged to elaborate on their prior knowledge. These findings seem to be at variance with the findings of curriculum comparison studies that generally do not report PBL to have effects superior to those of forms of conventional training.2 It seems that although effects are found at the micro level, these do not translate into visible effects at the curriculum level. This then raises the question of why this is so. What might explain this apparent paradox? We can offer only some tentative hypotheses. The first is a ‘compensation’ hypothesis. There is some evidence that students who study under less favourable circumstances tend to compensate by working harder.98,99 This would imply that although students profit more from PBL, they compensate for this

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

801

H G Schmidt et al additional support by working less hard. This is unlikely because students in PBL curricula tend to spend more time on individual learning than those in more conventional environments. A second hypothesis, more parsimonious with the micro-level finding reviewed in this article, has been proposed recently. This hypothesis assumes that differential dropout and study duration mask the effects of PBL at the curriculum level. It takes as its starting point the observation that PBL curricula tend to show less dropout and shorter study duration than conventional schools.52,97 Comparisons between PBL and conventional schools may therefore be biased against PBL; effects of PBL become hidden by differential attrition and differential study duration. To test this hypothesis, Schmidt and colleagues (unpublished data) re-analysed 104 curriculum comparisons among schools for which dropout rates and study duration information were available. By correcting for differences on these variables, they were able to demonstrate robust effects of PBL on both knowledge attainment and diagnostic performance. It seems that micro-level effects of PBL do replicate at the curriculum level. Given that well over 5000 studies referring to PBL are represented in PubMed, what interesting avenues remain for further research? We will confine ourselves to two suggestions here. The tutorial group, vital to the inner workings of PBL, remains in many respects a black box. For instance, differences in epistemological beliefs among students have been shown to influence achievement (i.e. students who believe that learning is knowledge construction perform better than other students100). Why is this so? Are groups that are heterogeneous in terms of prior knowledge more successful than homogeneous groups, as some studies seem to suggest?101 Despite the importance in the small group of elaboration on what one already knows, students tend to avoid elaboration on the incorrect assumption that everybody in the group already knows what the individual knows. Would it help if tutors explicitly prompted for further elaboration? Tutors were shown to be more effective when they were able to explain things in a simple way. What does this imply? Do cognitively congruent tutors explain more items in the tutorial group? What would happen if tutors were instructed to explicitly provide scaffolds whenever their group reached a dead end in their discussion? The second suggestion refers to long-term memory for what has been learned. Some literature implies that

802

PBL has a particular effect on the long-term memorability of learned material, even if initially no effects are found.22–24 If this is true, there may be several reasons for it. For instance, as the problem that originally drove the learning will be stored in the memory along with the material learned, it is possible that such a problem acts as a cue for the retrieval of the knowledge acquired and thus increases its accessibility at a later stage.102 A related explanation may be that students in PBL organise knowledge in memory differently in a manner that facilitates its later retrieval. Studies of possible differences in knowledge organisation that affect the use of that knowledge in new contexts – the transfer issue – are virtually non-existent in education research,103 but are sorely needed. In the Introduction to this paper, we suggested that different authors tend to define the ultimate goals of PBL in different ways. This paper took as its point of departure cognitive psychology and well-known principles of constructivist learning, with an emphasis on how people acquire knowledge. However, a point of view particularly prevalent in many medical schools tends to define PBL as a process of inquiry.104,105 From this perspective, the ultimate objective of PBL is to help students develop the ‘inquiry’ or ‘problem-solving’ skills of an expert by imitating his or her thinking processes. Working on a problem is seen as a simulation of what the doctor does, particularly in its emphasis on data gathering and interpretation. In this view of PBL, the role of knowledge acquisition is somewhat vague. Although most of its proponents acknowledge that knowledge is needed to fill gaps in the student’s knowledge base, from this perspective PBL is clearly focused on the acquisition of problemsolving skills. This perspective is attractive to medical educators because professionals in the field seem to think that their expertise is indeed partly based on the possession of such elusive skills. However, thirty years of research in this domain have made it clear that it is unlikely that problem-solving skills can be learned through education, or even that such things as problem-solving skills, independent of subject matter knowledge, exist.106 If this is true, it indicates that there are no shortcuts to acquiring expertise in medicine; that there are no domain-independent problemsolving skills the acquisition of which could compensate to some extent for lack of knowledge acquisition. The cognitive constructivist perspective on PBL incorporates this view that clinical reasoning is knowledgebased. It sees learning in the field of medicine as a series of attempts to construct mental models of the underlying mechanisms of disease in terms of how it is produced, the conditions under which it appears, and

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning its consequences in terms of observable signs and symptoms in patients. These mental models of disease, or ‘illness scripts’, are the cognitive tools that doctors use when they engage in clinical reasoning.107 The task of medical education is to help students develop these cognitive structures. Problem-based learning may be a useful way of doing so.

Contributors: HS initiated and coordinated the writing of this review. He provided the conceptual outline and final plan of the manuscript. JR contributed sections on situational interest research and the micro-analytical approach. He also reviewed the role of problems in problem-based learning (PBL). EY reviewed studies on the role of the tutor, scaffolds used in PBL and research on the cumulative nature of learning. All authors contributed to the critical revision of the paper and approved the final manuscript for submission. Acknowledgements: none. Funding: none. Conflicts of interest: none. Ethical approval: not applicable.

REFERENCES 1 Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med 1993;68 (1):52–81. 2 Colliver JA. Effectiveness of problem-based learning curricula: research and theory. Acad Med 2000;75 (3):259–66. 3 Dochy F, Segers M, van den Bossche P, Gijbels D. Effects of problem-based learning: a meta-analysis. Learn Instruct 2003;13 (5):533–68. 4 Gijbels D, Dochy F, van den Bossche P, Segers M. Effects of problem-based learning: a meta-analysis from the angle of assessment. Rev Educ Res 2005;75 (1):27–61. 5 Schmidt HG, Dauphinee WD, Patel VL. Comparing the effects of problem-based and conventional curricula in an international sample. J Med Educ 1987;62 (4):305–15. 6 Vernon DTA, Blake RL. Does problem-based learning work? - A meta-analysis of evaluative research. Acad Med 1993;68 (7):550–63. 7 Schmidt HG, van der Molen HT, Te Winkel WWR, Wijnen WHFW. Constructivist, problem-based learning does work: a meta-analysis of curricular comparisons involving a single medical school. Educ Psychol 2009;44 (4):227–49. 8 Barrows HS. How to Design a Problem-Based Curriculum for the Preclinical Years. New York, NY: Springer Publishing 1985. 9 Evensen DH, Hmelo CE, eds. Problem-Based Learning: A Research Perspective on Learning Interactions. Mahwah, NJ: Lawrence Erlbaum 2000.

10 Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev 2004;16 (3):235–66. 11 Schmidt HG. Problem-based learning: rationale and description. Med Educ 1983;17 (1):11–6. 12 Schmidt HG. Foundations of problem-based learning – some explanatory notes. Med Educ 1993;27 (5):422–32. 13 Pressley M, Wood E, Woloshyn VE, Martin V, King A, Menke D. Encouraging mindful use of prior knowledge – attempting to construct explanatory answers facilitates learning. Educ Psychol 1992;27 (1):91–109. 14 Reder LM. The role of elaboration in the comprehension and retention of prose: a critical review. Rev Educ Res 1980;50 (1):5–53. 15 Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79 (10 Suppl):70– 81. 16 Chi MTH, Siler SA, Jeong H, Yamauchi T, Hausmann RG. Learning from human tutoring. Cogn Sci 2001;25 (4):471–533. 17 Moust JHC, Schmidt HG, De Volder ML, Belie¨n JJJ, De Grave WS. Effects of verbal participation in smallgroup discussion on learning. In: Richardson JTE, Eysenck ME, Piper DW, eds. Student Learning: Research in Education and Cognitive Psychology. Milton Keynes: Open University Press 1987;212–18. 18 Schmidt HG. Activatie van voorkennis en tekstverwerking [Activation of prior knowledge and text processing]. Ned Tijdschr Psychol 1984;39:335–47. 19 Schmidt HG, De Grave WS, De Volder ML, Moust JHC, Patel VL. Explanatory Models in the processing of science text: the role of prior knowledge activation through small-group discussion. J Educ Psychol 1989;81:610–9. 20 De Grave WS, Schmidt HG, Belie¨n JJ, Moust JHC, De Volder ML, Kerkhofs LMM. Effecten van verschillende typen van activatie van voorkennis op recall, gemeten met een aanvultoets [Effects of different types of activation of prior knowledge on recall, measured with a completion procedure]. Tilburg: Onderwijs Research Dagen 1985. 21 van Blankenstein FM, Dolmans DHJM, van der Vleuten CPM, Schmidt HG. Which cognitive processes support learning during small-group discussion? The role of providing explanations and listening to others. Instr Sci 2009;32 (2):1–16. 22 Eisenstaedt RS, Barry WE, Glanz K. Problem-based learning: cognitive retention and cohort traits of randomly selected participants and decliners. Acad Med 1990;65 (9 Suppl):11–2. 23 Ma˚rtenson D, Eriksson H, Ingelman-Sundberg M. Medical chemistry: evaluation of active and problemoriented teaching methods. Med Educ 1985;19: 34–42. 24 Tans RW, Schmidt HG, Schade-Hoogeveen BEJ, Gijselaers WH. Sturing van het onderwijsleerproces door middel van problemen: een veldexperiment [Guiding the learning process by means of

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

803

H G Schmidt et al

25

26

27 28

29

30

31

32

33

34

35

36

37

38

39

40

41

804

problems: a field experiment]. Tijdschr Onderwijsres 1986;11 (1):38–48. Hidi S, Renninger KA. The four-phase model of interest development. Educ Psychol 2006;41 (2): 111–27. Loewenstein G. The psychology of curiosity: a review and reinterpretation. Psychol Bull 1994;116 (1): 75–98. Berlyne DE. Curiosity and learning. Motiv Emotion 1978;2 (2):97–175. Litman JA. Interest and deprivation factors of epistemic curiosity. Pers Indiv Differ 2008;44 (7):1585– 95. Litman JA. Curiosity and the pleasures of learning: wanting and liking new information. Cognition Emotion 2005;19 (6):793–814. Rotgans JI, Schmidt HG. The role of teachers in facilitating situational interest in an active-learning classroom. Teach Educ 2011;27 (1):37–42. De Volder ML, Schmidt HG, Moust JHC, De Grave WS. Problem-based learning and intrinsic motivation. In: van der Berchen JHC, Bergen TCM, De Bruyn EEI, eds. Achievement and Task Motivation. Berwyn, IL: Swets North America 1986;123–7. Schmidt HG. Intrinsieke motivatie en studieprestatie: enkele verkennende onderzoekingen [Intrinsic motivation and achievement: some investigations]. Pedagog Stud 1983;60:385–95. Rotgans JI, Schmidt HG. Situational interest and academic achievement in the active-learning classroom. Learn Instr 2011;21 (1):58–67. Des Marchais JE. A Delphi technique to identify and evaluate criteria for construction of PBL problems. Med Educ 1999;33 (7):504–8. Jacobs AEJP, Dolmans DHJM, Wolfhagen IHAP, Scherpbier AJJA. Validation of a short questionnaire to assess the degree of complexity and structuredness of PBL problems. Med Educ 2003;37 (11):1001–7. Kim S, Phillips WR, Pinsky L, Brock D, Phillips K, Keary J. A conceptual framework for developing teaching cases: a review and synthesis of the literature across disciplines. Med Educ 2006;40 (9):867–76. Sockalingam N, Rotgans JI, Schmidt HG. Student and tutor perceptions on attributes of effective problems in problem-based learning. Higher Educ 2010; 62 (1): 1–16. Dolmans DHJM, Gijselaers WH, Schmidt HG, van der Meer SB. Problem effectiveness in a course using problem-based learning. Acad Med 1993;68 (3): 207–13. Mpofu DJS, Das M, Murdoch JC, Lanphear JH. Effectiveness of problems used in problem-based learning. Med Educ 1997;31 (5):330–4. Soppe M, Schmidt HG, Bruysten RJMP. Influence of problem familiarity on learning in a problem-based course. Instr Sci 2005;33 (3):271–81. Hung W. The 3C3R model: a conceptual framework for designing problems in PBL. Interdisciplinary J Prob Based Learn 2006;1 (1):55–77.

42 Gijselaers WH, Schmidt HG. Development and evaluation of a causal model of problem-based learning. In: Nooman ZH, Schmidt HG, Ezzat ES, eds. Innovation in Medical Education: An Evaluation of its Present Status. New York, NY: Springer Publishing 1990;95–113. 43 van Berkel H, Schmidt H. On the additional value of lectures in a problem-based curriculum. Educ Health (Abingdon) 2005;18 (1):45–61. 44 van Berkel HJM, Schmidt HG. Motivation to commit oneself as a determinant of achievement in problem-based learning. Higher Educ 2000;40 (2):231– 42. 45 van den Hurk MM, Dolmans D, Wolfhagen I, van der Vleuten CPM. Testing a causal model for learning in a problem-based curriculum. Adv Health Sci Educ 2001;6 (2):141–9. 46 Slavin R. Cooperative Learning: Theory, Research and Practice. Boston, MA: Allyn & Bacon 1990. 47 Springer L, Stanne ME, Donovan SS. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a metaanalysis. Rev Educ Res 1999;69 (1):21–51. 48 Severiens SE, Schmidt HG. Academic and social integration and study progress in problem-based learning. Higher Educ 2009;58 (1):59–69. 49 Tinto V. Classrooms as communities – exploring the educational character of student persistence. J High Educ 1997;68 (6):599–623. 50 Burch VC, Sikakana CNT, Seggie JL, Schmidt HG. Performance of academically at-risk medical students in a problem-based learning programme. A preliminary report. Adv Health Sci Educ 2007;12 (3):345–58. 51 Howell H. Ten years of hybrid PBL at Harvard School of Dental Medicine. Fourth International Symposium on Problem-Based Learning in Dental Education, 23– 28 October 2005, Nakorn Pathom. 52 Iputo JE, Kwizera E. Problem-based learning improves the academic performance of medical students in South Africa. Med Educ 2005;39 (4):388–93. 53 Barrows HS, Tamblyn R. Problem-Based Learning: An Approach to Medical Education. New York, NY: Springer Publishing 1980. 54 Maudsley G. Roles and responsibilities of the problembased learning tutor in the undergraduate medical curriculum. BMJ 1999;318:657–61. 55 Mayo WP, Donnelly MB, Schwartz RW. Characteristics of the ideal problem-based learning tutor in clinical medicine. Eval Health Prof 1995;18:124–36. 56 Wetzel MS. Developing the role of the tutor ⁄ facilitator. Postgrad Med J 1996;72:474–7. 57 De Grave WS, Dolmans DHJM, van der Vleuten CPM. Profiles of effective tutors in problem-based learning: scaffolding student learning. Med Educ 1999;33 (12):901–6. 58 Collins A, Brown JS, Newman SE, eds. Cognitive Apprenticeship: Teaching the Crafts of Reading, Writing, and Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates 1989.

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

What works and why in problem-based learning 59 Davis WK, Nairn R, Paine ME, Anderson RM, Oh MS. Effects of expert and non-expert facilitators on the small-group process and on student performance. Acad Med 1992;67 (7):470–4. 60 Des Marchais JE, Black R. Effect of Tutor Content Expertise on Student Academic Achievement in the Sherbrooke Problem-Based Curriculum. Sherbrooke, QC: University of Sherbrooke 1991. 61 Swanson DB, Stalenhoef-Halling BF, van der Vleuten CPM. Effect of tutor characteristics on test performance of students in a problem-based curriculum. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, eds. Teaching and Assessing Clinical Competence. Groningen: BoekWerk Publications 1990;129–34. 62 Schmidt HG. Resolving inconsistencies in tutor expertise research: does lack of structure cause students to seek tutor guidance? Acad med 1994;69:656–62. 63 Silver M, Wilkerson LA. Effects of tutors with subject expertise on the problem-based tutorial process. Acad Med 1991;66 (5):298–300. 64 Eagle CJ, Harasym PH, Mandin H. Effects of tutors with case expertise on problem-based learning issues. Acad Med 1992;67 (7):465–9. 65 Moust JHC. De Rol van Tutoren in Probleemgestuurd Onderwijs. Contrasten Tussen Student- en Docenttutoren [On the Role of Tutors in Problem-Based Learning: Contrasting Student-Guided with Staff-Guided Tutorials]. Maastricht: University of Limburg 1993. 66 Schmidt HG, Moust JHC. What makes a tutor effective - a structural-equations modeling approach to learning in problem-based curricula. Acad Med 1995;70 (8):708– 14. 67 Chng E, Yew EHJ, Schmidt HG. Effects of tutorrelated behaviours on the process of problem-based learning. Adv Health Sci Educ 2011 (In press). 68 Gukas ID, Leinster SJ, Walker R. Verbal and nonverbal indices of learning during problem-based learning (PBL) among first year medical students and the threshold for tutor intervention. Med Teach 2010;32 (1):5–11. 69 Kindler P, Grant C, Kulla S, Poole G, Godolphin W. Difficult incidents and tutor interventions in problembased learning tutorials. Med Educ 2009;43:866–73. 70 Hmelo-Silver CE, Barrows HS. Goals and strategies of a problem-based learning facilitator. Int J Probl Based Learn 2006;1 (1):21–39. 71 Kirschner PA, Sweller J, Clark RE. Why minimal guidance during instruction does not work: an analysis of the failure of constructivist, discovery, problembased, experiential, and inquiry-based teaching. Educ Psychol 2006;41 (2):75–86. 72 Belland B, Glazewski K, Richardson J. A scaffolding framework to support the construction of evidencebased arguments among middle school students. Educ Tech Res Dev 2008;56 (4):401–422. 73 Schmidt HG, Loyens SMM, Van Gog T, Paas F. Problembased learning is compatible with human cognitive architecture: commentary on Kirschner, Sweller, and Clark (2006). Educ Psychol 2007;42 (2):91–7.

74 Wood D, Bruner JS, Ross G. The role of tutoring in problem solving. J Child Psychol Psychiatry 1976;17:89– 100. 75 Saye JW, Brush T. Scaffolding critical reasoning about history and social issues in multimedia-supported learning environments. Educ Tech Res Dev 2002;50 (3):77–96. 76 Berk LE, Winsler A. Scaffolding Children’s Learning: Vygotsky and Early Childhood Education. Washington, DC: National Association for the Education of Young Children 1995. 77 Roehler LR, Cantlon DJ, eds. Scaffolding: A Powerful Tool in Social Constructivist Classrooms. Cambridge, MA: Brookline 1997. 78 Merrie¨nboer JJGV. Training Complex Cognitive Skills: A Four-Component Instructional Design Model for Technical Training. Englewood Cliffs, NJ: Educational Technology 1997. 79 van de Pol J, Volman M, Beishuizen J. Scaffolding in teacher–student interaction: a decade of research. Educ Psychol Rev 2010;22 (3):271–96. 80 Simons K, Klein J. The impact of scaffolding and student achievement levels in a problem-based learning environment. Instr Sci 2007;35:41–72. 81 Choo SSY, Rotgans JI, Yew EHJ, Schmidt HG. Effect of worksheet scaffolds on student learning in problembased learning. Adv Health Sci Educ 2011 (In Press). 82 Schmidt HG, Bouhuijs PAJ. Effecten van Structurering van Patie¨ntenproblemen op Leerresultaat en Satisfactie van Studenten [Effects of Structuring of Problems on Achievement and Satisfaction of Students]. Amsterdam: Vrije Universiteit 1977. 83 Verkoeijen PPJL, Rikers RMJP, Winkel WWT, van den Hurk MM. Do student-defined learning issues increase quality and quantity of individual study? Adv Health Sci Educ 2006;11 (4):337–47. 84 Gallagher SA, Stepien WJ, Rosenthal H. The effects of problem-based learning on problem solving. Gifted Child Q 1992;36:195–200. 85 Kolodner JL, Camp PJ, Crismond D, Fasse B, Gray J, Holbrook J, Puntambekar S, Ryan M. Problem-based learning meets case-based reasoning in the middleschool science classroom: putting Learning by Design into practice. J Learn Sci 2003;12 (4):495– 547. 86 Dolmans DHJM, Schmidt HG, Gijselaers WH. The relationship between student-generated learning issues and self-study in problem-based learning. Instr Sci 1995;22 (4):251–67. 87 van den Hurk MM, Wolfhagen IHAP, Dolmans D, van der Vleuten CPM. The impact of student-generated learning issues on individual study time and academic achievement. Med Educ 1999;33 (11):808–14. 88 Blumberg P, Michael JA. Development of self-directed learning behaviours in a partially teacher-directed problem-based learning curriculum. Teach Learn Med 1992;4:3–8. 89 Marshall JG, Fitzgerald D, Busby L, Heaton G. A study of library use in problem-based and traditional

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806

805

H G Schmidt et al

90 91

92

93

94

95

96

97

98

806

medical curricula. Bull Med Libr Assoc 1993;81 (3):299–305. Rankin JA. Problem-based medical education – effect on library use. Bull Med Libr Assoc 1992;80 (1):36–43. Zimmerman BJ. Investigating self-regulation and motivation: historical background, methodological developments, and future prospects. Am Educ Res J 2008;45 (1):166–83. Yew EHJ, Schmidt HG. Is learning in problem-based learning cumulative? Adv Health Sci Educ 2010; doi: 10.1007/s10459-010-9267-y. Cobb P. Where is the mind? Constructivist and sociocultural perspectives on mathematical development. Educ Res 1994;23:13–20. Driver R, Asoko H, Leach J, Mortimer E, Scott P. Constructing scientific knowledge in the classroom. Educ Res 1994;23:5–12. Zimmerman BJ. Self-regulated learning and academic achievement – an overview. Educ Psychol 1990;25 (1):3– 17. De Grave WS, Schmidt HG, Boshuizen HPA. Effects of problem-based discussion on studying a subsequent text: a randomised trial among first year medical students. Instr Sci 2001;29:33–44. Schmidt HG, Cohen-Schotanus J, Arends L. Impact of problem-based, active, learning on graduation rates of ten generations of Dutch medical students. Med Educ 2009;43 (3):211–8. Dubin R, Taveggia TC. The Teaching–Learning Paradox. A Comparative Analysis of College Teaching Methods. Eugene, OR: Center for the Advanced Study of Educational Administration 1968.

99 Johnson W, Kieras D. Representation saving – effects of prior knowledge in memory for simple technical prose. Mem Cognition 1983;11:456–66. 100 Loyens SMM, Rikers RMJP, Schmidt HG. The impact of students’ conceptions of constructivist assumptions on academic achievement and dropout. Stud High Educ 2007;32:581–602. 101 Gutierrez R, Slavin RE. Achievement effects of the non-graded elementary school – a best-evidence synthesis. Rev Educ Res 1992;62 (4):333–76. 102 Tulving E, Thomson DM. Encoding specificity and retrieval processes in episodic memory. Psychol Rev 1973;80 (5):352–73. 103 Eva KW, Neville AJ, Norman GR. Exploring the etiologic of content specificity: factors influencing analogic transfer and problem solving. Acad Med 1998;73 (10 Suppl):1–5. 104 Barrows HS. Inquiry: the pedagogical importance of a skill central to clinical practice. Med Educ 1990;24:3–5. 105 Hmelo CE, Ferrari M. The problem-based learning tutorial: cultivating higher order thinking skills. J Educ Gifted 1997;20 (4):401–22. 106 Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005;39 (4):418–27. 107 Schmidt HG, Rikers R. How expertise develops in medicine: knowledge encapsulation and illness script formation. Med Educ 2007;41 (12):1133–9. Received 7 March 2011; editorial comments to authors 11 March 2011; accepted for publication 5 April 2011

ª Blackwell Publishing Ltd 2011. MEDICAL EDUCATION 2011; 45: 792–806