Designing and Assessing Significant eLearning Experiences in the Classroom Bette Poutre, MS Interim Director eLearning and Academic Technologies School of Pharmacy and Health Professions Office of eLearning and Academic Technologies Creighton University, Omaha, Nebraska http://www.creighton.edu

Objectives 1.  Examine best practices for designing and assessing online learning experiences. 2.  Examine strategies to engage learners in an online learning community. 3.  Describe assessment processes that can be used to measure the outcomes of online learning.

Creighton University •  A Catholic, Jesuit University founded in 1878; one of 28 Jesuit colleges and universities •  Urban campus in Omaha, Nebraska •  Comprehensive academic programs in Arts & Sciences, Law, Business, Dentistry, Nursing, Medicine, Pharmacy & Health Professions, Graduate & University College •  Student body of more than 7,700 students from all 50 states and 41 countries

School of Pharmacy & Health Professions (SPAHP) Degrees Offered Doctor of Pharmacy – PharmD –  Entry-level (campus 1976 and distance 1994) Doctor of Occupational Therapy – OTD –  Entry level (campus- 1999 and distance 2008) –  Post professional (distance 1995) Doctor of Physical Therapy – DPT –  Entry level (campus - 1993)

Student Enrollment by Profession Program

Campus

Distance

Total

Pharmacy

455

284

739

Occupational Therapy

151

43

199

44

44

Post-professional Occupational Therapy Physical Therapy Total enrollment

179

179 1161

Each student in the School receives a tablet computer, training and maintenance as part of tuition.

Let’s all get on the same terminology page…

eLearning Definitions •  Synchronous Learning –  Learners access learning environment at the same time supported by media (video conferencing/chat) to support learners with the development of learning communities

•  Asynchronous Learning –  Learners access learning environment any time

•  Blended Learning –  Combining face-to-face (F2F) and non-F2F online activities •  Some performed synchronously and some asynchronously •  Some seat time is replaced with online activities

–  Other terms: hybrid, mixed-mode, web-enhanced learning

Diaz, V. & Brown, M. (2010). Blended learning: A report on the ELI focus session (ID: ELI3023), Educause Learning Initiative (ELI).

Online Programming Definitions •  Online Courses – those in which all or the vast majority (75% or more) of the instruction and interaction occurs via electronic communication or equivalent mechanisms, with the faculty and students physically separated from each other •  Online Programs –those in which a student could earn the credential by taking 50% or more of the work in the distance education format North Central Association of Colleges and Schools

Variations of Blended Learning Program

Diaz, V & Brown, M. (2010). Blended learning: A report on the ELI focus session (ID: ELI3023), Educause Learning Initiative (ELI).

Significant learning experiences…

Framework Based on L. Dee Fink’s, Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses

Fink, L. D. (2003). Creating significant learning experiences. San Francisco, CA: Jossey-Bass. A Self-Directed Guide to Designing Courses for Significant Learning http://www.deefinkandassociates.com/GuidetoCourseDesignAug05.pdf

Good courses are those that… •  Challenge students to significant learning •  Use active forms of learning •  Have teachers who care about the subject student and their own teaching and learning •  Have teachers who interact well with students •  Have a good system of feedback, assessment and grading

Foundational Knowledge •  Knowing is the ability to understand and remember specific information and ideas •  Major ideas & perspectives •  Foundational knowledge provides the basic understanding that is necessary for other kinds of learning •  What key ideas or perspectives are important for students to understand in this course?

Application •  Learning how to engage in various kinds of thinking (critical, creative, practical) is a form a application •  Includes developing certain skills or learning how to manage complex projects •  Application learning allows other kinds of learning to become useful •  What important skills do students need to gain?

Integration •  Connections between different things: –  Ideas, People, Realms of life •  What connections should be made among: –  Ideas within the course? –  Information, ideas and perspective in this course and those in other courses or areas? –  Material in this course and the students’ own personal, social, &/or work life? •  The act of making new connections gives learners a new form of power, especially intellectual power

Human Dimension •  Learning about: –  Oneself –  Others

•  What could/should students learn about themselves? •  What could/should student learn about understanding others &/or interacting with them?

Caring •  Developing new: –  Feelings, Interests, Values

•  When students care about something, they have the energy they need for learning more and making it part of their lives •  Without the energy for learning nothing happens •  What changes or values do you hope students will adopt? –  Feelings, Interests, Ideas?

Learning to Learn •  Becoming a better student •  Inquiring about a subject •  Self-directing learners •  How would you like students to learn about: –  How to be good students in a course like this? –  How to learn about this particular subject? –  How to become a self-directed learner of this subject?

Interactive Nature •  Taxonomy is relational, not hierarchical •  Achieving one type of learning simultaneously enhances the possibility of achieving other kinds of learning •  Finding a way to help students achieve one kind of learning enhances student achievement in other kinds of learning •  Example: using the information and concepts in a course to effectively solve problems (application) makes it easier for students to get excited about the value of the subject (caring) •  Example: Students learn to effectively relate the subject to other ideas and subjects (integration) which makes it easier for them to see the significance of the course material for themselves and others (human dimension)

Course Goals Formulation for Significant Learning Example By the end of this course, the student will: •  Understand and remember key concepts, terms, relationships, and so on… (Foundation) •  Know how to use the content. (Application) •  Relate the subject to other subjects. (Integration) •  Understand the personal and social implications of knowing about this subject. (Human Dimension) •  Care about the subject and learning about learning more on the subject. (Caring) •  Know how to keep on learning about this subject after the course is over. (Learning How to Learn)

Example…World Geography Foundational Knowledge •  Have a mental map of the world and correctly locate major places – countries, oceans, cities, etc. Application •  Be able to find information on and analyze regional problems from a geographical perspective Integration •  Identify the interactions between geography and other realms of knowledge such as history, politics, economics, etc.

Example…World Geography Human Dimension •  Be able to intelligently discuss world events with other people and the impact of geography on these events. Caring •  Be interested in other places/want to continue to learn more via reading, Internet, etc. Learning How to Learn •  Be able to interpret the geographic significance of new information and ideas

Technological Tools that Match Each Dimension…

Pedagogy MUST Drive Technology •  Examine course / lesson objectives •  Then determine how technology can assist in student accomplishing the objective

Foundational Information Developing a full understanding of the concepts associated with a subject •  Web research •  Videos •  Ted Talks www.ted.com •  Offline/online readings

Application Critical thinking, practical thinking, creativity, managing complex projects, performance •  Video recordings or performances •  Presentations •  Simulations or role playing •  Case studies •  Problem solving

Types of Thinking 1.  Critical Thinking: analyzing and evaluating 2.  Practical Thinking: learning how to answer questions, making decisions, and solving problems 3.  Creative Thinking: thinking outside the box; creating new ideas, new ways of doing things

Integration Interdisciplinary learning, learning communities •  Mind Mapping –  Helps students move beyond text-only readings to allow them to interpret concepts, facts, or ideas in a graphicmanner www.mindmeister.com

•  Wordles –  Wordle is a toy for generating “word clouds” from text that you provide. The clouds give greater prominence to words that appear more frequently in the source text. Clouds can be tweaked with different fonts, layouts, and color schemes. The images can be printed, or save them to the Wordle gallery to share with friends. –  http://www.wordle.net/

Mind Mapping Example

http://www.bing.com/images/search?q=Concept +Map&Form=R5FD1#view=detail&id=9A2177B54BD0A43D6A55272EA800583146FD4D37&selectedIndex=1

Wordle Example

https://www.google.com/search? hl=en&site=imghp&tbm=isch&source=hp&biw=1391&bih=889&q=wordle&oq=wordle&gs_l=img. 3..0l10.2285.3676.0.4251.6.6.0.0.0.0.83.421.6.6.0.chm_pq_qw%2Chmss2%3Dfalse...0...1.1.36.img..0.6.420.ST16SypWkBE

Human Dimension Leadership, character-building, contributing to a team, serving others •  Team projects •  Blogs •  Wikis •  Microblogs / Social Media •  Discussion boards

Caring Wanting to be a good student, excitement about a subject/topic, commitment to healthy living •  Discussion boards •  GPA and/or honors •  Volunteering/helping/mentoring

Learning How to Learn Learning how to be a better student through self-regulation or deep learning, self-directed and intention learning, reflective •  Self assessments •  Rubrics •  Reflections •  Development plans

Creating an effective online community…

Seven Principles of Effective Teaching (Chickering & Gamson, 1987 and Graham, et al. 2001)

Good Practice Encourages: 1.  Student-Faculty Contact 2.  Cooperation among Students 3.  Active Learning 4.  Prompt Feedback 5.  Emphasizing Time on Task 6.  Communication of High Expectations 7.  Respecting Diverse Talents and Ways of Learning

1. Encourages Student-Faculty Contact Provide clear guidelines for interaction with students •  Hold online office hours: online chat hours •  Most important for faculty to increase student motivation and involvement •  Application to online teaching: –  –  –  –  – 

Discussion boards Web conferencing Skype Email Establish boundaries: timelines for responding, etc.

2. Encourages Cooperation among Students Establish well-designed discussion assignments to facilitate meaningful cooperation among students •  Working with others = sharing ideas and improving thinking •  Group projects •  Well designed discussion boards –  Small groups expectations –  Focused on a task –  Required participation

- Post discussion - Feedback on discussion - Student lead

3. Good Practice Encourages Active Learning Students should be present in course projects •  Learning is not a spectator sport; active vs. passive •  Students must talk about what they’re learning and relate it to post experiences and apply it to daily lives –  Learning by doing –  Gaming –  Community engagement from a distance –  Reflection postings on a discussion forum

4. Good Practice Gives Prompt Feedback Instructor needs to provide prompt feedback •  Two types: –  Information: feedback on content of an assignment; grade; answer to a question –  Acknowledgement: confirmation of an act that’s occurred

•  •  •  • 

Track changes feature in Microsoft Word Inserting comments using the hidden text option Quick quizzes/self assessment tools Portfolio evaluation: excellent options for timeconsuming & costly to record (e.g., leadership skills, multi-cultural interactions, etc.)

5. Good Practice Emphasizes Time on Task Online courses need deadlines •  Time + energy = learning •  Allocate realistic time frames to allow for effective learning to occur •  Flexibility due to student’s life context •  Intermittent deadlines = avoid procrastination •  Learning efficiency: web tools, electronic databases, ereserve readings, etc. •  Use calendar tool as a way to keep students informed •  Timed tests/quizzes emphasize time on task

6. Communicates High Expectations Challenging tasks, sample cases, praise for quality work all communicate high expectations •  Challenging tasks, sample cases, and praise for quality work communicate high expectations. •  Provide students with models to follow, along with comments explaining why the examples are good. •  Publicly praising exemplary work communicates high expectations.

7: Respects Diverse Talents and Ways of Learning Allow students to choose project topics to incorporate diverse views •  Adhere to a template for the lessons, but vary assignments •  Create assignments that require the student to become more openminded; self selected topics

•  Create diverse groups to work on group projects •  Create ways for students to demonstrate their talents (i.e., use a variety of assessment strategies) •  Push students to learn in new ways that do not come so easily to them

How do I actively engage online students? Community of Inquiry Theory

Garrison, D. et.al. (2000). Critical inquiry in a text-based environment: computer conferencing in higher education. The Internet and Higher Education 2(2-3): 87-105.

Read more: Community of Inquiry web site: http://communitiesofinquiry.com includes scholarly articles, explanations and examples for the three components of the community of inquiry model; teaching presence, social presence, and cognitive presence.

Creating an Online Community (Garrison, 2000)

•  Cognitive Presence –  Relates to thinking and involves the ability of learners to start, create, and validate meaning through reflection and dialogue in the online environment

•  Social Presence –  Involves personal and emotional connection to the group where individuals are able to express themselves socially and emotionally in a genuine manner

•  Instructor Presence –  Voice of the facilitator serving as a model for the discussion, providing constructive and formative feedback

A successful educational experience involves the balance and interaction of all three

Designing Significant Learning Experiences…

Content-Centered vs. Learning-Centered Paradigms Content-centered –  Describing topics or content determined by text books and additional material

Example Topics: A, B, C, D …….

Learning-centered –  Teaching is driven to incorporate new kinds of learning rather that new content

Example Foundational knowledge to learning how to learn

Paradigm Comparisons Learning How to Learn The learning-centered paradigm pushes teaching and learning in this direction, into multiple dimensions on learning.

Caring Human Dimension Integration Application Foundational Knowledge: Topics A, B, C, D

The content-centered paradigm pushes teaching and learning in this direction, along one dimension of learning. Fink, L. D. (2003). Creating significant learning experiences. San Francisco, CA: Jossey-Bass.

Course Development Process Before the course begins •  Instructor knowledge of subject matter and design of their instruction After the course begins •  Teacher/student interactions and course management Improve teaching by improving competence in one or more of the domains Fink, L. D. (2003). Creating significant learning experiences. San Francisco, CA: Jossey-Bass.

Course Development Process Teacher-student interactions •  Methods of interacting with students: lecturing, discussions, email, etc. •  Improving interactions would be a major advance Course management •  Having course ready for the events that take place in the course (discussions, submissions, assessments, etc.) Design of instruction •  Knowledge usually obtained via a faculty development session but not key for most faculty •  Most follow traditional method of teaching and lack conceptual tools to rethink the teaching and learning activities

How to design quality into my course? Content-centered approach –  Look at subject > Create list of 8-12 topics > Create lectures > Add a few assignments, exams and a final > DONE

Learning-centered approach –  Backward design – imagine when the course is over and perhaps several years past •  Ask yourself, “What is it I hope that students will have learned and still has value several years after the course is over?”

Integrated not Linear Course Design Step 1: What is it I hope that students will have learned and still has value several years after the course is over?

Step 3: What would students need to do during the course to be able to do well on the assessment activities?

Step 2: What would students have to do to convince me that they achieved the learning goals/objectives?

NOTE: Any break in any one of the three connections = course cannot benefit from integration and is a broken course = will not work effectively.

Information needing to be gathered (text book, other content and activities) Fink, L. D. (2003). Creating significant learning experiences. San Francisco, CA: JosseyBass.

Definitions Situational Factors •  Information needing to be gathered Circles •  Decisions that need to be made Arrows from Situational Factors •  Information used in the process of making they three key sets of decisions Arrows Connecting Circles •  Components needing to be connected and support each other

Integrated Course Design Key questions when designing learning experiences: 1.  What are the important situational factors? 2.  What are the learning goals/objectives? 3.  What types are feedback and assessment are provided? 4.  What types of teaching and leaning activities will suffice to achieve the learning goals/objectives? 5.  Are the components connected and integrated – are they consistent and supportive of each other?

12 Steps of Integrated Course Design Steps in Integrated Course Design Initial Phase: Components 1.  Situational Factors 2.  Learning goals 3.  Feedback & Assessment 4.  Teaching & Learning Activities 5.  Integrate Component Parts Intermediate Phase: Coherent Whole 6.  Course Structure 7.  Teaching Strategies 8.  Overall Set of Learning Activities Final Phase: Four Remaining Tasks 9.  Grading system 10.  Possible Problems 11.  Write Syllabus 12.  Evaluation of Course and Teaching

Important Situational Factors –  –  –  –  –  – 

Context of the Teaching and Learning Situation Expectations of External Groups Nature of the Subject Characteristics of the Learners Characteristics of the Teacher Special Pedagogical Challenge

Formulate Significant Learning Goals –  –  –  –  –  – 

Foundational Knowledge Application Integration Human dimension Caring Learning How to Learn

Formulate Feedback and Assessment Procedures –  “Audit-ive” vs. Educative Assessments –  FIDeLity – Frequent, Immediate, and Discriminating (based on clear criteria) and Lovingly

Teaching & Learning Activities –  Passive vs. Active Learning

Integration of Component Parts –  Assess Integration

Phases 2 & 3: Coherent Whole and Remaining Tasks Steps in Integrated Course Design Initial Phase: Components 1.  Situational Factors 2.  Learning goals 3.  Feedback & Assessment 4.  Teaching & Learning Activities 5.  Integrate Component Parts Intermediate Phase: Coherent Whole 6.  Course Structure 7.  Teaching Strategies 8.  Overall Set of Learning Activities Final Phase: Four Remaining Tasks 9.  Grading system 10.  Possible Problems 11.  Write Syllabus 12.  Evaluation of Course and Teaching

Course Structure –  –  – 

Where do you want to go? How will you know when you get there? Determine Resources

Teaching Strategies –  –  –  –  – 

Create Thematic Course Structure Determine Major Topics Identify Specific Learning Activities Develop communication Plan to Students Develop course evaluation plan

Overall Set of Learning Activities –  Passive vs. Active Learning –  What will students need to do in and out of class? –  Develop overall scheme of learning activities

Grading –  Develop Grading

Possible Problems –  What could go wrong and anticipate possible problems

Syllabus –  How will students know what you are planning to do – now write the syllabus

Evaluation –  Plan evaluation of the course and your teaching performance

Assessments

To create authentic assessment 1.  Realistic 2.  Require judgment and innovation 3.  Ask the student to do the subject 4.  Replicate or simulate the contexts in which adults are tested in the workplace, in civic life, and in personal life

To create authentic assessment 5. Assess the student’s ability to use a repertoire of knowledge and skill efficiently & effectively to negotiate a complex task 6. Allow appropriate opportunities for student to rehearse, practice, consult resources, and get feedback on and refine performances and products

Rich Learning Experiences In Class: •  Debates •  Role playing •  Simulations •  Dramatizations

Outside of Class: •  Service learning •  Situational Observations •  Authentic Projects

In Depth Reflective Dialogue With whom? •  Oneself •  Others (teacher, clients, other students, public) About what? •  Subjective of the course •  Learning process: What am I learning? Of what values is this? How did I learn: best, most comfortably, with difficulty, etc.? •  What else do I need to learn?

Assessment of Dimensions

Types of Assessments Dimension

Types of Assessments

Foundational

Knowledge probes, frequent quizzes, content reviews

Application

Rubrics, simulations, practicums, case reviews, required self-assessments

Integration

Essays, case studies, required peer assessments, papers, interdisciplinary assessments, problem solving

Human dimension

Leadership, effective team contribution assessments

Caring

Academic honors, community engagement, volunteering/mentoring

Learning how to learn

Required reflections or self-assessment, development plans, problem-solving or self-directed learning

Pulling it all together example Communication Class Readings

Feedback

Lecture

Reflection

Choice of engagement

Engagement

1.  In which community engagement did you participate? 2.  What strategies did you purposefully use when communicating with participants. Give an example. 3.  What communication barriers were present in your interactions with participants? Give examples. 4.  What did you learn or what learning was affirmed during your interaction with others? 5.  What surprised you about communication with people in this setting? 6.  What specific components of your communication skills do you wish to develop in the future?

Why? •  Authentic experience •  Used foundational knowledge to –  apply it in a real situation, –  connect communication strategies with people they were unfamiliar with, –  reflect about what they learned and how they felt, and finally –  think about their future growth/learning.

References •  Caulfield, J. (2011). How to design and teach a hybrid course: Achieving student-centered learning through blended classroom, online, and experiential activities. Sterling, VA: Stylus Publishing. •  Chickering, A. & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 3-7. •  Diaz, V. & Brown, M. (2010). Blended learning: A report on the ELI focus session (ID: ELI3023), Educause Learning Initiative (ELI). •  Fink, L. D. (2003). A self-directed guide to designing courses for significant learning. Download available at http://www.deefinkandassociates.com/index.php/resources/

References •  Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass. •  Garrison, D., Anderson, T. & Archer, W. (200o). The internet and higher education 2 (2-3): 87-105. •  Graham, C., Cagiltay, K., Byung-Ro, L., Craner, J., & Thomas, M. (2001). Seven principles of effective teaching: A practical lens for evaluating online courses. Technology Source, March/April 2001. •  Manning, S. & Johnson, K. (2011) The technology toolbelt for teaching. San Francisco, CA: Jossey-Bass. •  Quinn, C. (2012). Mobile academy: mLearning for higher education. San Francisco, CA: Jossey-Bass.

Questions Contact information: Bette Poutre, MS Office of eLearning and Academic Technology School of Pharmacy and Health Professions Creighton University Omaha, NE Email: [email protected] Phone: 402.280.3798 Website: http://spahp.creighton.edu/departments-offices/ elearning-and-academic-technologies