Creating Rubrics for MODEL Assessment David Dirlam Senior Assessment Coordinator and Thomas Gattis Chair of Industrial Design Savannah College of Art and Design

What are you assessing for?

For Competence Use PAGE Rubrics Excellent

6

Satisfaction

Good Average 0

-6

Poor Functionality Must-Be Expected One Dimensional Desired

Exciting Attractive

Rating Product or Service Quality Check one box for each criteria Criteria 1 2 3 4 5

Poor

Average

Good

Excellent

Reaction Time

For Performance Use SWELL Rubrics 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

Sequences Which Expand Little by Little

1

6

11

16

21

Days

Time to recognize a sentence from Anderson, 1983

Rating Performance Speed and Accuracy Check one box for each criteria Criteria 1 2 3 4 5

None

Some

More

A Lot

For Transformation Use the Law of Succession Beginning Common at entry No growth No strength

Easy Extreme growth Little strength Inspiring Slow growth Great strength

Practical Moderate growth Moderate strength

Record a Transforming Choice Circle the most accurate summary of the person’s choices Beginning Development of a Particular Activity

Describe a practice that people do when they first try the activity (e.g., a hands-on workshop).

Easy

Practical

Inspiring

Describe the easy things people learn to do in the activity during their first few months of effort (e.g., an introductory course with a project).

Describe what people do after they have spent a few years learning the activity (e.g., a senior thesis in a major field).

Describe methods people use to change the activity for themselves and others (e.g., a contribution to the discipline).

For Innovation Use MODEL Rubrics Term

Problem

Solution

Matrices

Trillions of ways to design. Limited memory for terms.

Multiple discrete dimensions

Organized

Interacting with students requires fast retrieval.

2-4 terms per dimension

Developmentally

A common basis is needed for developmental theories.

Use the law of succession

Expertise

Where can we get the theories?

Interview Experts

Labeling

How can we combine theories from different experts?

Analyze keyword networks

Interview Tool Beginning

Learning

Skilled

Innovating

Decision

Try

Learn

Become proficient

Make contributions

Practice Time

None

Weeks to months

A few years

5 years to start, 10 to use regularly

Effects

Peripheral Participation

Take little practice; get some reward

Enable living wages but no excitement

Enable Discoveries

Helpful Prompts

Save this What have your If unsure, If a student still uses “sandwich filling” students or save for after this strategy when for after the Easy colleagues done that the Easy about to graduate, and Innovating surprised you with strategy. you feel discouraged. approaches. its appropriateness?

Interview Process*

Beginning Strategy  Start with a friend or two. Easy Strategies  Interview people in their office or classroom (memory is place dependent).  Remind them of their classes and students by using “targeted small talk.”  Introduce the transforming choice. Practical Strategies  Present the Law of Succession, the MODEL rubrics table, and the Interview Tool in 5-10 minutes.  Record brainstorms in the first 10 minutes and later remind them of the dimensions these suggest.  Use reminders of the developmental progression (interview starting ideas) in the first ten minutes. Rehearse the meanings of the four levels. Make sure that “easy” strategies are fast growing or that “inspiring” ones require deep understanding of the needs of their discipline.  Remind the interviewee that the goal is to describe behaviors or strategies that people can reliably identify. Try to imagine what observable “traces” or “products” would be clear indications.  Make sure the interviewee explains technical or disciplinary language enough so that an introductory student could understand.  Expect 7-15 dimensions.  Group the results of several interviews into 10-20 total dimensions (3-5 dimensions per outcome)

* Prepared after discussion with fellow MODEL rubrics interviewer, Regina Lowery

Interview Process (continued) Inspiring Strategies  Use Keith Sawyer’s “Yes, and…” approach. Make connections with own experiences or with other interviews. Especially mention ideas that add to the understanding of both you and the interviewee.  If you have more than 50 dimensions from several interviews, contact us for our organizing tool: a MS Excel workbook to do keyword network deconstruction. After grouping the dimensions, write a 50-word maximum abstract of each level. Then give an easily remembered title to each abstract.  Compare your results with Trillions of Ways to Design and send comments to [email protected] (all comments used in book will be acknowledged).

* Prepared after discussion with fellow MODEL rubrics interviewer, Regina Lowery

Benefits  Unified theory of the development of disciplinary expertise  Collective—less biased, more buy-in  Comprehensive—the 18 Industrial Design dimensions create 70 billion ways to design; the 21 dimensions from 20 disciplines create 4 trillion ways.  Efficient—2 hours per faculty member, 4 hours each for coordinator

 Stimulate dialog  Recognize unique contributions  Applicable to client-designer relations  Use for authentic, yet expert interactions with students

If you would like to be a Trillion Ways to Design collaborator, please add your name, email and some areas from the chart below to the sign-up list.