Best Practices in Presenting Data and Information NEAIR 2010 Summer Drive-In Workshop Terra Schehr Assistant Vice President for Institutional Research and Effectiveness Loyola University Maryland
Agenda Underlying philosophy Sectors/audiences Deliverables (Word, PPT, Excel, PDF . . .) Tables, charts, and supergraphics What is the point?
T Schehr
Summer 2010
2
Underlying Philosophy
T Schehr
Summer 2010
3
What we are known for
Useful
Trust
T Schehr
Summer 2010
4
Knowledge Management Traditional IR Information put to use
Data
Knowledge
Data placed in context T Schehr
Information
Summer 2010
Resource: ISKME
5
Approaching the Data 3 Rs Reduce Reuse Recycle
T Schehr
What kind of cop are you going to be? Joe Friday Vic Mackey ?
Summer 2010
6
Small group activity What do you wonder/worry about when preparing/ presenting data? T Schehr
Summer 2010
7
Sectors and audiences
T Schehr
Summer 2010
8
Audiences Target Audience
Where to Aim
Board of Trustees / Regents
Moderate detail; illustrative quotes; summaries that help them make connections; erring on side of policy-setting vs. micromanaging
Cabinet / Executive
Moderate detail; low amount of narrative; tables / bullet lists that help consolidate data into broad topical categories; conclusions, implications, recommendations clearly stated
Faculty and Other Experts
Fairly high amount of detail in tables; graphics that display results; define terminology; clear inferences and conclusions; references and citations
Lay People
Simple graphics; illustrative quotes
All
Organize around themes; clear labeling of sections so reader can skim/skip; technical and statistical details in an appendix
T Schehr
Summer 2010
Resource: Bers & Seybert
9
Audience Appropriate Reporting
Data
faculty & other experts
all
Knowledge cabinet / executive, faculty & other experts, lay people
Information T Schehr
Summer 2010
Resource: ISKME 10
Dale’s Cone of Experience
Dale, E (1969) T Schehr
Summer 2010
11
Dale’s Cone of Experience
http://www.marketingplannow. com/index.asp?page=2090 T Schehr
Summer 2010
12
Deliverables
T Schehr
Summer 2010
13
Purposes of Reports Historical record Support for planning Support for policy/program development Support for policy/program improvement Public relations Compliance T Schehr
Summer 2010
Resource: Bers & Seybert
14
Small group activity What are the advantages/ disadvantages of: Word, Excel, PPT, & PDF as the platform for deliverables? T Schehr
Summer 2010
15
Tools for Deliverables Tool
Pros
Cons
Excel
Can interact with the data
Usually no narrative summary provided—conclusions are up to the user
PDF
All content is protected from change Not interactive—if we want our work to be used we need to let people use it (interact, copy-paste, etc.)
PPT
Bells and whistles
Can be challenging to make it “stand-alone”
Word
Can get a lot of content on a page
Visually less interesting than PPT
T Schehr
Summer 2010
16
Essential Content Meaningful title Leader/header on each page Author/office of origin Source of data Page numbers
T Schehr
Summer 2010
17
Organizing Content Focus on the big ideas/news Aggregate first then drill down to subgroups Show
trends when possible Identify significant differences
T Schehr
With survey items start with the “overall” item (e.g. overall satisfaction) and then discuss more specific items Summer 2010
18
Organizing Content – cont. Unless there is a meaningful order to the categories in a table/chart, sort results from largest to smallest Percentages may be more meaningful than means Top/Bottom Two Box %
T Schehr
Summer 2010
19
Tables and Charts
T Schehr
Summer 2010
20
Tables vs. Charts Tables
Charts
Data
Exact values, end-user can manipulate the data (depending on which deliverable tool is used), works for qualitative data
Sometimes requires user to estimate the values, data is not (usually) interactive, not appropriate for qualitative data
Trends
May take some study to see trends and patterns--this increases as the number of data elements increases
Easy to see trends and patterns
Interactions May take some study to see interactions--this increases as the number of data elements increases T Schehr
Summer 2010
Easy to see interactions
21
Question: What other things go into your decision to use either tables or charts? T Schehr
Summer 2010
22
Overall Tips Don’t settle for the default settings Maximize the amount of “ink” used on the data and minimize the “ink” everywhere else (grid lines, boxes, etc.) If something stands out (different font, different colored bar in a chart, etc.) it should be for a reason Round results (except things like GPA) T Schehr
Summer 2010
23
Tables
T Schehr
Summer 2010
24
Tables – Basic Organization Organize columns and rows in a meaningful way Natural order (past to future) Meaningful grouping (ranks of faculty) Alphabetical Magnitude of values (largest to smallest)
T Schehr
Summer 2010
25
Tables – Basic Organization – cont. Only one piece of data per cell Maintain consistent alignment of data Column labels centered Numbers to right Decimals and percentage signs aligned Text to left T Schehr
Summer 2010
26
Tables – Text Avoid orientating text differently than from left –to-right (horizontal) Avoid using ALL CAPS Use meaningful variable names and labels Try for consistent length of variable labels
T Schehr
Summer 2010
27
Tables – Numbers Use appropriate number formats Commas in whole numbers % next to every percentage value Include column and row summaries as appropriate Totals Means/Medians T Schehr
Summer 2010
28
Tables – Example 1 Consistent length of label?
Meaningful label?
More than one piece of data
F06
F07
F08
F09
Took placement test
1828 83.5%
1855 83.2%
2221 86.5%
1889 95.6%
5904 84.5%
Did not take Placement test
361 16.5%
375 16.8%
346 13.4%
85 4.3%
1082 15.4%
Total
2189
2230
2567
1974
6986
Shading is too dark
Inconsistent alignment of data
Number format and decimal places T Schehr
Total
Summer 2010
Meaningful summary?
29
Tables – Better Example 1 Took Placement Test?
Fall 2006
Fall 2007
Fall 2008
Fall 2009
N
%
N
%
N
%
N
%
Total
Avg
Yes
1,828
84%
1,855
83%
2,221
87%
1,889
96%
5,904
88%
No
361
16%
375
17%
346
13%
85
4%
1,082
13%
Total
T Schehr
2,189 100%
2,230 100%
2,567 100%
Summer 2010
1,974 100%
Summary
6,986 100%
30
Tables – Even Better Example 1 Took Placement Test?
Fall 2006
Fall 2007
Fall 2008
Fall 2009
Avg
Yes
84%
83%
87%
96%
88%
No
16%
17%
13%
4%
13%
T Schehr
Summer 2010
31
organization is not meaningful
all CAPS and centered
T Schehr
Tables – Example 2
Fictitious Data on Where Urban Studies Graduate Students Enroll Row Labels ANTIOCH UNIVERSITY - NEW ENGLAND BALTIMORE CITY COMMUNITY COLLEGE BAYLOR UNIVERSITY (blank) BOSTON UNIVERSITY CREIGHTON UNIVERSITY IMMACULATA UNIVERSITY LAKE TAHOE COMMUNITY COLLEGE LOUISIANA TECH UNIVERSITY NORTHEASTERN UNIVERSITY SALT LAKE COMMUNITY COLLEGE ST JOHNS UNIVERSITY TOWSON UNIVERSITY TULANE UNIVERSITY UNIVERSITY OF KANSAS UNIVERSITY OF PHOENIX VILLANOVA UNIVERSITY Grand Total
2006
2007 2008 2009 7 1 5 2 2 1 2
3 3 9 2 5 2
2 2
11 5 2 8 1 25
Summer 2010
34
3
2 1
4 18
8
Grand Total 7 1 7 5 3 3 9 2 5 4 2 13 9 2 8 1 4 85
difficult to track across the rows
Data are not aligned in columns
32
Tables – Better Example 2 Number of Students Admitted in fall of . . . Where do Urban Studies graduate students enroll? St. Johns Towson Immaculata University of Kansas Baylor Antioch University – New England Louisiana Tech Villanova Northeastern Creighton Boston University Tulane Salt Lake Community College Baltimore City Community College Grand Total T Schehr
2006 11
2007
2008
5
3
2009 2 1
5
2
9 8 7 5 4 2 3
2 3 2 2 25
Summer 2010
29
17
1 6
Total 13 9 9 8 7 7 5 4 4 3 3 2 2 1 77 33
Tables – Example 3 Table 3. Amount of Time in a Typical Week During the Prior Year Spent in Prayer or Meditation Loyola Class of 2008 as Entering FirstYear Students
Loyola Class of 2008 as Juniors
Class of 2008 Juniors at Catholic Institutions
None
21%
23%C
30%
Less than 1 hour
39%
41%
34%
One to Two Hours
31%
25%
26%
Three hours or more
9%
11%
10%
C – indicates a significant difference between Loyola juniors and juniors at other Catholic institutions.
T Schehr
Summer 2010
34
Tables – The Last Word When a table is a cross-tab put the independent variable on the columns and use column percentages If a table breaks across pages, repeat column and row labels as appropriate Pick a format that works for you (and your audiences) and stick with it T Schehr
Summer 2010
35
Charts
T Schehr
Summer 2010
36
Charts – Basic Guidelines Organize the data in a meaningful way Use prominent and clear graphical elements to show data Don’t clutter the interior of a chart Avoid using 3-dimensional charts
T Schehr
Summer 2010
37
Charts – Basic Guidelines – cont. Include the detail about the data points on the chart whenever possible Keep series labels and legends short and easy to read When possible, label the chart data directly instead of using a legend
T Schehr
Summer 2010
38
Types of Charts Depict size Depict change over time Depict what is typical or, alternatively, exceptional Depict relationships or predictions
T Schehr
Summer 2010
Resource: Bers & Seybert
39
Charts Depicting Size
T Schehr
Summer 2010
40
Pie Chart – Example 1 Is this college your . . .
1st Choice 2nd Choice 3rd Choice > 3rd Choice
T Schehr
Summer 2010
41
Pie Chart – Better Example 1 Is this college your . . . 2nd Choice 26%
3rd Choice 7% > 3rd Choice 4%
1st Choice 64%
T Schehr
Summer 2010
42
Pie Chart – Comparison Is this college your . . .
Is this college your . . .
1st Choice
2nd Choice 26%
2nd Choice
3rd Choice 7% > 3rd Choice 4%
3rd Choice > 3rd Choice
T Schehr
Summer 2010
1st Choice 64%
43
Bar Chart –Example 2 Percent of seniors who spent 6 or more hours a week in each leisure-time activity Exercising Online social networks Partying Playing video games Prayer/meditation Reading for pleasure Socializing with friends Student clubs Surfing the net Volunteering 0% T Schehr
20% Summer 2010
40%
60%
80%
100% 44
Bar Chart – Better Example 2 Percent of seniors who spent 6 or more hours a week in each leisure-time activity Socializing with friends Partying Surfing the net Exercising Student clubs Online social networks Volunteering Reading for pleasure Playing video games Prayer/meditation
88% 52% 32% 28% 14% 12% 6% 4% 2% 1% 0%
T Schehr
20% Summer 2010
40%
60%
80%
100% 45
Bar Chart – Comparison Percent of seniors who spent 6 or more hours a week in each leisure-time activity
Percent of seniors who spent 6 or more hours a week in each leisure-time activity
Socializing with friends Partying Surfing the net Exercising Student clubs Online social networks Volunteering Reading for pleasure Playing video games Prayer/meditation
Exercising Online social networks Partying Playing video games Prayer/meditation Reading for pleasure Socializing with friends Student clubs Surfing the net Volunteering 0%
T Schehr
20%
40%
60%
80%
100%
Summer 2010
88% 52% 32% 28% 14% 12% 6% 4% 2% 1% 0%
20%
40%
60%
80%
100%
46
Charts Depicting Change Over Time
T Schehr
Summer 2010
47
Line Chart – Example 3 High School Graduates Who Enroll in College Within 12 Months of Graduating 100% 90% 80% 70% 60% 50% 40% 30%
73% 69%69%69% 69% 68% 67% 66% 66% 65% 65% 64% 62% 64% 64%66% 63% 63% 62% 61% 61% 60%59% 59% 59% 59% 59% 57%56% 57% 56% 56% 55% 55% 55% 55% 55% 53% 52%53% 51%66% 51% 50% 48% 62% 62% 47% 46% 44% 59% 58% 43% 43% 57% 57% 42% 55% 55% 54% 40% 54% 53%54% 54% 38% 52% 52% 37% 51% 36% 49%51%47% 52% 43%44%44% 43% 42%
African-American Hispanic White
34%
20% 10% 0%
T Schehr
Summer 2010
48
Line Chart – Better Example 3 High School Graduates Who Enroll in College Within 12 Months of Graduating 80% 69%
70%
58%
60% 52%
55%
50% 50%
40% 43%
30%
African-American
T Schehr
Summer 2010
Hispanic
White
49
Line Chart – Comparison High School Graduates Who Enroll in College Within 12 Months of Graduating
High School Graduates Who Enroll in College Within 12 Months of Graduating 80%
100% 90% 80% 70% 60% 50% 40% 30%
73% 69%69%69% 69% 68% 67% 66% 66% 66% 65% 65% 64% 64% 64% 63% 63% 62% 62% 61% 61% 60%59% 59% 59%59% 59% 56% 56% 55% 55% 57% 53% 55%57%56% 55% 55% 53% 52% 51%66% 51% 50% 48% 62% 62% 47% 46% 59% 58% 43% 43% 57% 42% 44% 55%57% 55% 54% 40% 54% 54% 54% 53% 38% 52% 52% 37% 52% 51% 51% 36% 49% 47% 43%44%44% 43% 42%
20%
69%
70%
African-American Hispanic
58%
60% 52%
White
55%
50% 50%
34% 40% 43%
10% 0%
30%
African-American
T Schehr
Summer 2010
Hispanic
White
50
The Area Chart Debate – Example 4 Private 4-Year Published and Net Tuition and Fees in Constant 2007 Dollars $25,000 $23,712
$20,000
39%
$14,755
$15,000
36%
$14,400
$10,000 $9,377
$5,000
$0
Tuition and Fees
T Schehr
Summer 2010
Net Tuition and Fees
51
The Area Chart Debate – Example 4 Private 4-Year Published and Net Tuition and Fees in Constant 2007 Dollars $25,000 $23,712
$20,000
39%
$14,755
$15,000
36%
$14,400
$10,000 $9,377
$5,000
$0
Tuition and Fees
T Schehr
Summer 2010
Net Tuition and Fees
52
Column Charts – Example 5 Enriching Educational Experiences Seniors 100
Effect size .35
.50
^
^
75
51.2 50
45.5
53.0
.26
48.2
.15
.67
50.3 40.8
25
0
Statistically significant difference from Loyola-2009 score
^ Statistically significant difference from Loyola-2004 score T Schehr
Summer 2010
53
Charts Depicting What is Typical or Exceptional
T Schehr
Summer 2010
54
Histogram – Example 6 Linguistics Dept.
Urban Studies Dept.
T Schehr
Summer 2010
55
Histogram and Number of Bins
T Schehr
Summer 2010
Resource: Robbins 56
Box and Whisker Plots – Example 7 Individual outlier cases + 1.5 IQR 75Th %-ile Median 25Th %-ile - 1.5 IQR
T Schehr
Summer 2010
57
Trick Excel into Box-like Plots Using Stock Plots – Example 8 2002 U.S. NEWS Academic Reputation Score 5 Range of Values 4 Average Value 3 Institution A 2
1 Tier 1
Tier 2
Weight towards overall score = 25%
T Schehr
Summer 2010
58
Charts Depicting Relationships or Predictions
T Schehr
Summer 2010
59
Scatterplot – Example 9 2007 SAT Reading and Writing Scores 650
SAT Writing Score
600 550 500 450 400 350 300 300
350
400
450
500
550
600
650
SAT Critical Reading Score T Schehr
Summer 2010
60
Scatterplot – Better Example 9 2007 SAT Reading and Writing Scores 650
SAT Writing Score
600 550 500 450 400 350 300 300
350
400
450
500
550
600
650
SAT Critical Reading Score T Schehr
Summer 2010
61
Scatterplot – Value of the Fit Line 2007 SAT Math and Writing Scores
650
650
600
600
SAT Writing Score
SAT Writing Score
2007 SAT Reading and Writing Scores
550 500 450 400 350
550 500 450 400 350
300
300 300
350
400
450
500
550
600
650
SAT Critical Reading Score
T Schehr
300
350
400
450
500
550
600
650
SAT Math Score
Summer 2010
62
HIGH
Scatterplot of Summary Data – Example 10 NEEDS ATTENTION
SUCCESS Interaction with faculty
Social life on campus
Tutorial help / academic assistance Faculty availability
Derived Importance
Major advising Culture & fine arts programming
Student center programs
Library facilities/ resources Computer services/support
Classroom/ lab facilities
Financial aid office Registrar's office Computer facilities/ resources
LOW
First-year advising
UNDERRECOGNIZED
LOW PRIORITY DISSATISFIED
VERY SATISFIED
Satisfaction
T Schehr
Summer 2010
63
Charts – The Last Word When using several charts in a document they should be consistent Use common chart types for similar types of data Use common color schemes and sizes Use a common baseline and scale
T Schehr
Summer 2010
64
Supergraphics
T Schehr
Summer 2010
65
What is a Supergraphic? Data rich Multi-graphic Usually a high resolution physical handout Ex: map, weather section in a news paper
T Schehr
Summer 2010
66
The Best Supergraphic Ever?
T Schehr
Summer 2010
Resource: Tufte
67
The Supergraphic
T Schehr
Summer 2010
68
http://awesome.good.is/transparency/web/1006/rise-of-walking-and-biking/flat.html
T Schehr
Summer 2010
69
http://awesome.good.is/transparency/web/0908/trans0809joblessinthecity.html
T Schehr
Summer 2010
70
Supergraphics – The Last Word “[Tufte] and I disagree. He thinks people are a lot smarter than I do. He likes packing a ton of information into a slide and letting people tease it out (same as the Napoleon graph in his first book). I go in the opposite direction. If you can get the info across at first glance, you win.” - Seth Godin
T Schehr
Summer 2010
71
Closing Thoughts
T Schehr
Summer 2010
72
Displaying Data and Information: What is the Point? Connect with the audience Direct audience’s attention Promote understanding and memory
T Schehr
Summer 2010
73
Connect With the Audience Principle of relevance Do not give too much or too little information Principle of appropriate knowledge Avoid concepts, jargon, and symbols that can not be easily explained in the display
T Schehr
Summer 2010
Resource: Kosslyn
74
Direct Audience’s Attention Principle of salience Make sure perception is reality Use formatting to highlight the differences you want readers to focus on Principle of perceptual organization If you want elements grouped in a particular way, do it yourself don’t leave it up to the reader T Schehr
Summer 2010
Resource: Kosslyn
75
Promote Understanding and Memory Principle of informative changes Be intentional about formatting Principle of capacity limitations Be careful with supergraphics
T Schehr
Summer 2010
Resource: Kosslyn
76
Resources Bers, T.H. & Seybert, J.A. (1999). Effective reporting. (Resources in Institutional Research, 12) Tallahasee, FL: The Association for Institutional Research. Institute for the Study of Knowledge Management in Education www.ISKME.org Kosslyn, S. (2007). Clear and to the Point: 8 Psychological Principles for Compelling PowerPoint Presentations. New York, NY: Oxford University Press. Robbins, N. (2005). Creating more effective graphics. Hoboken, NJ: John Wiley & Sons, Inc.. Tufte, Edward www.edwardtufte.com/tufte/index
T Schehr
Summer 2010
77
Terra Schehr Assistant Vice President for Institutional Research and Effectiveness Loyola University Maryland
[email protected]