Journey from Subjective to Objective: Capturing User Experience Digital User Experience (DUX) Working Group in Collaboration with Sciences Librarian Indiana University – Purdue University Indianapolis Yoo Young Lee Eric Snajdr Lisa Calvert Andy Smith
Image Credit: h ttp://www.ploink-‐brothers.com/2016/03/23/get-‐results-‐by-‐not-‐making-‐these-‐web-‐design-‐mistakes/
@DUX_YooYoung
Table of Contents 1. Introduction 2. Usability vs. User Experience 3. UX Method & Data Analysis 4. Results 5. Discussion & Next Steps 6. Q&A @DUX_YooYoung
Introduction Background Information Project launch
User research
Design & Implementation
2014
Evaluation
2015.07 Launch • DUX working group formed • Project of the redesign of UL website initiated
• Surveys • Google Analytics • User behavior
• Wireframes
• UX in the library
• Mockups
instruction
• Prototypes • Informal UX Testing
@DUX_YooYoung
Introduction Research Questions? 1. Measure overall UX evaluation 2. Measure overall task performance 3. Difference between what they said and what they did? 4. Library instruction affects UX? If so, what ways?
User experience for the web is all about how users feel when interacting with a website or
Information architecture
interface. (Schmidt, 2012)
Web UX
Usability
@DUX_YooYoung
UX Method How to capture UX?
Why in the library instruction?
• In the library instruction: 9 different sessions • Total participants: 213 students
Participants • Enrolled in sciences program
• Valid data: 160 students
• Lower level: 107 students
Valid data
• Upper level: 53 students
• Over 18 years old • Completed all four: tasks/ pre&post surveys/ instruction
@DUX_YooYoung
UX Method How to capture UX?
Tasks
Pre-‐Survey
• Brief overview
• 9 questions
• 6 tasks under 5
• 5 minutes
minutes • Tool: Screencast-‐
• Tool: Survey Monkey
Library instruction
Post-‐ Survey
• Information
• 9 questions
literacy
• 5 minutes • Tool: Survey Monkey
O-‐Matic/ Survey Monkey @DUX_YooYoung
UX Method Tools used • Online survey: Survey Monkey • Screen capture: Screencast-‐O-‐Matic • Analysis: R for data analysis and R and Excel for display
@DUX_YooYoung
Video
PLAY
@DUX_YooYoung
UX Method Data Coding
@DUX_YooYoung
UX Method Data Coding • Task: Yes/ No/ Skip/ Non complete/ Title of the page • Behavior • Time
@DUX_YooYoung
Results Overall UX Evaluation on Website (Pre-‐survey right after tasks)
5: Strongly agree 4: Agree
Negative Aspects
Positive Aspects Easy to u se 5.00
3: Neither agree nor disagree 2: Disagree
Difficult to u se 5.00
4.00
1: Strongly disagree 0: N/A
4.00 Satisfied
3.00
Fun to u se
2.00
Uncertain
3.00 2.00
1.00
1.00
0.00
0.00
Confident
Frustrating to u se
Helpful
Frustrated
Useless
Confusing User-‐friendly
Data sets: All
@DUX_YooYoung
Results Overall UX Evaluation on Website (Pre-‐survey right after tasks)
5: Strongly agree 4: Agree 3: Neither agree nor disagree
Negative Aspects
Positive Aspects
2: Disagree All
Lower
Upper
All
Easy to u se 5
3
Data sets: All α = 0.05
Fun to u se
Uncertain
3.00 2.00
1
1.00
0
0.00
Helpful
User-‐friendly
1: Strongly disagree 0: N/A
4.00
2
Confident
Upper
Difficult to u se 5.00
4
Satisfied
Lower
Frustrated
Frustrating to u se
Useless
Confusing
@DUX_YooYoung
Results Overall Task Performance T1: Find information on how many books you can borrow. T2: Find your subject guide. T3: Find information about the 3D Printing Studio. T4: Where would you go when you don’t know where to start on your research project? T5: Find one scholarly article in your major. T6: Find one database in your major.
@DUX_YooYoung
Results Overall Task Performance (Right Answer) 5: Answer right 5 out of 5 60
4: Answer right 4 out of 5 3: Answer right 3 out of 5 2: Answer right 2 out of 5
48
50
1: Answer right 1 out of 5 0: No right answer
40
# of students 30 30 24
26
26
20
10
6
0 0
Data sets: All
1
2
Scores
3
4
5
@DUX_YooYoung
Results Overall Task Performance Upper Level
Lower Level
Mean of Upper: 2.77 Mean of Lower: 1.86 P-‐value: 0.0001851
α = 0.05
@DUX_YooYoung
Results Overall Task Performance (Previous Library Instruction) A
A: Students who have never had l ibrary
B
Mean
2.07
2.12
Variance
1.48
2.01
46.00
42.00
Observations Pooled Variance
1.73
Hypothesized Mean Difference
0.00
df
86.00
t S tat
-‐0.19
instruction before B: Students who have had 1 -‐3 library instruction sessions before
5: Answer right 5 out of 5 4: Answer right 4 out of 5 3: Answer right 3 out of 5
P(T non-‐valid data
•
Professor? Instructor?
2. Data coding •
160 pre & post survey data
•
160 video to be analyzed
3. We knew what they did, but didn’t know why they did it @DUX_YooYoung
Q&A
Questions?
@DUX_YooYoung
Journey from Subjective to Objective: Capturing User Experience Digital User Experience (DUX) Working Group in Collaboration with Sciences Librarian Indiana University – Purdue University Indianapolis Yoo Young Lee Eric Snajdr Lisa Calvert Andy Smith
Image Credit: h ttp://www.ploink-‐brothers.com/2016/03/23/get-‐results-‐by-‐not-‐making-‐these-‐web-‐design-‐mistakes/
