Using Structural Cues to Guide Readers

Jan H. Spyridakis, Ph.D. Professor November 2006 ©Copyright 2006. Jan H. Spyridakis. All rights reserved.

Designing the Future of Communication

Introduction ™ Emerging tools and technology for reading ™ Print materials moving to the Web ¾ Conflated layouts: carry-over from the print world (e.g., long scrolling articles, lack of concrete links, global menus that disappear)

™ Rhetorical cueing strategies often based on guidelines from print ™ Need for empirically based guidelines for design of online information Department of Technical Communication, University of Washington

Presentation Overview ™ Supporting literature ¾ Text linearity ¾ Signaling & schema theory

™ Recent work investigating the effect of structural cueing on comprehension, perceptions, and behavior ™ Our experiment generator toolkit—WebLab UX ™ Conclusions ™ Future Directions Department of Technical Communication, University of Washington

Literature: Linear vs. Nonlinear Text ™ Who controls the text? Author? Reader? ™ Linear text facilitates comprehension ™ Nonlinear text (e.g., hypertext) facilitates perceptions of use, learner control, interactivity, and cognitive flexibility ¾ But it can cause cognitive overload & disorientation

™ Need to mitigate problems w/ hypertext to maximize the benefits

Department of Technical Communication, University of Washington

Literature: Signaling in Print ™ E.g., Headings, previews, logical connectives, TOCs, indices ™ Create schemata—critical for forming a coherent text representation ™ Facilitative effect especially when readers find information only moderately familiar, interesting, or easy Department of Technical Communication, University of Washington

Literature: Signaling in Hypertext ™ E.g., Previews, links, menus, TOCs, site maps, indices, graphical overviews ™ Previews improve knowledge acquisition with search tasks and also perceptions ™ Graphical overviews enhance performance (especially for readers with low prior knowledge) ™ Poor signals degrade performance ¾ E.g., High frequency headings reduces comprehension of linear documents with online readers Department of Technical Communication, University of Washington

Gaps, Problems, Solutions ™ Findings primarily from print or from closed hypertext systems ™ Web studies often focus on searching, not browsing ™ Users aren’t assessed in their natural environments ™ Need to gain empirical support for effective design of Web-based signals ™ Advance capabilities of technology to support remote user testing, e.g., WebLab UX Department of Technical Communication, University of Washington

Some of Our Recent Work ™ Explicitness of local navigational links ™ Intriguing and informative hyperlink phrasing ™ Presence/absence of text previews and menus Measures: comprehension, perceptions, and behavior Department of Technical Communication, University of Washington

Methods Shared across Studies ™ Studies conducted remotely through the Internet at participants’ own computer/ location/time (announced through flyers, discussion boards, hyperlinks on sites)

™ Participants ¾ Engineering writing classes (M age ~ 21; gender ~ 75% male) OR ¾ General population, e.g., users of medical Web sites (M age ~ 47; gender ~ 75% female) ¾ Experienced Web users ¾ Well-educated but they do rate study sites in midrange of familiarity, difficulty, interest Department of Technical Communication, University of Washington

Methods Shared across Studies cont. ™ Browsing scenarios ™ Ss asked to spend ~ 20 min. browsing (Actual M ~ 11-14 min.) ™ Pre/post browsing surveys ™ Participation incentives ™ Naturally occurring Web sites ¾ Not too familiar, interesting, or easy ¾ Explorable in 15-20 min. ¾ Content with general OR specific appeal

Department of Technical Communication, University of Washington

Methods Shared across Studies cont. ™ Nonnative English speakers excluded from analyses in ALL studies ¾ They significantly differed from native English speakers on comp. and perceptions ¾ Too few to analyze by experimental conditions

™ Significant (sig.) results: p ≤ .05

Department of Technical Communication, University of Washington

Study Introduction & Consent Form

Department of Technical Communication, University of Washington

Study Instructions

Department of Technical Communication, University of Washington

Browsing Scenarios

Department of Technical Communication, University of Washington

Pre-Browsing Surveys

Department of Technical Communication, University of Washington

A Study Web Site

Department of Technical Communication, University of Washington

Post-Browsing Surveys Comprehension

Perceptions

Department of Technical Communication, University of Washington

Post-Browsing Surveys cont. More Perceptions

Department of Technical Communication, University of Washington

Study 1: Explicitness of Local Navigational Links ™ Effect of explicitness of wording of local navigational links on users’ comprehension, perceptions of use, and behavior ™ Replacement of generic link labels with explicit labels will lead to: ¾ Higher comprehension ¾ More positive perceptions of use ¾ Greater site exploration Department of Technical Communication, University of Washington

Specific Literature: Explicitness of Link Labels ™ Labels that reveal relationship of source and target positively affect comp. & perceptions of use ™ Labels that reveal target content promote focused reading ™ Local nav. links can conflict with browser functionality (e.g., browser back button)

Department of Technical Communication, University of Washington

Methods ™ 84 participants from general pop. ™ Web site on Osteoarthritis: 4 linear articles, 28 pages ™ 4 Conditions: varying in semantic and organizational explicitness of local nav. links ™ Comprehension test: 18 questions

Department of Technical Communication, University of Washington

Four Web Sites: 4 Versions of Link Labels Generic

Semantically Explicit

Organizationally Explicit

Semantically & Organizationally Explicit Department of Technical Communication, University of Washington

Department of Technical Communication, University of Washington

Results: Comprehension

# Comp. Questions Correct

™ Sig. lower comp. for subjects in Org. cond. (e.g., “Section 3 of 9”) 14 12 10 8 6 4 2 0 Next/Pre

Sem

Org

Sem/Org

Department of Technical Communication, University of Washington

Results: Perceptions of Use ™ Sem/Org cond. (e.g., “Section 3 of 9: Causes”) reported following > embedded links ™ Positive comments > most explicit cues (Sem/Org and Org) ™ Neg. comments > least explicit cues (Next/Pre & Sem) 100

% Pos.

% Comments

90

% Neg.

80 70 60 50 40 30 20 10 0

Next/Pre

Sem

Org

Sem/Org

Department of Technical Communication, University of Washington

Results: Log Files ™ Sem/Org & Next/Pre led to seeing most pages; Org led to least 50

% Pages Visited

40

30

20

10

0 Next/Pre

Sem

Org

Sem/Org

Department of Technical Communication, University of Washington

Conclusions ™ Implications for Web authors ¾ Choose link wording carefully—some cueing can lower comprehension ¾ Double up on signals ¾ Users like what they are used to seeing

Department of Technical Communication, University of Washington

Study 2: Intriguing & Informative Hyperlink Phrasing ™ Effect of generic, informative, and intriguing link labels (global nav. and embedded) on users’ comp., perceptions of use, and behavior ™ Informative links will improve comprehension, particularly inferential comprehension ™ Intriguing links will encourage site exploration Department of Technical Communication, University of Washington

Specific Literature: Hyperlink Phrasing ™ Informative link labels improve search accuracy—if not too long and if clearly worded ™ Seductive details divert readers from main points ¾ Interfere with understanding the main topic ¾ May lead to misinterpretation ¾ May lead to wider exploration of content Department of Technical Communication, University of Washington

Method ™ 475 participants from Engr. classes ™ Scenario: You’re a new park ranger… ™ Browsed 1 of 5 Web sites on American Samoa, 19 pages (15 min) ™ 5 conditions: embedded and global links varying by informativeness and intriguingness ™ Comprehension (19 factual and 19 inferential MC questions) ™ Created toolkit to support the work Department of Technical Communication, University of Washington

Shifting Gears: WebLab UX ™ Remote research toolkit that we have been developing—WebLab UX ™ WebLab UX Automatically… ¾ Generates conditions from single source code ¾ Assigns participants to conditions ¾ Generates surveys from flat text files ¾ Records user data: individual’s behavior in customized log files and survey results Department of Technical Communication, University of Washington

Link Versions Generic

Intriguing

Informative

Biodiversity

Tropical Menagerie

Native Animal Diversity

Volcanoes

Samoa’s Fiery Past

Samoa’s Volcanic Origins

Department of Technical Communication, University of Washington

Link Conditions Nav/Embed Matched Yes

Navbar links Generic

Embedded links Generic

No

Generic

Intriguing

No

Generic

Informative

Yes

Intriguing

Intriguing

Yes

Informative

Informative

Department of Technical Communication, University of Washington

Department of Technical Communication, University of Washington

Results: Comprehension ™ Gen-Inf links sig. > total comp. than Inf-Inf links %. Comp. Ques. Correct

% Inf. Comp. Ques. Correct

™ Gen-Inf links sig. > infer. comp. than Intrg-Intrg links 60 50 40 30 20 10 0

60 50 40 30 20 10 0

GenGen

G enInt r g

G en- I nf

It r g It r g

Inf - Inf

GenGen

GenInt rg

Gen- Inf

It rg It rg

Inf - Inf

Department of Technical Communication, University of Washington

Results: Matched vs. Unmatched ™ Key differences between “matched” and “unmatched” nav. link vs. embedded link conditions Matched

Unmatched

Generic-generic

Generic-intriguing

Informative-informative Generic-informative Intriguing-intriguing

Department of Technical Communication, University of Washington

Results: Subjects in Unmatched Conditions ™ Visited more pages ™ Revisited pages more often—used more links ™ Correctly answered more comprehension questions, overall ™ Correctly answered more inferential comprehension questions about visited pages ™ Believed they learned more from visiting the site Department of Technical Communication, University of Washington

Conclusions ™ Detailed embedded links encourage browsing and learning ¾ Intriguing OR informative embedded links

™ Detailed navbar links don’t affect browsing behavior or learning

Department of Technical Communication, University of Washington

Study 3: Text Previews and Menus ™ Effect of text previews (preview/embedded links, preview/link list; link list only) and menus (present, absent) on comprehension, perceptions of use, and behavior ™ Previews will increase inferential and factual comp., time spent on pages, and ratings of content relatedness ™ Nav. menus will increase factual comp., site exploration, and accuracy of perceptions of site size and usefulness Department of Technical Communication, University of Washington

Specific Literature: Previews & Menus ™ Previews improve knowledge acquisition w/ search tasks and also perceptions ™ Menus increase site exploration ™ Poor signals degrade performance

Department of Technical Communication, University of Washington

Methods ™ 282 participants (Engr. Classes) ™ Scenario: Learn about Big Bend park to support upcoming job as tour guide. ™ 6 conditions: varying in text previews (3 levels) and navigation tab menus (2 levels) ™ Comprehension (16 inferential, 16 factual MC questions) Department of Technical Communication, University of Washington

Text Previews (menus absent) • List only • Preview w/ list • Preview w/ embedded links

Department of Technical Communication, University of Washington

Text Previews (menus present) • List only • Preview w/ list • Preview w/ embedded links

Department of Technical Communication, University of Washington

Results: Previews & Comp.

% Inferential Comp.

™ Previews with embedded links sig. > inferential comp. than previews with link list 50 40 30 20 10 0

Prev/Embd

Prev/List

List Only

Department of Technical Communication, University of Washington

Results: Nav. Menus & Comprehension ™ Nav. Menus did not influence comprehension! Absent

Percent Comprehension

50

Present

40 30 20 10 0 Factual

Inferential

Department of Technical Communication, University of Washington

Results: Previews & Perceptions ™ Previews with list and link list only sig. > SUS ratings than previews with embedded links (Use ratings very similar) SUS on 100 pt. scale

100 90 80 70 60 50

Prev/Embd

Prev/List

List Only

Department of Technical Communication, University of Washington

Results: Menus & Perceptions ™ Presence of menus sig. > SUS ratings vs. absence of menus (Use SUS on 100 pt. scale

ratings very similar) 100 90 80 70 60 50 Absent

Present

Department of Technical Communication, University of Washington

Results: Interactions on SUS & Use

SUS on 100 pt. scale

™ Previews with embedded links are most negatively affected by the absence of menus (Use ratings very similar) 100 90

Prev/Embd Prev/List List Only

80 70 60 50 Absent Present Department of Technical Communication, University of Washington

Results: Previews & Percent of Pages Visited ™ List only led to more exploration than previews with a list (trend toward sig., p = .059) Percent Pages Visited

100 90 80 70 60 50 Prev/Embd

Prev/List

List Only

Department of Technical Communication, University of Washington

Results: Menus & Percent of Pages Visited ™ Pres. of menus led to greater site exploration than absence of menus (trend toward sig., p = .061) Percent Pages Visited

100 90 80 70 60 50

Absent

Present

Department of Technical Communication, University of Washington

Conclusion ™ Previews w/embedded links positively influence inferential comp. but reduce perceptual ratings ™ Nav. menus do not affect comp. but do improve perceptual judgments ™ Link lists or menus lead to greater site exploration

Department of Technical Communication, University of Washington

Conclusions Across the 3 Studies ™ Designers of online information should understand: ¾ The need for cues in online documents ¾ That signals may not equally affect print and online readers ¾ That readers like what they are use to but can perform well with multiple types of signals ¾ That readers may perform differently than they perceive—and that design decisions must be made in context Department of Technical Communication, University of Washington

Design Recommendations ™ Choose link wording carefully—cueing can > or < comprehension ™ Be willing to double up on signals if they serve different purposes ™ Consider detailed wording for embedded links—they can increase comp. and site exploration ™ Be willing to mix different embedded links vs. nav links ™ If inf. comp. is a goal, use a preview with embedded links on the home page and reinforce it with a nav. menu Department of Technical Communication, University of Washington

Future Directions ™ Finish analyzing data, open-ended responses, and navigational paths ™ Continue investigating the effect of design variables on users who are reading the Web in order to learn information ™ Examine actual readers of natural Web sites ™ Continue development of WebLab UX ™ Convince usability researchers of the ecological and external validity—and the value of remote user testing ™ Look at design of Web sites for portable devices Department of Technical Communication, University of Washington

For more information http://depts.washington.edu/intres

Department of Technical Communication, University of Washington