Learning AAC & Eye gaze is hard! Where do we start?

#AATC16

#eyegazeSmartPath @JessWright_AAC @thinksmartbox

Today’s workshop: We will:

• Explore key stages to developing eye gaze skills: from cause and effect through to communication & computer control • Explore resources developed to provide a fun, engaging and structured platform to achieve the key stages & skills • Equip you with the ; a framework & strategies to help you improve the eye gaze skills of any user

Would you like to receive a CPD certificate?

Please fill in your name & email address

Eye gaze • Eye gaze skills are acquired, not inherent

• Much like learning to use a switch, eye gaze is a skill that needs to be taught and developed

How does eye gaze work Pupil

Glint: Reflection of light

Camera Near-Infrared light

Reflection of near-infrared light from the eye into the camera. Eye movement converted into cursor movement.

Where to start?

• Fun, motivating & engaging activities • Enable the user to learn through doing • Empower the user to learn that they are in control & they are manipulating the screen • Not with a calibration • Not with a vocabulary package • But with errorless learning & no-fail activities

Things to think about

• Positioning: the camera needs to see the eyes! How is the user’s head control? Is the user comfortable? Device? • Light sources: other sources; windows, walls, floor or ceiling • Environment: is the location distracting? • Glasses: are they clean? Reflective frames? Coatings? Position? • Eyes: are they fully open, is there anything obstructing the view of the pupil? Looking down?

Calibration

• Don’t start will a calibration! • User first needs to learn the reason, why to look at the device & targets! • Needs to be able to hold gaze on stimulus • Use the calibration from someone else: ideally, same eye colour & ethnicity

What is calibration? Glint: Reflection of light

• Looking at stimulus in different locatio • Pupil moves, the glint does not.

• Camera measures the distance betwee the pupil and glint. Pupil

Calibration Tips • • • • •

Make the stimulus interesting to the user, e.g. favourite picture Get the user to look at the middle of the stimulus Don’t calibrate every time using eye gaze Perform the calibration after engaging the user 1 point calibration gives opportunity to try a calibration sooner!

Hints & Tips for using eye gaze

• Positioning is important with eye gaze • User needs to be comfortable, e.g. reclining chairs • Eye gaze monitor! • Customise activity to make it more engaging or accessible • Users may react to the device, this doesn't mean they want to use their hand over their eyes, it’s just what they are used to • Don’t model touching the screen, use a keyboard or mouse

Position - Grid 3, eye gaze monitor

Unhappy red face - Distance not correct - Either too close or too far away

Happy face Position is good

Happy face with one eye - Position is good - Camera can only see one eye

Position - Alea IntelliGaze eye gaze monitor

Red face with one eye -Distance not correct -Either too close or too far away -Only tracking one eye

Green face with two eyes Position & distance are good

Yellow face with no eyes - Adjust distance - Camera can’t see any eyes

The face gets bigger if the user is too close and smaller if the user is too far away.

A motivating and fun way to get started with eye gaze technology 66 Activities Free trial - https://thinksmartbox.com/product/look-to-learn/

Interactive Learning

A completely new way for people to learn and explore through animated activities

Symbol Communication

Use symbols to represent words and phrases for choosing what you want to say. A wide range of ready-made grid sets.

Computer Control

Control every application on your computer.

workshop

• Work through the 11 Smart Path steps • For each step, I will need a volunteer • Recommended activities for each step are shown on the right hand side of each slide

Smart-path 1.

Screen engagement

3.

Visual scene targeting

2. 4. 5. 6. 7. 8. 9.

Exploration

Fail safe targeting Targeting

Developing a gaze

Choice making and selecting Static grid selection

Modelling and exposure to vocabulary

10. Communication

11. Computer Controls

Activity

1. Screen Engagement

• Look, something instantly happens • Developing understanding of cause & effect and control • Engaging screen • Manipulating screen • Usual for user to just look at one area of the screen • Facilitator involvement adds to user understanding: “Fred the music stopped, can you make some more?”

Magic Mou

Magic Squa

Acti

2. Exploration

• Exploring screen • Looking around the whole screen • Develop understanding; look and things happen • No-fail activities: errorless learning • Engaging screen • Manipulating screen

Xylophone Guitar

Reveal Blocks

Reveal Shapes Scratch

Cannon Graffiti

Acti

3. Explore visual scene targeting

Park Life

• Whole screen interaction

Planet Pog

• Engaging with interactive scenes • Identify targets

• Developing ‘looking to locate’

• No-fail activities: errorless learning

• Facilitator modelling verbal language during activity and user responds to facilitator: • Where is the boy on the bike? • Or user sets the conversation

Classroom

Rumble in the Fun House

Snow Town Lazy Lake

Big City

Ancient Egypt

Wild West

Acti

4. Fail safe targeting

• Multiple targets • Fail safe = user has success • Developing focus on targets • Problem solving: develop knowledge & judgement; looking at a particular target • User responds to facilitator “Can you splat the green alien?”

Xylophone

DJ Star

Spider Web

Balloon Pop

Music Squar Fart Clouds Bottles

Drum Kit

Face Splat

Acti

5. Targeting

• Targeting only item on screen • No longer fail safe • Developing ‘looking to locate’ • Problem solving: develop knowledge & judgement; look at a particular target/area • Facilitation may be required initially to aid success • Reward for looking at target

Custard Pie

Ghost Hunt

Snowball Fi

Funny Face

Creative Col

Acti

6. Developing a gaze (dwell)

• Targeting then maintaining • Highly visual dwell indicator to aid learning • Developing a dwell • Developing ‘searching & selecting’ • Problem solving: develop knowledge & judgement; look at a particular target/dwell time • Facilitator initially provides verbal prompting

Egg

Barn

Rock Band

Happy Birds

Bunny & the Bea Forest Race

Desert Race

7. Choice making & selecting

• Developing ‘search then selecting’

• Making a choice

• Continue to develop dwell; less visual indicator and facilitator prompts

• Using ‘looking to locate’ skill to find chosen choice

• Problem solving: develop knowledge & judgement; look at a particular target & dwell time

Dinner Time

Javelin

Snowman Types

Drums

Opinions

Monster Factory

Paper, Scissor, Stone

8. Static grid selection

• Accessing a static grid • Keeping it fun and engaging • Highly fun animated activity within a static grid • Develop looking from cell to the animation • Making a choice, selection & performing a dwell • Using ‘looking to locate’ & ‘search and select’ skills developed

9. Modelling & exposure to vocabulary

• We learn to communicate through exposure to language • Model vocabulary in relation to the activity • Fun and engaging

10. Communication

• Starts actively selecting vocabulary • Communicating message appropriately • Navigating to communication grid • Can keyword link • Can navigate between grids using categories • Can create a sentence

11. Computer Control

• True empowerment is independently being able to control every aspect of your computer • Developing click and drag • Mouse control; dwell or Zoom to click mouse control • Grid 3 computer control – keyboard shortcuts

Smart-path 1.

Screen engagement

3.

Visual scene targeting

2. 4. 5. 6. 7. 8. 9.

Exploration

Fail safe targeting Targeting

Developing a gaze

Choice making and selecting Static grid selection

Modelling and exposure to vocabulary

10. Communication

11. Computer Controls

Customising

• Improve engagement • Make it more accessible

Customising for access • Think about the layout

• Use Look to Learn analysis to custom the grid to aid the user access

Look to Learn Analyser • Built in analysis tool for every activity • Workbooks to track and encourage progression

Form & Individual Plan

Important to remember

• Make it fun! • Keep pushing the boundaries! • Don’t get stuck doing what they can do! Try new activities & skills.

Questions?

ARATA Stand: 38

@JessWright_AAC @thinksmartbox

Facebook group: Smartbox Community www.thinksmartbox.com

[email protected]