Safe Interaction, Connectivity and State

Hui Zhong Advanced Technology & Research, Volvo Group Trucks Technology

Objectives

 Main goal is to safely deliver connected lifestyle functionality while also improving driver attention  Overcome challenges related to driver distraction guidelines and regulation  Meet increased consumer demand for connected lifestyle  Integrate state-of-the-art on-board driver inattention monitoring

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Outline

 SP1 HMI Guidelines and Validation Processes

 SP2 Safe HMI designs  SP3 Safe Connectivity  SP4 Driver Inattention Monitoring

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

SP1 and SP2

 Research questions:

– What is the reliability and validity of performance criteria? – What is the feasibility of testing procedures? – What are the practical implications of performance testing on HMI design?

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

A typical simulator experiment

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

NHTSA test carried out at Volvo Cars

 A full-scale NHTSA test with two test samples, in total 48 participants

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Acceptance criteria

No more than 3 out of 24 of the participants shall fail to meet each of the following criteria:  Acceptance criteria 1 (AC1): For at least 21 of the 24 test participants, no more than 15 percent (rounded up to the next whole number) of each participant’s total number of eye glances away from the forward road scene have durations of greater than 2.0 seconds while performing the testable task one time

 Acceptance criteria 2 (AC2): For at least 21 of the 24 test participants, the mean duration of each participant’s eye glances away from the forward road scene is less than or equal to 2.0 seconds while performing the testable task one time  Acceptance criteria 3 (AC3): For at least 21 of the 24 test participants, the sum of the durations of each individual participant’s eye glances away from the forward road scene is less than or equal to 12.0 seconds while performing the testable task one time Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

What is the reliability and validity of performance criteria? Do two tests yield the same results?  When comparing the test outcomes between two groups, it was found that for 6 of the 10 tested tasks, pass/fail outcomes differed between the two groups Are the criteria well founded?

 Individual variations in glance strategy exist and that these variations seem to have a non-negligible influence on the NHTSA acceptance test criteria Are there better ways to calculate pass/fail criteria?  Group means rather than the proposed frequency count of individual passes and failures, would effect between-group robustness in task compliance testing

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Experiments carried out at Volvo Trucks

1.2) Embedded occlusion vs glasses occlusion part 2 (difficult tasks) 1.3) Effects of simulator view (truck vs car) and display position (low/high) on glance measures 1.4) Occlusion vs eyetracking

Validation methods

1.1) Embedded occlusion vs glasses occlusion part 1 (normal tasks)

2.0) On road study with eyetracking, current FH truck => Expert evaluation of problematic tasks

2.2) Using sound to reduce distraction part 2: speech feedback for handwriting text input

2.3) Text presentation part 1: Effects of font type 2.4) Text presentation part 2: Effects of polarity (Black text on white bg vs. White text on black bg) 2.5) Text input method: Keyboard vs. Handwriting

Safe HMI recommendations, input to standards, guidelines Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Safe HMI designs

2.1) Using sound to reduce distraction part 1: speech/sound feedback for menu navigation

Distraction-reducing auditory display

 Can distraction from in-vehicle interfaces be reduced by adding guiding sound to e.g. menu browsing?  Two sound concepts based on previous research developed: Earcons (musical sounds), Spearcons (speech-based sounds)

Earcons

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Spearcons

Total glance time 9.000

8.032

7.557

Earcon

Baseline

8.000 7.000 6.000 5.000

4.147 4.000 3.000 2.000 1.000 .000 Spearcon

Significant difference between Spearcons and Earcons/Baseline (p < .01)

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

SP1-2 conclusions

Overall the work has

 played an important role in the correspondence with HMI guidelines, e.g. ESoP, JAMA, ISO, NHTSA  evolved design guidelines used within the product development process  provided insights on the feasibility of test methods  provided practical recommendations on safe HMI design

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

SP3 Safe Connectivity No or limited “Driver Distraction” control over use of Apps directly on Smartphone (Use, HMI design, content) OEM Certified Apps

Head-Unit HMI

High control of App (Use, Content, HMI design, Notification, Driver distraction)

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

Safe Connectivity Platform Safe HMI design recommendation

Assessment Tools

App approval process Safety API

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

App contest  Chalmers course: DAT255, Software Engineering Project, 7.5 HEC, HT2014  8-week competition (September to October 2014)  172 students

 19 teams  Challenge: Design, develop & assess Apps for truck drivers running on the Automotive-Grade Platform, & using the SICS project background & inputs

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016

SP4 Driver Inattention Monitoring

 Investigate state of art Driver State Monitoring Camera System which

– monitors the driver’s head pose, gaze direction and eyelid closure – detects driver attention such as eyes on/off road, driver lap, instrument cluster, right/left side, micro sleep and asleep.

Volvo Group Trucks Technology Hui Zhong, SICS project, SAFER Thursday Seminar, January 28, 2016

Sensor Performance Testing

 Test setup

 Analysis

– 13 sticker test

– Uptime

– Re-initialization

– Re-initialization

– Naturalistic driving

– Noise

– Radio task

– Accuracy

– Blinking task

– On/Off road

Volvo Group Trucks Technology Hui Zhong, SICS project, SAFER Thursday Seminar, January 28, 2016

Test Result Summary

 The overall system shows good reliability and accuracy.  Critical to have a good mounting position  Gaze Uptime is very low which makes it unusable in practice  Head Uptime is pretty good, however not sufficient for more than basic on/off road detection

Volvo Group Trucks Technology Hui Zhong, SICS project, SAFER Thursday Seminar, January 28, 2016

Gaze Zone Estimation

 Sensor data set:

– Head pose – Head pose + eye closure – Head pose + gaze data  Calculate the probabilities of the driver’s visual focus at each of a set of defined sectors using Bayesian filtering

 Gaze mapping from head rotation and eye closure using Gaussian processes shows improved gaze zone estimation performance compared to using filtered data Volvo Group Trucks Technology Hui Zhong, SICS project, SAFER Thursday Seminar, January 28, 2016

Thanks for your attention! Hui Zhong Advanced Technology & Research Volvo Group Trucks Technology

Volvo Group Trucks Technology Hui Zhong, SICS project, Resultatkonferens TSAF, September 21, 2016