Motion Sensing Technology for Wearable and VR Applications Shunnan Liu GM, CyweeMotion Inc.
© 2016 CyweeMotion Ltd. All rights reserved.
1
Motion Sensing Technology Evolution Wii(1st), iPhone 2/3G
Acc Only (3 axis)
Delay, Force Interfusion, Pitch singularity
Air Mouse
Acc + Gyro (6 axis)
Gyro temperature drift issue, Accumulated error and relative position issue
iPhone 3GS
Acc + Mag (6 axis)
Delay, Magnetic disturbance
Current Android Smartphone / iPhone
Acc + Gyro + Mag (9 axis)
Provide real-time & precise motion sensing with low latency orientation and absolute position output.
© 2016 CyweeMotion Ltd. All rights reserved.
2
Motion Sensing Technology Evolution Wii(1st), iPhone 2/3G
Acc Only (3 axis)
Delay, Force Interfusion, Pitch singularity
Air Mouse
Acc + Gyro (6 axis)
Gyro temperature drift issue, Accumulated error and relative position issue
iPhone 3GS
Acc + Mag (6 axis)
Delay, Magnetic disturbance
Current Android Smartphone / iPhone
Acc + Gyro + Mag (9 axis)
Provide real-time & precise motion sensing with low latency orientation and absolute position output.
© 2016 CyweeMotion Ltd. All rights reserved.
3
The Fact of Motion Processing • Without proper interpretation, motion sensor data is hard to read • Motion Processing Algorithm is all about complicated math calculation • Motion Processing must be always-on @ background © 2016 CyweeMotion Ltd. All rights reserved.
4
Always-On Sensor Hub design Application processor (AP)
Application processor (AP)
Android Applications
Android Applications
Application framework
Application framework
Sensor HAL + sensor fusion & algorithm
Sensor HAL
Kernel
Kernel I2C/SPI
Sensor hub (MCU/DSP)
I2C
Motion algorithms I2C
Accelero sensor
Gyroscope sensor
Compass sensor
Pressure sensor
Before => sensor fusion running on AP • High power consumption (120 to 190mA) • Effort to maintain sensor fusion in Android user space • Motion sensor vendor dependent • Repeated software porting effort for new motion sensors
© 2016 CyweeMotion Ltd. All rights reserved.
Accelero sensor
Gyroscope sensor
Compass sensor
Pressure sensor
Now => dedicated MCU/DSP to run the sensor fusion • Low power consumption (~1mA) for Always-On
• 9-axis fusion algorithm sensor independent • Real time environment for real time algorithms
• Host OS independent • Host offloading
5
Motion Processing Hub Applications in Anywhere
Activity Detection Motion Tracking
© 2016 CyweeMotion Ltd. All rights reserved.
6
© 2016 CyweeMotion Ltd. All rights reserved.
7
Wearable market overview Mainstream
Fast growing
Source: Statista, Feb. 2016
© 2016 CyweeMotion Ltd. All rights reserved.
8
Top 5 Applications for Smart Wearables • • • • •
Activity Recognition Sports tracking Sleep Monitoring Health Monitoring Notification & control
© 2016 CyweeMotion Ltd. All rights reserved.
9
Highlights for Activity Recognition • • • • •
Activity Recognition will be the key focus in Android It should be activated 7/24 at very low power manner Automatic detection! Must be accurate! Keep increasing in the future
© 2016 CyweeMotion Ltd. All rights reserved.
10
VR Market Estimation • Gaming to lead the first-stage VR cycle. • After Gaming application the Business, Entertainment will be coming soon.
More Application Coming
Source: Yuanta Investment Consulting
© 2016 CyweeMotion Ltd. All rights reserved.
11
Virtual comes to Real! • VR is set to kick-off this year as big brands showcase their revamped VR handset in CES 2016 – PC/Game console based – Smartphone based
© 2016 CyweeMotion Ltd. All rights reserved.
12
CyWeeMotion Sensor Processing Solution
© 2016 CyweeMotion Ltd. All rights reserved.
13
Company Fact Sheet Soft Bank China Venture Capital
Industrial Technology Research Institute (ITRI)
CyweeMotion Quick Facts
Founded in 2007 by ITRI and SBCVC as a joint technology/business alliance Motion processing application/algorithm development pioneered by ITRI in 2003 Number of employees: 40 Number of patents held: 144 (115 granted)
© 2016 CyweeMotion Ltd. All rights reserved.
14
Core Technology Overview • Our motion technology powers on mobile/wearable devices and platform services. Core Technology
Key Sensor Fusion Applied Application Processor(AP)
Sensor Fusion Algorithm
MCU/DSP(algorithm)
(Sensor Fusion Hub)
(Smart Phone)
(Wearable Device/ Accessory)
(Motion Platform)
Sensor Fusion Algorithm
© 2016 CyweeMotion Ltd. All rights reserved.
15
Motion Solution for Mobile/Wearable Devices Sensor Fusion & Motion Detection Sensor HUB
9/10-axis Sensor Fusion Library
Gestures Recognition Libraries
Application Processor(AP)
MCU/DSP(algorithm)
Context Awareness
© 2016 CyweeMotion Ltd. All rights reserved.
Motion Tracking
Navigation PDR for mobiles
16
Human Activity Motion Engine •
Human Activity Motion Engine will make it possible for you to enjoy amazing fitness programs.
Smart Phone & Wearable Device
Human Activity Motion Engine © 2016 CyweeMotion Ltd. All rights reserved.
17
Current VR Problem • Latency & Judder effect – Judder is caused by the mismatch of video frame and head movement, improve fusion accuracy and response will improve it – Higher sensor ODR can remove judder? The answer is “Not exactly”
• Looks like a dog? – Current VR relies on IR sensor to track your position, accurate motion sensing algorithm can make it better
© 2016 CyweeMotion Ltd. All rights reserved.
18
Virtual Reality Gaming Motion Engine •
VR Motion Tracking Engine will lead you into various virtual reality scenarios.
Smart Phone & Wearable Device
VR Motion Tracking Engine © 2016 CyweeMotion Ltd. All rights reserved.
Virtual Reality Platform
19
CyweeMotion Fusion Solution for VR • Sensor ODR (100Hz) defined in Android Mobile is not sufficient for VR application • Higher sensing rate will gain better accuracy & performance – Angular error @ 100Hz is ±0.7° – Angular error @ 400Hz is ±0.46°, 34% off 慢速 (1 rps, 10 revolution) 中速 (3 rps, 45 revolution) 100 Hz 400 Hz 100 Hz 400 Hz 1 4.620 -3.410 1 29.960 -15.850 2 4.380 -4.390 2 32.030 -23.260 3 -4.200 5.200 3 33.000 -23.430 4 4.350 -3.370 4 28.190 -18.120 5 5.650 -4.830 5 33.710 -22.950 Avg 4.640 4.240 Avg 31.378 20.722 0.46 0.42 0.7 0.46
© 2016 CyweeMotion Ltd. All rights reserved.
20
CyweeMotion fulfills Google’s VR requirement Features CyweeMotion Accelerometer CyweeMotion Magnetic field sensor
CyweeMotion Magnetic field uncalibrated CyweeMotion Gyroscope
Description An accelerometer sensor reports the acceleration of the device along the 3 sensor axes. A magnetic field sensor (also known as magnetometer) reports the ambient magnetic field, as measured along the 3 sensor axes.
Sensor used 加速度計
磁場感測器
An uncalibrated magnetic field sensor reports the ambient magnetic field together with a 磁場感測器 (未校正) hard iron calibration estimate. A gyroscope sensor reports the rate of rotation of the device around the 3 sensor axes.
陀螺儀
CyweeMotion Gyroscope uncalibrated
An uncalibrated gyroscope reports the rate of rotation around the sensor axes without 陀螺儀 (未校正) applying bias compensation to them, along with a bias estimate.
CyweeMotion Orientation
An orientation sensor reports the attitude of the device.
© 2016 CyweeMotion Ltd. All rights reserved.
手機姿態 (角度表示)
Data format data[0] = X軸加速度 (m/s^2) data[1] = Y軸加速度 (m/s^2) data[2] = Z軸加速度 (m/s^2) data[0] = X軸磁場強度 (uT) data[1] = Y軸磁場強度 (uT) data[2] = Z軸磁場強度 (uT) data[0] = X軸磁場強度(未校正) (uT) data[1] = Y軸磁場強度(未校正) (uT) data[2] = Z軸磁場強度(未校正) (uT) data[3] = X軸磁場強度偏移量 (uT) data[4] = Y軸磁場強度偏移量 (uT) data[5] = Z軸磁場強度偏移量 (uT) data[0] = X軸角速度 (rad/s) data[1] = Y軸角速度 (rad/s) data[2] = Z軸角速度 (rad/s) data[0] = X軸角速度(未校正) (rad/s) data[1] = Y軸角速度(未校正) (rad/s) data[2] = Z軸角速度(未校正) (rad/s) data[3] = X軸角速度偏移量 (rad/s) data[4] = Y軸角速度偏移量 (rad/s) data[5] = Z軸角速度偏移量 (rad/s) data[0] = azimuth (degree) data[1] = pitch (degree) data[2] = roll (degree)
21
Advanced Sport Tracking Motion Engine •
Advanced Sport Tracking Motion Engine will help you become more professional in different sports.
Smart Phone & Wearable Device
Advanced Sport Tracking Motion Engine © 2016 CyweeMotion Ltd. All rights reserved.
22
Navigation PDR Motion Engine •
Navigation Engine will improve your experience with better location-based services.
Smart Phone & Wearable Device
Navigation Engine
© 2016 CyweeMotion Ltd. All rights reserved.
Navigation Related Service Platform 23
CyweeMotion Sensor Hub Approaches Standard standalone Sensor Hub Higher System Flexibility
Specific Sensor Hub Support Voice trigger or location based technology
Simplified Sensor Hub Less chip count, PCB space (lower BOM cost)
Combo MEMS Sensor Hub Integrate inertial sensor for mid/entry level product
Mobile Application Processor
Mobile Application Processor
Mobile Application Processor
Mobile Application Processor
Cortex - M4 MCU
DSP or GPS SoC
Cortex-M4 or DSP
Motion MEMS Sensor Cortex - M4
Sensors (A+G+M +Pressure+Prox…)
Sensors (A+G+M +Pressure+Prox…)
Sensors (A+G+M +Pressure+Prox…)
(松果) © 2016 CyweeMotion Ltd. All rights reserved.
Motion MEMS Sensor Partners
24
CyweeMotion Sensor Hub Framework
© 2016 CyweeMotion Ltd. All rights reserved.
25
We’re the leader of Motion Fusion
© 2016 CyweeMotion Ltd. All rights reserved.
26
MEMS Sensor Partners
© 2016 CyweeMotion Ltd. All rights reserved.
27
Supported Sensor List Sensor P/N
Sensor Type
LSM330DLC
3 axis accelerometer + 3 axis gyroscope
LSM330TR
3 axis accelerometer + 3 axis gyroscope
L3GD20TR
3 axis gyroscope
LSM303D
3 axis accelerometer + 3 axis magnetometer
LSM303DLHC
3 axis accelerometer + 3 axis magnetometer
LSM6DS0
3 axis accelerometer + 3 axis gyroscope
LSM6DS3
3 axis accelerometer + 3 axis gyroscope
LPS331AP
pressure + temperature
LPS25
pressure + temperature
HT221
Humidity + temperature
AMI306
3 axis magnetometer
BMI055
3 axis accelerometer + 3 axis gyroscope
BMI160
3 axis accelerometer + 3 axis gyroscope
BMM150
3 axis magnetometer
BMP280 / BMP180
pressure + temperature
BMX055
3 axis accelerometer + 3 axis gyroscope +3 axis magnetometer
BME280
pressure + humidity + temperature
©©2016 CyweeMotion Ltd. All rights reserved. 2014 CyWee Motion Inc. All rights reserved.
28
Confidential.
28
Supported Sensor List(cont.) Sensor P/N
Sensor Type
YAS533
3 axis magnetometer
YAS530B
3 axis magnetometer
YAS532B
3 axis magnetometer
AK8963
3 axis magnetometer
AK09911
3 axis magnetometer
AK09912
3 axis magnetometer
APDS9960
Proximity + ambient light + gesture
CM36283A3OP
Proximity + ambient light
CM36682M3OE
Proximity + ambient light
CM36671
Proximity + ambient light
MPU6500
3 axis accelerometer + 3 axis gyroscope
MPU6515
3 axis accelerometer + 3 axis gyroscope
MPU6880
3 axis accelerometer + 3 axis gyroscope
TMD2771
Proximity + ambient light
TMD27723
Proximity + ambient light
HSCDTD801
3 axis magnetometer
PAH8001EI-QG
Heart rate detection function (HRD)
AS7000
Heart rate detection
©©2016 CyweeMotion Ltd. All rights reserved. 2014 CyWee Motion Inc. All rights reserved.
29
Confidential.
29
Thank You. Max Lin, VP of Sales
[email protected] +886-919-221-013
© 2016 CyweeMotion Ltd. All rights reserved.
Shun-Nan Liu, GM
[email protected] +886-928-068-853
30