Virtual Reality and Oculus Rift Visual Imaging in the Electronic Age
Donald P. Greenberg November 17, 2015 Lecture #24
Virtual Reality
• A term used to describe a computer generated environment which can simulate the perception of PRESENCE.
Virtual Reality
• A person immersed within this virtual world can manipulate objects, interact with the environment, and explore the virtual world in the same perceptual way as one interacts with the physical world.
Augmented Reality Augmented reality is the interpretation of digital graphical information with live video or the user’s physical environment in real time.
UNC
Human in the Loop • • • •
Abstract Interpretation Viewing a Picture on Television
Cinema Viewing Presence
Microsoft’s Hololens
Magic Leap
Virtual Reality • • • •
Field of view and resolution Color and dynamic range Speed (Refresh and update rates) Bandwidth
Cross Section of Eye & Retina
Rods and Cones
Rods and Cones
Measures of Acuity
detection
min. detectable 0.5” Upper figure adapted from Cornsweet. Visual Perception © Academic Press, 1971.
Two rightmost figures from Graham, Vision and Visual Perception © Wiley, 1966.
resolution
min. resolvable 30”
Snellen letters 30”
localization
vernier 5-7”
Resolution of the Human Eye
Resolution of the Human Eye • Humans can tell visual details at distances larger than 0.3 arc minutes • The Field of View (FOV) of the human eye can be generously estimated as 120 by 90 degrees
Resolution of the Human Eye • (120 degrees x 60 arcminutes / degree x 1 pixel / 0.3 arcminutes) x (90 degrees x 60 arcminutes / degree x 1 pixel / 0.3 arcminutes)
• 431,568,000 pixels; 432 MegaPixels. A 1080p display is 2.1 megapixels.
The Significance of Movement-Sensitive Cells
- David H. Hubbel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.
The Significance of Movement-Sensitive Cells
- David H. Hubbel. EYE, BRAIN, AND VISION, 1988 Scientific American Books, Inc. p. 80.
Lytro Camera
Vergence- Accommodation Conflict
Vergence – Accommodation Conflict
Samsung AMOLED Displays
Standard LCD RGB Display
PenTile Display
Virtual Reality • • • •
Field of view and resolution Color and dynamic range Speed (Refresh and update rates) Bandwidth
Refresh vs. Update Rate • The “refresh rate” is the number of times per second the entire image is drawn • The “update rate” is the number of times per second the image is changed
Refresh Rates • Note that an insufficient refresh rate will cause “flicker.” • For most cases the displays must be refreshed at least 50 cycles/second. • Current virtual reality displays (2015) operate at 90 Hz.
Update Rates • The human eye can process approximately 10-12 separate images per second. To simulate motion, update rates must be greater than this.
• The movie industry has historically used 24 fps and the television industry 30 fps.
• Recently, James Cameron (Avatar) and Peter Jackson (The Hobbit) have been filming at 48 fps.
• Since in VR/AR continuous motion is paramount, then rendering must be in (1000/24), about 40 milliseconds or less.
Cones interconnect in the retina, eventually leading to opponent-type signals. Roy S. Berns. “Billmeyer and Saltzman’s Principles of Color Technology, 3rd Ed. 2000, John Wiley & Sons, Inc. p. 16.
Color-Receptive Fields Channel
Combination
Resolution
black - white
M+L
very high
Green - red
M–L
High
yellow - blue
M+L–S
low
Full Color Image
YIQ http://en.wikipedia.org/wiki/YIQ
Luminance Image (Y)
YIQ http://en.wikipedia.org/wiki/YIQ
Color Addition (Red/Green Axis)(I)
YIQ http://en.wikipedia.org/wiki/YIQ
Color Addition (Yellow/Blue Axis)(Q)
YIQ http://en.wikipedia.org/wiki/YIQ
Research on Foveated Displays
Truck Art
DORSEY 1993
DORSEY 1993
DORSEY 1993
Oculus Rift
Oculus Rift
Components
Accelerometers and logic board Lenses Flat 1080P AMOLED Display
Oculus Rift DK2
2014
Steam VR
2015
Crescent Bay
2015
Virtual Reality • • • • •
Tracking and sensor response Algorithmic computational time
Data capture Heat and power (de-tethering) Economy
Oculus Rift
Angular Rotation
Distortion Strategy
Oculus Rift
Distortion Strategy
Oculus Rift
Distorted Image
Oculus Rift
Distorted Image
1 2 3 Catch
Autodesk
Project Beyond
Samsung 2015
Project Beyond
Samsung 2015
Google’s Recording Rig
2015
End..
