3D vision overview Cameras, calibration and projective geometry Multiple camera stereo Single camera stereo and the fundamental matrix Shape from X – photometric stereo
Sections covered: Sonka 9 Additional Material Forsyth and Ponce http://research.microsoft.com/en-us/um/people/szeliski/Book/ Augmented Reality 2010
Wednesday 9 February 2011
2
2
How do we measure Depth in images? Monocular Cues Depth from motion - TTC Perspective Depth from Focus Occlusion Texture Gradient Illumination Gradient (shading/shadows) … 3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
3
3
How do we measure Depth in images? Stereo (multiple viewpoint) Cues Triangulation/Disparity Only works to about 3M in humans Motion Parallax Accommodation and Convergence Eyes actively change lens and gaze directions to keep objects in focus Robotics need active stereo head 3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
4
4
Monocular vs. stereo Fooling the monocular cues Hollow face illusion video
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
5
5
NASA lecture on stereo
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
6
6
Anthropomorphic Design
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
7
7
Multiple view stereo from a single camera
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
8
8
3D vision at work in movies
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
9
9
3D at work in simulation Surgical Simulation
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
10
10
3D Vision Many Applications need 3D Robotics Guidance Medical Modeling Modeling for the Games & Movie Industry Augmented Reality Movies New Games / New Apps
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
11
11
The challenge of 3D 2D images of a 3D world Extracting depth information Interpreting depth information
Developments in 3D vision theory Marr 1982: 3D reconstruction “From an image (or a series of images) of a scene, derive an accurate three-dimensional geometric description of the scene and quantitatively determine the properties of the object in the scene” Aloimonos and Shulman 1989: + understanding “images of a moving or stationary object or scene taken by a monocular or polynocular moving or stationary observer, to understand the object or the scene and its three-dimensional properties” Wechsler 1990: + interacting in the world “The visual system casts most visual tasks as minimization problems and solves them using distributed computation and enforcing non-accidental, natural constraints.”
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
13
13
Aloimonos 1993: + comparison with natural vision systems What principles enable us to (i) understand existing vision systems, (ii) give machines the ability to see… Hartley & Zisserman 2000: multiple view geometry in computer vision, a mathematical approach for extracting scene geometry from single and multiple images using the constrains imposed by perspective projection - accuracy via bundle adjustment - used in move matching software like Boujou (see 2d3.com) Davidson 2003: visual SLAM Building a 3D model in realtime while moving in the world. This is important for mobile robotics and AR
3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
14
14
Sub-problems & approaches Three intertwined problems: Feature observability Representation Interpretation Ambiguity and Optical Illusions http://www.lhup.edu/~dsimanek/3d/illus2.htm
Two main approaches
Reconstruction approach (bottom-up) Image data driven
Recognition approach (top-down) Model driven
3D models? Using multiple 2D models 3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
15
15
Other approaches – Active perception Currently two schools trying to explain the vision mechanism Extracting geometric models bottom-up View based models – used for interpretation Static vs. Active perception Static cameras with fixed characteristics Active vision systems Observer is active &Controls its visual sensors Makes ill-posed problems tractable Shape from X… 3D Vision
Wednesday 9 February 2011
Computer Vision Lecture 2
16
16
Other approaches – Qualitative and Purposive Qualitative Vision
Qualitative description of objects Only represent what is needed Viewpoint independence Different scales
Purposive vision Identify the goal Just compute the required information E.g. Collision avoidance
