3D Computer Vision II
Reminder
Camera Models
Nassir Navab" based on a course given at UNC by Marc Pollefeys & the book “Multiple View Geometry” by Hartley & Zisserman"
October 27, 2009"
Outline – Camera Models"
•
Geometric Parameters of a Finite Camera"
•
Projective Camera Model"
Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"
Decomposition of Camera Matrix"
•
Cameras at Infinity"
•
Other Camera Models (Pushbroom and Line Cameras)"
3D Computer Vision II - Camera Models"
2"
Pinhole Camera Model"
• • •
Mapping between 3D world and 2D image" Central projection" Models are described in matrices with particular properties"
3D Computer Vision II - Camera Models"
3"
Homogeneous Coordinates"
3D Computer Vision II - Camera Models"
4"
Central Projection"
3D Computer Vision II - Camera Models"
5"
Principal Point Offset"
principal point (perpendicular intersection point of principal axis and image plane)
where are the coordinates of the principal point 3D Computer Vision II - Camera Models"
6"
Principal Point Offset"
where
3D Computer Vision II - Camera Models"
is called camera calibration matrix
7"
Camera Rotation and Translation" Inhomogeneous coordinates
where
represents the point in world coordinates represents the same point in camera coordinates represents the coordinates of the camera origin in the world coordinate frame
3D Computer Vision II - Camera Models"
8"
Camera Rotation and Translation" Homogeneous coordinates
projection to image plane from camera coordinates projection to image plane from world coordinates
3D Computer Vision II - Camera Models"
9"
Extrinsic and Intrinsic Parameters"
where projection matrix of a general pinhole camera with 9 DOF
intrinsic camera parameters with 3 DOF extrinsic camera parameters with each 3 DOF (camera orientation, position in world coordinates) 3D Computer Vision II - Camera Models"
10"
Camera Rotation and Translation" No explicit camera center
where from
3D Computer Vision II - Camera Models"
11"
CCD Cameras – Non-Square Pixels" number of pixels per unit distance
4 DOF
10 DOF 3D Computer Vision II - Camera Models"
12"
Skew Parameter" skew parameter
5 DOF
finite projective camera with 11 DOF
3D Computer Vision II - Camera Models"
13"
Finite Projective Camera – Summary"
projection matrix 11 DOF (5+3+3)
non-singular
3D Computer Vision II - Camera Models"
14"
Finite Projective Camera – Decomposition of P" non-singular 3x3 matrix (8 DOF)
decompose projection matrix P in K,R,C
QR matrix decomposition
3D Computer Vision II - Camera Models"
15"
Finite Projective Camera – Summary"
where
•
Camera matrices P are identical with the set of homogeneous 3x4 matrices for which the left 3x3 sub-matrix is non-singular"
• •
If rank(P) = 3, but rank(M) < 3, then camera at infinity" if rank(P) < 3 the matrix mapping will be a line or a point and not a plane (not a 2D image)
3D Computer Vision II - Camera Models"
16"
Outline – Camera Models"
•
Geometric Parameters of a Finite Camera"
•
Projective Camera Model"
Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"
Decomposition of Camera Matrix"
•
Cameras at Infinity"
•
Other Camera Models (Pushbroom and Line Cameras)"
3D Computer Vision II - Camera Models"
17"
Camera Anatomy"
• • • • • •
Camera center" Column vectors" Principal plane" Axis plane" Principal point" Principal ray"
3D Computer Vision II - Camera Models"
18"
Camera Center" P has a 1D null-space we will prove that the 4-vector C is the camera center points on a line through A and C since All 3D points on the line are mapped on the same 2D image point, and thus the line is a ray through the camera center Finite cameras: Infinite cameras: 3D Computer Vision II - Camera Models"
19"
Column Vectors"
Column vectors are the image points which project the axis directions (X,Y,Z) and the origin Example for the image of the y-axis is the image of the world origin 3D Computer Vision II - Camera Models"
20"
Row Vectors"
Represent geometrically particular world planes.
row vectors
column vectors
3D Computer Vision II - Camera Models"
21"
Row Vectors of the Projection Matrix" p1 is defined by the camera center and the line x=0 on the image. p2 is defined by the camera center and the line y=0 on the image.
Example p2
respectively for p1 3D Computer Vision II - Camera Models"
22"
Principal Plane" Plane through camera center and parallel to the image plane.
points X are imaged on the line at infinity if X is on the principle plane especially 3D Computer Vision II - Camera Models"
23"
Principal Point" The line through camera center and perpendicular to principal plane is the principal axis. The intersection of the principal axis with the image plane is the principal point. normal direction to the principal plane principal point where and 3D Computer Vision II - Camera Models"
third row of M
Principal Axis Vector" Ambiguity that principal axis points towards the front of the camera (positive direction)
towards the front of the camera direction unaffected by scaling since
3D Computer Vision II - Camera Models"
25"
Forward Projection" Maps a point in space on the image plane
Vanishing points
Only M affects the projection of vanishing points 3D Computer Vision II - Camera Models"
26"
Back-Projection to Rays" Points on the reconstructed ray camera center C
(pseudo-inverse) Ray is the line formed by those two points intersection of the ray with the plane at infinity
3D Computer Vision II - Camera Models"
27"
Depth of Points"
w can be interpreted as the dot product of the ray CX with the principal ray direction
(PC=0) If
,
then m3 is a unit vector pointing in positive axis direction Suppose
3D Computer Vision II - Camera Models"
. Then
28"
Depth of Points: Examples"
3D Computer Vision II - Camera Models"
29"
Outline – Camera Models"
•
Geometric Parameters of a Finite Camera"
•
Projective Camera Model"
Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"
Decomposition of Camera Matrix"
•
Cameras at Infinity"
•
Other Camera Models (Pushbroom and Line Cameras)"
3D Computer Vision II - Camera Models"
30"
Camera Matrix Decomposition" Finding the camera center C
numerically: find right null-space by SVD of P
Algebraically:
where 3D Computer Vision II - Camera Models"
31"
Camera Matrix Decomposition" Finding the camera center C Any plane π going through C will be a linear combination of the three planes defined by the rows of P. Therefore:
where 3D Computer Vision II - Camera Models"
32"
Camera Matrix Decomposition" Finding the camera orientation and internal parameters
using RQ decomposition
Decompose
=(
Q
) =
R -1
R -1 Q -1
Ambiguity removed by enforcing positive diagonal entries 3D Computer Vision II - Camera Models"
33"
When is Skew Non-zero?" arctan(1/s) 1
γ
for CCD/CMOS, always s=0 Image from image, s≠0 possible (non coinciding principal axis) resulting camera: where H is a 3x3 homography 3D Computer Vision II - Camera Models"
34"
Euclidean vs. Projective Spaces" General projective interpretation
• Meaningful decomposition in K,R,t requires Euclidean image and space • Camera center is still valid in projective space • Principal plane requires affine image and space • Principal ray requires affine image and Euclidean space 3D Computer Vision II - Camera Models"
35"
Outline – Camera Models"
•
Geometric Parameters of a Finite Camera"
•
Projective Camera Model"
Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"
Decomposition of Camera Matrix"
•
Cameras at Infinity"
•
Other Camera Models (Pushbroom and Line Cameras)"
3D Computer Vision II - Camera Models"
36"
Cameras at Infinity"
Cameras with their center lying at infinity
M is singular
Two types of cameras at infinity: Affine and non-affine cameras
3D Computer Vision II - Camera Models"
37"
Affine Cameras"
Definition: An affine camera is a camera with a camera matrix P in which the last row p3T is of the form (0,0,0,1)T .
Points at infinity are mapped to points at infinity 3D Computer Vision II - Camera Models"
38"
Affine Cameras"
3D Computer Vision II - Camera Models"
39"
Affine Cameras"
distance of the world origin from the camera center in direction of the principal ray
modifying p34 corresponds to moving along principal ray 3D Computer Vision II - Camera Models"
40"
Affine Cameras" Combine tracking back and zooming magnification factor k=dt/d0 remains fixed
3D Computer Vision II - Camera Models"
41"
Error in Employing Affine Cameras" point on plane parallel with principal plane and through origin, then
general points (not on the parallel plane) with distance from the plane
3D Computer Vision II - Camera Models"
42"
Error in Employing Affine Cameras"
3D Computer Vision II - Camera Models"
Affine Imaging Conditions"
Approximation should only cause small error 1. Δ much smaller than d0 2. points close to principal ray (i.e. small field of view)
3D Computer Vision II - Camera Models"
44"
Decomposition of P ∞"
absorb d0 in K2x2
alternatives, because 8dof (3+3+2), not more 3D Computer Vision II - Camera Models"
45"
Summary of Parallel Projections"
canonical representation
calibration matrix
principal point is not defined
3D Computer Vision II - Camera Models"
46"
Hierarchy of Affine Cameras"
dropping the z-coordinate
orthographic projection
(5dof)
3D Computer Vision II - Camera Models"
47"
Hierarchy of Affine Cameras"
scaled orthographic projection
(6dof)
3D Computer Vision II - Camera Models"
48"
Hierarchy of Affine Cameras"
weak perspective projection
(7dof)
3D Computer Vision II - Camera Models"
49"
Hierarchy of Affine Cameras" Affine camera
(8dof) full generality of an affine camera
Affine camera is a projective camera with principal plane at infinity Affine camera maps parallel world lines to parallel image lines No center of projection, but direction of projection PAD=0 3D Computer Vision II - Camera Models"
50"
General Camera at Infinity"
M is singular, but last row not zero
Camera center is on plane at infinity Principal plane is not plane at infinity Images of points at infinity are in general not mapped to infinity on the image plane
3D Computer Vision II - Camera Models"
Outline – Camera Models"
•
Geometric Parameters of a Finite Camera"
•
Projective Camera Model"
Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"
Decomposition of Camera Matrix"
•
Cameras at Infinity"
•
Other Camera Models (Pushbroom and Line Cameras)"
3D Computer Vision II - Camera Models"
52"
Pushbroom Cameras"
(11dof)
Straight lines are not mapped to straight lines! (otherwise it would be a projective camera) 3D Computer Vision II - Camera Models"
53"
Line Cameras" (5dof)
Null-space PC=0 yields camera center Also decomposition
3D Computer Vision II - Camera Models"
54"
Summary Camera Models" • • •
Photometric and radiometric properties of a camera" Geometric parameters of a finite camera" Projective cameras" – Camera anatomy (camera center, principle plane, principle point, and principle axis)" – Camera matrix decomposition (camera center, orientation, and intrinsic parameter"
•
Cameras at infinity" – Affine cameras" – Non-affine cameras"
•
Alternative models (pushbroom cameras, line cameras)"
3D Computer Vision II - Camera Models"
55"
Literature on Camera Models" • • • • •
Chapter 6 in R. Hartley and A. Zisserman, “Multiple View Geometry”, 2nd edition, Cambridge University Press, 2003. " Chapter 3 in O. Faugeras, “Three-dimensional Computer Vision”, MIT Press, 1993." Chapter 2 in E. Trucco and A. Verri, “Introductory Techniques for 3-D Computer Vision”, Prentice Hall, 1998." H. Gernsheim, “The Origins of Photography”, Thames and Hudson, 1982." A. Shashua. “Geometry and Photometry in 3D Visual Recognition”, Ph.D. Thesis, MIT, Nov. 1992. AITR-1401. "
3D Computer Vision II - Camera Models"
56"
