Robert Collins CSE486, Penn State
Lecture 12: Camera Projection Reading: T&V Section 2.4
Robert Collins CSE486, Penn State
Imaging Geometry W
Object of Interest in World Coordinate System (U,V,W) U
V
Robert Collins CSE486, Penn State
Imaging Geometry Camera Coordinate System (X,Y,Z).
Y X
Z f
• Z is optic axis • Image plane located f units out along optic axis • f is called focal length
Robert Collins CSE486, Penn State
Imaging Geometry W
Y X
y Z
x U
Forward Projection onto image plane. 3D (X,Y,Z) projected to 2D (x,y)
V
Robert Collins CSE486, Penn State
Imaging Geometry W
Y X
y Z
x u
v
U Our image gets digitized into pixel coordinates (u,v)
V
Robert Collins CSE486, Penn State
Imaging Geometry
Image (film) Camera Coordinates Coordinates Y X
y Z
x U
u v
World W Coordinates
Pixel Coordinates
V
Robert Collins CSE486, Penn State
World Coords U V W
Forward Projection Camera Coords X Y Z
Film Coords
Pixel Coords
x y
u v
We want a mathematical model to describe how 3D World points get projected into 2D Pixel coordinates. Our goal: describe this sequence of transformations by a big matrix equation!
Robert Collins CSE486, Penn State
World Coords U V W
Backward Projection Camera Coords X Y Z
Film Coords
Pixel Coords
x y
u v
Note, much of vision concerns trying to derive backward projection equations to recover 3D scene structure from images (via stereo or motion) But first, we have to understand forward projection…
Robert Collins CSE486, Penn State
World Coords U V W
Forward Projection Camera Coords X Y Z
Film Coords x y
Pixel Coords u v
3D-to-2D Projection • perspective projection We will start here in the middle, since we’ve already talked about this when discussing stereo.
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
x
X O
O.Camps, PSU
Z
x
Y
p = (x,y,f)
Y
X
y f
Z
X x f Z Y y f Z
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
x
x
Y
p = (x,y,f)
X
y
Z
f
Z
O
derived via similar triangles rule x
f O.Camps, PSU
Y
X
X x f Z Y y f Z
Z
X
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
x
x
Y
p = (x,y,f)
X
y
Z
f
Z
O
derived via similar triangles rule x
f O.Camps, PSU
Y
X
X x f Z Y y f Z
Z
X
y f
Z
Y
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
x
X O
Z
x
Y
p = (x,y,f)
Y
X
y f
Z
X x f Z Y y f Z
So how do we represent this as a matrix equation? We need to introduce homogeneous coordinates. O.Camps, PSU
Robert Collins CSE486, Penn State
Homogeneous Coordinates
Represent a 2D point (x,y) by a 3D point (x’,y’,z’) by adding a “fictitious” third coordinate. By convention, we specify that given (x’,y’,z’) we can recover the 2D point (x,y) as
x' x z'
y' y z'
Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero k (can be negative as well as positive)
Robert Collins CSE486, Penn State
Perspective Matrix Equation (in Camera Coordinates)
X x f Z Y y f Z
x' f y ' 0 z ' 0
0 f 0
X 0 0 Y 0 0 Z 1 0 1
Robert Collins CSE486, Penn State
World Coords U V W
Forward Projection Camera Coords X Y Z
Film Coords
Pixel Coords
x y
Rigid Transformation (rotation+translation) between world and camera coordinate systems
u v
Robert Collins CSE486, Penn State
World to Camera Transformation PC PW
W
Y X Z
U V
Avoid confusion: Pw and Pc are not two different points. They are the same physical point, described in two different coordinate systems.
Robert Collins CSE486, Penn State
World to Camera Transformation PC PW
W
Y X
R Rotate to align axes
C
Z
U V
Translate by - C (align origins)
P C = R ( PW - C )
Robert Collins CSE486, Penn State
Matrix Form, Homogeneous Coords P C = R ( PW - C ) X Y Z 1
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
1 0 0 0
0 1 0 0
0 cx 0 cy 1 cz 0 1
U V W 1
Robert Collins CSE486, Penn State
Example: Simple Stereo System Z
Y left y camera located at (0,0,0)
(X,Y,Z)
z (
x Tx
,
z
)
y
(
,
)
right camera located at (Tx,0,0)
x
X Left camera located at world origin (0,0,0) and camera axes aligned with world coord axes.
Robert Collins CSE486, Penn State
X Y Z 1
Simple Stereo, Left Camera 1r11 0r21 0r31
r012 r013 r122 r023
r032 r133 0 0 0 1
camera axes aligned with world axes
=
1 0 0 0
0 1 0 0
0 0cx 0 0cy 1 0cz 0 1
U V W 1
located at world position (0,0,0)
Robert Collins CSE486, Penn State
Simple Stereo Projection Equations
Left camera
Robert Collins CSE486, Penn State
Example: Simple Stereo System Z
Y left y camera located at (0,0,0)
(X,Y,Z)
z (
x Tx
,
z
)
y
(
,
x
)
right camera located at (Tx,0,0)
X Right camera located at world location (Tx,0,0) and camera axes aligned with world coord axes.
Robert Collins CSE486, Penn State
X Y Z 1
Simple Stereo, Right Camera 1r11 0r21 0r31
r012 r013 r122 r023
r032 r133 0 0 0 1
camera axes aligned with world axes
=
1 0 0 0
0 1 0 0
0 -T cxx 0 0cy 1 0cz 0 1
U V W 1
located at world position (Tx,0,0)
Robert Collins CSE486, Penn State
Simple Stereo Projection Equations
Left camera
Right camera
Robert Collins CSE486, Penn State
X Y Z 1
Bob’s sure-fire way(s) to figure out the rotation r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
1 0 0 cx 1 about 0 thiscy 0forget while thinking 0 rotations 1 cz 0about 0 0 0 1
U V W 1
PC = R P W This equation says how vectors in the world coordinate system (including the coordinate axes) get transformed into the camera coordinate system.
Robert Collins CSE486, Penn State
X Y Z 1
Figuring out Rotations r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
U V W 1
PC = R P W
what if world x axis (1,0,0) corresponds to camera axis (a,b,c)?
X a Y b Z c 1
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0
1
U 1 V 0 0 W 1
X a Y b Z c 1
ra11 r12 r13 rb21 r22 r23 rc31 r32 r33 0 0 0
1
U 1 V 0 0 W 1
we can immediately write down the first column of R!
Robert Collins CSE486, Penn State
Figuring out Rotations
and likewise with world Y axis and world Z axis... same axis in camera coords
X Y Z 1 world X axis (1,0,0) in camera coords
axis is world coords
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
world Y axis (0,1,0) in camera coords
U V W 1 world Z axis (0,0,1) in camera coords
Robert Collins CSE486, Penn State
Figuring out Rotations
Alternative approach: sometimes it is easier to specify what camera X,Y,or Z axis is in world coordinates. Then do rearrange the equation as follows.
PC = R P W
R-1PC = PW
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0 1
X Y Z 1
RTPC = PW U V W 1
Robert Collins CSE486, Penn State
Figuring out Rotations
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0 1
X Y Z 1
U V W 1
RTPC = PW
what if camera X axis (1,0,0) corresponds to world axis (a,b,c)?
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0
1
X 1 Y 0 0 Z 1
U a V b c W 1
ra11 r21 r31 X U 1 a V rb12 r22 r32 Y 0 b 0 c W rc13 r23 r33 Z 0 0 0 1 1 1 we can immediately write down the first column of RT, (which is the first row of R).
Robert Collins CSE486, Penn State
Figuring out Rotations
and likewise with camera Y axis and camera Z axis... same axis in camera coords
X Y Z 1
axis is world coords
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
U V W 1
camera X axis (1,0,0) in world coords
camera Y axis (0,1,0) in world coords
camera Z axis (0,0,1) in world coords
Robert Collins CSE486, Penn State
Example
y z x
Rtrain
0 0 1 0 -1 0 1 0 0
Rfly
0 -1 0 0 0 1 -1 0 0
Robert Collins CSE486, Penn State
X Y Z 1
Note: External Parameters also often written as R,T r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
R ( PW - C ) = R PW - R C = R PW + T
1 0 0 0
0 1 0 0
r11 r12 r13 tx r21 r22 r23 ty r31 r32 r33 tz 0 0 0 1
0 cx 0 cy 1 cz 0 1
U V W 1
Robert Collins CSE486, Penn State
World Coords U V W
Summary Camera Coords X Y Z
Film Coords x y
Pixel Coords u v
We now know how to transform 3D world coordinate points into camera coords, and then do perspective project to get 2D points in the film plane. Next time: pixel coordinates
