Robert Collins CSE486, Penn State
Robert Collins CSE486, Penn State
Imaging Geometry W
Object of Interest in World Coordinate System (U,V,W)
Lecture 12: Camera Projection
Robert Collins CSE486, Penn State
Y
• Z is optic axis • Image plane located f units out along optic axis • f is called focal length
f
Imaging Geometry W
Camera Coordinate System (X,Y,Z). Z
X
Robert Collins CSE486, Penn State
Imaging Geometry
Y
V
U
Reading: T&V Section 2.4
y Z
X
x V
U Forward Projection onto image plane. 3D (X,Y,Z) projected to 2D (x,y)
Robert Collins CSE486, Penn State
Robert Collins CSE486, Penn State
Imaging Geometry
Camera Coordinates
W Y X
x u
v
Image (film) Coordinates
Y
y Z
Imaging Geometry
X U Our image gets digitized into pixel coordinates (u,v)
V
y Z
x U
u v
World W Coordinates
V
Pixel Coordinates
1
Robert Collins CSE486, Penn State
Forward Projection Camera Coords X Y Z
World Coords U V W
Film Coords
Pixel Coords u v
x y
Film Coords
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Film Coords
Pixel Coords
x y
u v
x y
But first, we have to understand forward projection…
Forward Projection Camera Coords X Y Z
Pixel Coords
Note, much of vision concerns trying to derive backward projection equations to recover 3D scene structure from images (via stereo or motion)
Our goal: describe this sequence of transformations by a big matrix equation!
World Coords U V W
Camera Coords X Y Z
World Coords U V W
We want a mathematical model to describe how 3D World points get projected into 2D Pixel coordinates.
Robert Collins CSE486, Penn State
Backward Projection
u v
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
p = (x,y,f)
Y
x
x
Robert Collins CSE486, Penn State
X
3D-to-2D Projection • perspective projection
Y
X Z
y f
Z
O
X Z Y y f Z
x f
We will start here in the middle, since we’ve already talked about this when discussing stereo. O.Camps, PSU
Basic Perspective Projection
Scene Point
y
x
X
y
Z
f
Z
O
derived via similar triangles rule x
f O.Camps, PSU
X
X Z Y y f Z
x f
Z
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
Y
p = (x,y,f) x
X O
X f O.Camps, PSU
Y
X
y
Z
f
Z
X Z Y y f Z
x f
derived via similar triangles rule x
p = (x,y,f) x
Y
Y
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z)
Image Point
Robert Collins CSE486, Penn State
x
Robert Collins CSE486, Penn State
Z
X
y f
Y
Z
2
Robert Collins CSE486, Penn State
Basic Perspective Projection
Scene Point
Perspective Projection Eqns
P = (X,Y,Z) y
Image Point
p = (x,y,f)
Y
x
x
X O
y
X Z Y y f Z
x f
Y
X Z
f
Z
So how do we represent this as a matrix equation? We need to introduce homogeneous coordinates.
Robert Collins CSE486, Penn State
Homogeneous Coordinates
Represent a 2D point (x,y) by a 3D point (x’,y’,z’) by adding a “fictitious” third coordinate. By convention, we specify that given (x’,y’,z’) we can recover the 2D point (x,y) as
x
x' z'
y
y' z'
Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero k (can be negative as well as positive)
O.Camps, PSU
Robert Collins CSE486, Penn State
Perspective Matrix Equation
Robert Collins CSE486, Penn State
Forward Projection
(in Camera Coordinates)
X Z Y y f Z
x' f y ' 0 z ' 0
x f
X 0 0 Y 0 0 Z 1 0 1
0 f 0
Camera Coords X Y Z
World Coords U V W
Film Coords
Pixel Coords u v
x y
Rigid Transformation (rotation+translation) between world and camera coordinate systems
Robert Collins CSE486, Penn State
World to Camera Transformation PC PW
Robert Collins CSE486, Penn State
World to Camera Transformation PC PW
W
Y
W
Y
X
X Z
U V
Avoid confusion: Pw and Pc are not two different points. They are the same physical point, described in two different coordinate systems.
R Rotate to align axes
C
Z
U V
Translate by - C (align origins)
PC = R ( PW - C )
3
Robert Collins CSE486, Penn State
Matrix Form, Homogeneous Coords
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
0 1 0 0
1 0 0 0
Example: Simple Stereo System Z
Y
PC = R ( PW - C ) X Y Z 1
Robert Collins CSE486, Penn State
0 cx 0 cy 1 cz 0 1
U V W 1
(X,Y,Z)
z
left y camera located at (0,0,0)
( ,
x
z
)
y
( ,
Tx
)
right camera located at (Tx,0,0)
x
X Left camera located at world origin (0,0,0) and camera axes aligned with world coord axes.
Robert Collins CSE486, Penn State
Robert Collins CSE486, Penn State
Simple Stereo Projection Equations
Simple Stereo, Left Camera
X Y Z 1
r111 r012 r0 13 r21 r122 r0 0 23 r31 r032 r1 0 33 0 0 0 1
camera axes aligned with world axes
Left camera
0 0cx 0 0cy 1 0cz 0 1
0 1 0 0
1 0 0 0
U V W 1
located at world position (0,0,0)
=
Robert Collins CSE486, Penn State
Example: Simple Stereo System Z
Y left y camera located at (0,0,0)
(X,Y,Z)
z
z
( , )
x Tx
y
( , )
x
right camera located at (Tx,0,0)
X Right camera located at world location (Tx,0,0) and camera axes aligned with world coord axes.
Robert Collins CSE486, Penn State
X Y Z 1
Simple Stereo, Right Camera r111 r021 0r31
r012 r0 13 r122 r0 23
r032 r1 33 0 0 0 1
camera axes aligned with world axes
1 0 0 0
0 1 0 0
0 -T cxx 0 0cy 1 0cz 0 1
U V W 1
located at world position (Tx,0,0)
=
4
Robert Collins CSE486, Penn State
Simple Stereo Projection Equations
Left camera
Bob’s sure-fire way(s) to figure out the rotation
Robert Collins CSE486, Penn State
X Y Z 1
Right camera
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
1 0 0 cx 1 about 0 thiscy 0forget while thinking 0 rotations 1 cz 0about 0 0 0 1
U V W 1
PC = R PW This equation says how vectors in the world coordinate system (including the coordinate axes) get transformed into the camera coordinate system.
Robert Collins CSE486, Penn State
X Y Z 1
Robert Collins CSE486, Penn State
Figuring out Rotations r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
U V W 1
Figuring out Rotations
and likewise with world Y axis and world Z axis...
PC = R PW
same axis in camera coords
X Y Z 1
what if world x axis (1,0,0) corresponds to camera axis (a,b,c)?
X a Y b Z c 1
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0
1
U 1 V 0 0 W 1
X a Y b Z c 1
ra11 r12 r13 rb21 r22 r23 rc31 r32 r33 0 0 0
1
U 1 V 0 0 W 1
axis is world coords
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
world X axis (1,0,0) in camera coords
world Y axis (0,1,0) in camera coords
U V W 1 world Z axis (0,0,1) in camera coords
we can immediately write down the first column of R!
Robert Collins CSE486, Penn State
Figuring out Rotations
Alternative approach: sometimes it is easier to specify what camera X,Y,or Z axis is in world coordinates. Then do rearrange the equation as follows.
PC = R PW
R-1PC = PW
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0 1
X Y Z 1
RTPC = PW U V W 1
Robert Collins CSE486, Penn State
Figuring out Rotations
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0 1
X Y Z 1
U V W 1
RTPC = PW
what if camera X axis (1,0,0) corresponds to world axis (a,b,c)?
r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0
1
X 1 Y 0 0 Z 1
U a V b W c 1
ra11 r21 r31 X U 1 a V rb12 r22 r32 Y 0 b 0 c W rc13 r23 r33 Z 0 0 0 1 1 1 we can immediately write down the first column of RT, (which is the first row of R).
5
Robert Collins CSE486, Penn State
Robert Collins CSE486, Penn State
Figuring out Rotations
Example
and likewise with camera Y axis and camera Z axis... same axis in camera coords
X Y Z 1
axis is world coords
r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
y
U V W 1
z x
camera X axis (1,0,0) in world coords
Robert Collins CSE486, Penn State
X Y Z 1
0 0 1 Rtrain 0 -1 0 1 0 0
camera Z axis (0,0,1) in world coords
camera Y axis (0,1,0) in world coords
Note: External Parameters also often written as R,T r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0 1
R ( PW - C ) = R PW - R C = R PW + T
1 0 0 0
0 1 0 0
r11 r12 r13 tx r21 r22 r23 ty r31 r32 r33 tz 0 0 0 1
0 cx 0 cy 1 cz 0 1
Robert Collins CSE486, Penn State
U V W 1
World Coords U V W
Rfly
0 -1 0 0 0 1 -1 0 0
Summary Camera Coords X Y Z
Film Coords x y
Pixel Coords u v
We now know how to transform 3D world coordinate points into camera coords, and then do perspective project to get 2D points in the film plane. Next time: pixel coordinates
6