Imaging Geometry. Lecture 12: Camera Projection. Imaging Geometry. Imaging Geometry. Imaging Geometry. Imaging Geometry

Robert Collins CSE486, Penn State Robert Collins CSE486, Penn State Imaging Geometry W Object of Interest in World Coordinate System (U,V,W) Lectu...
Author: Randell Kelley
7 downloads 1 Views 915KB Size
Robert Collins CSE486, Penn State

Robert Collins CSE486, Penn State

Imaging Geometry W

Object of Interest in World Coordinate System (U,V,W)

Lecture 12: Camera Projection

Robert Collins CSE486, Penn State

Y

• Z is optic axis • Image plane located f units out along optic axis • f is called focal length

f

Imaging Geometry W

Camera Coordinate System (X,Y,Z). Z

X

Robert Collins CSE486, Penn State

Imaging Geometry

Y

V

U

Reading: T&V Section 2.4

y Z

X

x V

U Forward Projection onto image plane. 3D (X,Y,Z) projected to 2D (x,y)

Robert Collins CSE486, Penn State

Robert Collins CSE486, Penn State

Imaging Geometry

Camera Coordinates

W Y X

x u

v

Image (film) Coordinates

Y

y Z

Imaging Geometry

X U Our image gets digitized into pixel coordinates (u,v)

V

y Z

x U

u v

World W Coordinates

V

Pixel Coordinates

1

Robert Collins CSE486, Penn State

Forward Projection Camera Coords X Y Z

World Coords U V W

Film Coords

Pixel Coords u v

x y

Film Coords

Robert Collins CSE486, Penn State

Basic Perspective Projection

Scene Point

Film Coords

Pixel Coords

x y

u v

x y

But first, we have to understand forward projection…

Forward Projection Camera Coords X Y Z

Pixel Coords

Note, much of vision concerns trying to derive backward projection equations to recover 3D scene structure from images (via stereo or motion)

Our goal: describe this sequence of transformations by a big matrix equation!

World Coords U V W

Camera Coords X Y Z

World Coords U V W

We want a mathematical model to describe how 3D World points get projected into 2D Pixel coordinates.

Robert Collins CSE486, Penn State

Backward Projection

u v

Perspective Projection Eqns

P = (X,Y,Z) y

Image Point

p = (x,y,f)

Y

x

x

Robert Collins CSE486, Penn State

X

3D-to-2D Projection • perspective projection

Y

X Z

y f

Z

O

X Z Y y f Z

x f

We will start here in the middle, since we’ve already talked about this when discussing stereo. O.Camps, PSU

Basic Perspective Projection

Scene Point

y

x

X

y

Z

f

Z

O

derived via similar triangles rule x

f O.Camps, PSU

X

X Z Y y f Z

x f

Z

Perspective Projection Eqns

P = (X,Y,Z) y

Image Point

Y

p = (x,y,f) x

X O

X f O.Camps, PSU

Y

X

y

Z

f

Z

X Z Y y f Z

x f

derived via similar triangles rule x

p = (x,y,f) x

Y

Y

Basic Perspective Projection

Scene Point

Perspective Projection Eqns

P = (X,Y,Z)

Image Point

Robert Collins CSE486, Penn State

x

Robert Collins CSE486, Penn State

Z

X

y f

Y

Z

2

Robert Collins CSE486, Penn State

Basic Perspective Projection

Scene Point

Perspective Projection Eqns

P = (X,Y,Z) y

Image Point

p = (x,y,f)

Y

x

x

X O

y

X Z Y y f Z

x f

Y

X Z

f

Z

So how do we represent this as a matrix equation? We need to introduce homogeneous coordinates.

Robert Collins CSE486, Penn State

Homogeneous Coordinates

Represent a 2D point (x,y) by a 3D point (x’,y’,z’) by adding a “fictitious” third coordinate. By convention, we specify that given (x’,y’,z’) we can recover the 2D point (x,y) as

x

x' z'

y

y' z'

Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero k (can be negative as well as positive)

O.Camps, PSU

Robert Collins CSE486, Penn State

Perspective Matrix Equation

Robert Collins CSE486, Penn State

Forward Projection

(in Camera Coordinates)

X Z Y y f Z

 x'   f  y '   0     z '   0

x f

X  0 0   Y 0 0   Z  1 0   1

0 f 0

Camera Coords X Y Z

World Coords U V W

Film Coords

Pixel Coords u v

x y

Rigid Transformation (rotation+translation) between world and camera coordinate systems

Robert Collins CSE486, Penn State

World to Camera Transformation PC PW

Robert Collins CSE486, Penn State

World to Camera Transformation PC PW

W

Y

W

Y

X

X Z

U V

Avoid confusion: Pw and Pc are not two different points. They are the same physical point, described in two different coordinate systems.

R Rotate to align axes

C

Z

U V

Translate by - C (align origins)

PC = R ( PW - C )

3

Robert Collins CSE486, Penn State

Matrix Form, Homogeneous Coords

r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

0 1 0 0

1 0 0 0

Example: Simple Stereo System Z

Y

PC = R ( PW - C ) X Y Z 1

Robert Collins CSE486, Penn State

0  cx 0  cy 1  cz 0 1

U V W 1

(X,Y,Z)

z

left y camera located at (0,0,0)

( ,

x

z

)

y

( ,

Tx

)

right camera located at (Tx,0,0)

x

X Left camera located at world origin (0,0,0) and camera axes aligned with world coord axes.

Robert Collins CSE486, Penn State

Robert Collins CSE486, Penn State

Simple Stereo Projection Equations

Simple Stereo, Left Camera

X Y Z 1

r111 r012 r0 13  r21 r122 r0 0 23  r31 r032 r1 0 33  0 0 0 1

camera axes aligned with world axes

Left camera

0 0cx 0 0cy 1 0cz 0 1

0 1 0 0

1 0 0 0

U V W 1

located at world position (0,0,0)

=

Robert Collins CSE486, Penn State

Example: Simple Stereo System Z

Y left y camera located at (0,0,0)

(X,Y,Z)

z

z

( , )

x Tx

y

( , )

x

right camera located at (Tx,0,0)

X Right camera located at world location (Tx,0,0) and camera axes aligned with world coord axes.

Robert Collins CSE486, Penn State

X Y Z 1

Simple Stereo, Right Camera r111 r021 0r31

r012 r0 13  r122 r0 23 

r032 r1 33  0 0 0 1

camera axes aligned with world axes

1 0 0 0

0 1 0 0

0 -T  cxx 0 0cy 1 0cz 0 1

U V W 1

located at world position (Tx,0,0)

=

4

Robert Collins CSE486, Penn State

Simple Stereo Projection Equations

Left camera

Bob’s sure-fire way(s) to figure out the rotation

Robert Collins CSE486, Penn State

X Y Z 1

Right camera

r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

1 0 0  cx 1 about 0 thiscy 0forget while thinking 0 rotations 1  cz 0about 0 0 0 1

U V W 1

PC = R PW This equation says how vectors in the world coordinate system (including the coordinate axes) get transformed into the camera coordinate system.

Robert Collins CSE486, Penn State

X Y Z 1

Robert Collins CSE486, Penn State

Figuring out Rotations r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

U V W 1

Figuring out Rotations

and likewise with world Y axis and world Z axis...

PC = R PW

same axis in camera coords

X Y Z 1

what if world x axis (1,0,0) corresponds to camera axis (a,b,c)?

X a Y b Z c 1

r11 r12 r13 r21 r22 r23 r31 r32 r33 0 0 0

   1

U 1 V 0 0 W 1

X a Y b Z c 1

ra11 r12 r13 rb21 r22 r23 rc31 r32 r33 0 0 0

   1

U 1 V 0 0 W 1

axis is world coords

r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

world X axis (1,0,0) in camera coords

world Y axis (0,1,0) in camera coords

U V W 1 world Z axis (0,0,1) in camera coords

we can immediately write down the first column of R!

Robert Collins CSE486, Penn State

Figuring out Rotations

Alternative approach: sometimes it is easier to specify what camera X,Y,or Z axis is in world coordinates. Then do rearrange the equation as follows.

PC = R PW

R-1PC = PW

r11 r21 r31  r12 r22 r32  r13 r23 r33  0 0 0 1

X Y Z 1

RTPC = PW U V W 1

Robert Collins CSE486, Penn State

Figuring out Rotations

r11 r21 r31  r12 r22 r32  r13 r23 r33  0 0 0 1

X Y Z 1

U V W 1

RTPC = PW

what if camera X axis (1,0,0) corresponds to world axis (a,b,c)?

r11 r21 r31 r12 r22 r32 r13 r23 r33 0 0 0

   1

X 1 Y 0 0 Z 1

U a V b W c 1

ra11 r21 r31  X U 1 a V rb12 r22 r32  Y 0 b 0 c W rc13 r23 r33  Z 0 0 0 1 1 1 we can immediately write down the first column of RT, (which is the first row of R).

5

Robert Collins CSE486, Penn State

Robert Collins CSE486, Penn State

Figuring out Rotations

Example

and likewise with camera Y axis and camera Z axis... same axis in camera coords

X Y Z 1

axis is world coords

r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

y

U V W 1

z x

camera X axis (1,0,0) in world coords

Robert Collins CSE486, Penn State

X Y Z 1

0 0 1 Rtrain 0 -1 0 1 0 0

camera Z axis (0,0,1) in world coords

camera Y axis (0,1,0) in world coords

Note: External Parameters also often written as R,T r11 r12 r13  r21 r22 r23  r31 r32 r33  0 0 0 1

R ( PW - C ) = R PW - R C = R PW + T

1 0 0 0

0 1 0 0

r11 r12 r13 tx r21 r22 r23 ty r31 r32 r33 tz 0 0 0 1

0  cx 0  cy 1  cz 0 1

Robert Collins CSE486, Penn State

U V W 1

World Coords U V W

Rfly

0 -1 0 0 0 1 -1 0 0

Summary Camera Coords X Y Z

Film Coords x y

Pixel Coords u v

We now know how to transform 3D world coordinate points into camera coords, and then do perspective project to get 2D points in the film plane. Next time: pixel coordinates

6