Todays Lecture Camera Calibration The inner parameters - K . Projective vs. Euclidean Reconstruction. Finding the camera matrix. DLT - Direct Linear Transformation. Normalization of uncalibrated cameras. Radial distortion
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
2 / 28
The Inner Parameters -K The matrix K is the upper triangular γf K = 0 0
matrix: sf f 0
x0 y0 . 1
f - focal length γ - aspect ratio s - skew (x0 , y0 ) - principal points
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
3 / 28
The Inner Parameters -K The focal length f
fx f 0 0 x fy = 0 f 0 y 1 0 0 1 1 Re scales the images (e.g. meters → pixels).
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
4 / 28
The Inner Parameters -K The principal point (x0 , y0 ) fx + x0 f 0 x0 x fy + y0 = 0 f y0 y 1 0 0 1 1 Re centers the image. Typically transforms the point (0, 0, 1) to the middle of the image.
480
0 0
Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
640
2016-01-27
5 / 28
The Inner Parameters -K Aspect ratio
γfx + x0 γf fy + y0 = 0 1 0
0 x0 x f y0 y 0 1 1
Pixels are not always squares but can be rectangular. In such cases the scaling in the x-direction should be different from the y-direction. Skew
γf 0 0
sf f 0
x0 y0 1
logoonly
Corrects for tilted pixels. Typically zero. Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
6 / 28
Projective vs. Euclidean Reconstruction Calibrated Cameras A camera P = K [R
t] ,
where the inner parameters K are known is called calibrated. If we change coordinates in the image using ˜ x = K −1 x, we get a so called normalized (calibrated) camera ˜x = K −1 K [R
t] X = [R
t] X.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
7 / 28
Projective vs. Euclidean Reconstruction Projective The reconstruction is determined up to a projective transformation. If λx = PX, then for any projective transformation ˜ = H −1 X X we have ˜ λx = PHH −1 X = PH X. PH is also a valid camera.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
8 / 28
Projective vs. Euclidean Reconstruction Euclidean The reconstruction is determined up to a similarity transformation. If λx = [R t]X, then for any similarity transformation ˜= X we have λ x = [R s
t]
sQ v 0 1
Q v /s 0 1/s
−1 X
˜ = [RQ X
R
v t ˜ + ]X. s s
Since RQ is a rotation this is a normalized camera.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
9 / 28
Projective vs. Euclidean Reconstruction
Projective
Euclidean
Arc de triomphe, Paris. The reconstructions have exactly the same reprojection error. But the projective coordinate system makes things look strange. logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
10 / 28
Projective vs. Euclidean Reconstruction Demo.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
11 / 28
Finding K See lecture notes.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
12 / 28
RQ-factorization Theorem If A is an n × n matrix then there is an orthogonal matrix Q and a right triangular matrix R such that A = RQ. (If A is invertible and the diagonal elements are chosen to be positive, then the factorization is unique.) Note: In our case we will use K for the triangular matrix and R for the rotation.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
13 / 28
Finding K See lecture notes.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
14 / 28
Direct Linear Transformation - DLT Finding the camera matrix Use images of a known object to eliminate the projective ambiguity. If Xi are 3d-points of a known object, and xi corresponding projections we have λ1 x1 = PX1 λ2 x2 = PX2 .. . λN xN = PXN . There are 3N equations and 11 + N unknowns. We need 3N ≥ 11 + N ⇒ N ≥ 6 points to solve the problem.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
15 / 28
Direct Linear Transformation - DLT Matrix Formulation p1T P = p2T p3T
where pi are the rows of P The first equality is XT 1 p1 − λ1 x1 = 0 XT 1 p2 − λ1 y1 = 0 XT 1 p3 − λ1 = 0, where x1 = (x1 , y1 , 1). In matrix form XT 1 0 0
Direct Linear Transformation - DLT Homogeneous Least Squares See lecture notes...
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
18 / 28
Singular values decomposition Theorem Each m × n matrix M (with real coefficients) can be factorized into M = USV T , where U and V are orthogonal (m × m and n × n respectively), diag(σ1 , σ2 , ..., σr ) 0 S= , 0 0 σ1 ≥ σ2 ≥ ... ≥ σr > 0 and r is the rank of the matrix.
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
19 / 28
Direct Linear Transformation - DLT Homogeneous Least Squares See lecture notes...
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
20 / 28
Direct Linear Transformation - DLT Improving the Numerics (Normalization of uncalibrated cameras) The matrix contains entries xi , yj and ones. Since xi and yi can be about a thousand, the numerics are often greatly improved by translating the coordinates such that their “center of mass” is zero and then rescaling the coordinates to be roughly 1. Change coordinates according to s 0 −s x¯ ˜ x = 0 s −s y¯ x. 0 0 1 Solve the homogeneous linear least squares system and transform back to the original coordinate system. Similar transformations for the 3D-points Xi may also improve the logoonly results. Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
21 / 28
Pose estimation using DLT
3D points measured using scanning arm. Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
2016-01-27
22 / 28
Pose estimation using DLT
14 points used for computing the camera matrix. Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
2016-01-27
23 / 28
Pose estimation using DLT
14 points used for computing the camera matrix. Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
2016-01-27
24 / 28
Texturing the chair
Project the rest of the points into the image. Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
2016-01-27
25 / 28
Texturing the chair
Form triangles. Use the texture from the image. Magnus Oskarsson
Computer Vision: Lecture 3
logoonly
2016-01-27
26 / 28
Textured chair
logoonly
Magnus Oskarsson
Computer Vision: Lecture 3
2016-01-27
27 / 28
Radial Distortion
Not modeled by the K -matrix. Cannot be removed by a projective mapping since lines are not mapped onto lines (see Szeliski). Magnus Oskarsson
