Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012

Camera Self-Calibration by an Equilateral Triangle A.Baataoui, I.El Batteoui

A.Saaidi Et K.Satori

LIIAN, Department of Mathematics and informatics Faculty of Sciences Dhar-Mahraz P.O.Box 1796 Atlas-Fes, Morocco

LIIAN, Department of Mathematics and informatics Faculty of Sciences Dhar-Mahraz P.O.Box 1796 Atlas-Fes, Morocco

ABSTRACT In this article, we present a technique for self-calibration of a CCD camera with constant focal distance using a planar scene. The particularity of our technique is the use of triangle equilateral which two vertices are defined from the matches detected in images taken by the camera. Using these vertices to estimate all the projection matrices which are operated with homographies between the images to estimate the intrinsic parameters of the camera. Experimental results show the robustness of our algorithms in terms of stability and convergence.

Finally, three images are sufficient to determine the intrinsic parameters of camera by solving a system of nonlinear equation that needs initialization and optimization of a cost function associated to the parameters sought by the bais of 'Levenberg-Marquardt algorithm [15]. The paper is organized as follows: The second part presents the model of camera and equilateral triangle. The projection of the triangle is described in third part. Selfcalibration of cameras presented in fourth part. The experiments are presented in the fifth and concluding part in the sixth game.

Keywords: Self-Calibration, Equilatéral Triangle,

2. CAMERA AND EQUILATERAL TRIANGLE

Absolute Conic , Homography.

A.

1. INTRODUCTION

We consider the pinhole camera model to transform a point in the scene in his image, this model is defined by the perspective projection matrix P of 34 given by the

The calibration of a camera is to determine the intrinsic and extrinsic parameters using a known object called calibration pattern [8, 9, 10, 18]. The grid may be threedimensional.(calibration 3D) or plane (calibration 2D). This constraint is not always present in the applications of computer vision, which gives rise to new methods called self-calibration which allow to calculate the intrinsic and extrinsic parameters without any prior knowledge on the stage. The latter method uses 3D scenes [1, 2, 3, 21, 22, 23] or planar scenes [4, 5, 6, 7, 9] to calculate the camera settings automatically, but they face many problems, such as the majority of methods encounters the problem of systems to solve non-linear, which must be initialized while adding constraints on the model of self-calibration, for example in [16] a constraint is added to the movement of camera that undergoes a translation and a small rotation, this constraint permits to estimate the homography of the plane at infinity and to calculate parameters of the camera. In this paper we are interested on the camera selfcalibration with fewer constraints by the use of the object of equilateral triangle any scene 2D and movement rigid camera in space. Our technique is to estimate the homography matrix between two images by the RANSAC algorithm [11] based on points of interest detected by Harris [12] which are matched by the correlation function ZNCC [13, 14, 19 ]. Next, this matrix will be used with two matches (the projections of the two vertices of an equilateral triangle) to estimate the projection matrix and determined the system of equations.

Model of camera

following formula : P  K ( R t ) with : 

( R t) is the extrinsic parameter matrix, with R is the rotation matrix and t the translation vector of camera in space.



K is the intrinsic parameter matrix defined by:

 f   0 u0    K 0  f v0  0 0 1  

(1)

f is the focal length,  is the scale factors, (u0 , v0 ) are the image coordinates of the principal point ant  is the obliquity factor. B.

Triangle équilatéral

For any two points A and B in the plane of the stage, there is a point C so that ABC is an equilateral triangle set by the length of its side (Figure 1).

29

Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012



R  seuil : near a point of interest

B. Correlation Measure We use the correlation measure ZNCC  Zero mean Normalised Cross Correlation  to match the bridges of interest detected by the Harris algorithm, this measure is characterized by the invariance to changes in local luminance linear and defined by the following formula :

ZNCC (qi , q j ) 

The three heights (median, bisector and mediator) are the same measure. They are

3 simultaneously equal a with a is length of 2 one side of an equilateral triangle. The three angles are the same as: 60 .

3. EQUILATERAL TRIANGLE PROJECTION Harris Detector [12, 17]: To extract the high points in terms of information, we used the Harris detector is based on the following function: (2)

n

qi and q j two points of Harris detected in the two images i and j .

xn  I (qi  n)  I (qi ) yn  I '(q j  n)  I '(q j )

C. Homography between images RANSAC [11] : is an algorithm that estimates geometric entities (homography between images in our situation) from a dataset (matched points), whose error with respect to the entity is found above a threshold .

33 matrix transformation linking two points belonging to image i and j by the following equation:

with :

Homography btween images : is a

( x, y) : The coordinates of pixel in question. A

C B 

M= 

C

A

2 I x

2

B

(4)

I (qi ) and I '(q j ) are means of pixel luminance on a window centered respectively in qi and q j .

A. Interest points

E ( x, y )  ( x. y ).M ( x, y )t ,

 xn2  yn2

with :

a few properties of the equilateral triangle:



n

n

Figure 1. Equilateral triangle



 xn yn

 I y

2

2

w

q j : Hij qi

(5)

2

w

w( x, y ) 

1 2 2

C (

 I xy

with qi and q j are respectively the projections of the

)w

same point in the scene in the picture i and j .

( x2  y 2 ) 2 2 e 

H ij is the homography matrix between two images i

The matrix M characterizes the local behavior of the function E . To detect points of interest, we evaluated the following measure:

 R  Det ( M )  k  Trace 2 ( M )  2  with : Det ( M )  AB  C Trace( M )  A  B and k  0.04  

R  0 : at the vicinity of an edge



R  0 : in a homogeneous region

(3)

and j , estimated by the knowledge of at least four matches by applying the RANSAC algorithm. D. Projection matrix of the equilateral triangle Projection Matrixes : to estimate the projection matrix we will use two references : reference Affine and reference Euclidean ( B, X a ,Ya , Z a )

( B, X e ,Ye , Ze ) with Z e and Z a are perpendicular to the plane of the triangle ABC . Table 1 presents the coordinates of the vertices of an equilateral triangle in two references Affine and Euclidean.

30

Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012 Table 1: Homogeneous Coordinates Of Vertices Of The Triangle In The Two References Affine And Euclidean Point

B

Affine plan

Euclidean plan

A

Q1  (0, 1, 1)T

a 3 Q '1  ( , a, 1)T 2 2

B

Q2  (0, 0, 1)T

Q '2  (0, 0, 1)T

C

Q3  (1, 0, 1)T

Q '3  (a, 0, 1)T

(6)

scale factor, by:

(7)

of camera in space to project the scene in the image i and a   0 a 2     3 a 0  is the passage matrix, between the S  0 2   0 1 0     reference Affine and Euclidean, of vertices of the triangle as:

 S Qd

Lj Image j A

B

B

C

C

Figure 2. Projection of triangle ABC in the two images i and j by the two matrixes Li and L j

1 0   T  H i : KRi  0 1 Ri ti  The matrix is the 0 0    homography that permits to project the plan of the scene in the image i , therefor formul (7) becomes:

Li : Hi S

Ri , ti represent, respectively, orientation and position

Qd'

Li A

T With 1 d 3 , qid  (uid , vid ) is a point in the 'image i represents the projection of a vertices of the equilateral triangle and Li a 33 matrix that can be defined, up to a

1 0   T  Li : KRi  0 1 Ri ti  S 0 0   

C

Image i

The triangle ABC , expressed in the Affine plan, is projected in the image i by a matrix Li (Figure 2) as:

(uid , vid , 1)T : Li Qd

Planar scene

A

(8)

In practice there is not an automatic method to determine the triangle in the image i . But we can always determine which two vertices of the triangle can be deduced four linear equations from the equation (6). herefore we need other equations to calculate matrix Li .

(9)

And for the image j we can write:

Lj : H j S

(10)

From equations (9) and (10) we deduce that:

L j : Hij Li

(11)

with H ij is the homography betwin image i and j as:

Hij : H j Hi1 The matrix H ij

(12) is determined by the

method

explained in the part C. in the two images i and j the projections of the two vertices are given by:

(uid , vid , 1)T : Li Qd

(13)

(u jd , v jd , 1)T : L j Qd

(14)

with 1 d 3 . We devlop the two relations (11) and (14) we find that:

(u jd , v jd , 1)T : Hij Li Qd

(15)

31

Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012

Equations (13) and (15) are sufficient to determine the matrix Li , for this we use a method of singular value

Cost function : To solve the system (21). We minimize the following cost function:

min

4. CAMERA SELF-CALIBRATION

with

with

    S '     

0

'



(16)

a 2 3 a 2 0

a

0

    .    

the

matrix

Ri

is

T orthogonal ( Ri Ri  I3 ) , So the relation (16) can be written as follows:

LTi  Li

 S 'T S ' S 'T RiT ti  :    tT R S ' tiT ti  i i



T with   KK



(17)

is the projection of the absolute conic

: I3 in all images. z3i   the matrix contains the first z2i 

T two lines and columns of the matrix Li  Li , therfor from equation (17) we find that:

T

Mi : S ' S

 ij2  z3i z2 j  z2i z3 j n represents the number of images used. To solve the function (22) using the Levenberg-Marquardt algorithm [15] which requires a very important step initialization. Initialization : to initialize the function (22), we assume that certain conditions are satisfied on the vision system. 

The principal point is to the center of the image, therefore u0 and v0 are known.



Pixels are squared therefore   1 .



by replacing these data in the system of equations (21), we find the following system between image i and j :

 

1

 z1i We note by M i    z3i

(22)

l 2ij  z3i z1 j  z1i z3 j

After the development of the Equation (7) we obtain:



j i 1 i 1

 ij2  z1i z2 j  z2i z1 j

A. Self-calibration equations

K 1Li : Ri S ' RiT ti



n 1

  ij2  l 2ij  ij2  n

decomposition. the L j matrix is determined by equation (11).

'

(18)

This formula expresses the relation between the intrinsic camera parameters and those of a triangle. Therefore for two images i and j , relation (18) can be written :

Mi : M j



with   f 4 f 2

(23)



T

,  is a 3n 2 matrix and  is a 3n

vector such as  and  elements are expressed in function of u0 , v0 ,  , Li et L j .

5. EXPERIMENTATIONS To experiment with this technique and demonstrate its effectiveness, we took three images of 512 × 512 of two 2D scenes unknown by an camera whose intrinsic parameters are kept constant (Figure 3).

(19)

From the relation (19) we deduce the following equalities between image i and j :

z1 j z3i z3 j z3i z3 j z1i  ,  ,  z2i z2 j z1i z1 j z2i z2 j

Image 1

(20)

These equations (20) brought us to a system of three nonlinear equations:

 z1i z2 j  z2i z1 j  0   z3i z1 j  z1i z3 j  0   z3i z2 j  z2i z3 j  0

(21)

Image 2

Image 3

Figure 3.The three images of a scene unknown 2D used for self-calibration of the camera

To detect the rich points in terms of information, we used the Harris algorithm that eliminates the noise by a Gaussian filter and gives better results deal with the transformations of the image related to the rotation, scaling, change of views, change brightness and noise related to the sensor [20, 16, 17]. the Harris points are shown in figure 4:

32

Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012

6. CONCLUSION

Image 1

In these papers, we treated the problem of camera selfcalibration plan, using only two points of interest, which is the projection of the vertices of a triangle in each image. These points will be used to estimate the homography matrix between images and the projection matrix for each pair of image, leading to the end to a system of nonlinear equations for determining the intrinsic parameters of camera. Our technique therefore allows the camera self-calibration with a 2D scene known, with a simple, reliable and robust compared to other methods.

Image 2

7. REFERENCES Image 1

[1]

A.Saaidi, A.Halli, H.Tairi and K.Satori. Self – Calibration Using a Particular Motion of Camera. To appear in Wseas Transaction on Computer Research. Issue 4, Vol. 3, April 2008.

[2]

P.Sturm, A case against Kruppa's equations for camera self-calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, Issue 10, pp. 1199-1204, October 2000.

[3]

Manolis I.A. Lourakis and R.Deriche. Camera selfcalibration using the kruppa equations and the SVD of the fundamental matrix: the case of varying intrinsic parameters. Technical Report 3911, INRIA, 2000.

[4]

P.Gurdjos and P.Sturm. Methods and Geometry for Plane-Based Self-Calibration. CVPR, pp. 491-496, 2003.

[5]

A. Saaidi, A. Halli, H. Tairi and K. Satori. SelfCalibration Using a Planar Scene and Parallelogram. ICGST-GVIP, ISSN 1687398X,February 2009

[6]

P.Gurdjos, A.Crouzil and R.Payrissat. Another Way of Looking at Plane-Based Calibration: The Centre Circle Constraint. ECCV, 2002

[7]

B.Triggs. Autocalibration from planar sequences, In Proceedings of 5th European Conference on Computer Vision, Freiburg, Allemagne, Juin 1998.

[8]

P.F.Sturm and S.J.Maybank. On Plane-Based Camera Calibration: A General Algorithm, Singularities, Applications. In Proceedings of the CVPR-IEEE, Vol. 1, pp. 432-437, 1999.

[9]

Z.Zhang. A Flexible New Technique for camera Calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No.11, pp. 1330-1334, 2000.

[10]

M.Wilczkowiak, E.Boyer, P.Sturm. Camera Calibration and 3D Reconstruction from Single Images Using Parallelepipeds. In ICCV, Vancouver, Canada, pp. 142-148, July 2001.

[11]

M.A.Fischler et R.C.Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated

Image 3

Figure 4. The Harris points

For matching points detected by Harris in the three images, we used ZNCC correlation measure which is invariant to linear changes in local luminance, these are presented in Figure 5: 57 105 116 85 102 115 89 94 108 106 93104 95 101 82 78 96 77

177 166 153 171203 164 162

272 275 243 266 277 213239 273 216 199 228 144 172 264 174 148 180 223 257 187 229 240 145181 135 263 109 210 157 107 173206 260 158 134 140 215 224 236 249 150 86 254 169 204 237 252 184 72 185 195 225 65 127 188 61 128 217 205 66 62 124 182 192 183 57 178 67 87 167 121 155 37 24 20 25 30

149 189 98 125 136 99 130

7 105 116 85 102 115 89 94 108 106 254 93 104 95101 82 78 77 96

333

270 327 301 4047 14 100 63 91 810 110 159 22 39 41 5873 12 15 314 317 79 23 48 142 151 34 9 331 126 200 304 179 131 212 231 259 280 51 221 253 80 117 170 193 1829 31 45 129 322 230 55 70 214 255 284 88 103 141 197 274 302 310 186 233 295 313 323 332 81 118 71 244 267 111 298 146 285 56 278 21 122 64 74 198222 139 147 28 38 46 238 160194 289 309 330 59 90 50 54 137 201227 245 33 42 76 84 311 325 262279 119 161 250 287303 83 112 207 235 4 6 1627 132 175 165 4453 196 256 290 296 318 241 143 133 60 305 131932 75 247 163 226 312 319 49 123 291 176202 329 211234 251 265 281 52 68 292 154 17 35 168 114138 219248 299 326 261 269 293 306 320 283 190 11 26 288 113 120 152 191 208232 258276 300 246 335 69 282 297 209 321 324 271 286 220 242 268 294308 315 36 92 43 218 97 156

1

2 3

301

272 275 243 266 277 213239 273 216 199 264 144172 174 148 257 180 223 187 229 240 145 181 135 263 109 210 157 173 206 260 107 158 134 140 215 224 236 72 249 86 150 204 237 169 252 184 185 65 195 225 127 188 217 61 128 205 66 62 124 182 192 183 178 57 167 67 87 121 155

334

316 328 307

177 5 166 153 171203 164 162

37 125 24 136 316 189 98 20 130 25 99 328 307 30 270 327 4047 100 1810 14 63 91 110 159 314 41 22 39 2 12 15 79 151 304 317 331 73 142 48 126 179 200 23 34 259 9 131 212 280 253 221 193 322 51 117 170 284 231 70 80 230 255 274 129 214 1829 31 45 55228 302 141 197 295 310 332 88 103 233 313 323 118 244 267 298 278 285 71 81 111 146 186 222 198 56 122 64 74 238 21 139 147 309 330 2838 46 289 160 245 227 90 201 54 59 76 137 311 325 50 68 250 194 262279 33 287 303 84 119 3 207 235 83 132161 165 175 318 112 256 4 290 6 196 241 16 2729642 4453 305 133 143 312 319 247 291 329 226 60 123 163 211 1319 251 32 234 265 281 292 49 202 299 154 168 326 52 219 261 269 293 58 75 114 306 176 320 172635 283 190 248 288 335 258276 208232 300 113 138 152 191 11 246 282 297 321 324 209 149 120 271 286 220 242 268 294308 315 69 36 92 43 97 218 156

Image 1 57 105 116 85 102 115 89 94 108 106 93104 95 101 82 78 96 77

177 166 153 171203 164 162

334

333

Image 2 2

333

57 53 56

269 3

54 41 48 37 49 36

272 275 243 266 277 213239 273 216 199 228 144 172 264 174 148 180 223 257 187 229 240 145181 263 109135 210 157 107 173 260 158 206 134 140 215 224 236 249 150 86 254 169 204 237 252 184 72 185 195 225 65 127 188 61 128 217 205 66 62 124 182 192 183 57 178 67 87 167 121 155

208 210 175 201 212 209 148 88 108134 82 127 92 110 189 114 157 67 89 107 121 162 192 115 90 198 58 97 109139 194 6578 8498 147 50 158 170 182 25 187 44 46 106 137 171 185 118 196 119 130 159 122 149 22 73 138 26 70 116 117 104 20 27 66 95

334

236 13 143 5 47

154 254 181 207 250 266 75 245 261 52 265 200 205 240 239 270 246 259 150 123 71 99 23 242 260 11 10 234 86 165 268 38 145 186 215 219 241 251 32 155 253262 72 193 113 76 230 14 128 163 146 235 220 6174 167 213 132 176202 267 39 16 85 101 120 248255 19 30 156 224 133 172 40 5562 31 231252 264 6881 91 124 222 197214 135161 24 33 183 100 8 129 79 169 232 225 256 177 243 18 83 140 188 15 63 35 43 111 173 227 153 257 190 141 179 42 59 7787103131 216 228 160 6 184 199 237 144 168 244 271263 34 102 204 112136 195 218 17 69 223 249 258 2128 151 180 238 105 191211 226 125 217 233 8094 178 142166 221 206 93 126 229247 60 64 174 203 164 152 29 12 45 4 51 7 9 96

1

37 24 20 25 30

149 189 98 125 136 316 99 130 328 307 270 327 301 4047 14 100 63 91 810 110 159 22 39 41 5873 2 12 15 314 317 79 23 48 142 151 34 9 331 126 200 304 179 131 212 231 259 280 51 221 253 80 117 170 193 1829 31 45 129 322 230 3 55 70 214 255 274 284 88 103 141 197 302 310 186 233 295 313 323 332 81 118 71 244 267 111 298 146 285 56 278 21 122 64 74 198222 139 147 28 38 46 238 160 289 309 330 59 90 50 54 137 201 245 33 42 76 194 227 84 112 311 325 262279 119 161 250 287303 83 207 235 4 6 1627 132 175 165 4453 196 256 290 296 318 241 143 133 60 75 305 131932 247 163 226 312319 49 123 291 176202 329 211 234 251 265 281 52 68 292 154 17 35 168 114138 219 299 326 261 269 293 306 320 283 190 11 26 248 288 113 120 152 191 208232 258276 300 246 335 69 282 297 209 321 324 271 286 220 242 268 294308 315 36 92 43 218 97 156

1

Image 1

Image 3

Figure 5. The Matched points

The homographies between images and projections matrixes of a triangle (define by two vertices from interest points, the third vertices is not used) in all images are calculated, in short the resolution of a non-linear equation system permits to estimate elements of the image of the absolute conic and to calculate the intrinsic camera parameters. Table 2 presents the initial result and optimal intrinsic parameters of the camera of two scenes: Table 2: Initial And Optimal Solution Of The Intrinsic Camera Parameters.

Initial solution Optimal solution

f



u0

v0

1125 1175

1 0.92

256 261

256 263

33

Special Issue of International Journal of Computer Applications (0975 – 8887) on Software Engineering, Databases and Expert Systems – SEDEXS, September 2012

cartography. Graphics and Image Processing, Juin 1981. [12]

C.Harris et M.Stephens. A combined Corner et Edge Detector. 4th Alvey vision Conference. pp. 147-151, 1988.

[13]

M.Lhuillier and L.Quan. Quasi-dense reconstruction from image sequence. ECCV, 2002.

[14]

A.Saaidi, H.Tairi and K.Satori. Fast Stereo Matching Using Rectification and Correlation Techniques. ISCCSP, Second International Symposium on Communications, Control and Signal Processing, Marrakech, Morrocco, March 2006.

[15]

[16]

[17]

J.More. The levenberg-marquardt algorithm,implementation and theory. In G.A.Watson, editor, Numerical Analysis, Lecture Notes in Mathematics 630. Springer-Verlag, 1977. A.Saaidi. Contribution à l’Amélioration des Méthodes et des Algorithmes d’Autocalibrage des cameras, pour la Reconstruction des Scènes Tridimensionnelle. Thèse de doctorat, FSDM, Maroc, 2010. N.Elakkad, A.Baataoui, A.El abderrahmani, A.Saaidi et K.Satori. « Etude Comparative des détecteurs des points d’intérêt »WCCCS 11,2011.

[18]

A.Baataoui, M.Merras, N.Elakkad, I.El batteoui N.Elakkad, A.El abderrahmani, A.Saaidi et K.Satori. « Nouvelle Méthode De Calibrage De Caméra CCD Par Une Scène Plane Inconnue » WCCCS 11,2011.

[19]

A.Baataoui,N.Elakkad,N.Elakkad, A.El abderrahmani, A.Saaidi et K.Satori. « Etude Compartive des Méthodes de Mise en Correspondance» JDTIC ,2011

[20]

C. Schmid. Appariement d'Images par Invariants Locaux de Niveaux de Gris. Thèse de Doctorat, INPG, France, 1996.

[21]

A.El abderrahmani, A.Saaidi, K. Satori., "Robust Technique for Self-Calibration of Cameras based on a Circle". ICGST-GVIP, Vol 10, Issue 5, December 2010

[22]

A.El abderrahmani, A.Saaidi, K. Satori., “Planar Self-Calibration with Less Constraint". IJCST, Vol 2, Issue 2, June 2011

[23]

Adnane EL-ATTAR, Mohemed KARIM, Hamid TAIRI, Silviu IONITA “a robust multistage algorithm for camera self-calibration dealing with varying intrinsic parameters” JATIT 15th October 2011.Vol.32 No.1

34