1

Biometrics Project: Bayesian Face Recognition Jinwei Gu Computer Science Department [email protected]

Abstract This project is to implement a 2D face recognition algorithm proposed in [2], which models the density of intrapersonal and extrapersonal face space separately with a single Gaussian for each, and thus uses Bayesian theory to do classification. It includes both maximum a posteriori (MAP) and maximum likelihood (ML) decision. Besides, we will try two improvements: one is to use Gaussian Mixture Model for density modelling since there will be multiple modes in intrapersonal face space, and the other one is to use Gabor feature jets instead of pixel intensity in face representation. The traditional eigenfaces [1] method is also implemented as a base line for comparison. The experiments will be carried on the ORL database containing 40 subjects and each one has 10 images under different lighting, pose, facial expression, and facial details. All the recognition algorithms are evaluated by the Cumulative Rank Curve.

I. L ECTURE R EVIEW 2D face recognition has been a popular and challenging research area since last decade. It arises general interests in computer vision, image analysis, psychology, etc. The problem can be formulated as that: given a set of face images with labelled identity (the gallery) and a set of unlabelled face images (the probe), identify the person in the probe images. Many works have been proposed for this problem till now. Below we briefly introduce the linear space methods, including eigenfaces and fisherfaces, and the method based on Bayesian theory on intrapersonal and extrapersonal classes. These methods are proved to be generally effective for many real applications [4]. A. Linear Methods In 1991, Turk and Pentland [1] proposed “eigenfaces” method. They generate “eigenfaces” from the training face images by principal component analysis (PCA), and classify the new November 11, 2005

DRAFT

2

image by computing the distance with the given faces in the feature subspace spanned by the “eigenfaces”. This algorithm, although seems simple, is pretty effective as long as the faces are nearly frontal view and under normal lighting. As shown in [4], with carefully choosing the distance metric (e.g. Mahanobis distance), Eigenfaces method can achieve as good performance as other more complex recognition algorithms. We will implement this algorithm in this project as a baseline for comparison. However, the drawback of eigenfaces is that it does not utilize the discriminant information for different classes (subjects) in the training data. Since we do PCA for all the images, if the change of lighting condition is more obvious than the difference of subjects, the principal components will capture the lighting instead of the different between subjects. To deal with this problem, Belhumeur et.al. [3] proposed a method called “fisherfaces”, which finds the best projection matrix by maximizing the distance between classes while minimizing the distance within classes. More specifically, as shown in [3], let x1 , . . . , xN denote N given face images, and they belong to X1 , . . . , Xc subjects. Both eigenfaces and fisherfaces want to find an orthogonal matrix W and project each face into a subspace spanned by the vectors of W : y = W · x. In eigenfaces, we determine W by: W ∗ = arg max W T ST W , W

while in fisherface, W is given by: |W T SB W | , W |W T SW W | and SB are the within-class scatter matrix and between-

W ∗ = arg max where ST is the total scatter matrix, SW class matrix, respectively: ST

N X = (xk − µ)(xk − µ)T , k=1

SW =

c X X

(xk − µi )(xk − µi )T ,

i=1 xk ∈Xc

SB =

c X

Ni (µi − µ)(µi − µ)T .

i=1

The experimental results in [3] show that fisherfaces achieves smaller recognition error than eigenfaces. November 11, 2005

DRAFT

3

B. Bayesian Method Both of these methods are to find a linear projection from the original feature space to a subspace, and then do template-matching for classification. In many situations, however, the faces of different subjects might not be linearly separable. Some researchers thus proposed to use kernel methods which is analogous to that in support vector machine (SVM). On the other hand, recall that Bayesian decision theory is the optimal choice in terms of classification error. We can not estimate the pdf (probability distribution function) for each subject because we do not have enough samples, but we can estimate the pdf for two classes: intrapersonal class ΩI (two images belong to the same subject), and extrapersonal class ΩE (two images belong to different subject). This method was proposed by Moghaddam et.al. [2]. Suppose ∆ = x1 − x2 is the difference of two images, the similarity of x1 and x2 is then measured by: S(x1 , x2 ) = P (∆ ∈ ΩI ) = P (ΩI |∆).

(1)

By using the Bayesian Theorem, we can deduce as follows: S = P (ΩI |∆) =

P (∆|ΩI )P (ΩI ) , P (∆|ΩI )P (ΩI ) + P (∆|ΩE )P (ΩE )

(2)

where P (ΩI ) and P (ΩE ) are the prior and can be set for specific problems, and is not very relevant to the final results [4]. In many cases, we can set them as 1/2. The terms P (∆|ΩI ) and P (∆|ΩE ) are the corresponding likelihoods. To compute the likelihood, the authors proposed to use P CA to do dimension reduction and model the distribution as a multi-variate gaussian:   2  P yi2    (∆) exp − 12 M exp − ǫ 2ρ i=1 λi  , Pˆ (∆|Ω) =  Q 1/2 (N −M )/2 (2πρ) (2π)M/2 M λ i=1 i 

(3)

where the first part is the probability in the cutoff subspace spanned by the first M eigenvectors, and the second term is the estimated probability of ∆ in the orthogonal complement space. ρ is the average of the eigenvalues of the complement space: N X 1 ρ= λi . N − M i=M +1

The authors also pointed out that since the intrapersonal eigenspace corresponds to a more tightly constrained subspace, we can only use P (∆|ΩI ) as a simplified similarity measure. The

November 11, 2005

DRAFT

4

experimental results show that the performance based on this maximum likelihood (ML) measure will not degrade much as that based on maximum a posteriori (MAP) measure in Equation (1). In practice, it is found that the probability computed by Equation (3) is very close to zero which will cause numerical problems. Teixeria [4] proposed to use the following equivalent similarity measures to ensure the numerical stability: M

SM L (∆) = − and

I 2 yI,i ǫ2 (∆) X − , ρI λ I,i i=1

# " # ME MI 2 2 yE,i yI,i ǫ2 (∆) X ǫ2 (∆) X SM AP (∆) = + − + . ρE λ ρ λ E,i I I,i i=1 i=1 "

(4)

(5)

II. P ROJECT P ROPOSAL This project is to implement the above Bayesian face recognition algorithm, compare it with the eigenfaces method, and try some possible improvements. A. Database The database we plan to use is the ORL Database of Faces [5]. It contains 10 different images of 40 distinct subjects. The images were taken at different times, varying the lighting, facial expressions and facial details. The images are almost frontal view. A summary of these images is shown in Figure 1. It is available at: http://www.cl.cam.ac.uk/Research/DTG/attarchive/facedatabase.html

B. Implementation and Experiments Our experiment will contain the following steps: •

Data preprocessing. This step is to normalize the sample images. We label the eyes of each images manually, and use them to normalize all the samples. As suggested in [4], we will use an ellipsoid mask to remove the boundary region of the face (hair, ears, etc), and then normalize the region inside the mask such that the mean is 0 and the standard deviation is 1.

•

Training. Compute the ΩI and ΩE from the gallery images, then perform PCA and save the eigenvalues and eigenvectors.

November 11, 2005

DRAFT

5

Fig. 1.

ORL face database.

November 11, 2005

DRAFT

6

•

Testing. For each image in the probe set, compute the similarity between the image with all the images in the gallery, and identify the id of the one with the best similarity. We use cumulative rank curve to evaluate the classifier’s performance. We also implement eigenfaces for comparison.

•

Possible improvements. We will try two improvements in this project. – Instead of using a single multivariate gaussian to model P (∆|Ω) in Equation (3), we will try to use Gaussian Mixture Model (GMM) to model the density. The logic behind is obvious, the intrapersonal class will have multiple modes, each of which for each different subject. – Instead of representing ∆ as the difference of the images, we will try to use Gabor filter jet on some given points of the face as the feature, and represent ∆ as the difference of two feature vectors. Using Gabor filter jet at given points to represent face is proposed in [6], which is proved to be effective by capturing the major characteristic of the faces while suppressing the noises of the images.

•

We also plan to build an automatic face recognition system, i.e. with automatic face detection and eye localization. Face detection will be done by the AdaBoost method built in OpenCV, and eye localization is done by local search.

The project will be built using C++ and Matlab. The output of the project are: •

Project report containing the experimental results and analysis

•

Programs including: – labelling tool and data normalization tool. – bayesian face recognition method, and eigenfaces method. – face recognition method with the improvements. III. T IME S CHEDULE

The time schedule of the project is as follows: •

Nov.15-Nov.18: data labelling and normalization

•

Nov.19-Nov.24: implement the algorithms and the experiments.

•

Nov.25-Nov.30: implement the improvements and the comparison experiments.

•

Nov.31-Dec.5: finish the project report.

November 11, 2005

DRAFT

7

R EFERENCES [1] M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 1991. [2] B. Moghaddam, W. Wahid and A. Pentland. Beyond Eigenfaces: probabilistic matching for face recognition. IEEE Int’l Conference on Automatic Face & Gesture Recognition, April 1998. [3] P.N.Belhumeur, J.P.Hespanha, and D.Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE PAMI, July 1997. [4] M. L. Teixeira. The Bayesian Intrapersonal/Extrapersonal Classfier, David Bolme, Masters Thesis, CSU Computer Science Department, July 2003. [5] F. Samaria and A. Harter. Parameterisation of a stochastic model for human face identification. 2nd IEEE Workshop on Applications of Computer Vision December 1994, Sarasota (Florida). [6] L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg. Face Recognition by Elastic Bunch Graph Matching, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, 1997, pp. 776-779.

November 11, 2005

DRAFT