123

CHAPTER 6 MULTIMODAL BIOMETRIC RECOGNITION SYSTEM

Some of the challenges faced by the unimodal biometric systems are noisy data, lack of universal data, less performance improvement and vulnerable to spoof attacks. Hence, multimodal biometric system is preferred, which refers to the system that establishes identity based on combining two or more biometric traits, i.e., fuse multiple traits of an individual or extraction of multiple features and matching algorithms operating on the same biometric. The most compelling reason to combine different biometric traits is to improve the identification accuracy significantly and provide larger population coverage. This can be done when features extracted from different biometrics are statistically independent. 6.1

PROPOSED MULTIMODAL BIOMETRIC SYSTEM The block diagram of the proposed multimodal biometric

recognition system is shown in Figure 6.1. The biometric images obtained from each modality are preprocessed, transformed using multi-resolution transforms such as Curvelet and Ridgelet, and then, features are extracted from these transformed output images. These extracted multi-resolution features are compared against the stored templates to generate the match scores.

124

DAIris

DAFingerprint

DAFace

Preprocessing

Preprocessing

Preprocessing

Multi-resolution Transforms Features Extraction

Features Extraction

Features Extraction

Matching Score

Matching Score

Matching Score

Score level Fusion

Stored Templates

Decision ?

No

Yes Genuine

Imposter DA

Figure 6.1

Data Acquisition

Block Diagram of Proposed Multimodal Biometric Recognition System

A match score is a measure of similarity between the input and template biometric feature vectors. When match score output by different

125

biometric matchers are consolidated in order to arrive at a final recognition decision, fusion is said to be done at the match score level. This is also known as fusion at the measurement level or confidence level. After generating matching scores for each modality, the scores are normalized using various techniques such as min-max (MM), max (MX), median-MAD (MAD), tanh (TH), double-sigmoid (DS), and z-score (ZS). 6.2

DIFFERENT LEVELS OF FUSION TECHNIQUES The various levels of fusion techniques are broadly categorized into

two namely, pre-classification or fusion before matching, and postclassification or fusion after matching (Jain et al 2008). Different fusion techniques that can be accomplished at various levels in a biometric system are shown in Figure 6.2.

Biometric Fusion

Before matching

Sensor Level

Feature Level

Match-Score Level

After matching

Classifier Fusion

Rank Level

Decision Level

Figure 6.2 Different Levels of Fusion Techniques in a Biometric System

126

The sensor level and feature level techniques fall into the first category, while the second category fusion techniques are match-score level, rank level and decision level. 6.2.1

Sensor Level Fusion Technique It combines information at the raw data obtained using multiple

sensors or multiple snapshots of a biometric using a single sensor. Mosaicing multiple samples of the same biometric is an example of this type of fusion. 6.2.2

Feature Level Fusion Technique It typically involves enhancing the feature vectors arising from

multiple feature extractors and subjecting the fused feature vector to a feature transformation algorithm. This enables the system to compute discriminatory features that can enhance matching performance. 6.2.3

Match-Score Level Fusion Technique Each individual biometric process typically outputs a match score

but possibly, multiple scores. The fusion process fuses these scores into a single score, which is then compared to the system acceptable threshold. If the classification approach to score fusion is allowed, then the output may be direct decisions instead of scores. Fusion methods at this level can be broadly classified into three categories namely, density-based, transformation-based and classifier-based schemes (Ross et al 2006). In the density-based scheme, a large number of training samples are necessary to reliably estimate the joint-density function especially if the dimensionality of the feature vector is large. In transformation-based scheme, a score normalization process is essential to transform the multiple match

127

scores generated by multiple matchers into a common domain. The goal of classifier-based scheme is to directly estimate the class rather than to compute an intermediate scalar value. This fusion scheme assumes the availability of a large representative number of genuine and impostor scores during the training phase of the classifier when its parameters are computed. 6.2.4

Rank Level Fusion Technique When a biometric system operates in the identification mode, the

system output can be viewed as a ranking of the enrolled identities. In this case, the output indicates the set of possible matching identities sorted in decreasing order of confidence. The goal of rank level fusion scheme is to consolidate the rank output by the individual biometric subsystems in order to derive a consensus rank for each identity. This fusion technique does not need normalization techniques. 6.2.5

Decision Level Fusion Each individual biometric process outputs its Boolean result. The

fusion process fuses them together by a combination algorithm such as AND, OR rules etc. Other decision level fusion technique includes majority voting, weighted majority voting, Bayesian decision fusion, etc. 6.3

PROPOSED SCORE LEVEL FUSION TECHNIQUES The matching scores at the output of the individual matchers may

not be homogeneous. For example, one matcher may output a dissimilarity measure while another may output a similarity measure. Further, the scores of individual measures need not be on the same numerical scale and may follow different statistical distribution. Due to these reasons, score normalization is

128

essential to transform the scores of the individual matchers into a common domain prior to combining them. 6.3.1

Score Normalization Techniques Score normalization refers to changing the location and scale

parameters of matching score distributions at the output of the individual matchers, so that the scores of different matchers are transformed into a common domain. When the parameters used for normalization are determined using a fixed training set, it is referred to as fixed score normalization. In such case, the matching score distribution of the training set is examined and a suitable model is chosen to fit the distribution. Based on the model, the normalization parameters are determined. In adaptive score normalization, the normalization parameters are estimated based on the current feature vector. This approach has the ability to adapt to variations in the input data such as the change in the length of the speech signal in speaker recognition systems. The score normalization techniques used in our work include min-max (MM), max (MX), median-MAD (MAD), tanh (TH), double-sigmoid (DS), and Z-score (ZS) normalizations. Let M be the total number of matchers, Xi be the set of all the raw output values (matching scores) of the corresponding matcher i and xi i

Xi,

M, where xi is the output raw value of the matcher i. The

corresponding normalized matching score is denoted as nsi

NSi, where NSi

is the set of normalized matching scores of the matcher i. Min-max normalization (MM): It is the simplest normalization technique that normalizes the numerical range of the scores to [0, 1] and also retains the shapes of the original distributions. The min-max normalization function is given by

129

where min(Xi) and max(Xi) are the minimum and maximum values of the matching scores. Max normalization (MX): This normalization technique is constructed from the above min-max normalization technique by assigning zero to the min value. The max normalization function is given by

Median-MAD normalization (MAD): This normalization scheme would be relatively robust to the presence of noise in the training data and is given by

The denominator value is Median Absolute Deviation (MAD), which is an estimate of the scale parameter of the feature value. Tanh-normalization (TH): It is similar to Z-score normalization, which is also computed using mean and standard deviation of the match scores. The tanh-normalization function is given as:

where µ and respectively.

denote the mean and standard deviation of the match scores

130

Double-sigmoid normalization (DS): This normalization scheme provides a linear transformation of the scores in the region of overlap, while the scores outside this region are transformed non-linearly. The double-sigmoid normalization function is given by

where

is the reference operating point and

1

and

2

denote the left and

right edges of the region in which the function is linear. The function exhibits linear characteristics in the interval ( -

1,

-

2).

Z-score normalization (ZS): The most commonly used score normalization technique is the Z-score which is calculated using the arithmetic mean and standard deviation of the match scores. It is given by

where µi and

i

are the arithmetic mean and standard deviation respectively

for the ith matcher. This scheme can be expected to perform well if the average and the variance of the score distributions of the matchers are available, if not known, they can be estimated based on the given training set. Table 6.1 shows the summary of the various matching score level normalization techniques as cited in Ross et al 2008. It states that the tanh and double sigmoid techniques are more robust and their efficiencies are high.

131

Table 6.1 Summary of Normalization Techniques Normalization techniques

6.3.2

Robustness

Efficiency

Min-max

No

N/A

Max

No

N/A

Median-MAD

Yes

Moderate

Tanh

Yes

High

Double-sigmoid

Yes

High

Z-score

No

High

Various Score-level Fusion Techniques Consider NSiris, NSfp, and NSface as the normalized scores of iris,

fingerprint and face respectively. Let a, b and c be the weights assigned respectively to the biometric modalities iris, fingerprint and face. During the implementation of weighted fusion techniques, three test cases are considered. For the first test case, equal weights (0.33) are assigned to all the three modalities and for the remaining two test cases, more weights are assigned to iris, next less weight to fingerprint and least weight to the face modality. The chosen weights are based on the possibility of spoofing attacks. Table 6.2 Different Weights Assignment Strategy Weights assigned for Test case # Iris

Fingerprint

Face

1

0.33

0.33

0.33

2

0.4

0.3

0.3

3

0.5

0.3

0.2

132

Since, this system aims to reduce (or) decrease the vulnerability of spoofing attacks, falsifying iris trait is tough when compared to fingerprint and is still tough when compared to the face modality. Hence, the iris modality is assigned with the highest weight thus leading to better multimodal recognition rate. b takes either 0.3 or 0.33

a takes the value either 0.33 or c takes either 0.2 or 0.3 or 0.33, as

listed in Table 6.2. The eight score level fusion techniques namely sum, mean, product, min, max, median, tanh and exponential methods are considered and are listed in Table 6.3. Here, NSfinal denotes the final normalized score after applying individual fusion techniques. Table 6.3 Various Score Level Fusion Techniques Fusion techniques Weighted Sum Weighted Mean Weighted Min Weighted Max Weighted Median Weighted Product Weighted Tanh Weighted  Exponential

Formula

133

6.3.3

Decision-Making Technique Threshold value is chosen adaptively by collecting the wrongly

classified subjects

minimum match scores and applying them in the

corresponding fusion techniques. A user is considered to be genuine, if the match score value after fusion is less than the chosen threshold; otherwise as an imposter. 6.4

EXPERIMENTAL RESULTS AND DISCUSSION The proposed multimodal biometric recognition system is tested

with the virtual multimodal database formed from the public biometric database. To demonstrate the effectiveness of the proposed methodologies on the virtual multimodal database, the public database considered here are CASIA-IrisV3-Interval database for Iris, FVC2004 DB1_A for fingerprint and ORL database for face. The subjects chosen from each biometric trait is forty with eight samples per subject, leading to 160 images for training and 160 for testing for each biometric modality. Table 6.4 Recognition Rate of the Public Database

Modality

No. of Subjects

No. of images used for

Recognition Rate (%)

training

testing

CT

RT

Iris

40

160

160

76.625

93.625

Fingerprint

40

160

160

88.125

71.875

Face

40

160

160

96.875

79.375

The reason for choosing forty subjects is that the ORL database contains a maximum of forty subjects and reason for choosing eight samples per subject,

134

is that FVC2004 DB1_A database as a maximum of eight samples per subject. Table 6.4 illustrates the recognition rate of individual biometric traits from the public biometric database. It is seen from the Table 6.4 that the Ridgelet Transform performs better for iris images and Curvelet Transform performs better for fingerprint and face images. 6.4.1

Recognition Results for Virtual Multimodal Database Table 6.5 shows the multimodal biometric recognition rate using

different normalization and fusion techniques for the virtual multimodal database by considering the above mentioned three test-cases. Eight different score fusion techniques and six different normalization techniques are considered for experimentation. The experimented results are shown in the Table 6.5. Each row in the table defines different fusion techniques and each column defines various normalization techniques. Table 6.5 Recognition Rate using Different Normalization and Fusion Techniques for Virtual Multimodal Database (a) Test Case-1 (b) Test Case-2 and (c) Test Case-3 Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

98.13

93.13

76.25

95.00

91.88

98.13

Weighted Mean

94.38

93.75

68.13

96.25

92.50

95.63

Weighted Min

96.25

92.50

85.63

98.13

96.25

95.63

Weighted Max

83.75

70.00

56.88

78.75

61.25

78.75

Weighted Median

88.13

93.75

93.13

86.25

83.13

82.50

Weighted Product

95.00

93.13

91.25

95.00

92.50

98.75

Weighted Tanh

95.00

93.13

93.75

95.00

91.88

93.75

Weighted Exponential

95.00

93.13

91.25

95.00

92.50

93.75

(a)

135

Table 6.5 (Continued) Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

95.00

93.75

68.13

96.25

92.50

95.63

Weighted Mean

96.25

94.38

76.25

96.25

93.75

95.63

Weighted Min

96.25

93.13

93.13

96.25

93.75

95.63

Weighted Max

96.25

96.25

57.50

97.50

93.13

92.50

Weighted Median

74.38

82.50

96.25

55.63

68.13

80.00

Weighted Product

93.13

94.38

89.38

96.25

93.75

94.38

Weighted Tanh

94.38

94.38

93.13

96.25

93.13

94.38

Weighted Exponential

95.00

93.13

60.63

96.25

91.88

90.00

(b) Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

95.00

94.38

60.63

96.25

91.25

97.50

Weighted Mean

95.00

94.38

60.63

96.25

91.25

97.50

Weighted Min

98.13

96.25

93.13

96.88

93.75

98.75

Weighted Max

97.50

97.50

59.38

97.50

93.13

97.50

Weighted Median

76.88

73.13

95.00

66.25

76.88

81.25

Weighted Product

96.88

94.38

85.00

96.88

93.13

98.75

Weighted Tanh

96.25

94.38

90.63

96.25

93.13

98.75

Weighted Exponential

95.00

94.38

60.00

96.25

90.63

91.88

(c) In the case of virtual multimodal database recognition rates, from Table 6.5, it is clearly seen that for the test case-1, 98.75 % is obtained for the weighted-product fusion technique; for the test case-2, the maximum

136

recognition rate of 97.5 % is obtained for weighted-max fusion technique; for the test case-3, the recognition rate of 98.75 % is achieved for weighted-min, weighted-product and weighted-tanh fusion techniques. 6.4.2

Recognition Results for Self-built Multimodal Database Table 6.6 shows the multimodal biometric recognition rate for the

self-built multimodal database for the above mentioned three test cases. Table 6.6 Recognition Rate using Different Normalization and Fusion Techniques for Self-Built Multimodal Database (a) Test Case-1 (b) Test Case-2 and (c) Test Case-3 Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

100.00

100.00

100.00

98.50

100.00

100.00

Weighted Mean

100.00

100.00

100.00

98.50

100.00

100.00

Weighted Min

100.00

100.00

81.00

100.00

100.00

100.00

Weighted Max

98.50

98.50

98.50

97.00

97.50

94.50

Weighted Median

100.00

100.00

99.00

99.50

100.00

99.00

Weighted Product

100.00

100.00

99.50

98.50

100.00

100.00

Weighted Tanh

100.00

100.00

97.50

98.50

100.00

100.00

Weighted Exponential

100.00

100.00

99.50

98.50

100.00

99.50

(a)

137

Table 6.6 (Continued) Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

99.00

100.00

100.00

98.50

100.00

99.00

Weighted Mean

99.00

100.00

100.00

98.50

100.00

99.00

Weighted Min

100.00

100.00

99.50

99.50

100.00

100.00

Weighted Max

99.50

97.00

98.50

93.00

94.50

94.00

Weighted Median

100.00

92.50

85.50

92.00

99.50

98.50

Weighted Product

88.00

100.00

98.50

98.50

100.00

100.00

Weighted Tanh

98.00

100.00

97.00

98.50

100.00

100.00

Weighted Exponential

99.00

100.00

99.50

98.50

100.00

96.50

(b) Normalization techniques Fusion techniques MM

MX

MAD

TH

DS

ZS

Weighted Sum

100.00

99.50

90.50

98.50

100.00

96.00

Weighted Mean

100.00

99.50

100.00

98.50

100.00

96.00

Weighted Min

100.00

100.00

100.00

99.50

100.00

100.00

Weighted Max

99.50

96.00

98.50

93.50

94.50

84.50

Weighted Median

99.00

98.50

80.00

98.00

99.00

98.50

Weighted Product

100.00

100.00

97.00

98.50

100.00

98.00

Weighted Tanh

100.00

99.50

96.00

98.50

100.00

99.00

Weighted Exponential

100.00

99.50

99.50

98.50

100.00

95.50

(c) From Table 6.6, it is observed for the self-built biometric data set that the 100 % recognition rate is achieved for many score normalization and fusion techniques. The test case-3, i.e., weight assignments of 0.5 for iris, 0.3 for fingerprint and 0.2 for face, results in better performance in an overall sense, for all fusion and normalization techniques while weighted-min fusion

138

technique outperforms other fusion techniques with 100 % recognition rate for almost all normalization techniques and for the three test cases, as evident from Table 6.6. 6.5

PERFORMANCE COMPARISON Liu et al 2009 used three biometric traits namely iris, fingerprint

and face with forty subjects and obtained a recognition rate of 95.6% using score level fusion by employing the Gabor Wavelet and the Posterior Union Decision-Based Neural Network (PUDBNN) while our proposed work, achieves a recognition rate of 99.38% for forty subjects which is shown in Table 6.7. Genuine Acceptance Rate (GAR) obtained for the self-built multimodal database is 99.5%. Table 6.7 Performance Comparison with Liu et al (2009) (Against Score Level Fusion) Recognition Rate (in %) Modalities Liu et al (2009) method

Proposed method

Iris

87.60

93.63

Fingerprint

85.20

88.13

Face

83.50

96.88

Iris + Fingerprint + Face

95.60

99.38

Gan et al (2008) conducted experiments on ORL face database and CASIA

iris

database.

They

extracted

the

two

features

using

Two-Dimensional Discrete Cosine Transform (2-D DCT) and used the Kernel Fisher Discriminant Analysis (KFDA) to produce the optimal discriminant feature at the feature level. They considered 7 frontal samples per subject with front 4 samples for each class as training samples, and the remainder 3

139

samples for each class as testing samples. Their obtained correct recognition rate based on appointed and random training and test samples were 96.67% and 94.17% respectively. Our experimental results done on the face and iris database produced the recognition rates of 97.5 % and 95.38 % at the feature level for the appointed and random training and test samples respectively. It is shown in Table 6.8. Table 6.8

Performance Comparison with Gan et al (2008) (Against Feature Level Fusion)

RR of Gan

method

RR of proposed method

(in %)

(in %)

96.67

97.50

Al-Hijaili & AbdulAziz (2011) performed the fusion of two features obtained from the iris and face modalities at the matching score-level by assigning weights 0.8 and 0.2 for the iris and face respectively. Iris and face images were recognized using the method proposed by Masek (2003) and Turk & Pentland (1991) respectively. The recognition rates for iris and face images were 95.63% and 75 % for 40 subjects respectively. When the two scores were fused at the score-level, the obtained recognition rate after fusion was 98.75 % and the genuine acceptance rate (GAR) was 98 %. Our proposed method has provided a recognition rate of 100% at the score level, with a genuine acceptance rate (GAR) of 98%. The recognition rates (RR) of these works are tabulated in Table 6.9.

140

Table 6.9 Performance Comparison with Al-Hijaili & AbdulAziz (2011) (Against Score Level Fusion) RR of Al-Hijaili & AbdulAziz (2011) method (in %)

RR of proposed method

98.75

100.00

(in %)

The improvement in the performance is due to the extraction of statistical / co-occurrence features by deploying multi-resolution transforms such as Curvelet and Ridgelet. Also, the match-score level fusion technique with high weightage assigned to iris biometric modality leads to better recognition rate. 6.7

SUMMARY The proposed framework for multimodal biometric recognition

system extracts the multi-resolution features from the Curvelet and Ridgelet transformed output images and the extracted features are matched at the matching score level. The performance of the multimodal biometric system is examined with different score normalization techniques and fusion methods by assigning different weights on both virtual and self-built multimodal database. The proposed weighted-min fusion technique performs well among all the normalization techniques. It is proved that the combination of iris, fingerprint and face biometric modalities for the proposed multimodal biometric recognition system has higher performance than each unimodal separately. The next chapter concludes this thesis work and addresses the scope for further enhancements in this work.