Biometric Recognition Approaches for Access Control Prof. Shi-Jinn Horng

洪西进 教授

Department of Computer Science and Information Engineering National Taiwan University of Science and Technology [email protected] (O)02-27376700; 0930165009

OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

1. INTRODUCTION 

Motivation and purpose



Background of Biometric recognition and related research

1.1 Motivation and purpose (cont.) ◦ Biometric is better in security  Don’t need to memorize passwords , carry credentials or keys  Cannot be piracy and counterfeiting

◦ The security of public places  Airports, public transportation, Offices, Domitory,…

1.1 Motivation and purpose (cont.) A n ATM Machine with Finger Vein Recongintion System

1.1 Motivation and purpose (cont.) A n ATM Machine with Finger Vein Recongintion System

Eye Locker Installed in ATM Machine(台灣由田)

Face Recognition System Used in Immigration Office (台灣優奎士)

Hand Recognition Technique Using hand gesture to operate the TV, smart phone, to input the password for access control 

1.1 Motivation and purpose (cont.) 

Biometric authentication ◦ Fingerprint, iris, face, voice, hand geometry, vein, etc. Table1 Comparison of the biometric identifications

Source of the Table1: （社）日本自動認識システム協會HP

1.2 Background of Biometric recognition and related research (cont.) 

Face biometric • • • • •



Resistance is very low Easy to implement Accuracy is not so good Cheap Face can be copied or counterfeit

Vein be an ideal biometric ◦ ◦ ◦ ◦

The growth of veins change slowly Vein image by infrared light with special wavelengths to irradiate hand Finger vein, palm vein and arm vein recognition technology Near-infrared light and far-infrared light

1.2 Background of vein recognition and related research (cont.) 

Compared to other biometric technologies, palm vein has many advantages: ◦ ◦ ◦ ◦

High recognition rate Singularities Cost slightly lower High reliability, vein cannot be copied or counterfeit

1.2 Background of Biometric recognition and related research (cont.)

1.2 Background of Biometric recognition and related research (cont.)

1.2 Background of Biometric recognition and related research (cont.) The biometric industry's leading vendors

AuthenTec Cogent Systems

Hitachi ImageWare Systems

Cognitec Systems L-1 Identity Solutions Daon

LG Electronics

DigitalPersona Fujitsu

Lumidigm Motorola

NEC Nuance Precise Biometrics Sagem MORPHO Schlage UPEK

OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

2. Image Processing A New Finger Vein Feature Extraction Algorithm: by Caixia Liu

 A finger vein feature extraction method based on

improved adaptive Niblack threshold segmentation

A Method of Multispectral Finger-Vein Image Fusion: by Jinfeng Yang, Yongshuang Jia  To fuse finger vein images in different bands

Finger-vein Image Restoration Based on Skin Optical Property: by Jinfeng Yang, Guangliang Bai  To restore the vein via skin optical property

A Method of Multispectral FingerVein Image Fusion Jinfeng Yang, Yongshuang Jia Tianjin Key Lab for Advanced Signal Processing Civil Aviation University of China, CAUC, Tianjin, P.R. China

Image Denoising 

To use Pulse Coupled Neural Networks (PCNN) method for image denoising, PCNN denoising can be completed by adjusting brightness of image noise pixels, which can very well maintain image edge details and cost a few milliseconds.

Image Brightness Adjustment 



To present a linear gray transformation method to adjust the brightness value of image, not change the quality The gray-scale ranges of original images are [a,b]

Image Fusion 

Image denoising and registration are applied to finger vein image A(x, y) for band A and B(x, y) for band B.



Image brightness adjustment is applied to finger vein image



The fusion is based on the improved regional energy method



The improved regional energy method is listed as follows: For finger-vein images A(x, y) and B(x, y), regional pixel value in a 3*3 sliding window as fusion intensity can be expressed in (2) and (3). By comparing EA(x, y) with EB(x, y), regional smaller pixel value is utilized as the pixel of the fused image.

Image Enhancement 

B(x, y) represents original image, C(x, y) denotes the result image, z represents 20*20 regional variance of original image, and m represents 20*20 regional mean image of original image, then the result of finger-vein enhancement is shown in Fig. 5.

Experimental Results

Experimental Results

Experimental Results

Finger-vein Image Restoration Based on Skin Optical Property Jinfeng Yang, Guangliang Bai Tianjin Key Lab for Advanced Signal Processing Civil Aviation University of China, P.O. Box 9, Tianjin, 300300, China

PSF(Point Spread Function)



Here we assume that F(u,v), G(u,v) and H(u,v) repre­sent the Fourier transforms of f(x, y), g(x, y) and h(x, y), respectively, so the Weiner deconvolution filter is expressed as follows:

Scattering Noise Removal

OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

3. Feature Extraction  Finger-vein Authentication Based on Wide Line Detector and Pattern Normalization: by Beining Huang, Yanggang Dai, Rongfeng Li, Darun Tang Personal

Identification For Single Sample Using Finger Vein Location and Direction Coding: by Wenming Yang, Qing Rao,Qingmin Liao Finger

Vein Recognition Using Local Line Binary Pattern:

by Bakhtiar Affendi Rosdi,Chai Wuh Shing,Shahrel Azmin Suandi

Finger-vein Authentication Based on Wide Line Detector and Pattern Normalization Beining Huang, Yanggang Dai, Rongfeng Li, Darun Tang



A wide line detector for feature extraction, which can obtain precise width information of the vein and increase the information of the extracted feature from low quality image. [5]

Personal Identification for Single Sample Using Finger Vein Location and Direction Coding Wenming Yang, Qing Rao,Qingmin Liao

LDC (Location and Direction Coding)

• Valley shaped characteristic in vein area’s brightness: the cross-sectional brightness goes from high to low, and then to high.

•

A point has a high probability of being in the vein area when it is in the valley, and vise versa.

LDC can be any angles, here, restrict the search in eight

directions, 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°. The direction with the largest valley depth is taken as the direction of the pixel. Thus, obtain the valley depth image G and direction map of the vein image.



• 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5° are coded as 1, 2, 3, 4, 5, 6, 7, 8, respectively.

Vein Location Extraction from G and  

Step1: We use two thresholds, high threshold Th and low threshold Tl , to do the segmentation. ambiguous points



Step2: We use local threshold method for the further segmentation of the ambiguous area.

Vein Location Extraction 

Step3: Smooth the image with median filter. fix the small holes with morphological operation, and eliminate the noise points.

Vein Location and Direction Coding 

we code the directions 4, 5, 6 into one, and simplify the original 8 directions into 6 directions. Also 0 represents no vein.



Finger vein location and direction information can be coded into a feature map.



For example, a pixel has the largest

3

valley depth when the direction is 45. Then it is coded as 3 Feature map

Template Matching Based on Feature Map 

For corresponding points in two different feature map A and B, we assume they are matched if they are non-zero and equal. m

n

 ( A, B)   f ( A(i, j ), B(i, j )) i 1 j 1

1, s  t , s  0, t  0 f ( s, t )   0, others  

R(A, B) is the similarity function with value interval [0,1].



The larger the similarity, the higher probability of matching.



Taking into account of the possible shift of the finger position in the image, we perform a shift correction when the similarity is calculated.

Finger Vein Recognition Using Local Line Binary Pattern Bakhtiar Affendi Rosdi,Chai Wuh Shing,Shahrel Azmin Suandi

Feature Extraction 

LBP (Local Binary Pattern)

Feature Extraction 

Local Line Binary Pattern (LLBP) ◦ The operator consists of two components: horizontal component and vertical component. The magnitude of LLBP can be obtained by calculating the line binary codes for both components.

Feature Extraction 

LLBP

Feature Extraction 

Local Line Binary Pattern (LLBP) ◦ The operator consists of two components: horizontal component and vertical component. The magnitude of LLBP can be obtained by calculating the line binary codes for both components. ◦ Each image pixel is coded by LLBP

Matching 

Hamming Distance (HD)

⊗ is a Boolean exclusive-OR operator between corresponding pair of bits. The codeA and codeB are the extracted binary and enrolled codes, respectively. CodeLength is the total number of bits of the enrolled codes.  When the two codes are from two different fingers, the HD is close to 1.

Experimental Results (EER)

Experimental Results 

Time consuming

OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

4. Feature Matching Template Matching Based on Feature Map Hamming Distance POC Hausdorff Distance

POC (Phase only Correlation) matching[7]: 2D Fourier Transform of image f(k1, k2) and g(k1, k2)





Cross Phase Spectrum



When f(n1, n2)=g(n1, n2) (same image), we have

When two images are similar, their POC function gives a distinct sharp peak; otherwise, drops significantly.



Hausdorff Distance等距同構的概念,是一個 用來測量兩個點集合在空間上之差異性的方 法，一個集合到離另一個集合最近的點的最 大距離。Mismatch of two sets.



較歐式距離比對快速，O(n+m). Two sets are the same, Hausdroff Distance = 0



OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

5. Lab Work Using 2D Gabor Filters and SIFT Features to design an effective Palm Vein and Palmprint Recognition device  System Devices  System Architecture 

5.1 System Device 

Device components ◦ Infrared camera ◦ Infrared light sensors ◦ The filter

Figure1 device components and system device

5.2 System Architecture 

The recognition system performs the following steps: ◦ ◦ ◦ ◦ ◦

Real-time hands detection Image pre-processing SIFT features extraction Features matching Experimental results

5.2 System Architecture (cont.) Palm image No Palm identify Yes Image preprocessing

Database

SIFT Feature matching

Similarity comparing

No

Yes Pass

Fig. 2 System Flow Chart

Not pass

5.2.1 Real-time Hand Detection 

Easier use identification system ◦ Focused on the center of the palm image ◦ Draw the scan lines of the square area on the center of the image ◦ Calculate the pixels that were scanned by all the scanning lines

Figure3 Scanning lines of the vein image

5.2.2 IMAGE PREPROCESSING Gabor Filter B. Image Features Enhanced A.

A.Gabor filter  

Gabor filter is a linear filter The main characteristics of Gabor filter display frequency distribution of the local image or signal ◦ Based multi-channel principle ◦ Different direction and size of the frequency bandpass filter ◦ Suitable for analysis local feature and texture extraction ◦ Signal detection, feature extraction, texture analysis, image segmentation and identification

B. Image Features Enhanced 

Gabor Filter

 ,  z   k  ,  k e

5 scales and 8 directions

i 

k  ,



2

2

e

 k   , 

2

z

2

k  k max / 

/ 2 2  

e 

ik

 , z

e

    /8

 2 / 2

 

B. Image Features Enhanced (cont.) Table2 Comparison parameters of Gabor filter 1 direction Orientation (0)

0

Frequency Sigma Average features of the images Average time of SIFT processing（ms） Average time of the features matching（ms）

0.5

3 directions

4 5 directions directions -90,-45,0,45 90,0,90,1 45,0,45,9 80 0 0.5 0.5 0.5

128

417

525

688

182

568

732

937

113.9

375.95

485.55

657.75

B. Image Features Enhanced (cont.)

Figure5 original image

Figure6 original image processed by the direction -450 of Gabor filter

Figure7 original image processed by the direction 00 of Gabor filter

Figure8 original image processed by the direction 450 of Gabor filter

5.2.3 SIFT FEATURE EXTRACTION 

Environmental factors leading to poor results ◦ Luminance, scale, rotation, translation and noise, etc.



David G. Lowe “Scale Invariant Feature Transform”（SIFT） ◦ Describe the local invariants ◦ Scale-space extremas ◦ Resistance to some changes in translation, rotation, scale and noise

5.2.3 SIFT FEATURE EXTRACTION (cont.) 

SIFT following four steps ： A. Detection of scale-space extrema B. Orientation assignment C. The local image descriptor

A. Detection of scale-space extrema 



It has been shown by Koenderink (1984) and Lindeberg (1994) that under a variety of reasonable assumptions the only possible scale-space kernel is the Gaussian function The scale space of an image is defined as a function, L( x, y,σ), that is produced from the convolution of a variablescale Gaussian, G( x, y,σ), with an input image, I( x,y ):

A. Detection of scale-space extrema (cont.) 

The difference-of-Gaussian function convolved with the image, D( x, y,σ), which can be computed from the difference of two nearby scales separated by a constant multiplicative factor k:

Figure9 Build first-order scale space by DoG Source of the figure9: David G. Lowe “Local feature view clustering for 3D object recognition”

A. Detection of scale-space extrema (cont.)

Figure10 Build Image Pyramid by DoG

A. Detection of scale-space extrema (cont.) 

Maxima and minima of the difference-ofGaussian images are detected by comparing a pixel to its 26 neighbors.

Figure11 Extreme detection schematic Source of the figure11: David G. Lowe “Distinctive Image Features from Scale-Invarient Keypoints”

B. Orientation assignment 

By assigning a consistent orientation to each keypoint based on local image properties ◦ Achieve invariance to image rotation

B. Orientation assignment (cont.) 

For each image sample, L(x,y), at this scale, the gradient magnitude, m(x,y), and orientation, θ(x,y), is precomputed using pixel differences:

C. The local image descriptor 



In order to achieve orientation invariance, the coordinates of the descriptor and the gradient orientations are rotated relatively to the keypoint orientation. A Gaussian weighting function with σ equal to one half the width of the descriptor window is used to assign a weight to the magnitude of each sample point.

D. The local image descriptor (cont.)  

Window Size 16 *16= 16 (4*4 )array 4×4 array of histograms with 8 orientation bins

Figure13 Description of the Keypoints 16*8= 128 dimensions Source of the figure13: David G. Lowe “Distinctive Image Features from Scale-Invarient Keypoints”

SIFT FEATURE EXTRACTION

Figure14 the process of SIFT on the image MuserH1

Figure15 the process of SIFT on the image MuserI1

5.2.4 FEATURE MATCHING 

Calculate similarity

Figure16 2 images matching in 5 points

◦ Obtained by the first point A and measure other points with distance this map :

Figure17 the distance between basis points and the other points

5.2.4 FEATURE MATCHING (cont.) 

Suppose two images have n matching points, and distance similarity of two images：



If Match points are less than 5, then don’t calculate the similarity and set similarity be 0

5.2.5 EXPERIMENTAL RESULTS 

Development environment ◦ ◦ ◦ ◦ ◦

Intel Core 2 Due E6550 (2.33GHZ) DDRII 2GB Windows XP Service Pack 3 Microsoft Visual C++ 2008 Open Source Computer Vision Library Ver.1.1

5.2.5 EXPERIMENTAL RESULTS (cont.) The database consists of 1000 people including 746 men and 254 women. Each person has 4 palm images. Totally, there are 4000 palm images.  Three experiments 

◦ Case One: 1：1 verification system ◦ Case Two: 1：N recognition system ◦ Case Three: rotation

Case One: 1：1 Verification System (cont.) Table 5 1：1 Verification Results Number of people

100

500

1000

Threshold d

25

25

25

P positive set

1200

6000

12000

N negative set

39600

998000

3996000

R rejected number

7

21

46

A accepted number

0

0

0

FRR（％）

0.58

0.35

0.383

FAR（％）

0

0

0

Time（ms）

355.667 for 1000 people

Case One: 1：1 Verification System (cont.)

Fig. 18 1：1 Features Distribution

Case Two: 1：N Recognition (cont.) How to solve the computation time for many users?  Cloud computing?  Filter?

Case Two: 1：N Recognition (cont.) Table 7 1：N Recognition Results

people

1000

people

time（ms）

Threshold d

25

50

934.82

P positive set

12000

100

1543.34

N negative set

24000

200

2417.25

R rejected number

46

500

6136.762

A accepted number

0

1000

12178.37

FRR（％） FAR（％）

0.383 0

Case Three: Rotation Table9 Experimental results of rotation Rotation degree

+-15

+-30

all

25

25

25

P

2000

2000

4000

N

2000

2000

4000

R

101

136

237

A

0

0

0

FRR（％）

5.05

6.8

5.93

FAR（％）

0

0

0

Threshold d

Case Three: Rotation (cont.) Table10 Comparison of different methods Number of FAR samples Y.Ding, D.Zhung and K. Wang[23] L. Wang and G. Leedham[26] Ajay Kumar, K. Venkata Prathyusha [11]

48

0

FRR 0.9%

32

1.5% 3.5%

100

1.14 %

Zhang [27]

100

0

Proposed method

1000

0

1.14 % 0.91 % 0.383 %

5.2.6. A Practical System  

Fusion Technology Face and Palm System

5.2.6. 1 Fusion Technology Fusion Levels 1. Sensor module 2. Feature extraction module 3. Matching module 4. Decision-making module 

5.2.6.1 Fusion Technology (cont.)

5.2.6.2 Face and Palm System  

Face and Palm System http://www.youtube.com/watch?v=BNcoA89GJLw

OUTLINE Introduction  Image Processing  Feature Extraction  Feature Matching  Lab Work 

Design an Efficient Palm Vein Recognition System Using SIFT Features A Practical System 

Conclusions

6. CONCLUSIONS  

Any Niche Market? Palm Vein Verifier US 5,000 Finger Vein Verifier US 6,000

6. CONCLUSIONS (cont.)  

A commercial system is under construction Want to join our laboratory?

6. CONCLUSIONS (cont.) 2013年亞洲百大大學入選名單(泰晤士高等教育)

名次/學校 14(14)台灣大學

名次/學校 4北京大学

27(27)清華大學

6清华大学

32(32)交通大學

24复旦大学

46(41)中山大學

25中国科技大学

47(41)成功大學

35南京大学

52(52)台灣科技大學

40上海交通大学

54(52)中央大學

41中国人民大学

68(－)台灣師範大學

45浙江大学

69(－)中國醫藥大學

51中山大学

71(－)元智大學

58武汉大学

72(－)中原大學

58武汉理工大学

76(－)海洋大學

66哈尔滨工业大学

83(－)中正大學

80大连理工大学

89(－)陽明大學

89西安交通大学

92(－)中興大學 98(－)逢甲大學 100(－)台北科技大學

99华中科技大学 13項指標：教學、研究、論文引

用次數、產業評分、國際化

6. CONCLUSIONS (cont.) 2014 QS 學科排名 台灣科技大學七領域進入 前兩百大

Q&A

Thank you for listening

Reference 1. Caixia Liu, A New Finger Vein Feature Extraction Algorithm, 6th International Congress on Image and Signal Processing (CISP 2013) 2. Jinfeng Yang, Yongshuang Jia, A Method of Multispectral Finger-Vein Image Fusion, ICSP2012 Proceedings 3. Jinfeng Yang, Guangliang Bai, Finger-vein Image Restoration Based on Skin Optical Property, ICSP2012 Proceedings 4. Wenming Yang, Qing Rao,Qingmin Liao, Personal Identification For Single Sample Using Finger Vein Location and Direction Coding, HandBased Biometrics (ICHB), 2011 International Conference on 5. Beining Huang, Yanggang Dai, Rongfeng Li, Darun Tang, Finger-vein Authentication Based on Wide Line Detector and Pattern Normalization, Pattern Recognition (ICPR), 2010 20th International Conference 6. Bakhtiar Affendi Rosdi,Chai Wuh Shing,Shahrel Azmin Suandi,Finger Vein Recognition Using Local Line Binary Pattern, Sensors 2011 7. Koichi Ito etc., A Finger Print Matching using Phase only Correlation, IEICE 2004