Information Fusion in Multibiometric Systems

1
Information Fusion in Multibiometric Systems
Md. Maruf Monwar Computer Science University of
Calgary 2008
2
Outline

• Biometric and Multibiometric Systems
• Issues Involved in Multibiometric Systems Design
• Information Fusion
• A Sample Multibiometric System
• Conclusion

3
Biometric System

• Biometric system
• An automatic pattern recognition system that
  recognizes a person by determining the
  authenticity of a specific biological and/or
  behavioral characteristic (biometric) possessed
  by that person
• Physiological biometric identifiers
• - Fingerprints,
• - Hand geometry,
• - Ear patterns
• - Eye patterns (iris and retina),
• - Facial features
• and other physical characteristics.
• Behavioral identifiers
• - Voice,
• - Signature
• - Typing patterns
• and others.

4
Biometric System

• Application of Biometric Systems
• Physical access control of, for example, an
  airport. Here the airport infrastructure, or
  travel infrastructure in general, is the
  application.
• Logical access control of, for example, a bank
  account i.e., the application is the access to
  and the handling of money.
• Ensuring uniqueness of individuals. Here the
  focus is typically on preventing double
  enrollment in some application, for example, a
  social benefits program.

5
Operational Modes of a Biometric System

• Verification Mode
• One-to-One transaction
• The user effectively claims an identity by
  providing
• some information which is typically used to
  call up a
• reference number from a database.
• Identification Mode
• One-to-Many transaction
• The users information is compared against a
  database
• of reference templates and the users
  identity determined
• as a match against one of these templates

6
Multibiometric System

• Most biometric systems deployed in real-world
  applications are monomodal, i.e. only one source
  of information is used for authentication.
• These systems often face numerous limitations,
  such as, susceptibility of the result to quality
  of the sample, its orientation/rotation and
  distortion, noise, intra-class variability,
  non-distinctiveness, non-universality, and
  others.
• Some of the limitations imposed by monomodal
  biometric systems can be overcome by including
  multiple sources of information for establishing
  identity. Such systems are known as
  Multibiometric Systems.

7
Advantages of Multibiometric Systems

• Multibiometric systems can offer substantial
• improvement in the matching accuracy.
• Reduced FAR and FRR Increase feature space and
  hence capacity of identification system
• Multibiometric addresses the issue of non-
• universality or insufficient population
  coverage.
• Achieved a certain degree of flexibility If a
  dry finger prevent user from enrolling ..
• Multibiometric system makes the life of any
• impostor harder.
• Facilitates challenge-response mechanism by
  asking the user to present a random subset of
  traits
• Can be used in single biometric system also

8
Advantages of Multibiometric Systems

• Multibiometric systems can effectively address
  the problem of noisy data.
• In case of poor voice, face can be used for
  verification
• Multibiometric systems can be effectively used
  in
• tracking or continuous monitoring system where
  
• only one trait is not sufficient.
• A person can be monitored by face and gait
  patterns, but in crowded place or for distance
  from camera, this is not possible simultaneously
• A multibiometric system may also be viewed as a
• fault tolerant system.
• Continues to operate even when certain source
  becomes unreliable

9
Single and Multibiometric Systems
10
Design Issues of Multibiometric Systems

• Cost benefits
• Tradeoff between the added cost and the
  improvement in matching
• Determining sources of biometric information
• Which of the sources are relevant to the
  application at hand?
• Acquisition and processing sequence
• Simultaneously or one-by-one Processing can also
  be done one after another or simultaneously
• Type of information used
• What type of information is to be fused?
• True or virtual multimodal database

11
Design Issues of Multibiometric Systems

• Fusion methodologies adopted
• Sum rule, maximum, minimum, Borda count
• Operational Mode
• Identification or verification
• Assigning weights to Biometric
• Weights need to be assigned or not?
• Multimodal Database
• True or virtual multimodal database

12
Sources of Multiple Evidences
- Multi-sensor systems (single biometric
trait-multiple sensors) 2D texture content using
CCD and 3D surface shape using range camera
13
Sources of Multiple Evidences
- Multi-algorithm systems (single biometric
trait-multiple algorithm)
14
Sources of Multiple Evidences
- Multi-instance systems (multiple instances of
same body traits) For dry skin, one fingerprint
might not work
15
Sources of Multiple Evidences
- Multi-sample systems (multiple samples of same
biometric traits)
A small size sensor may acquire multiple dab
prints of an individuals finger and later done
mosaicing
16
Sources of Multiple Evidences
- Multimodal systems (multiple biometric traits)
17
Sources of Multiple Evidences
- Hybrid systems
Two speaker recognition algorithm combined with
three face recognition algorithm
18
Fusion

• The goal of Fusion is to determine the best
  set of experts in a given problem domain and
  devise an appropriate function that can optimally
  combine the decisions rendered by the individual
  experts.
• Two subcategories
• Prior to matching
• - Sensor level fusion
• - Feature level fusion
• After Matching
• - Rank level fusion
• - Match score level fusion
• - Decision level fusion

19
Prior to Matching Fusion

• Sensor level
• The raw data acquired from multiple sensors can
  be processed and integrated to generate new data
  from which features can be extracted.
• For example, in the case of face
  biometrics, both 2D texture information and 3D
  depth (range) information (obtained using two
  different sensors) may be fused to generate a 3D
  texture image of the face which could then be
  subjected to feature extraction and matching.
• Feature level
• The feature sets extracted from multiple data
  sources can be fused to create a new feature set
  to represent the individual.
• The geometric features of the hand, for
  example, may be augmented with the
  eigen-coefficients of the face in order to
  construct a new high-dimension feature vector. A
  feature selection/transformation procedure may be
  adopted to elicit a minimal feature set from the
  high-dimensional feature vector .

20
After Matching Fusion

• Match Score Level
• Multiple classifiers output a set of match scores
  which are fused to generate a single scalar
  score.
• As an example, the match scores generated
  by the face and hand modalities of a user may be
  combined via the simple sum rule in order to
  obtain a new match score which is then used to
  make the final decision.
• Rank level
• This type of fusion is relevant in identification
  systems where each classifier associates a rank
  with every enrolled identity (a higher rank
  indicating a good match).
• Thus, fusion entails consolidating the
  multiple ranks associated with an identity and
  determining a new rank that would aid in
  establishing the final decision. Techniques such
  as the Borda count may be used to make the final
  decision

21
After Matching Fusion
Decision level When each matcher outputs its own
class label (i.e., accept or reject in a
verification system, or the identity of a user in
an identification system), a single class label
can be obtained by employing techniques such as
majority voting or AND/OR rules.
22
Various Fusion Level Possibilities
23
Some Multibiometric Systems
24
Example Multibiometric System

• Developed by Ross and Jain in 2003
• Used face, fingerprint and hand geometry.
• Match score level fusion method is used to
  consolidate the information from these three
  monomodal experts.

25
Example Multibiometric System
26
Example Multibiometric System

• Face Verification Eigenface technique is used.
• Fingerprint Verification Minutiae matching
  technique is used.
• Hand Geometry Verification Simple distance
  measure (between test and enrolled hand geometry
  feature vectors) technique is used.

27
Minutiae Matching
28
Example Multibiometric System

• Used virtual multimodal database face and
  fingerprint from the same subjects, but hand
  geometry from different users.
• Mutual non-independence of the biometric
  indicators allows to assign the biometric data of
  one user to another.

29
Example Multibiometric System

• Match Score level Fusion
• The database consisted of matching scores
  obtained from three different modalitiesface,
  fingerprint and hand geometry.
• This data was used to generate 1000 ( 50x20 - 5
  scores from 5 faces, 5 scores from 5 fingerprints
  and 10 scores from hand geometry) genuine scores
  and 24500 (50x10x49 - each feature set of a user
  was compared against one feature set each of all
  other users, 5 match score selected from either
  face and fingerprint scores and 5 match scores
  selected from 10 hand geometry scores) impostor
  scores.

30
Example Multibiometric System

• Match Score level Fusion
• All scores were mapped to the range 0, 100.
• Since the face and hand scores were not
  similarity scores (they were distance scores),
  they were converted to similarity scores by
  simply subtracting them from 100.
• A score vector represents the scores of multiple
  classifiers.
• The vector (x1, x2, x3) is a score vector,
  where x1, x2 and x3 correspond to the
  (similarity) scores obtained from the classifiers
  corresponding to the face, fingerprint and hand
  geometry systems, respectively.

31
Example Multibiometric System

• Sum Rule
• The simplest form of combination would be to
  take the weighted average of the scores from the
  
• multiple modalities. This strategy was
  applied to all
• possible combinations of the three
  modalities.
• Equal weights were assigned to each modality as
  
• the bias of each classifier was not computed.

32
Match Score Fusion Example
33
Example Multibiometric System
Experiment Results
34
Example Multibiometric System
Experiment Results
35
Example Multibiometric System
Experiment Results
36
Example Multibiometric System

• Experiment Results
• Their experiments suggest that the sum rule
• performs better than the decision tree.
• The FAR of the tree classifier is 0.036 and
  the
• FRR is 9.63. The sum rule that combines all
  
• three scores has a corresponding FAR of
  0.03
• and a FRR of 1.78.

37
Conclusion

• Advantages of multibiometric systems over
  single
• biometric systems and the issues that need to
  be
• considered during designing such systems have
  been
• discussed.
• Information fusion plays a key role in a
  multibiometric
• system and the success of such system depends
  heavily
• on fusion methods.
• Initial results obtained on a match score level
  fusion based
• multibiometric system proposed by Ross and
  Jain
• in 2003 that uses face, fingerprint and hand
  geometry
• features for biometric verification purposes.

38
One Other Important Issue
Score Normalization
Table. Genuine acceptance rate (GAR) () of
different normalization and fusion techniques at
the 0.1 false acceptance rate (FAR)
39
Some References
1 Arun A. Ross, K. Nandakumar and Anil K.
Jain, Handbook of Multibiometrics, Springer,
NY, USA, 2006. 2 L. Hong and A. K. Jain,
Integrating faces and fingerprints for personal
identification, IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 20, no.
12, pp. 12951307, 1998. 3 A. K. Jain, S.
Prabhakar, and S. Chen, Combining multiple
matchers for high security fingerprint
verification system, Pattern Recognition
Letters, vol. 20, no. 11-13, pp. 13711379,
1999. 4 R. Frischholz, U. Dieckmann, BioID
A Multimodal Biometric Identification System, In
IEEE Computer, vol. 33, no. 2, 2000, 64-68. 5
J. Fierrez-Aguilar, J. Ortega-Garcia, D.
Garcia-Romero, and J. Gonzalez-Rodriguez, A
comparative evaluation of fusion strategies for
multimodal biometric verification, in
Proceedings of 4th International Conference on
Audio- and Video-Based Biometric Person
Authentication (AVBPA), J. Kittler and M. Nixon,
Eds., vol. LNCS 2688, 2003, pp. 830837. 6 A.
Kumar, D. C. Wong, H. C. Shen, and A. K. Jain,
Personal verification using palmprint and hand
geometry biometric, in Proceedings of 4th
International Conference on Audio- and
Video-Based Biometric Person Authentication
(AVBPA), J. Kittler and M. Nixon, Eds., vol. LNCS
2688, 2003, pp. 668 678.
40
Thank You