SecurePhone: a mobile phone with biometric authentication and e-signature support for dealing secure transactions on the fly

1
SecurePhone a mobile phone with biometric
authentication and e-signature support for
dealing secure transactions on the fly
IST-2002-506883 Secure contracts signed by mobile
Phone
2
Presentation Outline

• SecurePhone concept and use
• Project aim 1 secure exchange
• Project aim 2 multi-modal Biometric Recogniser
• Performance on PDA
• Implementation constraints and performance on
  SIMcard
• PDA selection
• Conclusion

3
What is a SecurePhone?

• A SecurePhone is a new mobile communications
  device that enables users to exchange text/audio
  documents during a phone call to draw legally
  valid transactions.
• It combines biometric identity verification with
  e-signing in a system using front-edge
  technologies (wireless networking, double-key
  cryptography).
• Biometric recognizer enables strong
  authentication by comparing live biometric
  features with models trained on enrollment data
  which were previously stored on the PDA or on the
  devices SIM card.
• Biometric authentication grants access to
  built-in e-signing facilities, also integrated
  on the PDA/SIM.

4
User interface

• User Interface implementation includes
• biometric data management modules
• capture and pre-processing of enrollment or
  live test data on PDA
• training of GMM models (templates) for identity
  verification on PC (or PDA)
• Verification on PDA (now)/SIM (underway)
• SharedDoc module
• interactive modification and exchange of a text
  file, the e-contract
• exchange of audio files

5
Project aim 1 secure exchange
Secure PKI (personal key infrastructure)
Deal secure m-contracts during a mobile phone call

• secure private key stored on SIM card
• dependable multi-modal voice, face, signature
• user-friendly familiar, intuitive, non-intrusive
• flexible legally binding text/audio transactions
• dynamic mobile (anytime, anywhere)

6
PK technology in SecurePhone

• Public key encryption technology is used for
  e-signature, i.e. to enforce data integrity and
  non-repudiation in P2B, public-key technology is
  used for authentication over networks and/or form
  e-signing.
• SIM card is used as a tamper-proof device for
  e-signing and storing the users e-signature
  private key (strong signature if the
  corresponding digital certificate is e-signed by
  a valid CA).
• Standard e-signature certificates and procedures
  are used for certificate verification and
  management, so documents e-signed by means of the
  SecurePhone have the same legal validity as
  documents e-signed by other means.

7
Biometric verification architectures

• Biometric templates can be stored
• on the SIM card (ToC)
• on the PDA/host (ToH)
• on a Trusted Third Party (TTP) server (ToS)
• Matching/verification can be performed
• by an applet running on the SIM card (MoC)
• by an application running on the PDA/host (MoH)
• by an application running on a TTP server (MoS)
• Only ToC MoC meets requirements on
• security
• privacy and user acceptance

8
Person-to-Person (P2P) user scenario

• During a phone call, two SecurePhone end users
  (actors) agree on drawing a distance contract by
  setting up a direct m-transaction
• One actor (proposer) sends an e-document
  (e-contract, i.e. text/audio file) to the other
  actor (endorser)
• In case of text files, the e-contract can be
  interactively modified and transmitted back and
  forth until a formal agreement on its contents is
  reached
• To finalize the m-transaction, the endorser
  e-signs the e-contract and sends it to the
  proposer as evidence of formal acceptance of the
  terms contained therein
• Depending on the e-contract type, the proposer
  may also be requested to e-sign the e-contract

9
Person-to-Business (P2B) scenario

• Scenario compatible with SecurePhone
  architecture, but not implemented in the project
• SecurePhone user accesses the server of service
  provider using his browser
• Server sets up an SSL/TLS communication channel
  with strong client authentication
• Browser triggers local authentication, which
  releases private key
• The e-signing of web-based forms is accepted by
  service provider as evidence of agreed e-contracts

10
Project aim 2 biometric verification

• In both P2P and P2B, the user (i.e. a host
  application) needs to locally authenticate in
  order to unlock cryptographic functions and
  access the private key securely stored on the SIM
  card
• PIN- or password-based authentication is
  admissible yet weak and unsatisfactory for
  security-critical applications (e-commerce,
  e-health- e-government)
• Local authentication strengthened in order to
  increase users trust in the system by combining
• WYK a token that only the user knows (signature)
• WYH a token that only the user holds (PDA with
  SIM card)
• WYA biometric identity

11
User verification system

• User requests PDA to verify their identity
• PDA requests user to
• read prompt (face in box)
• sign signature
• Feature processing applied to each
  modalitysilence removal, histogram
  equalisation, MFCC or Haar wavelets, online CMS,
  delta features, etc.
• for each modality S(i)log p(XiC)-log p(XiI)
• if S(i) lt ?(i) for any (i) please repeat
• else fused-score log p(SC) - log p(SI)
• if fused-score gt f user accepted
• else user rejected

12
Multi-modal biometric verification
face
voice
signature
preprocessing
preprocessing
preprocessing
modelling
modelling
modelling
fusion
user profile
client impostor joint-score models
accept userrelease private key
reject user
13
Voice verification (SU / GET ENST)

• Fixed 5-digits prompt conceptually neutral,
  easily extendable, requires few Gaussians
• 22 KHz sampling
• Online energy based non-speech frame removal
• MFCCs with online CMS and first-order time
  difference features slow to compute, but fixed
  point faster than floating point
• Features modelled by 100-Gaussian GMM pdf, with
  UBM for model initialisation and score
  normalisation
• Training on data from 2 indoor and 2 outdoor
  recordings from one session, testing on similar
  data from another session

14
Face verification (BU)

• Static face recognition 10 grey-scale images
  selected at random, 160x192 pixels
• Histogram equalisation and z-score normalisation
  of features
• Haar low-low-4 (or low-high) wavelet features
  fast to compute
• Features modelled by only 4 Gaussian GMM pdf
  UBM used for model initialisation and score
  normalisation
• Training on data from 2 indoor and 2 outdoor
  recordings from one session, testing on similar
  data from another session

15
Signature verification (GET INT)

• Shift normalisation, but no rotation or scaling
• 2D coordinates (100 Hz) augmented by time
  difference features, curvature, etc. total 19
  featuresNote no pressure or angles available,
  since obtained from PDAs touch screen, not
  from writing pad
• Fast to compute
• Features modelled by 100 Gaussian GMM pdf UBM
  used for model initialisation and score
  normalisation
• Training and testing on data from one session

16
Fusion (GET INT)

• For each modality S(i) log p(XiC) - log
  p(XiI)
• LLR score fusion was tested by
• Optimal linear weighted sumFused-score sum
  over i of w(i) S(i)
• GMM scores modelling, i.e. modelling both client
  and impostor joint score pdfs by diagonal
  covariance GMMsFused-score log p(SC) - log
  p(SI)

17
PDAtabase

• After initial development with many databases,
  CSLU/BANCA-like database recorded on Qtek2020 PDA
  for realistic conditions (sensors, environment)
• 60 English subjects 24 for UBM, 18 for g1, 18
  for g2.Accept/reject threshold optimised on g1,
  then evaluated on g2, vice versa
• Video (voice face) 6 x 5-digit, 10-digit and
  phrase prompts2 sessions, with 2 inside and 2
  outside recordings per session
• Signatures in one session, 20 expert
  impostorisations for each
• Virtual couplings of audio-visual with signature
  data (independent)
• Automatic test script allows to test many
  possible configuration
• User just provides executables for feature
  modelling, scores generation and scores fusion

18
Performance on PDA
DET curves for prompts T1 (5 digits, left), T2
(10 digits, middle) and T3 (short phrases,
right) in PDAtabase
19
Performance on PDA
EER R1WER (FAR/FRR) R0.1WER (FAR/FRR) R10WER (FAR/FRR)
T1 2.39 2.40 (1.57/3.24) 1.87 (4.97/1.56) 1.02 (0.43/6.95)
T2 1.54 1.60 (0.89/3.32) 1.37 (3.05/1.20) 0.63 (0.25/4.37)
T3 2.30 2.37 (1.61/3.14) 2.03 (4.54/1.78) 0.92 (0.38/6.34)
Fusion results ( WER, FAR and FRR) for the best
fusion method (Min-Max GMM), for the 3 prompt
types in the PDAtabase
20
Implementation constraints

• PDA main processor is much slower than PC, but
  does speech preprocessing in real time for 22 kHz
  signalsNote speech signal taken directly from
  mic, therefore gt 8 kHz
• Only data on the SIM card is secure, so all
  biometric models must be stored and processed on
  the SIM, which has very limited computational
  resources
• SIM model storage limited to 40 K text-dependent
  promptsNote text-independent prompts or varied
  text-dependent prompts are more secure, but would
  require 200-400 K
• GMM based verification is well suited to integer
  computation
• Enrolment can use only one short indoor session

21
Performance on SIMcard

• SIM processor very slow single verification
  takes 53 minutes!
• Most time goes to voice and signature processing
  these use a large number of frames and models
  with a lot of Gaussians.
• Not acceptable for any practical application.
• Drastic measures needed global processing.
• By using means and standard deviations across all
  parameters for all frames in the
  utterance/signature, the number of frames is
  reduced to one.
• Since the data are much simpler, only a few
  Gaussian mixtures are needed for modelling
• Single verification now under 1 second, but
  performance for T1 is now 10.5 EER.

22
Remarks on PDA selection

• No suitable off-the-shelf products at moment of
  selection fulfilled all SecurePhone requirements
• Limitations of Qtek 2020
• Class B GPRS ? impossible to transmit voice and
  data simultaneously
• Camera is on the rear ? difficulties with video
  acquisition and text prompt reading
• Proprietary video SDK, not freely available ?
  problems with low-level raw image data recording
• Now available Qtek 9000 solves first two
  problems, solution to last problem may be usuable
  with Qtek 9000!

23
Conclusion

• The SecurePhone
• combines secure communication with user
  authentication
• is user-friendly and respects privacy
• does not require special hardware
• enables m-business with legal validity
• can easily be extended to other applications
• delivers proof-of-concept
• has very high performance on PDA, performance on
  SIM must still be improved.

24
http//www.secure-phone.info