Testing and Certification of Biometric Components and System in Europe a report on the intermediate findings of the BioTesting Europe Project

1
Testing and Certification of Biometric
Components and System in Europe a report on the
intermediate findings of the BioTesting Europe
Project

• Maria Margarida Castro NevesFraunhofer IGD,
  Germany
• Margarida.Castro-Neves_at_igd.fraunhofer.de

2
Agenda

• About the BioTesting Europe project
• Identified EU needs for testing in biometrics
• Issues Gaps in testing capabilities
• Improving EU capabilities for assuring
  performance

3
BioTesting Europe

• Project details
• 9 month project finishing by Dec 2007
• Supporting Activity under Preparatory Actions
  for Security Research
• Partners
• European Biometrics Forum (coordinator)
• National Physical Laboratory (UK)
• Fraunhofer IGD (Germany)
• EC/JRC Ispra (Italy)
• Objectives
• Consult to determine EUs needs for testing of
  biometrics (Inventory)
• Identify where improved testing capabilities
  required (Gap Analysis)
• Prepare work plan/roadmap of coordinated actions
  to further develop biometrics testing and
  certification capabilities
• Define the business case for testing

4
European Approach

• This is why national governments / authorities
  should support a European approach for testing
  certificates
• The vendors would not survive to pay for 27
  national tests/certificates
• Not-testing (before installing) would undermine
  the EU-widesecurity policy for the border
  control process
• We need to provide a comparable security at all
  border control points along the EU perimeter
• Vice-Versa recognition works (well)
  forCC-certification. It should work alsofor
  Biometric Performance certification!

5
Project scope

• Stakeholders consulted
• Suppliers
• Vendors
• System Integrators
• Operators
• end customer
• Test organisations
• Independent 3rd party labs
• In-house test labs
• Certification authorities
• Academics

• Applications considered
• (Criteria relevance and urgency)
• Passports
• AFIS
• Visas (VIS BMS)
• Identity documents
• Registered traveller
• Potential Scope
• Systems
• Sub-systems
• Devices
• Processes
• Personnel (training education)

6
Questions to be answered

• What testing is needed?
• Which components should be certified?
• Who should perform these tests?
• What standards are applicable?
• What do we already have what needs to be
  developed ?
• What RD is needed?
• What are the costs and who will pay/invest?
• Inventory based on 38 Questionnaires

7
Example e-borders

• What needs testing for e-Passports and border
  control e-Gates?
• Qualities of enrolment
• Procedures
• Operating environment
• Interoperability
• Efficiency at the border
• Throughput
• Accuracy
• Accessibility
• Usability
• Consistency of processes

8
Testing needed / Tests conducted
Needs to be tested Who tests Comments
Operators Suppliers Test-labs Operators Suppliers Test-labs Operators Suppliers Test-labs
Performance S T Component level tests
Performance O (Sub-)System level tests
Accuracy 11 O S T
Accuracy 1N (with large N) O S Need v.large databases
Failure to Enrol/Acquire O T Need representative population environment
Throughput O S T
Interoperability T E.g. MINEX, MTIT
Conformance
Data format (levels12) S T Some test tools
Data format (level 3 semantic level) S? No methodologies / reference data
API O S
9
To be tested Who tests Comments
Biometric data quality S T Standards being developed
Software kit to assess data quality ? Validation / Calibration needed?
Sensor testing
Quality Conformance S E.g. Appendix F
Sensor ruggedness S T traditional type of test
Production quality S? Are all sensors the same quality as the tested/certified one
Usability / Accessibility O? Not tested to any standard
Security
Anti-spoofing S T Few products tested under CC
Data protection T? Similar to security audit
Safety S T CE plus?
Personnel O
10
Observations

• Testing is carried out by Suppliers, Operators,
  and Test Organisations
• Mostly by suppliers operators
• Most current test needs are being addressed
• By ad-hoc means rather than using standard schema
  / references
• 3rd party tests certification will be
  complementaryto suppliers and operators tests
• Suppliers will test during development
  production
• Operators need to test on their own data
• Helps us understand our system
• Standard tests certification must meet real
  needs
• Certify against applicable levels of performance,
  test scenario, etc.
• Must be a return on investment in carrying out
  the tests

11
Observations / Gaps

• Fragmented approach to testing
• Few common requirements identified
• Disconnect between component-level tests
  system-level tests
• Component-level performance not predictive of
  system-level performance
• No methodologies / standards for some key areas
  of testing
• Usability/Accessibility (of particular EU
  interest)
• Level-3 conformance to data format standards
• i.e. is the record an accurate representation of
  the characteristic
• Biometrics not a mature technology still many
  unknowns about performance
• E.g. long-term performance of face, fingerprint,
  iris
• Ageing of face compared to photo image over
  lifetime of passport
• Performance expectations fingerprinting children
  (age limits)

12
Observations / Gaps

• Usability and Accessibility
• Diverse concepts for Human-Computer-Interface
  (HCI) among vendors, creating confusion for data
  subjects
• Standardization of usability related issues is
  not progressed far ISO 24779 (Icons Symbols)
  is in early Working Draft status
• RD How can we separate out usability impacts on
  biometric performance?
• Need for test data
• Determining high accuracy requires a lot of data
• Data protection legislation often prevents
  sharing/saving data
• Release of any data may compromise its use in
  testing
• Possible Technical Solutions
• Possibility to consider synthetic data?
• If the test data can not travel to the
  System-Under-Test could the system travel to the
  data?

13
Improving test capabilities

• Broader access to test results
• Show a more complete picture of performance
• Consistent presentation may assist operators
  understanding of results
• Use standards in development for machine readable
  test results ?
• Biometric testing API
• Easier to implement tests
• Common test tools
• Running performance trials
• Analysis visualisation of results

14
Organisational structures (under consideration)

• Do we need a network of test organisations?
• European International?
• Which existing institution can take the role of
  an accreditation body?
• Criteria for including a test laboratory in such
  a network?
• Which type of labs are accepted
• Governmental lab / Independent lab
• Consultant / integrators lab
• Industry lab
• No closed group - transparent conditions needed
• What are the criteria that a lab drops out of the
  network

15
Organisational structures (under consideration)

• As resources are limited - where should the focus
  of testing be?
• Biometric Performance testing
• Protocol testing (according to SC17.3 work)
• Security testing along Common Criteria
• What role for Qualified product lists /
  certification?
• Some performance aspects better suited to
  certification than others
• Conformance to standard
• Interoperability
• FAR/FRR too dependent on target
  population/environment
• Scope of certificate
• Application specific?
• Duration?

16
Conclusions

• BioTesting project underway
• Project finishes soon, but comments/opinions
  welcomed
• Testing of usability issues is becoming urgent to
  achieve desired levels of performance
  interoperability
• Focus of test and certification seems certain to
  change as industry matures

17
Further information

• Contact points
• max.snijder_at_eubiometricsforum.com
• 31 624 603809 (direct)
• 353 1 488 5810 (secretariat)
• tony.mansfield_at_npl.co.uk
• 44 20 8943 7029
• christoph.busch_at_igd.fraunhofer.de
• 49 6151 155 536
• margarida.castro-neves_at_igd.fraunhofer.de
• 49 6151 155 535
• Website
• www.biotestingeurope.eu