What is Field Testing

1
What is Field Testing Why

• The main reason for Field Testing is to verify
  the compatibility of the mobile terminal against
  different networks
• Field testing is carried out in 'live' commercial
  networks because
• Systematic lab tests cannot simulate all CDMA
  network configurations.
• Network functionality varies when comparing
  networks of different vendors.
• Although the network vendor may be the same,
  operators have possibilities to adjust network
  parameters in many ways.
• To provide a quality product by ensuring
• its complete interoperability with different
  infrastructures
• its robustness in good and adverse field
  conditions

2
Field Testing Goals

• Field Testing should fulfill the following
  goals
• To discover such problems that can't be
  found out in systematic laboratory
• testing
• To confirm that all features of the phone
  will work without problems in all major
• cellular networks
• To ensure that the phone works according
  to end user expectations, mainly
• based on user manual
• To ensure that the quality of the product
  will satisfy typical end-user and the
• quality will be better than in the
  competitors phones.

3
Test Categories

• Field test cases are aimed to cover all phone
  features
• Test cases are usually divided into following
  groups
• CDMA specific idle activity
• Voice call handling and quality
• Short Message Service (SMS)
• Supplementary Services (SS)
• Data
• Internet / WAP
• Tests are also divided according to test
  environment
• Stationary tests (good signaling conditions,
  verifies signaling interoperability)
• Tests when moving (Handovers HO, Location
  Updates LU, cell re-selection...)
• Tests in weak field (searching for service,
  call crashing, call success rates...)

4
Field Test Levels

• Level 1 Lab Testing
• Level 2 DEBUG and Feature verification Testing
• Level 3 Local Field Test Network Testing
• Level 4 Field Testing with Different
  Infrastructures / Network

5
Field Test Levels

• Level 1 Lab Testing
• Objective To evaluate the protocol status of
  the SW and the minimum product performance needed
  to begin testing the product on a live network
• Type of Testing
• CDMA Protocol Test Rack
• HP 8924C/E8285A Digital Parametric and Digital
  Call Processing Test
• HP 8924C/E8285A Analog Parametric and Analog Call
  Processing Test
• HP 8924C/E8285A Analog Parametric and Analog Call
  Processing Test
• Mobile Originated Testing on HP8924C/E8285A TOM
  
• Mobile Terminated Testing on HP8924C/E8285A TOM
  
• n. Final UI and Flash and Align
• Minimum Criteria to Pass
• Product must pass Test n, no exceptions.
• Product must pass all critical tests cases in
  Test 1-4

6
Field Test Levels

• Level 2 DEBUG and Feature Verification Testing
• Objective To verify fixed errors in the field,
  new feature implementation and perform minimal
  call statistics
• Type of Testing
• Feature verification will travel to different
  networks if feature is not supported in local
  network
• Error/ Bug fix verification in local field test
  network
• Call Origination and Termination in local field
  test network (no more than 100 calls per phone)
• Call Maintenance in local field test network (no
  more than 5 hrs per phone)
• Minimum Criteria to Pass
• Product must still pass all minimum criteria for
  Field Test Level 1
• Product should have performance at least equal to
  current products being sold

7
Field Test Levels

• Level 3 Local Field Test Network Testing
• Objective To evaluate the stability and
  performance of a product before taking it to
  other networks/infrastructures
• Type of Testing
• All test cases outlined in the Projects Field
  Test Specification
• Call Origination and Termination in local field
  test network (min 100 calls per phone)
• Call Origination and Termination in local field
  test network (no more than 100 calls per phone)
• Call Maintenance in local field test network (min
  10 hrs per phone)
• Minimum Criteria to Pass
• Product must pass all Tests in Field Test Level 1
  
• Product must have no power downs
• Product should have performance at least equal to
  current products being sold
• Product must pass critical test cases in Test 1
  2

8
Field Test Levels

• Level 4 Field Testing with Different
  Infrastructures /Networks
• Objective To evaluate product performance and
  stability versus different infrastructures,
  various network providers, and various markets
• Type of Testing
• All test cases outlined in the Projects Field
  Test Specification
• Call Origination and Termination in local field
  test network (min 100 calls per phone)
• Call Maintenance in local field test network (min
  10 hrs per phone)
• All Features testing that is specific to a
  particular network
• Error/bug fix verification that is specific to a
  particular network
• Minimum Criteria to Pass
• Product must pass all minimum criteria in Field
  Test Level 3
• Product must pass all test cases in Test 1 2
  for all infrastructures
• Product must pass all Features requirement for a
  particular Carrier/Customer
• Product must perform better than current
  products being sold

9
The Field Testing Process
L 3
L 4
L 2
L 2
Preliminary Check
Preliminary Field Testing
Field Testing
Prepared for Preliminary FT
Prepared for FT
Level 1 Lab Testing
Level 2 Preliminary Field Testing
Level 3 Main Countries
Level 3 All Major Countries
Final Mobile
Level 4 Correction verification
Yes
No
Corrections
Corrections
Corrections
Correct-ions
Errors
Errors
Product Development
Errors
Error Correction/Internal Testing
Integration Testing
Regression Testing
10
Field Test Responsible

• Field test responsible is a person who organises
  the FT activities
• Field test responsible will do
• Planning for
• Resources
• Timing (protos, accessories and SW)
• Estimated cost calculation
• Country selection (according network
  infrastructure, subscribers and services)
• Tests to be executed
• Arrangements for
• Field test training
• Field test resources
• Contact persons in target countries and local
  SIM-cards
• Information about network status, vaccination,
  visa, ata carnet, ..
• Co-operation with other projects
• Dedicated plan for each field test trip

11
Field Test Responsible

• Field test responsible will do (cont)
• Reporting for
• test manager (status reports)
• project (errors /usability issues / maturity
  estimations)
• Follow up work for
• found errors (error reproducing at office / fix
  verification)
• error check lists
• FT reports
• FT plan
• HW/SW Updates
• local global field test database
• network news