Title: CARSAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes
1CAR/SAM Regional Guidance Material on Air Traffic
Services Quality Assurance Programmes
- NAM/CAR/SAM Quality Assurance Workshop
- Mark Ríos
- Air Traffic Management Officer
- ICAO NACC Regional Office
NAM/CAR/SAM Air Traffic Services Quality
Assurance Workshop (Port of Spain, Trinidad and
Tobago, 26 to 30 November 2001
2INTERNATIONAL CIVIL AVIATION ORGANIZATION NORTH
AMERICAN, CENTRAL AMERICAN AND CARIBBEAN
OFFICE CAR/SAM REGIONAL GUIDANCE MATERIAL
ON AIR TRAFFIC SERVICES QUALITY ASSURANCE
PROGRAMMES Chapters 5 8 Version
1.0 November 2001
3Contents
- Chapter 5. Air Traffic Incident Reporting,
Investigations, and
Investigation Teams - Chapter 6. Air Traffic Incident Prevention
Programmes - Chapter 7. Air Traffic Services Evaluation
Programme - Chapter 8. Quality of Services Improvement
Programmes
4Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Introduction
- Air traffic incidents
- AIRPROX
- Operational error
- Operational deviation
- Runway incursion
- Pilot deviation
- Casual factor
- Procedural
- Ground facilities (equipment)
- Human
5Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- General reporting procedures
- Reporting by pilots
- Reporting by ATS
- ATS incident report form for use by pilots and
controllers - Investigation and documentation
- Analysis of ATS incident
- Release of information
- In the interest of accident/incident prevention
- Publish report as soon as possible
6Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Air traffic incident investigation process
- Step-by-step process
- Suspected ATS incident event
- Identify and report ATS deficiencies
- That problems could be corrected
- Reporting of ATS incidents
- Requirement within States
- Responsibility of all ATS personnel
- Immediately report suspected ATS incidents to ATS
supervisor or appropriate ATS administration
official - Without fear of reprisal
7Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Preliminary ATS incident investigation
- Fact finding
- Determine what occurred in the ATS system
- Determine validity of the suspected ATS incident
- Confirmation of ATS incident
- Take appropriate action
- Advise other ATS unit if involved
- Relieve controller(s) may be involved
- Gather pertinent information (e.g. flight
progress strips, etc.) - Review voice recordings and prepare working copy
8Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Review computer and radar data
- Conduct preliminary interviews
- Notify ATS management
- Complete preliminary ATS Incident Investigation
Worksheet - Follow-up investigation
- ATS unit chief should
- Ensure ATS incident investigations are conducted
- Coordinate with other ATS unit chief if involved
- Designate ATS incident Investigator-In-Charge
(ATS-IIIC) - Assemble ATS incident investigation team
- Ensure complete, thorough, objective and timely
investigation
9Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- ATS-IIIC
- Conduct complete investigation
- Forward findings and recommendations to ATS unit
chief - Determine scope of other ATS unit investigation
if involved and exchange information - Assign duties to team members
- Conduct interviews in accordance with national
and local requirements - ATS incident investigation team
- Assist ATS-IIIC
- Remain under guidance of ATS-IIIC during
investigation process
10Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Investigation process
- Fact finding
- In-depth inquiry into all causal factors
- Procedures
- Training
- Supervision
- Equipment
- Control room environment
- External factors
- Controller actions
- Airspace configuration
- Traffic flows
- Pilot actions
- Consequence of ACAS/TCAS event
- Meteorological conditions
- Control position configuration
- Coordination procedures
- Aerodrome environment
- Human factors
- Accuracy of automated radar systems
- Radar data
11Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Investigation process (cont..)
- Interviews
- Conduct interviews with all possible personnel
involved or who may have knowledgeable of the
incident - Conduct interview in atmosphere of shared concern
- Consider national and/or local interview
procedures - Allow written comments and recommendations from
interviewee - ATS-IIIC or investigation team should conduct
interviews
12Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Investigation process (cont.)
- Voice recordings
- Review voice recording
- Make certified copy of original recording
- 5 minutes before to 5 minutes after last contact
- Include track time if available
- Written transcript of communications only when
needed - Radar data
- Radar data may be used to validate occurrence of
ATS incident - Written procedures should be developed explaining
process - Consider automation capabilities and accuracy of
data
13Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Reclassification of ATS incident
- What if during the investigation data indicates a
reclassification is warranted? - The incident may be reclassified as
- Pilot deviation
- Other ATS unit deviation
- From operational error to operational deviation
- From operational deviation to operational error
- No occurrence
14Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Return to operational duties
- ATS chiefs should remain involved in the post ATS
incident process to ensure complete and
consistent handling - Reasons for suspension of controllers rating and
assignment of proficiency training - Assist and enhance controllers performance
- Best of his/her abilities
- Meet minimum quality standards
- Improve overall performance and quality of ATS
15Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Return to operational duties (cont.)
- Consider the following when returning a
controller to operational duties - Events leading up to and surrounding ATS incident
- Controllers statement
- Procedure or separation minimum involved
- Voice recordings
- Controllers training record
- Controller proficiency checks
- Proficiency training
- Controller currency on the control position
- Computer data
- Involvement in previous ATS incidents
- Results of interviews and recommendations made by
ATS-IIIC and investigation team
16Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Return to operational duties (cont.)
- Deficiencies in controllers performance
- Determine whether to suspend controllers rating
- Suspension of rating not based solely on
involvement in ATS incident but based on
performance - Determine appropriate actions and refresher
training - Complete refresher training
- Re-certify controller through controller
proficiency check
17Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Final ATS incident reports
- ATS unit chief
- Analyze data submitted by ATS-IIIC and
investigation team in Final ATS Incident
Investigation Report to determine and/or endorse
classification of incident and - Categorization of ATS incident (i.e, procedural,
facility, human factor, etc.) - Causal factors
- Recommendation and corrective actions
- Complete Final ATS Incident Report within 30 days
18Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- ATS incident tracking process
- Annual ATS incidents tracking process
- Will assist CAAs with follow-up investigations
and trend analysis - Sample
- ATS Incident Report XXXX-A-00-001
- ATS Incident Report XXXX-T-00-002
- ATS Incident Report XXXX-F-00-003
- Legend
- XXXX ATS unit identification
- A, T, or F Type of ATS unit (A Area (ACC)
T Approach (APP) and - Aerodrome Control Tower F FIC, AFIS, etc.)
- 00 Last two digits of the year
- 001 ATS incident number in sequence by year
for the unit
19Chapter 5. Air Traffic Incident Reporting,
Investigations, and Investigation Teams
- Training and proficiency records
- Document in training and proficiency records
- ATS incident records retention
- Determine retention period
- 2 ½ years is recommended
- National ATS authority responsibilities
- Maintain central source of ATS incident data
- Review final ATS incident investigation reports
- Develop and distribute annual ATS incident
analysis report - Determine effectiveness and efficiency of
programme - Maintain liaison with ATS unit chiefs
- Maintain oversight of ATS incident reports
20Chapter 6. Air Traffic Incident Prevention
Programmes
- Introduction
- Establish policies and programmes that result in
quality incident free ATS - Requires commitment of all ATS personnel
- Consistently perform duties and responsibilities
in accordance with ICAO SARPs, CAA and ATS unit
directives and procedures - Accept responsibility for maintaining the safest
and highest levels of operational performance - Goals/objectives
- ATS incident free environment
- Safe, orderly, and expeditious ATS to users
- Everyone is responsible for ATS incident
prevention - Know, apply, and adhere to requirements in
performance of operational duties and
responsibilities
21Chapter 6. Air Traffic Incident Prevention
Programmes
- ATS incident prevention
- Many initiatives can be pursued
- Five primary areas
- Communications
- Phraseology
- Supervision
- Teamwork
- ATC proficiency
22Chapter 6. Air Traffic Incident Prevention
Programmes
Prevention objectives
- Identify activities/efforts which enhance
individual, ATS unit, and system performance - Recognize commendable activities/efforts
- Identify deficiencies (individual, equipment,
procedural) - Promptly correct deficiencies
- Ensure actions taken corrected the deficiencies
or problems - Initiatives to ensure effective controller
performance, teamwork, communications - Communicate performance expectations
- Maintain summary of ATS incidents and incorporate
into training - Brief ATS personnel on ATS incidents
- Monitor/evaluate voice recordings
- Review local/national/regional ATS incidents
- Annual QA review
- ATS unit staff meetings
- Improve communications among all ATS personnel to
create atmosphere conducive to sharing
information - Attendance to local, national, regional QA
seminars, conferences, etc.
23Chapter 6. Air Traffic Incident Prevention
Programmes
ATS supervisors
- Communicate performance expectations with
controllers - Take prompt action when performance does not meet
expectations (on-the-spot corrections, etc.) - Address individual/team accountability
- Efficient/consistent oversight of ATS unit
operation - Ensure distractions and noise levels in ATS unit
are kept at a minimum
- Require personnel to maintain high degree of
professionalism, teamwork, control position
discipline, awareness - Promote open flow of communications
- Conduct ATS unit team meetings
- Place emphasis on hearback/readback
- Keep ATS unit chief/support staff informed
(operations concerns, etc.)
24Chapter 6. Air Traffic Incident Prevention
Programmes
ATS quality assurance officers/specialist
- Random voice recording reviews
- Brief ATS unit on trends and causal factors
related to ATS incidents, random recording
reviews, evaluations, etc. - Maintain awareness of ATS operational environment
- Provide staff expertise and support
- Assist in ensuring QA goals and objectives are
being met
- Ensure scenarios involving hearback/readback
errors are included in training - Prepare/distribute ATS incident summaries to all
ATS personnel - Ensure ATS unit maintains summary of ATS units,
causal factors, trends, etc. - Incorporate into classroom and annual proficiency
training
25Chapter 6. Air Traffic Incident Prevention
Programmes
ATC personnel
- Keep ATS supervisors advised of traffic problems
and equipment limitations - Suggest improvements to ATS unit and/or incident
prevention - Maintain situational awareness
- Assist busier control positions
- Assist busier control positions
- Review own operating techniques and ATS unit
procedures to effect highest quality of
performance - Promptly report ATS incidents
- Utilize memory aids
26Chapter 6. Air Traffic Incident Prevention
Programmes
- Voice recording evaluations
- To ensure proper phraseology, good operating
practices, adherence to ICAO standards,
national/local directives and practices - Guidelines
- Conduct semi-annually on all ATS operational
personnel - Review of voice recording by ATS supervisor/ATS
QA specialist - Document comments
- Develop action plan to address performance
deficiencies - ATS supervisor/QA specialist review and discuss
voice recording with controller
27Chapter 6. Air Traffic Incident Prevention
Programmes
- Review of ATC procedures and practices that may
contribute to ATS incidents - Constant review of ATC procedures and practices
- Identify, report, recommend, and implement
amendments - Ensure safety of aviation and prevention of ATS
incidents
28Chapter 6. Air Traffic Incident Prevention
Programmes
- ATS incident review groups
- Implement ATS incident review groups
- National/local levels
- Purpose
- Periodic reviews of prior ATS incidents
- Identify other potential problem areas requiring
attention - Effort to prevent ATS incidents
- Composition
- ATS unit chief
- Controllers
- Other ATS staff
- ATS unit QA specialist
- Pilots/airlines when appropriate
29Chapter 6. Air Traffic Incident Prevention
Programmes
- Safety information sharing networks and
international organizations - Several safety information networks available
- Facilitate free exchange of information of actual
and potential safety deficiencies - Expanding focus to include air traffic control
services - Goals
- Collect data
- Share information
- Assist with identification and prevention of
incidents and accidents
30Chapter 6. Air Traffic Incident Prevention
Programmes
- Safety information sharing networks and
international organizations (cont.) - ADREP
- ICAO Accident/Incident Reporting Manual, Doc 9156
- Global Aviation Safety Plan (GASP)
- Endorsed by 32nd Session of the ICAO Assembly
- Objectives
- Safety initiatives that offer best safety
dividend - Annual reviews of accident statistics
- Decrease in worldwide accident rate
- Enhance identification of short comings and
deficiencies - Increase and improve ICAOs own capability to
compile, assess and disseminate safety-related
information - ICAO Website www.icao.int
31Chapter 6. Air Traffic Incident Prevention
Programmes
- Global Aviation Information Network
- To encourage the sharing of aviation safety data
worldwide in an effort to further reduce
accidents - Plans to expand programme to include air traffic
control - Website www.gainweb.org
- Aviation Safety Reporting Programme (ASRP)
- Reporting programme designed to identify unsafe
operating conditions - Deficiencies and discrepancies
- Limited immunity
- Voluntary and confidential
32Chapter 6. Air Traffic Incident Prevention
Programmes
- PAN American Aviation Safety Team (PAAST)
- Multi-organizational team
- Pool essential resources
- Addresses priority safety areas
- Deliver practical products to aviation community
in Latin American and Caribbean Regions - Participation in international organization study
groups to analyze and recommend solutions to ATS
incidents is encouraged
33Chapter 6. Air Traffic Incident Prevention
Programmes
- Runway incursion prevention plan
- Increase in runway incursions
- Aerodrome operations increasing throughout the
world - Potential for runway incursions increase
- Need for prevention plan which specifically
addresses runway incursions - Strategies
- Position relief briefings
- Position relief checklist
- Proficiency training Prevention of runway
incursions - Feedback
34Chapter 6. Air Traffic Incident Prevention
Programmes
- Runway incursion prevention plan (cont.)
- Memory aids
- Use of flight progress strips/boards
- Runway in use sign
- Review usage of memory aids on semi-annual basis
- Procedures
- Review aerodrome movement operating practices
- Designate runway crossing point(s)
- Place special emphasis on safe runway crossings
- Use of proper procedures in runway crossings
including interphone procedures, runway in use
sign by ATS supervisors and controllers
35Chapter 6. Air Traffic Incident Prevention
Programmes
- Procedures (cont.)
- Avoid use of conditional clearances except as
authorized by ICAO SARPs - Certain conditional clearances are permitted
however, they should be kept to a minimum, the
exception rather than the norm should be
practiced, and under extreme caution - Include runway check procedures in letter of
agreement - Define vehicular movement procedures/areas in
letter of agreement - Vehicular movements should be restricted to
perimeter roads, vehicle access roads and other
non-movement areas - Describe closed runway procedures
36Chapter 6. Air Traffic Incident Prevention
Programmes
- Quality Assurance Review (QAR)
- Essential to the effectiveness of the ATS System
and prevention of ATS Incidents that all
incidents be investigated - QARs provide for identification, investigation
and resolutions of these incidents through
corrective training of controller performance
deficiencies
37Chapter 6. Air Traffic Incident Prevention
Programmes
- Quality Assurance Review (QAR) (contd)
- Conduct QARs when
- Aircraft accidents
- Other incidents not categorized as ATS incidents
- ACAS/TCAS event
- Other miscellaneous incident reports that involve
loss of separations - Aircraft go-arounds
- Public inquiries regarding ATS
38Chapter 6. Air Traffic Incident Prevention
Programmes
- Quality Assurance Review (QAR) (contd)
- Process
- Designate person to conduct QARs
- Conduct QAR in depth so as to assess controller
performance - Accomplish corrective training for all identified
controller performance deficiencies - Communicate QAR conclusions
- Record notification and conclusion of QAR in ATS
unit daily operations log
39Chapter 7. ATS Evaluation
- Introduction
- Chapter deals with need for constant and
continuous evaluation of individual ATS units and
of the overall ATS system - Purpose and scope
- Ensure that provision of service is maintained at
the highest standard - Units/personnel apply polices, standards, rules,
procedures and separation minimum in approved
manner
40Chapter 7. ATS Evaluation
- ATS evaluation activities include
- Assessing service provided to users
- Ensuring operating procedures conform with
standards - Assessing operational requirements
- Identifying potential unsafe procedures or
operating practices - Detecting problem areas or deficiencies
- Examining effectiveness of intra/inter-unit
communications and coordination - Examining personnel utilization, position
workload and establishments to ensure
compatibility - Recommending corrective actions
41Chapter 7. ATS Evaluation
- Conduct of evaluation
- Designate team of experienced personnel to
conduct ATS evaluations - At least once every two years
- Notify ATS unit chief prior to conducting
evaluation - Meeting at completion of evaluation with ATS unit
chief to discuss significant findings and
recommendations - Prepare written report ATS evaluation team
- Prepare action plan ATS unit
42Chapter 7. ATS Evaluation
- ATS evaluation procedures
- This section provides standardized procedures for
evaluating compliance with ICAO SARPs and
specified directives and procedures at the
national and local ATS unit level within a State - Evaluation process
- Full unit ATS evaluation
- Preparation and notification
- Use ATS Unit Evaluation Checklist
- Notify ATS unit chief 30 days prior to evaluation
- Request data from ATS unit for pre-evaluation
review - Solicit special interest items that ATS unit
would like assessed
43Chapter 7. ATS Evaluation
- Evaluation process (cont.)
- In-briefing
- Introduce team members
- Evaluation schedule and activities
- ATS unit chief staff
- Conducting the evaluation
- Direct observation
- Control position and/or voice recording or data
monitor - Attendance at staff meetings
- Observation of training
- Review of administrative records
- Interviews/discussions
- Review of in-flight/preflight evaluation reports
44Chapter 7. ATS Evaluation
- Evaluation process (cont.)
- Daily briefings
- Progress of evaluation
- Provided by lead ATS unit evaluator to ATS unit
chief - Out-briefing
- Briefing of ATS evaluator/teams findings
- Attendance
- ATS unit chief
- ATS unit personnel encouraged
- Present draft copy of ATS evaluation report
- Evaluation critique
- Critique form to be completed by ATS unit chief
45Chapter 7. ATS Evaluation
- Follow-up evaluations
- Preparation and notification
- Unannounced or minimum notification
- On-site evaluation
- Desk audit
- Combination of both
- Conduct no earlier than 6 months after full unit
ATS evaluation - May request data for pre-evaluation review
- Basically follow same process as full unit ATS
evaluation
46Chapter 7. ATS Evaluation
- In-flight evaluations
- Conduct at least 2 per year on each ATS unit
- Conducted by ATS evaluators and other authorized
ATS personnel - Use airlines or other operators in which an
agreement to conduct such observations is in
place - The following air traffic services may be
assessed - Air traffic services reporting office
- Automatic terminal information service
- Aerodrome control service
- Approach control service
- Air traffic advisory service
- Other services deemed appropriate
47Chapter 7. ATS Evaluation
- Special evaluations
- May be conducted when deemed necessary by the ATS
authority or if requested by the ATS unit - Evaluation reports
- Document results of all ATS evaluations
- Complete and distribute final ATS evaluation
reports within 30 days of the out-briefing - Contents of final ATS evaluation reports
- Results of evaluation pertaining to areas
assessed - Describe reportable items
- Assign tracking control numbers to all items
identified
48Chapter 7. ATS Evaluation
- Responses to ATS unit evaluations
- Required on all items rated less than
satisfactory - Three-step closure process
- Action plan
- Develop and forward action plans to appropriate
ATS authority within 30 days of receiving final
ATS unit evaluation report - First response
- Completed and distributed to ATS authority with
60 days of ATS unit out-briefing - Second response
- Completed and distributed with 180 days after
date of the ATS unit evaluation out-briefing and
every 180 days thereafter until all items are
closed
49Three-step Closure process
- Three-step closure process. The three-step
closure process is the method by which less than
satisfactory items of an evaluation are to be
closed. The required responses should be
available after 60 days and 180 calendar days and
should describe the three steps as follows - a) Corrective action. The initial action taken
by the ATS unit to correct the discrepancy - b) Follow-up action. The action taken over a
period of time to validate that the initial
action corrected the discrepancy. Include the
date(s) when this was accomplished and the
results and - c) Management control. The action taken by the
ATS Authority or ATS Unit that will remain in
place to ensure the problem does not reoccur.
The action should include identifying those posts
within the ATS unit that have responsibility to
periodically review the corrected discrepancy and
when the review will be accomplished.
50Chapter 8. Quality of Services Improvement
Programmes
- Introduction
- Initiatives that can improve the overall quality
of air traffic services being provided - Random voice recording reviews
- ATS unit quality assurance officers/specialists
should complete several random voice recording
reviews on a monthly basis - Ensure overall quality of ATS is maintained
51Chapter 8. Quality of Services Improvement
Programmes
- ICAO phraseology improvement programme
- Implement ATS unit phraseology improvement
programme - Benefit to pilots and controllers
- Results can improve quality of service and
prevention of ATS incidents - Method
- Random voice recording reviews
- Voice recording monitor evaluations
- Direct observations
- Recognition of ATS personnel who demonstrate
outstanding phraseology or significant improvement
52Chapter 8. Quality of Services Improvement
Programmes
- ATS customer service/feedback
- Important to establish good communications
between ATS authorities/units and users - Users can offer valuable feedback
- Feedback from others organizations/departments/int
ernally - Obtain feedback through surveys
- ATS quality assurance survey
- Conduct annually
- External
- Internal
- Analyzed and validate data collected
- Share results with ATS personnel
- Issues affecting quality of service should be
addressed
53Chapter 8. Quality of Services Improvement
Programmes
- Pilot/controller user forums
- Conduct annually
- Can produce good relations and enhance
communications between ATS authorities, pilots
and controllers - Main objective
- Bring together people in the ATS system that work
together on a daily basis - Pilot on the flight deck
- Controller in the control room/tower
- Better understanding of each others
responsibilities and duties - Avoid conducting in a meeting type format
- Can also be used to present and explain
information on the local or national ATS system
and procedures
54Chapter 8. Quality of Services Improvement
Programmes
- Participation in pilot safety seminars
- Participation of ATS authorities in pilot safety
seminars - Present information regarding the ATS system,
safety and quality assurance - Pilot visits to ATS units
- Encourage pilot visits to ATS units
- ATS system familiarization
- ATS system familiarization/education for pilots
- Consider developing an ATS system education
programme for pilots - Education pilots in how to best utilize the ATS
system - Functions
- Responsibilities
- Benefits
- Services
55Chapter 8. Quality of Services Improvement
Programmes
- Familiarization training flights for ATS
personnel - Establish programmes with airlines
- ATS supervisors and controllers should be
encourage to participate - Allows ATS unit personnel to experience first
hand the activities on the flight deck - View programme as proficiency training
56Chapter 8. Quality of Services Improvement
Programmes
- Recognition of quality performance
- Recognition of positive quality performance is as
important as identifying deficiencies - Recognition as individuals or as a team
- Providing a high standard of performance and
quality of service
57Chapter 8. Quality of Services Improvement
Programmes
- ATS performance measurements
- Important that ATS providers find ways to
continually improve the safety and efficiency of
ATS services - Factors when measuring performance and quality of
ATS services - Safety
- Delay
- Predictability
- Flexibility
- Efficiency
- Availability
- Access
- Cost of Service
- Runway Capacity
58THANK YOU
THE END