ICAO Accident Prevention Programme

_______________________________________________

2005

International Civil Aviation Organization

FOREWORD

The First Edition of the ICAO Accident Prevention Manual (Doc 9422) was published in 1984 and has proven to be useful to States and the aviation community for developing and maintaining accident prevention programmes. For the most part, the principles and concepts for accident prevention outlined in the First Edition are as valid today as in 1984. However, since its initial print, developments in civil aviation have had a significant impact on aviation safety, for example: — Widespread deregulation of the aviation industry, resulting in an extremely competitive business environment and significant growth in the volume of travel; — Privatization of many State-owned and operated airlines and many services previously provided by the State (such as Air Traffic Control and Airport Management); — Technological advances in the design and reliability of aircraft and the other equipment used in the provision of aviation services; — Improved understanding of the role of Human Factors in accident causation; and — Increasing acceptance by management of the role of organizational factors upon safety, etc. Within this changing context, many accident prevention lessons from other fields of industrial safety have been adapted for use in aviation. Innovative development and application of new methods continue to gain widespread acceptance. What was often controversial twenty years ago (such as Crew Resource Management) is accepted as incontrovertible today. The industry seeks increased efficiency and safety and is ever mindful of any developments that will assist in this pursuit. Examples of improving accident prevention methodologies include: — — — —

Widespread collection and analysis of safety-related data; Use of voluntary, non-punitive incident reporting systems; Proactive (as opposed to reactive) accident prevention measures; International cooperation and collaborative efforts with all sections of the industry in the development of standard procedures and the sharing of safety-related information; — Increased understanding of the integrated nature of system safety and a consequent reliance on Safety Management Systems; and — Use of Risk Management methods for the identification of safety hazards, and the assessment and control of the attendant risks. Recognizing these developments, a draft Second Edition of the ICAO Accident Prevention Manual was produced in 2004. In the same year, two other draft ICAO manuals were developed, one addressing safety management systems (SMS) in air traffic management and the other, SMS in aerodrome operations. As the three draft manuals addressed safety management, albeit from different perspectives, it was decided to combine the three draft manuals into one manual called the ICAO Safety Management Manual. As the second draft of the Accident Prevention Manual (APM) was believed to contain valuable information for the establishment and operation of an accident prevention programme, and not all of this information would be incorporated in the new safety management manual, it was decided that the ICAO Accident Investigation and Prevention Section would, in the interest of accident prevention, make the draft APM freely available on its website and via a CD. As ICAO was not publishing this material, it could no longer be called an ICAO manual. The document has therefore been renamed as the ICAO Accident Prevention Programme.

i

Although many of the accident prevention principles and concepts in the document have evolved from flight operations, most have potential application in other areas such as cabin safety, aircraft maintenance, ground handling, air traffic services and aerodrome management. As such, this document should be useful to States and the aviation community in developing and maintaining their accident prevention programmes. While much of this document will be of use to safety specialists (accident prevention advisers), it aims to go further and addresses accident prevention activities to line management and other high-level decision makers. The material in this document is neither exhaustive, nor prescriptive. Users are encouraged to expand and adapt its concepts to suit their own requirements or needs. Creation of such a document would not have been possible without substantial reliance on numerous external sources — all motivated by accident prevention and advancing aviation safety. It would not be possible to identify all those who have contributed in one way or another to the compilation of this work. However, a few major sources merit mention: — Aircraft manufacturers (notably the Boeing Aeroplane Company and Airbus); — The International Air Transport Association (IATA) and its member airlines; — Civil aviation authorities (in particular the Joint Aviation Authority, the Federal Aviation Administration of the United States, the Civil Aviation Authority of the United Kingdom, Transport Canada and the Civil Aviation Safety Authority of Australia); — Major Professional associations (such as the International Federation of Air Line Pilots’ Associations (IFALPA) and the International Federation of Air Traffic Controllers’ Associations (IFATCA); — The Global Aviation Information Network (GAIN); and — The published works of many researchers and academics dedicated to improving aviation safety from around the world.

___________________

ii

TABLE OF CONTENTS

Chapter 1.

Overview

Chapter 2.

Roles and Responsibilities for Accident Prevention

Chapter 3.

Safety Basics

Chapter 4.

Managing Safety

Chapter 5.

Risk Management

Chapter 6.

Incident Reporting Systems

Chapter 7.

Flight Data Analysis Programmes

Chapter 8.

Line Operations Safety Audits (LOSA)

Chapter 9.

Management of Safety Information

Chapter 10.

Safety Analysis, Studies and Surveys

Chapter 11.

Information Exchange

Chapter 12.

Assessing Safety Performance

Chapter 13.

Establishing an Accident Prevention Programme

Chapter 14.

Practical Considerations for Operating an Accident Prevention Programme

Chapter 15.

Investigating for Accident Prevention

Chapter 16.

Emergency Response Planning

Chapter 17.

Accident Prevention in the Cabin

Chapter 18.

Accident Prevention in Air Traffic Services (ATS)

Chapter 19.

Accident Prevention at Airports

Chapter 20.

Accident Prevention in Aircraft Maintenance

Appendix

Glossary of Terms

Bibliography ___________________

Chapter 1 OVERVIEW

Accident Prevention • General • Need for accident prevention (Cost of accidents, etc.) • Stakeholders in safety • Scope of accident prevention: (Flight safety, Ground safety, vs. Environmental safety, Occupational health and safety, etc.) • Approaches to accident prevention — Traditional — Contemporary • ICAO requirements ICAO Accident Prevention Programme • Objective • Target audience • Gender • Relationship to other ICAO documents • How to use the Accident Prevention Programme

1-1

This page intentionally left blank.

1-2

Chapter 1 OVERVIEW

The true value of safety is often only appreciated in its absence.

ACCIDENT PREVENTION General Aviation is remarkable for the giant technological leaps it has made over the last century. This progress would not have been possible without parallel achievements in the control and reduction of aviation’s safety hazards. Given the many ways that aviation can result in injury or harm, those involved with aviation have been pre-occupied with preventing accidents since the earliest days of flying. Through the discipline of “flight safety,” the frequency and severity of aviation occurrences have declined significantly. From an operational perspective, few industries enjoy the outstanding safety record of the air transportation industry. However, flying activity in scheduled and non-scheduled operations is expected to double over the next two decades. Unless there is a reduction in current accident rates, this increase in traffic will result in a significant increase in the number of major accidents globally. Despite the industry’s enviable safety record, a significant increase in the number of major accidents may cause the public’s confidence in aviation to be seriously undermined.

Need for accident prevention Although major air disasters are rare events, less catastrophic accidents and a whole range of incidents occur more frequently. These lesser safety events may be harbingers of underlying safety problems. They provide evidence of conditions ripe for failure. Ignoring the underlying safety hazards that facilitate such events can pave the way for an increase in the number of more serious accidents. Accidents (and incidents) cost money. Although purchasing “insurance” can spread the costs of an accident, accidents make bad business sense. Insurance may cover specified risks (direct costs), but there are many uninsured costs. For an airline, these uninsured costs may vastly exceed the insured costs, including the downtime (i.e. loss of revenue), system re-scheduling costs and less tangible costs such as the loss of confidence of the travelling public. An understanding of these uninsured (or indirect) costs is fundamental to understanding the economics of safety. In addition to the financial costs, aviation accidents can also demand an enormous social toll. These costs are less tangible. The grief resulting from the loss of relatives or friends and the costs to society resulting from the loss of skilled and valued members, are not readily quantifiable. The air transportation industry’s future viability may well be predicated on its ability to prevent accidents and sustain the public’s perceived comfort regarding their safety while traveling. Safety is therefore a prerequisite for a sustainable business. It is also a matter of ethics.

1-3

Stakeholders in safety Given the total costs of aviation accidents, many diverse groups have a stake in preventing accidents. Historically, flight safety was the domain of pilots. Today, a much broader perspective is required if industry-wide preventive measures are to be effective. Many players outside the cockpit create conditions or hazards that may compromise safe flight operations. Following are the principal stakeholders in safety: a) Aircraft owners and operators; b) Manufacturers, (especially airframe and engine manufacturers); c) Aviation regulatory authorities (e.g. CAA, FAA, EASA); d) Industry trade associations (e.g. IATA, ATA); e) Professional associations and unions (e.g. IFALPA, IFATCA); f) International aviation organizations (e.g. ICAO); g) Investigative agencies (e.g. US NTSB); and h) The flying public. Major aviation occurrences, particularly those in which there is an investigation, invariably involve additional groups with an interest in accident prevention. For example: a) Next of kin, victims, or persons injured in the accident; b) Insurance companies; c) Travel industry; d) Safety training and educational institutions (e.g. Flight Safety Foundation); e) Other government departments and agencies; f) Elected government officials (i.e. politicians); g) Investors; h) Coroners and police; i)

Media;

j)

General public;

k) Lawyers and consultants; and l)

Diverse special interest groups.

1-4

These stakeholders may often have unique expectations of those responsible for the management of aviation safety. They may not always share a common objective in advancing aviation safety. Other motives may drive them.

Scope of accident prevention For many years, “flight safety” was the term frequently used to describe accident prevention activities in aviation. Today, accident prevention is considered to be multi-disciplinary, embracing many distinct activities and levels of authority, from the front line worker to senior management. The major operational areas addressed in this manual include: a) Flight operations; b) Airworthiness; c) Cabin safety; d) Air traffic services; and e) Aerodrome operations, including ground handling and aircraft servicing; etc. Accident prevention in aviation is sometimes considered to include occupational safety and health (OSH) and environmental issues. Also, security issues are sometimes considered as safety issues. However, this manual does not directly address issues of security, environmental safety, or occupational safety and health (OSH).

Approaches to accident prevention With global aviation activity forecast to continue to rise, there is concern that traditional methods for preventing accidents will not be sufficient to keep the number of accidents at a level that is acceptable to society. New methods for understanding safety and taking preventive actions are therefore evolving. Accident prevention may be approached from two different directions: a) The traditional approach which responds to particular safety events; and b) The contemporary approach which actively seeks out those conditions which might enable a safety event, and takes appropriate action to reduce the risks — before an accident confirms the existence of a safety problem. Traditional Approach. Traditionally, flight safety focused on compliance with and enforcement of increasingly complex regulatory requirements. This approach worked well up until the late 1970s when the accident rate levelled off with little subsequent improvement in the industry’s “level of safety”. Accidents continued to occur in spite of all the rules and regulations.

1-5

The traditional approach to safety, sometimes called reactive safety, reacts to undesirable events by prescribing measures to prevent recurrence. Rather than defining best practices or desired standards, this approach is really based upon ensuring minimum standards are met. With an overall fatal accident rate in the vicinity of 10-6 (i.e. one fatal accident per million flights) further safety improvements are becoming increasingly difficult to achieve using this approach. Contemporary Approach. In order to keep accident numbers at an acceptable level, accident prevention needs to shift from a reactive mode to a proactive mode. In addition to a solid framework of legislation, regulatory requirements, approved procedures, and the enforcement of those requirements, a number of other factors are considered to be effective in preventing accidents. Some of these are listed below: a) Senior management’s commitment to the company safety programme; b) A corporate safety culture that fosters safe practices, encourages safety communications and actively manages safety with the same attention to results as financial management; c) Effective implementation of Standard Operating Procedures (SOPs), including the use of checklists and briefings; d) A non-punitive environment to foster effective incident and hazard reporting systems (at both company and national level); e) Systems to collect, analyse, and share safety-related data arising from normal operation (gathered through such programmes as Flight Data Analysis (FDA) and Line Operations Safety Audits (LOSA); f) Application of scientifically-based, risk management methods; g) Competent investigation of accidents and serious incidents, identifying systemic safety deficiencies (rather than just targets for blame); h) Integration of safety training (including Human Factors training) into training programmes for operations personnel; i)

Sharing safety lessons learned and best practices through the active exchange of safety information (among companies and States); and

j)

Systematic safety oversight programmes aimed at assessing safety performance and reducing or eliminating emerging problem areas.

No single element will provide the necessary safety net to meet today’s expectations for accident prevention. Rather, an integrated application of most of these elements will significantly increase the aviation system’s resistance to unsafe acts and conditions in the workplace. However, even with an effective accident prevention programme, there are no guarantees that all accidents can be prevented.

1-6

ICAO requirements ICAO requires that operators establish and maintain an accident prevention and flight safety programme.1 Some of the basic ingredients of an effective accident prevention and flight safety programme are specified in Annex 132. For example, requirements for: a) Incident reporting systems; b) Database systems; c) Analysis of data and preventive action; and d) Exchange of safety information. The accident prevention and flight safety programme should be documented in a company’s operation manual, including a statement on the company’s safety policy and the responsibility of personnel. To ensure the necessary focus, the safety programme is often contained in a separate volume of the company operations manual. Recognizing the need for an integrated approach to accident prevention, ICAO has also introduced requirements for the implementation of safety management systems in the areas of Air Traffic Services and Aerodromes management.

ICAO ACCIDENT PREVENTION PROGRAMME Objective The objective of this Accident Prevention Programme is to: a) Outline contemporary accident prevention concepts and methods, thereby increasing awareness of the many facets of accident prevention today; b) Provide examples of practical applications, thereby assisting stakeholders in safety to establish or improve their accident prevention programmes; and c) Foster an exchange of ideas, tools and methodologies for accident prevention.

Target audience The methods and procedures described in this programme have been compiled from experience gained in the successful development and management of aviation safety programmes by aviation operators, ATS providers, aerodrome, and maintenance organizations. In addition, the programme embodies best practices from sources such as governments, manufacturers and other reputable aviation organizations.

1

ICAO Annex 6 — Operation of Aircraft, Part I — International Commercial Air Transport — Aeroplanes and Part III — International Operations — Helicopters. 2 See Annex 13 — Aircraft Accident and Incident Investigation, Chapter 8.

1-7

Application of the guidance material herein is not limited to operational personnel. Rather it should be relevant to the full spectrum of stakeholders including senior management. In particular, this programme is aimed at those personnel who are responsible for designing, implementing and managing effective accident prevention programmes. Specifically: Government Officials with responsibilities for regulating the aviation system; Management of operational organizations, such as operators, ATS providers, aerodromes and maintenance; and Safety Practitioners, such as accident prevention advisers and safety advisers.

Gender Notwithstanding any references that are specifically aligned with the male gender, this document is gender neutral — having equal application for men or women.

Relationship to other ICAO documents ICAO Annexes include Standards and Recommended Practices (SARPs), many of which have a direct bearing on safe flying operations and accident prevention, thereby providing much of the direction for the writing of this manual. For example: a) Annex 6, which governs the Operation of Aircraft, requires that “An operator shall establish and maintain an accident prevention and flight safety programme.” (See Part 1 Chapter 3, 3.2.1); b) Annex 11, which governs Air Traffic Services, provides direction for implementing ATS safety management programmes. (See Chapter 2, 2.26); c) Annex 13 provides direction for the investigation of accidents and incidents and includes Recommendations to States for the promotion of accident prevention by analysis of accident and incident data and by the prompt exchange of safety information. (See Chapter 8); and d) Annex 14 includes requirements for the design and operation of aerodromes, including the management of safety at aerodromes. The Accident Prevention Programme is also intended to be a companion document for other ICAO documents, including: a) Preparation of an Operations Manual (Doc 9376) which provides detailed guidance to operators in such areas as training and the supervision of operations, and includes direction on the need to maintain an accident prevention programme; b) Airworthiness Manual (Doc 9760) which provides guidance for the conduct of a continuing airworthiness programme; c) Human Factors Training Manual (Doc 9683) which describes in greater detail much of the underlying approach to the human performance aspects of accident prevention in this Manual;

1-8

d) Safety Oversight Audit Manual (Doc 9735) which provides guidance for the systematic conduct of safety audits, in particular for licensing, air operations, and airworthiness matters; e) Human Factors Guidelines for Safety Audits Manual (Doc 9806) which provides guidelines for anyone preparing for or conducting a safety oversight audit which includes consideration of human performance and limitations – the source of most accidents; f) Manual of Aircraft Accident and Incident Investigation (Doc 9756) which provides information and guidance to States on the procedures, practices and techniques that can be used in aircraft accident investigations; g) Cabin Attendants’ Safety Training (Doc 7192) which provides for the training for cabin attendants required by Annex 6; and

How to use the Accident Prevention Programme In many ways, this programme is a stand-alone document. The user should find sufficient information to guide the justification for, initiation and operation of a viable accident prevention programme. The programme is not prescriptive. However, based on an understanding of the philosophy, principles and practices discussed herein, organizations should be able to develop an approach to accident prevention that is best suited to the local conditions. A bibliography at the end of the programme identifies sources of information used in the document and can be used to guide the reader to further information in particular areas; however, ICAO can not guarantee the currency of the addresses provided. Cautious use of the World Wide Web (www) can also assist in obtaining related information.

____________________

1-9

Chapter 2 ROLES AND RESPONSIBILITIES FOR ACCIDENT PREVENTION

Organizational Responsibilities • Introduction • ICAO • States • State civil aviation administrations (CAAs) • Aircraft manufacturers • Airline operators • General aviation • Service providers • Third party contractors • Business and professional associations Global Cooperation Management’s Special Responsibility for Safety

2-1

This page intentionally left blank.

2-2

Chapter 2 ROLES AND RESPONSIBILITIES FOR ACCIDENT PREVENTION

Safety is everybody’s business.

ORGANIZATIONAL RESPONSIBILITIES Introduction The responsibility for preventing accidents goes well beyond the cockpit. It is a shared responsibility involving a wide spectrum of organizations and institutions. These include international organizations, major aircraft and power-plant manufacturers, State regulatory authorities for civil aviation, owners and operators, maintenance organizations, industry and professional associations, aviation education and training institutions, etc. Further, third parties that provide aviation support services (including contracted services) must also share in the responsibility for accident prevention — observing the safety standards pertinent to their areas of endeavour. To reduce the severity and probability of mishaps, each of these institutional stakeholders has particular roles to perform diligently. Generally, these organizational responsibilities fall into the following areas: a) Defining policies and standards affecting accident prevention; b) Allocating resources to sustain accident prevention initiatives and activities; c) Providing expertise for the identification and evaluation of safety hazards; d) Taking safety action to eliminate or reduce systemic hazards to what has been decided is an acceptable level of risk; e) Incorporating technical advances in design and maintenance of equipment; f) Conducting safety oversight and accident prevention programme evaluation; g) Contributing to the investigation of accidents and serious incidents; h) Keeping abreast of best industry practices, adopting these as appropriate; i)

Promoting aviation safety (including the exchange of safety-related information); and

j)

Amending regulations governing civil aviation safety as required.

2-3

ICAO From a regulatory perspective, ICAO’s role is to provide procedures and guidance for the safe conduct of international aircraft operations and to foster the planning and development of air transport. This is largely achieved by developing Standards and Recommended Practices (SARPs), which are contained in the Annexes to the Chicago Convention and reflect the operational experience of States. Procedures for Air Navigation Services (PANS) contain practices beyond the scope of SARPs, where a measure of international uniformity is desirable for safety and efficiency. Regional Air Navigation Plans detail requirements for facilities and services specific to ICAO regions. In essence, these documents define the international framework for promoting safety and efficiency in aviation. In addition to this regulatory framework, ICAO contributes to accident prevention by promoting those safety practices of States which are based on best safety experience. More specifically, ICAO: a)

Provides guidance material for States and operators covering most aspects of aviation safety, (including flight operations, airworthiness, air traffic services, aerodromes and airport security). Generally this guidance material is in the form of manuals or circulars;

b)

Developed this Manual which outlines accident prevention concepts and provides guidance for the conduct of effective aviation safety programmes;

c)

Defines international procedures for accident and incident investigation and reporting. These are contained in Annex 13 — Aircraft Accident and Incident Investigation, the Manual of Aircraft Accident and Incident Investigation (Doc 9756) and the Accident/Incident Reporting (ADREP) Manual (Doc 9156);

d)

Promotes aviation safety on a continuing basis by: 1) Disseminating accident and incident information through the ADREP system and by other means; 2) Disseminating aviation safety information in publications, and more recently in electronic formats; and 3) Conducting conferences and seminars, etc. addressing specific aspects of aviation safety (i.e. accident investigation, accident prevention and human factors); and

e)

Conducting ICAO’s Universal Safety Oversight Audit Programme (USOAP).

States States bear significant responsibility for establishing an environment conducive to safe and efficient flight operations. They fulfill much of this responsibility by establishing a legislative framework governing all aspects of civil aviation within their jurisdiction. The laws potentially impacting on accident prevention cover a wide gamut, some with a more direct impact than others. Some of the principal areas of legislation potentially affecting safety in aviation include:

2-4

a) Aeronautics law establishes a State’s objectives for aviation — both commercial and private. Typically, this legislation includes the State’s vision for aviation safety and delineates the broad responsibilities, accountabilities and authorities for fulfilling those objectives; b) Manufacturing and Trade laws govern the production and sale of safe aeronautic equipment and services; c) Labour laws (including Occupational Safety and Health (OSH) laws) set the rules for the work environment in which aviation employees are expected to perform their duties safely; d) Security laws contribute to safety in the workplace and govern who may enter into operational areas and under what terms; and e) Environmental laws affecting the siting of airports and navigation aids, impact on flight operations (such as noise abatement procedures). Having established the legal framework for safety in aviation, States have a responsibility for establishing an administrative agency (commonly referred to as the Civil Aviation Authority or CAA) for implementing the provisions of their aeronautics law. This responsibility includes: a) Establishing the necessary statutory authority and delegations to regulate the aviation industry; b) Ensuring it is adequately staffed with competent, suitably equipped technical officials; and c) Maintaining an effective system of safety oversight to assess how well regulatory requirements are being met. Safe and efficient aviation requires significant infrastructure and aeronautic services, including airports, navigation aids, air traffic management, meteorological services, flight information services, etc. Some States own and operate their own air navigation services and major airports; others own and operate their own national airline. However, many States have corporatized these operations, operating under the oversight of the State. Regardless of the approach taken, States must ensure that the infrastructure and services in support of aviation are provided and maintained to meet international obligations and the needs of the State. Many States delegate responsibility for the investigation of accidents and serious incidents (pursuant to Annex 13) to their State aviation administrations. However, this practice raises a potential conflict of interest whereby the investigators may be required to report on shortcomings in the State’s safety oversight performance (perhaps even their own performance as regulators). Increasingly, States are creating specialist investigative agencies, independent of the regulatory authorities. A number of these agencies are ‘multi-modal’ handling investigations of all transportation modes, e.g. air, land, sea and pipeline. Finally, States have a responsibility to be “good citizens” in the international community of aviation. They can best do this by ensuring that their governing legislation and regulations conform to the Chicago Convention and ICAO’s SARPs. When a State cannot, adapt their national legislation and regulations to SARPs they are required to file a “difference”. ICAO publishes these differences so that other States may be aware of departures from internationally agreed standards. The ICAO USOAP programme is used to determine States compliance with safety critical SARPs.

2-5

State civil aviation administrations (CAAs) As stated above, States have a responsibility for accident prevention which they address by setting the legislation for aviation and putting in place the necessary administrative arrangements to ensure that these laws are effectively implemented. On aviation safety matters, effective State administrations are guided by: a) A clear statement of their vision and mission (regarding safety); b) A well understood and accepted set of: 1) Operating principles, such as delivering safe and efficient service consistent with public expectations and at reasonable cost; treating clients and employees with respect, etc.; and 2) Corporate values such as competence, openness, fairness, integrity, respect, responsiveness to client needs, etc.; c) A statement of the Administration’s safety objectives; for example, reduce the probability and consequences of unsafe aviation occurrences, improve understanding throughout the aviation industry and general public of the State’s actual safety performance; and d) Strategies for fulfilling their objectives; for example, reduction of safety risks to aviation through the identification of those operations that fall below accepted levels, encouraging their return to an acceptable level of safety or, if necessary, rescinding their certification. Based on such broad direction, State administrations typically have responsibilities for some or all of the following: a) Establishing and implementing the rules, regulations and procedures for safe and efficient aviation. For example: 1) Personnel licensing; 2) Procedures for obtaining and renewing: — Operating Certificates; — Airworthiness Certificates; — Airport Certification, etc.; 3) Operation of Air Traffic Services; 4) (In many States) conduct of accident and incident investigations, etc. b) Implementing a system for safety oversight of the entire civil aviation system by surveillance, inspections and safety audits, etc.; c) Carrying out enforcement actions as necessary;

2-6

d) Monitoring technological developments and best industry practices with a view to improving the State’s aviation system performance; e) Maintaining a system of aviation records, including licenses and certificates, infractions, reported accidents and incidents, etc.; f) Conducting analyses of safety trends, including accident/incident data, service difficulty reports, etc.; and g) Promoting safety through the dissemination of specific safety materials, conducting safety seminars, etc. Most States provide at least some of the services required in operating a national aviation system, such as ATS. Notwithstanding this common practice, civil aviation authorities should recognize the potential for a conflict of interest between providing an aviation service and, as the regulatory authority, fulfilling the responsibilities for safety oversight. A growing number of States believe there are operational efficiencies and economics to be had by corporatized services as ATC or airport management. This corporatization may take a number of forms.

Aircraft manufacturers The design and manufacture of aircraft and their components improve with advancing technology. Each new generation of aircraft incorporates improvements based on the latest Astate of the art@ and operational experience. Manufacturers produce aircraft which comply with the airworthiness regulations of domestic and foreign governments, and meet the economic and performance requirements of purchasers. Manufacturers also produce manuals and other documentation to support their products. In some States these may be the only guidance material available for the operation of a specific aircraft type or piece of equipment. Thus the standard of documentation provided by the manufacturer is very important. Additionally, through their responsibilities for providing product support, training, etc. manufacturers are the best source for the overall safety record of a particular aircraft type or the in-service record of a component. Aircraft manufacturers employ various specialists in the fields of design, manufacture and operation of their aircraft, as well as accident investigators. This expertise is usually available for the investigation of accidents or incidents to aircraft of their manufacture. In addition, the major aircraft manufacturers have active safety departments whose roles include, monitoring in-service experience, providing feedback to the manufacturing process and disseminating safety information to customer airlines. Manufacturers can face costly litigation following an aircraft accident. On one hand, this is a spur to optimize safety, while on the other, it can act as a deterrent to the voluntary correction of faults when this could be regarded as an admission of design or manufacturing deficiencies.

2-7

Airline operators Most major airlines employ many of the accident prevention activities outlined in this manual, while many of the smaller airlines may not employ any. Where such activities exist, they are usually carried out by a section or safety office which monitors overall operating experience and provides independent advice to company management on the preventive action needed to eliminate or avoid identified hazards. Such activities may also lead to economies in the airline=s operation. These prevention activities usually include some form of incident reporting, safety surveys and audits and information feedback by means of periodic safety magazines, bulletins, newsletters, or the company’s website. The safety aspects of the engineering/manufacturing side of an airline are often the responsibility of a Quality Control Manager/Chief Inspector. Accident prevention programmes have tended to be oriented towards the flight operations side of the organization. Safety, however, must embrace the total airline and it is essential that a close working relationship be maintained between all parts of the organization. A State’s civil aviation authority uses regulations, standards, recommended procedures and other guidelines to help operators manage the risks inherent in aviation. However, regulations may not always fit an airline’s safety needs perfectly. Airlines which rely on regulatory compliance as the cornerstone of their accident prevention and risk management programmes, may not achieve the results they desire.

General aviation In many States, general aviation accidents constitute a major loss of resources. As a consequence, substantial benefits are to be gained from accident prevention programmes aimed at this group. In addition, general aviation operators often share facilities such as aerodromes, air traffic services, etc. with airline operators. This mixing of operations with differing requirements and performance standards may introduce hazards. General aviation embraces a wide range of aircraft types, crew qualifications and operating environments. In many States it includes the expanding areas of corporate or business flying, often operating sophisticated aeroplanes or helicopters flown by professional pilots; through to non-professional pilots who only fly occasionally for pleasure. Motivating an interest and awareness of safe aviation practices is a challenge for an accident prevention programme aimed at this varied group. Specialized general aviation aerial work operations, such as firefighting and aerial application, create unique hazards which have led some States to conduct safety programmes aimed specifically at these groups.

Service providers Safe and efficient flight operations depend on effective delivery of a variety of supporting services. Operators may provide some of these services themselves, or they may contract these services out to specialist service providers. Such services include: a) Aircraft maintenance, repair and overhaul; b) Flight planning, flight dispatch and flight following;

2-8

c) Ramp handling; and d) Crew training, etc. Other key services, which are supplied by providers external to the operator, include: a) Air Traffic Control; b) Aerodrome operations, including airport emergency services; c) Airport security; and d) Navigation aids. Traditionally, such external services have been provided by the State — usually through their civil or military aviation authorities. However, civil aviation authorities in some States have discovered potential conflicts of interests in the dual roles of the State as both a regulator and as a service provider. Moreover, some States believe that there are operational efficiencies and economies to be gained from the corporatization of many of these services. As a result, some States have delegated responsibilities for the provision of many such services. Regardless of the ownership or management structure for the provision of such supporting services, responsible managers are expected to develop and implement accident prevention (or loss control) programmes within these separate areas of expertise. The guidance material provided in this manual applies equally to the provision of such support services, regardless of whether they are governed by State-run or corporate management.

Third party contractors The provision of services supporting flight operations has long involved private contractors in such areas as refuelling, catering and other aircraft ground services, runway and taxiway construction and repair, etc. Indeed, the number of disparate vehicles on any busy airport ramp reflects the number of third party contractors. Whether a large corporate contractor or small entrepreneur, the contracting authority holds overall responsibility for managing the safety risks taken by the contractor. The contract must specify safety standards to be met. The contracting authority then has the responsibility of ensuring the contractor complies with the safety standards prescribed in the contract. The relationship between the contracting authority and a contractor is more than a legal situation. It represents the best interests of both parties. For example, an airline must protect its revenue source (the fare-paying public) by ensuring that its approved maintenance organization (AMO) provides airworthy aircraft; and the AMO would understand that sub-standard service would compromise future work with the airline.

2-9

Business and professional associations Business and professional associations also play a vital role in accident prevention. International, national, and regional airline associations are formed to advance commercial interests; however, airlines increasingly recognize the strong links between aviation safety and profitability. Airlines see that an accident by one airline can seriously compromise their own business. Thus, airline associations maintain an active watch of industry developments in technology, procedures and practices. Their members collaborate in the identification of safety hazards and in the advocacy required for reducing or eliminating those deficiencies. Through such associations, many airlines are now sharing safety-related data with a view to preventing accidents. In a similar manner, professional associations representing the interests of various professional groups (e.g. pilots, air traffic controllers, cabin attendants, etc.) are active in the pursuit of accident prevention. Through studies, analysis and advocacy such groups provide much of the subject matter expertise required for identifying and ameliorating safety hazards. Increasingly, airlines are joining partnerships or alliances with other airlines to extend their effective route structure through code-sharing agreements. This can result in a flight segment being operated by an airline other than that expected by the passenger. These arrangements can have safety implications. No airline wants to be tied to an unsafe partner. To protect their own interests, the alliance partners conduct mutual safety audits — thereby enhancing airline safety.

GLOBAL COOPERATION Although the organizational elements described above have specific roles and responsibilities for accident prevention, the international nature of aviation demands that their individual efforts be integrated into a coherent, global aviation safety system, requiring cooperation and collaboration at all levels. Global collaboration occurs in international fora such as: a) Airline associations (such as IATA, ATA); b) International federations of national associations (such as IFALPA, IFATCA); c) International safety bodies (such as the Flight Safety Foundation, ISASI); and d) Industry/government groups (such as CAST, GAIN); e) Major manufacturers safety forums. Such organizations are able to provide ‘subject matter experts’ for meetings and studies. For example, manufacturers may invite input through “user” groups, and the users themselves may invite manufacturer’s input to better understand particular recommended operating practices. As a result, there is a healthy cross-pollination of safety-related information and knowledge. Such collaborative efforts are not based solely on noble intentions; they make good business sense for the following reasons: a) The air transport industry is strongly interdependent. The consequences of a major air disaster can affect many of the stakeholders. Mutual concern over damage to the industry’s reputation,

2-10

goodwill and public confidence tends to promote collective action over the parochial pursuit of special interests; b) There is strength in collective action; and c) Globalization of markets has diminished the significance of State borders and authority. Examples of the ways in which such global collaboration improves the efficiency and effectiveness of accident prevention efforts include: a) Harmonization, coherence and interoperability through universal design standards, standard operating procedures, terminology, etc.; b) Global sharing of safety-related information; c) Earlier identification and resolution of global systemic hazards; and d) Back-up and mutual reinforcement through overlapping effort and sharing of specialist resources, etc. MANAGEMENT’S SPECIAL RESPONSIBILITY FOR SAFETY3 In a major study of airlines around the world, it was found that the safest airlines had a clear safety mission, starting at the top of the organization and guiding actions right down to the operational level4. Lautman and Gallimore found that in the safest airlines: “…Flight operations and training managers recognize their responsibility to flight safety and are dedicated to creating and enforcing safety oriented policies…There is a method of getting information to the flight crews expeditiously and a policy that encourages confidential feedback from pilots to management…The management attitude, …is a dynamic force that sets the stage for the standardization and discipline in the cockpit brought about by a training programme oriented to safety issues.” The safest organizations are often the most efficient. Although trade-offs between safety risk management and costs may occur, management need to recognize the hidden costs of accidents and that safety is good for business. By taking a systematic approach to corporate decision-making and risk management, they reduce their accidental losses. Management has the authority and the responsibility to manage safety risks in the company. It achieves this by establishing a systematic method for identifying hazards, assessing risks, assigning priorities to these risks and then by reducing or eliminating those hazards which pose the greatest potential loss. It has the ability to introduce changes in the organization, its structure, its staffing, its equipment, policies and procedures.

3

Human Factors Training Manual (Doc 9683) Part 1 Ch 2 further addresses the importance of management in the establishment of a positive safety culture. Also see TC TP 12883 Nov 96: Human Factors: Management & Organization: Management=s Role in Safety. 4 See Control of Crew-Caused Accidents by L.G. Lautman and P.L. Gallimore (both of the Boeing Commercial Airplane Company) reproduced in Flight Safety Foundation: Flight Safety Digest, October 1989.

2-11

Above all, management sets the organizational climate for safety. Without management’s wholehearted commitment to safety, any accident prevention programme will be largely ineffective. In positively reinforcing safety actions, management sends the message to all staff that it really cares about safety and they had better too. To establish safety as a core value of the organization, it is necessary to make safety an integral part of the management plan. This can be done by setting objectives and safety goals, then holding managers and employees accountable for achieving those goals. Staff, then look to management for: Clear direction in the form of credible policies, objectives, goals, standards, etc.; Time for meetings, setting and communicating policies and standards, etc.; Adequate resources to fulfill assigned tasks safely and efficiently; and Expertise in terms of access to experience through safety literature, training, seminars, etc. The special onus on management for accident prevention applies, regardless of the size or type of organization providing the aviation service. The role of management in accident prevention is a recurring theme throughout this document.

____________________

2-12

Chapter 3 SAFETY BASICS

Introduction • Accidents vs. incidents: What is an accident? What is an incident? Accident and Incident Causation • Traditional view of causation • Contemporary view of causation • Findings and causes • Incidents: Precursors of accidents — 1:600 Rule Understanding the Accident/Incident Context • Equipment design • Supporting infrastructure • Human factors — SHEL model • Cultural factors — National — Professional — Organizational • Corporate safety culture — Positive safety cultures — Indications of a positive safety culture – Informed culture – Learning culture – Reporting culture – Just culture — Error tolerance — Blame and punishment Human Error • Control of human error — Error reduction — Error capturing — Error tolerance Accident Prevention Cycle Cost Considerations • Cost of accidents • Cost of incidents • Cost of accident prevention

3-1

This page intentionally left blank.

3-2

Chapter 3 SAFETY BASICS INTRODUCTION Traditionally, safety has been perceived to be the absence of accidents. However, flight operations are subject to a diverse set of often-conflicting factors, any one of which may contribute to an accident. Thus, the absence of accidents may not reflect the absence of these risks. Depending on individual perspectives, the concept of safety carries many different connotations, such as: a) Zero accidents (or serious incidents); b) A freedom from danger or risks; i.e. those factors which cause or are likely to cause harm; c) An attitude towards unsafe acts and conditions by employees (reflecting a “safe” corporate culture); d) The degree to which the inherent risks in aviation are ‘acceptable’; e) The process of hazard identification and risk management; and f) Control of accidental loss (of persons, property or damage to the environment). Safety is increasingly viewed as the management of risk. Thus, for the purposes of this programme: Safety is considered to be the state in which the risk of harm to persons or property damage is reduced to, and maintained at or below, an acceptable level through a continuing process of hazard identification and risk management. No human activity or man-made system can be guaranteed to be absolutely safe, i.e. free from risk. Safety is a relative notion, whereby inherent risks are acceptable in a “safe” system. As long as everyone – from politicians to the travelling public — understands and accepts this, there is a coherent foundation for aviation safety (i.e. accident prevention). The roles and responsibilities for ensuring safety (i.e. the prevention of accidents) were addressed in Chapter 2 (Roles and Responsibilities for Accident Prevention). Of note, the responsibilities for safety go well beyond the flight deck to include staff, managers and officials across all organizations constituting the aviation system.

3-3

Accidents vs. Incidents: What is an accident? What is an incident? ICAO Annex 13 provides definitions of accidents and incidents that may be summarized as follows: a) An accident is an occurrence during the operation of an aircraft, that entails: 1) A fatality or serious injury; 2) Substantial damage to the aircraft involving structural failure or requiring major repair of the aircraft; or 3) The aircraft is missing. b) An incident is an occurrence, other than an accident, associated with the operation of an aircraft that affects or could affect the safety of operation. A serious incident is an incident involving circumstances indicating that an accident nearly occurred. The ICAO definitions use the word “occurrence” to indicate an accident or incident. From the perspective of accident prevention, there is a danger in concentrating on the difference between accidents and incidents through definitions that may be arbitrary and limiting. Many incidents occur every day which may, or may not, be reported to the investigation authority, but which come very close to being accidents. Because there is no injury, or little or no damage, they might not be investigated. This is unfortunate because investigation of an incident may yield better results for accident prevention, than the investigation of an accident. The difference between an accident and an incident may just be an element of chance, rather than effective intervention. Indeed, an incident may be thought of as an undesired event that, under slightly different circumstances, could have resulted in harm to people or damage to property and thus have been classified as an accident.

ACCIDENT AND INCIDENT CAUSATION Understanding accident and incident causation is a key to accident prevention. Since accidents and incidents are so closely related, no attempt is made to differentiate accident causation from incident causation.

Traditional view of causation Following a major air disaster, the questioning process begins. For example: a) How and why did competent personnel make the errors necessary to precipitate the accident? and b) Could something like this happen again? Traditionally, investigators have examined a chain of events or circumstances which ultimately led to someone doing something inappropriate thereby triggering the accident. This inappropriate behaviour may have been an error in judgement (such as a deviation from standard operating procedures), an error due to inattention, or a deliberate violation of the rules. Following the traditional approach, the investigative focus was more often than not on finding someone to blame (and punish) for the accident. At best, accident prevention efforts were concentrated on finding

3-4

ways of reducing the risk that such unsafe acts would be committed in the first place. However, the errors or violations that trigger accidents seem to occur randomly. With no particular pattern to pursue, accident prevention efforts to reduce or eliminate random events may be ineffective. Analysis of accident data all too often reveals that the situation prior to the accident was Aripe for an accident@. Safety-minded persons may even have been saying that Ait is just a matter of time before these circumstances lead to an accident”. And when the accident occurs, all too often healthy, qualified, experienced, motivated and well-equipped personnel committed errors that triggered the accident. They (and their colleagues) may have committed these errors or unsafe practices many times before without adverse consequences. Further, some of the unsafe conditions in which they were operating may have been present for years, again without causing an accident. In other words, there is an element of chance present. Sometimes these unsafe conditions were the consequence of decisions by management; they recognized the risks, but often priorities required a trade off. Indeed, front-line personnel work in a context that is defined by organizational and management factors often beyond their control. The front-line employees are but part of a larger system. To be successful, accident prevention requires an alternative understanding of accident causation, one that depends on examining the total context (i.e. the system) in which these people work.

Contemporary view of causation According to contemporary thinking, accidents require the coming together of a number of enabling factors, each one necessary, but in itself not sufficient to breach system defences. Major equipment failures or operational personnel errors are seldom the sole cause of breaches in safety defences. Often these breakdowns are the consequence of human failures in decision-making. These breakdowns may occur at the operational level (active failures) or be considered latent failures sometimes resulting from management/boardroom decisions. Most accidents include both active and latent failures. Figure 3-1 portrays an accident causation model5 that assists in understanding the interplay of organizational and management factors (i.e. system factors) in accident prevention. Various “defences” are built into the aviation system to protect against inappropriate performance or poor decisions at all levels of the system: the front-line workplace, the supervisory levels and senior management. This model shows that while organizational factors, including management decisions, can create latent failure conditions that could lead to an accident, they also contribute to the system defences. Errors and violations having an immediate adverse effect can be viewed as unsafe acts; these are generally associated with front-line personnel (pilots, controllers, mechanics, etc.). These unsafe acts may penetrate the various defences put in place to protect the aviation system by company management, the regulatory authorities, ICAO, etc., resulting in an accident. These unsafe acts may be the result of normal errors, or they may result from deliberate violations of prescribed procedures and practices. The model recognizes that there are many error-producing or violation-producing conditions in the work environment that may affect individual or team behaviour.

5

Adapted from James Reason, ACollective Mistakes in Aviation: >The Last Great Frontier=@, Flight Deck, Summer 1992, Issue

4.

3-5

Organization

Workplace

Organization & Management Decisions

Error & Violation Producing Conditions

Crew/Team

Errors & Violations

Defences

Outcome

Accident

Figure 3-1. Accident causation model (Adapted from Dr. James Reason 1995)

These unsafe acts are committed in an operational context which includes latent unsafe conditions (latent failures). A latent failure is the result of an action or decision made well before an accident. Its consequences may remain dormant for a long time. Individually, these latent failures are usually not harmful since they are not perceived as being failures in the first place.6 Latent unsafe conditions may only become evident once the system=s defences have been breached. They may be present in the system well before an accident and are generally created by decision-makers, regulators and other people far removed in time and space from the accident. Front-line operational personnel can inherit defects in the system, such as those created by poor equipment or task design; conflicting goals (e.g. on-time service vs. safety); defective organizations (e.g. poor internal communications); or bad management decisions (e.g. deferral of a maintenance item). Effective accident prevention efforts aim to identify and mitigate these latent unsafe conditions on a system-wide basis, rather than by localized efforts to minimize unsafe acts by individuals. Such unsafe acts may only be symptoms of safety problems, not causes. Most latent unsafe conditions start with the decision-makers, even in the best-run organizations. These decision-makers are also subject to normal human biases and limitations, as well as to very real constraints of time, budget, politics, etc. Since some of these unsafe decisions can not be prevented, steps must be taken to detect them and to reduce their adverse consequences. Fallible decisions by line-management may take the form of inadequate procedures, poor scheduling or neglect of recognizable hazards. They may lead to inadequate knowledge and skills or inappropriate operating procedures. How well line-management and the organization perform their functions sets the scene for error, or violation, producing conditions. For example, how effective is management with respect to setting attainable work goals, organizing tasks and resources, managing day-to-day affairs, 6

Wood, Richard H., Aviation Safety Programs: A Management Handbook, 3rd edition, Englewood, Co.: Jeppesen, 2003

3-6

communicating internally and externally, etc.? The fallible decisions made by company management and regulatory authorities are too often the consequence of inadequate resources. However, avoiding the costs of strengthening the safety of the system can facilitate accidents that are so expensive as to bankrupt the operator.

Findings and causes One of the difficulties in determining accident causes involves the level of confidence that can be placed in the investigative findings. In criminal proceedings, the burden of proof is heavy, seeking virtual certainty (e.g. “beyond reasonable doubt”). However, in accident investigations, it is often impossible to determine causes with absolute certainty. Following an investigation, some stakeholders, especially the general public, media and legal fraternity are primarily interested in Athe cause@ of the accident. More enlightened stakeholders will seek an understanding of all the causes. However, excessive emphasis on determining the causes may be at the expense of accident prevention. If the investigation went deep enough, beyond the determination of the causes, it may reveal systemic safety deficiencies that must be corrected in the interests of accident prevention. A thorough investigation will reveal many findings, perhaps including latent unsafe conditions that had no role whatsoever in the accident causation. From an accident prevention point of view, significant safety deficiencies found in an investigation that may be completely unrelated to the accident must be eliminated, or reduced. A thorough investigation will reveal a number of findings. Some of these findings could be considered causal. Others may have contributed to the accident in some way (sometimes referred to as contributory factors). On the basis that Annex 13 defines the objective of an investigation as accident prevention, a growing number of investigations are determining findings without attempting to identify which of the findings are causal.

Incidents: Precursors of accidents Regardless of the accident causation model used, typically there would have been precursors evident before the accident. All too often, these precursors only become evident with hindsight. Latent unsafe conditions may have existed at the time of the occurrence. Identifying and validating these unsafe conditions requires an objective, in-depth risk analysis. Although it is important to fully investigate accidents with high numbers of fatalities, this may not be the most fruitful means for identifying safety deficiencies. Care must be taken to ensure that the “blood priority” (often prevalent in the media after significant loss of life) does not detract from a rational risk analysis of unsafe conditions in aviation. While using accident investigations to identify hazards is important, it is reactive. 1:600 Rule7. Research into industrial safety in 1969 indicated that for every 600 reported occurrences with no injury or damage, there were some: — 30 incidents involving property damage, — 10 accidents involving serious injuries, and — 1 major or fatal injury. 7

Frank E. Byrd, Jr., 1969

3-7

Fatal Accident

1 Accidents

10 Reportable Incidents

30 Incidents

600

Figure 3-2. 1:600 rule

The 1-10-30-600 ratio shown in Figure 3-2 is indicative of a wasted opportunity, if investigative efforts are focused only on those rare occurrences where there is serious injury, or significant damage. The latent factors contributing to such accidents may be present in hundreds of other incidents, and could be identified – before serious injury or damage ensues. Effective accident prevention requires that all staff and management identify and analyse hazards before they result in accidents. In aviation incidents, injury and damage (and therefore, liability) are generally less significant than in accidents. Accordingly, there is less publicity associated with these occurrences. In principle, more information regarding such occurrences should be available (e.g. live witnesses, undamaged flight recorders, etc.). Without the threat of substantial damage suits, there also tends to be less of an adversarial atmosphere during the investigation. Thus, there should be a better opportunity to identify why the incidents occurred and, equally, how the defences in place prevented them from becoming accidents. In an ideal world, the underlying safety deficiencies can all be identified, and preventive measures to ameliorate these unsafe conditions initiated, before an accident occurs.

UNDERSTANDING THE ACCIDENT/INCIDENT CONTEXT Accidents and incidents occur within a defined set of circumstances and conditions. These include the aircraft and other equipment, the weather, the airport and flight services, etc. They also include the regulatory, industry and corporate operating climate. Above all, they include the permutations and

3-8

combinations of human behaviour. At any given time, some of these factors may converge in such a way as to create conditions that are ripe for an accident. Understanding the context in which accidents occur is fundamental to accident prevention. Some of the principal factors shaping the context for accidents and incidents include equipment design, supporting infrastructure, human and cultural factors, corporate safety culture and cost factors. Each is discussed in this chapter.

Equipment design Equipment (and job) design is fundamental to both safe operations and maintenance. Simplistically, the designer is concerned with such questions as: a) Does the equipment do what it is supposed to do? b) Does the equipment interface well with the operator? Is it “user-friendly”? c) Does the equipment fit in the allocated space? etc. From the equipment operator’s perspective, the equipment must “work as advertised”. The ergonomic design must minimize the risk (and consequences) of errors. Are the switches accessible? Is the controlling action intuitive? Are the dials and displays adequate under all operating conditions, etc.? Is the equipment resistant to mistakes, e.g. “Are you sure you want to delete this file?” Each such factor has accident potential. The designer also needs to consider the equipment maintainer’s perspective; there must be sufficient space available to permit access for required maintenance under typical working conditions and with normal human strength and reach limitations. The design must also incorporate adequate feedback to warn of an incorrect assembly. With advances in automation, design considerations become even more apparent. Whether it is the pilot in the cockpit, air traffic controllers at their consoles, or a maintenance engineer using automated diagnostic equipment, the scope for new types of human errors has expanded significantly. Equipment operators may no longer be certain at all times of what is happening, hence the question: “What is the computer doing now?” Their situational awareness may be affected accordingly. Although increased automation has reduced the potential for many types of accidents, accident prevention now faces new challenges induced by that automation, such as lack of situational awareness, boredom, etc.

Supporting infrastructure From an operator or service provider’s perspective, the availability of adequate supporting infrastructure is essential to the safe operation of aircraft. This includes the adequacy of the State’s performance with respect to such things as: a) Personnel licensing; b) Certification of aircraft, operators, service providers and aerodromes; c) Ensuring the provision of required services; d) Investigation of accidents and incidents; and

3-9

e) Providing operational safety oversight. From a pilot’s perspective, supporting infrastructure includes such things as: a) Airworthy aircraft suitable for the type of operation; b) Adequate and reliable CNS services; c) Adequate and reliable aerodrome, ground handling, and flight planning services; and d) Effective support from the parent organization with respect to initial and recurrent training, scheduling, flight dispatch or flight following system, etc. An air traffic controller is similarly concerned with such things as: a) Availability of operable (CNS) equipment suitable for the operational task; b) Effective procedures for the safe and expeditious handling of aircraft; and c) Effective support from the parent organization with respect to initial and recurrent training, rostering and general working conditions. Human factors8, 9 In a high technology industry like aviation, the focus of problem solving is often on technology. However, the accident record repeatedly demonstrates that at least three out of four accidents involve performance errors made by apparently healthy and appropriately qualified individuals. In the rush to embrace new technologies, the fallible mortals who must interface with and use this equipment are often overlooked. The sources of some of the problems causing or contributing to these accidents may be traced to poor equipment or procedure design, or to inadequate training or operating instructions. But whatever the origin, understanding normal human performance capabilities, limitations and behaviour in the operational context is central to understanding accident prevention. An intuitive approach to Human Factors is no longer appropriate. The human element is the most flexible and adaptable part of the aviation system, but it is also the most vulnerable to influences that can adversely affect its performance. With the majority of accidents resulting from less than optimum human performance, there has been a tendency to merely attribute them to human error. However, the term Ahuman error@ is of little help in accident prevention. Although it may indicate where in the system the breakdown occurred, it provides no guidance as to why it occurred. An error attributed to humans may have been design-induced, or stimulated by inadequate equipment or training, badly designed procedures, or a poor layout of checklists or manuals. Further, the term Ahuman error@ allows concealment of the underlying factors that must be brought to the fore if accidents are to be 8

Adapted from Ch 2 of Human Factors Guidelines for Safety Audits Manual (Doc 9806) Readers are referred to Human Factors Training Manual (Doc 9683) for a more comprehensive coverage of the theoretical and practical aspects of Human Factors. 9

3-10

prevented. In contemporary safety thinking, human error is the starting point, rather than the stopping point in accident investigation and prevention. Accident prevention initiatives seek ways of minimizing or preventing human errors that might jeopardize safety. This requires an understanding of the operating context in which humans err, (i.e. an understanding of the factors and conditions affecting human performance in the workplace).

SHEL Model The workplace typically involves a complex set of interrelated factors and conditions, which may affect human performance. The SHEL model10 (sometimes referred to as the SHELL model) can be used to help visualize the interrelationships among the various components of the aviation system. The SHEL model is a development of the traditional Aman-machine-environment@ system, (the name being derived from the initial letters of its components). SHEL places emphasis on the human being and the human=s interfaces with the other components of the aviation system. The following nomenclature is applied: a) Liveware (L) (humans in the workplace), b) Hardware (H) (machine and equipment), c) Software (S) (procedures, training, support, etc.), and d) Environment (E) (the operating circumstances in which the rest of the L-H-S system must function).

Figure 3-3 depicts the SHEL model. This building block diagram is intended to provide a basic understanding of the relationship of the human to other factors in the workplace.

H

S

E

L

S – software H – hardware E – environment L – liveware

L

Figure 3-3. SHEL Model 10 The SHEL concept was first developed by Professor Elwyn Edwards in 1972, with a modified diagram to illustrate the model developed by Frank Hawkins in 1975.

3-11

Liveware. In the centre of the model are those persons at the front line of operations. Although people are remarkably adaptable, they are subject to considerable variations in performance. Humans are not standardized to the same degree as hardware; so the edges of this block are not simple and straight. People do not interface perfectly with the various components of the world in which they work. To avoid tensions that may compromise human performance, the effects of irregularities at the interfaces between the various SHEL blocks and the central Liveware block, must be understood. The other components of the system must be carefully matched to humans if stresses in the system with accident potential are to be avoided. Several different factors put the rough edges on the Liveware block; some of the more important factors affecting individual performance are: Physical Factors include the individual=s physical capabilities to perform the required tasks, (e.g. strength, height, reach, vision, and hearing). Physiological Factors include those factors which affect the human=s internal physical processes, which can compromise the crew=s physical and cognitive performance, e.g. oxygen availability, general health and fitness, disease or illness, tobacco, drug or alcohol use, personal stress, fatigue, or pregnancy. Psychological Factors include those factors affecting the psychological preparedness of the individual to meet all the circumstances that might occur during a flight, e.g. adequacy of training, knowledge and experience, visual illusions and workload. The individual=s psychological fitness for duty includes motivation and judgement, attitude towards risky behaviour, confidence, stress, etc. Psycho-social Factors include all those external factors in the individual=s social system that bring pressure to bear on them, both in their work and their non-work environments, e.g. argument with a supervisor, labour-management disputes, a death in the family, personal financial problems or other domestic tension. The SHEL model is particularly useful in visualizing the interfaces between the various components of the aviation system. These include: Liveware-Hardware (L-H). The interface between the human and the machine is the one most commonly considered when speaking of Human Factors. It determines how the human interfaces with the physical work environment, e.g. design of seats to fit the sitting characteristics of the human body, displays to match the sensory and information processing characteristics of the user, controls with proper movement, coding and location. However, there is a natural human tendency to adapt to L-H mismatches. This tendency may mask serious deficiencies, which may only become evident after an accident. Liveware-Software (L-S). The L-S interface is the relationship between the individual and the supporting systems found in the workplace, e.g. the regulations, manuals, checklists, publications, standard operating procedures and computer software. It includes such Auser friendliness@ issues as currency, accuracy, format and presentation, vocabulary, clarity, symbology, etc. Increasingly, cockpit automation has altered the nature of crew duties. Workload may have been increased to such an extent during some phases of flight that crew members= attitudes towards each other may be affected (i.e. the L-L interface).

3-12

Liveware-Liveware (L-L). The L-L interface is the relationship between the individual and other persons in the workplace. Flight crews, air traffic controllers, maintenance technicians and other operational personnel function as groups and group influences play a role in determining human behaviour and performance. This interface is concerned with leadership, crew cooperation, teamwork and personality interactions. In aviation, the advent of Crew Resource Management (CRM) has resulted in considerable focus on this interface. CRM training promotes teamwork and focuses on the management of normal human errors. The L-L interface goes well beyond the crew relationship in the cockpit. Staff/management relationships are also within the scope of this interface, as are corporate culture, corporate climate and company operating pressures that can all significantly affect human performance. Liveware-Environment (L-E). This interface involves the relationship between the individual and the internal and external environments. The internal workplace environment includes such physical considerations as temperature, ambient light, noise, vibration, air quality, etc. The external environment (for pilots) includes such things as visibility, turbulence, terrain, etc. Increasingly, the work environment for flight crews includes disturbances to normal biological rhythms, e.g. sleep patterns. Further, the aviation system operates within a context of broad political and economic constraints, which in turn affect the overall corporate environment. Included here are such factors as the adequacy of physical facilities and supporting infrastructure, the local financial situation, regulatory effectiveness, etc. Just as a crew=s immediate work environment may create pressures to take short cuts, inadequate infrastructure support may also compromise the quality of crew decisionmaking. For the most part, the rough edges of these interfaces can be managed. For example: a) The designer can ensure the performance reliability of the equipment under specified operating conditions; b) During the certification process, the regulatory authority can define the conditions under which that equipment may be used; c) The organization’s management can specify standard operating procedures and provide initial and recurrent training for the safe use of the equipment; and d) Individual equipment operators can ensure their familiarity and confidence in using the equipment safely under all required operating conditions, etc. Cultural factors11 Culture influences the values, beliefs and behaviours that we share with the other members of our various social groups. Culture serves to bind us together as members of groups and to provide clues as to how to behave in both normal and unusual situations. Some see culture as the “collective programming of the mind”. It is the complex, social dynamic that sets the rules of the game, or the framework for all our interpersonal interactions. Culture is the sum total of the way people conducts their affairs in a particular social milieu. Culture provides a context in which things happen. For accident prevention, understanding this context called culture is an important determinant of human performance and its limitations.

11

This section is adapted from Human Factors Guidelines for Safety Audits Manual (Doc 9806).

3-13

The western world=s approach to management is often based on an emotionally detached rationality, which is considered to be Ascientifically” based. It assumes that human cultures in the workplace resemble the laws of physics and engineering, which are universal in application. This assumption reflects a Western cultural bias. Aviation safety must transcend national boundaries, including all the cultures therein. On a global scale, the aviation industry has achieved a remarkable level of standardization across aircraft types, countries and peoples. Nevertheless, it is not difficult to detect differences in how people respond in similar situations. As people in the industry interact (the Liveware-Liveware (L-L) interface), their transactions are affected by the differences in their cultural backgrounds. Different cultures have different ways of dealing with common problems. Organizations are not immune to cultural considerations. Organizational behaviour is subject to these influences at every level. The following three levels of culture have relevance to accident prevention initiatives: a) National culture differentiates the national characteristics and values system of particular nations. People of different nationalities differ for example in their response to authority, how they deal with uncertainty and ambiguity, and how they express their individuality. They are not all attuned to the collective needs of the group (team or organization) in the same way. In collectivist cultures, there is acceptance of unequal status and deference to leaders. Such factors may affect the willingness of individuals to question decisions or actions — an important consideration in Crew Resource Management (CRM) for example. Crew assignments that mix national cultures may also affect team performance by creating misunderstandings. b) Professional culture differentiates the behaviour and characteristics of particular professional groups (e.g. the typical behaviour of pilots vis à vis that of air traffic controllers, or maintenance engineers). Through personnel selection, education and training, on-the-job experience, etc., professionals tend to adopt the value system and develop behaviour patterns consistent with their peers; they learn to Awalk and talk@ alike. Pilots generally share a pride in their profession and are motivated to excel in their flying. On the other hand, pilots frequently have a sense of personal invulnerability, e.g. they feel that their performance is not affected by personal problems, or they do not make errors in situations of high stress. c) Organizational culture differentiates the behaviour and values of particular organizations (e.g. the behaviour of members of one company vs. that of another company, or government vs. private sector behaviour). Organizations provide a shell for national and professional cultures. In an airline for example, pilots may come from different professional backgrounds (e.g. military vs. civilian experience, bush or commuter operations vs. development within a large carrier). They may also come from different organizational cultures due to corporate mergers or lay-offs. Generally, personnel in the aviation industry enjoy a sense of belonging. They are influenced in their day-to-day behaviour by the values of their organization. Does the organization recognize merit? Promote individual initiative? Encourage risk taking? Tolerate breeches of SOPs? Promote open two-way communications, etc.? Thus, the organization is a major determinant of employee behaviour. The greatest scope for creating and nourishing a culture of safety is at the organizational level. This is commonly referred to as corporate safety culture and is discussed further below.

3-14

The three cultural sets described above are important to safe flight operations. They determine how juniors will relate to their seniors, how information is shared, how personnel will react under stress, how particular technologies will be embraced and used, how authority will be acted upon, how organizations react to human errors (e.g. punish offenders, or learn from experience). Culture will be a factor in how automation is applied in flight operations; how procedures (SOPs) are developed and implemented; how documentation is prepared, presented, and received; how training is developed and delivered; how crew assignments are made; relationships between pilots, operations and ATC; relationships with unions, etc. In other words, culture impacts on virtually every type of interpersonal transaction. In addition, cultural considerations creep into the design of equipment and tools. Technology may appear to be culture-neutral, but it reflects the biases of the manufacturer (e.g. consider the English language bias implicit in much of the world=s computer software). Yet, there is no right and no wrong culture; they are what they are and they each possess a blend of strengths and weaknesses. The challenge for accident prevention advisers is to understand how culture affects both individuals and aviation organizations and how that relationship can put safety at risk, or serve to enhance it.

Corporate safety culture As seen above, many factors create the context for human behaviour in the workplace. Organizational or corporate culture sets the boundaries for accepted human behaviour in the workplace by establishing the behavioural norms and limits. Thus, organizational or corporate culture provides a cornerstone for managerial and employee decision-making; “This is how we do things here!” Safety culture is a natural bi-product of corporate culture. The corporate attitude towards safety influences employees’ collective approach to safety. Safety culture consists of shared beliefs, practices and attitudes. The tone for safety culture is set and nurtured by the words and actions of senior management. Corporate safety culture then is the atmosphere created by management which shapes workers’ attitudes towards safety and accident prevention. Safety culture is affected by such factors as: a) Management’s action and priorities; b) Policies and procedures; c) Supervisory practices; d) Safety planning and goals; e) Actions in response to unsafe behaviours; f) Employee training and motivation; and g) Employee involvement or Abuy in@. The ultimate responsibility for safety rests with the directors and management of the organization – whether it is an airline, a service provider (e.g. ATS) or an Approved Maintenance Organization (AMO).

3-15

The safety ethos of an organization is established from the outset by the extent to which senior management accepts responsibility for safe operations and for the management of risk.12 Positive safety culture13 Although compliance with safety regulations is fundamental to accident prevention, contemporary safety thinking is that much more is required. Operators that simply comply with the minimum standards set by the regulations are not well situated to identify emerging safety problems. An effective way to promote a safe operation is to ensure that an operator has a positive safety culture. Simply put, all staff must be responsible for and consider the impact of safety on everything they do. This way of thinking must be so deep-rooted that it truly becomes a >culture’. All decisions, either by the Board of Directors, by a driver on the ramp, or by an engineer, need to consider the implications on safety. A positive safety culture must be generated from the >top down’ and relies on a high degree of trust and respect between workers and management. Workers must believe that they will be supported in any decisions made in the interests of safety. They must also understand that intentional breaches of safety that jeopardize the operation will not be tolerated. In order to ensure a positive safety culture, management must convince employees that while schedule delivery and costs are important, safety is paramount. Some organizations go so far as to enunciate a formal corporate policy on their commitment to a positive safety culture.

Indications of a positive safety culture A positive safety culture demonstrates such attributes as: a) Senior management place strong emphasis on safety as part of the strategy of controlling risks (i.e. minimizing losses); b) Decision-makers and operational personnel hold a realistic view of the short- and long-term hazards involved in the organization’s activities; c) Those in top positions: 1) Foster a climate in which there is a positive attitude towards criticisms, comments and feedback from lower levels of the organization on safety matters; 2) Do not use their influence to force their views on subordinates; and 3) Implement measures to contain the consequences of identified safety deficiencies; d) Senior management promote a non-punitive working environment; they tolerate legitimate errors and systematically attempt to derive safety lessons from them;

12 13

Adapted from CASA Aviation Safety Management: An Operator’s Guide to Building a Safety Program Ch 2 Adapted from Airbus Safety Strategy (OSPH Draft: Sec 1 - Introduction Sep 1999 p 15)

3-16

e) There is an awareness of the importance of communicating relevant safety information at all levels of the organization (both within and with outside entities); f) There is promotion of realistic and workable rules relating to hazards, to safety and to potential sources of damage; and g) Personnel are well trained and fully understand the consequences of unsafe acts. Positive safety cultures typically are: Informed cultures. Management fosters a culture where people understand the hazards and risks inherent in their area of operations. Personnel are provided with the necessary knowledge, skills and job experience to work safely, and they are encouraged to identify the threats to their safety and seek the changes necessary to overcome them. Learning cultures. Learning is seen as more than a requirement for initial skills training; rather it is valued as a lifetime process. People are encouraged to develop and apply their own skills and knowledge to enhance organizational safety. Staff are updated on safety issues by management and safety reports are fed back to staff so that everyone can learn the pertinent safety lessons. Reporting cultures. Managers and operational personnel freely share critical safety information without the threat of punitive action. This is frequently referred to as creating a corporate reporting culture. Personnel are able to report hazards or safety concerns as they become aware of them, without fear of sanction or embarrassment. Just cultures. While a non-punitive environment is fundamental for a good reporting culture, the workforce must know and agree on what is acceptable and what is unacceptable behaviour. Deliberate violations must not be tolerated by management or by workers. A culture that recognizes that, in certain circumstances, there may be a need for punitive action is considered a just culture. Personnel tend to be self-disciplined in a just culture. Table 3-1 below summarizes three corporate responses to safety issues ranging from a poor safety culture, through the bureaucratic approach which only meets minimum acceptable requirements, to the ideal positive safety culture.

3-17

Table 3-1. Characteristics of different safety cultures

Poor

Bureaucratic

Positive

Hazard information is:

Suppressed

Ignored

Actively sought

Safety messengers are:

Discouraged or punished Avoided Discouraged

Tolerated

Trained and encouraged Shared Rewarded

Safety Culture: Characteristics

Responsibility for safety is: Dissemination of safety information is: Failures lead to: New ideas are:

Cover ups Crushed

Fragmented Allowed but discouraged Local fixes New problems (not opportunities)

Inquiries and systemic reform Welcomed

Error tolerance An important dimension of a positive safety culture is the organization’s attitude towards errors and the perceptions it creates among staff in how it responded to errors. Error tolerance is the term used to describe the ability of a system to accept an error without serious consequence. The concept of ‘error tolerance’ was first applied to the ergonomic design of equipment which incorporated physical defences against inappropriate human acts, for example, air/ground logic to prevent inadvertent gear retraction on the ground. In addition, procedural actions, such as checklists, crosschecks and readbacks, provide error tolerance by identifying unsafe conditions before a mishap occurs. Increasingly, the concept of error tolerance is being extended beyond equipment and job design into corporate safety culture. Creating a positive corporate safety culture is so dependent on effective two-way communications between management and front-line personnel, that organizations are increasingly recognizing the value of voluntary incident reporting systems that provide immunity to the reporter. However, the effectiveness of such reporting systems depends largely on the ‘error tolerance’ of the company.

Blame and punishment Once an investigation has identified the cause of an occurrence, it is usually evident who “caused” the event. Traditionally, blame (and punishment) could then be assigned. While the legal environments vary widely between States, many States still focus their investigations on determining blame and apportioning liability. For them, punishment remains a principal tool. Philosophically, punishment is appealing from several points of view, such as:

3-18

a) Seeking revenge for a breach of trust; b) Protecting society from repeat offenders; c) Altering individual behaviour; or d) Setting an example for others. Punishment may have a role to play in dealing with violations where crews intentionally contravene the “rules”. Arguably, such sanctions may deter the perpetrator of the violation (or others in similar circumstances) from jeopardizing safe flight operations. In principle, such punishment should be awarded, regardless of the outcome of the violation, i.e. punishment should not be awarded only to those whose misdemeanour leads to actual losses for the company. If the accident was the result of an error in judgement or technique, it is almost impossible to effectively punish that error. Change could be made in selection or training processes, or the system made more tolerant of such errors. If punishment is selected in such cases, two outcomes are almost certain. Firstly, no further reports will be received of such errors. Secondly, since nothing has been done to change the situation, the same accident could be expected again. Perhaps, society needs to use punishment in order to mete out justice. However, the global experience suggests that punishment has little, if any, systemic value for accident prevention. Except in wilful cases of negligent behaviour, with deliberate violations of the norms, punishment serves little purpose from a safety perspective. In much of the international aviation community, a more enlightened view of the role of punishment is emerging. In part, this parallels a growing understanding of the causes of human errors (as opposed to violations). Errors are now being viewed as the results of some situation or circumstance, not necessarily the causes of them. As a result, managers are beginning to seek out the unsafe conditions that facilitate such errors. They are beginning to find that the systematic identification of organizational weaknesses and safety deficiencies pays a much higher dividend for accident prevention than punishing individuals. (That is not to say that these enlightened organizations are not required to take action against individuals who fail to improve after counselling and/or extra training.) While many airlines are taking this more positive approach to the active management of safety, others have been slow to adopt and implement effective ‘non-punitive policies’. Still, others have been slow to extend their non-punitive policies beyond flight operations on a corporate wide-basis (to include maintenance and ground operations).14

HUMAN ERROR Human error is cited as being a causal or contributing factor in the majority of aviation occurrences. All too often, competent personnel commit such errors, although clearly they did not plan to have an accident. Errors are not some type of aberrant behaviour; they are a natural bi-product of virtually all human endeavour. Error must be accepted as a normal component of any system where humans and technology interact. ATo err is human.@

14

From IATA Subject: Non-Punitive Policy Survey March 2002

3-19

The factors discussed above create the context in which humans commit errors. Given the rough interfaces of the aviation system (as depicted in the SHEL model), the scope for human errors in aviation is enormous. Understanding how normal people commit errors is fundamental to accident prevention. Only then can effective measures be implemented to minimize the effects of human errors on safety. Even if not altogether avoidable, human errors are manageable through the application of improved technology, relevant training and appropriate regulations and procedures. Most measures aimed at error management involve front-line personnel. However, the performance of pilots, controllers, technicians, etc. can be strongly influenced by organizational, regulatory, cultural and environmental factors affecting the workplace. For example, organizational processes constitute the breeding grounds for many predictable human errors, such as inadequate communication facilities, ambiguous procedures, unsatisfactory scheduling, insufficient resources, unrealistic budgeting — really all processes that the organization can control. Figure 3-4 summarizes some of the factors contributing to human errors — and to accidents.

Culture Accidents

Training

Incidents

Personal Factors

HUMAN ERRORS

Other Factors

Procedures Equipment Design

Organizational Factors

Figure 3-4. Contributing factors to human error

Control of human error Fortunately, few errors lead to adverse consequences, let alone accidents. Typically, errors are identified and corrected with no undesirable outcomes, for example, selecting an incorrect frequency or setting the bug to the wrong altitude. On the understanding that errors are normal in human behaviour, the total

3-20

elimination of human error would be an unrealistic goal. The challenge then is not merely to prevent errors, but to learn to safely manage the inevitable errors. Three strategies for managing human errors are briefly discussed below. Such strategies are relevant in flight operations, air traffic control or aircraft maintenance.15 a) Error Reduction strategies intervene directly at the source of the error by reducing or eliminating the contributing factors to the error. They aim at eliminating any adverse conditions that increase the risk of error. Examples of error reduction strategies include improving the access to an aircraft component for maintenance, improving the lighting in which the task is to be performed, reducing environmental distractions and providing better training. b) Error Capturing assumes the error has already been made. The intent is to Acapture@ the error before any adverse consequences of the error are felt. Error capturing is different from error reduction in that it does not directly serve to reduce or eliminate the error. Examples of errorcapturing strategies include crosschecking to verify correct task completion, functional test flights, etc. c) Error Tolerance refers to the ability of a system to accept an error without serious consequence. Examples of measures to increase error tolerance are the incorporation of multiple hydraulic or electrical systems on an aircraft to provide redundancy or a structural inspection programme that provides multiple opportunities to detect a fatigue crack - before it reaches critical length. Some airlines have implemented error management strategies that have significantly reduced human errors in such areas as rushed or non-stabilized approaches and incorrect use of checklists. Reducing the frequency and consequences of human error provides enormous scope for accident prevention.

ACCIDENT PREVENTION CYCLE

Accident prevention begins with an appreciation of the operational context in which accidents occur.

Given the number and potential relationships of all the factors that may affect safety, an effective management system is required for accident prevention. An example of the type of systematic process required is shown in Figure 3-5 Accident Prevention Cycle. A brief description of the cycle follows.

15

From Human Factors Training Manual (Doc 9683)

3-21

Identify Hazard

M o n ito r Progress

Assess Risks

Control Options

Ta k e Action

Risk Communication

Figure 3-5. Accident prevention cycle Hazard identification is the critical first step in managing safety. Hard evidence of hazards is required and may be obtained in a number of ways, from a variety of sources, for example: a) Hazard and incident reporting programmes; b) Investigation and follow-up of reported hazards and incidents; c) Trend analysis; d) Feedback from training; e) Flight data analysis; f) Safety surveys and operational oversight safety audits; g) Monitoring normal line operations; h) State investigation of accidents and serious incidents; and i)

Information exchange programmes.

3-22

Each hazard identified must be evaluated and prioritized. This evaluation requires the compilation and analysis of all available data. The data is then assessed to determine the extent of the hazard; is it a “oneof-a-kind”, or is it systemic? A database may be required to facilitate the storage and retrieval of the data. Appropriate tools are then needed to analyse the data. Having validated a safety deficiency, decisions must then be made as to the most appropriate action to be taken to reduce or eliminate the hazard. The solution must take into account the local conditions, as “one size” does not fit all situations. Care must be taken that the solution does not introduce new hazards. This is the process of risk management. Once appropriate safety action has been implemented, performance must be monitored to ensure that the desired outcome has been achieved, for example: a) The hazard has been eliminated (or at least been reduced in severity); b) The action taken permits coping satisfactorily with the hazard; and c) No new hazards have been introduced into the system. If the outcomes are unsatisfactory, the whole process must be repeated. Accident prevention is a dynamic process that requires change. People are inherently resistant to change. Management may be especially resistant to change if it costs money. Supposition and conjecture are unlikely to convince decision-makers to spend money on measures with ill-defined prospects for significant benefit. Thus, providing a compelling argument for change is a fundamental challenge for those managing safety.

COST CONSIDERATIONS Operating a profitable, yet safe airline requires a constant balancing act between the need to fulfill production goals (such as on-time departures) vs. safety goals (which may require taking extra time to ensure that a door is properly secured). The aviation workplace is filled with potentially unsafe conditions which will not all be eliminated; yet, operations must continue. Some airlines adopt a goal of Azero accidents@ and state that Asafety is their number one priority@. The reality is that airlines are in the business of making money. Profit or loss is the immediate indicator of the company=s success in meeting its production goals. However, safety is a prerequisite for a sustainable aviation business, as a company tempted to cut corners will eventually realize. For most companies, safety can best be measured by the absence of accidental losses. Companies may realize they have a safety problem following a major accident or loss, in part because it will impact on the profit/loss statement. However, a company may operate for years with many potentially unsafe conditions without adverse consequence. In the absence of an effective safety management programme to identify and correct these unsafe conditions, the company may assume that it is meeting its safety objectives as evidenced by the “absence of losses”. In reality, it has been lucky.

3-23

Total Costs

Costs

Risk Reduction

Losses

Protection Figure 3-6. Safety vs. costs Safety and profit are not mutually exclusive. Indeed, quality airlines realize that expenditures on the correction of unsafe conditions are an investment toward long-term profitability. Losses cost money. As money is spent on risk reduction measures, costly losses are reduced – as shown in Figure 3-6. However, by spending more and more money on risk reduction, the gains made through reduced losses may not be in proportion to the expenditure. Companies must balance the costs of losses and expenditures on risk reduction measures. In other words, some level of loss is acceptable from a straight profit and loss point of view. However, few organizations can survive the economic consequences of a major accident. Hence, there is a strong economic case for an effective accident prevention programme. Accident prevention programmes require energy and persistence, but not always a large budget.

Cost of accidents There are three types of costs associated with an accident or serious incident: Direct, Indirect and Industry/social costs. Direct costs: These are the obvious costs, which are fairly easily determined. They mostly relate to physical damage, and include rectifying, replacing or compensating for injuries, aircraft equipment and property damage. The high costs of an accident can be reduced by insurance coverage. (Some large organizations effectively self-insured by putting funds aside to cover their risks). Indirect costs: While insurance may cover specified accident costs, there are many uninsured costs. An understanding of these uninsured costs (or indirect costs) is fundamental to understanding the economics of safety and hence, the viability of measures for accident prevention.

3-24

Indirect costs include all those things that are not directly covered by insurance and usually total much more than the direct costs resulting from an accident. Such costs are sometimes not obvious and are often delayed. Some examples of uninsured costs that may accrue from an accident include.16 a) Loss of business and damage to the reputation of the organization. Many organizations will not allow their personnel to fly with an operator with a questionable safety record. b) Loss of Use of Equipment equates to lost revenue. Replacement equipment may have to be purchased or leased. Companies operating a one-of-a-kind aircraft may find that their spares inventory and the people specially trained for such an aircraft become surplus. c) Loss of staff productivity. If people are injured in an accident, and are unable to work, many States require that they continue to be paid. Also, they will need to be replaced at least for the short term, incurring the costs of wages, overtime (and possibly training), as well as imposing an increased workload on the experienced workers. d) Investigation and clean-up are usually uninsured costs. Operators may incur costs from the investigation including the costs of their staff involvement in the investigation, the costs of tests and analysis, wreckage recovery, restoring the accident site, etc. e) Insurance deductibles, the policyholder=s obligation to cover the first portion of the cost of any accident must be paid. A claim will also put a company into a higher risk category for insurance purposes, and therefore may result in increased premiums. (Conversely, the implementation of a comprehensive accident prevention programme could help a company to negotiate a lower premium.) f) Legal action and damage claims. Legal costs can accrue rapidly. While it is possible to take out insurance for public liability and damages, it is virtually impossible to cover the cost of time lost handling legal action and damage claims. g) Fines and Citations by governmental authorities may be imposed, including possibly shuttingdown unsafe operations. Industry and social costs: In addition to the dollar costs identified above, an aviation disaster may compromise the aviation industry’s overall reputation and market much more widely than just the accident airline. (Events after 11 September 2001 are instructive.) Travelers switching to alternate means of transportation (such as road travel) may be exposed to additional risks – a social cost.

Costs of incidents Serious aviation incidents, which result in minor damage or injuries, can also incur many of these indirect or uninsured costs. Typical cost factors arising from such incidents can include: a) Fight delays and cancellations; b) Alternate passenger transportation, accommodation, complaints, etc.; 16

Wood, Richard H., Aviation Safety Programs: A Management Handbook, 3rd edition, Englewood, Co.: Jeppesen, 2003

3-25

c) Crew change and positioning; d) Loss of revenue and reputation, etc. e) Aircraft recovery, repair and test flight; and f) Incident investigation.

Costs of accident prevention The costs of accident prevention are probably even more difficult to quantify than the full costs of accidents — in part, because of the difficulty in assessing the value of accidents that have been prevented. Nevertheless, some airlines have attempted to quantify the costs and benefits of introducing safety management systems (SMS). They have found the cost savings to be substantial. Performing a cost benefit analysis is complicated due to the small number of accidents. However, it is an exercise that should be undertaken, as senior management are not inclined to spend money if there is no quantifiable benefit. One way of addressing this issue is to separate the costs of the accident prevention programme from the cost of correcting safety deficiencies, by charging the programme costs to the safety department and the safety deficiency costs to the line management most responsible. This exercise involves senior management in considering costs and benefits.

If you think safety is expensive, try an accident.

____________________

3-26

Chapter 4 MANAGING SAFETY

Introduction • System safety • Strategies for safety management • ICAO requirements for safety management programmes Safety Management Systems (SMS) • Introduction • The SMS process • Relationship of safety management systems to quality assurance systems Organizing for Safety Management • Introduction • Safety policy, objectives and goals • Structure Appendix 1. Three Cornerstones of an SMS

4-1

This page intentionally left blank

4-2

Chapter 4 MANAGING SAFETY INTRODUCTION In view of the total costs of a major accident, management needs to conserve the organization’s assets and minimize its risks. Effective risk minimization requires the management of all the factors which can impact on safety. Successful aviation organizations have a coherent system for managing safety. The management of safety is one of their core business functions — just as financial management is. The application of contemporary risk management methods facilitates the identification of weaknesses and guides management towards the cost-effective resolution of unacceptable risks. Effective information management is required to support safety analyses and for the sharing of safety lessons and best practices across the industry. Finally, some system of performance measurement is required to confirm the effectiveness of the organization’s safety management system and to test the validity of steps taken to reduce risks. Accident prevention therefore, is multi-disciplinary, requiring the systematic application of a variety of techniques and activities across the spectrum of aviation activities; it is also a continuous process. Bearing in mind that the objective of an aviation organization is usually the production of a service or product, the effective management of safety requires a realistic balance between safety and productivity goals. Thus, a systemic approach, in which the organization's goals and resources are analysed, helps ensure that decisions concerning safety are realistic and complementary to the purposes of the organization. The finite limits of financing and operational performance must be accepted in any industry. Defining acceptable and unacceptable risks is therefore important for cost-effective accident prevention. However, it is worth remembering that properly implemented, accident prevention measures not only increase safety, but also improve the operational effectiveness of an organization. System safety17 System safety was developed as an engineering discipline in the 1950s and was used by the National Aeronautics and Space Administration (NASA) and the United States Air Force in missile and space programmes. In the 1960s, it was applied to the development of new military aircraft. The system safety programme requirements are defined in MIL-STD-882. The Standard has been updated several times and it is still the basic document that defines system safety. System safety is described as the application of special technical and managerial skills to the systematic, forward-looking identification and control of hazards throughout the life cycle of a project, programme, or activity. The primary objective of system safety is accident prevention and it is achieved by focusing on the control of hazards associated with a system or product. By proactively identifying, assessing, and eliminating or controlling safety-related hazards to acceptable levels, accident prevention could be achieved.

17

Based on Wood, Richard H., Aviation Safety Programs: A Management Handbook, 3rd edition, Englewood, Co.: Jeppesen, 2003

4-3

As an engineering discipline, system safety practitioners were safety engineers, not operational specialists. As a result, their focus tended to be on designing and building fail-safe systems. On the other hand, civil aviation tended to focus on the operation of the aeroplane and safety managers often came from the ranks of pilots. They had little interest or experience with passenger safety, cabin safety, maintenance safety, ground operations safety and so on. In pursuing improve safety, it became necessary to view aviation safety as more than just the aeroplane and its pilots. Aviation is a total system that includes everything needed to keep an aeroplane in the air safely. The ‘system’ includes the airport, air traffic control, maintenance, flight attendants, ground operational support, dispatch and others. Thus, the total aviation system is more than just the aeroplane and its pilots, and safety management programmes need to address all parts of it. Strategies for safety management18 “Safety management is the systematic management of the risks associated with flight operations, related ground operations and aircraft engineering or maintenance activities to achieve high levels of safety performance.”19 Safety management begins with defining an organization’s strategy for accident prevention. The strategy will reflect the corporate safety culture and may range from purely reactive, responding only to accidents, through to strategies that are highly proactive in their search for safety problems. Depending on the accident prevention strategy adopted, different methods and tools need to be employed. a) Reactive strategy: Investigate accidents and reportable incidents. This strategy is useful for situations involving failures in technology or unusual events. The utility of the reactive approach for accident prevention purposes depends on the extent to which the investigation goes beyond determining the causes, to include an examination of all the contributory factors. The reactive approach to accident prevention tends to be marked by: 1) Management’s safety focus being on compliance with minimum requirements; 2) Safety measurement being based on reportable accidents and incidents with such limitations in value as: — Any analysis is limited to examining actual failures;

— Insufficient data is available to accurately determine trends, especially those attributable to human errors; and — Little insight is available into the >root causes’ and latent unsafe conditions, which facilitate human errors; and 3) Constant catching up is required to match human inventiveness for new types of errors. b) Proactive safety strategy: Prevent accidents by aggressively seeking information from a variety of sources which may be indicative of emerging safety problems. Organizations pursuing a proactive strategy for accident prevention believe that the risk of accidents can be minimized by identifying

18 19

Adapted from Human Performance and Safety Consultants Inc 2001 CAP 712 Safety Management Systems for Commercial Air Transport Operations, CAA UK

4-4

vulnerabilities before they fail, and by taking the necessary actions to reduce those risks. To do this, they actively seek systemic unsafe conditions through such tools as: 1) Hazard and incident reporting systems (preferably confidential and non-punitive systems) which promote the identification of latent unsafe conditions; 2) Safety surveys to elicit feedback from front-line personnel about areas of dissatisfaction and unsatisfactory conditions which may have accident potential; 3) Flight data recorder analysis for identifying operational exceedances and confirming normal operating procedures; 4) Operational inspections or audits of all aspects of flight operations, to identify vulnerable areas before accidents, incidents, or minor safety events confirm a problem exists; and 5) A policy for consideration and embodiment of manufacturers service bulletins.

ICAO requirements for safety management programmes ICAO recognizes the importance of safety management programmes for effective accident prevention and has introduced requirements for programmes in the following areas of aviation activity: a) Air Traffic Services: (Annex 11) requires that States implement systematic and appropriate ATS safety management programmes to ensure that safety is maintained in the provision of ATS within their airspaces and at their aerodromes;20 and b) Aerodromes: (Annex 14) recommends that a certified aerodrome should have in operation a safety management system.21

SAFETY MANAGEMENT SYSTEMS (SMS) Introduction A safety management system (SMS) is an element of management responsibility which defines a company=s safety policy and sets out how it intends to manage safety as an integral part of its overall business. Safety measures taken in the context of a SMS tend to raise the level of safety performance above the minimum acceptable standards implied by mere compliance with regulatory requirements. A SMS provides the organizational framework for managing safety. Sound business practices are pursued with the object of improving organizational safety and efficiency. A SMS comprises the normal management functions of goal setting, planning and measuring performance. SMSs include a corporate vision, emanating from the Board of Directors and the Chief Executive Officer, and reaching into all departments of the organization whose activities contribute to safety performance. In essence, it embeds safety as a core value of the organization.

20 21

Annex 11: 2.26.1 Annex 14, Vol. 1, 1.3.4

4-5

An effective SMS is becoming as vital to the survival of an aviation business as is an effective financial management system. SMS has particular application in organizations devoted to flight operations, aircraft engineering and maintenance, air traffic services and aerodrome operations. An SMS must also include accountability for those suppliers, sub-contractors and business partners with the potential to affect the company=s safety performance. A SMS creates the corporate culture in which employees at all levels practice safety in their assigned responsibilities and it makes safety Aeverybody=s business”, albeit with explicit objectives and lines of accountability. Effective safety management systems comprise three defining cornerstones: a) A comprehensive corporate approach sets the tone for the management of safety. The corporate approach builds upon the safety culture of the organization and embraces the organization’s safety policies, objectives and goals, and perhaps most importantly, senior management’s commitment to safety; b) Effective organizational tools are needed to deliver the necessary activities and programmes to advance safety. It includes how the organization arranges its affairs to fulfil its safety policies, objectives and goals, how it establishes standards and allocates resources, etc. The principal focus is on hazards and their potential effects on safety-critical activities; and c) A system for safety oversight to confirm the organization’s continuing fulfilment of its corporate safety policy, objectives, goals and standards. Feedback mechanisms facilitate continuous improvement. These include such activities as monitoring and analysis of flight data, safety audits and performance analysis, the identification and adoption of best industry practices, etc. A more detailed examination of each of these cornerstones is provided at Appendix 1 (Three Cornerstones of SMS) to this Chapter.

The SMS process Conceptually, the SMS process parallels the cycle of accident prevention described in Figure 3-5. Both involve a continuous loop process as represented in Figure 4-1 below.

4-6

Collect Data Latent Unsafe Conditions Re-evaluate Situation

Latent Unsafe Conditions Collect Additional Data

Analyse Data

Prioritize Unsafe Conditions

Implement Strategies

Assign Responsibilities

Develop Strategies Approve Strategies

Figure 4-1. SMS process A SMS is evidence-based, in that it requires the analysis of actual data to identify hazards. Using risk assessment techniques, priorities are set for reducing the potential consequences of the hazards. Strategies to reduce or eliminate the hazards are then developed and implemented with clearly established accountabilities. The situation is reassessed on a recurrent basis and additional measures are implemented as required. Following are brief descriptions of the steps of the SMS process outlined in Figure 4-1: Data collection. The first step in the SMS process is to define the objectives of the system under review. The system and its various components are reviewed, including the interactions among people, procedures, tools, materials, equipment, facilities, software and the environment. Applicable regulations, documented goals, objectives, and performance specifications are gathered. Finally, the necessary data to support the safety analyses are compiled. Analyse the data. By analysing all the pertinent information, safety hazards can be identified. The conditions under which the hazards pose real risks, their potential consequences and the likelihood of occurrence can be determined; in other words What can happen? How? and When? This analysis can be both qualitative and quantitative. The inability to quantify and/or the lack of historical data on a particular hazard do not exclude the hazard from the analysis.

4-7

Prioritize the unsafe conditions. The seriousness of hazards is determined by considering them relative to each other, and against an agreed set of acceptability criteria. Those posing the greatest risks are considered for safety action. This may require a cost-benefit analysis. Develop strategies. Beginning with the highest priority risks, several options for safety action may be considered, for example: a) Spread the risk across as large a base of risk-takers as practicable. (This is the basis of insurance.) b) Transfer the risk (or liability) to some other organization or agency (such as through lease vs. buy decisions). c) Eliminate the risk entirely (possibly by ceasing that operation or practice). d) Accept the risk, and continue operations unchanged. e) Mitigate the risk by implementing measures to reduce the risk or at least facilitate coping with the risk. In selecting a remedial strategy, care is required to avoid introducing new (and perhaps worse) risks. Approve strategies. Having analysed the risks and decided on an appropriate course of action, management’s approval is required to proceed. The challenge here is the formulation of a convincing argument for (perhaps expensive) change. Assign responsibilities and implement strategies. Following the decision to proceed, the “nuts and bolts” of implementation must be worked out. This includes a determination of resource allocation, assignment of responsibilities, scheduling, revisions to operating procedures, etc. Re-evaluate situation. Implementation is seldom as successful as initially envisaged. Feedback is required to close the loop. What new problems may have been introduced? How well is the agreed strategy for risk reduction meeting performance expectations? What modifications to the system or process may be required? Collect additional data. Depending on the re-evaluation step, new information may be required and the full SMS cycle reiterated to refine the safety action. Implementing a SMS requires skills in safety analysis, skills that may not be practiced by management. The more complex the analysis, the more important is the need for the application of the most appropriate analytical tools. A key feature of the SMS process is that it is a closed loop process. It requires feedback to ensure that management can test the validity of their decisions and assess the effectiveness of their implementation.

Relationship of safety management systems to quality assurance systems A quality assurance system (QAS) defines and establishes an organization=s quality policy and objectives. It ensures that the organization has in place those elements necessary to improve efficiency and reduce risks. If properly implemented, a QAS ensures that procedures are carried out consistently, that problems are identified and resolved, and that the organization continuously reviews and improves its

4-8

procedures, products and services. A QAS should proactively identify problems and improve procedures in order to meet corporate objectives.22 In a SMS, these same functions are applied to understanding the human and organizational issues that can impact on safety. A SMS employs similar methods to identify safety problems within the organization and to reduce or eliminate the potential for related accidents through improved procedures. Safety management focuses on the identification and control of safety risks. Notwithstanding the apparent close relationship between quality assurance and safety management, care must be taken not to blur the distinction through the use of terms such as “safety assurance”. Safety cannot be assured.

ORGANIZING FOR SAFETY MANAGEMENT Introduction Safety management (or accident prevention) is not a discrete function to be assigned to a particular organizational element. It is a “modus operandi” which must permeate all aspects of the operation. The safest organizations recognize that accident prevention is a shared responsibility involving many organizational elements. They take a systemic approach to accident prevention, organizing and managing their operations such that they experience proportionally fewer adverse occurrences. What are some of the traits of the safest organizations? In general terms, they23: a) Pursue safety as one of the objectives of the organization and regard safety as a major contributor in achieving production goals; b) Have developed risk management structures, which allow for an appropriate balance between production management and risk management; c) Enjoy a corporate culture in which the active promotion of safety is pervasive; d) Possess an organizational structure which has been designed with a suitable balance of complexity, standardized procedures and centralized decision-making consistent with the objectives of the organization and the characteristics of the environment; e) Rely on internal responsibility, rather than regulatory compliance to achieve safety objectives; and f) Respond to systemic safety deficiencies with long-term, as well as with short-term, measures. Within the framework for safety established by the State, safe organizations make provision for several activities essential for effective accident prevention efforts. Some of these include24: a) Arrangements for the recruitment, development and training of suitably qualified personnel;

22

QASs are discussed further in Chapter 12, Assessing Safety Performance. Control of Crew-Caused Accidents by L.G. Lautman and P.L. Gallimore (both of the Boeing Commercial Airplane Company) reproduced in Flight Safety Foundation: Flight Safety Digest October 1989. 24 Adapted from CAP 712 23

4-9

b) Safety awareness training for management and staff; c) Defined standards and auditing of all operations and services, including those provided by outside contractors; d) Monitoring performance of safety significant equipment, systems and services; e) Hazard identification and risk assessment methods; f) Extra managerial vigilance during the implementation of substantive change to organizational processes, equipment, or procedures; g) Arrangements for staff to easily communicate their significant safety concerns to management; and h) Emergency response planning and testing.

Safety policy, objectives and goals The safety management of so-called ‘safe’ organizations live up to a well-defined safety policy, have realistic safety objectives and seek to fulfil these objectives through achievable safety goals. Each is addressed below: Safety policy. Safety management begins with the development and implementation of clear direction for the conduct of day-to-day affairs. Good safety policy clearly states management’s intentions and aspirations for continuous improvement in the level of safety. Safety policy should flow naturally from the organization’s safety culture; for example, the safety policies of a reactive organization will differ from those which are proactive. Senior management must develop and communicate safety policies that allocate responsibilities and hold people accountable for meeting the organization’s safety goals. Safety policies also describe management’s intentions with respect to the organizational processes and structures to be used. Senior management’s commitment to a positive safety culture begins with the clear direction of its safety policies. As a minimum, a safety policy should include: a) A clear declaration of the organization’s commitment and objectives for safety; b) Safety goals and a regular review of safety performance; c) Clear statements of safety responsibilities for all functional areas; d) Clearly stated accountabilities for safety; e) A means for ensuring compliance with all safety-related rules and regulations; f) A means for ensuring adequate safety management knowledge and skills at all levels; and g) Integration of safety management with other management systems.

4-10

Once the safety policy is defined, procedures must be devised to implement the policy. An organization’s operating procedures must be consistent with its safety policy and appropriate for the personnel responsible for performing them and the tasks being undertaken. Linked closely to an organization’s safety policy are the organization’s safety objectives and goals. a) Objectives provide specific directions for the organization’s planned safety activities (consistent with the safety policy). For example, a company safety policy might include the promotion of open communications with all levels of staff on matters pertaining to safety. To fulfil this policy, the organization may set as an objective the implementation of a voluntary incident reporting system by the end of next year. b) Goals may include the specific steps (including the timetable) for the attainment of the objective. Goals may be seen as the waypoints for achieving the organization’s objectives. For example, goals may be set for achieving a particular rate of incident reporting (say 30 reports per month). This may indicate the effectiveness of the programme in meeting the company’s safety policy by promoting open communications through the implementing of a successful incident reporting system. Objectives and goals must be achievable and clearly define the limits within which the organization will operate. They must be unambiguous, well documented, readily accessible and be reviewed on a regular basis. Structure25 An organization’s structure plays an important part in its safety performance and culture. Organizations can be simple or complex. They can centralize decision-making authority or they can delegate it widely throughout the organization. Several aspects of the organizational structure can facilitate or hinder effective safety management: Complexity is the product of several factors such as the number of management levels, the division of labour, job specialization, centralized vs. decentralized facilities, and technologies used for organizational communications. Standardization defines the extent to which the organization is uniformly structured to meet its objectives. (Uniformity extends into equipment acquisitions and operating procedures.) Centralization refers to the formal decision-making process. (A centralized decision-making process is most effective in a stable environment; however, in an unpredictable environment requiring rapid on-the-spot decisions, a decentralized process may be more appropriate.) Adaptability to change can be the key to success and ultimately the survival of the organization. Organizations must be able to adapt quickly to external changes. There is no “ideal” organizational structure. What works for one organization at a specific time may be totally unsuited for another organization. A large airline with internal maintenance and training 25

Adapted from Human Factors: Management & Organization: Management=s Role in Safety (TC TP 12883 Nov 96) and ICAO document Human Factors Training Manual (Doc 9683)

4-11

departments requires a completely different organization than a small charter operator, which contracts out its maintenance and training. Similarly, there is no single solution as to how best to structure the organization’s safety management functions. However, one key characteristic marks successful safety management organizations. The person appointed to be the organization’s APA reports directly to the most senior levels of the organization. When the APA reports at a subordinate level of management, there is a potential conflict of interest, as intermediate managers may suppress potentially embarrassing safety issues. Senior management may only be told what it wants to hear, rather than what it needs to know about safety vulnerabilities.

— — — — — — — —-

4-12

Appendix 1 to Chapter 4 THREE CORNERSTONES OF AN SMS 26 Effective safety management systems comprise three defining cornerstones. The characteristics for each are outlined below: a) A comprehensive corporate approach to safety which provides for such things as: — Ultimate accountability for corporate safety is assigned to the Board of Directors and Chief Executive Officer (CEO) with evidence of corporate commitment to safety from the highest organizational levels; — A clearly enunciated safety philosophy, with supporting corporate policies, including a non-punitive policy for disciplinary matters; — Corporate safety goals, with a management plan for meeting these goals; — Well defined roles and responsibilities with specific accountabilities for safety published and available to all personnel involved in safety; — A requirement for an independent safety officer (or Accident Prevention Adviser); — Demonstrable evidence of a positive safety culture throughout the organization; — Commitment to a safety oversight process which is independent of line management; — A system of documentation of those business policies, principles, procedures and practices with safety implications; — Regular review of safety improvement plans; and — Formal safety review processes. b) Effective organizational tools for delivering on safety standards through such activities as: — Risk-based resource allocation; — Effective selection, recruitment, development and training of personnel; — Implementation of Standard Operating Procedures (SOPs) developed in cooperation with affected personnel; — Corporate definition of specific competencies (and safety training requirements) for all personnel with duties relating to safety performance; — Defined standards for, and auditing of, asset purchases and contracted services; 26

CAP 712 Op. Cit.

4-13

— Controls for the early detection of - and action on - any deterioration in the performance of safety-significant equipment, systems or services; — Controls for monitoring and recording the overall safety standards of the organization; — The application of appropriate hazard identification, risk assessment and effective management of resources to control identified risks; — Provision for the management of major changes in such areas as the introduction of new equipment, procedures or types of operation, turnover of key personnel, mass layoffs or rapid expansion, mergers and acquisitions; — Arrangements enabling staff to communicate significant safety concerns to the appropriate level of management for resolution and feedback on actions taken; — Emergency response planning and simulated exercises to test the plan’s effectiveness; and — Assessment of commercial policies with regard to their impact on safety. c) A formal system for safety oversight with such desirable elements as: — A system for analysing flight recorder data for the purpose of monitoring flight operations and for detecting unreported safety events; — An organization-wide system for the capture of reports on safety events or unsafe conditions; — A planned and comprehensive safety audit review system which has the flexibility to focus on specific safety concerns as they arise; — A system for the conduct of internal safety investigations, the implementation of remedial actions and the dissemination of such information to all affected personnel; — Systems for the effective use of safety data for performance analysis and for monitoring organizational change as part of the risk management process; — Systematic review and assimilation of best safety practices from other operations; — Periodic review of the continued effectiveness of the safety management system by an independent body; — Line managers= monitoring of work in progress in all safety critical activities to confirm compliance with all regulatory requirements, company standards and procedures, with particular attention to local practices;

4-14

— A comprehensive system for documenting all applicable aviation safety regulations, corporate policies, safety goals, standards, SOPs, safety reports of all kinds, etc. and for making such documentation readily available for all affected personnel; and — Arrangements for ongoing safety promotion based on measured internal safety performance.

____________________

4-15

Chapter 5 RISK MANAGEMENT

General Hazard Identification Risk Assessment • Problem definition • Probability of adverse consequences • Severity of consequences of occurrence • Risk acceptability Risk Control • Defence analysis • Risk control strategies • Brainstorming • Evaluating risk control options Risk Communication Risk Management Considerations for State Administrations • Occasions warranting risk management by State • Communications by State administrations • Benefits of risk management by State administrations

5-1

This page intentionally left blank.

5-2

Chapter 5 RISK MANAGEMENT

Risk management serves to focus safety efforts on those hazards posing the greatest risks.

GENERAL The aviation industry faces a diversity of risks everyday, many capable of compromising the viability of an operator and some even posing a threat to the industry. Indeed, in the aviation industry, risk is a byproduct of doing business. Not all risks can be eliminated, nor are all conceivable accident prevention measures economically feasible. The risks and costs inherent in commercial aviation necessitate a rational process for decision-making. Daily, operators and managers make decisions in real time, weighing the probability and severity of any adverse consequences implied by the risk against the expected gain of taking the risk. This process is known as risk management. For the purposes of this manual risk management can be defined as: Risk Management: the identification, analysis and elimination (and/or control to an acceptable level) of those hazards, as well as the subsequent risks that threaten the viability of an organization. In other words, risk management facilitates the balancing act between assessed risks and viable risk control. It is an integral component of safety management programmes. Risk management involves a logical process of objective analysis, particularly in the evaluation of the risks. However, in striving for the highest level of safety, it must be accepted that absolute safety is unachievable. An overview of the process for Risk Management is summarized in the flow chart at Figure 5-1 below.

5-3

Identify the hazards to aircraft, personnel or the organization.

HAZARD IDENTIFICATION

RISK ASSESSMENT Severity / Criticality

Evaluate the seriousness of the consequences of the hazard occurring.

RISK ASSESSMENT Probability of Occurrence

What are the chances of it happening.

Is the consequent risk acceptable and within the organization’s safety performance criteria.

YES Accept the risk

RISK ASSESSMENT Acceptability

NO Take action to reduce the risk to an acceptable level

RISK CONTROL

Figure 5-1. Risk management process

As Figure 5-1 indicates, risk management comprises three essential elements: hazard identification, risk assessment and risk control. Although risk management is often thought of in the context of aircraft operators, the concepts have equal application in the decision-making by management in air traffic control, maintenance, airport management, manufacturer (e.g. aircraft, engines, components) and State administrations.

5-4

HAZARD IDENTIFICATION For the purposes of this manual, a hazard is any situation or condition that has the potential to cause adverse consequences, for example, injury or loss of life, property or environmental damage. In other words, hazards create the potential for unacceptable losses. Thus, a hazard includes any condition in the aviation system that could contribute to the unsafe operation of an aircraft. The scope of aviation hazards is wide, including aircraft design, manufacture, operation and maintenance. The SHEL Model (described earlier) provides a useful framework for visualising the scope for potential hazards. For example: a) Design factors, including equipment and task design; b) Procedures and operating practices, including their documentation and checklists, and their validation under actual operating conditions; c) Communications, including the most suitable medium, terminology and such barriers to effective communications as language; d) Personnel factors, such as company policies for recruitment, training, crew scheduling and remuneration; e) Organizational factors, such as the compatibility of production and safety goals, the allocation of resources, operating pressures, effectiveness of internal communications and the corporate safety culture; f) Work environment factors, such as ambient noise and vibration, temperature, lighting and the availability of protective equipment and clothing; g) Regulatory oversight factors, including the applicability and enforceability of regulations; the certification of equipment, personnel and procedures; and the adequacy of surveillance audits; and h) Defences include such factors as the provision of adequate detection and warning systems, the error tolerance of equipment and the extent to which the equipment is hardened against failures. Hazard identification is the act of recognizing the unsafe conditions, which create hazards and defining the characteristics of the hazard. Hazards may be recognized through actual safety events (accidents or incidents), or they may be identified through proactive programmes aimed at identifying hazards before they precipitate an occurrence. In practice, both reactive measures and proactive programmes provide an effective means of identifying hazards for accident prevention. Reported safety events are clear evidence of problems in the system and therefore, provide an opportunity to learn valuable safety lessons. Safety events should therefore be investigated to identify the hazards putting the system at risk. This involves investigating all the factors, including the organizational and human factors that played a role in the event. Guidance for investigating safety events for accident prevention is included in Chapter 15, Investigating for Accident Prevention.

5-5

Organizations with safety management systems will also use proactive processes for the identification of hazards. Methods of identifying hazards proactively include: a) Trend monitoring and analysis of safety data; b) Analysis of incident reports, maintenance service difficulty reports, etc; c) Analysis of operational data from flight recorders, line operational safety audits, etc.; d) Safety surveys of employees; and e) Safety inspections and audits; etc. Subsequent chapters provide more details on proactive methods of hazard identification. Although hazard identification is an on-going process, there are times in an organization’s life that special attention to hazard identification is warranted: a) Whenever there is an unexplained increase in safety-related events or safety infractions; b) When major operational changes are planned, including key personnel, routes, aircraft types, etc.; c) If the organization is undergoing change, such as rapid growth or contraction; and d) Mergers, acquisition or downsizing. Having identified a hazard suspected of posing a risk to the organization, the characteristics of the unsafe condition need to be defined. This requires considering factors such as: a) Extent to which the hazard exists in the organization (or aviation system). For example, is it limited to a particular group of personnel, aircraft type, operation, location; b) Adequacy of existing hazard control measures (or defences); and c) Extent to which the identified hazard is already being addressed? Such questions help define the characteristics of the problem and reduce the likelihood of initiating premature or inappropriate corrective action.

RISK ASSESSMENT Having confirmed the presence of a safety hazard, some form of analysis is required to assess its potential for harm or damage. Typically, this assessment of the hazard involves three considerations: a) The probability of the hazard precipitating an unsafe event (i.e. the probability of adverse consequences should the underlying unsafe conditions be allowed to persist); b) The severity of the potential adverse consequences, or the outcome of such an unsafe event; and

5-6

c) The rate of exposure to the hazards. The probability of adverse consequences increases through increased exposure to the unsafe conditions (thus, exposure may be viewed as another dimension of probability). Risk is the assessed potential for adverse consequences resulting from a hazard. It is the probability that during a defined period of activity, the hazard will result in an accident with definable consequences. It is the likelihood that the hazard=s potential to cause harm will be realized. Risk assessment involves consideration of both the probability and the severity of any adverse consequences; in other words, the loss potential is determined. In carrying out risk assessments, it is important to distinguish between hazards (the potential to cause harm) and risk (the likelihood of that harm being realized in a specified period of time). A risk assessment matrix (such as that provided in Table 5-1 below) is a useful tool for prioritizing the hazards most warranting attention. There are many ways to approach the analytical aspects of risk assessment — some more formal than others. For some risks, the number of variables and the availability of both suitable data and mathematical models may lead to credible results with quantitative methods (requiring mathematical analysis of specific data). However, few hazards in aviation lend themselves to credible analysis solely through numerical methods. Typically, these analyses are supplemented qualitatively through critical and logical analysis of the known facts and their relationships. Considerable literature is available on the types of analysis used in risk assessment. Chapter 10 (Safety Analysis, Studies and Assessments) includes some common analytical methods, including statistical analytical techniques and analytical tools for assessing potential hazards. For the risk assessments discussed in this manual, sophisticated methods are not required; a basic understanding of a few methods will suffice. Whatever methods are used, there are various ways by which risks may be expressed. For example: a) Number of deaths, loss of revenue, or loss of market share (i.e.. absolute numbers); b) Loss rates (e.g. number of fatalities per 1,000,000 seat miles flown); c) Probability of serious accidents (e.g. 1 every 50 years); d) Severity of outcomes (for example, injury severity); and e) Expected dollar value of losses vs. annual operating revenue (e.g. $1 million loss per $200 million revenue).

Problem definition In any analytical process, the problem must first be defined. In spite of identifying a perceived hazard, defining the characteristics of the hazard into a problem for resolution is not always easy. People from different backgrounds and experience will likely view the same evidence from different perspectives. Why something poses a significant risk will reflect these different backgrounds, exacerbated by normal human biases. Thus, engineers will tend to see problems in terms of engineering deficiencies; medical doctors as medical deficiencies, and psychologists as behavioural problems, etc. The anecdote in the following box exemplifies the multifaceted nature of defining a problem.

5-7

Charlie’s Accident Charlie has an emotional argument with his wife and proceeds to the local bar where he consumes several drinks. He departs the bar in his car at high speed. Minutes later, he loses control on the highway and is fatally injured. We know what happened; we must now determine WHY it happened. The investigation team is comprised of six specialists, each of whom has a completely different perspective on the root safety deficiency. The sociologist identifies a breakdown in interpersonal communications within the marriage. An enforcement officer from the Liquor Control Board notes the illegal sale of alcoholic beverages by the bar on a "two for one" basis. The pathologist determines that Charlie's blood alcohol was in excess of the legal limit. The highway engineer finds inadequate road banking and protective barriers for the posted speed. An automotive engineer determines that Charlie's car had a loose front end and bald tires. The policeman determines that the automobile was traveling at excessive speed for the prevailing conditions. Each of these different perspectives may result in a different definition of the underlying hazard.

Any or all of the factors cited in this example may be valid, underlining the nature of multi-causality. But, how the safety issue is defined will affect the course of action taken to reduce or eliminate the hazards. In assessing the risks, the analyst must evaluate all potentially valid perspectives and pursue the most suitable.

Probability of adverse consequences Regardless of the analytical methods used, the probability of causing harm or damage must be assessed. This probability will depend on answers to such questions as: a) Is there a history of occurrences like this, or is this an isolated occurrence? b) How many other aircraft, or components of this type, might have similar defects? c) How many operating or maintenance personnel are following, or are subject to, the procedures in question? d) What percentage of the time is the suspect equipment, or the questionable procedure, in use? and e) To what extent are there organizational, management or regulatory implications which might reflect larger threats to public safety? Based on such considerations, the likelihood of an event occurring can be assessed. For example: Unlikely to occur. Failures which are “unlikely to occur” include isolated occurrences, and risks where the exposure rate is very low or the fleet size is small. The complexity of the circumstances necessary to create an accident situation may be such that it is unlikely such a chain of events will arise again. For example, it is unlikely that independent systems would fail concurrently. However,

5-8

even if the possibility is only remote, the consequences of such concurrent failures may warrant follow-up. There is a natural tendency to attribute unlikely events to “coincidence”. Caution is advised. While coincidence may be statistically feasible, coincidence must not be used as an excuse for the absence of due analysis. May occur. Failures which "may occur" derive from hazards with a reasonable probability that similar patterns of human performance can be expected under similar working conditions, or that the same material defects exist elsewhere in the system. Probably will occur. Such occurrences reflect a pattern (or potential pattern) of material failures that have not yet been rectified. Given the design or maintenance of the equipment, its strength under known operating conditions, etc., continued operations will likely lead to failure. Similarly, given the empirical evidence on some aspect of human performance, it can be expected with some certainty that normal individuals, operating under similar working conditions, would likely commit the same errors or be subject to the same undesirable performance outcome.

Severity of consequences of occurrence Having determined the probability of occurrence, the nature of the adverse consequences if the event does occur needs to be assessed. The potential consequences govern the degree of urgency attached to the safety action required. If there is significant risk of catastrophic consequences, or if the risk of serious injury, property or environmental damage is high, urgent follow-up action is warranted. In assessing the severity of the consequences of occurrence, the following types of questions could apply: a) How many lives are at risk? Employees, fare paying passengers, bystanders or the general public; b) What is the likely extent of property or financial damage? Direct property loss to the operator, damage to aviation infrastructure, third party collateral damage, financial impact and economic impact for the State; c) What is the likelihood of environmental impact? Spill of fuel or other hazardous product and physical disruption of natural habitat; and d) What is the likely political implications and/or media interest?

Risk acceptability Based on the risk assessment, the risks can be prioritized relative to other, unresolved safety hazards. This is critical in making rational decisions to allocate limited resources against those hazards posing the greatest risks to the organization. Prioritizing risks requires a rational basis for ranking one risk vis à vis other risks. Criteria or standards are required to define what is an acceptable risk and what is an unacceptable risk. By weighing the likelihood of an undesirable outcome against the potential severity of that outcome, the risk can be categorized within a risk assessment matrix. Many versions of risk assessment matrices are available from

5-9

the literature. While the terminology or definition used for the different categories may vary, such tables generally reflect the ideas summarized in the Table below: Table 5-1. Risk assessment matrix27 SEVERITY OF CONSEQUENCES

LIKELIHOOD OF OCCURRENCE

Aviation definition

Meaning

Value

Qualitative definition

Meaning

Value

Catastrophic

Aircraft destroyed Multiple deaths

5

Frequent

Likely to occur many times

5

Hazardous

A large reduction in safety margins, physical distress or a workload that the crew cannot be relied upon to perform their tasks accurately or completely. Serious injury or death of a proportion of the occupants. Major aircraft damage.

4

Occasional

Likely to occur some times

4

Major

A significant reduction in safety margins, a reduction in the ability of the flight crew to cope with adverse operating conditions as a result of increase in workload or as a result of conditions impairing their efficiency. Aircraft serious incident. Injury to occupants.

3

Remote

Unlikely, but possible to occur

3

Minor

Nuisance. Operating limitations. Use of emergency procedures. Aircraft minor incident.

2

Improbable

Very unlikely to occur

2

Negligible

Little consequence

1

Extremely improbable

Almost inconceivable that the event will occur

1

27

Based on: An Introduction to Flight Safety Risk Assessment (Paper by Richard Profit UK CAA)

5-10

In this version of a risk assessment matrix: a) Severity of risk is ranked as Catastrophic, Hazardous, Major, Minor or Negligible with a

descriptor for each indicating the potential severity of consequences. As mentioned earlier, other definitions can be used, reflecting the nature of the operation being analysed; b) Probability (or Likelihood) of occurrence is also ranked through five different levels of

qualitative definition and descriptors are provided for each likelihood of occurrence; and c) Values may be assigned numerically, to weigh the relative importance of each level of

severity and probability. A composite assessment of risk, to assist in comparing risks, may then be derived by multiplying the severity and probability values. Having used a risk matrix to assign values to risks, a range of values may be assigned in order to categorize risks as acceptable, undesirable or unacceptable. a) Acceptable means that no further action needs to be taken, (unless the risk can be reduced further at little cost or effort). b) Undesirable (or tolerable) means that the affected persons are prepared to live with the risk in order to have certain benefits, in the understanding that the risk is being controlled as best as possible. c) Unacceptable means that operations under the current conditions must cease until the risk is reduced to at least the Tolerable level. A less numeric approach to determining the acceptability of particular risks includes consideration of such factors as: a) Managerial. Is the risk consistent with the organization’s safety policy and standards? b) Affordability. Does the nature of the risk defy cost-effective resolution? c) Legal. Is the risk in conformance with current regulatory standards and enforcement capabilities? d) Cultural. How will the organization’s personnel and other stakeholders view this risk? e) Market. Will the company=s competitiveness and well-being vis-à-vis other companies be compromised by not reducing or eliminating this risk? f) Political. Will there be a political price to pay for not reducing or eliminating this risk? g) Public. How influential will the media or special interest groups be in affecting public opinion regarding this risk?

5-11

RISK CONTROL Where risk is concerned, there is no such thing as absolute safety. Risks have to be managed to a level “as low as reasonably practicable” (ALARP28). This means that the risk must be balanced against the time, cost and difficulty of taking measures to reduce or eliminate the risk. When the acceptability of the risk has been found to be Undesirable or Unacceptable, control measures need to be introduced — the higher the risk, the greater the urgency. The level of risk can be reduced either by reducing the severity of the potential consequences, or by reducing the likelihood of occurrence. Safety action is required to reduce the risks. Optimal solutions will vary, depending on the local circumstances and exigencies. In formulating meaningful safety action, a good understanding of the adequacy of existing defences is required. Defence analysis29 A major component of any safety system is the defences put in place to protect people, property or the environment. These defences can be used to: a) Reduce the probability of unwanted events occurring; and b) Reduce the severity of the consequences associated with any unwanted events. Defences can be categorized into two types: a) Physical defences include objects that discourage or prevent inappropriate action, or which mitigate the consequences of events (for example, squat switches, switch covers, firewalls, survival equipment, warnings and alarms); and b) Administrative defences include procedures and practices that mitigate the probability of an accident (for example, safety regulations, standard operating procedures, supervision and inspection, and personal proficiency). Before selecting appropriate risk control strategies, it is important to understand why the existing system of defences was inadequate. The following line of questioning may pertain: a) Were defences provided to protect against such hazards? b) Did the defences function as intended? c) Were the defences practical for use under actual working conditions? d) Were affected staff aware of the risks and the defences in place? and e) Are additional risk control measures required?

28 29

From SRG SMS Policies and Guidelines Document: Safety Management Systems (Sponsor PS Griffith, Head A&ATSD) p7 From TSB’s Integrated Safety Investigation Methodology

5-12

Risk control strategies There is a range of strategies available for risk control. For example: a) Exposure avoidance. The risky task, practice, operation or activity, is avoided because the risk exceeds the benefits. b) Loss reduction. Activities that reduce the frequency of the unsafe events or the magnitude of the consequences. c) Segregation of exposure (separation or duplication). Activities that isolate the effects of the risk or build in redundancy to protect against the risks. In other words, reduce the severity of the risk. For example, protecting against collateral damage in the event of a material failure, or providing back-up systems to reduce the likelihood of total system failure.

Brainstorming Generating the ideas necessary to create suitable risk control measures poses a challenge. Developing risk control measures frequently requires creativity, ingenuity and above all an open mind to consider all possible solutions. The thinking of those closest to the problem (usually with the most experience) is often coloured by set-ways and natural biases. Broad participation, including representatives of the various stakeholders, tends to help overcome rigid mind-sets. Thinking “outside the box” is essential to effective problem solving in a complex world. All new ideas should be weighed carefully before rejecting any of them.

Evaluating risk control options In evaluating alternatives for risk control, not all are of equal potential for reducing risks. The effectiveness of each option needs to be evaluated before a decision can be taken. It is important that the full range of possible control measures is considered and that trade-offs between measures are considered to find an optimal solution. Each proposed risk control option should be examined from such perspectives as: a) Effectiveness. Will it reduce or eliminate the identified risks? To what extent do alternatives mitigate the risks? Effectiveness can be viewed as being somewhere along a continuum, as follows: 1) Level One: (Engineering actions) The safety action will eliminate the risk, for example, interlocks to prevent thrust reverser activation in-flight; 2) Level Two: (Control actions) The safety action accepts the risk but adjusts the system to control the risk by reducing it to a manageable level, such as, imposing more restrictive operating conditions; and 3) Level Three: (Personnel actions) The actions taken accept that the hazard can neither be eliminated (Level One) nor controlled (Level Two), so personnel must be taught how to cope with it, such as, the addition of a cockpit warning, revised check list and extra training.

5-13

b) Cost / benefit. Do the perceived benefits of the option outweigh the costs? Will the potential gains be proportional to the impact of the change required? c) Practicality. Is it doable and appropriate in terms of, available technology, financial feasibility, administrative feasibility, governing legislation and regulations, political will, etc.? d) Challenge. Can the risk control measure withstand critical scrutiny from all stakeholders? (Employees, managers, stockholders/State administrations, etc.) e) Acceptability to each stakeholder. How much buy-in (or resistance) from stakeholders can be expected? (Discussions with stakeholders during the Risk Assessment phase may indicate their preferred risk control option.) f) Enforceability. If new rules (SOPs, regulations, etc.) are implemented, are they enforceable? g) Durability. Will the measure withstand the test of time? Will it be of temporary benefit or will it have long-term utility? h) Residual Risks. After the safety action is taken, what will be the residual risks relative to the original unsafe condition? What is the ability to mitigate any residual risks? i)

New Problems. What new problems, or new (perhaps worse) risks will be introduced by the proposed change?

Obviously, preference should be given to corrective actions that will completely eliminate the risk. Regrettably, such solutions are often the most expensive. At the other end of the spectrum, when insufficient resources or organizational will are present, the problem is often deferred to the training department to teach staff to cope with the risks. In such cases, management may be avoiding hard decisions by delegating responsibility for the risk to subordinates.

RISK COMMUNICATION Risk communication includes any exchange of information about risks, i.e. any public or private communication that informs others about the existence, nature, form, severity or acceptability of risks. The information needs of the following groups may require special attention: a) Management must be apprised of all risks which present loss potential to the organization; b) Those exposed to the identified risks must be apprised of their severity and likelihood of occurrence; c) Those who identified the hazard need feedback on action proposed; d) Those affected by any planned changes need to be apprised of both the hazards and the rationale for the action taken; and e) Others with potential information needs regarding specific risks include: 1) Regulatory authorities;

5-14

2) Suppliers; 3) Industry associations; and 4) General public, etc. Effective communication of the risks (and plans for their resolution) adds value to the Risk Management process. The stakeholders can assist the decision-maker(s) if the risks are communicated early in a fair, objective and understandable way. Thus, risk communications should be: a) Two-way; b) Factual concerning the risks, benefits and uncertainties, and take into account the needs and concerns of the various stakeholders; c) Tested for acceptability by the decision-maker(s) and stakeholders; and d) Documented to reduce misunderstandings. Failure to communicate the safety lessons learned in a clear and timely fashion will undermine management’s credibility in promoting a positive safety culture. For safety messages to be credible, they must be consistent with the facts, with previous statements from management and with the messages from other authorities. These messages need to be framed in terms the stakeholders understand. RISK MANAGEMENT CONSIDERATIONS FOR STATE ADMINISTRATIONS30 Risk management techniques have implications for State administrations in areas ranging from policy development through to the “go/no-go” decisions confronting front-line State civil aviation inspectors. For example: a) Policy: To what extent should a State accept the certification paperwork of another State; b) Regulatory Change: From the many (often-conflicting) recommendations made for regulatory change, how are decisions made; c) Priority Setting: How are decisions made for determining those areas of safety warranting emphasis during safety oversight audits; d) Operational Management: How are decisions made when insufficient resources are available to carry out all planned activities; and e) Operational Inspections: At the front line, how are decisions made when errors are discovered outside of normal working hours.

30

Adapted from TP 13095 Risk Management & Decision Making, Transport Canada

5-15

Occasions warranting risk management by State administrations Some situations should alert State aviation administrations to the possible need for applying risk management methods, for example: a) Start-up or rapidly expanding companies; b) Corporate mergers; c) Companies facing bankruptcy or other financial difficulties; d) Companies facing serious labour-management difficulties; e) Introduction of major new equipment by an operator; f) Certification of a new aircraft type, new airport, etc.; g) Introduction of new communication, navigation or surveillance equipment and procedures; and h) Significant change to air regulations or other laws potentially impacting on aviation safety, etc. Risk management by State administrations will be affected by such factors as: a) Time available to make the decision (grounding an aircraft, revoking a certificate, etc.); b) Resources available to effect the necessary actions; c) Numbers of people affected by required actions (company-wide, fleet-wide, regional, national, international, etc.); d) The potential impact of the State’s decision for action (or inaction); and e) Cultural and political will to take the action required.

Communications by State administrations Effective communications by State Administrations are an important part of the risk management process. Establishing and maintaining good communications with stakeholders paves the way for effective decision-making. Stakeholders’ perceptions about risk will often differ from those of the regulatory authority. Communicating the reasons for public safety decisions requires care because such decisions often arouse strong emotions. Gaining acceptance of planned changes is more likely when key stakeholders are involved in the decision-making process. Such action reflects the safety culture of the State administration; is it one of management by decree, or one which values openness, respect, consultation, collaboration, partnership, etc.

5-16

Benefits of risk management for State administrations Applying risk management techniques in decision-making offers benefits for State administrations, including: a) Avoiding costly mistakes during the decision-making process; b) Ensuring all aspects of the risk are identified and considered when making decisions; c) Ensuring the legitimate interests of affected stakeholders are considered; d) Providing decision-makers with a solid defence in support of decisions; e) Making decisions easier to explain to stakeholders and the general public; and f) Providing significant savings in time and money.

____________________

5-17

Chapter 6 INCIDENT REPORTING SYSTEMS

Introduction to Reporting Systems • Value of reporting systems • ICAO requirements Types of Incident Reporting Systems • Mandatory incident reporting systems • Voluntary incident reporting systems • Confidential reporting systems Principles for Effective Incident Reporting Systems • Trust • Non-punitive • Inclusive reporting base • Independence • Ease of reporting • Acknowledgment • Promotion International Incident Reporting Programmes • ICAO Accident/Incident Reporting System (ADREP) • European Coordination Centre for Aviation Incident Reporting Systems (ECCAIRS) State Voluntary Incident Reporting Systems • Aviation Safety Reporting System (ASRS) • Confidential Human Factors Incident Reporting Programme (CHIRP) Company Reporting Systems • Benefits • Encouraging the free flow of safety information • Commercially available systems — BASIS — INDICATE Implementation of Incident Reporting Systems • What to report? • System management • Reporting method and format • Limitations on the use of incident data

6-1

Appendices 1. Sample airline policy on non-punitive hazard reporting 2. Examples of items to be reported in an airline occurrence reporting system 3. Limitations on the use of data from voluntary incident reporting systems

6-2

Chapter 6 INCIDENT REPORTING SYSTEMS INTRODUCTION TO REPORTING SYSTEMS A great deal is learned about safety deficiencies from accident investigations. Fortunately, aviation accidents are rare events. They are, however, generally investigated more thoroughly than incidents. When safety initiatives rely exclusively on accident data, the limitations of small samples apply. As a result, the wrong conclusions may be drawn, or inappropriate corrective actions taken. Research leading to the 1:600 Rule showed that the number of incidents is significantly greater than the number of accidents for comparable types of occurrences. The causal and contributory factors associated with incidents may also culminate in accidents. Often, only good fortune prevents an incident from becoming an accident. Unfortunately, these incidents are not always known to those responsible for reducing or eliminating the associated risks. This may be due to the unavailability of reporting systems, or people not being sufficiently motivated to report incidents.

Value of reporting systems Recognizing that knowledge derived from incidents could provide significant insights into safety hazards, several types of incident reporting systems have been developed. Depending on the type of reporting programme, a rich source of data for safety analysis may be available. Some safety databases contain a large quantity of detailed information31. Although these occurrences may not be investigated to any depth, the anecdotal information they provide can offer meaningful insight into the perceptions and reactions of pilots, cabin attendants, mechanics and air traffic controllers. Safety reporting systems should not just be restricted to incidents, but should include provision for the reporting of hazards, i.e. unsafe conditions which have not yet caused an incident. For example, some organizations have programmes for reporting conditions deemed unsatisfactory from the perspective of experienced personnel (Unsatisfactory Condition Reports). In some States, Service Difficulty Reporting (SDR) systems are effective in identifying airworthiness hazards. Aggregating data from such hazard and incident reports provides a rich source of experience to support risk management programmes. Data from incident reporting systems can facilitate an understanding of the causes of hazards, help define intervention strategies, and the effectiveness of interventions. Depending on the depth to which they are investigated, incidents can provide a unique means of obtaining first-hand evidence on the factors associated with mishaps from the participants themselves. Reporters can describe the relationships between stimuli and their actions. They may provide their interpretation of the effects of various factors affecting their performance, such as fatigue, interpersonal interactions and distractions. Furthermore, many reporters are able to offer valuable suggestions for remedial action. Incident data have also been used to improve operating procedures, display and control design, and provide a better understanding of human performance associated with the operation of aircraft and air traffic control.

31

The US Aviation Safety Reporting System (ASRS) holds hundreds of thousands of aviation occurrences.

6-3

ICAO requirements32 ICAO requires that States establish a mandatory incident reporting system to facilitate collection of information on actual or potential safety deficiencies. In addition, States are encouraged to establish a voluntary incident reporting programme, adjusting their laws, regulations and policies so that the voluntary programme: a) Facilitates the collection of information that may not be captured by a mandatory incident reporting system; b) Is non-punitive; and c) Affords protection to the sources of the information.

TYPES OF INCIDENT REPORTING SYSTEMS In general, an incident involves an unsafe, or potentially unsafe, occurrence or condition that does not involve serious personal injury or significant property damage; that is, it does not meet the criteria for an accident, but could have. When an incident occurs, the individual(s) involved may or may not be required to submit a report. The reporting requirements vary with the laws of the State where the incident occurred. Even if not required by law, operators may require reporting of the occurrence to the company.

Mandatory incident reporting system Annex 13 requires States to establish a mandatory incident reporting system to facilitate the collection of information on actual or potential safety deficiencies. In a mandatory system, people are required to report certain types of incidents. This necessitates detailed regulations outlining who shall report and what shall be reported. The number of variables in aircraft operations is so great that it is difficult to provide a comprehensive list of items or conditions which should be reported. For example, loss of a single hydraulic system on an aircraft with only one such system is critical. On a type with three or four systems, it may not be. A relatively minor problem in one set of circumstances can, in different circumstances, result in a hazardous situation. However, the rule should be: AIf in doubt — report it.@ Because mandatory systems deal mainly with “hardware” matters, they tend to collect more information on technical failures than on the Human Factor aspects. To help overcome this problem, States with well-developed mandatory reporting systems are introducing voluntary incident reporting systems aimed specifically at acquiring more information on the human factor aspects of occurrences.

Voluntary incident reporting systems Annex 13 recommends that States introduce voluntary incident reporting systems to supplement the information obtained from mandatory reporting systems. In such systems, the reporter, without any legal or administrative requirement to do so, submits a voluntary incident report. In a voluntary reporting system, regulatory agencies may offer an incentive to report. For example, enforcement action may be 32

See Annex 13, Ch 8

6-4

waived for unintentional violations that are reported. The reported information should not be used against the reporters, i.e. such systems must be non-punitive to encourage the reporting of such information.

Confidential reporting systems

Confidential reporting systems aim to protect the identity of the reporter. This is one way of ensuring that voluntary reporting systems are non-punitive. Confidentiality is usually achieved by de-identification, often by not recording any identifying information of the occurrence. One such system returns to the user the identifying part of the reporting form and no record is kept of these details. Confidential incident reporting programmes facilitate the disclosure of human errors, enabling others to learn from mistakes made, without fear of retribution or embarrassment.

PRINCIPLES FOR EFFECTIVE INCIDENT REPORTING SYSTEMS People are understandably reluctant to report their mistakes to the company that employs them, or to the government department that regulates them. Too often following an occurrence, investigators learn that many people were aware of the unsafe conditions before the event. For whatever reasons, however, they did not report the perceived hazards, perhaps because of: a) Embarrassment in front of their peers; b) Self-incrimination, especially if they were responsible for creating the unsafe condition; c) Retaliation from their employer for having spoken out; or d) Sanction (such as enforcement action) by the regulatory authority. Use of the following principles help overcome the natural resistance to safety reporting.

Trust Persons reporting incidents must trust that the receiving organization (whether the State or company) will not use the information against them in any way. Without such confidence, people will be reluctant to report their mistakes and they may also be reluctant to report other hazards they are aware of. Trust begins with the design and implementation of the programme. Employee input into the development of a reporting system is vital. A positive safety culture in the organization generates the kind of trust necessary for a successful incident reporting system. Specifically, the culture must be error tolerant and non-punitive. In addition, incident reporting systems need to be perceived as being fair in how they treat unintentional errors or mistakes. (Most people do not expect an incident reporting system to exempt criminal acts, or deliberate violations, from prosecution or disciplinary action.) Some States consider such a process to be an example of a “Just Culture”.

6-5

Non-punitive Non-punitive reporting systems are based on confidentiality. Before employees will freely report incidents, they must receive a commitment from the regulatory authority or from top management that reported information would not be used punitively against them. The person reporting the incident (or unsafe condition) must be confident that anything said will be kept in confidence. In some States, “Access to Information” laws make it increasingly difficult to guarantee confidentiality. Where this happens, reported information will tend to be reduced to the minimum to meet mandatory reporting requirements. Sometimes reference is made to anonymous reporting systems. Reporting anonymously is not the same as confidential reporting. Most successful reporting programmes have some type of callback capability in order to confirm details, or obtain a better understanding of the occurrence. Reporting anonymously makes it impossible to ensure understanding and completeness of the information provided by the reporter. There is also a danger that anonymous reporting may be used for purposes other than safety.

Inclusive reporting base Early voluntary incident reporting programmes were targeted at flight crews. Pilots are in a position to observe a broad spectrum of the aviation system, and are therefore well situated to comment on the system=s health. Nonetheless, incident reporting systems which focus solely on the flight crew’s perspective, tend to reinforce the idea that everything comes down to pilot error. Taking a systemic approach to accident prevention requires that safety information be obtained from all parts of the operation. In State-run incident reporting systems, collecting information on the same occurrence from different perspectives facilitates forming a more complete impression of events. For example, ATC instructs an aircraft to ‘go around’ because there is a maintenance vehicle on the runway without authorization. Undoubtedly, the pilot, the controller and the vehicle operator would all have seen the situation from different perspectives. Relying on one perspective only may not provide a complete understanding of the event.

Independence Ideally, State-run voluntary incident reporting systems are operated by an organization separate from the aviation administration responsible for the enforcement of aviation regulations. Experience in several States has shown that voluntary reporting benefits from a trusted Athird party” managing the system. The Athird party” receives, processes and analyses the incident reports and feeds the results back to the aviation administration and the aviation community. With Amandatory@ reporting systems, it may not be possible to employ a Athird party@. Nevertheless, it is desirable that the aviation administration gives a clear undertaking that any information received will be used for accident prevention purposes only. The same principle applies to an airline or any other aviation operator that uses incident reporting as part of its accident prevention programme.

6-6

Ease of reporting The task of submitting incident reports should be as easy as possible for the reporter. Reporting forms should be readily available so that anyone wishing to file a report can do so easily. They should be simple to compile, with adequate space for a descriptive narrative and they should encourage suggestions on how to improve the situation or prevent a reoccurrence. To simplify completion, classifying information, such as the type of operation, light conditions, type of flight plan, weather, etc. can use a Atick-off” format.

Acknowledgment The reporting of incidents requires time and effort by the reporter and should be appropriately acknowledged. To encourage further reports, one State includes a blank report form with the acknowledgment letter. In addition, the reporter naturally expects feedback about actions taken in response to the reported safety concern.

Promotion The (de-identified) information received from an incident reporting system should be made available to the aviation community in a timely manner. This may also help to motivate people to report further incidents. Such promotion activities may take the form of monthly newsletters or periodic summaries. Ideally a variety of methods would be used with a view to achieving maximum exposure.

INTERNATIONAL INCIDENT REPORTING PROGRAMMES ICAO Accident/Incident Data Reporting System (ADREP) In accordance with Annex 13, States report to ICAO information on all aircraft accidents, which involve aircraft of a maximum certified take-off mass of over 2,250 kg. ICAO also gathers information on aircraft incidents (involving aircraft over 5,700 kg.) considered to be important for safety and accident prevention. This reporting programme is known as ADREP. States report specific data in a predetermined (and coded) format to ICAO. When ADREP reports are received from States, the information is checked and electronically stored, constituting a databank of worldwide occurrences. ICAO does not require States to investigate incidents. However, if a State does investigate a serious incident, they are requested to submit formatted data to ICAO. The types of serious incidents of interest to ICAO include: a) Multiple system failures; b) Fires or smoke on-board an aircraft; c) Terrain and obstacle clearance incidents; d) Flight control and stability problems; e) Take-off and landing incidents;

6-7

f) Flight crew incapacitation; g) Decompression; and h) Near collisions and other air traffic incidents. European Coordination Centre for Aviation Incident Reporting Systems (ECCAIRS) 33 Many aviation authorities in Europe have collected information about aviation accidents and incidents. However, the number of significant occurrences in individual States was usually not sufficient to give an early indication of potentially serious hazards or to identify meaningful trends. Since many States had incompatible data storage formats, pooling of safety information was almost impossible. To improve this situation, the European Union (EU) introduced occurrence-reporting requirements and developed the ECCAIRS safety database system. The objective of these moves was to improve aviation safety in Europe through the early detection of potentially hazardous situations. The ECCAIRS system includes capabilities for analysing and presenting the information in a variety of formats. The database is compatible with some other incident reporting systems, such as ADREP. Some non-European States have also chosen to implement the ECCAIRS system to take advantage of common classification taxonomies, etc.

STATE VOLUNTARY INCIDENT REPORTING SYSTEMS A number of States operate successful voluntary incident reporting systems that utilize common features. Three such systems are described below: Aviation Safety Reporting System (ASRS)34 The United States operates a large aviation occurrence reporting system, known as the Aviation Safety Reporting System (ASRS). The ASRS operates independently from the Federal Aviation Administration (FAA) and is administered by NASA. Pilots, air traffic controllers, cabin crew, mechanics, ground personnel, and others involved in aviation operations may submit reports when aviation safety has been considered to be compromised. Samples of reporting forms are at the ASRS Website. Reports sent to the ASRS are held in strict confidence. All reports are de-identified before being entered into the database. All personal and organizational names are removed. Dates, times and related information, which might reveal an identity, are either generalized or eliminated. ASRS data are used to: a) Identify systemic hazards in the national aviation system for remedial action by appropriate authorities; b) Support policy formulation and planning in the national aviation system; c) Support research and studies in aviation, especially including human factors safety research; and d) Providing information to promote accident prevention. 33 34

For more information on ECCAIRS visit their website at http://eccairs-www.jrc.it The ASRS website is at http://asrs.arc.nasa.gov

6-8

The FAA recognizes the importance of voluntary incident reporting to accident prevention and offers ASRS reporters some immunity from enforcement actions, waiving penalties for unintentional violations reported to ASRS. With over 300,000 reports now on file, this database supports research in the aviation safety — especially relating to human factors. Confidential Human Factors Incident Reporting Programme (CHIRP)35 CHIRP contributes to the enhancement of flight safety in the United Kingdom, by providing a confidential reporting system for all individuals employed in aviation. It complements the United Kingdom’s Mandatory Occurrence Reporting system. Noteworthy features of CHIRP include: a) Independence from the regulatory authority; b) Broad availability (including flight crew members, air traffic control officers, licensed aircraft maintenance engineers, cabin crew and the general aviation community); c) Confidentiality of reporters’ identities; d) Analysis by experienced safety officers; e) Newsletters with broad distribution to improve safety standards by sharing safety information; and f) Participation by CHIRP representatives on several aviation safety bodies to assist in resolving systemic safety issues. COMPANY REPORTING SYSTEMS36 In addition to State-operated incident reporting systems (both mandatory and voluntary), many airlines, ATS providers and airport operators are implementing ‘in-house’ programmes for the reporting of safety hazards and incidents. If reporting is available to all personnel (not just flight crews), company reporting systems help promote a positive company-wide safety culture.

Benefits Incident reporting systems are one of an operator’s most effective tools for pro-active hazard identification, a key element in operating an effective safety management system. Policies, procedures and practices developed within an organization sometimes introduce unforeseen hazards into the airline operations system. These latent conditions (hazards) may lie dormant for years. They are usually introduced unknowingly, often with the best of intentions. Examples include poor equipment design, inappropriate management decisions, ambiguously written procedures and inadequate communication between management and line personnel. Line management can also introduce such hazards by instituting operating procedures that do not work as intended under Areal world” conditions. In short, hazards may have their origins far removed in space and time from the incidents that may eventually result from them. 35 36

Visit the CHIRP website at http://www.chirp.co.uk/air FSF Digest Aviation Safety: US Efforts to Implement Flight Operational Quality Assurance Programs, Jul-Sep 1998 p51

6-9

An accident or incident may not result from these hazards immediately because Afront line operators” (whether they be pilots, controllers or mechanics) develop their own ways of coping with the hazard — sometimes described as “workarounds”. However, if the hazards are not identified and addressed, sooner or later the coping mechanisms (or deficiencies) fail and an accident or incident ensues. A properly managed in-house reporting programme can help companies identify many of these hazards. By collecting, aggregating and then analysing hazard and incident reports, safety officers can better understand specific problems encountered during operations. Armed with this knowledge, they can initiate systemic solutions, rather than short-term fixes that may only hide the real problems.

Encouraging the free flow of safety information The trust of employees in the incident reporting programme is fundamental to the quality, accuracy and substance of data reported. If hazard and incident data is collected in a corporate atmosphere where employees feel free to openly share safety information, the data will contain much useful detail. The programme will be helpful in determining contributing factors and areas of concern since it will be representative of the actual line environment. On the other hand, if the company uses incident reports for disciplinary purposes, the company incident reporting programme will only receive the minimum information required to comply with company rules. Little useful information can be expected. The trust necessary for the free flow of useful safety information is very fragile. It may take years to establish; yet, one breech of that trust may undermine the effectiveness of the programme for a long time. Building the necessary trust begins with a formal statement of company policy on its approach to open and free incident reporting. A sample of one company’s policy on non-punitive hazard reporting is at Appendix 1 to this Chapter.

Commercially available systems An increasing number of commercially available incident reporting programmes that can be run on personal computers (PCs) and are available at relatively low cost, have proven to be well suited for managing company systems. These off-the-shelf software packages that can be used for reporting safety events and hazards are all inclusive: they collect and store data, generate reports, and can be used for trend analysis and safety performance monitoring. The following are three examples of such systems: British Airways Safety Information System (BASIS) was created as a company incident-reporting programme for flight crews. This PC-based programme has matured to become a quasi-industry standard for collecting and managing safety information. It is currently used by more than 100 airlines and aviation organizations. Newer systems coming online are frequently developed to be compatible with BASIS. A number of BASIS modules are now available covering a broad spectrum of activities relevant to accident prevention. For further information on BASIS, visit their website at http://www.winbasis.com.

6-10

INDICATE (Identifying Needed Defences in the Civil Air Transport Environment) is a safety management programme developed in Australia to provide a simple, cost-effective and reliable means of capturing, monitoring and reporting information about safety hazards. The INDICATE software was created in Microsoft Access and is easily installed on a Windowscompatible PC. It provides a logical and consistent methodology for recording and categorizing hazards; a means of quickly and easily recording recommendations and responses; a database on which safety hazards can be recorded and tracked; an automated facility for producing reports about hazards so that information can be disseminated easily to everyone who needs to know. It is also a useful tool for safety audit purposes. The Australian Transport Safety Bureau (ATSB) provides the INDICATE software at no cost. For further information on INDICATE, visit their website at http://www.atsb.gov.au.

IMPLEMENTATION OF INCIDENT REPORTING SYSTEMS If implemented in a non-punitive work environment, an incident reporting system can go a long way toward creating a positive safety culture. Depending on the size of the organization, the most expedient method for incident and hazard reporting is to use existing “paperwork” such as safety reports and maintenance reports. However, as the volume of reports increases, some sort of computerized system will be required to handle the task.

What to report Any hazard which has the potential to cause damage or injury or which threatens the organization’s viability should be reported. Hazards and incidents should be reported if it is believed that: a) Something can be done to reduce the accident potential; b) Other aviation personnel could learn from the report; or c) The system and its inherent defences did not work “as advertised.” In short, if in doubt as to the events safety significance, report it. (Those incidents and accidents that are required to be reported in accordance with State laws or regulations governing accident or incident reporting should also be included in an operator’s reporting database.) A sample list of the types of occurrences or events to be reported to an airline’s reporting system is at Appendix 2 to this Chapter. To be effective, an airline’s reporting programme should include hazard and incident reports from at least flight operations personnel, maintenance technicians and cabin crew. State safety incident reporting programmes should also include reports from ATS personnel and airport employees.

System management In establishing a reporting system, management in consultation with the APA and senior operations personnel need to determine the operating procedures for the system. For example:

6-11

a) What types of occurrence, event, or hazard should be reported; b) Where do all the reports go initially; c) How will the reporting be acknowledged; d) Who is responsible for any investigation that may be required; e) How will confidentiality be protected; f) What degree of immunity will be provided and the criteria that will apply for such immunity; g) What is the role of the safety committee; h) What are the criteria for bringing a (de-identified) report to management’s attention; and i)

How will any safety lessons learned and actions taken be disseminated to relevant staff.

Such procedures should be widely disseminated to encourage use of the programme.

Reporting method and format The method and format chosen for a reporting system matters little as long as it encourages personnel to report all hazards or incidents. The reporting process should be as simple as possible, and be well documented, including details as to what, where and when to report. In designing reporting forms, the layout should facilitate the submission of information. Sufficient space should be provided to encourage reporters to identify suggested corrective actions. Other factors to be considered in designing a system and reporting forms include: a) Pilots are generally not prolific writers, therefore the form should be kept as short as possible; b) Reporters are not safety analysts, therefore, the questions should be in simple, everyday language; c) Use of non-directive questions instead of leading questions. (Non-directive questions include: What happened? Why? How was it fixed? What should be done?); d) Prompts may be required for the reporter to think about >system failures= (how close were they to an accident?), and to consider their error management strategies; e) Focus should be on the detection and recovery from an unsafe situation or condition; and f) Reporters should be encouraged to consider the wider safety lessons inherent in the report, e.g. how the organization and the aviation system could benefit from it. Regardless of the source or method of submission, once the information is received it must be stored in a manner suitable for easy retrieval and analysis.

6-12

Limitations on the use of incident data Data gathered through voluntary hazard and incident reporting systems may be subject to a number of limitations. In analysing such data, care must be taken to avoid erroneous conclusions and inappropriate safety actions. Some of the factors to be considered in using incident data include: a) Information may not have been verified; b) Reporters are naturally biased (for example, pilots may see the circumstances of an occurrence differently to an air traffic controller because of their different perspectives); c) Reporting forms may inadvertently reflect bias (such as eliciting irrelevant information and avoiding more crucial information); d) Incident-reporting databases may be biased (by forcing the artificial classification of some data); and e) Shortcomings in trend analyses due to the nature and structure of the recorded information.

————————

6-13

This page intentionally left blank.

6-14

Appendix 1 to Chapter 6 SAMPLE COMPANY POLICY ON NON-PUNITIVE HAZARD REPORTING XYZ Airline=s non-punitive reporting policy XYZ Airline is committed to the safest flight operating standards possible. To achieve this, it is imperative that we have uninhibited reporting of all incidents and occurrences which may compromise the safe conduct of our operations. To this end, every employee is responsible for communicating any information that may affect the integrity of flight safety. Such communication must be completely free of any form of reprisal. XYZ Airline will not take disciplinary action against any employee who discloses an incident or occurrence involving flight safety. This policy shall not apply to information received by the Company from a source other than the employee, or which involves an illegal act, or a deliberate or wilful disregard of promulgated regulations or procedures. The primary responsibility for flight safety rests with line managers, however, flight safety is everyone=s concern. Our method of collecting recording and disseminating information obtained from Air Safety Reports has been developed to protect, to the extent permissible by law, the identity of any employee who provides flight safety information. I urge all staff to use our flight safety programme to help XYZ Airline become a leader in providing our customers and employees with the highest level of flight safety

Signed_________________________________________ Chairman & CEO

————————

6-15

This page intentionally left blank.

6-16

Appendix 2 to Chapter 6 EXAMPLES OF ITEMS TO BE REPORTED IN AN AIRLINE OCCURRENCE REPORTING SYSTEM Following is a listing of the types of occurrences, or safety events to be reported under the company’s incident reporting system. The list is neither exhaustive nor in any order of importance. (Some items may be required to be reported under State laws or regulations.) • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Any system defect which adversely affects the handling or operation of the aircraft; Warning of smoke or fire, including the activation of toilet smoke detectors and galley fires; An emergency is declared; The aircraft is evacuated by means of the emergency exits/slides; Safety equipment or procedures are defective or inadequate; Serious deficiencies in operational documentation; Incorrect loading of fuel, cargo or dangerous goods; Significant deviation from normal operating procedures; A go-around is carried out from below 1000 ft above ground level; An engine is shut down or fails at any stage of the flight; Ground damage occurs; A take-off is rejected after take-off power is established; The aircraft leaves the runway or taxiway or other hardstanding; A navigation error involving a significant deviation from track; An altitude excursion of more than 500 feet occurs; Unstabilized approach under 500 feet; Exceeding the limiting parameters for the aircraft configuration; Communications fail or are impaired; A stall warning occurs; GPWS activation; A heavy landing check is required; Hazardous surface conditions, e.g. icy, slush, poor braking; Aircraft lands with reserve fuel or less remaining; A TCAS RA event; A serious ATC incident, e.g. near mid-air collision, runway incursion, incorrect clearance; Significant wake turbulence, turbulence, windshear or other severe weather; Crew or passengers become seriously ill, are injured, become incapacitated or deceased; Violent, armed or intoxicated passengers, or when restraint is necessary; Security procedures are breached; Bird strike or Foreign Object Damage (FOD); and Any other event considered likely to have an effect on safety or aircraft operations.

————————

6-17

This page intentionally left blank.

6-18

Appendix 3 to Chapter 6 LIMITATIONS ON THE USE OF DATA FROM VOLUNTARY INCIDENT REPORTING SYSTEMS37 38 Care needs to be taken when using data from voluntary submitted incident reports. In drawing conclusions based on such data, analysts should be aware of the following limitations. Information not validated. In some States, voluntary, confidential reports can be fully investigated and information from other sources brought to bear on the incident. However, the confidentiality provisions of smaller programmes (such as company reporting systems) make it difficult to adequately follow-up on the report without compromising the identity of the reporter. Thus, much of the reported information cannot be substantiated. Reporters may have a tendency to understate their errors and blame the occurrence on other parties. Incidents may also be embellished to benefit the reporters. For example, during contract negotiations the number of incidents reported may be artificially inflated in order to support one side’s bargaining position. Reporter biases. Two factors may bias voluntary incident data: who reports and what gets reported. For example, pilots expect air traffic controllers to prevent traffic conflicts at tower-equipped airports. When a conflict occurs, pilots may view it as a problem with the air traffic system and therefore feel it is useful to report the incident. At non-tower airports, pilots are responsible for seeing and avoiding the other aircraft. If they fail to do so, the same pilots may feel culpable and perceive little benefit in reporting the incident; particularly, if no one else is in a position or likely to follow-up on the error. Some of the factors contributing to the subjective nature of voluntary incident reports include: a) Reporters must be familiar with the programme, they must have access to reporting forms or phone numbers; b) Reporters’ motivation to report may vary due to the following factors: 1) Level of commitment to safety; 2) Awareness of the reporting programme; 3) Perception of the associated risks (local vs. systemic implications); 4) Operational conditions (some types of incident receive more attention than others);

37 Development of a Methodology for Operational Reporting and Analysis Systems (OIRAS), Appel d=offres DGAC No 96/01 by Jean Paries and Ashleigh Merritt 38

Adapted in part from a paper by Dr. Sheryl Chappell of NASA ASRS entitled “Using Voluntary Incident Reports for Human Factors Evaluations” Aviation Psychology in Practice, Avebury Press, 1994

6-19

5) Denial, ignorance of safety implications, desire to hide the problem, or fear of recrimination or even disciplinary action, (despite guarantees to the contrary); c) Different occupational groups see things differently, both in terms of interpreting the same event and in terms of deciding what is important; and d) Reporters must be aware of an incident to submit a report. Errors that go undetected are not reported. Report forms. Typically, incident reporting forms induce bias (including bias against reporting at all): a) A report form must be sufficiently short and easy to use that operators are encouraged to use it, thus the number of questions must be necessarily limited; b) Completely open questions (i.e. narratives only) can fail to elicit useful data; c) Questions can guide the reporter, but they can also distort perceptions by leading the reporter to biased conclusions; and d) The range of possible events is so broad that a standard structured form cannot capture all information. (Therefore, analysts may have to contact the reporter to gain specific information.) Incident reporting databases. Information must be categorized in accordance with a pre-determined structure of keywords or definitions for entry into the database for later retrieval. Typically this introduces bias into the databases, compromising their utility. For example: a)

Unlike objective physical flight parameters, descriptions of events and any causal attributions are more subjective;

b)

Categorization requires a system of pre-determined keywords or definitions biasing the database: 1)

Reports are analysed to Afit” the keywords. Details that do not fit are ignored;

2)

Impossibility of creating an exhaustive list of keywords for classifying information;

3)

Keywords are either present or not present providing a poor approximation of the real world;

4)

Information is retrieved according to how it is stored; hence categorization determines the output parameters. For example, if there is no keyword called >technical failure= then >technical failure= will never be found to be the cause of incidents from that database;

5)

The categorization system creates a ‘self-fulfilling prophecy’. For example, many incident reporting systems bias the keyword categorization toward CRM. Consequently, CRM is often cited as both the cause of the problem and its cure (more CRM training will redress the perceived CRM deficiency);

6-20

c) Much of the information in the databases is never retrieved once it is entered; and d) Given the generality of keywords, the analyst must frequently go back to the original report to understand contextual details. Relative frequency of occurrence. Since voluntary incident reporting systems do not receive information of the type needed to compute useful rate figures, any attempt to put the incident in the perspective of a frequency of occurrence vis à vis other occurrences will be an educated guess at best. For valid frequency comparisons, three types of data are required: the number of persons actually experiencing similar incidents (not just the reported incidents), the size of the population at risk of similar occurrences, and a measurement of the time period under consideration. Trend analysis. Meaningful trend analysis of the more subjective parameters recorded in incident reporting databases have not been particularly successful. Following are some of the reasons: a) Difficulties in using structured information; b) Limitations in capturing the context for the incident through keywords; c) Inadequate levels of detail and accuracy of recorded data; d) Poor inter-reliability of one report against another; e) Difficulties in merging data from different databases; and f) Difficulties in formulating meaningful queries for the database.

____________________

6-21

Chapter 7 FLIGHT DATA ANALYSIS PROGRAMMES

Introduction • What is an FDA programme • Benefits of FDA programmes ICAO Requirement Using an FDA Programme • Exceedence detection • Routine measurements • Incident investigation • Continuing airworthiness • Integrated safety analysis FDA Equipment • Airborne equipment • Ground replay and analysis equipment FDA in Practice • FDA process • Analysis and follow-up Conditions for Effective FDA Programmes • Protection of FDA data • Essential trust • Requisite safety culture Implementing an FDA Programme • Aims and objectives of an FDA programme • The FDA team • Off-the-shelf packages Appendices 1. Sample FDA Statement of Agreement

7-1

This page intentionally left blank.

7-2

Chapter 7 FLIGHT DATA ANALYSIS PROGRAMMES INTRODUCTION39 Flight Data Analysis (FDA) programmes, sometimes referred to as Flight Data Monitoring (FDM), or Flight Operations Quality Assurance (FOQA), provide another tool for the proactive identification of hazards. They are a logical complement to the incident reporting systems (discussed in the previous Chapter) and to LOSA programmes (discussed in the next Chapter).

What is an FDA programme Initially, the principal use of flight recorders was to aid accident investigators, especially in those accidents with no surviving crewmembers. However, early on, it was recognized that analysis of this recorded data was also useful for better understanding serious incidents. More progressive organizations saw further potential for analysing flight recorder data in aggregate. In the 1970s, several airlines realized that the flight parameters recorded in the mandatory Flight Data Recorders (FDR) provided valuable insights for safe flight operations. By routinely accessing these recorded flight parameters, using a secondary quick access recorder (QAR), much could be learnt about the safety of flight operations and the performance of airframes and engines. Valuable data about the things that go right in day-to-day operations was available, putting accident and incident data into perspective. As well, analysis of this de-identified data could assist in the identification of safety hazards before a serious incident or accident occurred. To capitalize on these benefits, such airlines set up systems and processes to retrieve and analyse flight data recorded across their fleets. Despite some controversy, the aviation industry is slowly adopting the practice of routinely analysing recorded data from routine operations in support of their accident prevention programmes. Now, rather than merely reacting to serious events, management has the capability to proactively identify safety hazards and mitigate risks. Any effective FDA programme requires the cooperation of the pilot group. It is essential that agreement is reached on the processes to be followed, in particular the non-punitive aspects of such a programme. Such details are normally contained in a formal agreement between management and their flight crew. An example of one such agreement is shown at the end of this chapter. For the purposes of this Manual, a Flight Data Analysis (FDA) Programme may be defined as: A proactive and non-punitive programme for gathering and analysing data recorded during routine flights to improve flight crew performance, operating procedures, flight training, air traffic control procedures, air navigation services, or aircraft maintenance and design.

39

Adapted from OFSH 3.8

7-3

Benefits of FDA programmes Today, Flight Data Analysis (FDA) programmes are increasingly used for the monitoring and analysis of flight operations and engineering performance. FDA programmes are a logical component of a mature safety management system, particularly for larger airlines. Successful programmes encourage adherence to Standard Operating Procedures, deter non-standard behaviour and so enhance flight safety. They can detect adverse trends in any part of the flight regime and so facilitate the investigation of events other than those which have had serious consequences. Flight data analysis is used to detect flight parameter exceedances and to identify non-standard or deficient procedures, weaknesses in the ATC system, and anomalies in aircraft performance. FDA allows the monitoring of various aspects of the flight profile, such as the adherence to the prescribed take-off, climb, cruise, descent, approach and landing SOPs. Specific aspects of flight operations can be examined, either retrospectively to identify problem areas, or proactively prior to introducing operational change and subsequently, to confirm its effectiveness. While using flight recorder data during incident analysis is to be recommended, such recorded data provides the ability to compare a specific flight with the fleet profile thereby providing the ability to analyse the systemic aspects of an incident. It may be that the parameters of the incident vary only slightly from many other flights, possible indicating a requirement for change in operating technique or training. For example, it would be possible to determine whether a tail-scrape on landing was an isolated event, or symptomatic of a wider mishandling problem, such as over-flaring on touchdown or improper thrust management procedures. Engine monitoring programmes require the automated analysis of flight recorder data for reliable trend analysis. The value of manually coded engine data is limited in terms of accuracy, timeliness and reliability. Using flight recorder data, accurate analysis is possible within a short time, thereby increasing the potential for preventive action. It is also possible to monitor other aspects of the airframe and systems.

ICAO REQUIREMENT Recognizing the potential for accident prevention, ICAO has introduced provisions for a flight data analysis programme to be part of an operator’s accident prevention and flight safety programme. Operators of larger aircraft authorized to conduct international commercial air transport operations will be accountable for the operation of a non-punitive FDA programme, which contains adequate safeguards to protect the source(s) of the data. They may obtain the services of a specialist contractor to operate the programme.

From 1 January 2005, an operator of an aeroplane of a maximum certificated take-off mass in excess of 27,000 kg shall establish and maintain a flight data analysis programme as part of its accident prevention and flight safety programme. ICAO Annex 6 Part 1 Chapter 3

7-4

USING AN FDA PROGRAMME As already seen, FDA programmes offer a wide spectrum of potential applications for accident prevention, as well as improvements in operational efficiency and economy. Data aggregated from many flights may be useful to: a) Determine operating norms for day-to-day performance; b) Identify unsafe trends; c) Facilitate certification of equipment and SOPs; d) Identify operational hazards in specific operating procedures, fleets, domiciles, airports, ATC procedures, etc.; e) Monitor the effectiveness of specific safety actions taken; f) Support quality assurance programmes and safety audits; g) Reduce operating and maintenance costs; h) Optimize training procedures; and i)

Provide a performance measurement tool for risk management programmes.

Typically, FDA data today are being used in five areas: a) Exceedence detection; b) Routine measurements; c) Incident investigations; d) Continuing airworthiness; and e) Linked databases (or integrated safety analysis).

Exceedence detection Initially, FDA programmes may be used for detecting exceedences or safety events, such as deviations from flight manual limits, standard operating procedures, or good airmanship. A set of core events (usually provided by the FDA software vendor in consultation with the operator/manufacturer) establishes the main areas of interest to operators. Example: High lift-off rotation rate, stall warning, GPWS warning, flap limit speed exceedence, fast approach, high/low on glide slope, heavy landing. FDA provides useful information from safety events which can complement that provided in crew reports.

7-5

Example: Reduced flap landing, emergency descent, engine failure, rejected take-off, go-around, TCAS or GPWS warning, system malfunctions, etc. Companies may also modify the set of core events (in accordance with the agreement with their pilots) to account for unique situations they regularly experience or the SOPs they use. Example: To avoid nuisance reports from a non-standard SID. They may also define new events (with the agreement of the pilots) to address specific problem areas. Example: Restrictions on the use of certain flap settings to increase component life. Care must be taken that in order to avoid an exceedence, crew do not attempt to fly the FDA profile rather than follow SOPs. Such an action can quickly turn a poor situation into something worse.

Routine measurements Increasingly, data is retained from all flights, not just the ones producing significant events. A selection of measures is retained that are sufficient to characterize each flight and allow comparative analysis of a wide range of operational variabilities. Trends may be identified before there are statistically significant numbers of events. Emerging trends and tendencies are monitored before the trigger levels associated with exceedences are reached. Examples of parameters monitored: take-off weight; flap setting; temperature; rotation and lift-off speeds vs. scheduled speeds; maximum pitch rate and attitude during rotation; gear retraction speeds, heights and times. Examples of comparative analyses: pitch rates from high vs. low take-off weights; good vs. bad weather approaches; and touchdowns on short vs. long runways.

Incident investigation Recorded data provide valuable information for follow-up to mandatory reportable incidents and other technical reports. Quantifiable recorded data have been useful in adding to the impressions and information recalled by the flight crew. The recorded data also provide an accurate indication of system status and performance, which may help in determining cause and effect relationships. Examples of incidents where recorded data could be useful: a) Emergencies, such as: 1) High speed rejected take-offs; 2) Flight control problems; 3) System failures, etc.;

7-6

b) High cockpit workload conditions as corroborated by such indicators as: 1) Late descent; 2) Late localizer and/or glide slope interception; 3) Large heading change below a specific height; 4) Late landing configuration; c) Unstabilized and rushed approaches, glide path excursions, etc.; d) Exceedences of prescribed operating limitations (such as flap limit speeds, engine overtemperatures, Vspeeds, stall onset conditions, etc.; and e) Wake vortex encounters, low-level wind shear, turbulence encounters or other vertical accelerations, etc.

Continuing airworthiness Both routine and event data can be utilized to assist the continuing airworthiness function. Traditionally, engine-monitoring programmes have looked for measures of engine performance to determine operating efficiency and predict impending failures. The engine manufacturer normally supplies these programmes. Examples of continuing airworthiness uses: engine thrust level and airframe drag measurements; avionics and other system performance monitoring; flying control performance; brake and landing gear usage.

Integrated safety analysis All the data gathered in an FDA programme should be kept in a central safety database. By linking the FDA database to other safety databases (such as incident reporting systems and technical fault reporting systems), a more complete understanding of events becomes possible through cross-referencing the various sources of information. Care must be taken however, to safeguard the confidentiality of FDA data when linking it to identified data. Example of integration: A heavy landing results in a crew report, an FDA event and an engineering report. The crew report provides the context, the FDA event the quantitative description and the engineering report the result. The integration of all available sources of safety data provides the company safety management system with viable information on the overall safety health of the operation.

FDA EQUIPMENT FDA programmes generally involve systems that capture flight data, transform the data into an appropriate format for analysis, and generate reports and visualization to assist in assessing the data. The

7-7

level of sophistication of the equipment can vary widely. Typically, however, the following equipment capabilities are required for effective FDA programmes: a) An on-board device to capture and record data on a wide range of in-flight parameters (such as altitude, airspeed, heading, aircraft attitude, aircraft configuration, etc.); b) A means to transfer the data recorded on-board the aircraft to a ground-based processing station. In the past, this largely involved the physical movement of the memory unit from the QAR (either tape, optical disc, or solid state). To reduce the physical effort required, later transfer methods utilize wireless technologies; c) A ground-based computer system (using specialized software) to analyse the data (from single flights and/or in an aggregated format), identify deviations from expected performance, generate reports to assist in interpreting the read-outs, etc.; and d) Optional software for a flight animation capability to integrate all data, presenting it as a simulation of inflight conditions, thereby facilitating visualization of actual events.

Airborne equipment Modern glass-cockpit and fly-by-wire aircraft are equipped with the necessary digital data buses from which information can be captured by a recording device for subsequent analysis. Older aircraft may be retrofitted to record additional parameters. However, for older (non-digital) aircraft, it is unlikely to be practical to record sufficient parameters to support a viable FDA programme. The number of parameters recorded by the mandatory FDR may determine the scope of an FDA programme. Unfortunately, in some cases the number of parameters and recording capacity required by law to be recorded to support accident investigations may be insufficient to support an effective FDA programme. Thus many operators are opting for additional recording capacity, capable of being easily downloaded for analysis. Quick access recorders (QAR) are installed in the aircraft and record flight data onto a low-cost removable medium such as tape cartridge, optical disk, or solid-state recording medium. The recording can be removed from the aircraft after a series of flights. New technology QARs are capable of supporting more than 2,000 parameters at much higher sampling rates than the FDR. The expanded data frame greatly increases the resolution and accuracy of the output from ground analysis programmes. To eliminate the task of moving the data from the aircraft to the ground station by physically removing the recording medium of the QAR, newer systems automatically download the recorded information via secure wireless systems when the aircraft is in the vicinity of the gate. In still other systems, the recorded data is analysed on-board while the aircraft is airborne. The encrypted data is then transmitted to a ground station using satellite communications.

Ground replay and analysis equipment Data is downloaded from the recording device into a central replay and analysis department, where the data is held securely to protect this sensitive information. A variety of computer platforms, including networked PCs, are capable of hosting the software needed to replay the recorded data. Replay software is

7-8

commercially available, however, the computer platform will require front-end interfaces (usually provided by the recorder manufacturers) to cope with the variety of QAR, FDR and other inputs available today. FDA programmes generate large amounts of data requiring specialized analytical tools. These tools, which are commercially available, facilitate the routine analysis of flight data in order to reveal situations that require corrective action. The analysis software checks the downloaded flight data for abnormalities. The exceedence detection software typically includes a large number of trigger logic expressions derived from a variety of sources, such as, flight performance curves; standard operating procedures; engine manufacturers= performance data; airfield layout and approach criteria. Trigger logic expressions may be simple exceedences, such as redline values. However, the majority are composites which define a certain flight mode, aircraft configuration or payload-related condition. Analysis software can also assign different sets of rules dependent on airport or geography. For example, noise sensitive airports may use higher than normal glide slopes on approach paths over populated areas. Events and measurements can be displayed on a ground computer screen in a variety of formats. Recorded flight data is usually shown in the form of colour-coded traces and associated engineering units, cockpit simulations or animations of the external view of the aircraft.

FDA IN PRACTICE FDA process40 Typically, operators follow a closed-loop process in applying an FDA programme, for example: Baseline established. Initially, operators establish a baseline of operational parameters against which changes can be detected and measured. Example: Rate of unstable approaches, or hard landings. Unusual or unsafe circumstances highlighted. The user determines when non-standard, unusual or basically unsafe circumstances occur; by comparing them to the baseline margins of safety, the changes can be quantified. Example: Increases in unstable approaches (or other unsafe events) at particular locations. Unsafe trends identified. Based on the frequency of occurrence, trends are identified. Combined with an estimation of the level of severity, the risks are assessed to determine which may become unacceptable if the trend continues. Example: A new procedure has resulted in high rates of descent that are nearly triggering GPWS warnings. Risks mitigated. Once an unacceptable risk has been identified, appropriate risk mitigation actions are decided and implemented.

40

Adapted from CAP 739

7-9

Example: Having found high rates of descent, the Standard Operating Procedures (SOPs) are changed to improve aircraft control for optimum/maximum rates of descent. Effectiveness monitored. Once a remedial action has been put in place its effectiveness is monitored, confirming that it has reduced the identified risk and that the risk has not been transferred elsewhere.

Example: Confirm that other safety measures at the airfield with high rates of descent do not change for the worse after changes in approach procedures. Analysis and follow-up FDA data are usually compiled on a monthly basis. The data should then be reviewed by a working group — looking for specific exceedences and for emerging undesirable trends and for dissemination of information to flight crews. If deficiencies in pilot handling technique are evident, the information is de-identified in order to protect the identity of the flight crew. The information on specific exceedences is passed to an agreed aircrew representative for confidential discussion with the pilot. The aircrew representative provides the necessary contact with the pilot in order to clarify the circumstances, obtain feedback, and give advice and recommendations for appropriate action, such as: re-training for the pilot (carried out in a positive and non-punitive way); revisions to operating and flight manuals; changes to ATC and airport operating procedures; etc. As well as reviewing specific exceedences, all events are archived in a database. The database is used to sort, validate and display the data in easy-to-understand management reports. Over time, this archived data can provide a picture of emerging trends and hazards which would otherwise go unnoticed. Where the development of an undesirable trend becomes evident (within a fleet, or at a particular phase of flight, or airport location), the fleet’s training department can implement measures to reverse the trend through modification of training exercises and/or operating procedures. Likewise with other areas of the operation requiring action, the data can then be used to confirm the effectiveness of any action taken. Lessons learned from the FDA programme may warrant inclusion in the company’s safety promotion programmes. However, care is required to ensure that any information acquired through FDA is studiously de-identified before using it in any training or promotional initiative. As in any closed-loop process, follow-up monitoring is required to assess the effectiveness of any corrective actions taken. Flight crew feedback is essential for the identification and resolution of safety problems. For example: a) Are the desired results being achieved soon enough; b) Have the problems really been corrected, or just relocated to another part of the system; and c) Have new problems been introduced. All successes and failures should be recorded, comparing planned programme objectives with expected results. This provides a basis for review of the FDA programme and the foundation for future programme development.

7-10

CONDITIONS FOR EFFECTIVE FDA PROGRAMMES Following are several conditions that are fundamental to successful FDA programmes.

Protection of FDA data Airline management and pilots both have legitimate concerns regarding the protection of FDA data, for example: a) Use of data for disciplinary purposes; b) Use of data for enforcement actions against individuals or against the company, except in cases of criminal intent or intentional disregard of safety; c) Disclosure to the media and the general public under the provisions of State laws for access to information; and d) Disclosure during civil litigation. The integrity of FDA programmes rests upon protection of the FDA data. Any disclosure for purposes other than accident prevention can compromise the voluntary provision of FDA data, thereby compromising flight safety. Thus, the prevention of misuse of FDA data is a common interest of the State, the airlines and the pilots.

Essential trust As with any successful incident reporting system, the trust established between management and its pilots is the foundation for a successful FDA programme. This trust can be built on: a) Early participation of the pilots’ association in the design, implementation and operation of the FDA programme; b) A formal agreement between management and the pilots identifying the procedures for the use and protection of data. (Appendix 1 to this Chapter provides a sample agreement between an airline and its aircrew); and c) Data security, optimized by: 1) Adhering to stringent agreements with the pilots’ associations; 2) Strictly limiting data access to selected individuals within the company; 3) Maintaining tight control to ensure that identifying data are removed from the flight data records as soon as possible; 4) Ensuring that operational problems are promptly addressed by management; and 5) Destruction of all identified data as soon as possible.

7-11

Access to crew identification information during follow-up should only be available to specifically authorized persons and only used for the purpose of an investigation. Subsequent to the analysis, the data enabling this identification should be destroyed.

Requisite safety culture Consistent and competent programme management characterize successful FDA programmes. Examples of an effective safety culture include: a) Top management’s demonstrated commitment to promoting a pro-active safety culture, championing the cooperation and accountability of all organizational levels and relevant aviation associations (pilots, cabin staff, engineers, dispatchers, etc.); b) A non-punitive company policy. (The main objective of the FDA programme must be to identify hazards, not to identify individuals who may have committed an unsafe act.); c) FDA programme management by a dedicated staff within the safety or operations departments with a high degree of specialization and logistical support; d) Potential risks are identified through the correlation of the results of the analysis by persons with appropriate expertise. (For example, pilots experienced on the aircraft type being analysed are required for the accurate diagnosis of operational hazards emerging from FDA analyses.); e) Primary focus on monitoring fleet trends aggregated from numerous operations rather than on specific events; the identification of systemic issues adds more value for accident prevention than (perhaps isolated) events; f) A well-structured, de-identification system to protect the confidentiality of the data; and g) An efficient communication system for disseminating hazard information (and subsequent risk assessments) to relevant departments and outside agencies to permit timely safety action.

IMPLEMENTING AN FDA PROGRAMME Typically, the following steps are required to implement an FDA programme: a) Implementation of pilot association agreements; b) Establishment and verification of operational and security procedures; c) Installation of equipment; d) Selection and training of dedicated and experienced staff to operate the programme; and e) Commencement of data analysis and validation. Bearing in mind the time required to get crew/management agreements and procedures developed, a startup airline with no FDA experience would not likely achieve an operational system in less than 12 months.

7-12

Another year may be required before any safety and cost benefits appear. Improvements in the analysis software, or the use of outside specialist service providers, may shorten these time frames. Integrating the FDA programme with other safety monitoring systems into a coherent safety management system will increase the potential benefits. Safety information gathered from other programmes of the SMS gives context to the FDA data. In turn, FDA can provide quantitative information to support investigations that otherwise would be based on less reliable subjective reports.

Aims and objectives of an FDA programme Define objectives of programme. As with any project there is a need to define the direction and objectives of the work. A phased approach is recommended so that the foundations are in place for possible subsequent expansion into other areas. Using a building block approach will allow expansion, diversification and evolution through experience. Example: With a modular system begin by looking at basic safety related issues only. Add engine health monitoring etc. in the second phase. Ensure compatibility with other systems. Set both short and long term goals. A staged set of objectives starting from the first week’s replay and moving through early production reports into regular routine analysis will contribute to a sense of achievement as milestones are met. Example: Short term a) Establish data download procedures, test replay software and identify aircraft defects; b) Validate and investigate exceedence data; and c) Establish a user-acceptable routine report format to highlight individual exceedences and facilitate the acquisition of relevant statistics. Medium term a) Produce annual report - include key performance indicators; b) Add other modules to analysis (e.g. Continuing Airworthiness); and c) Plan for next fleet to be added to programme. Long Term a) Network FDA information across all company safety information systems; b) Ensure FDA provision for any proposed advanced training programme; and c) Use utilization and condition monitoring to reduce spares holdings.

7-13

Initially, focusing on a few known areas of interest will help prove the system’s effectiveness. In contrast to an undisciplined “scatter-gun” approach, a focused approach is more likely to get early successes. Example: Rushed approaches, or rough runways at particular airports; unusual fuel usage on particular flight segments; etc. Analysis of such known problem areas may generate useful information for the analysis of other areas.

The FDA team Experience has shown that the “team” required to run an FDA programme could vary in size from one person with a small fleet (5 aircraft), to a dedicated section for large fleets. The descriptions below identify various functions to be fulfilled, not all of which need a dedicated position. For example, engineering may provide only part time support. All FDA team members require appropriate training or experience for their respective area of data analysis. Each team member must be allocated a realistic amount of time to regularly spend on FDA tasks. With insufficient available manpower, the entire programme will under-perform or even fail. Team leader. Team leaders must earn the trust and full support of both management and flight crews. They act independently of other line management to make recommendations that will be seen by all to have a high level of integrity and impartiality. The individual requires good analytical, presentation and management skills. Flight operations interpreter. This person normally is a current pilot (or perhaps a recently retired senior Captain or trainer), who knows the company’s route network and aircraft. Their in-depth knowledge of SOPs, aircraft handling characteristics, airfields and routes will be used to place the FDA data in a credible context. Technical interpreter. This person interprets FDA data with respect to the technical aspects of the aircraft operation. He is familiar with the powerplant, structures and systems departments’ requirements for information and any other engineering monitoring programmes in use by the airline. Aircrew representative. This person provides the link between the fleet or training managers and flight crew involved in circumstances highlighted by FDA. The position requires good people skills and a positive attitude towards safety education. The person is normally a representative of the flight crew association and should be the only person permitted to connect the identifying data with the event. The aircrew representative requires the trust of both crewmembers and managers for their integrity and good judgment. Engineering technical support. This person is normally an avionics specialist, involved in the supervision of mandatory serviceability requirements for FDR systems. They must be knowledgeable about FDA and the associated systems needed to run the programme. Air safety coordinator. This person cross-references FDA information with other air safety monitoring programmes (such as the company’s mandatory or confidential incident reporting programmes), creating a credible integrated context for all information. This function can reduce duplication of follow-up investigations.

7-14

Replay operative and administrator. This person is responsible for the day-to-day running of the system, producing reports and analysis. Methodical, with some knowledge of the general operating environment, this person keeps the programme moving.

Off-the-shelf packages The QARs available on most large, modern aircraft can be analysed on a suitably configured replay and analysis system. Even though the operators themselves can configure the various event equations and exceedence levels, suppliers of ground replay software offer both starter packs and advanced flight operations monitoring programmes for a variety of different aircraft types. It is not normally costeffective for new operators to configure FDA systems themselves, although most suppliers will review the relevance and levels of event triggers with each new operator.41 Some aircraft manufacturers actively support FDA programmes for their aircraft.42 They provide airlines with packages including tools and software, handbooks to support their flight data analysis methods and procedures, and additional assistance for operators implementing their programme. (They see the sharing of data and information provided by the airline as a means for improving their aircraft, SOPs and training.) Most system vendors provide one year of maintenance and support in the original package but charge an annual fee thereafter. In addition, other cost factors to be considered by prospective purchasers include: a) Installation costs; b) Training costs; c) Software upgrade costs (often included in the maintenance contracts); and d) Other software licence fees that may be necessary. FDA programmes are often viewed as one of the most expensive safety systems in terms of the initial outlay, software agreements and personnel requirements. In reality, they have the potential to save the company considerable money by reducing the risk of a major accident, improving operating standards, identifying external factors affecting the operation and improving engineering monitoring programmes.

————————

41 42

The OFSH includes a list of suppliers of flight/performance monitoring systems in Appendix B, pg B-12 See Airbus’ Flight Operations Monitoring System

7-15

This page intentionally left blank.

7-16

Appendix 1 to Chapter 7 SAMPLE MEMORANDUM OF UNDERSTANDING FOR THE OPERATION OF A FLIGHT DATA ANALYSIS (FDA) PROGRAMME BETWEEN AN AIRLINE AND A PILOTS ASSOCIATION

1

Background

1.1 The flight data analysis programme, FDA PROGRAMME, forms part of THE AIRLINE’s Safety Management System. Recorded flight data can contain information that has the potential to improve flight safety, but also has the potential, if used inappropriately, to be detrimental to individual crewmembers or to the airline as a whole. This document describes protocols that will enable the greatest safety benefit to be obtained from the data whilst satisfying the company’s need to be seen to be managing safety, and simultaneously ensuring fair treatment of employees. 1.2 The FDA PROGRAMME conforms with the intent of THE AIRLINE’s Standing Instruction number X (SIN X), Reporting of Safety Incidents, in that “The purpose of an investigation of any accident or incident is to establish the facts and cause, and therefore prevent further occurrence. The purpose is not to apportion blame or liability.” 1.3 It also conforms with the intent of ICAO Annex 6 (Part 1, Chapter 3) “A flight data analysis programme shall be non-punitive and contain safeguards to protect the source(s) of the data”. 2

General intentions

2.1 It has long been accepted by both THE AIRLINE and THE PILOTS ASSOCIATION that the greatest benefit will be derived from the FDA PROGRAMME by working in a spirit of mutual cooperation towards improving flight safety. A rigid set of rules can, on occasions, be obstructive, limiting or counter-productive, and it is preferred that those involved in the FDA PROGRAMME should be free to explore new avenues by mutual consent, always bearing in mind that the FDA PROGRAMME is a safety programme, not a disciplinary one. The absence of rigid rules means that the continued success of the FDA PROGRAMME depends on mutual trust – indeed this has always been a key feature of the programme. 2.2 The primary purpose of monitoring operational flight data by the FDA PROGRAMME is to enhance flight safety. Therefore the intention of any remedial action following discovery, through the FDA PROGRAMME, of a concern, is to learn as much as possible in order: a) To prevent a recurrence; and b) To add to our general operational knowledge. 2.3 A general intention is that concerns raised by the FDA PROGRAMME should, where possible, be resolved without identifying the crew concerned. However there may be occasions when anonymity is not appropriate, and this document gives protocols to be followed on such occasions in order to be in accordance with SIN X.

7-17

2.4 It is recognized that THE AIRLINE requires an audit trail of actions taken following FDA PROGRAMME investigations. It is intended that this audit trail will be held within THE AIRLINE in a manner that satisfies THE AIRLINE’s requirements without being placed on a crewmember’s file. 2.5 A further intention is to provide recorded flight data to outside parties (CAA, FAA, Universities, manufacturers, etc) for research into flight safety. THE PILOTS ASSOCIATION will be informed of each such provision and, if the data is only useful if identified (i.e. can be linked to a specific flight) then THE AIRLINE will agree with THE PILOTS ASSOCIATION the confidentiality terms under which the data is provided. 3

Constitution

3.1 The constitution and responsibilities of the Flight Data Recording Group (the "FDA PROGRAMME Group") are defined in FCO Y. The Group meets once a month. Membership consists of: The Chairman (Flight Manager FDA PROGRAMME) A representative from each Fleet's training section A representative from Flight Data Recording (Engineering) A representative from Flight Technical Support A Flight Data Analyst from Flight Operations Representatives from THE PILOTS ASSOCIATION (currently two short-haul representatives and one long-haul representative) 3.2 The constitution and responsibilities of the Operational Flight Data Recording Working Group are defined in FCO Y. The Group meets bimonthly. Membership consists of: The Chairman (Flight Manager FDA PROGRAMME) A Flight Data Analyst from Flight Operations Manager Flight Data Recording (Engineering) A representative from Flight Technical Support A representative from Safety Services A representative from the CAA Safety Group A representative from THE PILOTS ASSOCIATION 4

Handling

4.1

Scope

This section applies to “events” discovered by the routine running of the FDA PROGRAMME. If a pilot files an Air Safety Report or reports an event to his Manager, then the responsibility for investigation lies with the Fleet, although the FDA PROGRAMME group may provide assistance. In this case the pilot is, of course, identified. 4.2 The list below gives some of the possible follow-up actions that may be used to investigate a concern raised by the FDA PROGRAMME. It is not intended to be exhaustive and does not preclude any other action agreed between THE AIRLINE and THE PILOTS ASSOCIATION which is in accordance with the general intentions above. Which action is most appropriate in given circumstances will be discussed and agreed between THE AIRLINE, represented by Flight Manager FDA PROGRAMME and the Fleet FDA

7-18

PROGRAMME representative, and THE PILOTS ASSOCIATION, represented by the relevant PILOTS ASSOCIATION representative. A Fleet Manager may request follow-up action. He will make his request to his Fleet FDA PROGRAMME representative who will consult with Flight Manager FDA PROGRAMME and the relevant PILOTS ASSOCIATION Representative, as above. 4.2.1 THE PILOTS ASSOCIATION may be asked to telephone the crewmembers to debrief an “event”. The nature of the call can be praise for a well-handled situation, enquiry to elicit more information about the event and its causes, or a reminder of a relevant Standard Operating procedure. The Fleet management may ask for specific questions or points to be put to the pilots during such a call or calls. In this case the pilots remain unidentified, and a record of the debriefing will be held in accordance with section 5 of this agreement. 4.2.2 THE PILOTS ASSOCIATION may be asked to contact a pilot who has a higher than average FDA PROGRAMME event rate, to advise the pilot and to seek any underlying reason. Again, Fleet management may ask for specific questions or points to be put to the pilots during such a call or calls. In this case too, the pilots remain unidentified, and a record of the debriefing will be held in accordance with section 5 of this agreement. 4.2.3 The enquiries of paragraphs 4.2.1 and 4.2.2 above may indicate that “closure” may not be possible without further action being taken. The following are examples of possible further action: —

The filing of an ASR — see paragraph 4.2.4 below;

—

A request for the pilot to speak directly to Fleet management – see paragraph 4.2.5 below; and

—

A requirement for the pilot to undertake some training to regain the required standard in a particular area — see paragraph 4.2.6 below.

4.2.4 If the “event” clearly warrants an ASR, but none has been filed, then THE PILOTS ASSOCIATION may be asked to request that the pilot(s) files one. An ASR filed under these circumstances will be treated as if it was filed at the time of the event. 4.2.5 THE PILOTS ASSOCIATION may be asked to invite a pilot to be debriefed by his Fleet management. If the pilot agrees to this, then he will be deemed to have reported the event unprompted so that paragraph 10.1 of SIN X applies: “It is not normally the policy of THE AIRLINE to institute disciplinary proceedings in response to the reporting of any incident affecting air safety.” A record of any such debriefing will be sent to the pilot concerned and a copy held in THE AIRLINE in accordance with section 5 of this document.

7-19

If the pilot declines the above invitation, then THE PILOTS ASSOCIATION debriefing will be continued until closure can be achieved. A record of this debriefing will be kept in accordance with section 5 of this document. 4.2.6 A pilot may be required to undertake such extra training as may be deemed necessary after consultation with the Fleet concerned. THE AIRLINE will arrange the training, and THE PILOTS ASSOCIATION will liaise with the pilot. A record of any such training will be sent to the pilot concerned and a copy held in THE AIRLINE in accordance with section 5 of this document. 4.3 If an event or sequence of events is considered serious enough to have hazarded the aircraft or its occupants, then THE PILOTS ASSOCIATION will be asked to withdraw anonymity of the pilots. THE PILOTS ASSOCIATION recognizes that, in the interest of flight safety, it cannot condone unreasonable, negligent or dangerous pilot behaviour and will normally accede to such a request. Removal of anonymity will be effected by the senior PILOTS ASSOCIATION representative after consultation with THE PILOTS ASSOCIATION chairman. The pilot will be notified by the senior PILOTS ASSOCIATION representative that anonymity is being withdrawn, and advised that he or she may be accompanied at any subsequent interview by a PILOTS ASSOCIATION representative. If agreement cannot be reached between THE AIRLINE Flight Operations and THE PILOTS ASSOCIATION as to whether an event is sufficiently serious to warrant withdrawal of anonymity, then a final decision will be taken by a nominated person. This person will be either THE AIRLINE Head of Safety or another nominated senior AIRLINE Manager, and he/she will be confirmed in this role by THE PILOTS ASSOCIATION who will reaffirm this acceptability each year. 4.4

Wilful disregard of SOPs If a pilot is discovered, through the FDA PROGRAMME only, to have wilfully disregarded THE AIRLINE SOPs, then he will be treated as follows: If the breach of SOP did not endanger the aircraft or its occupants, then debriefing may be carried out by THE PILOTS ASSOCIATION representative, thus preserving anonymity; but the pilot will be sent a letter containing a clear warning that a second offence will result in withdrawal of anonymity. If the breach of SOP did endanger the aircraft or its occupants, then THE AIRLINE will request withdrawal of anonymity as in paragraph 4.3 above.

4.5 If a pilot fails to cooperate with THE PILOTS ASSOCIATION with regard to the provisions of this agreement, then THE AIRLINE will receive THE PILOTS ASSOCIATION approval to assume responsibility for contact with that pilot, and any subsequent action. Such a pilot will be reminded by THE PILOTS ASSOCIATION that SIN X cautions: “In the event of an employee failing to report a safety related incident that they have caused or discovered, they will be exposed to full disciplinary action.”

7-20

5

Closure

5.1 Most FDA PROGRAMME events are not serious enough to warrant follow-up action and so are automatically “closed”. Those events for which follow-up action is required are deemed “open”, and then need a positive closure when the action is complete. 5.2 A record will be kept in THE AIRLINE of all events for which action is required. For each such event, the actions taken will be recorded along with a date of closure. This record will be kept in the FDA PROGRAMME database against the event itself. No record will be kept on an individual pilot’s file. 5.3 A letter will be sent, by Fleet Management, to each pilot involved in follow-up action, unless that action consisted only of a telephone debriefing by THE PILOTS ASSOCIATION representative for a single event. Such a letter will record the original concern, the subsequent discussion and/or action, and the expectation for the future. The letter will not be addressed to the pilot by name, but will be handed to THE PILOTS ASSOCIATION for forwarding to the pilot concerned. 5.4

Contents of record in FDA PROGRAMME DATABASE (FPD): The following will be recorded in the FPD against the event: A record of any telephone debrief by THE PILOTS ASSOCIATION A record of any debrief by Fleet Management A copy of any letter sent to the pilot A record of any extra training given to the pilot Any other relevant document The record will not contain anything that could identify the pilot by name.

5.5

Visibility of record and pilot identity: Flight Operations Management’s access level to FPD will reveal only that action is “open" or “closed" for each event – the actual action record is not visible. Events are not identifiable to a particular flight or pilot. Flight manager FDA PROGRAMME level of access to FPD will reveal the actual actions taken, and can associate a pilot, by his 5-digit FDA PROGRAMME number, with that event. Actual pilot identity is not available. THE PILOTS ASSOCIATION representative’s access to FPD is the same as the flight manager FDA PROGRAMME, but in addition THE PILOTS ASSOCIATION representative has a decode disk to identify a pilot from his 5-digit FDA PROGRAMME number.

5.6 It is the responsibility of the flight manager FDA PROGRAMME to detect pilots with more than one action recorded against their 5-digit FDA PROGRAMME number within a reasonable time, and bring this to the attention of the Fleet.

7-21

6

Safety Data Request (SDR)

6.1 Flight data for the first 15 minutes and the last 15 minutes of every flight is stored in a database known as SDR. This data is available for viewing by a Flight Manager if, and only if: An ASR has been filed for that portion of that flight, or The Captain of the flight has given his specific permission for the data to be viewed. 6.2 In order to view data in SDR, the flight manager needs to indicate, in the SDR itself, the reason for looking at the data. The reason is recorded in each case, and THE PILOTS ASSOCIATION representatives are able to view these records. 7

Retention of data

7.1 For each FDA PROGRAMME event FPD stores the raw flight data which can be viewed as a trace or as an instrument animation. In addition, but not visible to Flight Operations management, FPD stores information which identifies the flight (by date and registration) and the pilot (by 5-digit FDA PROGRAMME number). This data and information is required to analyse the event and to monitor, anonymously over a period of time, individual pilots’ event rates. Furthermore, SDR stores some raw flight data from each flight, as described in section 6 above. 7.2 THE AIRLINE will not retain data any longer than is necessary, and will in any case delete all flight data, and all means of identifying flights and crew, within 2 years of the flight. 7.3 For flights more than 2 years old, the FDA PROGRAMME database (FPD) will continue to contain a record of the FDA PROGRAMME events, but with all flight and crew identification removed. 8

THE PILOTS ASSOCIATION representatives’ access to confidential information

8.1 In order to fulfil his/her FDA PROGRAMME obligations, THE PILOTS ASSOCIATION representative will need access to information which is confidential to THE AIRLINE, and may be subject to the Data Protection Act. Upon appointment, a representative will be required to sign a Confidentiality Agreement which specifies the terms under which information obtained from THE AIRLINE may be used. Breach of this agreement will lead to suspension from the FDA PROGRAMME group, and may be the subject of THE AIRLINE’s disciplinary procedures. 8.2

In order to contact the crew involved in a FDA PROGRAMME event (see section 4), THE PILOTS ASSOCIATION representative will need: — The identity of the flight (date, registration and flight number); — The ability to identify the crew of that flight, and how to contact them; and — An electronic copy of the flight data and a means of viewing it.

8.3 THE AIRLINE will provide each PILOTS ASSOCIATION representative with a laptop computer pre-loaded with software to meet the above requirements:

7-22

— The identity of the flight will be provided by e-mail from the FDA PROGRAMME Group; — The identity of the crew, and their contact details, will be determined by remote access to THE AIRLINE flight crew scheduling system; and — The flight data will be e-mailed by the FDA PROGRAMME group, and will be viewed using the pre-loaded software. 8.4 In order to identify a pilot from his/her 5-digit FDA PROGRAMME number (see paragraph 4.2.2), THE PILOTS ASSOCIATION representative will be provided with a decode disk, for use with FPD. 8.5 Upon finishing work with the FDA PROGRAMME group, THE PILOTS ASSOCIATION representative will return the laptop and disk to THE AIRLINE. No copy of THE AIRLINE-provided software may be retained.

Signed on behalf of THE AIRLINE:

Signed on behalf of THE PILOTS ASSOCIATION:

_________________________________

_________________________________

Name:___________________________

Name:____________________________

Date:____________________________

Date:_____________________________

____________________

7-23

Chapter 8 LINE OPERATIONS SAFETY AUDITS (LOSA)

Introduction • ICAO’S role Terminology • Threats • Errors • Threat and error management • Systemic countermeasures Defining Characteristics of LOSA Safety Change Process Implementing LOSA

8-1

This page intentionally left blank.

8-2

Chapter 8 LINE OPERATIONS SAFETY AUDITS (LOSA) INTRODUCTION Increasingly, the aviation industry is recognizing the need to anticipate the negative consequences of human error. Hazards can be identified, analysed and validated based on data collected through the monitoring of day-to-day operations. Line Operational Safety Audits (LOSA) is another method for monitoring normal flight operations for accident prevention purposes. Like Flight Data Analysis (FDA) programmes, LOSA facilitates hazard identification through the analysis of actual in-flight performances. Whereas FDA provides accurate data on exceedences from expected aircraft performance, LOSA provides information on human behaviour; it facilitates understanding the context for the behaviour that may have precipitated the exceedences. While FDA and LOSA are well suited for application in larger airline operations, they can both be used very effectively in smaller airlines and are increasingly being used by them. LOSA is a tool for the management of human errors in flight operations. It is used to identify the threats to aviation safety which lead to human errors, to minimize the risks that such threats may generate and to implement measures to manage these errors within the operational context. LOSA enables operators to assess their resistance to operational risks and front-line personnel errors. Using a data-driven approach, they can prioritize these risks and identify actions to prevent accidents. In short, LOSA is a risk management tool. By observing normal day-to-day flight operations, data about flight crew behaviour and situational factors in “normal” operations are collected. Thus, LOSA facilitates understanding both successful behaviour and failures. Hazards deriving from operational errors can be identified and effective countermeasures developed. LOSA uses experienced and specially trained observers to collect data about flight crew behaviour and situational factors on “normal” flights. During audited flights, observers record error-inducing circumstances and the crew’s responses to them. The audits are conducted under strict non-punitive conditions, without fear of disciplinary action for detected errors. Flight crews are not required to justify their actions. Data from LOSA also provide a picture of system operations that can guide strategies in regard to accident prevention, training and operations. Like FDA programmes, data collected through LOSA can provide a rich source of information for the pro-active identification of systemic safety hazards. A particular strength of LOSA is that it identifies examples of superior performance that can be reinforced and used as models for training. (Traditionally, the industry has collected information on failed performance and revised training programmes accordingly.) With LOSA, training interventions can be based on the most successful operational performance. For example, based on LOSA data, CRM training can be modified to reflect best practices for coping with particular types of unsafe conditions and for managing typical errors related to these conditions.

8-3

ICAO’s role ICAO endorses LOSA as a way to monitor normal flight operations. ICAO supports the industry’s initiatives with LOSA, serving as an enabling partner in the programme. ICAO’s role includes: a) Promoting the value of LOSA to the international civil aviation community; b) Facilitating research in order to collect necessary data; and c) Acting as a mediator in the culturally sensitive aspects of data collection. ICAO has published a manual, Line Operations Safety Audit (LOSA) Manual (Doc 9803), to provide guidance to operators regarding LOSA programmes.

TERMINOLOGY LOSA employs specific terminology concerning threats, errors, threat and error management and countermeasures:

Threats During normal flights, crews routinely face situations created outside the cockpit that they must manage. Such situations increase the operational complexity of their task and pose some level of safety risk. These external situations may be relatively minor (such as frequency congestion), through to major (such as an engine-fire warning). In LOSA, such situations are referred to as threats. Some threats can be anticipated (such as a high workload situation during approach) and the crew may brief in advance, for example, “In the event of a go-around….”. Other threats may be unexpected. Since they occur without warning, no advanced briefing is feasible, (for example, a TCAS advisory).

Errors Humans are generally quite effective in balancing the conflicting demands between “getting the job done” and “getting the job done safely”. However, errors are a normal part of all human behaviour — including the performance of flight crews. In an operational context, flight crew errors tend to reduce the margin of safety and increase the probability of accidents. Any action or inaction by the flight crew that leads to deviations from expected behaviour may be viewed as an error. Examples of crew errors might include non-compliance with regulations and SOPs, or unexpected deviation from company or ATC expectations. Errors may be minor (setting the wrong altitude, but correcting it quickly) or major (not completing an essential checklist item). LOSA employs five categories of crew errors. These include: a) Communication error: Miscommunication, misinterpretation, or failure to communicate pertinent information among the flight crew or between flight crew and an external agent (for example, ATC or ground operations personnel);

8-4

b) Proficiency error: Lack of knowledge or psychomotor (“stick and rudder”) skills; c) Operational decision error: Decision-making error that is not standardized by regulation or operator procedures and that unnecessarily compromises safety. (For example, a crew decision to fly through a known wind shear on approach instead of going-around.); d) Procedural error: Deviation in execution of regulatory and/or operator procedures. The intention is correct but the execution is flawed. This category also includes errors where a crew forgot to do something; and e) Intentional non-compliance error: Wilful deviation from regulations and/or operator procedures (i.e. violations). Threat and error management43 Since threats and errors are an integral part of daily flight operations, systematic understanding of them is required for safely dealing with them. LOSA offers an informed perspective on threats and errors from which suitable coping strategies can be developed. Specifically, quantifiable LOSA data are useful in answering such questions as: a) What type of threats do flight crews most frequently encounter? When and where do they occur, and what types are the most difficult to manage? b) What are the most frequently committed crew errors, and which ones are the most difficult to manage? c) What outcomes are associated with mismanaged errors? How many result in the aircraft being in an “undesired” state (such as fast/slow on final approach)? d) Are there significant differences between airports, fleets, routes or phases of flight vis à vis threats and errors?

Systemic countermeasures Accepting that error is inevitable, the most effective countermeasures go beyond trying to simply prevent errors. They need to highlight unsafe conditions early enough to permit flight crews to take corrective action before adverse consequences result from the error. In other words, they “trap” the error. The most effective countermeasures seek to improve the everyday work situation in which flight crews face the inevitable threats to safe performance, measures which give crews a “second chance” to recover from their errors. Such systemic countermeasures include changes in aircraft design, crew training, company operating procedures, management decisions, etc.

43

The University of Texas has developed a Threat and Error Management (UTTEM) Model.

8-5

DEFINING CHARACTERISTICS OF LOSA The following characteristics of LOSA ensure the integrity of the methodology and its data: a) Jump seat observations during normal flight operations: LOSA observations are limited to routine flights (as opposed to line checks, or other training flights). Check pilots add to an already high stress level, thus providing an unrealistic picture of performance. The best observers learn to be unobtrusive and non-threatening, recording minimum detail in the cockpit. b) Joint management / pilot sponsorship: In order for LOSA to succeed as a viable accident prevention programme, both management and pilots support the project. Joint sponsorship provides “checks and balances” for the project to ensure that any necessary change will be made as a result of LOSA data. A LOSA audit does not proceed without the endorsement of the pilots via a signed agreement with management. A LOSA steering committee with pilot and management representatives shares responsibility for the planning, scheduling, supporting observers and verifying the data. c) Voluntary crew participation: Maintaining the integrity of LOSA within the airline is extremely important for long-term success. One way to accomplish this goal is to collect all observations with voluntary crew participation. Before conducting LOSA observations, an observer obtains the flight crew’s permission. If an airline conducting LOSA has an unreasonably high number of refusals by pilots to be observed, this may indicate that there are critical “trust” issues to be dealt with first. d) Collection of only de-identified, confidential safety data: LOSA observers do not record names, flight numbers, dates or any other data that can identify a crew. This allows for a high level of protection against disciplinary action. Airlines should not squander an opportunity to gain insight into their operations by having pilots fearful that a LOSA observation could be used against them in disciplinary proceedings. In other words, LOSA must not only be seen to be non-punitive, it must be non-punitive. e) Targeted observations: All data is collected on a specifically designed LOSA Observation Form. (Examples of the forms are included in Doc 9803.) Typically, the following types of information are collected by the LOSA observer: 1) Flight and crew demographics such as city pairs, aircraft type, flight time, years of experience in that position and with that airline, and crew familiarity; 2) Written narratives describing what the crew did well and what they did poorly and how they managed threats or errors for each phase of the flight; 3) CRM performance ratings using validated behavioural markers; 4) Technical worksheet for the descent/approach/landing phases that highlight the type of approach flown, the landing runway and whether the crew met the parameters of a stabilized approach; 5) Threat management worksheet that details each threat and how it was handled; 6) Error management worksheet that lists each error observed, how each error was handled and the final outcome; and

8-6

7) Crew interview conducted during low workload periods of the flight, such as cruise, that asks pilots for their suggestions to improve safety, training, and flight operations. f) Trusted, trained and standardized observers: Observers are primarily pilots drawn from the line, training department, safety department, management, etc. Experienced LOSA observers from a non-affiliated airline may be more objective and serve to provide an anchor point for company observers, especially for companies initiating a new LOSA programme. Regardless of the source, it is critical that the observers are respected and trusted to ensure acceptance of LOSA by the line pilots. The observers must be trained in concepts of threat and error management and in the use of the LOSA rating forms. Standardized rating is vital to the validity of the programme. g) Trusted data collection site: In order to maintain confidentiality, airlines must have a trusted data collection site. No observations can be misplaced or improperly disseminated within the airline, without compromising LOSA integrity. Some airlines use a “third party” to provide a neutral party for objective analysis of results. h) Data verification round-tables: Data-driven programmes like LOSA require data quality management procedures and consistency checks. For LOSA, round table discussions with representatives of management and the pilots association scan raw data for inconsistencies. The database must be validated for consistency and accuracy before a statistical analysis can proceed. i)

Data-derived targets for enhancement: As the data are collected and analysed, patterns emerge. Certain errors occur frequently, certain airports or activities are problematic, certain SOPs are ignored or modified, and certain manoeuvres pose particular difficulties. These patterns become targets for enhancement. The airline then develops an action plan and implements appropriate change strategies based on the input of expertise available to the airline. Through subsequent LOSA audits, the effectiveness of the changes can be measured.

j)

Feedback of results to the line pilots: After a LOSA is completed; the airline’s management team and the pilots association have an obligation to communicate the findings to the line pilots. Pilots are interested not only in the results but also management’s plan for improvement.

SAFETY CHANGE PROCESS Like other tools for risk management, a closed-loop process is required to effect a safety change. Problems are identified and analysed, strategies developed, priorities established, remedial measures implemented, and effectiveness monitored to identify any residual problems. LOSA directs organizational attention to the most important safety issues in daily operations. However, LOSA does not provide the solutions; they must come from organizational strategies. The organization must evaluate the data obtained through LOSA, identify those hazards posing the greatest risks to the organization and then take the necessary actions to address them. LOSA can only reach its full potential if the organizational willingness and commitment exist to act upon the lessons of LOSA. Without meaningful safety action, LOSA data will join the tremendous banks of unused safety data already available within the international civil aviation community. Following are some typical safety change strategies for airlines following a LOSA audit: a) Redefining operational philosophies and guidelines;

8-7

b) Modifying existing procedures or implementing new ones; c) Arranging specific training in error management and crew countermeasures; d) Reviewing checklists to ensure relevance of the content and then issuing clear guidance for their initiation and execution; and e) Defining tolerances for stabilized approaches, as opposed to the “perfect approach” parameters promoted by existing SOPs. Early successes with LOSA have been most noticeable with respect to: a) Improving error management by flight crews, b) Reduction in checklist performance errors, and c) Reduction in unstabilized approaches.

IMPLEMENTING LOSA Undertaking a LOSA audit is a major safety initiative. It cannot be undertaken lightly. While LOSA is very suitable for application in larger airlines with mature accident prevention programmes, it is increasingly being adopted by medium and smaller sized operations. Like successful FDA and CRM training programmes, the knowledge and experience of specialists are required for the design and conduct of an effective LOSA. Organizations wishing to implement a LOSA programme should consult the ICAO Line Operations Safety Audit (LOSA) Manual (Doc 9803) and an airline experienced in operating LOSA. In particular, formal training in the methodology and the use of the specialized LOSA tools and in the handling of the highly sensitive data collected is essential. Since the support of all parties is required for a successful LOSA programme, representatives from flight operations, training and safety departments, as well as representatives from the pilots’ union should meet at the outset and agree on such issues as: a) Operational requirement for a LOSA and the likelihood of conducting a successful audit; b) Programme goals; c) Resources available to guide the conduct of the audit; d) Creation of a LOSA Steering Committee to assist in planning and obtaining buy-in to the programme (including but not limited to flight operations, training, safety department and pilots union); e) Suitable department to be responsible for administering the programme (for example, the safety department); f) Selection and training of credible observers;

8-8

g) Scheduling, targeted concerns (e.g. stabilized approaches), fleet coverage, etc.; h) Protocols to be followed by flight crews and observers; i)

Protocols for the protection of data;

j)

Analysis process;

k) Formal reporting requirements; l)

Communication of results; and

m) Process for implementing changes necessary to reduce or eliminate hazards identified. The best results are obtained when LOSA is conducted in an environment of trust. Line pilots must believe that there will be no repercussions at the individual level; otherwise their behaviour will not reflect daily reality and LOSA will be little more than an elaborate line check.

__________________

8-9

Chapter 9 MANAGEMENT OF SAFETY INFORMATION

General • ICAO recommendations • Accident/Incident Reporting (ADREP) System Information System Needs Understanding Databases • What is a database • Database limitations • Database integrity Database Management • Protection of safety data • Safety database capabilities • Database selection considerations

9-1

This page intentionally left blank.

9-2

Chapter 9 MANAGEMENT OF SAFETY INFORMATION

Databases contain a wealth of safety information; however, without the tools and skills necessary to access and analyse that data, it is essentially useless. GENERAL There is a wealth of aviation safety-related information collected and stored for accident prevention purposes; to the extent that there is a risk of information overload. With careful management, this information can contribute to the risk management and decision-making functions. However, this requires an understanding of data, databases, and the use of appropriate tools for effectively managing the consolidated data to reach timely and valid decisions. Increasingly, computer software is being used to facilitate the recording, storage, analysis and presentation of safety information. Such safety information systems are becoming more integrated and accessible. It is now possible to conduct sophisticated analysis on information in the databases to provide a useful tool for pinpointing where safety interventions are most needed. These interventions need to be based on sound risk management techniques so as to ensure the greatest potential for enhancing safety.44 The hazard identification programmes outlined in previous chapters are capable of generating voluminous information, not all of it relevant to accident prevention. Yet, quality data is the lifeblood of safety analysis (and accident prevention). Decision-making in effective Safety Management Systems is “data driven”. To be convincing, the necessary argument for safety change must be based on the analysis of consolidated and quality data. The establishment and maintenance of a safety database (or databases) provides an essential tool for company managers, accident prevention advisers and for regulatory authorities monitoring safety issues. Unfortunately, many databases lack the data quality necessary to provide a reliable basis for adjusting safety priorities, evaluating the effectiveness of safety programme activities and initiating safety-related research.

ICAO recommendations Annex 13 recommends that States establish an accident and incident database to facilitate the effective analysis of safety information, including that from its incident reporting systems. The database systems should use standardized formats to facilitate data exchange, and States are encouraged to foster regional arrangements, as appropriate.

Accident/Incident Data Reporting (ADREP) System To assist States obtain safety data ICAO maintains the ADREP system. ADREP is a database of information on aircraft accidents and serious incidents worldwide.

44

Source TP 13521, 99/12 - Transport Canada

9-3

The ADREP system uses the ECCAIRS software. This database programme is available to States wishing to establish their own databases in support of accident prevention. The ADREP system provides: a) A significant database of international accident and incident experience for safety analysis and research; b) An internationally developed system for coding safety data to facilitate the exchange of safety data; and c) Regular summary reports and an analytical service in response to specific safety requests from States.

INFORMATION SYSTEM NEEDS Dependent on their size, aviation operators require a range of capabilities and outputs to manage their safety data.45 In general, they require: a) Capability for transforming large amount of safety data into useful information that supports decision-making; b) Workload reduction for managers and safety personnel; c) Automated systems, customizable to their own culture; and d) Relatively low cost. Typically, safety data is used for a variety of tasks, including: a) Trend analysis of operational events; b) Occurrence investigations; c) Hazard identification, risk assessment and risk control; d) Routine performance monitoring using FDA and LOSA data; e) Review of training programmes; f) Reports for management (e.g. quarterly summaries, safety promotion); and g) Comparisons with other like organizations.

UNDERSTANDING DATABASES To exploit the potential benefits of safety databases, a basic understanding of them is required.

45

Source: GAIN WG B: Analytical methods and Tools: Survey of Airline Flight Safety Offices to Define Safety Analysis Requirements: RESULTS

9-4

What is a database Any information that has been grouped together in an organized manner can be considered a database. Paper records can be maintained in a simple filing system (i.e. a manual “database”), but such a system will suffice only for the smallest of operations. Storage, recording, recall and retrieval of data are cumbersome tasks. Safety data of whatever origin should preferably be stored in an electronic database which facilitates the retrieval of the stored information in a variety of formats. The capability to manipulate information, analyse it, and retrieve it in a variety of ways is known as database management. Most database management software packages incorporate the following organizational elements for defining a database: a) Record: A grouping of information items that go together as a unit (such as all data concerning one occurrence); b) Field: Each separate information item in a Record (such as the date or location of an occurrence); and c) File: A group of Records having the same structure and an interrelationship (such as all enginerelated occurrences for a specific year). Databases are considered to be "structured" when each data field has a fixed length and its format type is clearly defined by a number, date, Ayes-no@ answer, character or text. Often only a fixed choice of values is available to the user. These values are stored in reference files, often referred to as base tables or list value tables; for example, selection of aircraft make and model from a pre-determined list. In order to facilitate quantitative analysis and systematic searches, free-form text entry in structured databases is minimized by confining it to a fixed field length. Often such information is categorized by a system of keywords. Databases are considered to be "text-based" when information holdings are primarily written documents, (for example, accident and incident summaries or written correspondence). The data are indexed and stored in free-form text fields. Some databases contain large amounts of text and structured data; however, modern databases are much more than electronic filing cabinets. From an accident prevention perspective, electronic databases can be used to: a) Alert departments to hazards as they are identified; b) Track the status of an investigation, together with any safety actions required; and c) Monitor implementation and effectiveness of actions taken to prevent recurrence.

Database limitations There are limitations to be considered in developing, maintaining or using databases. Some of the limitations relate directly to the database system, others to the usage of the data. If unsupportable conclusions and decisions are to be avoided, database users should understand these limitations. Database users should also know the purpose for which the database was assembled, its limitations and the credibility of the information entered by the agency which created and maintains it.

9-5

Database integrity Safety databases are a strategic element of an organization’s risk management programme. The data are vulnerable to corruption from many sources and extreme care must be taken to preserve the integrity of the data. Many personnel may have access to the database for inputting data. Others will require access to the data for the performance of their safety duties. Access from multiple sites of a networked system can increase the vulnerability of the database. The utility of a database will be compromised by inadequate attention to maintaining the data. Missing data, delays in inputting current data, inaccurate data entry, etc, corrupt the database. Even the application of the best analytical tools cannot compensate for bad data.

DATABASE MANAGEMENT Protection of safety data Given the concerns of the aviation industry regarding the potential misuse of safety data compiled strictly for the purposes of advancing aviation safety, in many ways database management begins with protection of the data. Database managers face a continuing struggle attempting to balance the needs for data protection and making data accessible to those who can advance aviation safety. Protection considerations include: a) Adequacy of “access to information” laws vis-à-vis long-term accident prevention requirements; b) Company policies on protection of safety data; c) De-identification, by removing all details which might lead a third party to infer the identity of individuals (flight numbers, dates/times, locations, aircraft type, etc.); d) Security of information systems, data storage and communication networks; e) Limiting access to databases to those with a “need to know”; and f) Prohibitions on unauthorized use of data.

Safety database capabilities The functional properties and attributes of different database management systems vary, and each should be considered before deciding on the most suitable system for an operator’s needs. Experience has shown that air safety-related incidents are best recorded and tracked using a PC-based database. The number of features available depends on the type of system selected. Basic features should enable the user to perform such tasks as: a) Log safety events under various categories; b) Link events to related documents (e.g. reports and photographs);

9-6

c) Monitor trends; d) Compile analyses, charts and reports; e) Check historical records; f) Data-share with other organizations; g) Monitor event investigations; h) Apply risk factors; i)

Flag overdue action responses; and

j)

Ensure action taken to avoid reoccurrence.

Databases are only as good as the ability to extract and present pertinent data to facilitate analysis and/or reporting. Different kinds of software tools are available to take advantage of the information contained in the databases, such as: Query or search. This type of tool allows a user to extract information from a database (whether it is structured or text-based). Reporting. Users are able to output data in user-friendly formats. Typically, the software provides users the flexibility to produce standard format reports or tailored outputs according to their analytical or presentation needs. Analysis. A variety of commercially available software support statistical analysis methods. Usually, the data must be extracted from the database first and rearranged for use with such packages. However, some packages provide a capability to directly access database contents and perform a range of statistical procedures. A number of systems used by airlines to collect, record, and categorize information about safety events also have analysis capabilities as well as features to facilitate action assignment, monitoring and data exchange.

Database selection considerations The selection of commercially available database systems will depend upon the user’s expectations, the data required, the computer operating system and the complexity of the queries to be handled. A variety of programmes with differing capabilities and skill demands is available. The choice of which type to use requires a balance of five considerations: User-friendliness. The system should be intuitively easy to use. Some programmes provide a wide range of features, but require significant training. Unfortunately, there are often trade-offs between the search power and user-friendliness; the more user-friendly the tool, the less likely it will be able to handle complex queries.

9-7

Access. Although access to all details stored in the database would be ideal, not all users require such access. The structure and complexity of the database will influence the choice of any particular query tools. Performance is a measure of how efficiently the system operates. It depends on such considerations as: a) How well the data are captured, maintained and monitored; b) Whether the data is stored in formats that facilitate trend or other analyses; c) Complexity of the database structure; and d) Design of the host computer system (or network). Flexibility is dependent on the system’s ability to: a) Process a variety of queries; b) Filter and sort data; c) Use binary logic (that is they can deal with “AND/OR” conditions such as “all pilots who are captains AND have 15,000 hour experience”, or “all pilots who are captains OR have 15,000 hours experience”); d) Perform basic analysis (counts and cross-tabulations); e) Produce user-defined outputs; and f) Connect with other databases to import or export data. Costs vary with individual company requirements. The price charged by some system vendors is a flat fee, which allows multiple users on any one licence. For others, the rate increases depending on the number of authorized users. The purchaser should take into consideration such associated cost factors as: a) Installation costs; b) Training costs; c) Software upgrade costs; d) Maintenance and support fees; and e) Other software licence fees that may be necessary.

____________________

9-8

Chapter 10 SAFETY ANALYSIS, STUDIES AND SURVEYS

Introduction • ICAO requirement • Safety analysis – what is it • Analytical thinking — Reasoning — Objectivity and bias — Challenge process Information Sources Analytical Methods and Tools • General methods and tools — Statistical analyses — Trend analysis — Normative comparisons — Simulation and testing — Expert panel — Cost-benefit analysis • Specialized methods and tools — Event reporting and analysis systems — Occurrence investigation and analysis — Human factors analysis — Flight data analysis — Other analytical tools Analysis Process • Data retrieval • Sorting the data • Analysing the data • Drawing conclusions • Common analytical errors Safety Studies • Selecting study issues • Information gathering • Participation Safety Surveys • Survey frequency • Where to look • Active monitoring techniques

10-1

Significant Issues Lists (SIL) Appendices 1. Understanding bias 2. Basics of statistical analysis

10-2

Chapter 10 SAFETY ANALYSIS, STUDIES AND SURVEYS INTRODUCTION Having completed the task of collecting and recording relevant safety data, meaningful and supportable conclusions for accident prevention can only be reached through analysis. Data reduction to simple statistics in itself serves little useful purpose without further evaluation of the practical significance of the statistics in defining a problem which can be resolved. Preventing accidents requires the initial hazard identification, followed by the collaborative effort of risk management. Often the expertise of a number of disciplines is required to analyse the unsafe conditions. During the risk assessment phase, the data are analysed, the probability and severity of risks evaluated, and the degree of acceptability of the risks determined. Ideally, this analysis process proceeds with certainty. In reality, facts are often less than “certain”. Some of the data may be contradictory, other data may be biased or misrepresented and much of the information needed will be missing. Such problems defy easy resolution. ICAO requirement46 ICAO recognizes the linkages between sound safety analysis and accident prevention. ICAO promotes accident prevention by the analysis of accident and incident data and by the prompt exchange of safety information. Having established an accident and incident database and an incident reporting system, States are required to analyse the information contained in their accident/incident reports and their databases to determine any preventive actions required. ICAO also recognizes the value of safety studies in recommending changes needed for accident prevention.

Safety analysis — what is it Analysis is the process of organizing facts using specific methods, tools or techniques. It may be used to: a) Verify the utility and limitations of available data; b) Assist in deciding what additional facts are needed; c) Establish consistency, validity and logic; d) Ascertain causal and contributory factors; and e) Assist in reaching valid conclusions; etc. Safety analysis is based on factual information, possibly from several sources. Relevant data must be collected, sorted and stored in such a manner that it is easily retrievable. Analytical methods and tools suitable to the analysis are then selected and applied. 46

See Annex 13 — Aircraft Accident and Incident Investigation.

10-3

Safety analysis may involve the use of mathematical calculations and/or models to simulate particular events. Safety analysis is often iterative, requiring multiple cycles. It may be quantitative or qualitative. The absence of quantitative baseline data may force a reliance on more qualitative methods of analysis. Regardless, credible safety analysis must evaluate all hypotheses and remain objective, free from conjecture, emotions and politics.

Analytical thinking Analytical thinking is the systematic way by which order is drawn from all the available information. Through analytical thinking, conclusions are reached based on data or known information that a particular event occurred (or might occur). Typically, four activities are involved: Observation (collecting and studying data relevant to the safety problem); Hypothesis (theorizing on any patterns noted in the data); Prediction (concluding what might happen, if the hypothesis under consideration is valid); and Verification (collecting new facts to test the predictions based on the hypothesis).

Reasoning Credible safety analysis is built upon logical (unemotional) reasoning. The reasoning necessary for the reconstruction of an accident sequence differs from the kinds of reasoning used in analysing large amounts of safety data. Following are some of the reasoning processes typically encountered in safety analysis: Logic. Logic is implicit in all reasoning. It helps us link seemingly unrelated information into meaningful patterns, for example, logic is used to support conclusions that, given these circumstances, this event will happen with certainty. Logic is used to determine what conditions are necessary for an event to occur and to what extent these conditions are sufficient for the event to occur. For example, fuel is necessary for a fire; however, the actual amount of fuel present in an occurrence may not have been sufficient to cause a fire of the size which occurred. Deduction. There are two principal logic processes used in every day life: deduction and induction. Deductive logic goes from the broadly known, down to narrow or specific conclusions. Specific facts, theories or events are used to explain how various information pieces are tied together (that is, their interrelationships). In reconstructing the chronology and causes of an occurrence or event, deductive reasoning is required to move from the generally known to form increasingly specific conclusions. This process is essential in understanding why and how something happened. Induction is a process of drawing general conclusions from specific observations or experience. Induction implies that what happened once will happen again under the same circumstances. For effective safety analysis, inductive reasoning should be based upon a large body of consistent information; isolated events may not be sufficient to form a general conclusion. Inductive conclusions are really just theories which can only be confirmed through practical experience. Hypothesis. Early in the analysis process, hypotheses or scenarios are created to attempt to connect the common elements of available data. They require imagination. Preconceived notions are abandoned and as many hypotheses as possible should be considered. No hypothesis can be accepted as truth until it has been tested against available data and information. A hypothesis should fit the

10-4

facts; the urge to make the facts fit the hypothesis must be resisted. As the analysis proceeds, initial hypotheses may have to be abandoned in view of contrary data and information. (However, new data may subsequently come to light to support reconsidering a discarded hypothesis.)

Objectivity and bias Not all information is equally reliable. It is necessary to treat information with judgement; a high degree of skepticism is appropriate. An open mind is required to give due consideration to all relevant information. Notwithstanding the desire to remain objective, time does not always permit the collection and careful evaluation of sufficient data essential to objectivity. Intuitive conclusions may be reached which are not consistent with the objectivity required for credible safety analysis. We are all subject to some level of bias in our judgement (and therefore, in the weighting and evaluation of information). Past experience will influence our judgement, as well as our creativity in establishing hypotheses. One of the most frequent forms of judgement error is known as Aconfirmation bias@. This is the tendency to seek and retain information that confirms what we already believe to be true. As soon as we create a hypothesis, we become susceptible to confirmation bias. Appendix 1 to this Chapter includes a discussion of several concepts of bias that are relevant to the drawing of questionable conclusions in safety analysis.

Challenge process Fundamental to strong analytical thinking is a challenge process. For any analysis to be credible, it must have gone through an introspective process, starting with rigorous review by peers. Adequate skills are necessary in the challenging team which will normally be comprised of managers/specialists/experts. Only if Asevere self-criticism@ has been exercised throughout the analytical process will the argument for change stand up to the rigours of external challenge. All personnel engaged in safety analysis should encourage constructive criticism as a normal part of their work.

INFORMATION SOURCES Much information is available from a variety of sources to support effective safety analysis. However, new problems are continually being introduced by changes in technology, environmental concerns and operational conditions. Thus the past may not be an ideal indication of future problems. The following are the more widely used source of safety information: a) Review of Current Databases including: 1) Investigation reports of accidents and incidents; 2) Safety database(s) (both internal and external to the organization.) These include the mandatory occurrence reporting record (e.g. ADREP), service difficulty reports, industry safety data exchanges (such as STEADES47) etc.);

47

Administered by IATA.

10-5

3) Voluntary incident reports (recognizing that such information is anecdotal and may not be statistically valid); 4) Flight monitoring systems such as FDA and LOSA; and 5) Engine conditions monitoring programmes; b) Review of regulations (governing State as well as regulations from other States to help identify inadequate safety defences. Note that some of these regulations may be out-of-date, incomplete, ambiguous, and even contradictory); c) Review of current corporate situation including: 1) Safety reports and committee minutes; 2) Workplace opinions; and 3) Audit reports; d) Specialist advice from recognized industry experts such as manufacturer's representatives or academics (e.g. sleep researchers). Their perspective may be invaluable in defining the scope and potential consequences of a particular hazardous situation; and e) Literature searches (professional and industry journals and published academic papers) often provide an appreciation of the underlying causes and effects of particular hazards. Indeed, there is so much safety-related information available, accident prevention advisers, managers and regulatory officials may face an over-abundance of information — not all of which is potentially useful. Care is required to select only data and information that are valid, reliable, relevant, etc. for the purposes of the safety analysis.

ANALYTICAL METHODS AND TOOLS Having acquired the information needed to support a safety analysis, the choice of available analytical methods and tools is wide. Some of the methods used in safety analysis are automated, some are not. Several software-based tools (requiring different levels of expertise for effective application) are also available. Some of the relevant methods and tools are general, having broad application beyond the specific needs of aviation safety; and some are unique to aviation.48

48 The website for the Global Aviation Information Network (GAIN) includes descriptions of some of the industry’s best practices with respect to methods and tools effective in aviation safety analysis (www.gainweb.org).

10-6

General methods and tools Some of the available general methods and tools include: Statistical analyses. Many of the analytical methods and tools used in safety analysis are built upon statistical procedures and concepts. For example, risk analysis utilizes concepts of statistical probability. Statistics play a major role in safety analysis by helping to quantify situations, providing insight through numbers. This generates more credible results for a convincing safety argument. The type of safety analysis conducted at the level of a company accident prevention programme requires basic skills for analysing numeric data, identifying trends and for making basic statistical computations such as arithmetic means, percentiles and medians. Statistical methods are also useful for graphical presentations of these analyses. Computers can handle the manipulation of large volumes of data. Most statistical analysis procedures are available in commercial software packages (such as MS Excel). Using such applications, data can be entered directly into a pre-programmed procedure. While a detailed understanding of the statistical theory behind the technique is not necessary, the analyst should understand what the procedure does and what the results are intended to convey. While statistics are a powerful tool for safety analysis, they can also be misused, leading to erroneous conclusions. Care must be taken in the selection and use of the data used in statistical analysis. To ensure appropriate application of the more complex methods, the assistance of specialists in statistical analysis may be required. Such specialists may be helpful in the following circumstances: a) Conducting more complex statistical analytical procedures; b) Developing sampling techniques; c) Interpreting statistical outputs particularly when data samples are small; d) Advising on the use of appropriate normative data; e) Assisting in the use of specialized databases, extraction and analysis tools; f) Detecting data corruption; g) Advising on the use and interpretation of data from external sources, etc.; and h) Consolidating data, checking its homogeneity and relevance. A summary of some of the basics of statistical analysis is included at Appendix 2 to this Chapter. Trend analysis. By monitoring trends in safety data, predictions may be made about coming events. Emerging trends may be indicative of embryonic hazards. Statistical methods can be used to assess the significance of perceived trends. Upper and lower limits of acceptable performance may be defined against which to compare current performance. Trend analysis can be used to trigger “alarms” when performance is about to depart from accepted limits. Desktop computer analysis software programmes, such as MS Excel have the capability to support many types of trend analysis (such as linear, exponential and forecasting).

10-7

Normative comparisons. Sufficient data may not be available to provide a factual basis against which to compare the circumstances of the event or situation under examination with everyday experience (which routinely copes with the same conditions). The absence of credible normative data often compromises the utility of safety analyses. In such cases, it may be necessary to sample real world experience under similar operating conditions, using such methods as: a) Direct observation of skilled personnel performing similar tasks under similar conditions, or observation of like components in service; b) Sampling through the use of questionnaires, surveys or interviews of employees engaged in similar activities; and c) Simulation and testing. Note: By sampling actual flight operations, FDA and LOSA programmes provide much useful normative data for the analysis of unsafe practices in flight operations. Both these programmes were discussed in Chapters 7 and 8 of this manual. Simulation and testing. In particularly difficult cases, the underlying safety hazard may become more evident through testing. For example, laboratory testing may be required for analysing material defects. For suspect operational procedures, simulation in the field under actual operating conditions, or in a flight simulator may be warranted. Expert panel. Given the diverse nature of safety hazards, the variables in assessing the inherent risks, and the different perspectives possible in evaluating any particular unsafe condition, the views of others should be sought, including peers and specialists. When a multidisciplinary team is formed to evaluate the evidence of an unsafe condition, it can also assist in identifying and evaluating the best course for corrective action. Cost-benefit analysis. The acceptance of recommended risk control measures may be dependent on credible cost-benefit analyses. The costs of implementing the proposed measure are weighed against the expected benefits over time. Indeed, cost-benefit analysis may suggest that accepting the risk is preferable to the time, effort and cost of implementing corrective action.

Specialized methods and tools In addition to the methods and tools having general application for safety analysis, specialized methods and tools have proven their value and are regularly used by the aviation community throughout the world. GAIN and the OFSH both provide information and guidance on such analytical methods and tools.49 Such inventories of methods and tools are designed to help safety analysts identify existing computer programmes and/or methodologies that can be used to turn aviation safety data into safety information that is suitable for decision-making. The level of complexity and sophistication of these methods and tools varies widely. Some go well beyond desktop PC skills and capabilities. Thus, the equipment and skill requirements for successfully applying them vary.

49

Operational Flight Safety Handbook (OFSH), Appendix C and Global Aviation Information Network (GAIN) Working Group B

10-8

Event reporting and analysis systems include a capability for analysing occurrence data. BASIS (originally designed as an easy-to-use reporting system for safety events) is a good example of such a system. Add-on BASIS modules permit analysis beyond the data of the event reporting capability to include analysing human factors and inflight recordings.50 Occurrence investigation and analysis. Occurrence investigation increasingly employs a variety of standard analytical methods and tools. By adhering to a systematic methodology, consistency in the identification of hazards and assessment of risks is enhanced. Some of the tools not only help identify root causes of occurrences, but also tie-in to organizational occurrence databases thereby facilitating trend analysis. Human factors analysis. Linked closely to the methods and tools coming into use for occurrence investigation and analysis, are specialized methods and tools for better understanding the impact of human factors on specific occurrences, as well as families of similar occurrences. Flight data analysis. Given the rapid evolution of FDA (FOQA) programmes, there has been a parallel development of analytical methods and tools for capitalizing on data recorded during a flight, permitting trend analysis and the identification of systemic hazards. With the advent of LOSA, methods and tools are being developed to integrate the data from FDA and LOSA. Other analytical tools. A number of non-specialized analytical tools are available, however when pursuing these specialized types of analysis, it would be wise to utilize an experienced analyst: a) Events and Causal Factor (E&CF) Analysis; b) Change Analysis; c) Hazard-Barrier-Target (HBT) Analysis; and d) Fault Tree Analysis (FTA), etc.

ANALYSIS PROCESS Regardless of the analytical tools selected, the safety analysis process involves some or all of the following activities. Data retrieval Safety analysis generally involves an examination of many data points. Different search strategies may be required to get the appropriate data and care is required in determining the search criteria.51 Two types of errors may occur during data retrieval that may compromise the validity of the safety analysis: False positive returns may appear appropriate but on further examination are found to be irrelevant; and False negative returns are relevant data that are not identified during the retrieval process.

50 51

For further information on BASIS, visit the BASIS website at http://www.winbasis.com/ See Chapter 9, Management of Safety Information, for a discussion of data quality.

10-9

Sorting the data The data retrieved must be sorted and arranged in a way that will support the analysis. Irrelevant data (such as false positives) need to be rejected. Judgements are thus made as to the adequacy of the remaining data. If necessary, additional information is sought, perhaps using different methods or tools.

Analysing the data Having retrieved and compiled the requisite data, the most appropriate analytical methods and tools and analytical thought processes are applied. Safety analysis is often an iterative process, sometimes requiring new or additional data or the selection of alternative methods or tools. Hazards are sometimes not visible unless viewed in a particular way. Many cross sections through the data may be required, examining the data from different perspectives. For example, it may not be possible to uncover a safety problem when looking at all recorded events, but a distribution by location may reveal an unacceptable number of occurrences at a particular location, while the other locations have a good safety record.

Drawing conclusions The analyst begins to form impressions from the outset. Preliminary findings are made as to what is known with certainty, the relationships between facts, areas where insufficient or inadequate data have been found, etc. Care must be taken to avoid jumping to premature conclusions based on strongly held impressions or beliefs. The analyst should seek to maximize certainty by ensuring an objective and comprehensive understanding of the facts. Preliminary conclusions must withstand the test of such questions as why and so what. Caution is advised when relying largely on occurrence data. Safety databases generally contain only reported information; little data is available on events which were not reported. For example, the database may hold more traffic conflicts that occurred at airports operating air traffic control towers than at airports with no tower. This does not imply that the towers are the cause of the conflicts. Airports with towers generally have more traffic, providing more opportunities for aircraft to come too close to each other. Also, the controller and/or pilots are more likely to report such occurrences.

Common analytical errors The following are some common pitfalls in analytical thinking: a) Basing a general conclusion on too little evidence, e.g. using words such as "always" or "never"; b) Drawing premature conclusions from insufficient data; c) Stacking the evidence by withholding facts so that the evidence points to only one conclusion; d) Linking two events as if one caused the other when the relationship between them may be more complex; e) Assuming that, because one event follows another, it was caused by the first event;

10-10

f) Assuming that a complicated question has only two possible answers; g) Drawing a conclusion that bears no logical relation to the facts (the “apples and oranges” argument); h) Suggesting that, because two things or situations share some similarities, they must be alike in other ways; and i)

Relying on an expert for issues beyond that person’s qualifications and experience.

SAFETY STUDIES Some complex or pervasive safety issues can best be understood through an examination in the broadest possible context. At the top of the aviation safety spectrum, systemic safety concerns may be addressed on an industry-wide or a global scale. For example, the industry collectively has been concerned with the frequency and severity of approach and landing accidents and has undertaken major studies, made many safety recommendations and implemented global measures to reduce the risks of accidents during the critical approach and landing phases of flight. The convincing argument necessary to achieve large or systemic changes requires significant data, information, analysis and effective communication. Safety argument based on isolated occurrences and anecdotal information will fall on deaf ears. Data from many sources must be coherently integrated and synthesized to permit drawing valid conclusions. For the purposes of this manual, these larger, more complex safety analyses are referred to as Safety Studies. The term includes many types of studies and analysis conducted by State authorities, airlines, manufacturers, and professional and industry associations. ICAO recognizes that safety recommendations may arise not only from the investigation of accidents and serious incidents, but also from safety studies.52 Safety studies have application in hazard identification and analysis relating to issues arising from flight operations, maintenance, cabin safety, air traffic control, airport operations, etc. Safety studies of industry-wide concerns generally require a major sponsor. The Flight Safety Foundation, in collaboration with major aircraft manufacturers, ICAO, NASA and other key industry stakeholders has taken a leading role in many such studies. Civil aviation authorities of specific States have also conducted major safety studies, many identifying safety risks of global interest. Several State authorities have also used safety studies for identifying and resolving hazards in their national aviation systems. Although, it is unlikely that small or medium-sized operators would undertake a major safety study, large operators and regulatory officials may well be involved in identifying widespread systemic safety issues through such means. Consistent with normal safety management processes, safety studies and safety surveys typically include the following steps: a) Information gathering; b) Recording of pertinent data; c) Preliminary analysis and hazard identification; 52

See Annex 13, Chapter 8

10-11

d) Risk assessment, including prioritization of risks; e) Development of risk control strategies; f) Implementation of preferred risk control options; and g) Monitoring and evaluation to determine the effectiveness of the actions taken, the residual risks and any further action required.

Selecting study issues Large operators, manufacturers, safety organizations and regulatory authorities may maintain a list of significant safety issues. (The topic of maintaining a Significant Issues List (SIL) is described later in this Chapter.) Such lists may be based on the accident and incident record in such areas as runway incursions, ground proximity warnings, TCAS advisories, etc. These issues may be prioritized in terms of the risks to the organization or the industry. Given the degree of collaboration and information-sharing necessary to conduct an effective safety study, issues selected for study must enjoy a broad base of support among participants and contributors.

Information gathering The information sources cited earlier with respect to safety analysis are equally applicable for safety studies. Several additional methods (outlined below) are available for acquiring the relevant information necessary to support the broad-based analysis of a safety study. Review of occurrence records. Investigated occurrences may be reviewed by selecting those occurrences which meet some pre-defined characteristics such as Controlled Flight Into Terrain, cabin fires or crew fatigue. By reviewing all available material on file, specific elements may be identified that are suitable for further analysis. Structured interviews. Much useful information can be acquired through structured interviews. Care must be taken in selecting the interviewees to avoid biasing the results. Starting from a series of carefully crafted and sequenced questions, the interviewer can probe particular aspects as deeply as warranted. Structured interviews depend very much on trust between the parties and must be conducted in a non-punitive environment. While they can be time-consuming, such interviews offer the potential for acquiring quality information, even though it may not be a statistically representative sample. Success with structured interviews will depend on the quality of the questions and follow-up questions, and the ability of the analyst to reduce much anecdotal information to useful data for analytical purposes. Directed field investigations of relatively insignificant occurrences (which might normally not be investigated) may uncover sufficient additional information to permit a deeper analysis than that obtained from mandatory reporting systems. By investigating a sample of like-occurrences over a defined period, specific information can be collected in a structured way. Although few of these investigations when considered individually would have contributed much to the collective knowledge of the factors contributing to such occurrences, collectively they may reveal behavioural

10-12

patterns which are compromising safe operations. Field investigations, supplemented by some of the other methods described here, provide an efficient means for validating systemic risks. Literature search. Whether the safety issues under examination have to do with particular equipment, technology, maintenance, human performance, environmental factors, or organizational and management issues, undoubtedly much has already been written on the subject by Aexperts@. Safety analysts conducting major studies require library search skills to locate and review the most compelling works written by scholars and other subject matter experts. Careful use of the Internet provides a vast source of knowledge. Prior to commencing a safety study, it may be appropriate to carry out a literature search on the issue under consideration. This search may be useful in deciding if any further action is required; specifically, is it a safety issue to which meaningful value might be added. Experts’ testimony. Direct contact with recognized subject matter experts may be warranted. For example, if crew fatigue is the issue, it may be appropriate to consult directly with those conducting scientific sleep research. Such experts may be contacted informally over the Internet or telephone; they may also be invited to provide more formal input through submissions to a hearing or public inquiry. Public inquiries. For major safety issues that must be considered from many perspectives, State authorities may convene some form of public inquiry. These provide all stakeholders, individually or as representatives of particular interest groups, an opportunity to present their views in an open, impartial process. However, prior to deciding upon convening a public inquiry, considerable preparatory work is required to demonstrate the need for such an expenditure of resources. Hearings. Less formal meetings (than public inquiries) may be convened with a view to hearing the different (and often divergent) views of the major aviation stakeholders, perhaps involving unions or professional associations, regulatory authorities, operators’ and industry associations. As opposed to a public inquiry, the stakeholders are heard in camera (or private); in this way, they may be more candid in stating their positions.

Participation To maximize the benefits from a safety study, the widest possible participation is advisable. Inputs from a broad cross section of interest groups will ensure that all perspectives are considered. The solution to one interest group’s problem may create problems and hazards for another. Similarly, the information gathered should be examined from a multidisciplinary perspective. SAFETY SURVEYS53 Understanding the systemic hazards and inherent risks associated with everyday activities allows an organization to minimize unsafe acts and respond proactively by improving the processes, conditions and other systemic issues that lead to unsafe acts. Safety surveys are one way to systematically examine particular organizational elements or the processes used to perform a specific operation — either generally, or from a particular safety perspective. Safety surveys are a form of safety study at the organizational level. They have potential application for airlines, maintenance organizations, air traffic control centres, etc. 53

Adapted from TC 13881 Safety Management Systems for Flight Operations and Aircraft Maintenance Organizations

10-13

Surveys are particularly useful in assessing attitudes of selected populations, for example, pilots engaged in extensive trans-meridian flight. Bias in sampling can be minimized if those surveyed are randomly picked. The same rigour as required for structured interviews is required in carefully formulating the questions and their sequence. However, in a survey, open-ended questions requiring narrative responses should be avoided. Rather, questions should elicit specific responses (which can be scored). These might include evaluating an opinion along some pre-determined scale e.g. from strongly disagree through neither agree nor disagree to completely agree. Surveys require prior coordination with the authorities governing the target respondents. For example, a survey may be doomed from the outset without the support of the pertinent unions and professional associations. Safety surveys are a powerful risk management tool. The activities of safety surveys can span the complete risk management cycle from hazard identification, through risk assessment into safety oversight and quality assurance. They are most likely to be conducted by organizations that have truly made the transition from a reactive to a proactive safety culture.

Survey frequency Some organizations advocate conducting safety surveys at regular intervals as an integral component of their safety management systems. Surveys have a particular application when an organization is undergoing significant change, for example: a) During the introduction of a major new safety programme, such as FDA or LOSA; b) During rapid organizational change due to growth and expansion; c) When major changes in the nature of the organization’s operations are planned (introduction of a new fleet, mergers with other airlines, etc.); d) During major labour/management differences (such as contract negotiations, strike action, etc.); or e) Following the change of key personnel (such as Chief Pilot, Maintenance Manager, etc.).

Where to look The information sources outlined earlier in this Chapter can contribute much to the understanding of potential risks facing the organization. Audit reports may provide a structured record of areas of concern in a prioritized format. Since changing accountable managers has a tendency to shorten corporate memories, follow-up assessments of formal audit reports may reveal lingering safety hazards, i.e. a review of action plans and actions actually taken. Typically, employees know where best to look. Line managers and front line workers often have valid perceptions of where the greatest risks are in their areas of responsibility. Their input can be sought through focus groups, consultations with employee representatives and structured interviews with subordinate managers and supervisors.

10-14

Active monitoring techniques Several active monitoring techniques can also be employed in safety surveys to build an accurate picture of the organization’s current situation. For example: a) Inspections are useful for assessing adherence to requirements, plans and procedures by actually inspecting the premises, plant, and equipment or operating practices. Inspections tend to focus at the task level, and may be as simple as informal walk-around by the accident prevention adviser. b) Audits verify conformance with established requirements and standards. They are usually achieved through an independent review of an organization’s systems, personnel, facilities, etc. examining predetermined aspects. Audits tend to be focused at the process level. c) Reviews are useful for assessing the processes involved in a work area or system for their effectiveness and appropriateness — in particular whether the resource allocations are adequate. d) Checklists are useful for consistently collecting specific data related to the system. Such data are essential to adequately portray the level of safety within the company, develop a convincing safety argument for necessary change, and to communicate high priority safety hazards.

SIGNIFICANT SAFETY ISSUES LIST (SIL) Some State regulatory authorities, investigative agencies and large operators have found that maintaining a list of high priority safety issues is an effective means for highlighting areas warranting attention. These lists are sometimes referred to as the “Top Ten” or the “Most Wanted” lists. Such lists prioritize those safety issues that put the aviation system (or the organization) at risk. If such lists are to be of value in guiding the work of those involved in accident prevention, they must not chronicle every perceived hazard. Thus, SILs should be limited to not more than ten issues. Typical issues that may warrant inclusion on an SIL might include: a) Frequency of GPWS warnings; b) Frequency of TCAS advisories; c) Runway incursions; d) Altitude deviations (busts); e) Call sign confusion; f) Unstabilized approaches; and g) Air proximities (near misses) at selected aerodromes, etc. SILs should be reviewed and updated annually, adding new high-risk issues and deleting lesser risk issues.

———————— 10-15

Appendix 1 to Chapter 10 UNDERSTANDING BIAS54 Everyone=s judgement is shaped by their personal experience. Notwithstanding the quest for objectivity, time does not always permit the collection and careful evaluation of sufficient data to ensure objectivity. Based on a lifetime of personal experiences, we all develop mental models that generally serve us well in quickly evaluating everyday situations, Aintuitively@ without a complete set of facts. Unfortunately, many of these mental models reflect personal bias. Bias is the tendency to apply a particular response regardless of the situation. Following are some of the basic biases that can affect the validity of safety analyses: Frequency bias: We tend to over- or under-estimate the probability of occurrence of a particular event because our evaluation is based solely on our personal experience. We assume that our limited experience is representative of the global situation. Selectivity bias: Our personal preferences give us a tendency to select items based on a restricted core of facts. We have a tendency to ignore those facts which do not quite fit the pattern we expect. We may focus our attention on physically important characteristics, or obvious evidence (e.g. loud, bright, recent) and ignore cues that might provide more relevant information about the nature of the situation. Familiarity bias: In any given situation, we tend to choose the most familiar solutions and patterns. Those facts and processes which match our own mental models (or preconceived notions) are more easily assimilated. We tend to do things in accordance with the patterns of our previous experience, even if they are not the optimum solutions for the current situation, e.g. the route we pick to go somewhere may not always be the most efficient under changing circumstances. Experience can be valuable in helping us focus our attention on those things which are most likely to be problematic, but we should recognize that in following these familiar patterns we may be overlooking critical information. Today, management gurus exhort us to Athink outside the box@. Conformity bias: We have a tendency to look for results which support our decision rather than information which would contradict it. As the strength of our mental model increases, we are reluctant to accept facts which do not line up nicely with what we already Aknow@. Time pressures can lead to erroneous assumptions that do not accurately reflect the current reality. We tend to seek information that will confirm what we already believe to be true. Information that is inconsistent with our chosen hypothesis is then ignored or discounted. A frequently cited causal factor in aviation accidents is Aexpectancy@, i.e. individuals see what they want to, or expect to see, and they hear what they want to, or expect to hear. Expectancy is a form of conformity bias. Group conformity or >Group think=: A variation on conformity bias is Agroup think@. Most of us have a tendency to agree with majority decisions; we yield to group pressures to bring our own thinking in line with the group=s. We do not want to break the group=s harmony by upsetting the prevalent mental model. In the interests of expediency, it is a natural pattern to fall into.

54

Adapted from Human Factors Guidelines for Safety Audits Manual (Doc 9806)

10-16

Overconfidence bias. There is a tendency for people to overestimate their knowledge of the situation and its outcome. The result is that attention is placed only on information that supports their choice and ignores contradictory evidence. The defining characteristic of an overconfidence bias is that attention is given to certain information because an individual overrates his/her knowledge of the actual situation. Without the tempering afforded by on-the-job experiences, an inexperienced individual may overrate the value of "classroom" theory versus the more "work-shop"-oriented knowledge used by peers. On the other hand, more seasoned personnel may also let overconfidence bias affect their judgement, having “seen it all before@.

————————

10-17

Appendix 2 to Chapter 10 BASICS OF STATISTICAL ANALYSIS Purpose This précis provides supplemental material in support of statistical analysis for accident prevention.

Data collection To begin an analysis, normally a cursory review of overall counts of the available data and simple crosstabulations of the data are conducted. However, the data that is readily available through routine databases may be insufficient to carry out a credible analysis or risk assessment. Specific additional data may be required. Two methods for acquiring the extra data are: Sampling is one way of acquiring sufficient information for a valid analysis. If a sample of data representative of the larger population of data is compiled, credible conclusions may be drawn from the statistical analysis. Some of the considerations to be taken into account in preparing a valid sample include: a) Size of the sample (the larger the sample size, the more precise the conclusion); b) Selection method (e.g. random samples produce unbiased estimates); c) Representativeness of data (to avoid comparing apples with oranges); d) Homogeneity of sample data (in terms of data definitions, time-frames, operating conditions, etc.); and e) Completeness of data (sufficient to be truly representative of the full population). Because of the resource implications, the sampling process must be limited to only that data necessary to support the scope of the analysis. It may be limited in terms of: a) Duration (e.g. collect data for one year); b) Area (e.g. limit sample to a particular location or fleet); and c) Data type (e.g. use readily obtainable numeric data vs. deriving numbers from narratives). If the sample is a valid representation of population, the statistical analysis should yield credible conclusions on all like-occurrences at a much lower cost than observing or testing every item in the population. Surveys. One way of sampling a large population is through surveys or questionnaires. In addition to the sampling principles described above, some other considerations for survey design include:

10-18

a) Population to be surveyed? How are they to be selected? b) Sample size (vs. accuracy requirements)? c) When? (not after an event that could influence the results); d) Sampling method? (Contacting participants? Format (oral/written)? Tic box vs. narrative?) e) Questions to be asked? (Careful formulation of questions is necessary to get the information really wanted, willing responses, with non-ambiguous answers). Surveys are also a useful means for the collection of normative data (which otherwise might not be available).

Describing and presenting data Quantitative information must be presented in a way suitable for facilitating analysis and promoting understanding. Tables are usually the first step in organizing data and graphs provide the most vivid presentation of the overall picture. However, the more specific aspects of the data contained therein such as their averages, variability, and interrelationships are most succinctly summarized by appropriately chosen numerical measures (such as means and medians). Tabulation Tables are most frequently used to organize data and present numeric information, cross tabulating different variables. For example, events of a particular type by month, or by location, compared to similar data for preceding years. Tables are suitable for summarizing data, identifying trends and conveying analytical conclusions. Tables can also be used for examining single variables (incidents vs. age of fleet) or multiple variables (incidents vs. age of fleet by year and by phase of flight). The following considerations apply when preparing a table: Orientation: A well-chosen title, row and column headings supplemented by explanatory information and footnotes help to orient the reader. Assumptions and recognized anomalies in the data should be provided to avoid incorrect conclusions. There should also be an indication of the data sources. Analysis: The overall features of the table should be examined to confirm that the table works. For example, an appropriate measuring system and units have been used, overall sums/averages are reasonable, and there is consistency between summary rows and columns (e.g. their association follows logically and their variability is credible). Graphical presentations Graphs or charts summarize tabular data in a way which facilitates visualizing the relationship between the variables. Graphs reveal facts about data that would otherwise require careful study to detect in a table. It is important that the initial impression portrayed by a graph be an accurate impression. Here too, some basic principles have to be kept in mind in their preparation to avoid misleading conclusions. For example:

10-19

a) Omission of the line representing zero units on the vertical scale can significantly magnify a change; if it must be omitted it should be clearly illustrated (perhaps by a jagged break). b) Perspective diagrams (e.g. three-dimensional) can present a distorted representation of any fluctuations. c) Disproportionate scales and X/Y axes can exaggerate trends or relative changes. d) Inappropriate labelling of scales, legends and titles can compromise the graph's credibility. Many types of graphs can be used in presenting numeric data, depending on the nature of the data and the purpose intended. Line graphs are useful for illustrating time trends, with time on the horizontal axis. Bar graphs allow several variables to be compared over time or against one another. Scatter plots are used to graph data with two variables, the independent variable being on the horizontal axis. These are sometimes used to develop a simple trend for activity data to get an approximate forecast of an accident rate. Pie charts describe relative proportions of the components of a data set. Histograms, which look like bar charts, depict the distribution of data. Arithmetic means and medians Having organized and presented the data in a useful format, it is also important to understand their relationships. Arithmetic mean. The most commonly used numeric relationship is the arithmetic mean (commonly referred to as the average). The arithmetic mean represents a complete set of data through a single value. When data are arranged according to magnitude, the average lies somewhere near the centre of the data set; therefore averages are often referred to as a measure of centre. The median (discussed below) is also a measure of centre. Medians are useful for ranking variables, for example organizing people by age. Medians do not employ the actual numerical value of the data. Rather, the median of a set of numbers is that number located at the midpoint when they are arranged in increasing order. For example, suppose that the age of pilots involved in accidents ranges between 25 and 60 years. The mean age may be computed to be 45, which might suggest that older pilots have more accidents. However, the median age, may be only 30, which confirms that half the pilots in accidents are actually under 30: the mean was influenced by a few pilots who were perhaps closer to 60. Measures of variation of data Measures of centre are usually incomplete and misleading without some accompanying indication of how spread-out the data are. The degree to which data tend to spread about an average value is called the variation or dispersion of the data. As the example on the mean age of pilots above illustrated, use of it alone lumps the minimum and maximum extreme cases together. Measures of dispersion include percentiles, variance and standard deviation. When using the median to measure centre, use is made of percentiles to indicate the variability or spread. The nth percentile of a data set is a value such that n percent of the observations fall below it.

10-20

The median is therefore the 50th percentile; the 90th percentile would include the first 90 out of 100 ranked pieces of data. Help from statistical specialists may be required to better understand what the variation in a particular data set means. Time series and trend analysis A set of observations or measurements of the same variable made at different times, usually at equal intervals, is called a time series. They can be represented pictorially by constructing a graph of the variable versus time; for example, the number of incident reports received per month. When the frequency of a particular event is measured over time, it is normal to note a sequential increase or decrease in a run. The question is whether or not this run (up or down) is significant? There are many formulas for evaluating runs; one such rule suggests that a run of seven or more sequentially increasing or sequentially decreasing points be further analyzed. On the other hand, shorter runs may also be informative, suggesting that somehow the system is compensating for the change and returning to more “normal” levels.

Making valid comparisons For useful conclusions to be drawn from statistics it requires the valid comparisons of the various data. For example, in comparing accident and incident statistics, more valid comparisons are obtained using rate information than raw numbers of occurrences. If two types of aircraft are compared and type A flies one million hours in one year resulting in one accident and type B flies five million hours in a year resulting in five accidents, the accident rate based on hours flown is the same for both types (one accident per one million flying hours). Rates express the numerical proportions of two sets of data. In accident statistics this usually involves numbers of accidents, incidents, injuries or damage as one set and some measure of exposure, such as flying hours, or numbers of flights as the other. Generally, rates are suited to establishing the general measure of safety of an operation, rather than evaluating specific prevention measures. For a rate to be valid, the sets of data and time frame used must be compatible. For example, long and short haul operators do not fly the same numbers of flights. Their flights are usually of different duration, using aircraft that may have widely varying performance capabilities, and carrying differing numbers of passengers. Therefore, a comparison of the relative safety of these two types of operation will largely depend on whether the exposure base includes numbers of flights, flying hours, miles, passengers or some combination of these. Statistics must be used with caution. For instance, they may show that pilots in a certain age group or with a certain number of flying hours have the most accidents. Arithmetically, these figures are correct; however, they imply that the pilots in these groups are Aless safe@ than pilots in other groups. Before such a determination can be made, it is necessary to determine the total number of pilots in each age group, since the pilot group with the highest number of accidents may also contain the greatest number of pilots. This principle needs to be considered whenever comparisons are made. Many accident statistics are of little value for accident prevention purposes because they provide no valid means of comparison. Traditionally, airline safety statistics have used seat-kilometres (or miles) as an exposure base. With wide-bodied aircraft the number of available seats has increased significantly although the passenger loading can vary considerably. Also, the longer range of such aircraft means that they spend a greater

10-21

proportion of each flight in the cruise phase. Thus seat kilometres are not a particularly useful basis for measuring safety. Most accidents occur during the landing and take-off phases and this is a constant for all flights, irrespective of other factors. In addition, any flight that results in an accident can be seen to represent a failure of the safety process, irrespective of how long it has been airborne, how far it has flown, or how many seats it has. Accordingly, when comparing safety levels of airline operations, the rate of accidents to numbers of flights (or departures) may be a more appropriate measure than the use of flight hours or seat-kilometres. For most general aviation operations (which normally have shorter flights), the number of accidents or incidents and an exposure base measured in flight hours are often used. The resulting rate is then an expression of accidents or incidents per 1 000 000 hours, 100 000 hours, or 10 000 hours. The base chosen will depend on the amount of aviation activity being considered in order to provide the resulting rate as an easily handled number. In special operations involving unique hazards, it may be desirable to use a different exposure base. For example, aerial application operations usually involve many flights per hour. The large number of take-offs and landings substantially increase the likelihood of an accident during these critical phases of flight. For such operations it may be more useful to compare numbers of accidents with numbers of flights. When considering safety trends, the current record is often compared against a base period to determine whether there has been an improvement or decline in safety. This analysis method can also provide a useful hazard alerting technique. However, when the numbers of accidents/incidents are comparatively small, slight changes in numbers, for instance from one year to another, can provide an erratic and virtually meaningless result. To overcome this, some form of averaging can be used. For instance, the number of accidents in the subject year can be compared to the average number of accidents in a preceding three or five-year period. Alternatively, the number of accidents in the subject year can be added to the number of accidents of the previous two or four years, from which a so-called rolling three or five year average is calculated. While the foregoing methods may give an indication of the safety trends of a particular organization, or operation, it has to be remembered that accidents are infrequent and random events. Thus caution should be applied when using such methods as an indication of relative safety.

Common pitfalls As will be apparent, there are many potential pitfalls in the use of statistics to reach meaningful conclusions. The following outlines a few: Understanding of numbers. Numbers can be misleading unless they are used with appropriate care concerning their accuracy, their associated units and the manipulation they are subjected to during the analysis process. Following are examples of common weaknesses in the use of numbers that may compromise the credibility of the statistical analysis: a) Absence of a unit of measure to give the number meaning; b) Manipulation of numbers with inconsistent units of measure; c) Using unsupportable levels of accuracy (especially when using data from more than one source); d) Inconsistency in rounding-off the level of significance of numbers; and

10-22

e) Making extrapolations beyond the confidence limits of the available data. Rules of accuracy. Understanding can be enhanced by respecting some basic rules of accuracy in the use of numbers. For example: a) Quote numbers that can be understood (e.g. avoid numbers with exponential powers if possible); b) Indicate the measurement unit (e.g. accidents per 100 000 departures); c) Ensure consistency in the type of measurement used (e.g. metric vs. imperial); and d) Round data when appropriate (e.g. 5 deaths is more meaningful than 5.21). Significant figures. Misunderstanding in the use of numbers can arise from the misuse of significant figures. Differentiation needs be made between what may be precisely measured to many significant figures and the message that is conveyed. Common sense is often the best guide. For example, although it may be determined that the aircraft had 19,989.5 litres of fuel on board, for safety argument sake, is this not the same as 20,000. If there have been only four occurrences of a particular type, two of which were fatal, is it meaningful to imply great precision by saying that 50.00 % of such occurrences are fatal. Likewise, a numeric drop in CFIT accidents from 10 to 8 (20%) should not be described as significant if the activity rate declined proportionately. Different data sources. Not all safety databases are the same. Each database possesses unique characteristics. When using data from different sources (such as other organizations, manufacturers or States) caution is advised. Failure to take the following issues into consideration may result in “comparing apples with oranges”. a) Terminology. Differences in definitions for occurrence type, event or causal and contributory factors, (for example, near-collision, non-fatal accident). b) Storage. Differences in the capture or updating of data; c) Requirements. Differences in reporting requirements; d) Operations. Differences in the size of the infrastructure, number of employees, the volume of traffic, maintenance and safety programmes, etc.; e) Traffic type. Short haul vs. long haul, passenger vs. freight, etc.; and f) Equipment. Age and extent of modernization of the fleet. Trend analysis. Incorrect conclusions can easily be drawn in examining data covering an extended time period. These include: a) A comparison may be made between annual totals several years apart. The base year for calculating the change may be a "blip" (with an unusually low or high frequency of the event) thereby giving a false impression of an increase or decrease.

10-23

b) Comparisons may be made between two time periods to show the effect of a change (in the operating environment). The result will be meaningless unless the periods chosen adequately reflect the different operating environments. For example, if the reporting rules for gathering the data have changed over the period, compensation must be made. c) In computing averages over a number of years, the individual data points within the time span should be examined to adjust for factors contributing to data points well outside the expected range (e.g. a labour-management dispute, an unusually high number of casualties from a single accident, etc.). d) In quoting an annual average over a recent number of years, the inclusion of a recent random increase or decrease in the calculation may result in misleading figures. e) When describing a trend over a long time period, it may be misleading to ignore large fluctuations particularly in the opposite direction – even though the general trend is clear.

___________________

10-24

Chapter 11 INFORMATION EXCHANGE

Introduction • The need • ICAO requirements Information Sharing Systems • Airlines • ICAO • GAIN • IATA STEADES • ECCAIRS • BASIS/SIE • Other commercially available database systems Disseminating the Safety Message • Websites and networks • Industry and professional associations • Manufacturers • Academia • Regulatory authorities • Company information dissemination Impediments to Information Collection and Sharing Elements for Successful Information Exchange

11-1

This page intentionally left blank.

11-2

Chapter 11 INFORMATION EXCHANGE

Those who ignore the lessons of history are doomed to repeat them. Santyana INTRODUCTION Due to concerns that the worldwide commercial aviation accident rate appeared to have “plateaued”, many aviation safety experts have suggested that, to further decrease the worldwide accident rate, the aviation community will have to improve the sharing of aviation safety information. All too often during accident investigations, experts (including pilots, engineers, air traffic controllers) reveal Awe all knew about that problem@. The challenge is to discover those problems and fix them before they cause an accident or incident, i.e. to improve our foresight with better use of hindsight. Accident prevention requires effective communication at a number of different levels (international, State and organization). Open communication is a noticeable characteristic of organizations with a positive safety culture. On the other hand, organizations with poor safety cultures often have poor or rumourdriven communications.

The need Everyone interested in accident prevention requires access to a variety of information sources. Decisionmaking in effective Safety Management Systems is “data driven”. Thus, managers, accident prevention advisers, manufacturers, and regulatory authorities all need access to relevant safety information. The exchange of safety information is a fundamental element of accident prevention programmes. This includes the dissemination of information both internally, within the organization, and externally with other agencies. Some information is exchanged or disseminated in accordance with specific requirements, such as: Internal a) Monthly or quarterly reports to management; b) Notifications of validated hazards to affected personnel; c) Feedback to reporters to the safety incident reporting system; d) Dissemination of incident investigation reports; and e) Promotion of specific safety issues and practices; etc.

11-3

External a) State accident/incident reporting systems, MORs, etc.; b) Formal accident and incident investigation reports; c) Aircraft service difficulty reports; and d) Safety reports to manufacturers, airline associations, etc. The size and complexity of a safety information system needs to be tailored to meet the organization’s needs. A small organization may be able to manage the relevant information exchange manually. Larger enterprises are best served by automating much of the system. Some larger organizations may require staff to administer the safety information system. Typically, we think of sharing information as the exchange of data (facts and figures). However, users require more than just data for their accident prevention programmes. To give meaning to the wealth of data already available, they need appropriate analytical methods and tools. ICAO requirements55 ICAO emphasizes the need to exchange safety information in the interests of accident prevention. States that have identified safety matters from their databases that are considered to be of interest to other States should forward that information to them as quickly as possible. Furthermore, States are encouraged to promote the establishment of safety information networks among all users of the aviation system and should facilitate the free exchange of information on actual and potential safety deficiencies. (ICAO recognizes the need for standardized definitions, classifications and formats to facilitate such data exchange.) In Europe, JAR–OPS requires that operators “Establish programmes … for the evaluation of relevant information relating to accidents and incidents and the promulgation of related information.”

INFORMATION SHARING SYSTEMS Information sharing is not a new concept. Yet safety information cannot be shared unless there is a system to collect, store and analyse safety data. Processes for exchanging information can then be established. Examples of efforts to facilitate the exchange of safety related information follow.

Airlines Airlines have long known the benefits of sharing experience from say one fleet, across other areas of their operation. Companies also require familiarity with industry’s best practices for the conduct of safe and efficient operations. To assist in this, parent airlines often share information with their subsidiary and commuter operators. Airlines in formal alliances with other airlines may also share safety information with a view to improving safety and efficiency. Increasingly, alliance airlines share data from flight data analysis (FDA) programmes and safety audits. On a broader scale, participating airlines share information 55

Annex 13

11-4

with industry associations, regionally, nationally and internationally. Through such cooperation, safety improvements are promoted and regulatory authorities urged to implement requirements which are safe and efficient. Several formal information-sharing systems are in widespread use.

ICAO In addition to the SARPs contained in Annex 13, ICAO conducts several activities to promote the sharing of safety information: a) Adopting Assembly resolutions to provide high level guidance and policy direct to States on facilitating the exchange of safety information; b) Providing States with guidance material on the subject (such as this manual); c) Working with industry to develop widely acceptable taxonomies for the classification and exchange of safety information; d) Reformatting the ADREP system to be compatible with the ECCAIRS database thus facilitating the exchange of safety data; and e) Working with industry groups (such as GAIN) on ways to facilitate the free exchange of safety information. Global Aviation Information Network (GAIN)56 GAIN is a voluntary government/industry initiative which aims to facilitate the global sharing of safety data, information and best practices. Building on existing safety information systems and the needs of States, operators and manufacturers, GAIN promotes and facilitates the implementation of systems to further the global sharing of aviation safety information. GAIN reports provide access to the industry’s best practices with respect to the conduct of effective safety analysis. As GAIN evolves, it will encourage mentoring and foster the use and development of existing and new analytical methods and tools.

IATA Safety Trend Evaluation, Analysis and Data Exchange System (STEADES) IATA operates a comprehensive safety information exchange system for participating airlines, using deidentified data submitted by airlines, STEADES uses the BASIS software which enables participants to share safety and security lessons arising from incident data. Data are forwarded to IATA for analysis. Regular reports by IATA review safety events, present analysis in the form of trend charts, and provide an overview of emerging safety concerns.

56

See GAIN Website (www.gainweb.org)

11-5

European Coordination Centre for Aviation Incident Reporting Systems (ECCAIRS) ECCAIRS is a database product developed to assist European States with the collection, storage, analysis and exchange of safety related data. Several non-European States have also adopted the ECCAIRS database which is available free of charge. By using standardized processes and taxonomies ECCAIRS offers the following benefits to its member States: a) Standardized collection (reporting) of safety data; b) Provides for the analysis of safety data; c) Facilitates data exchange (no need for data conversions); and d) Compatible with ICAO ADREP thus facilitating reporting to the ADREP system.

British Airways Safety Information System (BASIS) BASIS is a mature data management system that has been in utilized for a number of years and is used by more than 100 airlines and aviation organizations. BASIS has a safety information exchange module, (the Safety Information Exchange), which enables the exchange of standardized safety information.

Other commercially available database systems Other software products are being developed to facilitate the collection, storage, analysis and exchange of safety related data. One such system is the Aviation Quality Database (AQD) which is being used by an increasing number of operators. Users of compatible database products are able to readily exchange safety data and thus benefit from a wider range of safety information.

DISSEMINATING THE SAFETY MESSAGE There are a wide variety of vehicles for both disseminating and acquiring safety information. Some of the more commonly used methods for distributing the safety message are discussed below.

Websites and networks The global acceptance of the Internet has significantly increased the amount of information accessible from anywhere in the world. Literally, thousands of sources of safety information are now available by means of powerful Internet search engines. Because of the dynamic nature of this medium, users are cautioned as to the reliability and currency of much of the available information.

Industry and professional associations National, regional and international associations of airlines, operators, and professional groups meet regularly to discuss issues in the pursuit of accident prevention. Their deliberations reflect the collective wisdom of the group and thus help define best industry practices.

11-6

Manufacturers The major aircraft manufacturers are a valuable source of information for accident prevention purposes. In addition to their responsibility for continuing airworthiness and product improvement, the manufacturers have made substantive contributions to aviation safety by supporting collective initiatives (such as CFIT reduction programmes), participating widely in safety seminars and workshops, conducting training, and producing safety publications on a range of subjects.

Academia Considerable information on university research programmes related to aviation safety is available through the technical press and increasingly on the Internet. Much of this information is particularly relevant to Human Factors. Such research often enjoys the sponsorship of aircraft manufacturers, regulatory authorities or industry associations.

Regulatory authorities The Civil Aviation Authorities in many of the larger States disseminate information pertinent to accident prevention: Occurrence reports: Some States publish reported occurrences involving aircraft and equipment malfunctions, operational errors, etc. Information from de-identified reports submitted through voluntary reporting schemes is also available in some States. Accident reports. Accident final reports are normally published when the investigation is complete. Several State investigative authorities make their reports available either free, by subscription, or on payment for each report. Others post their reports on a website. Safety studies. Some States sponsor research and/or conduct special studies of particular safety hazards within their community. The reports of such studies are frequently available, either on a website or for a nominal fee. Promotional and training material. Some States prepare safety promotional packages, thereby making the industry’s collective wisdom accessible to the wider aviation community. Such promotional material may be in the form of promotional literature, posters, compact disks, videos, etc.

Company information dissemination Effective accident prevention at an operational level depends upon efficient internal communications. Such communications must allow for upwards as well as downwards transfer of information. Some of the traditional methods of information dissemination used by companies include:57 In-house flight safety magazines and newsletters. Typically, these periodicals present a selection of flight and ground safety topics. Although production of an in-house safety magazine is expensive, an

57

Adapted from OFSH Sec 3.9

11-7

informative and well-written publication keeps the accident prevention programme visible and demonstrates management’s commitment to the accident prevention programme. Flight crew notices, briefing sheets or crew NOTAMS. Operational information can be disseminated to flight crew via these methods depending on the urgency of the information. Intranet. In addition to the traditional methods of safety information dissemination, some companies are now using an in-house Intranet so that staff can access the information from a PC at their own convenience. Electronic mail can also be used to disseminate information to staff.

IMPEDIMENTS TO INFORMATION COLLECTION AND SHARING Properly collected, stored and disseminated, safety information can be a powerful accident prevention tool. However, if the information is used improperly, it can seriously affect the availability of such information in future with a consequent effect on accident prevention efforts. Some ways in which safety information can be misused are: a) Job sanctions by employers and/or enforcement action by government regulators based upon the information; b) Public disclosure of the information; c) Criminal sanctions based upon the information; or d) Misuse of the information in civil litigation. Other constraints that can impede the free flow of safety-related information include: a) Likely effect on professional reputations; b) Commercial considerations for operators; and c) Legal ramifications (liability, costs, penalties, etc.). As the industry develops better systems and protocols for information sharing, some of these barriers are gradually being reduced. However, the legal environment in some States (including freedom of information legislation) may discourage the free flow of safety information. An appropriate balance needs to be struck between the need to encourage the flow of safety information in the interest of accident prevention and the rights of the State and its citizens to legal redress. Some States have addressed this issue with legislation that for example defines the way in which cockpit voice recorder (CVR) information is to be handled, or to protect information supplied to certain safety reporting systems. Such actions help create an environment in which people are not afraid to report unsafe events and conditions with a consequent positive benefit for accident prevention.

11-8

ELEMENTS FOR SUCCESSFUL INFORMATION EXCHANGE Several factors will help ensure the free flow of relevant and timely safety information for accident prevention, including: a) Involve all relevant stakeholders, (operators, manufacturers, industry and professional associations, regulatory and investigative authorities, etc.); b) Establish a high level of trust among participants; c) Develop standards for data storage, analysis, and exchange; d) Providing a credible level of protection for the people and organizations involved; e) Data owners should control what data may be shared with whom; f) Provide confidentiality, while permitting follow-up action; g) Start small and expand as condition allows; h) Mentoring may help less experienced operators get started in the safety data sharing exercise; and i)

Informal information exchange processes (networking) can work well, especially for smaller operations.

___________________

11-9

Chapter 12 ASSESSING SAFETY PERFORMANCE

Introduction Safety Health • Assessing safety health — Symptoms of poor safety health — Indicators of improving safety health • Statistical safety performance indicators • Minimum levels of safety Quality Assurance Safety Oversight — International level — State level — Company level • Inspections • Surveys — Questionnaires & checklists Safety Audits – General • Conducting the audit • Self-audit • Airline code-share and alliance audits ICAO Universal Safety Oversight Audit Programme (USOAP) Regulatory Safety Audits • Purpose of regulatory safety audits — Surveillance and compliance — Areas and degree of risk — Competence and performance • Audit process — Preparation — Inspection — Reporting — Management of change Safety Programme Review Appendices 1. Sample indicators of safety health 2. Sample individual safety survey 3. Airline management self-audit checklist (FSF)

12-1

This page intentionally left blank.

12-2

Chapter 12 ASSESSING SAFETY PERFORMANCE INTRODUCTION Safety management systems require feedback on safety performance to complete the closed loop process. Through such feedback, system performance can be evaluated and any necessary changes effected. In addition, stakeholders require an indication of the level of safety within an organization. For example: a) Staff may need confidence in their organization’s ability to provide a safe work environment; b) Line management requires feedback on safety performance to assist in the allocation of resources between the often-conflicting goals of production and accident prevention; c) Passengers are concerned with their own mortality; d) Senior management seeks to protect the corporate image (and market share); and e) Shareholders wish to protect their investment, etc. Although the stakeholders in an organization’s safety process want feedback, their individual perspectives as to “what is safe?” vary considerably. Not everyone sees safety from the same perspective. Deciding what reliable indicators there are of acceptable safety performance depends largely upon how one views “safety”. For example: a) Senior managements may seek the unrealistic goal of Azero accidents@. Unfortunately, as long as aviation involves risk, there will be accidents, even though the accident rate may be very low. In an ideal world, there would be some way of measuring the number of accidents that have been prevented; unfortunately, such a method has not been found. b) Regulatory requirements normally define minimum Asafe@ operating parameters, e.g. cloud base and flight visibility limitations. Operations within these parameters contribute to “safety”, however, they do not guarantee it. c) Statistical measures are often used to indicate a level of safety, e.g. the number of accidents per hundred thousand hours, or fatalities per thousand sectors flown. Such quantitative indicators mean little by themselves but they are useful in assessing whether safety is getting better or worse over time.

SAFETY HEALTH Recognizing the complex interactions affecting safety and the difficulty of defining what is safe and what is not, some safety experts are referring to the “safety health” of an organization. The term safety health is an indication of an organization’s resistance to unexpected conditions, or acts by individuals. It reflects the systemic measures put in place by the organization to defend against the unknown. Further, it is an indication of the organization’s ability to adapt to the unknown. In effect, it

12-3

reflects the safety culture of the organization.

e ut

s is

sp

An a

e

at

m

tD

ra

ig h

og

Fl

Pr

te en

em pl

m

t en

ste

c id

Sy

In

ng

ty

rt i

fe

po

Sa

Re

im

em

d te

ag

op

an M ty

em st

Sy

fe Sa

ad

ag an m ew N

H e a lth y

d

en

t en em

d ra n ge er

m

t

a ly

di ur bo la

s i ld

bu M

r ie

nc

t io ex

pe

ra

e

na

g t in ra pe

pe O

O

Safety Health

(Resistance to Misadventure)

C

la

er

nd

t if

ic

m

at

an

e

ag

gr

an

em

te

en

d

t

Although the absence of safety-related events (accidents, incidents) does not necessarily indicate a “safe” operation, some operations are considered to be “safer” than others. Safety deals with risk reduction to an acceptable (or at least tolerable) level. The level of safety in an organization is unlikely to be static and will vary over time. As an organization adds defences against safety hazards, its safety health may be considered to be improving. However, various factors (hazards) may compromise that safety health, requiring additional measures to strengthen the organization’s resistance to misadventure. The concept of the safety health of an organization varying during its life cycle is depicted in Figure 12-1 below.

Z o n e R e g u la to r y ( m in im u m

U n h e a lth y

C o m p lia n c e

a c c e p ta b le

le v e l)

Z o n e

T im e

Figure 12–1. Variation in safety health

Assessing safety health In principle, the characteristics and safety performance of the “safest” organizations can be identified. These characteristics, which reflect industry’s best practices, can serve as benchmarks for assessing safety performance. Symptoms of poor safety health Continuing with the medical analogy, poor safety health may be indicated by symptoms that put elements of the organization at risk. Appendix 1 to this Chapter provides examples of symptoms for both the regulatory authority and operators which may be indicative of poor safety health. Weakness in any one area may be tolerable, however, many symptoms indicate serious systemic risks compromising the safety health of the organization. Indicators of improving safety health Appendix 1 also provides indications of improving safety health for both the regulatory authority and for operators. These reflect the industry’s “best practices” and a good safety culture. Organizations with the best safety records tend to “maintain or improve their safety fitness” by

12-4

implementing measures to increase their resistance to the unforeseen. They consistently go beyond the minimum regulatory requirements. Identifying such symptoms may provide a valid impression of an organization’s safety health. Even if this impression is valid, it may lack the depth necessary for effective decision-making. Additional tools are required for measuring safety performance in a systematic and convincing way.

Statistical safety performance indicators Statistical safety performance indicators illustrate historic safety achievement; they provide a “snap shot” of past events. Presented either numerically or graphically, they provide a simple, easily understood indication of the level of safety in a given aviation sector, in terms of the number or rate of accidents, incidents or casualties over a given time frame. At the highest level, this could be the number of fatal accidents per year over the past ten years. At a lower (more specific level), the safety performance indicators might include such factors as the rate of specific technical events (e.g. engine shut-downs, TCAS advisories, etc.). Statistical safety performance indicators can be focused on specific areas of the operation to monitor safety achievement or identifying areas of interest. This Aretrospective@ approach is useful in trend analysis, hazard identification, risk assessment and even the choice of risk control measures. Since accidents (and serious incidents) are relatively random and rare events in aviation, assessing safety health based solely on safety performance indicators may not provide a valid predictor of safety performance, especially in the absence of reliable exposure data. Looking backwards does little to assist organizations in their quest to be proactive, putting in place those systems most likely to protect against the unknown. The safest organizations employ additional means for assessing safety performance in their operations.

Minimum levels of safety Aviation organizations must meet regulatory requirements to ensure minimum acceptable levels of safety. Organizations that just meet these minimal requirements may not really be healthy from a safety point of view, but they have reduced their vulnerabilities to the unsafe acts and conditions most conducive to accidents. In short, they have taken minimum precautionary measures. Weak organizations that fail to meet the minimum standards will be removed from the aviation system; either proactively by the regulator removing their operating permit, or reactively, in response to commercial pressures, such as the high cost of accidents or serious incidents, or consumer resistance. QUALITY ASSURANCE58 The functions of a quality assurance programme help ensure that the requisite systemic measures have been taken to meet the organization’s safety goals. Quality assurance does not “assure safety”. 58

Adapted from TP13881, Safety Management Systems for Flight Operations and Aircraft Maintenance Organizations, Transport Canada, 2002

12-5

Rather, quality assurance measures help management ensure that the necessary systems are in place within their organization to reduce the risk of accidents. A quality assurance programme includes procedures for monitoring the performance of all aspects of an organization including such elements as: a) Well designed and documented procedures (e.g. standard operating procedures); b) Inspection and testing methods; c) Monitoring of equipment and operations; d) Internal and external audits; e) Monitoring of corrective actions taken; and f) The use of appropriate statistical analysis, when required. A number of internationally accepted quality assurance standards are in use today. The most appropriate system depends on the size, complexity and product of the organization. ISO 9000 is one set of international standards used by many companies to implement an in-house quality system. Such systems will also ensure that the organization’s suppliers have appropriate quality assurance systems in place.

SAFETY OVERSIGHT One of the three cornerstones for an effective Safety Management System is a formal system for safety oversight. Safety oversight involves regular (if not continuous) monitoring of all aspects of an organization’s operations. On the surface, safety oversight demonstrates compliance with State and company rules, regulations, standards, procedures, etc. However, its value goes much deeper. Monitoring provides another method for proactive hazard identification, validation of the effectiveness of safety actions taken and continuing evaluation of safety performance. Safety oversight can be conducted at the company, the State (or regulatory level) or at the international level. The “monitoring” functions of safety oversight take many forms with varying degrees of formality. International level. At the international level, the ICAO Universal Safety Oversight Audit Programme (USOAP) (described later in this Chapter) monitors the safety performance of all contracting States. International organizations like IATA are also engaging in the safety oversight of airlines through an audit programme. State level. At the State level, effective safety oversight can be maintained through a mix of some of the following elements: a) No-notice inspections to sample the actual performance of various aspects of the national aviation system; b) Formal (scheduled) inspections which follow a protocol which is clearly understood by the organization being inspected;

12-6

c) Discouraging non-compliant behaviour through enforcement actions (sanctions or fines); d) Monitoring quality of performance associated with all licensing and certification applications; e) Tracking the safety performance of the various sectors of the industry; f) Responding to occasions warranting extra safety vigilance (such as major labour disputes, airline bankruptcies, rapid expansion or contraction of activity, etc.); and g) Conducting formal safety oversight audits of airlines or service providers such as air traffic control, approved maintenance organizations, training centres or airport authorities, etc. Company level. At the company level, the size and complexity of the organization will determine the best methods for establishing and maintaining an effective safety oversight programme. Companies with adequate safety oversight employ some or all of the following methods: a) First-line supervisors maintain vigilance (from a safety perspective) by monitoring dayto-day activities; b) Regularly conduct inspections (formal or informal) of day-to-day activities in all safety critical areas; c) Sample employee views on safety (from both a general and a specific point of view) through safety surveys; d) Systematic review and follow-up on all reports of identified safety issues; e) Systematic capture of data which reflect actual day-to-day performance (such as FDA and LOSA); f) Conduct macro-analyses of safety performance (safety studies); g) Conduct a regular operational audit programme (including both internally and externally conducted safety audits); and h) Attention to communicating safety results to all affected personnel, etc. Safety oversight at the company level essentially includes oversight at the ‘individual level’.

Inspections Perhaps the simplest form of safety oversight involves the APA carrying out informal “walk-arounds” of all operational areas of the company. Talking to workers and supervisors, witnessing actual work practices, etc. in a non-structured way provides the APA with valuable insights into safety performance “at the coal face”. As the APA’s frequent presence becomes the norm, a level of trust can be established. The resulting feedback should help in fine-tuning the accident prevention programme.

12-7

To be of value to the organization, the focus of an inspection should be on the quality of the “end product”. Unfortunately, many inspections simply follow a tick-box format. These may be useful for verifying compliance with particular requirements, but are less effective for assessing systemic safety risks. Rather than a tick-box format, a checklist can be used as a guide to help ensure that parts of the operation are not overlooked. Management and line supervisors may also conduct safety inspections to assess adherence to organizational requirements, plans and procedures. However, such inspections may only provide a spot check of the operations, with little potential for systemic safety oversight.

Surveys Surveys of operations and facilities can provide management with an indication of the levels of safety and efficiency within its organization. In attempting to determine the underlying hazards in a system, such surveys are usually independent of routine inspections by government or company management. To determine if a particular facility or operation contains hazards, safety surveys usually involve the use of checklists and informal confidential interviews. Interviews in particular may elicit information which cannot be obtained any other way. The validity of the information obtained may need to be verified before corrective action is taken. Surveys completed by operational personnel can provide important diagnostic information about daily operations. They can provide an inexpensive mechanism to obtain significant information regarding many aspects of the organization, including: a) Perceptions and opinions of operational personnel; b) Level of teamwork and cooperation among various employee groups; c) Problem areas or bottlenecks in daily operations; d) Corporate safety culture; and e) Current areas of dissent or confusion. Like voluntary incident reporting systems, surveys are subjective, reflecting individuals’ perceptions. As such they are subject to the same kinds of limitations, such as the biases of the author, biases of the respondents, biases in interpreting the data, etc. Questionnaires and checklists. These provide an inexpensive, yet effective tool for surveying aspects of an operation. They provide specific data which is suitable for safety analysis, including assessment of safety performance. A sample survey is included at Appendix 2 to this Chapter.

SAFETY AUDITS — GENERAL Safety audits are probably the key tool for assessing safety performance. Like financial audits, safety audits provide a means for systematically assessing how well the organization is meeting its safety objectives. Safety audits should go beyond just checking compliance with regulatory requirements

12-8

and conformance with company standards, to include an assessment as to how well various components of the system are fulfilling their intended function. Safety audits may be conducted internally by the organization, or by an external safety auditor. Demonstrating safety performance for State regulatory authorities is the most common form of external safety audit. Increasingly, however, other stakeholders may require an independent audit as a precondition to providing a specific approval, such as for financing, insurance, partnerships with other airlines, entry into foreign airspace, etc. Regardless of the driving force for the audit, the purpose, activities and products from both internal and external audits are similar. The verification of compliance with regulations and standards is an integral part of safety management. Detailed records of audit findings, corrective actions taken and follow-up inspections should be kept. The results of a safety audit should be communicated throughout the organization, thus contributing to the accident prevention programme. Ideally, safety audits are conducted regularly, following a formal cycle which ensures each functional area is audited at least annually as a part of the organization’s plan for evaluating overall safety performance. Depending on the size of the organization and the availability of resources, experienced and trained individuals within the company may perform audits or they may assist external auditors. Wherever practical, having regard to the size of the organization, these functions should be undertaken by persons who are not responsible for, and have not been involved in, the design or performance of the tasks and functions being audited. In this way, the evaluation is neutral and independent from the operational aspects of the organization. Checklists identify what is to be reviewed during the audit in sufficient detail to ensure that all intended tasks and functions are covered. The extent and complexity of such checklists will depend on the size and complexity of the organization being audited. For the purposes of a safety audit, the checklist should address the following areas in an organization: a) Safety policies and standards; b) Structure of safety accountabilities; c) Safety culture (reactive or proactive); d) Hazard identification and risk management processes; e) Safety oversight capabilities (monitoring, inspections, audits, etc.); and f) Provisions for assuring safety performance of contractors. Since there may be considerable overlap between audits conducted for quality assurance purposes and safety audits, coordination and collaboration among those responsible will be required.

Conducting the audit Typically, an audit team will comprise specialists led by a team leader. The composition and number of those on the audit team will vary depending upon the size and complexity of the area to be audited.

12-9

It is important that there is sufficient expertise in specialist areas to ensure the credibility of the audit findings. Those chosen to undertake an audit must be suitably qualified and trained for the audit function. The audit process involves four distinct phases: preparation, inspection, reporting and management of change. Each of these phases is discussed below under Regulatory Safety Audits.

Self-audit Critical self-assessment (or self-audit) is a tool that management can employ to measure safety margins. A comprehensive questionnaire to assist airline management to conduct a self-audit of those factors affecting accident prevention is included at Appendix 3 to this Chapter. This self-audit checklist is designed for use by senior airline management to identify organizational events, policies, procedures or practices which may be indicative of safety hazards. There are no right or wrong answers applicable to all situations. Nor are all the questions relevant to many types of operations. However, the thrust of a response to a line of questioning may be revealing of the organization’s safety health. Although this self-audit was originally designed for use in flight operations, the line of questioning is relevant for the management of most operational aspects of civil aviation. Thus, this audit checklist can be adapted for application in a variety of situations with the potential to contribute to an accident.

Airline code-share and alliance audits Increasingly, airlines are entering into “alliances” and code-sharing agreements. Under such arrangements, the fare-paying public may have difficulty differentiating one airline from another; the expectation is that an equivalent level of service will prevail, including an equivalent level of safety. Cooperative audits among the participants of these agreements ensure a consistent level of safety across the alliance partners.

ICAO UNIVERSAL SAFETY OVERSIGHT AUDIT PROGRAMME (USOAP) ICAO recognizes the need for States to exercise effective safety oversight of their aviation industries. Thus, ICAO has established the Universal Safety Oversight Audit Programme (USOAP) 59. The primary objectives of USOAP are: a) To determine the degree of conformance by States in implementing ICAO Standards; b) To observe and assess the States’ adherence to ICAO Recommended Practices, associated procedures, guidance material and safety-related practices; c) To determine the effectiveness of States’ implementation of safety oversight systems through the establishment of appropriate legislation, regulations, safety authorities and inspections, and auditing capabilities; and 59

Guidance Material is available from ICAO to assist States in preparing for USOAP audits. See ICAO Docs: Safety Oversight Audit Manual (Doc 9735) and Human Factors Guidelines for Safety Audits Manual (Doc 9806).

12-10

d) To provide Contracting States with advice in order to improve their safety oversight capability. A first USOAP audit cycle of most ICAO Contracting States addressing Annex 1 — Personnel Licensing, Annex 6 — Operation of Aircraft and Annex 8 — Airworthiness of Aircraft has been completed. Summary reports of the audits containing an abstract of the findings, recommendations and the proposed State corrective actions are published and distributed by ICAO to enable other Contracting States to form an opinion on the status of aviation safety in the audited State. Future USOAP audit cycles will use a systemic approach, focusing on safety critical SARPs of all safety related annexes. The audit findings to date have revealed many shortcomings in individual State’s compliance with ICAO SARPs. REGULATORY SAFETY AUDITS60 Purpose of regulatory safety audits For some States, the ICAO USOAP audits are the only assessment made of the States’ aviation safety performance. However, many States do carry out a programme of safety audits to ensure the integrity of their national aviation system. Performance measurements and the appraisal of safety management documentation, including the processes for hazard identification and risk assessment, should be part of the regulatory safety audit. Typically, regulatory audits address three elements: a) Surveillance and compliance with requirements (State, ICAO, EASA, etc.); b) Areas of organizational risk and the systems in place to manage these risks; and c) Competence and performance of those most responsible for accident prevention. Surveillance and compliance. The regulatory authority needs to ensure that the required international, national or local standards are complied with prior to issuing any licence or approval and that the situation will apply for the duration of the licence or approval. The regulator determines an acceptable means for demonstrating compliance. The organization being audited is then required to provide documentary evidence that the regulatory requirements can and will be met. Areas and degree of risk. A regulatory audit should ensure that organizational systems are in place to periodically review procedures to ensure that all safety standards are being continuously met. Assessments should be made of how risks are identified and how any necessary changes are made. The audit should confirm that the individual parts of the organization are performing as an integrated system. Therefore, regulatory safety audits must be conducted in sufficient depth and scope to ensure that the organization has considered the various interrelationships in its management of safety. Competence and performance. In addition to confirming the continuing competency of all staff, the regulatory authority needs to assess the capabilities of personnel in key positions. The possession of a licence granting specific privileges does not necessarily measure the competence of the holder to perform managerial tasks assigned by the organization; for example, competence 60

Adapted from Safety Management Systems; SRG Policy and Guidelines (UK) Chapter 5.

12-11

as a pilot may not equate to managerial acumen. Where there are short term skills gaps, the organization will need to satisfy the Regulator that they have a viable plan to mitigate the situation as soon as practicable. In addition, the Regulator should be interested in the involvement of the highest level of management with responsibility for safety in the day-to-day safety of the organization.

Audit process The process for regulatory audits follows the four steps outlined above (preparation, inspection, reporting and management of change). Preparation. As part of the audit preparation process, the regulatory authority may consult with senior management of the organization to be audited. The organization may be requested to provide preparatory material in advance of the actual audit, for example, selected records, a completed pre-audit questionnaire, manuals, etc. There is an onus on the audit authority to ensure that the organization being audited believes that the audit will be: a) Objective and fair; b) Consistent with audits conducted of similar organizations; c) Conducted by competent auditors; and d) Open, in that audit results will be readily available to management, etc. The organization must have a clear understanding of the purpose, scope, resource requirements, audit and follow-up processes, etc. before the auditors arrive. Those chosen to undertake a Regulatory audit must be credible to the organization being audited. In short, they must be qualified and trained for the audit function in the appropriate areas of expertise. Specialist expertise from outside the audit authority may be required to participate in the audit. The team leader must be an effective communicator, capable of earning and sustaining the trust of the organization being audited. Inspection. The inspection phase is essentially the fact-finding phase of the audit. Information from almost any source may be reviewed as part of the audit. In conducting an inspection for a regulatory safety audit, there is a tendency to limit observations to items of regulatory non-compliance. Auditors must appreciate that such inspections have limitations: a) The organization may rely exclusively upon the Regulator to ensure that they are meeting the standards; b) The standards may only be met whilst the Regulator is undertaking the inspection; c) An inspection report will only highlight those areas of deficiency found at the time of the inspection; and

12-12

d) The inspection does not encourage the organization to be proactive, and often, only issues raised by the inspector will be rectified. Reporting. Management may require regular progress reports as well as a final report on the audit. Planned remedial actions are documented for all identified areas of safety concern. Following a regulatory safety audit, State authorities normally prepare a formal report. At the conclusion of the audit, the audit team leader meets with senior representatives of the organization to debrief the audit. Factual accuracy is confirmed and significant findings are highlighted. A draft copy of the final report is often left with management. The organization then has an opportunity for responding to the findings and recommendations of the audit. The audit findings may fall into three categories: a) Serious discrepancies of non-compliance warranting action to suspend a licence; b) Any discrepancy or non-compliance that must be rectified within an agreed time limit; and c) Observations on issues which are likely to impact on safety, or become a regulatory issue before the next audit. The Regulator may require the organization to provide a formal response to the audit, outlining proposed remedial actions and timelines for implementation. The Regulator may also request subsequent status reports on implementation. Management of change. Upon receipt of the final audit report, management must ensure that progress is made in reducing or eliminating the attendant risks. Follow-up is also required to ensure that any action taken pursuant to the audit does not in any way degrade safety. In other words, new hazards with potentially higher risks must not be allowed to enter the system as a consequence of the audit. A regulatory authority may require follow-up on the audit report to ensure that necessary safety action is taken. Follow-up visits may assist in the monitoring process and in maintaining effective lines of communication between the organization and the regulator concerning progress in implementing change. Satisfactory justification for deviations from agreed timelines may be required as a precondition to renewal of an operating certificate. Failure by the regulator to follow-up on lapses in implementing necessary (and agreed) safety actions will compromise the validity of the entire safety audit process.

SAFETY PROGRAMME REVIEW Any system requires feedback on the fulfillment of the system’s objectives in order to adjust the various inputs and processes. A programme review validates the accident prevention programme; it confirms that a systemic approach is being taken to accident prevention (as opposed to a patchwork of uncoordinated and unrelated safety initiatives). Through a regular review process, management can pursue continuous improvement in the programme. When an accident prevention programme is first implemented, typically there is enthusiasm, with new hopes and aspirations. As the various elements of the programme come into effect, managing the

12-13

programme shifts from implementation to maintenance. The initial enthusiasm may wane. As safety performance data builds, analysis may provide misleading conclusions unless a broad, system-wide perspective is taken. For example, the number of reports to the hazard and incident reporting system may increase during implementation, then reduce somewhat. This does not necessarily mean that the number of actual hazards has been reduced. Perhaps the system is not operating as intended. Staff may have lost confidence in the non-punitive nature of the programme, or in management’s readiness to address identified safety deficiencies. It is the responsibility of the Chief Executive and the APA to ensure that this does not happen.

————————

12-14

Appendix 1 to Chapter 12 SAMPLE INDICATORS OF SAFETY HEALTH Poor safety health

Improving safety health

CAA

CAA

•

•

• • • • • • •

Inadequate governing legislation and regulations; Potential conflicts of interest (such as regulator also being service provider); Inadequate civil aviation infrastructure and systems; Inadequate fulfillment of regulatory functions (such as licensing, surveillance and enforcement); Inadequate resources and organization for the magnitude and complexity of regulatory requirements; Instability and uncertainty within the CAA, compromising quality and timeliness of regulatory performance; Absence of formal safety programmes such as incident reporting and safety oversight; Stagnation in safety thinking (such as reluctance to embrace proven best practices).

• • • •

National incident reporting programmes (both mandatory and voluntary); National safety monitoring programmes including incident investigations, accessible safety databases, trend analysis, etc.; Regulatory oversight including routine surveillance, regular safety audits and monitoring of best industry practices; Risk based resource allocation for all regulatory functions; and Safety promotion programmes to assist operators.

Operator

Operator

•

• •

• • • • • • • • • •

Inadequate organization and resources for current operations; Instability and uncertainty due to recent organizational change; Poor financial situation; Unresolved labour – management disputes; Record of regulatory non-compliance; Low operational experience levels for type of equipment or operations; Fleet inadequacies such as age and mix; Poorly defined (or no) corporate flight safety function; Inadequate training programmes; Corporate complacency re safety record, current work practices, etc.; Poor safety culture.

•

•

• •

Proactive corporate safety culture; Investment in human resources in such areas as non-mandatory training; Formal safety programmes for maintaining safety database, incident reporting, investigation of incidents, safety communications, etc.; Operation of a comprehensive safety management system (i.e. appropriate corporate approach, organizational tools and safety oversight); Strong internal two-way communications in terms of openness, feedback, reporting culture, dissemination of lessons learned; Safety education and awareness in terms of data exchange, safety promotion, participation in safety fora, training aids.

————————

12-15

This page intentionally left blank.

12-16

Appendix 2 to Chapter 12 SAMPLE INDIVIDUAL SAFETY SURVEY61

Please answer the following questions. 1) Experience Time in Company Flight Crew ____ ____0-1 yr ____2-4 yr ____5-9 yr ____10 or more years Ground Crew ____ ____0-1 yr ____2-4 yr ____5-9 yr ____10 or more years 2) Time in present position: 3) What, in your opinion, will cause the next accident? Listed below are some plausible reasons to help you answer this question. Please choose an appropriate answer(s) and explain your choice in a sentence or two. a) Complacency b) Violation of rules c) Mechanical problems/equipment d) Pilot/crew error e) Fatigue or other physical factors f) Working conditions g) Procedures on the ground or in the air h) Other 4) What are the shortcomings of our Accident Prevention Programme as it now exists? Listed below are some plausible reasons to help you answer this question. Please choose an appropriate answer(s) and explain your choice in a sentence or two. a) Lack of discussion about procedures b) Safety publications c) Dissemination of information d) Standardization, training 61

Adapted from OFSH

12-17

e) Lack of support or participation f) Communications g) Suggestions, surveys, etc. h) Other 5) What "close call" experiences have you had in the last 6 months? 6) What do you like about the company’s accident prevention programme? 7) What ideas, comments or recommendations do you have about improving the accident prevention programme in general? 8) What other comments do you have regarding safety in the company? 9) Are there jobs that you do on a routine basis for which you don’t have suitable tools or equipment? Give specifics. 10) Have you received the amount of training you feel you needed to do your job well and safely? What additional training would you need? 11) Are there work routines/schedules that you would like to see changed? Why? 12) Are you aware of ground safety hazards that "we live with" that ought to be corrected? Please elaborate. 13) Are there ground or flight procedures in use, which, in your opinion ought to be changed to enhance safety? Please name.

————————

12-18

Appendix 3 to Chapter 12 AIRLINE MANAGEMENT SELF-AUDIT CHECKLIST (FSF)62 Objective This self-audit may be used by airline management to identify administrative, operational and maintenance processes and related training that might indicate safety hazards. The results can be used to focus management attention on those issues possibly posing a risk of incidents or accidents. Management and organization Management structure a) Does the company have a formal written statement of corporate safety policies and objectives? b) Are these adequately disseminated throughout the organization? Is there visible senior management support for these safety policies? c) Does the organization have a safety department or a designated Accident Prevention Adviser (APA)? d) Is this department or APA effective? e) Does the department/ APA report directly to senior corporate management? f) Does the organization support the periodic publication of a safety report or newsletter? g) Does the organization distribute safety reports or newsletters from other sources? h) Is there a formal system for regular communication of safety information between management and employees? i)

Are there periodic safety meetings?

j)

Does the company participate in industry safety activities, such as those sponsored by the Flight Safety Foundation (FSF), the International Air Transport Association (IATA) and others?

k) Does the organization formally investigate incidents and accidents? Are the results of these investigations disseminated to managers and operational personnel? l)

Does the organization have a confidential, non-punitive hazard and incident-reporting programme?

m) Does the organization maintain an incident database? n) Is the incident database routinely analysed to determine trends? 62

Adapted from Flight Safety Foundation: Flight safety Digest. May 1999

12-19

o) Does the company operate a Flight Data Analysis (FDA) programme? p) Does the company operate a Line Operational Safety Audit (LOSA) programme? q) Does the company conduct safety studies as a means of proactively identifying safety deficiencies? r) Does the organization use outside sources to conduct safety reviews or audits? s) Does the organization solicit input from aircraft manufacturers’ product support groups?

Management and corporate stability a) Have there been significant or frequent changes in ownership or senior management within the past three years? b) Have there been significant or frequent changes in the leadership of operational divisions within the organization in the past three years? c) Have any managers of operational divisions resigned because of disputes about safety matters, operating procedures or practices?

Financial stability of the organization a) Has the organization recently experienced financial instability, a merger, an acquisition or other major reorganization? b) Was consideration given to safety matters during and following the period of instability, merger, acquisition or reorganization? c) Are safety-related technological advances implemented before they are directed by regulatory requirement, i.e., is the organization proactive in using technology to meet safety objectives?

Management selection and training a) Are there well-defined management selection criteria? b) Is operational background and experience a requirement in the selection of management personnel? c) Are first-line operational managers selected from operationally qualified candidates? d) Do new management personnel receive formal safety indoctrination and training? e) Is there a well-defined career path for operational managers? f) Is there a formal process for the annual evaluation of managers?

12-20

Workforce a) Have there been recent layoffs by the organization? b) Are there a large number of personnel employed on a part-time or contractual basis? c) Does the company have formal rules or policies to manage the use of contract personnel? d) Is there open communication between management, the workforce and unions about safety issues? e) Is there a high rate of personnel turnover in operations or maintenance? f) Is the overall experience level of operations and maintenance personnel low or declining? g) Is the distribution of age or experience level within the organization considered in long-term organizational planning? h) Are the professional skills of candidates for operations and maintenance positions evaluated formally during the selection process? i)

Are multicultural processes and issues considered during employee selection and training?

j)

Is special attention given to safety issues during periods of labour-management disagreements or disputes?

k) Have there been recent changes in wages or work rules? l)

Does the organization have a corporate employee health maintenance programme?

m) Does the organization have an employee assistance programme that includes treatment for drug and alcohol abuse?

Fleet stability and standardization a) Is there a company policy concerning cockpit standardization within the organization’s fleet? b) Do pilots and flight operations personnel participate in fleet acquisition decisions?

Relationship with the regulatory authority a) Are safety standards set primarily by the organization or by the appropriate regulatory authority? b) Does the organization set higher standards than those required by the regulatory authority? c) Does the organization have a constructive, cooperative relationship with the regulatory authority?

12-21

d) Has the organization been subject to recent safety-enforcement action by the regulatory authority? e) Does the organization consider the differing experience levels and licensing standards of other States when reviewing applications for employment? f) Does the regulatory authority routinely evaluate the organization’s compliance with required safety standards?

Operations specifications a) Does the organization have formal flight-operations control, e.g. dispatch or flight following? b) Does the organization have special dispatch requirements for extended twin-engine operations (ETOPS)? c) Are fuel/route requirements determined by the regulatory authority? d) If not, what criteria does the company use? e) Does each flight crew member have a copy of the pertinent operations specifications?

Operations and maintenance training Training and checking standards a) Does the organization have written standards for satisfactory performance? b) Does the organization have a defined policy for dealing with unsatisfactory performance? c) Does the organization maintain a database of training performance? d) Is this database periodically reviewed for trends? e) Are check pilots periodically trained and evaluated? f) Does the organization have established criteria for instructor/check pilot qualification? g) Does the organization provide specialized training for instructors/check pilots? h) Are training and checking performed by formally organized, independent departments? i)

How effective is the coordination among flight operations, flight training and flight standards?

Operations training a) Does the company have appropriate training and checking syllabi? b) Does this training include:

12-22

1) Line-oriented flight training (LOFT)? 2) Crew resource management (CRM)? 3) Human factors? 4) Wind shear? 5) Dangerous goods? 6) Security? 7) Adverse weather operations? e.g., anti-icing and de-icing procedures? 8) Altitude and terrain awareness? 9) Aircraft performance? 10) Rejected take-offs? 11) ETOPS? 12) Instrument Landing System (ILS) Category II and Category III approaches? 13) Emergency-procedures training including pilot/flight attendant interaction? 14) International navigation and operational procedures? 15) Standard International Civil Aviation Organization (ICAO) radiotelephone phraseology? 16) Volcanic-ash avoidance/encounters? c) If a ground-proximity warning system (GPWS), traffic alert and collision avoidance system (TCAS) and other special systems are installed, is specific training provided for their use? Are there clearly established policies for their use? d) Are English language skills evaluated during training and checking? e) Is English language training provided? f) At a minimum, are the procedures contained in the manufacturer’s aircraft operations manual covered in the training programme? g) Are there formal means for modification of training programmes as a result of incidents, accidents, or other relevant operational performance?

12-23

Training devices a) Are approved simulators available and used for all required training? b) Is most of the organization’s training performed in the simulator? c) Do simulators include GPWS, TCAS, background communications and other advanced features? d) Are simulators and/or training devices configurations controlled? e) Has the organization established a simulator/training device quality assurance programme to ensure that these devices are maintained to acceptable standards? f) Does the regulatory authority formally evaluate and certify simulators?

Flight attendant training a) Do flight attendants receive comprehensive initial and recurrent safety training? b) Does this training include hands-on use of all required emergency and safety equipment? c) Is the safety training of flight attendants conducted jointly with pilots? d) Does this training establish policies and procedures for communications between cockpit and cabin crew? e) Are evacuation mock-up trainers that replicate emergency exits available for flight attendant training? Do they include smoke simulation?

Maintenance procedures, policies and training a) Does the regulatory agency require licensing of all maintenance personnel? b) Is formal maintenance training provided by the organization for all maintenance personnel? Is such training done on a recurrent basis? How is new equipment introduced? c) Does the organization have a maintenance quality assurance programme? d) If contract maintenance is used, is it included in the quality assurance programme? e) Is hands-on training required for maintenance personnel? f) Does the organization use a minimum equipment list (MEL)? Does the MEL meet or exceed the master MEL? g) Does the organization have a formal procedure covering communication between maintenance and flight personnel?

12-24

h) Are “inoperative” placards used to indicate deferred-maintenance items? Is there clear guidance provided for operations with deferred-maintenance items? i)

Are designated individuals responsible for monitoring fleet health?

j)

Does the organization have an aging-aircraft maintenance programme?

k) Is there open communication between the maintenance organization and other operational organizations, such as dispatch? How effective is this communication? l)

Does the organization use a formal, scheduled maintenance programme?

m) Are policies established for flight and/or maintenance personnel to ground an aircraft for maintenance? n) Are flight crew members ever pressured to accept an aircraft that they believe should be grounded?

Scheduling practices a) Are there flight and duty time limits for pilots? b) Are there flight and duty time limits for flight attendants? c) Do the flight and duty time limits meet or exceed Regulatory requirements? d) Does the organization train flight crew members to understand fatigue, circadian rhythms and other factors that affect crew performance? e) Does the organization allow napping in the cockpit? f) Are on-board crew-rest facilities provided? g) Are there minimum standards for the quality of layover rest facilities? h) Does the organization have a system for tracking flight and duty time limits? i)

Has the organization established minimum crew-rest requirements?

j)

Are augmented crews used for long-haul flights?

k) Are circadian rhythms considered in constructing flight schedules? l)

Are there duty time limits and rest requirements for maintenance personnel?

12-25

Crew qualifications a) Does the organization have a system to record and monitor flight crew currency? b) Does the record keeping system include initial qualification, proficiency checks and recurrent training, special airport qualifications, line-check observations for: 1) Pilots in command? 2) First and second officers? 3) Flight engineers? 4) Instructors and check pilots? 5) Flight attendants? c) Does the regulatory authority provide qualified oversight of instructor and check-pilot qualifications? d) Are the simulator instructors line-qualified pilots? e) Does the organization permit multiple-aircraft qualification for line pilots? f) Do organizational check pilots have complete authority over line-pilot qualification, without interference from management? g) If the organization operates long-haul flights, is there an established policy for pilot currency, including instrument approaches and landings? h) Does the organization have specific requirements for crew pairing for crew scheduling?

Publications, manuals and procedures a) Are all flight crew members issued personal copies of their type operations manuals/FCOM and any other controlled publications? b) How are revisions distributed? c) How are the issue and receipt of revisions recorded? d) Does the organization have an airline operations manual? e) Is the airline operations manual provided to each crew member? f) Is the airline operations manual periodically updated? g) Does the airline operations manual define:

12-26

1) Minimum number of flight crew members? 2) Pilot and dispatcher responsibilities? 3) Procedures for hand over of control of the aircraft? 4) Stabilized approach criteria? 5) Dangerous goods procedures? 6) Required crew briefings for selected operations, including cockpit and cabin crew members? 7) Specific pre-departure briefings for flights in areas of high terrain or obstacles? 8) Sterile-cockpit procedures? 9) Requirements for use of oxygen? 10) Access to cockpit by non-flight crew members? 11) Company communications? 12) Controlled flight into terrain (CFIT) avoidance procedures? 13) Procedures for operational emergencies, including medical emergencies and bomb threats? 14) Aircraft anti-icing and de-icing procedures? 15) Procedures for handling hijacking and disruptive passengers? 16) Company policy specifying that there will be no negative consequences for go-arounds and diversions when required operationally? 17) The scope of the captain’s authority? 18) A procedure for independent verification of key flight-planning and load information? 19) Weather minimums, maximum cross- and tail-wind components? 20) Special minimums for low-time captains? h) Are all manuals and charts subject to a review and revision schedule? i)

Does the organization have a system for distributing time-critical information to the personnel who need it?

j)

Is there a company manual specifying emergency response procedures?

k) Does the organization conduct periodic emergency response drills?

12-27

l)

Are airport facility inspections mandated by the company?

m) Do airport facility inspections include reviews of notices to airmen (NOTAMs)? Signage and lighting? Runway condition, such as rubber accumulations, foreign object damage (FOD) etc.? Aircraft rescue and fire-fighting (ARFF)? Navigational aids (NAVAIDS)? Fuel quality?

12-28

Dispatch, flight following and flight control a) Does initial/recurrent dispatcher training meet or exceed Regulatory requirements? b) Are operations during periods of reduced ARFF equipment availability covered in the flight operations manual? c) Do dispatchers/flight followers have duty-time limitations? d) Are computer-generated flight plans used? e) Are ETOPS alternates specified?

___________________

12-29

Chapter 13 ESTABLISHING AN ACCIDENT PREVENTION PROGRAMME

Introduction Ten steps to getting started 1. Senior management commitment 2. Policies and objectives 3. Organization — Accident Prevention Adviser (APA) — Organizational structures — Statement of safety responsibilities and accountabilities 4. Risk management 5. Hazard identification systems 6. Investigation capability 7. Safety analysis capability 8. Safety promotion, training and education 9. Safety information management system 10. Safety oversight and programme evaluation Appendix 1. Suggested topics to be included in a CEO Statement on Corporate Safety Commitment

13-1

This page intentionally left blank.

13-2

Chapter 13 ESTABLISHING AN ACCIDENT PREVENTION PROGRAMME INTRODUCTION Although there are several ways of establishing an accident prevention programme, there is no single model that “fits all sizes”. Size, complexity, and the type of operation, as well as the corporate safety culture and operating environment, will influence the structure most suited for individual organizations and their unique circumstances. In some companies, management may believe that they have adequately addressed “accident prevention” by appointing a Director of Safety or a Flight Safety Manager. Often, this person is expected to “manage safety” and “prevent accidents” without a clear set of objectives and priorities, with limited guidance about how to do the work and a lack of resources to adequately undertake the task. Effective accident prevention is not a single function carried out by a designated organizational element. It needs to be a “way of thinking”, shared by all elements of the organization. The safest organizations take a systemic approach to accident prevention, organizing and managing their operations such that they experience proportionally fewer serious occurrences. Safety management considerations are thus integrated into the organization in the same way that financial considerations are.

TEN STEPS TO GETTING STARTED Starting an accident prevention programme and operating an effective safety management system can be a daunting task. Taking a systems approach will help ensure that the elements necessary for building an effective programme are present. The following outlines ten steps for establishing an effective accident prevention programme. Integrating each of these elements into a coherent programme in effect implements a safety management system. A confirmation checklist is included at each step to highlight the necessary actions.

1. Senior management commitment The ultimate responsibility for safety rests with the directors and management of the organization. The whole ethos of a company’s attitude to safety – the company’s safety culture – is established from the outset by the extent to which senior management accepts responsibility for safe operations, particularly the proactive management of risk. Regardless of the size, complexity, or type of operation, the success of the accident prevention programme depends on the extent to which senior management devotes the necessary time, resources and attention to safety as a core management issue. Once hazards start to be identified through the Accident Prevention Programme, senior management must be prepared to commit resources to address those hazards. If left unattended, support for the Accident Prevention Programme will quickly evaporate.

13-3

Confirmation checklist °

Senior management is involved – and committed to – the Accident Prevention Programme.

°

Senior management has approved the organization’s safety policy and operational safety standards.

°

The safety policy and standards are communicated to all staff, with visible endorsement by senior management.

°

Appropriate resources are allocated to support the Accident Prevention Programme.

°

Senior management commits resources to correct hazards posing unacceptable risks.

°

Senior management has established an appropriate reporting chain for safety issues.

°

Senior management actively encourages participation in the Accident Prevention Programme.

°

Management promotes a positive safety culture whereby: — Safety information is actively sought; — Personnel are trained for their safety responsibilities; — Accident prevention is a shared responsibility; — Safety-related information is actively disseminated to all affected personnel; — Potential system failures and hazards lead to prompt managerial inquiries and any necessary reforms; — A formal programme is in place to regularly assess safety performance (e.g. safety surveys, safety audits); and — New ideas related to safety are welcomed.

2. Policies and objectives Policies The purpose of developing safety policies and objectives is to set out what the organization is striving to achieve, and how it is going to get there. Safety policies outline the methods and processes the organization will use to achieve desired safety outcomes. They serve as reminders as to “how we do business around here” and are a tangible indication that management is committed to safety. The creation of a positive safety culture begins with the issuance of clear, unequivocal direction. This policy statement should be written and communicated to all staff. An example of the types of topics covered in a CEO policy statement on corporate safety commitment is included as an attachment to this

13-4

chapter. In preparing a safety policy, senior management should consult widely with staff. Consultation ensures that the document is relevant to staff, giving them a sense of ownership in it. Corporate safety policy must also be consistent with relevant State regulations. Confirmation checklist °

A safety policy containing management’s expectations has been developed by management and staff and signed by the Chief Executive Officer.

°

The Safety Policy: — Enjoys the commitment and involvement of all staff; — Aligns with other operational policies; — Provides direction for implementing the policy; — States the responsibilities and accountabilities for directors, managers and employees; — Is reflected in the actions and decisions of all staff; — Has been communicated to all staff; and — Is reviewed periodically.

Objectives Related closely to safety policy (and safety culture) is how an organization sets its objectives. Clearly stated objectives can lead to a commitment to action which will enhance the safety of the organization. In some organizations no explicit safety objectives are stated. A few exceptional organizations set their objectives formally, clearly enunciating their vision, defining desired outcomes, spelling out the attainable steps for meeting the objectives, and documenting the process. Confirmation checklist °

Safety objectives and goals are practical, achievable and are regularly reviewed for relevance;

°

Performance standards are established;

°

Goals have deadlines for their fulfillment;

°

Responsibilities for action are clearly understood; and

°

Managers can follow through and hold those responsible to account for their progress towards those goals.

13-5

3. Organization How an organization arranges its method of doing business and managing safety will influence its resilience to misadventure and its ability to reduce risks. Two considerations are fundamental to establishing an effective organization that will support the accident prevention programme: — Accident Prevention Adviser; and — Organizational Structures. Accident Prevention Adviser (APA) Accident prevention activities need a focal point (or champion) as the driving force for the systemic changes necessary to effect accident prevention across the entire organization. In some States the regulatory authority requires an operator to nominate an individual to coordinate the Company’s flight safety (or accident prevention) programme. This task may be allocated to a pilot or other suitable qualified person who acts in the capacity of Safety Officer as a secondary duty. The effectiveness of this arrangement can vary, depending on the amount of time available to carry out the secondary duty and the operational style of the Company. The function is best accomplished by the appointment of a full-time Safety Officer whose responsibility is to promote safety awareness and ensure that the prevention of aircraft accidents is the priority throughout all divisions and departments in the organization. In this manual, such safety officers are referred to as Accident Prevention Advisers. Large organizations have “staff” specialists to advise and support the line managers. Staff officers do not have the “authority” of line managers to effect the changes necessary for mitigating safety deficiencies. In most organizations the Accident Prevention Adviser (APA) is a “staff” position, advising senior management on safety matters. Indeed, a potential conflict of interest arises if an APA also holds responsibilities for line management. Accident prevention then is a responsibility shared by each line manager, supported by the “staff” specialist, the APA. Accident prevention programmes are the line managers’ responsibility. Senior management must not hold the APA accountable for line managers’ responsibilities; rather the APA is accountable for rendering effective staff support for all line managers to ensure the success of their accident prevention programmes.63 Organizational structures Two different approaches to organizational structures that are consistent with the requirements for assuring both safety and efficiency in airline operations are outlined below. Both are designed to support a coherent “safety management system”. The solid lines represent formal reporting relationships, whilst the broken lines represent informal lines of communication.64

63 64

Adapted from Richard W Wood “Aviation Safety Programs: A Management Handbook” Adapted from TC TP 13881 Safety Management Systems p 16

13-6

Certificate Holder (Accountable Executive)

Director of Flight Operations

Director of Maintenance

Flight Safety Officer

Maintenance Quality Assurance Manager

Safety Office

Formal Reporting Informal Administrative Communications Figure 13-1. Sample organization A

The sample organization in Figure 13-1 is typical of many companies with good safety records. The Flight Safety Officer (FSO) reports directly to the Director of Flight Operations. However, the FSO does not have responsibilities for accident prevention in other departments. To cover considerations of safety in maintenance, a Maintenance Quality Assurance Manager (reporting directly to the Director of Maintenance) coordinates informally with the FSO through the “safety office”. Although the organization chart depicts an informal reporting relationship from the Safety Office to the executive level, this structure does not promote a truly systems approach to safety management. Rather, the organization focuses on safety issues from the perspectives of flight operations and maintenance, only.

13-7

Chief Executive Officer

Quality Manager

Safety Officer

Maintenance

Operations

Other

Formal Reporting Informal Reporting

Figure 13-2. Sample organization B

In the model depicted in Figure 13-2, Sample Organization B, both the APA and the Quality Manager perform the safety management system functions. However, they both have a direct reporting line to the CEO. The safety functions are dispersed throughout the organization to the Operations, Maintenance and other Departments. The APA and the Quality Manager then coordinate with each other and the Departmental Chiefs, assisting them in the fulfilment of their accident prevention functions. This model broadens the focus over that of Model A and is therefore more consistent with the systems approach to accident prevention. Statement of safety responsibilities and accountabilities Regardless of the organizational arrangement or “wiring diagram”, a formal statement of responsibilities and accountabilities is advisable, even in small organizations. This statement clarifies the formal and informal reporting lines on the organizational chart and specifies accountabilities for particular activities. The contents of the statement will vary depending on organizational size, complexity, and relationships. Confirmation checklist °

The organizational structure facilitates: — Line of communication between the APA and the CEO; — Clear definition of authorities, accountabilities and responsibilities avoiding misunderstanding, overlap and conflict (e.g. between APA and the Chief Pilot);

13-8

— Hazard identification and safety oversight. °

An Accident Prevention Adviser (with appropriate competencies and capacity) has been appointed.

°

The roles and responsibilities of the APA (and any staff) are clearly defined and documented.

°

The APA (and staff) have received appropriate briefings and training.

°

Staff and management understand and support the roles of the APA, and the APA enjoys the Chief Executive’s full support.

4. Risk management The risks and costs inherent in commercial aviation necessitate a rational process for decision-making. Implementation of risk management processes is critical to an effective accident prevention programme. Risks cannot always be eliminated; nor are all conceivable accident prevention measures economically feasible. Risk management facilitates this balancing act. Risk management begins with hazard identification. The hazards of any operation may be identified through a variety of reactive and proactive activities. Each identified hazard must be evaluated and its inherent risks assessed in terms of the probability of the hazard contributing to an unsafe event, and the adverse consequences in that eventuality. The next step in the risk management process is to critically assess the hazards and rank them, as far as possible, in order of their risk potential. Factors to consider are the likelihood of the occurrence and the severity of the consequences should there be an occurrence. In assessing the risks, the defences that have been put in place to protect against such hazards need to be evaluated. These defences can, through their absence, misuse, poor design, or condition contribute to the occurrence or exacerbate the risks. Through such a risk assessment process, a determination can be made as to whether the risk is being appropriately managed or controlled. If the risks are acceptable, the operation may continue. If not, then steps should be taken to increase the defences or to remove or avoid the hazard. Typically there is a wide range of potential risk control measures that may help limit exposure to identified risks. Each alternative risk control option needs to be evaluated, residual risks assessed and cost-benefits analysed. Having decided upon a suitable course of action, management must then communicate its safety concerns and planned actions to all persons affected by the acknowledged risks. Confirmation checklist °

A system is in place to pro-actively identify hazards.

°

Criteria are established for assessing risks.

°

Staff are involved in analysing and ranking identified risks.

°

Viable risk control measures are evaluated.

°

Management takes specific action to reduce, eliminate or avoid the risks.

°

Staff are aware of the actions taken and receive relevant training where appropriate.

13-9

°

Checks are in place to confirm that the actions taken are working as intended.

Chapter 5 contains a more complete discussion of the risk management process.

5. Hazard identification systems Isolated hazards and risks may not be a significant problem. However, when hazards or risks exist concurrently at many levels, there is an increased probability of an accident or incident. An effective hazard identification system is an essential element of risk management. Indeed, hazard identification is probably the major function of any accident prevention programme. The key features of an effective hazard identification system are: a) Identifying unsafe conditions; b) Collecting current and applicable hazard information; c) A procedure for receiving and actioning reports of hazards; d) A reliable method of accurately recording, storing, and retrieving hazard data; e) The capability to analyze hazard reports, both individually as well as in aggregate; f) A procedure for distributing lessons learned to affected staff and contractors; and g) Capable of being audited. By virtue of his unique perspective within the organization, the APA has a valuable role to play in the operation of all hazard identification programmes. Confirmation checklist °

A trusting (non-punitive) environment is fostered by management;

°

Formal mechanisms are in place for the systematic identification of hazards;

°

All identified hazards are recorded and investigated;

°

Staff involved in any recorded or reported incident are aware they will not be penalized for normal errors;

°

Affected staff are kept informed of efforts to reduce or eliminate identified hazards; and

°

A system is in place to allow the APA to monitor the status of each identified hazard.

Chapters 6, 7 and 8 provide further information on hazard identification systems.

13-10

6. Investigation capability While the State may investigate mandatorily reportable accidents and serious incidents, an effective accident prevention programme includes the capability to investigate such occurrences from a company perspective. The investigation of minor accidents and incidents (not reported to the State) provides another source for hazard identification. The accident prevention value of these investigations is proportional to the quality of the investigative effort. Without a structured methodology, it is difficult to integrate and analyse all pertinent information from such investigations so as to efficiently assess and prioritize the risks, and to recommend any necessary actions to advance safety. Determination of blame is not relevant to such safety investigations. Confirmation checklist °

The APA and key staff have received formal training in safety investigations;

°

Each hazard and incident report is evaluated with further safety investigation as necessary;

°

Management supports the acquisition and analysis of safety information;

°

Management takes an active interest in investigation findings;

°

Safety lessons learned are widely disseminated; and

°

The regulatory authority is apprised of significant safety concerns potentially affecting other operators, or requiring action by the regulatory authority.

Chapter 15 contains further information on investigating for accident prevention.

7. Safety analysis capability Safety analysis is the process of organizing and evaluating facts, so that valid conclusions can be drawn. It can also be used to identify hazards from aggregated data, to validate and assess the seriousness of identified risks, to evaluate risk control options and to assess the effectiveness of any actions taken to mitigate those risks, etc. A range of analytical methods and tools are available for conducting meaningful safety analyses. Credible risk assessments and convincing argument for mitigating risks depend on solid analytical capabilities. Confirmation checklist °

The APA is experienced in or has received training in analytical methods, or has access to competent safety analysts;

°

Analytical tools are available to support safety analyses;

°

The organization maintains a safety database;

°

Other information sources are accessible;

°

Hazard information and performance data is routinely monitored (trend analysis, etc.);

13-11

°

Safety analyses are subject to a challenge process (peer review);

°

Safety recommendations are made to management and appropriate corrective actions taken.

Chapter 10 contains further information on safety analysis.

8. Safety promotion, training and education Keeping staff informed about current safety issues through relevant training, safety literature, participation in safety courses and seminars, etc. improves the safety health of the organization. The provision of appropriate training to all staff (regardless of their professional discipline) is an indication of management’s commitment to an effective accident prevention programme. (Weak management may see training as an expense, rather than as an investment in the future viability of the organization.) New employees need to know what is required of them and how the organization’s accident prevention programme functions. Indoctrination training should emphasize, “How we do business here”. The APA is the logical resource person for providing a corporate perspective on the organization’s approach to accident prevention. Confirmation checklist °

Management recognizes that all levels of the organization require training in accident prevention, and that the needs vary across the organization;

°

All personnel receive safety indoctrination training and participate in specific on-going training for accident prevention;

°

The organization has an effective programme for the timely promotion of safety issues;

°

Staff are aware of their role in accident prevention and understand how the various elements of the accident prevention programme work;

°

Additional safety awareness training is provided when the operating environment changes (seasonal changes, operational conditions, regulatory requirements, etc.);

°

Staff understand that accident prevention has nothing to do with attributing blame.

Chapter 14 contains further information on training for accident prevention.

9. Safety information management system Operating an accident prevention programme generates significant amounts of information — some of it as documents, some of it data in electronic format. With careful management, this information can well serve the accident prevention programme, particularly the risk management process. However, without the tools and skills to record, store, and retrieve this information, it is essentially useless and its collection a waste of time.

13-12

Confirmation checklist °

Management supports the need for careful documentation and data control;

°

The accident prevention programme is documented;

°

Documents are readily available to those who need them;

°

Safety databases contain relevant, reliable and up-to-date information in a user-friendly format (i.e. readily accessible, standardized for comparative analysis, containing sufficient detail, etc.); and

°

Staff have received the necessary training for maintaining and using the safety information management system.

10. Safety oversight and programme evaluation Taking a systems approach to accident prevention requires “closing the loop”. Feedback is necessary to assess how well the first Nine steps are working. This is done through safety oversight and programme evaluation. Safety oversight can be achieved through inspections, surveys and audits. Are people doing what they are supposed to be doing? For many large organizations, formal safety audits are regularly conducted as a method of providing oversight of day-to-day operations. Safety audits assure staff and management that company activities are being performed as required (i.e. safely). Smaller organizations may get the necessary feedback less formally, through informal observations and discussion with personnel. Programme evaluation validates the accident prevention programme, not only confirming that people were doing what they were supposed to be doing, but that the impact of their collective efforts had achieved the programme’s objectives. Through regular review and evaluation, management can pursue continuous improvement in the accident prevention programme and ensure that the programme remains effective and relevant to the organization’s operation. Confirmation checklist °

Management understands the importance of (and the differences between) safety oversight and programme evaluation;

°

Adequate resources are allocated to the safety oversight and programme evaluation functions;

°

Staff input is sought and provided without fear of repercussion;

°

Regular safety audits are conducted for all functional areas of the organization (including the activities of contracting agencies);

°

Programme evaluations include the systematic review of all available feedback including: quality assurance programme results, safety trend analyses, safety surveys, safety audits, etc.; and

13-13

°

Findings are communicated to staff and reform measures are implemented as required to strengthen the system.

— — — — — — — —-

13-14

Appendix 1 to Chapter 13 SUGGESTED TOPICS TO BE INCLUDED IN A CEO STATEMENT ON CORPORATE SAFETY COMMITMENT Listed below are topics frequently covered in statements of corporate safety commitment. Following each topic are subjects commonly addressed to amplify the corporate position on that topic. Core values. Among our core values, we will include: a) Safety, health and the environment b) Ethical behaviour c) Valuing people Fundamental safety beliefs. Our fundamental safety beliefs are: a) Safety is a core business and personal value. b) Safety is a source of our competitive advantage. c) We will strengthen our business by making safety excellence an integral part of all flight and ground activities. d) We believe that all accidents and serious incidents are preventable. e) All levels of line management are accountable for our safety performance, starting with the Chief Executive Officer (CEO)/Managing Director. Core elements of our safety approach. The five core elements of our safety approach include: Top Management Commitment a) Safety excellence will be a component of our mission. b) Senior management will hold line management and all employees accountable for safety performance. Responsibility and Accountability of All Employees a) Safety performance will be an important part of our management/employee evaluation system. b) We will recognise and reward flight and ground safety performance. c) Before any work is done, we will make everyone aware of the safety rules and processes as well as their personal responsibility to observe them.

13-15

Clearly Communicated Expectations of Zero Accidents a) We will have a formal written safety goal, and we will ensure everyone understands and accepts that goal. b) We will have a communications and motivation system in place to keep our people focused on the safety goal. Auditing and Measuring for Improvement a) Management will ensure regular safety audits are conducted. b) We will focus our audits on the behaviour of people as well as on the conditions of the operating area. c) We will establish performance indicators to help us evaluate our safety performance. Responsibility of All Employees a) Each of us will be expected to accept responsibility and accountability for our own behaviour. b) Each of us will have an opportunity to participate in developing safety standards and procedures. c) We will openly communicate information about safety incidents and will share the lessons with others. d) Each of us will be concerned for the safety of others in our organisation. The objectives of the safety process a) ALL levels of management will be clearly committed to safety. b) We will have clear employee safety metrics, with clear accountability. c) We will have open safety communications. d) We will involve all relevant staff in the decision-making process. e) We will provide the necessary training to build and maintain meaningful ground and flight safety leadership skills. f) The safety of our employees, customers and suppliers will be a Company strategic issue.

(Signed) . CEO/Managing Director/or as appropriate ___________________

13-16

Chapter 14 PRACTICAL CONSIDERATIONS FOR OPERATING AN ACCIDENT PREVENTION PROGRAMME

The Safety Office • Safety office functions Accident Prevention Adviser (APA) • APA selection criteria • Leadership role • APAs in expanding or large airlines • APAs relationships Information Management Safety Committees • Committee chairman • Membership Conducting a Safety Survey Written Communications Disseminating Accident Prevention Information • Safety critical information • “Nice to know” information • Reporting to management Safety Promotion • Promotion methods Accident Prevention Training • Indoctrination for accident prevention • Safety training for management • Safety training for operational personnel • Training for APAs Appendix 1. Sample job description for APA

14-1

This page intentionally left blank.

14-2

Chapter 14 PRACTICAL CONSIDERATIONS FOR OPERATING AN ACCIDENT PREVENTION PROGRAMME THE SAFETY OFFICE65 The APA will require a suitably equipped office. The physical presence of the office (size and location) says a lot about the importance that management attaches to accident prevention and the role of the APA. The APA should be free to move around the organization — probing, questioning and observing. He needs to be readily accessible to anyone wishing to contact him, and should not shut himself in an office and wait for information to come to him. His physical location within the organization is important. If it is remote from the day-to-day operations, communications will inevitably suffer. It will also tend to make his own movement about the organization difficult and will certainly deter others from seeking him out. One of the main sources of safety information within an airline is its flight crew. Therefore, it may be desirable to locate the APA where flight crews can have ready access to him. This is particularly important in relation to human factor elements where the facility to discuss a problem, in confidence if necessary, immediately after a flight may be the deciding factor on whether the information is reported at all. The need for an operator to have a separate accident prevention, or safety office will vary depending on the size and complexity of the organization. Even though in most States there is no regulatory requirement to have a dedicated safety office, many medium to large operators choose to employ a safety office. The safety office provides a focal point for safety-related activities, acts as a repository for safetyrelated reports and information, and provides expertise on data analysis and risk management to functional managers. Just as aircraft operators can benefit from the creation of a dedicated safety office, major service providers (such as air traffic control, aerodromes and aircraft maintenance organizations) may benefit from a similar office.

Safety office functions Regardless of its parent organization, typically, a safety office fulfills a variety of corporate safety functions. Some of the more common functions include: a) Advising senior management on safety-related matters such as: 1) Setting safety policy; 2) Defining authorities and responsibilities for safety; 3) Establishing an effective corporate safety management system; 4) Recommending resource allocations in support of safety initiatives;

65

Adapted from TC TP 13881 p 18

14-3

5) Public communications on safety issues; and 6) Emergency response planning. b) Assisting line managers in: 1) Assessing identified risks; and 2) Selecting the most appropriate risk control measures for those risks deemed unacceptable. c) Overseeing hazard reporting systems: 1) Incident reporting systems; and 2) Flight data analysis programmes, etc. d) Managing safety databases. e) Conducting safety analyses: 1) Trend monitoring; and 2) Safety studies. f) Training on safety programmes and methods. g) Participating in safety committees. h) Safety promotion: 1) Sustaining awareness and understanding of accident prevention across all organizational functional areas; 2) Disseminating safety lessons in-house; and 3) Exchanging safety information with external agencies and similar operations. i)

Safety measurement: 1) Conducting safety surveys; and 2) Providing guidance on safety oversight and audits.

j)

Participating in accident and incident investigations.

k) Safety reporting to meet the requirements of: 1) Management (e.g. annually/quarterly review of safety trends and identification of unresolved safety issues); and 2) Regulator (CAA).

14-4

ACCIDENT PREVENTION ADVISER (APA) APA selection criteria The APA requires both technical and administrative competence, as well as strong inter-personal and communication skills. (Operational skills alone will not be sufficient.) APAs require strength in several areas to complement their professional expertise, such as: a) Broad knowledge of aviation and the company’s organizational functions and activities; b) People skills (such as tact, diplomacy, objectivity and fairness in treating all stakeholders equitably and with respect); c) Analytical and problem-solving skills; d) Project management skills; and e) Oral and written communications skills. A sample job description is contained in Appendix 1.

Leadership role From the outset, the APA must establish his persona. The APA is seen as a subject matter expert on accident prevention (possessing strong technical, administrative and communications skills). However, the APA’s strength is in convincing others of the need for change. This requires leadership skills. For an internal appointment, the APA might have to transition from being “one of the guys” to being a leader. In larger companies, the APA may require support staff, necessitating an ability to manage staff. Further considerations for developing the most suitable leadership style in a particular organization include: Personal example: The APA’s personal value system must include setting an example for all personnel, service providers and management. He must be seen at all times to be upholding the highest standards of accident prevention. The APA’s example cannot be one of “Do as I say, not as I do…” Courage of convictions. The APA must be willing to go against the tide if necessary. In some instances, the APA may be the lone voice for change. The need for change will not always be popular, either to management or to the affected personnel. Consensus builder. As a team builder, whether for office staff or in committee situations, the APA must build consensus, inspiring confidence while convincing key players of the need for change. Often this will require compromise and conflict resolution skills. Adaptable. The APA needs to steer a fine course through ever-changing circumstances and priorities, judging correctly when to speak out and when to give in. There is a fine line between perseverance and stubbornness, between flexibility and lack of personal resolve.

14-5

Self-starter. The effective APA does not wait for problems to present themselves. Consistent with a pro-active safety culture, initiative is required to search out hazards, validate safety deficiencies and provide argument for change. Innovative. There are few new messages in flight safety. Too many lessons have been learned and relearned. The APA must find innovative approaches to such age-old problems as complacency, short-cuts, “work arounds”, etc. Firm but fair. Effective leadership treats all people equitably – firmly in terms of what is required but fairly in being sensitive to unique circumstances.

APAs in expanding or large airlines As an operator expands it will become increasingly difficult for an APA to function as a single entity. An expanding route network may mean an increase in fleet size and perhaps the introduction of different types of aircraft. When this happens, the number of occurrences warranting the APA’s attention will increase. In such circumstances, a minimally staffed accident prevention department may not be able to provide an adequate monitoring function. Additional specialists will likely be needed to assist the APA, perhaps through secondary duty assignments, for example: a) Fleet flight safety officers (pilots qualified on type); b) Engineering safety officers (licensed ground engineers with broad experience); and c) Cabin safety officers (senior cabin crew members, experienced in cabin crew training, safety equipment and operating procedures). These specialists can assist with the monitoring of events peculiar to their own fleet or discipline and provide specialist input during the investigation of occurrences.

APA’s relationships The APA’s areas of interest are very broad. It includes flight operations, maintenance, training, dispatch, station managers, etc., as well as external relations with airport authorities, contractors, suppliers, manufacturers and officials of the regulatory authority. He must foster effective working relationships across the whole spectrum of those influencing accident prevention, at all levels. These relationships should be marked by: a) Competence and professionalism; b) Cordiality and courtesy; c) Fairness and integrity; and d) Openness.

14-6

The APA should be available to discuss accident prevention issues with anyone. A so-called “open-door” policy is not sufficient. The APA must be visible and approachable as he moves through all areas of flight operations and maintenance, and with external suppliers.

INFORMATION MANAGEMENT Collecting information about the health of the company’s operations through flight and maintenance reports, safety reports, audits, evaluations of work practices, etc., generates a lot of data. Some of the data are sensitive warranting measures to protect the identity of the individuals involved, for example, FDA and LOSA data. Safety data can be stored on paper or electronically; however, appropriate levels of security must be exercised for the storage, accessibility and permitted uses for all safety data. Sound management and control of the organization’s databases are fundamental to the effective performance of several accident prevention functions (such as trend monitoring, risk assessment, cost/benefit analyses, occurrence investigations, etc.). The potential volume of safety data can overwhelm APAs, thereby compromising its utility. Vetting procedures are required to ensure that only relevant safety data is retained. The systems and procedures for recording, storing and accessing this safety data need to be tailored to the needs of the organization. A wide range of relatively inexpensive electronic databases, capable of supporting the APA’s data management requirements, are commercially available for desktop computers. These stand-alone systems have the advantage of not using the organization’s main computer system, thus improving the security of the data. The chosen system must provide protection of the data and facilitate secure back-up. SAFETY COMMITTEES66 Depending on the size and complexity of the organization, the APA may benefit from the support of a safety committee. Smaller organizations may best discuss and resolve safety matters in an informal way. As long as there is good communication and staff and management are willing to provide advice and assistance to the APA, a formal safety committee may not be necessary. However, for larger organizations with several operational departments, communications are often “filtered” and more inter-departmental coordination is required. Safety issues often require inputs from a variety of different fields. Safety committees can provide a forum for discussing safety-related issues from different perspectives, especially for safety issues requiring a broader viewpoint. With multidisciplinary expertise, safety committees are natural forum for the “cross pollination” of ideas and for assessing safety performance from a “system” perspective; committees are more likely to see the “big picture”. They also provide a means by which safety achievements can be reviewed and safety information disseminated. The focus of safety committees should be on “action”, as opposed to “dialogue”. The role of the safety committee may include: a) Act as a source of expertise and advice; b) Review the progress on identified hazards and actions taken following accidents and incidents; 66

See OFSH Art 3.3

14-7

c) Make safety recommendations to address safety hazards; d) Review internal safety audit reports; e) Review and approve the audit response and the actions taken; f) Encourage lateral thinking about safety issues; g) Help identify hazards and defences; and h) Prepare and review safety reports to the Chief Executive Officer. Safety committees do not normally have the authority to direct individual departments. (Such authority would interfere with the formal lines of authority.) Rather such committees make recommendations for action by the responsible managers. However, because of accountability issues, some airlines have introduced safety committees at the Board level which ensures that corrective actions are taken.

Committee chairman The safety committee is often chaired by a senior executive with the APA acting as Secretary. This arrangement helps ensure that discussions do not avoid controversial issues. To be effective, the safety committee must enjoy the support of the CEO and departmental heads. Those with the capacity to make and authorize decisions should participate in the meetings for particular agenda items. Without the involvement of the decision-makers, the meetings may become “chat rooms” with much time wasted.

Membership Safety committees generally comprise representatives from key departments, such as Flight Operations, Engineering, Flight and Cabin Crew Training departments. Depending on the size of the organization, separate sub-committees may be required to address specific issues from these departments. The APA and the safety office coordinate activities and provide assistance to the safety committee and its subcommittees. Agenda. All committee members should have the opportunity to submit potential agenda items. If there are insufficient agenda items to warrant a regular meeting, it should be cancelled. The APA, as meeting Secretary, should finalize the agenda with the Chairman, providing the necessary background material for each item. Avoid standing (information) items in favour of items requiring decisions and action. The minutes. The APA, as the secretary of the meeting, should prepare draft minutes immediately following the meeting (before memories lapse). Once the Chairman has signed the minutes, they become an action document. The minutes should be distributed within a few working days of the meeting while those responsible for action items remember their commitment. Copies of the minutes should be distributed widely throughout the organization — for both line personnel and management. Follow up. After the meeting, other priorities may capture the attention of the action addressees. The APA should discretely monitor actions being taken (or not taken) and review progress with those who have undertaken a commitment to the committee.

14-8

CONDUCTING A SAFETY SURVEY Safety surveys offer the APA a flexible and cost-effective method for identifying hazards by sampling expert opinion. They may be used to review a particular area of safety concern where hazards appear, or are suspected, or as a monitoring tool to confirm that an existing situation is satisfactory. In either case, the principles and procedures are the same and they are equally applicable to large or small surveys. Surveys may be conducted using questionnaires or interviews. In either case, the respondent must receive an assurance of confidentiality regarding the information volunteered through the survey. Both methods require skill in formulating questions which will provide a valid reference point, without leading the person being surveyed. Interviewing requires particular skill in keeping the questions neutral and unbiased, avoiding negative feedback, encouraging openness, etc. The objectives of the survey should be clearly enunciated for all intended respondents. The sample size should be sufficient to permit valid conclusions to be drawn from the information obtained. The level of formality, the breadth of participation sought, etc. will depend on the scope of the survey. Other factors to be considered in conducting a survey include: a) Obtaining the cooperation of the people involved in the survey; b) Avoiding any perception of a Awitch-hunt@. (The objective is to gain knowledge, and any suggestions of blame or punishment will be counter-productive); c) Respecting the experience of the target respondents. (They are usually more experienced in their specialty than the surveyor.); d) Criticism (real or implied) can destroy the rapport with the person being interviewed; and e) Hearsay and rumour need to be substantiated before being accepted. When planning the survey, the following points need to be considered: a) Purpose of the survey; b) Selection of those best qualified to perform it; c) Defining the areas to be examined; and d) Ensuring that the management responsible for the area being surveyed is aware of the intended actions and supports its objectives. The gathering and analysis of the information, development of recommendations and the preparation of the final report of a survey will take time. It is therefore desirable to hold a brief review with those responsible as soon as the survey has been completed. If any conclusions are immediately obvious, they should be discussed informally. Recommendations should be practical and within the scope and ability of the organization concerned. Sensitive issues should not be avoided, but care should be taken to ensure they are presented in a fair, constructive and diplomatic manner.

14-9

WRITTEN COMMUNICATIONS Any significant recommendations for safety action should be provided in writing. This reduces the likelihood of misunderstandings ensuring that everyone is “singing from the same song sheet”. It also provides a baseline for evaluating the effectiveness of any implementation action. Regardless of the nature of the safety action being recommended, poorly written communications stand little chance of convincing the recipient to change priorities in order to address the risk. Therefore written communications should meet the following criteria: a) Clarity of purpose; b) Simplicity of language; c) Attention to detail, yet concise; d) Relevance of words and ideas; e) Logic and accuracy of argument; f) Objective, balanced and fair consideration of facts and analysis; g) Neutral (non-blameworthy) tone; and h) Timeliness.

DISSEMINATING ACCIDENT PREVENTION INFORMATION The APA should be the focal point for safety-related information – hazard reports, risk assessments, safety analyses, investigation reports, audit reports, meeting minutes, conference proceedings, etc. From all this information, the APA must sift the most relevant safety messages for dissemination. Some messages are urgent (before next flight), some are directive, some for background understanding, some seasonal, etc. Most staff do not have time to read all this information, so the APA must digest the salient points into easily understood safety messages. Several considerations should guide the APA in disseminating accident prevention information: a) Criticality of the safety information; b) The target audience; c) Best means for disseminating the information (briefings, directed letters, newsletters, company intranet, videos, posters, etc.); d) Timing strategy to maximize the impact of the message (winter briefings generate little interest in summer); e) Content (e.g. how much background information vs. the core message?); and

14-10

f) Wording – most appropriate vocabulary, style and tone.

Safety critical information Urgent safety information may be effectively disseminated using such means as: a) Direct message (oral or written) to responsible managers (such as Chief of Flight Operations, Chief Pilot, Chief of Maintenance); b) Direct briefings (e.g. for flight crew of a particular fleet); c) Passed through the flight dispatcher; d) Flight crew reading files; e) Direct mail (post, fax, or e-mail) — particularly for personnel who are away from home base.

“Nice to know” information The aviation industry produces considerable literature – some of it targeted at particular operations. This material includes State accident/incident reports, safety studies, aviation journals, proceedings of conferences and symposia, manufacturers’ reports, training videos, etc. Increasingly, this information is available electronically. Regardless of the format of the information, it may be made available to staff and/or management through: a) Internal circulation system; b) Safety library (probably the APA’s office); c) Summaries (probably by the APA) notifying staff of the receipt of such information; and d) Directed distribution to selected managers.

Reporting to management Keep it simple. Management does not have the time to sift through large amounts of material, some of which is probably irrelevant. They are interested basically in such questions as: a) What is the problem? b) How could it affect the company? c) How likely is it to happen? d) What is the cost if it does happen? e) How can the hazard be eliminated?

14-11

f) How can the risk be reduced? g) How much will it cost to fix? h) What are the downsides of such action?

SAFETY PROMOTION Safety promotion involves the communication of information with the objective of modifying behaviour or eliminating factors known to induce accidents. Traditionally, the emphasis has been on the communication of information. However, as increased human factor insights are gained, it is increasingly apparent that most accidents do not result from a lack of information so much as from deficiencies in attitude and behaviour. These, in turn, can erode the judgement on which decisions are based. If a message is to be learned and retained, the recipient first has to be positively motivated. Unless this is achieved, much well-intended effort will be wasted. Propaganda which merely exhorts people to take more care, avoid making errors, etc. is ineffective as it does not provide anything substantial which individuals can relate to. This approach has sometimes been described as the Abumper sticker@ approach to safety. Safety topics should be selected for promotional campaigns based on their potential to control and reduce losses due to accidents and incidents. Selection should therefore be based on the experience of past accidents or near misses, matters identified by hazard analysis and observations from routine safety audits. Employees should also be encouraged to submit suggestions for promotional campaigns.

Promotion methods All methods of dissemination C the spoken and written word, posters, videos, slide presentations, etc. C require talent, skill and experience to be effective. Poorly executed dissemination may be worse than none at all. Professional input is therefore advisable when disseminating information to a critical audience. Once a decision has been made to disseminate safety information, a number of important factors should be considered. They include: a) The audience: the message needs to be expressed in a terms and vernacular that reflect the knowledge of the audience. b) The response: what is hoped to be accomplished? c) The medium: while the printed word may be the easiest and cheapest, it is likely to be the least effective? d) The style of presentation: this may involve the use of humour, graphics, photography and other attention-getting techniques. Ideally, a safety promotion programme for accident prevention will use several different communication methods. The following methods are commonly used for this purpose:

14-12

a) Spoken Word: this is perhaps the most effective method, especially if supplemented with a visual presentation. But it is also the most expensive method, consuming time and effort to assemble the audience, aids and equipment. Some States employ safety specialists who visit various organizations, holding lectures and seminars. b) Written Word: by far the most popular because of speed and economy. However, the explosion of printed material tends to saturate our capacity to absorb it all. Printed safety promotion material competes for attention with considerable amounts of other printed material. In the digital era, the printed word has an even harder time competing for attention. Professional guidance or assistance may be desirable to ensure that the message is conveyed effectively. c) Videos: offer the advantages of dynamic imagery and sound to reinforce particular safety messages efficiently. However, they have two main limitations: expense of production and the need for special equipment for playback. Nonetheless, they can be effective in getting a particular message disseminated throughout a widely dispersed organizational structure – minimizing the need for staff travel. Today they may be distributed electronically or via CD. A variety of safety videos are available commercially, many listed on safety sites on the Internet. d) Displays: when a message is to be shown at a large gathering, the display booth is a good Aself-briefing@ technique. Imagination and display expertise are required to present not only the message but also the image of the organization. Its drawbacks are expense and, unless it is manned, a static and somewhat uninteresting appearance. Professional guidance or assistance is needed to ensure the message is conveyed effectively. e) Web Sites. Many of the foregoing promotion methods may have little appeal to generations that have grown up with PCs, digital game toys and Internet access. As previously outlined, the explosive growth of the Internet and Intranet technologies offer significant potential for improvement in the promotion of accident prevention. Even small companies can establish and maintain a website to disseminate safety information and promote accident prevention. This is an extremely powerful tool for APAs. f) Conferences, Symposia, Seminars, Workshops etc. provide ideal fora for promoting particular safety issues. The company, the regulator, industry associations, safety institutes, universities, manufacturers, etc. may sponsor these. The value of such fora often goes well beyond safety promotion by helping establish contacts with others in the safety field. When a major promotional programme is being contemplated, it is wise to seek advice from experienced communicators and knowledgeable representatives of the target groups involved.

ACCIDENT PREVENTION TRAINING The level of accident prevention training required will vary from general safety familiarization, to expert level for safety specialists. For example: a) Corporate indoctrination for all staff; b) Training aimed at management’s responsibilities; c) Training for operational personnel (pilots, flight attendants, maintenance technicians, ramp personnel, etc.); and

14-13

d) Training for aviation safety specialists (such as the APA, Flight Data Analysts, etc.).

Indoctrination for accident prevention Indoctrination training for all staff should include discussion of: a) Corporate safety philosophy, safety policies and safety standards (including corporate approach to disciplinary action vs. safety issues, integrated nature of safety management, risk management decision making, etc.); b) Corporate safety record, including discussion of areas of systemic weakness; c) Corporate safety goals and objectives; d) Organization for accident prevention; e) Corporate accident prevention programmes (SIRS, FDA, LOSA, etc.); f) Requirement for ongoing internal assessment of organizational safety performance (e.g. employee surveys, safety audits and assessments); g) Reporting accidents, incidents and perceived hazards; h) Lines of communications for safety matters; i)

Feedback and communication methods for the dissemination of safety information;

j)

Safety awards programmes (if applicable);

k) Safety audits and reviews; and l)

Safety promotion and information dissemination.

Safety training for management It is essential that the management team understands the principles on which the safety system is based. Corporate training ensures that managers and supervisors are familiar with the principles of the Safety Management System and their responsibilities and accountabilities for safety.

Safety training for operational personnel In addition to the corporate indoctrination outlined above, flight crews, flight attendants and maintenance technicians require more specific training with respect to: a) Procedures for accident, incident reporting; b) Unique hazards facing operational personnel such as:

14-14

1) Airfield, terrain and weather considerations; 2) Route structure, etc.; c) Procedures for hazard reporting; d) Specific accident prevention programmes, such as: 1) Flight Data Analysis (FDA) programme; 2) LOSA programme; and e) Safety Committee(s). Additional safety training may be required for line personnel when: a) The organization experiences significant operational change (such as changes to fleets, type of operations or route structure); b) State regulations or company policies are changed; c) New SOPs are introduced; d) Safety hazards posing unacceptable risks are identified; e) Results of formal safety audits are promulgated; f) Seasonal safety hazards and procedures (winter operations, etc.); and g) Emergency procedures are changed. Training for APAs67 The person selected as the company APA is expected to become familiar with most aspects of the organization, its activities and personnel. These requirements may be met in-house, or at external courses, however, much of the APA’s knowledge will be acquired by self-education. Areas where APAs may require formal training include: a) Familiarization with different fleets, types of operations, routes, etc.; b) Understanding the role of Human Factors in accident causation and prevention; c) Structure and operation of safety management systems; d) Safety analysis skills, in particular for risk management processes;

67

Adapted from OFSH Ch 2

14-15

e) Accident and incident investigation; f) Crisis management and emergency response planning; g) Safety promotion; h) Communications skills: 1) Oral for safety presentations 2) Interviewing; 3) Report writing; and 4) Computer skills such as word-processing, spreadsheets, and data base management. Dependent on the size of the operation, the APA may also require specialized training (or familiarization) for: a) CRM; b) Management of incident reporting systems; c) Flight Data Analysis; and d) LOSA methodology, etc. The APA can learn much about the organization’s activities through contacts with relevant managers. In particular, the APA should enjoy a good working relationship with the Chief Pilot(s), the Chief Operations Officer, the Chief Maintenance Officer and the Airport Manager(s). In larger airlines, the APA will likely maintain a close liaison with the Training Department and line station managers.

————————

14-16

Appendix 1 to Chapter 14 SAMPLE JOB DESCRIPTION ACCIDENT PREVENTION ADVISER68 Overall purpose The Accident Prevention Adviser is responsible for the operation of the Company’s accident prevention programme.

Dimension The position requires a meticulous approach and the ability to cope with changing circumstances and situations with little supervision. The Accident Prevention Adviser acts independently of other managers within the Company. The jobholder will be responsible for providing information and advice to senior management on matters relating to the safe operation of company aircraft. Tact, diplomacy and a high degree of integrity are therefore prerequisite. The job requires flexibility as assignments may be undertaken with little or no notice and outside normal work hours.

Nature and scope The Accident Prevention Adviser must interact with line flight crews, maintenance engineers, cabin crew, senior managers and departmental heads throughout the company. The Accident Prevention Adviser should also foster positive relationships with regulatory authorities and agencies, and service providers outside the company. The main functional points of contact within the company are: a) Flight Operations Management; b) Chief Pilot; c) Flight Crew Fleet Management; d) Flight Training and Standards Management; e) Cabin Crew Management; f) Engineering Management; 68

Adapted from OFSH 2.5

14-17

g) Ground Handling Management; h) Maintenance/Technical Control Management; and i)

Head of Security Services.

Other contacts will be established at a working level as appropriate.

Qualifications There are few individuals who possess all the skills and qualities necessary to fulfill this post. The suggested minimum attributes and qualifications required are: a) A broad aviation/technical knowledge; b) A sound knowledge of commercial operations, in particular flight operations procedures and activities; c) Experience as a flight crew member or engineer; d) The ability for clear expression in writing; e) Good presentation and interpersonal skills; f) Computer literacy; g) The ability to communicate at all levels, both inside and outside the Company; h) Organizational ability; i)

To be capable of working alone (at times under pressure);

j)

Good analytical skills;

k) To exhibit leadership and an authoritative approach; and l)

Be worthy of commanding respect among peers and management officials.

Authority On safety matters, the Accident Prevention Adviser has direct access to the CEO and appropriate management. The Accident Prevention Adviser is authorized to conduct safety audits of any aspect of the operation. The Accident Prevention Adviser has the authority to convene a company inquiry into an incident in accordance with the terms of the Company’s Operations Policy Manual.

14-18

Terms of reference To enable the Accident Prevention Adviser to manage the company’s accident prevention programme, the post-holder must have access to all departments at all levels. Although the primary responsibility of the incumbent is to provide information and advice on safety matters to the CEO, the Accident Prevention Adviser will also be responsible for: a) Maintaining the air safety occurrence reporting database; b) Monitoring corrective actions and flight safety trends; c) Coordinating and fulfilling the regulatory authority’s Mandatory Occurrence Reporting requirements; d) Liaising with the heads of all departments company-wide on safety matters; e) Acting as Chairman of the Company Safety Committee, arranging its meetings and keeping records of such meetings; f) Disseminating safety-related information company-wide; g) Maintaining liaison with manufacturers’ customer flight safety departments, government regulatory bodies and other flight safety organizations worldwide; h) Assisting with the investigation of accidents and conducting and coordinating investigations into incidents; i)

Carrying out safety audits and inspections;

j)

Maintaining familiarity with all aspects of the Company’s activities and its personnel;

k) Planning and controlling the budget of the safety office; l)

Managing or have oversight of the Flight Data Analysis Programme;

m) Publishing the company’s periodic safety magazine; and n) Participation in corporate strategic planning.

___________________

14-19

Chapter 15 INVESTIGATING FOR ACCIDENT PREVENTION

Introduction • State investigations — Accidents — Serious incidents • In-house investigations Scope of Safety Investigations • How much information is enough? Information Sources • Primary sources of information • Secondary sources of information Interviews • Conducting interviews • Caveat regarding witness interviews Investigation Methodology Investigating Human Performance Factors Investigating Human Errors Investigating Procedural Deviations • PEAT • MEDA Safety Recommendations Appendices 1. Interviewing Techniques

15-1

This page intentionally left blank.

15-2

Chapter 15 INVESTIGATING FOR ACCIDENT PREVENTION69

Investigation. A process conducted for the purpose of accident prevention which includes the gathering and analysis of information, the drawing of conclusions, including the determination of causes and, when appropriate, the making of safety recommendations. ICAO Annex 13

INTRODUCTION The accident prevention value of an accident, or a hazard, or incident report is proportional to the quality of the investigative effort. Ideally each reported hazard or occurrence is investigated to the point where all associated risks are identified. In the ICAO Annex 13 context, investigations are associated with accidents and serious incidents. For reportable accidents and serious incidents, the State will normally provide an investigative team. However, for such occurrences, the State investigators will often call upon the special knowledge of the APA and other company staff.70 The APA and those assigned as investigators must be technically competent both in the area under investigation, as well as in the methods of safety investigation. The APA must earn the confidence and respect of staff, and demonstrate the highest levels of integrity while objectively attempting to determine why things occurred and how potential hazards might impact on safety.

State investigations Accidents Accidents provide compelling and incontrovertible evidence of the severity of hazards. Too often it takes the catastrophic and grossly expensive nature of accidents to provide the spur for allocating resources to accident prevention to an extent otherwise unlikely. By definition, accidents result in damage and/or injury. If we concentrate on investigating the results, not the hazards or risks that cause them, we are being reactive rather than proactive and the investigation process from an accident prevention perspective is rather inefficient. The focus of an accident investigation should therefore be directed towards effective preventive action. With the investigation directed away from Athe chase for the guilty party@ and towards effective preventive action, cooperation will be fostered among those involved in the accident, 69

Based on TC TP 13881 Safety Management Systems for Flight Operations and Aircraft Maintenance, p 28. Manual of Aircraft Accident Investigation (Doc 6920) contains much useful information for APAs during formal State investigations. 70

15-3

facilitating the discovery of the underlying causes. The short-term expediency of finding someone to blame is detrimental to the long-term goal of preventing accidents.

Serious Incidents The term Aserious incident@ is used for those incidents which good fortune narrowly prevented from becoming an accident. For example, a near collision with another aircraft, or with the ground, involving large passenger aircraft. Because of the seriousness of such incidents, they should be thoroughly investigated. Some States treat these serious incidents as if they had been accidents. Thus they use an accident investigation team to carry out the investigation, including the publication of a Final Report and the forwarding to ICAO of an ADREP incident data report. This type of full-scale incident investigation has the advantage of providing hazard information to the same standard as that of an accident investigation, without the associated loss of life, aircraft or property.

In-house investigations Most occurrences do not warrant investigations by either the State investigative or regulatory authorities. Many incidents are not even required to be reported to the State. Nevertheless such incidents may be indicative of potentially serious hazards – perhaps systemic problems that will not be revealed unless the occurrence is properly investigated. For every accident or serious incident there may be hundreds of minor occurrences, many of which have the potential to become an accident. It is important that all reported hazards and incidents be reviewed and a decision taken on which ones should be investigated and how deeply. APAs will frequently be involved in investigating these occurrences with the view to accident prevention. For in-house investigations, the APA (or investigating team) may require specialist assistance, depending on the nature of the occurrence being investigated. For example: a) Cabin safety specialists for in-flight turbulence encounters, smoke or fumes in the cabin, galley fire, etc.; b) Air Traffic Services for loss of separation, near collisions, frequency congestion, etc.; c) Maintenance engineers for incidents involving material or system failures, smoke or fire, etc.; and d) Airport management advice for incidents involving FOD, snow and ice control, airfield maintenance, vehicle operations, etc.

SCOPE OF SAFETY INVESTIGATIONS How far should an investigation go into minor incidents and hazard reports? The extent of the investigation should depend on the actual or potential consequences of the occurrence or hazard. Hazard or incident reports that indicate high-risk potential should be investigated in greater depth than those with low potential. The depth and detail of the investigation should be that which is required to clearly identify and validate the underlying hazards. Understanding why something happened requires a broad appreciation of the

15-4

context for the occurrence. To develop this understanding of the unsafe conditions, the investigator should take a systems approach, perhaps drawing on the SHELL or 5M models. In deciding upon the scope of work required, APAs must accept that resources are limited. Thus the effort expended should be proportional to the perceived benefit, in terms of potential for identifying systemic hazards and risks to the organization.

How much information is enough? The investigative process should be comprehensive, attempting to address all the factors that contributed to the situation. Active failures (sometimes called triggering events) take place immediately prior to the event. They have a direct impact on system safety because of the immediacy of their effects. However, they are not usually the root cause of the event. As such, applying corrective actions to these active failures may not address the real cause(s) of the problem. A more detailed analysis is normally required to understand all the factors that may have contributed. How much data should be collected and analysed in order to develop an accurate picture of the contributing factors? How many employees, support personnel, supervisors, etc. should be interviewed? How far back in time should activities be investigated? To what extent should inter-personal relationships be examined? At what point does past behaviour cease to influence current behaviour? To what level of management should the investigation progress? There are no clear answers to such questions. If the information will not help explain why something happened (or happens), then it is not relevant to accident prevention. The questioning process can be carried to extremes. In practice, it is reasonable to stop the investigation at a point where corporate and regulatory authorities no longer have any power to change the underlying situation in the interests of accident prevention. Although the investigation should focus on the factors most likely to have influenced actions, the dividing line between relevance and irrelevance is often blurred. Data that initially may seem to be unrelated to the investigation could later prove to be extremely relevant after relationships between particular elements of the occurrence are better understood.

INFORMATION SOURCES Information relevant to a safety investigation can be acquired from a variety of sources which can be considered as primary and secondary sources of information. Primary sources of information include similarly configured equipment, documentation, recorder tapes, interviews, direct observation of personnel activities, and simulations. Secondary sources include occurrence databases, technical literature and the expertise of professionals or specialists.

Primary sources of information Physical examination of the equipment used during the safety event may yield useful information. This may include examining the aircraft, its components, or the workstations and equipment used by supporting personnel (e.g. air traffic controllers, maintenance and servicing personnel). Documentation spanning a broad spectrum of the operation is available, for example:

15-5

a) Maintenance records and logs; b) Personal records/logbooks; c) Certificates, licenses; d) Company personnel and training records, work schedules, etc.; e) Operator’s manuals and standard operating procedures; f) Training manuals and syllabi; g) Manufacturer’s data and manuals; h) Regulatory authority records; i)

Weather forecasts, records and briefing material; and

j)

Flight planning documents, etc.

Recordings (flight recorders, ATC radar and voice tapes, etc.) may provide useful information for determining the sequence of events. In addition to traditional flight data recordings, maintenance recorders in new generation aircraft are a potential additional source of information. Interviews conducted with individuals directly or indirectly involved in the safety event can provide a principal source of information for any investigation. In the absence of measurable data, interviews may be the only source of information. (Thus, APAs investigating for accident prevention need to be skilled in interviewing techniques.) Direct observation of actions performed by operating or maintenance personnel in their work environment can reveal information about potential unsafe conditions. However, the persons being observed must be aware of the purpose of the observations. Simulations permit reconstruction of an occurrence and can facilitate a better understanding of the sequence of events that led up to the occurrence, and the manner in which personnel responded to the event. Computer simulations can be used to reconstruct events using data from on-board recorders, air traffic control tapes and other physical evidence.

Secondary sources of information Additional information collected from other sources can facilitate the analysis of factual information. The APA and company investigators cannot be experts in every field related to the operational environment. It is important that they realize their limitations. When necessary, they must be willing to consult with other professionals during an investigation. Depending on the nature of the hazard and any required analysis, advice from such disciplines as engineering, medicine, psychology and statistics may be required.

15-6

Some of the most useful sources of supporting information come from accident/incident databases, company hazard and incident reporting systems, confidential reporting programmes, flight data analysis and LOSA programmes, manufacturers’ databases, etc. A word of caution about databases: before using any data, the prudent APA will assess the limitations inherent in the database. Consideration should be given to the source of the data, its intended purpose, the definitions used in recording the data, its currency, etc.

INTERVIEWS Information acquired through interviews can help clarify the context for unsafe acts and conditions. It can be used to confirm, clarify or supplement information learned from other sources. Interviews can help to determine what happened. More importantly, interviews are often the only way to answer the important “why” questions which, in turn, can facilitate appropriate and effective safety recommendations. In preparation for an interview, the interviewer must expect that individuals will perceive and recall things differently. The captain’s recall of an in-flight turbulence incident will likely differ considerably from what the flight attendants reported. The details of a system defect reported by flight crew may differ from those observed by maintenance personnel during a ground check. Supervisors and management may perceive issues differently than line personnel. The interviewer must accept all views as worthy of further exploration. However, even qualified, experienced and well-intentioned witnesses could be mistaken in their recollection of events they have witnessed. In fact, in interviewing a number of persons on the same event, if different perspectives are not offered, it may be grounds to suspect the validity of the information being received.

Conducting interviews The effective interviewer adapts to these differing views, remaining objective and avoiding making an early evaluation of the content of the interview. An interview is a dynamic situation, and the skilled interviewer knows when to continue a line of questioning and when to back off. For best results, interviewers will likely employ a process as follows: a) Careful preparation and planning for the interview; b) Conducting the interview in accordance with a logical, well-planned structure; and c) Assessing the information gathered in the context of all other known information. Appendix 1 to this Chapter provides further guidance for conducting effective interviews for accident prevention.

Caveat regarding witness interviews Sorting through the often-conflicting nature of witness interviews requires caution. Intuitively, an interviewer may weigh the value of an interview dependent on the background and experience of the person being interviewed. However:

15-7

a) Persons judged as “good witnesses” may allow their perceptions to be influenced by their experience (i.e. they see and hear what they would “expect”). Consequently, their description of events may be biased; and b) On the other hand, people who have no knowledge of an occurrence they have witnessed are often able to accurately describe the sequence of events. They may be more objective in their observations. The skilled investigator does not overly rely on a single witness – even the testimony of an expert. Rather, information from as many sources as practical needs to be integrated to form an accurate perception of the situation.

Credibility is difficult to establish, easy to lose, and almost impossible to regain.

INVESTIGATION METHODOLOGY Accident prevention requires a range of activities related to the investigative process. In addition to the traditional field phase of the investigation which is used to identify and validate perceived safety hazards, accident prevention requires competent safety analysis to assess the risks, and effective communications to control the risks. In other words, an integrated approach to safety investigations is required. Some occurrences and hazards originate from material failures, or occur in unique environmental conditions. However, the majority of unsafe conditions are generated through human errors (Liveware). When considering human error, an understanding of the unsafe conditions that may have affected human performance or decision-making is required. These unsafe conditions may be indicative of systemic (or latent) hazards that put the entire aviation system at risk. Consistent with the systems approach to safety, an integrated approach to safety investigations considers all aspects that may have contributed to unsafe behaviour or created unsafe conditions. The logic flow for an integrated process for safety investigations is depicted in Figure 15-1 Integrated Safety Investigation Methodology (ISIM). Such a model can guide the APA or safety investigator from initial hazard or incident notification, through to the communication of safety lessons learned.

15-8

Hazard or Occurrence Notification & Assessment

Assess notification and decide to investigate or not

Data Collection Process

Identify events and underlying factors

Reconstruct logical progression of occurrence events

Sequence of Events

Analyse facts & determine findings re underlying factors and hazards

Integrated Investigation

Estimate risk and determine acceptability for each hazard

Risk Assessment Process

Identify defences that are missing or inadequate

Defence Analysis

Identify and evaluate risk control options

Communicate safety message to stakeholders

Risk Control Analysis

Safety Communication Process

Figure 15-1. Integrated safety investigation model (ISIM)71

71

Adapted from ISIM developed by TSB of Canada

15-9

Effective investigations for accident prevention do not follow a simple step-by-step process that starts at the beginning and proceeds directly through each phase to completion. Rather, it is an iterative process that may require going back and repeating steps as new data is acquired and/or as conclusions are reached. The APA and the investigative team must remain flexible, moving forwards and backwards from data collection through analysis and back again.

INVESTIGATING HUMAN PERFORMANCE FACTORS Investigators have been quite successful in analysing the measurable data as it pertains to human performance, e.g. strength requirements to move a control column, lighting requirements to read a display, ambient temperature and pressure requirements, etc. Unfortunately, the majority of safety deficiencies derive from issues that do not lend themselves to simple measurement and are thus not entirely predictable. As a result, the information available does not always allow an investigator to draw indisputable conclusions. Several factors typically reduce the effectiveness of a human performance analysis.72 These include: a) The lack of normative human performance data to use as a reference against which to judge observed individual behaviour. (FDA and LOSA data are now providing a baseline for better understanding normal day-to-day performance.) b) The lack of a practical methodology for generalizing from the experiences of an individual (crewmember) to an understanding of the probable effects on a large population performing similar duties. c) The lack of a common basis for interpreting human performance data among the many disciplines (e.g. engineering, operations and management) which make up the aviation community. d) The ease with which humans can adapt to different situations, further complicating the determination of what constitutes a breakdown in human performance. The logic necessary to convincingly analyse some of the less tangible human performance phenomena is different from that required for other aspects of an investigation. Deductive methods are relatively easy to present and lead to convincing conclusions. For example, a measured wind shear produced a calculated aircraft performance loss and a conclusion could be reached that the wind shear exceeded the aircraft’s performance capability. Such straight cause/effect relationships cannot be so easily established with some human performance issues such as complacency, fatigue, distraction or judgement. For example, if an investigation revealed that a crewmember made an error leading to an occurrence under particular conditions (such as fatigue, distractions, or complacency), it does not necessarily follow that the error was made because of these preconditions. There will inevitably be some degree of speculation involved in such a conclusion. The viability of such speculative conclusions is only as good as the reasoning process used and the weight of evidence available. Inductive reasoning involves probabilities. Inferences can be drawn on the most probable or most likely explanations of behavioural events. Inductive conclusions can always be challenged and their credibility depends on the weight of evidence supporting them. Accordingly, they must be based upon a consistent and accepted reasoning method. 72

A manager from the Boeing Commercial Aeroplane Company (Fadden), 1984

15-10

Other factors to be considered when investigating human performance issues are: a) How to assess the relevance of certain behaviour or actions deemed "abnormal" or "nonstandard"; b) Sensitivity and privacy considerations vs. accident cause/prevention significance; and c) Avoidance of speculation vs. logical and reasonable explanations. Analysis of the human factors must take into account the accident prevention objective of the investigation. Occurrences are seldom the result of a single cause. Although individual factors when viewed in isolation may seem insignificant, in combination they can result in a sequence of events and conditions that culminate in an accident. The SHEL model provides a systematic approach to examining the constituent elements of the system as well as the interfaces between them. The interactive system suggested by Reason also provides an excellent framework by which investigators can ensure that the analysis addresses all levels of the production system. The analysis must not focus only on the active failures of front line operators, but must include an analysis of the "fallible decisions" at all levels which may have interacted to create the occurrence "window of opportunity".

INVESTIGATING HUMAN ERRORS Understanding the context in which humans err is fundamental to understanding the unsafe conditions that may have affected their behaviour and decision-making. These unsafe conditions may be indicative of systemic risks posing significant accident potential. The Error-Type Decision Tree provides a useful guide for understanding human errors.

15-11

Was the error caused by a miscommunication, misinterpretation, or failure to communicate pertinent information?

Communication Error

NO YES Was the error caused by a lack of knowledge or basic aircraft handling proficiency?

Proficiency Error

NO

Was the error a decision that increased risk and for which there were no written regulations or procedures?

YES

Operational Decision Error

NO

Was the error associated with a crew intention not to follow written regulations or procedures?

YES

Intentional Non-compliance Error

NO

Procedural

Error

Figure 15-2. Error-type decision tree

Typically, incidents and accidents involve several, often inter-linked errors. Each error should be investigated to determine the most serious underlying contributory factors Systematically investigating these human errors requires an integrated approach, as described above. In attempting to identify the underlying factors affecting human behaviour which may have led to an occurrence, the APA (or investigator) might use the following reasoning:

15-12

a) Was the behaviour or decision intentional or unintentional? b) If it was unintentional: 1) Was it a slip, whereby, the person knew what was required but inadvertently did something else (e.g. selected the wrong frequency)? 2) Was it a lapse whereby the person again knew what was required, but forgot (e.g. missed an altitude call-out)? 3) Was the unintentional behaviour the result of: — Unfamiliarity with a new procedure or new work environment? — Lack of skill or currency? — Inattention (due to distraction, fatigue, pre-occupation, complacency, etc.)? c) If it was intentional: 1) Was it a genuine mistake (perhaps due to an inappropriate plan)? 2) Was it a deliberate violation of prescribed operating procedures? 3) Were any mistakes in decision-making due to: — Inadequate knowledge for the conditions? — Ambiguous or inadequate rules governing the procedure? — Biases in decision-making? — Memory problems? 4) Were any deliberate violations due to: — Misunderstanding of correct procedures? — Adaptations of SOPs (i.e. routine application of a non-standard, but accepted, practice)? — Corporate climate of indifference? — Exceptional deviation due to actual conditions (e.g. weather, aircraft serviceability, security issue)? Given the context for the human errors (or violations), the APA or investigator can determine the extent to which a systemic hazard exists. The normal risk assessment process can then be used to determine whether the risk is unacceptable, therefore, warranting safety action.

15-13

INVESTIGATING PROCEDURAL DEVIATIONS Several independent research initiatives have determined that a majority of hull-loss accidents would have been prevented if established procedures had been followed. Since incidents are often only separated from accidents by a fine margin of luck, non-compliance with established procedures is also likely to be causal in most incidents. Simply listing procedural deviations as a contributing factor to an occurrence implies that further exhortations to “be safe” will alter crew performance. However, understanding the causal and contributory context for such deviations from SOPs and accepted safe work practices is fundamental to accident prevention. The reasons behind flight crew non-compliance with procedures are not well understood. They may include ambiguously written, or poorly understood procedures, inadequate training, time pressures, design shortcomings, incompatible work environments, unexpected operational situations, or just plain poor judgement. Fortunately, following an incident the flight crew is available to share their experience, insights and rationale. Ironically, occurrences occasionally arise where deviations from accepted work practices do prevent an accident. These may also warrant investigation to identify the weaknesses in the established procedures. Boeing has developed the following investigation methodologies: a Procedural Event Analysis Tool (PEAT) and a Maintenance Error Decision Aid (MEDA). Both follow an iterative (integrated) approach. The objective of the process is to help the investigator to arrive at valid, effective recommendations aimed at preventing similar types of procedural deviation. PEAT is useful in investigating procedural deviations. It focuses on the key event elements and the underlying cognitive factors that contributed to the event. In contrast, MEDA looks at organizational factors that contribute to human error, such as poor communication, inadequate information and poor lighting. PEAT and MEDA recognize that traditional efforts to investigate errors are often aimed at identifying the employee who made the error. The usual result is that the employee is subjected to a combination of disciplinary action and recurrent training – with little systemic safety benefit. Therefore, both PEAT and MEDA are based on the philosophy that personnel seldom deliberately deviate from prescribed procedures, especially if doing so is a safety risk. PEAT and MEDA are both software-based analytical tools, requiring specialized forms for following the process in a systematic way. Consequently, specific training in the use of PEAT and MEDA is required to ensure their successful application. PEAT73 PEAT is designed to provide airlines with a tool for safety and risk management. It aims at enhancing reliability, consistency and effectiveness of the investigation process. The PEAT form is designed to facilitate the investigation of events involving non-adherence to procedures. The primary focus is to find out why a serious event occurred and if a procedural deviation was involved. 73

For further information on PEAT, see www.boeing.com/commercial/flighttechservices/ftssafety.html.

15-14

The PEAT methodology comprises three elements: Process. PEAT provides a structured process that guides investigators through the identification of key contributing factors and the development of effective recommendations aimed at the elimination of similar errors in future. This includes collecting information about the event, analysing the event for errors, classifying the error and identifying preliminary recommendations. Data Storage. To facilitate analysis, PEAT provides a database for the storage of procedurally related data. Although designed as a structured tool, PEAT also provides the flexibility to analyse narrative information as needed. The database also facilitates tracking progress in addressing issues revealed by PEAT analyses and the identification of emerging trends. Analysis. Using the PEAT tool in a typical analysis of a procedurally related event, a trained investigator will consider the following areas and assess their significance in contributing to flight crew decision errors: a) Flight Phase where the error occurred; b) Equipment factors; 1) Role of automation; 2) Flight deck indications; 3) Aircraft configuration; c) Other stimuli (beyond immediate indications); d) Environmental factors; e) Procedure from which the error resulted: 1) Status of the procedure; 2) Onboard source of the procedure; 3) Procedural factors (e.g. negative transfer, impractical, complexity); 4) Crew interpretation of the relevant procedure; f) Current policies, guidelines/policies aimed at prevention of event; g) Crew factors: 1) Crew interpretation of the relevant procedure; 2) Crew intentions; 3) Crew understanding of situation at the time of the procedural deviation;

15-15

4) Situational awareness factors (e.g. vigilance, attention, etc.); 5) Factors affecting individual performance (e.g. fatigue, workload, etc.); 6) Personal and corporate stressors, management or peer pressures, etc; 7) Crew coordination/communication; and 8) Technical knowledge/skills/experience. MEDA74 Maintenance and inspection errors remain a significant cause of aircraft accidents worldwide. The percentage of commercial aircraft accidents in which maintenance error was a primary cause has risen steeply in recent years. Again, it is accepted that the same type of factors that contribute to accidents can be contributory to incidents and lesser safety events. Consequently, aviation authorities are seeking methods for better managing maintenance errors. MEDA is designed to systematically investigate the causes of maintenance errors that lead to occurrences involving equipment damage or personal injury or to unplanned events (such as a return to the gate, a flight cancellation or delay, a diversion, etc.). MEDA is discussed in greater detail in Chapter 20, Accident Prevention in Aircraft Maintenance.

SAFETY RECOMMENDATIONS When an investigation identifies hazards, or unmitigated risks, safety action is required. The need for action must be communicated by means of safety recommendations to those with the authority to expend the necessary resources. Failure to make appropriate safety recommendations may leave the risk unattended. For those formulating safety recommendations, the following considerations may apply: Action agency. Who can best take the necessary corrective action. Who has the necessary authority and resources to intervene. Ideally, problems should be addressed at the lowest possible level of authority, such as the departmental or company level as opposed to the national or regulatory level. However, if several airlines are exposed to the same unsafe conditions, extending the recommended action may be warranted. State and international authorities, or multinational manufacturers may best be able to initiate the necessary safety action. What vs. how. Safety recommendations should clearly articulate what must be done, not how to do it. The focus is on communicating the nature of the risks requiring control measures. Detailed safety recommendations which spell out exactly how the problem should be fixed, should be avoided. The responsible manager should be in a better position to judge the specifics of the most appropriate action for the current operating conditions. The effectiveness of any recommendation will be measured in terms of the extent to which the risks have been reduced, rather than strict adherence to the wording in the recommendation. General vs. specific wording. Since the purpose of the safety recommendation is to convince others of an unsafe condition putting some or all of the system at risk, specific language should be used in 74

For further information on MEDA, see www.boeing.com/commercial/flighttechservices/ftssafety.html.

15-16

summarizing the scope and consequences of the identified risks. On the other hand, since the recommendation should specify what is to be done (not how to do it), concise wording is preferable. Recipient=s perspective. In recommending safety action, the following considerations pertain from the recipient=s perspective: a) The safety recommendation is addressed to the most appropriate action authority (i.e. they have the jurisdiction and authority to effect the necessary change). b) There are no surprises (i.e. there has been prior dialogue concerning the nature of the assessed risks). c) It articulates what must be done, while leaving the action authority with the latitude to determine how best to meet that objective. Failure to observe these basic considerations may compromise the risk from receiving the necessary safety attention. Formal safety recommendations warrant written communications. This ensures that everyone is “singing from the same song sheet” and provides the necessary baseline for evaluating the effectiveness of implementation. However, it is important to remember that safety recommendations are only effective if they are implemented by the responsible managers.

— — — — — — — —-

15-17

This page intentionally left blank.

15-18

Appendix 1 to Chapter 15 INTERVIEWING TECHNIQUES75 GENERAL Information from interviews can be used to confirm, clarify, or supplement information obtained from other sources. Moreover, in the absence of measurable data, interviews may be the only source of information. The interviewer’s role is to obtain information from the interviewee that is as accurate, complete, and detailed as possible. To accomplish this, the interviewer must: a) Be prepared. b) Have a clear objective. c) Have a good knowledge of the hazard or incident under investigation. d) Be able to adapt to the interviewee=s personal style. e) Be willing to go beyond simple facts. Interviews, particularly those involving human performance factors, must go beyond the AWhat and When@ of the occurrence; they must also attempt to find out AHow and Why” it occurred. To facilitate the interview, the interviewer must: a) Keep the number of people participating in the interview to a minimum. b) Include anyone (such as a technical expert, supervisor or union representative) that the interviewee has requested to be present. c) Brief others on their expected behaviour during the interview. d) Not tolerate disruptions of the interview by others.

PREPARING FOR THE INTERVIEW Personal preparations The success of the interview will closely relate to personal preparation. Tailor the preparations to the interview. For example, depending on the nature of the reported hazard or incident, review the following: a) The facts relating to any safety event.

75

Adapted from Transportation Safety Board of Canada (TSB) Manual of Investigations

15-19

b) Recorded information, if any. c) The aircraft systems, if applicable. d) Any operational peculiarities in procedures. e) The crew=s personal records. f) Communication, navigation, and approach facilities, as applicable; etc. Other preparatory factors include: a) If possible, visit the occurrence site. b) Define the general objectives of the interview. c) Prepare a set of appropriate questions to address areas of concern. d) Assess the audience and dress accordingly. e) Arrive for the interview with any required equipment or reference material. f) Consider requesting expert assistance for interviews of a highly technical nature.

Timing Timing is critical. In follow-up to an incident or safety event, interviews should be conducted as soon as practicable to avoid: a) Loss of perishable information from fading memory. b) Interpretation and rationalization of events. c) Contamination caused by exposure to others’ recollections or interpretations of events. If an immediate interview is impractical, request a written statement to ensure information is recorded while fresh in the interviewee’s mind.

Location If possible, select a location which is: a) Quiet and comfortable. b) Free from interruptions. c) For a witness, where he was when he witnessed the event, if at all possible.

15-20

The interview The opening of the interview should reassure the interviewee about: a) The purpose of the interview (investigation). b) Interviewee’s rights. c) Your role as the interviewer. d) The procedures to be followed. Establish a rapport with the witness at the outset. Consider the following: a) Be polite. b) Behave in a natural manner; do not make the interview seem artificial. c) Keep interruptions to a minimum. d) Strive for an atmosphere of friendly conversation. e) Intervene only enough to steer the conversation in the desired direction. f) Display a sincere interest. The success of the interview will depend on both the timing and the structure of the questions. Begin the interview with a AfreeBrecall@ question, letting the individual talk about what he or she knows of the occurrence or subject matter. A free recall allows the witness: a) To ease into the interview in a more relaxed manner. b) To feel that what he or she has to say is significant. c) More importantly, it is a source of information which is uncontaminated by the interviewer. Sequence the questions from: a) Easier to harder. b) General to specific. As the interview progresses, use a mixture of other types of questions: a) OpenBended or Atrailing-off@ questions evoke rapid and accurate descriptions of the events, and lead to more participation by the interviewee. (For example: “You said earlier that your training was…?”) b) Specific questions are necessary to obtain detailed information and may also prompt the person to recollect further details.

15-21

c) Closed questions produce Ayes@ or Ano@ answers (providing little insight beyond the response). d) Indirect questions might be useful in delicate situations. (For example, “You mentioned that the first officer was uneasy about flying that approach. Why?”) A good closing to the interview includes: a) Summary of the key points. b) An opportunity for the interviewee to expand on any points previously covered, or to add further points. c) Reassurance to the interviewee and thanks for cooperation. d) Determination of availability for further interviews (if required).

QUESTIONNING TECHNIQUES Following are some of the common traps interviewers may encounter. a) Avoid questions with the definite article unless the object in question has already been mentioned by the witness. 1) Use: ADid you see A broken strut?@ 2) Not: ADid you see THE broken strut?@ b) When asking a question, avoid leading questions, i.e. any question that contains the answer. Instead, use neutral sentences without adjectives or figurative verbs. 1) Use: AWhich way was the aircraft travelling?@ 2) Not: AWas the aircraft travelling west?@ c) A question which mentions some object (whether the object existed or not) causes a tendency for a witness to assert that he/she saw the object. Design your questions so that they do not mention objects before the interviewee mentions them. d) If you have to ask questions of a personal nature, it is even more important to ask indirect questions. For example: 1) Use: “Was there anything upsetting you on the day of the occurrence?” 2) Not: “How did your marital situation affect you that day?”

ADDITIONAL GUIDANCE Following are some additional hints for effective interviews:

15-22

a) The interview is a dynamic process which requires continuing adaptation to the situation and to the interviewee. b) People approach any occurrence or situation from different perspectives. c) Remain objective and avoid making evaluations early in the interview, concentrate on the questions to be asked. d) Be aware of possible biases when assessing what was said during the interview. e) Do not allow the interviewee=s personality to influence interpretation of the interview. f) Do not accept any information gained in an interview at face value. Use it to confirm, clarify, or supplement information from other sources. In some circumstances there may be many witnesses to be interviewed. The resultant (often conflicting) information must be summarized, sorted and compiled in a useful format. For example: a) Prepare a summary of each interview. b) If appropriate, prepare a dataBmatrix. c) Summarize each interview by consolidating information under meaningful headings. d) Write an overall description from each set of summaries. e) Do not draw conclusions from evidence which is too inconsistent to support them.

TEN COMMANDMENTS OF GOOD LISTENING Good interviews require good listening skills. 1) Stop talking: You cannot listen if you are talking. AGive every person thine ear, but few thy voice@ (Polonius). 2) Put the talker at ease: Help witnesses feel that they are free to talk. 3) Show them you want to listen: Look and act interested. (For example: Do not review documents while they talk.) 4) Remove distractions: Do not doodle, tap, or shuffle papers. 5) Empathize with them: Try to put yourself in their place so that you can see their points of view. 6) Be patient: Allow plenty of time. Do not interrupt. 7) Hold your temper: An angry person gets the wrong meaning from words.

15-23

8) Do not criticize or embarrass: Criticism puts respondents on the defensive. Do not embarrass witnesses by commenting on their lack of technical knowledge, education, choice of words. They may Aclam up@ or get angry. Do not argue: even if you win, you lose. 9) Ask questions: Asking questions encourage the respondent and shows that you are listening. It also helps to develop points further. 10) Stop talking: This is first and last because all the other commandments depend on it. You cannot do a good listening job while you are talking.

RECORDING THE INTERVIEW Each interview should be documented for future reference. Records may consist of transcripts, interview summaries, notes and tape-recordings (if taped). For major (State) investigations, complete transcripts of interview testimony may be required. A qualified stenographer should be hired specifically to provide a verbatim record of the interview which must not be edited. For formal interviews, both the interviewer and the interviewee may agree that the interview be taped. Recorded interviews may be summarized in whole or in part depending on the degree of detail required to support investigative findings. Consider the following factors in summarizing a taped interview: a) the meaning is not changed; b) subjective interpretations are not made; c) the interviewee’s attitude is not obscured; d) significant testimony is not omitted; and e) the nature of the summary is indicated in the record. Those portions of the recorded interviews not relevant to occurrence may be summarized. The majority of interviews will not be recorded. However, a summary of the interview should still be prepared, again ensuring that: a) subjective interpretations are not made; b) the interviewee’s attitude is not obscured; and c) significant testimony is not omitted. Information relevant to the hazard or situation under investigation derived from informal interviews or conversations may be summarized in concise notes.

__________________

15-24

Chapter 16 EMERGENCY RESPONSE PLANNING

Introduction • ICAO requirements Plan Contents • Governing policies • Organization • Notifications • Initial response • Additional assistance • Crisis Management Centre (CMC) • Records • Accident site • News media • Formal investigations • Family assistance • Post critical incident stress counseling • Post occurrence review Aircraft Operator’s Responsibilities Checklists Training and Exercises Involvement of the APA

16-1

This page intentionally left blank.

16-2

Chapter 16 EMERGENCY RESPONSE PLANNING INTRODUCTION Perhaps because aviation accidents are rare events, few organizations are prepared when one occurs. Many organizations (airports and airlines) do not have effective plans in place to manage events during an emergency or crisis. Whether an airline survives the aftermath of an accident can depend on how well it handles the first few hours and days following a major accident. An emergency response plan outlines in writing what should be done after an accident occurs, and who is responsible for each action. At first glance, the initial response to an emergency may appear to have little to do with accident prevention. However, emergency response provides an opportunity to learn as well as to apply safety lessons aimed at the reduction of damage or injury. When there is an accident, management will often turn to the Accident Prevention Adviser (APA) for advice. The APA’s broad knowledge of operations and maintenance, his skill set and network of contacts are an invaluable resource to the organization in the event of any emergency. As such, the APA should play a major role in any emergency response. Successful response to an emergency begins with effective planning for a range of possible events. An Emergency Response Plan (ERP) provides the basis for a systematic approach to managing the organization’s affairs in the aftermath of a significant unplanned event – in the worst case, a major accident. The APA can undoubtedly contribute to the planning process in developing an ERP. To be effective, an ERP should be: a) Be relevant and useful for the people who are likely to be on duty at the time of an accident; b) Include checklists and quick reference contact details of relevant personnel; c) Be regularly tested through exercises; d) Be updated when details change.

ICAO requirements The ICAO Annex 14 — Aerodromes states that, before operations commence at an airport, an emergency response plan should be in place to deal with an aircraft accident occurring on, or in the vicinity of the airport. The ICAO Manual entitled Preparation of an Operations Manual (Doc 9376) states that the operations manual of a company should give instructions and guidance on the duties and obligations of personnel following an accident. It should include guidance on the establishment and operation of a central accident/emergency response centre – the focal point for crisis management. In addition to guidance for accidents involving company aircraft, guidance should also be provided for accidents involving aircraft for which it is the handling agent (for example, through code-sharing agreements or contracted services).

16-3

Larger companies may choose to consolidate all this emergency planning information in a separate volume of their operations manual. The ICAO Airport Services Manual, Part 7 — Airport Emergency Planning (Doc 9137) gives guidance to both airport authorities and aircraft operators on preplanning for emergencies, as well as on coordination between the different airport agencies, including the operator. Doc 9137 states that the purpose of an airport emergency response plan is to ensure that there is: a) Orderly and efficient transition from normal to emergency operations; b) Delegation of airport emergency authority; c) Assignment of emergency responsibilities; d) Authorization by key personnel for actions contained in the plan; e) Coordination of efforts to cope with the emergency; and f) Safe continuation of aircraft operations or return to normal operations as soon as possible.

PLAN CONTENTS An Emergency Response Plan (ERP) would normally be documented in the format of a manual. It should set out the responsibilities and required roles and actions for the various agencies and personnel involved in dealing with emergencies. An ERP should take account of such considerations as:

Governing policies The ERP should provide direction for responding to emergencies, such as governing laws and regulations for accident investigations, agreements with local authorities, company policies and priorities, etc.

Organization The ERP should outline management’s intentions with respect to the responding organizations by: a) Designating who will lead and who will be assigned to the responding teams; b) Defining the roles and responsibilities for personnel assigned to the response teams; c) Clarifying the reporting lines of authority; d) Setting up of a Crisis Management Centre (CMC); e) Establishing procedures for receiving a large number of requests for information, especially during the first few days after a major accident; f) Designating the corporate spokesperson for dealing with the media;

16-4

g) Defining what resources will be available, including financial authorities for immediate activities; h) Designating the company representative to any formal investigations undertaken by State officials; and i)

Defining a call-out plan for key personnel, etc.

An organization or flow chart could be used to show organizational functions and communication relationships.

Notifications The plan should specify who in the organization should be notified of an emergency, who will make external notifications and by what means. The notification needs of the following should be considered: a) Management (chief pilot, chief maintenance engineer and senior management team, etc.); b) State authorities (Search and Rescue, regulatory authority, accident investigation board, etc.); c) Local emergency response services (airport authorities, fire fighters, police, ambulances, medical agencies, etc); d) Relatives of victims — a sensitive issue, in many States handled by the police with information provided by the operator; e) Company personnel; f) Media; and g) Legal, accounting, insurers, etc.

Initial response Depending on the circumstances, an initial response team may be dispatched to the accident site to augment local resources and oversee the operator’s interests. Factors to be considered for such a team include: a) Who would lead the emergency response team? b) Who would be included on the initial response team? c) Who would speak for the company at the accident site? d) What would be required by way of special equipment, clothing, documentation, transportation, accommodation, etc.?

16-5

Additional assistance Company employees with appropriate training and experience can provide useful support during the preparation, exercising and updating of a company’s ERP. Their expertise may be useful in planning and executing such tasks as: a) Acting as passengers in crash exercises; b) Handling of survivors; c) Assisting with interviewing passenger witnesses; and d) Dealing with next of kin, etc.

Crisis Management Centre (CMC) A CMC should be established at the operator’s headquarters once the activation criteria have been met. In addition, a command post (CP) may be established at or near the accident site. The ERP should address how the following requirements for the CMC and CP are to be met: a) Staffing (perhaps for 24 hours a day 7 days per week during the initial response period); b) Communications equipment (telephone, fax, Internet, etc.); c) Documentation requirements, maintenance of emergency activities logs; d) Impounding related company records; e) Office furnishings and supplies; f) Reference documents (such as emergency response checklists and procedures, company manuals, airport emergency plans, telephone lists, etc.); and g) Collection of latest information for daily updates, internal distribution, etc. The services of a crisis centre may be contracted from another airline or other specialist organization to look after the airline’s interests in a crisis away from home base. Company personnel would normally supplement such a contracted centre as soon as possible.

Records In addition to the organization’s need to maintain logs of events and activities post the occurrence, the organization will also be required to provide information for any State investigation team. The ERP should provide the following types of information to investigators: a) All records relevant to the aircraft, the flight crew and the operation; b) Lists of points of contact and any personnel associated with the occurrence;

16-6

c) Notes of any interviews (and statements) with anyone associated with the event; and d) Any photographic or other evidence, etc.

Accident site After a major accident, representatives from many jurisdictions have legitimate reasons for accessing the site, for example, police, fire fighters, medics, airport authorities, coroners (medical examining officers) to deal with fatalities, State accident investigators, relief agencies such as the Red Cross and even the media. Although coordination of the activities of these stakeholders is the responsibility of the State’s police and/or investigating authority, the aircraft operator should clarify the following aspects of activity at the accident site: a) Nominating a senior company representative at the accident site: 1) If at home base; 2) If away from home base; 3) If offshore or in a foreign State. b) Management of surviving passengers; c) Needs of relatives of victims; d) Security of wreckage; e) Handling of human remains and personal property of the deceased; f) Preservation of evidence; g) Provision of assistance (as required) to the investigation authorities; and h) Removal and disposal of wreckage; etc.

News media How the company responds to the media may affect how well the company recovers from the event. Clear direction is required. For example: a) What information is protected by statute (FDR data, CVR and ATC recordings, witness statements etc.); b) Who may speak on behalf of the company at head office and at the accident site (Public Relations Manager, Chief Executive Officer or other senior executive, manager, owner); c) Direction regarding a prepared statement for immediate response to media queries; d) What information may be released (What should be avoided);

16-7

e) The timing and content of the company’s initial statement; and f) Provisions for regular updates to the media.

Formal investigations Guidance for company personnel dealing with State accident investigators and police should be provided.

Family assistance The EPR should also include guidance on the organization’s approach to assisting the families of accident victims (crew and passengers). This guidance may include such things as: a) State requirements for the provision of family assistance services; b) Travel and accommodation arrangements to visit the accident location and survivors; c) Programme coordinator and point(s) of contact for each family; d) Provision of up-to-date information; e) Grief counselling, etc.; f) Immediate financial assistance to victims and their families; and g) Memorial services, etc. Some States define the types of assistance to be provided by an airline operating within and into the country.

Post critical incident stress counselling For personnel working in stressful situations, the ERP may include guidance, specifying duty limits and providing for post incident stress counselling.

Post occurrence review Direction should be provided to ensure that, following the emergency, key personnel carry out a full debrief and record all significant lessons learned which may result in amendments to the ERP and associated checklists.

16-8

AIRCRAFT OPERATOR’S RESPONSIBILITIES76 The aircraft operator’s emergency response plan should be coordinated with the airport emergency plan so that the operator’s personnel know which responsibilities the airport will assume and what response is required by the operator. As part of their emergency response planning, aircraft operators in conjunction with the airport operator are expected to77: a) Provide training to prepare personnel for emergencies; b) Make arrangements to handle incoming telephone queries concerning the emergency; c) Designate a suitable holding area for uninjured persons (meeters and greeters); d) Provide a description of duties for company personnel (e.g. person in command, receptionists for receiving passengers in holding areas); e) Gather essential passenger information and coordinating fulfillment of their needs; f) Develop arrangements with other operators and agencies for the provision of mutual support during the emergency; g) Prepare and maintain an emergency kit containing: 1) Necessary administrative supplies (forms, paper, name tags, computers, etc.); and 2) Critical telephone numbers (doctors, local hotels, linguists, caterers, airline transport companies, etc.). In the event of an aircraft accident on or near the airport, operators will be expected to take such actions as: a) Report to airport command post to coordinate the aircraft operator’s activities; b) Assist in the location and recovery of any flight recorders; c) Assist investigators with the identification of aircraft components and ensure that hazardous components are made safe; d) Provide information regarding passengers and flight crew, and the existence of any dangerous goods on board; e) Transport uninjured persons to the designated holding area; f) Make arrangements for any uninjured persons who may intend to continue their journey, or who need accommodations or other assistance; g) Release information to the media in coordination with the airport public information officer and police; and 76 77

See Chapter 19, Accident Prevention at Airports, for further discussion of airport emergency response planning. Airport Services Manual, Part 7 — Airport Emergency Planning (Doc 9137) Op. Cit.

16-9

h) Remove the aircraft (and/or wreckage) upon the authorization of the investigation authority. While this paragraph is oriented towards the airline operator, some of the concepts would also apply to emergency planning by aerodrome operators and air traffic service providers.

CHECKLISTS Everyone involved in the initial response to a major aircraft accident will be suffering from some degree of shock. Therefore, the emergency response process lends itself to the use of checklists. These checklists can form an integral part of the company’s Operations or Emergency Response Manuals. To be effective, checklists must be regularly: a) Reviewed and updated (for example, currency of callout lists, contact details, company documentation for the CMC, etc.); and b) Tested through realistic exercises.

TRAINING AND EXERCISES An emergency response plan is a paper indication of intent. Hopefully, much of an ERP will never be tested under actual conditions. Training is required to ensure that these intentions are backed by operational capabilities. Since training has a short “shelf-life”, regular drills and exercises are advisable. Some portions of the ERP, such as the callout and communications plan can be tested by “desktop” exercises. Other aspects, such as “on-site” activities involving other agencies, need to be exercised at regular intervals. Such exercises have the advantage of demonstrating deficiencies in the plan, which can be rectified before an actual emergency.

INVOLVEMENT OF THE APA Following are some of the tasks that may be assigned to the APA in Emergency Response Planning: a) Advise senior management of the need for an ERP, on appropriate contents, and on training and exercising of the plan; b) Assist operational management in planning their portions of the ERP; c) Liaise with the Airport Manager(s) to coordinate emergency response requirements; d) Liaise with companies or organizations that may provide emergency response service; e) Ensure any necessary training has been provided; f) Ensure coordination of ERP with that of other airline partners; and g) Ensure the plan is tested regularly and is updated as required. __________________

16-10

Chapter 17 ACCIDENT PREVENTION IN THE CABIN

Cabin Safety — General • ICAO requirements • Operational Flight Safety Handbook (OFSH) — Cabin Safety Compendium Managing Cabin Safety • Commitment • Positive safety culture • SOPs, checklists and briefings • Incident reporting system • Training for accident prevention • Safety oversight Human Factors Affecting Cabin Safety Appendix 1. Safety reporting for flight attendants

17-1

This page intentionally left blank.

17-2

Chapter 17 ACCIDENT PREVENTION IN THE CABIN CABIN SAFETY — GENERAL Cabin safety constitutes an important element of the safety management programme of many airlines. Typically, a cabin safety programme is integrated into the airline’s overall accident prevention programme. The cabin environment presents a range of opportunities for compromising safety, possibly exposing cabin staff and passengers to such safety risks as: a) In-flight turbulence; b) Smoke or fire in the cabin; c) Decompression; d) Emergency landings; e) Emergency evacuations; and f) Unruly passengers. Cabin safety may be seen as the activity of reducing fatalities and injuries to passengers and crew. By reducing or eliminating hazards with the potential for creating injuries, cabin safety provides for a safer environment for persons in the cabin. The cabin crew are usually the only company representatives that passengers see while in the aircraft. From the passengers’ perspective, the cabin crew are there to provide in-flight service. From the perspective of senior management, the cabin crew may have more to do with creating a favourable corporate image. From a regulatory and operational perspective, flight attendants are on-board to manage adverse situations that may develop in the aircraft cabin and to provide assistance to passengers during an emergency, in order to ensure cabin safety. Following a major aviation accident, investigative attention will likely focus initially on flight operations. As guided by the evidence, the investigation may then expand to include other issues. The triggering event for an accident rarely begins in the passenger compartment. However, improper response by cabin crew members to events in the cabin may have more serious consequences. For example: a) Incorrect loading of passengers (e.g. weight and balance considerations); b) Failure to properly secure the cabin and galleys for take-off and landing and in turbulence; c) Delayed reaction to warnings (e.g. of in-flight turbulence); d) Inappropriate response to events in the cabin (e.g. electrical short-circuits, smoke, fumes, or an oven fire);

17-3

e) Failure to report significant observations (such as fluid leaks, or wings contaminated by snow or ice) to the flight crew; and f) Unruly passengers. Risks encountered in the cabin can be minimized through an effective accident prevention programme with emphasis on the early identification of hazards and effective cockpit/cabin communications. Hazards may arise from issues such as, aircraft design, cabin configuration, operational practices, cabin service procedures, aircraft maintenance, crew training, etc. With much of the flight attendants’ routine activities focused on cabin service, extra effort is required to ensure that cabin service is not provided at the expense of fulfilling their primary responsibility for passenger safety. It is essential that training and operating procedures for cabin crew address the full range of issues that could have safety consequences.

ICAO requirements Although ICAO does not require that cabin crew be licensed, Chapter 12 of Annex 6 — Operation of Aircraft specifies requirements for cabin crew with respect to: a) Assignment of emergency duties; b) Role during emergency evacuations; c) Use of emergency equipment; d) Flight- and duty-time limits; and e) Training. Operators are required to establish and maintain an approved training programme (including recurrent training) to be completed by all persons before being assigned as a cabin crew member. This training is aimed at ensuring the competence of cabin crew to perform in emergency situations. Preparation of an Operations Manual (Doc 9376) provides further guidance for cabin crew training including: a) Joint training with flight crews in handling of emergencies; and b) Training in assisting flight crews (of two pilot crews) in the event of flight crew incapacitation. Human Factors Guidelines for Safety Audits Manual (Doc 9806) also provides guidance for training about human performance relating to passenger cabin safety duties including flight crew – cabin crew coordination.

17-4

Operator’s Flight Safety Handbook (OFSH) — Cabin Safety Compendium Initiating a cabin safety programme poses a significant challenge. Several major operators and key industry representatives recognized that few airlines had a documented systematic approach to safety management. The OFSH was developed in response to this need. The Cabin Safety Compendium to the OFSH extends safety management programmes to include cabin safety. The Compendium documents proven safety practices built on worldwide experience. In addition to outlining routine and emergency safety procedures, it includes several appendices containing reference material, examples of checklists, minimum equipment lists, etc.

MANAGING CABIN SAFETY Chapter 13, Establishing an Accident Prevention Programme, outlined ten basic steps for establishing an accident prevention programme. These same ten steps offer sound guidance for integrating an accident prevention programme for cabin operations into the organization’s overall safety management system. The “Confirmation Checklists” also have application for cabin safety. It is suggested that the reader review the ten steps prior to initiating a cabin safety programme. Building on the ten step framework, the following additional considerations apply for the effective management of cabin safety.

Commitment The provision of cabin service may be viewed as a marketing or customer service function; however, cabin safety is clearly an operational function. Corporate policy should reflect this, and management needs to demonstrate its commitment to cabin safety with more than words. Common indicators of management’s commitment to cabin safety include: a) Allocation of sufficient resources (adequate staffing of cabin crew positions, initial and recurrent training, training facilities, etc.); b) Clearly defined responsibilities, including the setting, monitoring and enforcing of practical Standard Operating Procedures (SOPs) for safety; and c) Fostering of a positive safety culture, etc.

Positive safety culture Chapter 3, Safety Basics, emphasized the importance of safety culture to the effectiveness of an organization’s accident prevention programme. Creating a positive safety culture for cabin crews begins with departmental organization. If, as in many airlines, the flight attendants receive their principal direction from marketing, rather than from the flight operations department, the focus of flight attendants will probably not be on cabin safety. Other considerations for the promotion of a positive safety culture include: a) The relationship between flight crew and cabin crew, for example:

17-5

1) Spirit of cooperation, marked by mutual respect and understanding; 2) Effective communications between flight crew and cabin crew78; 3) Regular review of SOPs to ensure compatibility between flight deck and cabin procedures; 4) Joint pre-flight briefings for flight and cabin crews; and 5) Joint debriefings following safety-related occurrences, etc. b) Cabin crew participation in safety management programmes: 1) Involvement of APA in cabin safety issues; 2) Avenues for offering cabin safety expertise and advice (safety committee meetings, etc); 3) Participation in developing policies, objectives and SOPs affecting cabin safety; and 4) Participation in Safety Incident Reporting System, etc.

SOPs, checklists and briefings As in flight-deck operations, cabin safety requires strict adherence to well thought-out and practical SOPs, including the use of checklists and briefings of cabin crew. Procedures include, but are not limited to the following: passenger boarding; seat assignment; stowage of carry-on baggage; emergency exit accessibility and availability; passenger safety briefing; service equipment storage and use; emergency medical equipment storage and use (oxygen, defibrillator, first aid kit, etc.); handling of medical emergencies; non-medical emergency equipment storage and use (fire extinguishers, protective breathing equipment, etc.); in-flight emergency procedures (smoke, fire, etc.); cabin crew announcements; turbulence procedures (including securing the cabin); handling unruly passengers; emergency evacuations; and routine deplaning. Procedures for Air Navigation Services — Aircraft Operations (PANS-OPS, Doc 8168) includes guidance material on SOPs, checklists and crew briefings. The OFSH Cabin Safety Compendium also includes extensive guidance for establishing safe procedures for both normal and emergency operations. Incident reporting system79 Cabin crew must be able to report hazards, incidents and safety concerns as they become aware of them without fear of embarrassment, incrimination or disciplinary action. Flight attendants, their supervisors and the APA should have no doubts about: a) The types of hazards that should be reported; 78

As a result of recent security measures requiring the flight deck door to be locked during flight, extra effort is required to maintain effective on-board communications between the flight and cabin crew. 79 Chapter 6, Incident Reporting Systems, provides guidance on the set-up and use of incident reporting systems. The OFSH Cabin Safety Compendium includes further guidance for hazard reporting and tracking programmes

17-6

b) The appropriate reporting mechanisms; c) Their job security (following the reporting of a safety concern); and d) Any safety actions taken to follow-up on identified hazards. A summary of considerations for a cabin safety Incident Reporting System is included at Appendix 1 to this Chapter.

Training for accident prevention Safety training for cabin crew is largely aimed at accident prevention and emergency evacuation. Cabin crew should continue to learn about safety matters throughout their career through such training as: a) Initial indoctrination covering basic theory of flight, meteorology, physiology of flight, psychology of passenger behaviour, aviation terminology, etc. b) Hands-on training (if practicable using cabin simulators for fire, smoke and evacuation drills); c) In-flight supervision (on-job-training); d) Annual recurrent training and re-qualification; e) Knowledge and skills in Crew Resource Management, including coordinating activities with the flight crew; and f) Joint training exercises with flight crew to practice drills and procedures used in-flight and in emergency evacuations, etc. In an emergency, the expertise of the cabin crew will be required with little or no warning. Thus, effective safety training for cabin crew requires practice to maintain the sharpness necessary in an emergency.

Safety oversight Maintaining the required level of operational readiness of cabin crew for emergency situations calls for an effective programme of safety oversight. Safety inspections, safety surveys of cabin crew, and regular safety audits are tools to ensure that requisite safety standards are being maintained. Once an operator is certificated, safety oversight for cabin safety may be achieved through ongoing programme of: a) Aircraft inspections (e.g. emergency exits, emergency equipment, galleys); b) Pre-flight (ramp) inspections; c) Inflight cabin inspections (e.g. passenger briefings and demonstrations, crew briefings and use of checklists, crew communications, discipline, and situational awareness); d) Training inspections (e.g. facilities, quality of instruction, records); and

17-7

e) Base inspections (e.g. crew scheduling, dispatch, safety incident reporting and response), etc. A company’s internal safety audit programme should include the cabin crew department. The audit process should include a review of all cabin operations as well as an audit of cabin safety procedures, training, cabin crew member’s operating manual, etc. HUMAN FACTORS AFFECTING CABIN SAFETY80 The work environment and working conditions for cabin crew are influenced by a diverse set of human factors. Some of the more common factors to consider in developing a cabin safety programme include: Crew Resource Management (CRM). With ever-larger cabin crews, the cabin crew must work together as a team. CRM training for cabin crew could include: a) Communications and interpersonal skills. Hesitancy to communicate important data to other team members could jeopardize a flight. Polite assertiveness is required for effective teamwork; b) Situational awareness. Maintaining an accurate perception of evolving events requires questioning, cross-checking, refinement and updating of perception; c) Problem solving, decision-making skills and judgement may be critical in the event of an inflight emergency or in a situation requiring emergency evacuation or ditching; and d) Leadership/followership skills. While in charge flight attendants require well-developed leadership skills, individual cabin crew members must respect command authority unquestioningly during an emergency. Fatigue. Circadian disrhythmia (i.e. jet-lag) and other disturbances to normal sleep patterns are a part of the job. Yet, fatigue can seriously compromise cabin crew response in an emergency. Maximum alertness is required during the approach and landing phase, often at the end of a long duty period. Personality factors. Cabin crew members require skill in handling diverse personality types. In addition, cultural diversity can influence outcomes in an emergency, not only among the passengers, but also in culturally mixed crews. Workload and stress. The pace of cabin duties varies widely, especially during long-haul operations. Learning to cope with the stress of intense workloads and boredom are fundamental to maintaining situational awareness and the mental acuity required in an emergency. Competence, a function of experience and currency, is vital to maximizing effectiveness. Multipletype currencies resulting in transferring from one aircraft type to another may compromise effective emergency response due to difficult and possibly inappropriate habit transfer.

80 For further understanding of Human Factors relevant to cabin safety programmes see Human Factors Training Manual (Doc 9683) and Human Factors Guidelines for Safety Audits Manual (Doc 9806).

17-8

Equipment design. During safety audits, attention should be paid to equipment design factors that may compromise safe performance of duties by cabin crew (strength requirements, reach, userfriendliness, etc.).

— — — — — — — —-

17-9

This page intentionally left blank.

17-10

Appendix 1 to Chapter 17 SAFETY REPORTING FOR FLIGHT ATTENDANTS81 The Incident Reporting System of the airline should include provision for the reporting of safety concerns by flight attendants. If necessary, these reports may be treated in confidence. The following issues pertain to reporting for flight attendants: Who can report? All cabin crew qualified for flight operations. Who reviews safety concerns? A trusted and competent person (a trusted agent) who is mutually acceptable to cabin crew and management and who records and considers each report in confidence. How to report? a) Any of the following methods may be used, with or without a structured pre-printed format: b) Normal mail delivery to a dedicated (secure) address for the trusted agent; c) E-mail to the same trusted agent (for those less concerned about protection of identity); and d) Telephone or Fax to the trusted agent. What to report? Any safety-related incidents or events involving: a) Yourself; b) Other company staff or service providers; and c) Your organization or associated organizations (such as contractors).

Safety-related incidents or events may include: a) Errors b) Shortcomings in individual performance;

81

Adapted from CHIRP

17-11

c) Health or safety matters affecting operational procedures; d) Regulatory anomalies or deviations; and e) Any other unsafe aspects. What NOT to report? a) Incidents or events with no safety content; b) Issues involving conflicts of personalities provided they have no impact on the ability to ensure cabin safety; and c) Industrial relations and/or terms and conditions of employment problems. When to report? a) When others may benefit from an important "Lesson Learned"; b) When other reporting procedures are not appropriate or are not available; c) When you are concerned to protect your identity; (Note that anonymous reports should not be accepted); and d) When company/regulatory reporting procedures have been exhausted without the issue having been addressed.

__________________

17-12

Chapter 18 ACCIDENT PREVENTION IN AIR TRAFFIC SERVICES (ATS)

ATS Safety — General • ICAO requirements Managing Accident Prevention • Effective direction • Safety organization • Risk management • Incident reporting systems • Emergency response • Safety investigations • Safety oversight • Managing change Human Factors in ATS

18-1

This page intentionally left blank.

18-2

Chapter 18 ACCIDENT PREVENTION IN AIR TRAFFIC SERVICES (ATS) ATS SAFETY — GENERAL Although aviation accidents caused by shortcomings in air traffic services are rare, their consequences are potentially disastrous. Safety in ATS presupposes an effective safety management system. Today’s ATS systems have multi-layered defences through such things as: a) Rigid selection criteria and training for controllers; b) Clearly defined performance standards, such as separation criteria; c) Strict adherence to proven standard operating procedures (SOPs); d) Significant international cooperation; e) Utilization of technological advances; and f) Continuing system of evaluation monitoring and improvement. Keeping aircraft safely separated while expediting the flow of traffic in a highly dynamic situation presents unique challenges. Increasingly, controller workload, traffic density and complexity continue to pose significant risks to aviation. The frequency of air proximity and near mid-air collision reports, runway incursions, technical losses of required separation, etc. are indicative of the continuing accident potential in the provision of ATS. The provision of ATS is being further challenged by organizational change. Although State authorities have traditionally provided air traffic services, in a growing number of States, service delivery is being corporatized. Other States are joining regional consortia, such as EUROCONTROL for the delivery of services. From a regulatory perspective, safety oversight for aerodrome and air traffic service (ATS) units has traditionally been conducted through a prescriptive process where detailed requirements were published and compliance was confirmed through inspection. This approach encourages a safety culture of compliance, with little thought being given to proactive risk management. In view of increasing volumes of air traffic and a flat accident rate, efforts to improve safety through the implementation of safety management systems are increasing, including safety management systems for aerodromes and ATS units.

ICAO requirements Annex 11 — Air Traffic Services requires that States implement safety management programmes to ensure that safety is maintained in the provision of ATS within airspaces and at aerodromes. Through an effective safety management programme, actual and potential hazards can be identified, necessary remedial actions implemented and assurance provided that an acceptable level of safety is being maintained on a continuing basis.

18-3

ICAO Procedures for Air Navigation Services — Air Traffic Management (PANS-ATM, Doc 4444) provides guidance for safety management in ATS. Inter alia, an ATS safety management programme should include the following: a) Monitoring of overall safety levels and detection of any adverse trends; 1) Collection and evaluation of safety-related data. 2) Review of incident and other safety-related reports; b) Safety reviews of ATS units; 1) Regulatory issues; 2) Operational and technical issues; 3) Licensing and training issues. c) Safety assessments in respect of the planned implementation of airspace reorganization, the introduction of new equipment, systems, or facilities, and new or changed ATS procedures; and d) Mechanisms for identifying the need for safety-enhancing measures.

MANAGING ACCIDENT PREVENTION An earlier chapter provided ten steps for “getting started” in setting up a safety management system. The ten steps have equal application to safety management in ATS. That chapter should be read in conjunction with this section. In addition, the following considerations apply to safely managing the provision of air traffic services.

Effective direction ATS Centres and Units are parts of a larger ATS organization whether it be corporate, State, regional or international. The cooperation required among ATS facilities requires a high degree of standardization and interoperability, beginning with well thought-out, clearly understood policies and SOPs. Without consistency in the application of policy and SOPs, safety may be compromised. The Centre Manager or Unit Chief’s commitment to accident prevention must be demonstrated by living up to the spirit and intent of formal policy statements on safety – through actions, not just words. Commitment to safety begins with insistence on the highest standards: i.e. adherence to all prescribed operating procedures in day-to-day activities. This also requires matching management’s performance expectations with sufficient resources to do the job safely (including competent personnel and up-to-date equipment). In addition, the relationship management establishes with line personnel affects the success (or failure) of the two-way communication, fundamental to positive safety culture and accident prevention. Management and the controllers must enjoy an atmosphere of mutual trust and respect.

18-4

Safety organization How the ATS centre or unit is organized for accident prevention will be to a large extent a function of the volume and complexity of their activities. For example, at a large centre such as at an international airport, there are several discrete ATS activities (en-route, terminal, arrival and departure, tower, ground, etc.). The effectiveness of the safety decision-making processes will be largely dependent on how the diverse interests of all the service providers are integrated into a coherent “system”. The Centre Manager or Unit Chief alone may not be able to implement an accident prevention programme. They may require the guidance and assistance of a dedicated safety manager or Accident Prevention Adviser (APA). In appointing an APA, management must avoid the temptation to delegate accountability for safety to the safety manager, rather than to all managers and employees. At large centres, user-groups may be required to consider various perspectives. For example, one group, including representation from flying operators, terminal control, tower control and perhaps the airport authority, may address specific concerns regarding arrival and departure procedures. The involvement of air operators is essential for the development of practical and safe procedures.

Risk management As in other aviation activities, the provision of ATS requires a risk-based approach to decision-making. The same processes described elsewhere in this manual are required for reducing or eliminating risks in the provision of ATS. Risk management requires a coherent system for identifying hazards, assessing the risks and implementing viable measures for controlling the risks. ICAO Doc 4444 (PANS-ATM) requires that all reports of incidents, or reports concerning the serviceability of ATS facilities and systems, (such as failures or degradation of communications, surveillance and other safety significant systems and equipment), be systematically reviewed by the appropriate ATS authority in order to detect any trends in the operation of such systems which may have an adverse effect on safety. Once hazards have been identified, effective risk management depends on the validity of the risk assessment. An acceptable level of risk may be specified in qualitative or quantitative terms. Following are examples where defined values could be used to express the acceptable level of risk, such as, what is the: a) Maximum acceptable probability of an undesirable event, such as collision, loss of separation or runway incursion; b) Maximum number of incidents per 10,000 aircraft movements; c) Maximum acceptable number of separation losses per 10,000 trans-Atlantic crossings; and d) Maximum number of short-term conflict alerts (STCA) per 10,000 aircraft movements. Chapter 5, Risk Management, provides a more complete discussion of the principles of risk management.

18-5

Incident reporting systems As part of an ATS accident prevention programme, a confidential voluntary Safety Incident Reporting System provides one of the best tools for hazard identification. Doc 4444 (PANS-ATM) requires a formal incident reporting system for ATS personnel to facilitate the collection of information on actual or potential safety hazards or deficiencies related to the provision of ATS, including: a) Route structures, b) Procedures, c) Communications, navigation and surveillance systems; d) Other safety significant systems and equipment, and e) Controller workloads. In addition to mandatory State requirements for reporting accidents and incidents, the ATS organization may define the types of hazards, events or occurrences with risk potential that staff are expected to report. An effective reporting system makes provision for the voluntary reporting of any situation or condition that an employee believes poses accident potential in a blame-free, non-punitive environment.

Emergency response ATS personnel must be prepared to continue to provide services through emergency situations, such as, following an accident, a power or communication failure, loss of radar coverage, security threat, etc. Emergency procedures must be in place to guide operations without further compromising safety. The appropriate response of the unit requires a sound Emergency Response Plan (ERP) 82. The ERP should reflect a collaborative effort between management and the operational personnel who will have to execute it, in particular the line controllers. Backup procedures must be in place and regularly tested to ensure the continued provision of services to maintain the safe, expeditious and orderly flow of air traffic — perhaps at a degraded level. For example, shifting to procedural control in the event of a radar failure.

Safety investigations When accidents or serious incidents happen, competent investigators must be available to conduct an investigation in order to: a) Better understand the events leading up to the occurrence; b) Identify hazards and conduct risk assessments; c) Make recommendations to reduce or eliminate unacceptable risks; and

82

See Chapter 16 for guidance on emergency response planning for dealing with an accident or a major incident with ATS involvement.

18-6

d) Communicate the safety messages to the appropriate stakeholders. The investigation of minor incidents, such as losses of separation may yield evidence of systemic hazards. For maximum effectiveness, management should focus on determining risks rather than identifying persons to discipline. How this is done will be influenced by the safety culture of the organization. The credibility of the investigative process will hinge largely on the technical competence and objectivity of the investigators.

Safety oversight Maintenance of high standards of accident prevention in ATS implies a programme of monitoring and surveillance of the activities of all controllers and supporting staff, as well as of the reliability and performance of their equipment. Doc 4444 requires that a safety assessment be carried out in respect of any proposals for significant airspace reorganizations, for significant changes in the provisions of ATS procedures applicable to a defined airspace or an aerodrome, and for the introduction of new equipment, systems or facilities. An effective safety management system for ATS should incorporate an effective safety audit programme which covers all functions of the centre or unit. Doc 4444 requires that qualified personnel having a full understanding of relevant procedures, practices and factors affecting human performance, conduct safety reviews of ATS units on a regular and systematic basis. Doc 4444 also requires that data used in safety monitoring programmes be collected from as wide a range of sources as possible, as the safety-related consequences of particular procedures or systems may not be realized until after an incident has occurred. Thus, the audit programme should include the safety interfaces with all users of the ATS system, operators, airport management, and any contracted service providers.

Managing change The provision of ATS is a dynamic activity. Doc 4444 requires that all proposals for significant changes be evaluated through a safety assessment. Examples of significant changes include: a) Reduced separation minima; b) New operating procedures, including arrival and departure procedures (STARS and SIDS); c) Reorganization of the ATC route structure; d) Re-sectorization of an airspace; and e) Implementation of new communications, surveillance or other safety-significant systems and equipment, including those providing new functionality and/or capabilities. Safety factors to consider in conducting such a safety assessment include: a) Types of aircraft and their performance characteristics, including their navigation capabilities and performance;

18-7

b) Traffic density and distribution; c) Airspace complexity, ATS route structure and the classification of the airspace; d) Aerodrome layout, including runway and taxiway configurations and preferences; e) Air-ground communications capabilities and usage; f) Surveillance and alerting systems; and g) Significant local topography or weather phenomena, etc. HUMAN FACTORS IN ATS83 The world’s worst aviation disaster (a collision on the runway between two B747s at Tenerife in 1977) was attributed in part to human factors. Like pilots, air traffic controllers are at the forefront of aviation activity. They must interface in real time with pilots, other controllers, complex equipment and a plethora of procedures and standards, etc. Accident prevention in ATS requires a solid understanding of these interfaces and the human factors issues that can limit the performance of controllers. Some of the more common human factors that could create accident potential in ATS include: a) Physiological limitations: 1) Vision — the ability to physically see events unfolding (say from a control tower); 2) Hearing — the ability to discriminate different speech patterns in a noisy environment; and 3) Chronic fatigue affecting judgement, cognitive skills, and memory, etc. b) Psychological variables: 1) Memory (essential to maintaining a three dimensional picture of a dynamic situation); 2) Vigilance vs. distractions and boredom; 3) Operating pressures (from supervisors or management and from peers); 4) Motivation and frame of mind (perhaps affected by domestic or other outside pressures); 5) Stress tolerance (and consequential stress-related illnesses); 6) Judgement; 7) Habit patterns (perhaps taking procedural shortcuts); and

83

See Human Factors Training Manual (Doc 9683) for a more complete discussion of Human Factors in ATS.

18-8

8) Cultural diversity of the many users of the ATS system (such as military vs. civilian, different companies, foreign vs. domestic, different languages and behavioural patterns, etc.), all potentially capable of affecting the controllers’ expectancy, etc. c) Equipment factors: 1) Display design and workstation layout; 2) User-friendliness of software, including flexibility to adapt to changing situations; and 3) Use of automation, etc. d) Information transfer problems including: 1) Frequency congestion; 2) Call-sign confusion; 3) Hearing expectancy; 4) Language comprehension and accent; and 5) Use of non-standard phraseology, etc. e) Workload considerations: 1) Volume and complexity of traffic; 2) Number of sectors in use; 3) Situational awareness (maintaining the “big picture”); 4) Mental models used in decision-making (e.g. “rules of thumb”); 5) Time since last break; 6) Impact of shift work, scheduling and overtime; and 7) Chronic fatigue, etc. f) Organizational factors: 1) Corporate safety culture; 2) Approach to teamwork (and use of Team Resource Management (TRM)); 3) Adequacy of training; 4) Controller experience, competence and currency; 5) Quality of first line supervision;

18-9

6) Controller/management relationship; 7) Effective standardization of procedures and phraseology; and 8) Effective monitoring of day-to-day operations, etc. As traffic volumes continue to increase, ATS supervisors, investigators of ATS occurrences and APAs will require an increasing knowledge of the effects of such human factors on the performance of ATS personnel.

__________________

18-10

Chapter 19 ACCIDENT PREVENTION AT AIRPORTS “Ground Safety”

Introduction Airport Safety — General • Safety management system • ICAO requirement • Airport management Managing Airport Safety • Coherent direction • APA and safety committee(s) • Incident reporting • Safety oversight • Emergency response Airport Emergency Response Planning • Coordinated response • Airport emergency response exercises Airport Ramp Safety • Ramp work environment • Causes of ramp accidents • Safety management on the ramp • Vehicle operations Role of Operator’s APA in Ground Safety

19-1

This page intentionally left blank.

19-2

Chapter 19 ACCIDENT PREVENTION AT AIRPORTS “Ground Safety” INTRODUCTION The focus of aviation safety management is often on flight operations. However, occurrences during ground operations can also affect an airline’s financial position. Although a catastrophic accident during ground operations is unlikely, the probability of minor ground accidents is high. Each year, airlines incur significant financial losses associated with accidents during ground handling. Accidents and incidents occurring in-flight are generally well reported and investigated. However, ground accidents do not always receive the same level of attention. Minor accidents and incidents may not be reported to the airport management by the operators, tenants and service providers at the airport. These minor accidents and incidents may be a breeding ground for more serious accidents. Understanding the conditions that create hazards to safety at airports is vital to any successful accident prevention programme. Accident prevention at airports requires much the same approach to safety management as that required for safe flight operations. The concentration of different activities at airports creates unique circumstances with significant accident potential.

AIRPORT SAFETY — GENERAL Ground occurrences must be seen within the overall context of airport operations. Airports bring together a volatile mixture of activities with high-risk potential. Some of the factors contributing to this risk potential include: a) Traffic volume and mixture (including domestic and international, scheduled and non-scheduled, charter and specialty operations, commercial and recreational aviation, fixed and rotary wing, etc.); b) Vulnerability of aircraft on the ground (awkward, fragile, etc.); c) Abundance of high energy sources (including jet blast, propellers, fuels, high air pressures, etc); d) Extremes of weather (temperatures, winds, precipitation, obstructions to visibility); e) Wildlife (birds and animals) control; f) Aerodrome layout (especially taxiway routings and congested ramp areas); g) Adequacy of signage and markings, lighting, etc.; h) Non-adherence to established procedures (especially at uncontrolled airports);

19-3

i)

Control of vehicles on the ramp;

j)

Problems in information transfer (communications) with those operating on the airside;

k) Runway usage (including simultaneous multiple runway usage, intersection departures, preferential runways); l)

Ground and apron control (sometimes compromised by frequency congestion, use of nonstandard phraseology, language difficulties, mistaken call-signs, etc.);

m) Adequacy and reliability of approach aids; n) Adequacy and reliability of communications systems (air-ground as well as inter-vehicular); o) Airspace limitations (topography, obstructions, noise abatement requirements, etc.); and p) Security issues. Within this operating context, the airport provides a diverse set of services to support flight operations. Some of these include: a) Flight planning, including weather services; b) Navigation, approach and landing aids; c) Communication services; d) Air traffic, ground and ramp control; e) Runway and ramp maintenance (including snow and ice control, bird and wildlife control, FOD control, etc); f) Aircraft servicing of all types; g) Airport security; h) Airport emergency services (i.e. crash and fire rescue services); i)

Management of tenants (aviation operators, service contractors, etc.); and

j)

Customer management (passengers, freight shippers, etc.).

Safety management system (SMS) Given the complexity of the airport environment, a systematic approach to safety is required in order to coordinate the various activities for the safe delivery of services. A safety management system (SMS) ensures such a coherent approach. Implementing an SMS establishes the organizational framework for managing all the safety interests of the airport. In so doing, the safety Philosophy and the supporting Policies are developed, operating Procedures are coordinated and implemented, and day-to-day

19-4

operational Practices are systematically monitored. In short, an SMS helps create an airport safety culture conducive to accident prevention.

ICAO requirement Annex 14 — Aerodromes requires that aerodromes used for international operations be certified in accordance with ICAO specifications, through an appropriate regulatory authority. As part of this process, a certified airport is required to have a safety management system (SMS). The intent of requiring an SMS is to ensure an organized and orderly approach to the management of safety by the aerodrome operator. The SMS should provide for the organizational structure, responsibilities, procedures, processes and provisions for the implementation of aerodrome safety policies by an airport operator.

Airport management Traditionally, major airports have been owned and operated by the State. Increasingly, this is changing as airports are corporatized and management is turned over from government officials to airport authorities. Regardless of whether the airport manager is a government official or a senior manager for a nongoverment corporation, accident prevention remains a primary concern. Within the framework of an airport SMS, the airport management must oversee the activities of all the service providers, tenants, contractors and others to ensure the safest and most efficient performance of the airport. An effective airport accident prevention programme should begin with a strong corporate knowledge of the aviation business including best safety practices. The airport management must promote a positive safety culture. In part this will depend on: the resources dedicated to the accident prevention programme; the feedback mechanisms put in place — and how they are managed on a day-to-day basis; the promotion of sharing of safety-related information among stakeholders in the airport’s operation; and a constant striving for improvement.

MANAGING AIRPORT SAFETY Chapter 13 provides a comprehensive plan for establishing an effective accident prevention programme. The ten steps outlined in that chapter also apply for airport managers (APMs) in establishing a successful accident prevention programme for airports and ground safety. The following paragraphs are intended to supplement the material of that Chapter.

Coherent direction Given the complexity of the factors creating risk potential at airports, the APM must coordinate the activities of the diverse stakeholders at an airport — often with conflicting expectations and priorities. He must foster the sharing of a common focus amongst the stakeholders, most of whom are employees of agencies other than the airport authority. In addition, he may have to obtain resource commitments from the airlines, service providers and other contractors. The airport’s accident prevention programme begins with the development of appropriate safety policies and operating procedures. These policies and operating procedures are more likely to be implemented if stakeholders participate in their development and if they are included in appropriate contractual documents, such as leases, operating authorities, etc. A high degree of cooperation by all stakeholders

19-5

will also be necessary to achieve the desired level of standardization and interoperability required for safe ground operations. The APM must be careful to ensure that commercial interests, upon which the financial viability of the airport depends, are not met at the expense of operational issues that may affect safety. For example, increasing the number of aircraft gates may increase airport revenue but may also increase ramp congestion, presenting additional safety risks. Many large airports have a strong users’ group or committee, formed with representatives of airport tenants, operators, service providers, contractors, etc., which can assist airport management in resolving many of the problems before they become major safety issues.

APA and safety committee(s) Large airports may benefit from the appointment of a dedicated Accident Prevention Adviser (APA). The appointment of an APA, however, does not relieve the APM of the accountability for an effective accident prevention programme. In addition, large airports may require a safety committee. Since the safety committee would involve participation by many of the organizations that are on the user’s group referred to above, the APM and the stakeholders may find that the airport safety committee is an effective vehicle for integrating their diverse views. For example, such a committee would be essential in preparing the airport emergency response plan (discussed later in this Chapter). An airport APA would logically coordinate the activities of the airport safety committee. Further, given the requirement to integrate so many, often-conflicting interests, several safety sub-committees may be required. For example, separate groups may be formed to address particular areas of safety concern, such as airport security, vehicle operations on the airside, snow and ice removal, wildlife control, etc.

Incident reporting

Risk management begins with effective hazard identification. One of the most powerful tools for proactively identifying safety hazards is voluntary and confidential incident reporting. Through a nonpunitive, incident-reporting programme, the APM can tap the diversity of views available at an airport in identifying underlying situations or conditions with the potential for creating losses. In implementing an incident reporting system, airport employees, contractors and tenants should be clear on: a) The types of hazards that should be reported; b) The reporting mechanisms; c) Their job security; and d) Actions taken in following-up on identified hazards.

19-6

However, given the number of stakeholder groups involved, with their diverse interests and priorities, establishing and running an effective incident reporting system on an airport presents a considerable challenge.

Safety oversight Given the diverse activities of many different agencies, the maintenance of high safety standards at airports implies a regular programme of monitoring and surveillance. At the interfaces between stakeholders there may be a tendency to shirk responsibilities, stating that “it is not my problem”; for example, airport employees vs. the employees of airlines or contracted service providers. That is why it is essential that roles and responsibilities are clearly defined. Change is everywhere as airports expand to meet increasing demand. New runways and taxiways, terminal buildings, shops and warehouses, etc., have the potential to introduce new safety hazards. The APM may require that a safety assessment be carried out in respect of any proposals for significant changes in the operation of the aerodrome, or for the introduction of new equipment, systems or facilities. An effective safety management system for an airport should also incorporate a safety audit programme which covers the activities conducted at the airport that come under the purview of the airport management. Such safety reviews would cover the ramp activities of service providers and operators. A good understanding of human factors issues involving groups of employees, such as baggage handlers, vehicle operators, etc. will provide insights into safety hazards. Cooperative arrangements with the management of like-sized airports may provide the opportunity to gain additional expertise and experience for effective safety reviews and audits.

Emergency response Most accidents occur on or in the vicinity of airports, creating significant strain on the resources of the airport. Responding appropriately to an aircraft emergency is one of the more critical challenges facing airport management. To ensure an appropriate response at such times of high stress, a sound Emergency Response Plan (ERP) is essential. The ERP reflects a collaborative effort between airport management, the resident stakeholders and those who will have to execute the plan. The following section elaborates on planning for an airport emergency response. AIRPORT EMERGENCY RESPONSE PLANNING84 Airport emergency planning is the process of preparing an airport to cope with an emergency occurring at the airport or in its vicinity. The object of airport emergency planning is to minimize the effects of an emergency, particularly in respect of saving lives and minimizing the effect on aircraft operations. The airport emergency plan sets forth the procedures for coordinating the response of different airport agencies (or services) and those agencies in the surrounding community that could be of assistance in responding to the emergency. ICAO Airport Services Manual (Doc 9137) requires that an airport emergency plan will be implemented irrespective of whether an occurrence is an ‘on-airport’, or an ‘off-airport’ accident/incident. The ERP 84

This section is based upon ICAO Airport Services Manual, Part 7 — Airport Emergency Planning (Doc 9137).

19-7

should take into account operations in all weather conditions. The ERP should also make provision for potential accident locations in difficult terrain surrounding the airport, i.e. bodies of water, roads, depressions and other problem areas.

Coordinated response The plan should spell out the response, or participation, of all agencies which, in the opinion of the airport authority, could be of assistance in an emergency. Examples of such agencies are: a) On the airport: 1) Rescue and firefighting service; 2) Medical services; 3) Police and/or security services; 4) Airport administrations; air traffic services; and aircraft operators. b) Off the airport: 1) Police; 2) Local fire departments; 3) Medical services; 4) Hospitals; 5) Government authorities; 6) Military; 7) Harbour patrol and coast guard; and 8) Other relevant agencies.

Airport emergency response exercises The airport emergency response plan provides the theoretical framework for response. However, testing is crucial for determining where gaps may exist in the plan. For example, resolving misunderstandings among participants, procedures unworkable in practice, unrealistic estimates of requirements (time, resources, etc.). Testing the plan also allows participants to get to know each other and to learn how other services operate. It also confirms the vital communication links. There are three methods of testing an airport emergency plan: Full-scale exercises. Realistic, comprehensive simulations for testing all capabilities, facilities and agencies participating in an emergency response conducted at least once every two years.

19-8

Partial exercises. Simulations for selected emergency response functions, such as firefighting conducted at least once each year in which a full-scale exercise is not conducted, or as required to maintain proficiency. Table-top exercises. For updating procedures, checklists, telephone lists etc. and for integrating emergency response resources without expense coordinated at least semi-annually. Following are some of the more important considerations in preparing an exercise plan for the Airport ERP: a) Airport emergency service personnel are regularly tested on: 1) Emergency response procedures, first aid, etc.; 2) Fire-fighting; and 3) Emergency evacuations, including knowledge of relevant aircraft systems and evacuation routes; etc. b) Communication and call-out procedures are tested and up-to-date; c) Crash and fire routes are well understood, kept clear and inspected regularly; d) Command post is designated, equipped and tested; e) Temporary morgue facilities are available; f) Procedures are in place (and regularly tested) for: 1) Crowd control; 2) Media access; and 3) Receiving families and next of kin of accident victims; g) Clearing of aircraft wreckage or recovery of aircraft; and h) Provision for restoration of service or continued operation of the airport, etc. AIRPORT RAMP SAFETY85 Ground accidents usually involve relatively minor damage. Aircraft skin and ground-servicing equipment may be damaged and/or employees injured. Although much less frequent, a door or panel, which has been improperly secured, may in the worst case open in flight, leading to structural failure of the aircraft. The close linkage between ground safety and flight safety is clear.

85

Much of this section is based upon The Management of Safety on the Airport Ramp by Nick MacDonald and Ray Fuller in Aviation Psychology in Practice Avebury Technical, Aldershot UK 1994

19-9

Aircraft are easy to damage and expensive to repair. Even minor ground handling accidents are expensive as they incur such indirect costs as schedule disruptions, passenger accommodations, etc. Yet, because such occurrences may not fall within the definition of an aviation accident, aviation organizations frequently view such accidents from the perspective of occupational health and safety or environmental safety — as opposed to a critical aspect of maintaining safe and efficient flight operations. The concept of creating and fostering a positive safety culture on the ramp is often not well developed.

Ramp work environment The ramp can provide a difficult work environment that is often not ideal for safe operations from a human factors perspective. Difficulties can arise from the variety of activities, congestion in a restricted environment, tight time pressures, and often poor weather or lighting conditions. For example, the following activities are routinely conducted in the vicinity of aircraft: a) Aircraft ground handling comprises all the activities required to turn an aircraft around including: 1) Marshalling and chocking arriving aircraft; 2) Refueling; 3) Correcting maintenance defects and performing routine maintenance and inspections; 4) De-icing and anti-icing; 5) Catering, cleaning cabins and servicing water and toilets; 6) Passenger embarkation/disembarkation; 7) Loading and unloading of baggage and freight; and 8) Aircraft towing and pushback, etc; b) Airport emergency services; c) Airport security; and d) Ramp maintenance, etc. In addition to the complexity of ramp operations, the nature of ground handling poses significant scope for safety hazards due in part to: a) Aircraft size and shape vs. vehicle driver’s susceptibility to misperceptions and misjudgments of distance and location; b) Fragile skin and appendages which are easily damaged; c) Need to preserve aerodynamic and structural integrity of the aircraft; d) Constraints of space and time; and

19-10

e) Number of unskilled, low-paid and poorly motivated workers. Several human factors exacerbate the accident potential of the foregoing. The following factors typically characterize the workplace and content of ground-handling duties: a) Hostile work environment (noise, jet-blast, etc. in all kinds of weather); b) Working in limited (often height-restricted) space in midst of congestion of other servicing vehicles, personnel and adjoining aircraft movements; c) Time pressures for on-time departures (or to make-up for late running); d) Cyclical workload with peak demands followed by complete lulls between transiting aircraft; e) Frequent shift work; f) Requirement to operate a variety of expensive, specialized servicing equipment; g) Workforce (especially for loaders) often comprises casual labour, selected more for physical strength than intellectual capacity and skills; h) Ramp workers are often employed by organizations other than the airport authority (e.g. airlines, service providers, catering companies, etc.); and i)

Organizational factors deriving from management’s failure to provide a parallel level of attention to ground safety as it does to flight safety.

All things considered, the potential for accidents and injuries in the ramp environment is high. Reducing that potential requires a multidisciplinary effort across many departments of the airport, airline and contractors staffs.

Causes of ramp accidents Although many airlines have their own internal accident/incident databases, there are few public sources for data on ramp accidents. Most ground occurrences are not reported to any State authority. Nevertheless, based on industry experience, some general statements can be made about the causes of ramp accidents: a) Regulations or Standard Operating Procedures are inadequate or not followed. b) Poor discipline and inadequate supervision set up many accidents (particularly those involving excessive speed). c) Equipment. Incorrect use or abuse of the ground handling equipment often results in personal injuries. d) Dynamic environment with constant motion (and commotion) makes maintenance of situational awareness difficult even for experienced personnel. e) Weather — outdoors, round the clock in all seasons — limits human performance.

19-11

f) Training vs. exposure to risk. Companies generally train their skilled employees adequately. However, the relatively unskilled workers on the ramp who are exposed daily to significant risk usually receive little safety training and supervision. g) Human factors. Ramp accidents often involve human factors from such things as misjudgment, obscured vision, stress, distraction, time (or peer) pressures, complacency, ignorance, fatigue, etc.

Safety management on the ramp Ramp operations present scenarios with often conflicting goals that require rapid risk management decisions. Balancing the requirement for safety against operating pressures to provide a quick turnaround of the aircraft to avoid delays and disruptions calls for trade-offs. Shortcuts in following standard operating procedures may be taken to facilitate on-time departures, usually without adverse consequences. Workers may be chastised (perhaps even penalized) for failure to keep things moving. Yet, they may be “punished” if the practices they followed contributed to an accident. How can this vicious cycle be broken? The three cornerstones for an effective safety management system and the corresponding activities were discussed in an earlier chapter. With minor modifications these apply to preventing ramp accidents. Some factors warranting especial consideration include: a) Formal structured training geared to staff capabilities including: 1) Orientation for accident prevention; 2) Safe operation of ground support equipment; 3) Need for compliance with SOPs; and 4) Skills training such as marshalling signals and seasonal skills such as de-icing; b) Clear practical SOPs which are understood, practiced and enforced; c) Hazard and incident reporting system which encourages input from ground servicing personnel; d) Competent investigation of ramp mishaps, with particular emphasis on the human factors aspects; e) Effective collection and analysis of relevant ground safety data; f) Fostering a positive safety culture for all ramp workers, whereby they take “ownership” for their accident prevention record; g) Representation of ground handlers and servicing personnel on safety committees, perhaps including a separate sub-committee for ground safety; h) Feedback to workers regarding identified hazards and actions taken to reduce or eliminate them; i)

Continuing programme of accident prevention awareness; and

19-12

j)

Monitoring of ground system safety (through regular assessments and audits), etc. Vehicle operations86

Aircraft on the ground attract vehicles; the larger the aircraft, the more vehicles are required. Unfortunately, aircraft and vehicles do not mix well. The risk of collision is ever-present and the potential for serious consequences is great. Excessive speed in confined areas and in close proximity to aircraft is a major cause of ramp accidents. A systems approach is required for organizing and controlling vehicular traffic on the ramp, in order to reduce the risk of accidents. Most vehicle operators on the ramp are not airport authority employees; for example, they may work for commercial service providers, such as airlines, refuelling companies, catering and cleaning companies, etc. Most of these personnel are beyond the control of any single organization, however, they normally require some form of approval issued by the airport authority to drive on the ramp. Following are some methods for safely controlling vehicles that airport safety committees and APAs should consider: Vehicle control plan. This plan is usually developed by the airport authority and applies to all ramp areas and vehicles operated on them. All airport tenants are expected to know and follow this plan which should prescribe vehicle flow, signage and markings for vehicles and traffic control devices. Vehicle operating standards. These are the basic “rules of the road” for how vehicles are to be operated on the airport — including limits on speed and proximity to aircraft, right of way, etc. They are normally developed by the airport authority, perhaps with the advice and assistance from major tenants. Vehicle limitations. A basic rule is to limit the number of vehicles on the ramp to the minimum number needed to do the job. Each vehicle has to be justified. All vehicles should be company-owned with no privately owned vehicles authorized. Vehicle operator training. All drivers on the ramp must have some form of training (and perhaps even certification) before they are allowed to operate vehicles there. This programme may be administered by the airport authority or by the major airport tenants in accordance with guidelines from the airport authority. Enforcement. None of this effort is worthwhile, unless drivers comply with the plan and the standards set. Close supervision and monitoring are required to ensure that all users of the ramp uphold safety standards. This includes enforcement action against those who do not comply.

ROLE OF THE OPERATOR’S APA IN GROUND SAFETY An operator’s APA (whether supported by a ground safety officer or not) can make a significant contribution to flight safety, operational effectiveness, and thus the company’s finances through accident prevention efforts in ground handling. The APA must ensure that management sees that ground handling of aircraft is an integral part of flight operations. Ground safety merits the same systematic approach and attention to detail as the flight safety programme. The company’s programme for preventing accidents on the ground should therefore embody all the elements of the programme for flight operations, (hazard 86

Based on Wood, Richard H., Aviation Safety Programs: A Management Handbook, Jeppeson Sanderson Inc., Englewood, Colorado, 1997

19-13

reporting systems, safety committees, risk management processes, competent investigations, safety oversight, etc.). A successful accident prevention programme requires a solid working relationship between the airport managers of line station airports and the company APA. Indeed, an airline APA should visit all the airport managers on routes regularly serviced by the airline. During these liaison visits the APA should be interested in the adequacy of airport defences against ground accidents such as: a) Routine maintenance (for hard surfaces, lighting, signage and markings, etc); b) Planned new construction; c) Airport and ramp inspections, including control of Foreign Object Damage (FOD); d) Control of vehicle operations; e) Wildlife hazard control, especially birds; f) Snow and ice removal; g) Fuel quality and handling procedures; h) Emergency response plan; i)

Safety committees; and

j)

Communications of safety information at the local level, etc.

__________________

19-14

Chapter 20 ACCIDENT PREVENTION IN AIRCRAFT MAINTENANCE

Introduction Maintenance Safety — General • Maintenance working conditions — Organizational issues — Work site conditions — Human factors in maintenance Managing Safety in Maintenance • Corporate approach to safety — Organizing for safety — Documentation and records management — Resource allocations — Safety culture • Tools for programme delivery • Safety oversight and programme evaluation Managing Procedural Deviations in Maintenance • Maintenance Error Decision Aid (MEDA) APA’s Concerns Appendix 1. Maintenance Error Decision Aid (MEDA)

20-1

This page intentionally left blank.

20-2

Chapter 20 ACCIDENT PREVENTION AND AIRCRAFT MAINTENANCE INTRODUCTION Until recently, accident prevention programmes have tended to focus on the safety of flight operations. Less attention was devoted to systematically reducing risks arising from the maintenance activity. Yet, maintenance and inspection errors are cited as a cause for a number of accidents worldwide each year. Such errors are contributory in even more accidents and serious incidents. The safety of flight is dependent on the airworthiness of the aircraft. Accident prevention programmes in the areas of maintenance, inspection, repair and overhaul are therefore vital to flight safety. Maintenance organizations need to follow the same disciplined approach to safety management as is required for flight operations. Adhering to such a discipline in maintenance can be difficult. The maintenance activities may be conducted by the airline itself, or they may be contracted out to approved maintenance organizations. Furthermore, some of these activities may take place at a considerable distance from the aircraft’s home base. The term “safety” in an aircraft maintenance context carries two connotations: one with an emphasis on industrial safety and hygiene for the protection of Aircraft Maintenance Technicians (AMTs), facilities and equipment. The second is the programme for ensuring that AMTs provide airworthy aircraft for flight operations. Although the two may be inextricably linked, this chapter concentrates on the latter, with little reference to Occupational Safety and Health (OSH) issues.

MAINTENANCE SAFETY — GENERAL Conditions for maintenance-related failures may be set in place long before the eventual failure. For example, an undetected fatigue crack may take years to progress to the point of failure. Unlike flight crews who have near real-time feedback on their errors, maintenance personnel usually receive little feedback on their work until failure occurs. During this time lag, maintenance workers may continue to create the same latent unsafe conditions. As a consequence, the maintenance world incorporates a combination of safety defences, including multiple redundancies of aircraft systems to strengthen the system. These defences also include such things as certification of maintenance organizations, licenced mechanics, airworthiness directives, detailed SOPs, job cards, inspection of work, sign-offs and records of work completed, etc.

Maintenance working conditions The conditions under which aircraft maintenance is conducted often pose variables conducive to introducing risk potential. Some of the broader issues in maintenance potentially affecting safety include: Organizational issues: a) Time pressures to sustain on-time departures and around-the-clock operations (vs. shortcuts by workers);

20-3

b) Ageing aircraft requiring intensive inspections for fatigue, corrosion and overall condition, etc.; c) New technologies requiring new tools and test equipment, new work procedures, costly retraining, etc.; d) “Fix-it” focus to stay on schedule, replacing broken parts without determination as to why they failed (perhaps due to poor design or misassemble); e) Airline expansions and mergers, combining maintenance departments with different work practices and safety cultures; f) Outsourcing of services to sub-contractors (e.g. for heavy maintenance and overhaul); g) Unwitting introduction of (lower cost, substandard) bogus parts, etc.; and h) Licensing of AMTs for different aircraft, aircraft generation, types and manufacturers. Work site conditions: a) Aircraft designs which are not user-friendly from a maintenance perspective (for example, cramped access to components, height off the ground); b) Control of aircraft configurations (which are ever subject to modifications) vs. standardization of maintenance tasks and procedures; c) Availability (and accessibility) of spares, tools, documentation, etc.; d) Requirements for having ready access to voluminous technical information and the need for maintaining detailed work records; e) Variable environmental factors (for example, conditions on ramp vs. technical workshop vs. hangar floor); f) Unique operating conditions created by concurrent activities and inclement weather on the ramp; and g) Shortcomings in the provision of timely, accurate, understandable discrepancy reports by flight crews, etc. Human factors in maintenance. Some of the more common human factors likely to induce maintenance hazards include: a) Organizational and working conditions (as described above); b) Environmental factors (including temperature, lighting, noise, etc.); c) Individual factors (workload, physical demands or maintenance, etc.); d) Scheduling (shift work, night work and overtime) vs. adequacy of rest periods;

20-4

e) Appropriateness of SOPs (correctness, understandability and usability, etc.); f) Quality of supervision; g)

Proper use of job-cards, etc. (i.e. do actual floor practices comply with SOPs?);

h) Adequacy of formal training, on-the-job training (OJT), and recurrent training, human factors training; i)

Adequacy of handovers at shift changes and record keeping, etc.;

j)

Boredom; and

k) Cultural factors (AMT professionalism, openness to report errors and hazards, etc.).

MANAGING SAFETY IN MAINTENANCE Given the nature of the maintenance functions, the working environment for AMTs and the many Human Factors which may compromise their expected performance, a systematic approach to accident prevention is called for, i.e. a safety management system. Chapter 4, Managing Safety, describes how successful safety management systems recognize organizational inter-dependencies and interactions, with the need to integrate safety efforts across the entire organization. They are built upon the three cornerstones of: a) Corporate approach to safety; b) Effective tools for programme delivery; and c) Formal system for safety oversight and programme evaluation. Each of these aspects of a safety management system is discussed below.

Corporate approach to safety The corporate approach to safety sets the tone for how the organization develops its safety philosophy and policies, its safety culture, etc. In deciding on the approach the organization wishes to take towards safety management, the following factors may be relevant: a) Size of the organization (large carriers tend to require more structure); b) Nature of the operations (e.g. round-the-clock, international, scheduled operations vs. local or unscheduled operations); c) Organizational status (department of an airline vs. an independent enterprise); d) Maturity of the organization and its workforce (i.e. corporate stability and experience); e) Labour management relationships (recent history, complexity); f) Current corporate culture (vs. desired safety culture); and

20-5

g) Scope of maintenance work (line servicing vs. heavy overhaul of aircraft or major systems), etc. Organizing for safety. Chapter 13, Establishing an Accident Prevention Programme, offered two sample organizational structures for an airline, both of which reflect direct but informal reporting lines between operations, safety and maintenance (including the maintenance quality assurance manager). Such communications channels depend on the trust and respect established in the day-today working relationships of those involved. In an airline, the company APA must have clearly defined responsibilities and reporting lines with respect to accident prevent programmes in maintenance. The maintenance organization may require a technical specialist to work with the company APA. As a minimum, the company APA will require specialist advice from the maintenance department. The company’s safety committee should include representation from the maintenance department. In large airlines, a dedicated sub-committee for maintenance safety may be warranted. Documentation and records management. Maintenance departments depend heavily on quality systems for systematically acquiring, storing and retrieving the voluminous information required for safety management. Examples include: a) Technical libraries must be kept current (for such things as engineering orders, type certifications, airworthiness directives, service bulletins, etc.); b) Maintenance defects and work completed must be recorded in detail; c) Performance and system monitoring data must be retained for trend analysis; d) Corporate safety policies, objectives and goals require formal documentation and distribution; e) Records on personnel training, qualification and currency, etc.; and f) Component history, life, etc. In a large airline, much of this information will be computerized. Therefore, the success of a safety management system in a maintenance organization will largely depend on the quality and timeliness of its document and records management systems. Resource allocations. The best accident prevention programme on paper will be useless without adequate resources. To provide protection against significant losses due to an accident, significant expenditure is required. For example: a) Personnel with expertise to design and implement the accident prevention programme activities; b) Training in accident prevention for all staff; c) Information management systems to store safety data and expertise to analyse that data; and d) Office administration and support, etc.

20-6

Safety culture. A poor safety culture in a maintenance organization can lead to unsafe work practices not being corrected; possibly creating latent unsafe conditions that may not cause a problem for years. Management’s success in fostering a positive safety culture in the maintenance department will derive in large measure from how the foregoing issues are addressed and from how the programme is actually implemented.

Tools for programme delivery Effective delivery of a safety management system for maintenance builds upon risk-based decisionmaking, a concept that has long been integral to maintenance practices. For example, maintenance cycles are built upon probabilities that systems and components would not fail for the period of the cycle. Components are often replaced because they are “time expired” even though they may remain functionally serviceable. Based on knowledge and experience, risks of unsuspected failure are reduced to acceptable levels. Some of the principal tools (described elsewhere in this manual) for delivering a safety management system include: a) Clearly defined and enforced Standard Operating Procedures (SOPS); b) Risk-based resource allocations; c) Hazard and incident reporting systems; d) Flight data analysis programme; e) Trend monitoring and safety analyses (including cost-benefit analyses); f) Competent investigation of maintenance-related accidents and incidents; g) Training in safety management and accident prevention; and h) Communication and feedback systems (including information exchange, safety promotion, etc.).

Safety oversight and programme evaluation As with any “system”, feedback is required to ensure that the individual elements of the maintenance safety management system are functioning as intended. Continuing high standards of accident prevention in a maintenance organization implies a regular programme of monitoring and surveillance of all maintenance activities. This is especially so at the interfaces between workers (such as between maintenance personnel and flight crews, between different trades, or between changing work shifts) to avoid problems “falling through the cracks”. An earlier chapter discussed methods for maintaining safety oversight, including the conduct of regular safety audits. Change is inevitable in the aviation industry and the maintenance area is no exception. The Director of Maintenance may require that a safety assessment be carried out in respect of any significant changes in the maintenance organization. Circumstances that might warrant a safety assessment include: a corporate merger, introduction of a new fleet, equipment, systems or facilities, etc. In this way, the need for any programme adjustments can be identified and corrected.

20-7

As discussed in an earlier chapter, the overall accident prevention programme should be regularly evaluated to ensure that expected results are being achieved. Programme evaluation should provide satisfactory responses to such questions as: a) To what extent has management succeeded in establishing a positive safety culture? b) What are the trends in hazard and incident reporting? (including by technical trade or by aircraft fleet) c) Are hazards being identified and resolved — before a serious incident or accident occurs? d) Have adequate resources been provided for the accident prevention programme?

MANAGING PROCEDURAL DEVIATIONS IN MAINTENANCE The maintenance system includes not only the AMTs on the shop floor, but also all the other technicians, engineers, planners, managers, stores keepers and other persons that contribute to the maintenance process. In such a broad system, procedural deviations and errors in maintenance are inevitable and pervasive. Accidents and incidents attributable to maintenance are more likely to be caused by the actions of humans than by mechanical failure. Often they involve a deviation from established procedures and practices. Even mechanical failures may reflect errors in observing (or reporting) minor defects before they progress to the point of failure. Maintenance errors are often facilitated by contributory factors beyond the control of the AMT, for example: a) Information required to do the job; b) Equipment and tools required; c) Aircraft design limitations; d) Job or task requirements; e) Technical knowledge or skills requirements; f) Factors affecting individual performance (i.e. SHEL factors); g) Environmental or workplace factors; h) Organizational factors such as corporate climate; i)

Leadership and supervision; and

j)

Breakdowns in essential communications, etc.

20-8

Safe maintenance organizations foster the conscientious reporting of maintenance errors, especially those that jeopardize airworthiness, so that effective action can be taken. This requires a culture in which staff feel comfortable reporting errors to their supervisor or line manager once an error is recognized. New systems are being developed for managing procedural deviations (and errors) in aircraft maintenance. Typically these systems are a subset of an overall maintenance safety management system and exhibit the following characteristics: a) Encourage uninhibited reporting of occurrences which would not otherwise be required to be reported; b) Provide training for staff on the purpose and procedures for using the maintenance safety management system, including clear definition of departmental disciplinary policies (e.g. Disciplinary action should only be necessary in instances of what can objectively be termed recklessness); c) Incorporate competent safety investigation of reported errors; d) Seek appropriate safety action in follow-up to identified safety deficiencies; e) Provide feedback to the workforce; and f) Provide data suitable for trend analysis.

Maintenance Error Decision Aid (MEDA) One major programme for managing procedural deviations is the Maintenance Error Decision Aid (MEDA) developed by the Boeing Company. MEDA provides the first-line supervisor (and the APA) with a structured method for analysing and tracking the contributing factors leading to maintenance errors and for recommending error prevention strategies. In the MEDA process there are five basic steps: Event. Following an event, it is the responsibility of the maintenance organization to select the errorcaused aspects that will be investigated. Decision. After fixing the problem and returning the aircraft to service, the operator makes a decision: Was the event maintenance-related? If yes, the operator performs a MEDA investigation. Investigation. Following a structured form (specifically designed for MEDA), the operator carries out an investigation. The investigator records general information about the aeroplane, when the maintenance and the event occurred, the event that precipitated the investigation, the error that caused the event, the factors that contributed to the error and a list of possible prevention strategies. Prevention Strategies. Management reviews, prioritizes, implements, and then tracks prevention strategies (process improvements) in order to avoid or reduce the likelihood of similar errors in the future. Feedback is provided to the maintenance workforce so technicians know that changes have been made to the maintenance system as a result of the MEDA process. Management is responsible for

20-9

affirming the effectiveness of employees’ participation and validating their contribution to the MEDA process by sharing investigation results with them. The attachment to this chapter provides a more detailed description of MEDA.

APA’S CONCERNS A company APA faces challenges in providing sound advice to senior management on the maintenance portion of the accident prevention programme — especially if the APA’s background is not in aircraft maintenance. For example: a) Understanding accident prevention in the context in which maintenance work is carried out; b) Developing personal credibility especially in acquiring sufficient knowledge of accepted safe industry work practices and maintaining currency with respect to industry developments in aircraft maintenance. (One way for the APA to better understand the complex nature of the aircraft maintenance is to consult with maintenance managers and become familiar with the various facets of the attached MEDA checklist.); c) Developing and maintaining effective working relationships with: 1) Manager accountable for aircraft maintenance (and the manager responsible for quality assurance) for integrating the maintenance accident prevention programme into the overall corporate safety management system; and 2) Potential technical advisers through formal and informal relationships; d) Developing a synergy between the company accident prevention programme and the maintenance programme; e) Developing a spirit of cooperation and routine coordination of activities between flight operations and maintenance, particularly on such matters as adequacy of discrepancy reporting, or operating a Flight Data Analysis programme; f) Timely and credible analysis of safety data gathered through the various tools used for hazard identification; and g) Obtaining the participation and commitment of the maintenance department on company Safety Committees. In reviewing the effectiveness of accident prevention programmes in maintenance, APAs might pay particular attention to such issues as: a) Adequacy of maintenance documentation; b) Quality of communications up and down, as well as laterally within the maintenance organization; c) Attention to environmental factors affecting human performance;

20-10

d) Quality of training programmes, both for job-related knowledge and technical skills; e) Error reporting and trend analysis systems aimed at the identification of systemic hazards; f) The means for effecting any necessary changes to reduce or eliminate identified safety deficiencies; and g) An error tolerant and blame-free safety culture.

— — — — — — — —-

20-11

This page intentionally left blank.

20-12

Appendix 1 to Chapter 20 MAINTENANCE ERROR DECISION AID (MEDA) The Maintenance Error Decision Aid (MEDA) provides a structured framework for documenting contributing factors to errors and for recommending suitable error prevention strategies. MEDA is founded on the following basic tenets: a) Maintenance errors are not made on purpose; b) Most maintenance errors result from a series of contributing factors; and c) Many of these contributing factors are part of an airline’s processes, and therefore can be managed. The traditional approach in following-up on maintenance errors was all too often to identify the event caused by a maintenance error and then to administer discipline to whoever made that error. The MEDA process goes much further (without the disciplinary follow-up unless there has been a clear violation of procedures). Having investigated the event caused by a maintenance error and identifying who made the error, MEDA facilitates the following: a) Determination of those factors which contributed to the error; b) Interviewing the responsible persons (and others if necessary) to obtain all the pertinent information; c) Identification of those organizational or system barriers that failed to prevent the error (and the contributing factors as to why they failed); d) Gathering ideas for process improvement from the responsible persons (and others as applicable); e) Maintaining a maintenance error database; f) Analysis of patterns in maintenance errors; g) Implementation of process improvements based on error investigations and analyses; and h) Provision of feedback to all employees affected by these process improvements. MEDA checklists facilitate the interview process (i.e. data acquisition) and data storage in a maintenance error database. With a view to understanding the context in which maintenance errors are committed, following are ten areas where data should be collected. Information includes work-cards, maintenance procedures manuals, service bulletins, engineering orders, illustrated parts catalogues and any other written or computerized information provided either internally or by the manufacturer which is considered necessary for the fulfilment of the aircraft maintenance technician=s job. Some of the contributing factors as to why the information was problematic or was not used include:

20-13

a) Understandability (including format, level of detail, use of language, clarity of illustrations, completeness, etc.); b) Availability and accessibility; c) Accuracy, validity and currency; and d) Conflicting information. Equipment/tools includes all the tools and materials necessary for the correct completion of the maintenance or inspection task. In addition to routine drills, wrenches, screwdrivers, etc., it includes nondestructive test equipment, work-stands, test boxes, and special tools identified in the maintenance procedures. Some of the contributing factors as to how equipment or tools can compromise the performance of the aircraft maintenance technician include: a) Unsafe for use by the technician (e.g. protective devices missing, unstable); b) Unreliable, damaged or worn out; c) Poor layout of controls or displays; d) Mis-calibrated or incorrect scale readings; e) Unsuitable for task; f) Unavailable; g) Cannot be used in intended environment (e.g. space limitations, presence of moisture); h) Instructions missing; and i)

Too complicated.

Aircraft design/configuration/parts. This category includes those aspects of individual aircraft design or configuration that limit the technician=s access for maintenance. In addition, it includes replacement parts that are either incorrectly labelled or are not available, leading to the use of substitute parts. Contributing factors here that may lead to errors by the aircraft maintenance technician include: a) Complexity of installation or test procedures; b) Bulk or weight of component; c) Inaccessibility; d) Configuration variability (e.g. due to different models of same aircraft type or modifications); e) Parts not available or incorrectly labelled; and f) Easy to install incorrectly (e.g. due to inadequate feedback or absence of orientation or flow direction indicators, identical connectors).

20-14

Job/task includes the nature of the work to be completed including the combination and sequence of the various tasks comprising the job. Some of the contributing factors conducive to facilitating maintenance errors in this area include: a) Repetitive or monotonous task; b) Complex or confusing task (e.g. long procedure with multiple or concurrent tasks, exceptional mental or physical effort required); c) New or changed task; and d) Task or procedure varies by aircraft model or maintenance location. Technical knowledge/skills includes the airline process knowledge, aircraft system knowledge and maintenance task knowledge, as well as the technical skills to perform the assigned tasks or sub-tasks without error. Some of the related contributing factors compromising job performance are: a) Inadequate skills in spite of training, trouble with memory items, or poor decision making; b) Inadequate task knowledge due to inadequate training or practice; c) Inadequate task planning leading to interrupted procedures or too many scheduled tasks for time available (e.g. failure to get all necessary tools and materials first); d) Inadequate airline process knowledge, perhaps due to inadequate training and orientation (e.g. failure to order necessary parts on time); and e) Inadequate aircraft system knowledge (e.g. incomplete post-installation test and fault isolation). Many of the foregoing deficiencies call for improved tracking and measuring of the aircraft maintenance technician=s technical performance on the job. Individual factors include the factors affecting individual job performance which vary from person to person. They include those things brought to the job by the individual (e.g. body size/strength, health and personal events) as well as those caused by interpersonal or organizational factors (e.g. peer pressure, time constraints, and fatigue due to the job itself, scheduling or shift work). The MEDA checklist includes possible factors contributing to maintenance errors as follows: a) Physical health including sensory acuity, pre-existing disease or injury, chronic pain, medications, drug or alcohol abuse, etc.; b) Fatigue due to task saturation, workload, shift scheduling, lack of sleep or personal factors; c) Time constraints due to fast work pace, resource availability for assigned workload, pressures to meet aircraft gate time, etc.; d) Peer pressures to follow group=s unsafe practices, ignoring written information, etc.; e) Complacency, perhaps due to over-familiarity with repetitive task or hazardous attitudes of invulnerability or over-confidence;

20-15

f) Body size or strength not suitable for reach or strength requirements, perhaps in confined spaces; g) Personal events such as death of a family member, marital problems, change in financial wellbeing; and h) Workplace distractions, perhaps due to interruptions in a dynamically changing work environment. Environment/facilities include all those factors which can affect not only the comfort of the aircraft maintenance technician, but can also create health or safety concerns which may become a distraction to the maintenance technician. Following are some of the environmental factors that MEDA identifies as being potentially contributory to maintenance errors: a) High noise levels that compromise communications or feedback, or affect concentration, etc.; b) Excessive heat affecting technician=s ability to physically handle parts or equipment or causing personal fatigue; c) Prolonged cold that affects sense of touch or smell; d) Humidity or rain that affects aircraft, part or tool surfaces, including use of paper documents; e) Precipitation affecting visibility or necessitating bulky protective clothing; f) Lighting insufficient for reading instructions or placards, conducting visual inspections or performing the task; g) Wind affecting ability to hear or communicate, or irritating eyes, ears, nose or throat; h) Vibrations making instrument reading difficult or inducing fatigue in hands or arms; i)

Cleanliness affecting ability to perform visual inspections, compromising footing or grip, or reducing available workspace;

j)

Hazardous or toxic substances affecting sensory acuity, causing headaches, dizziness or other discomfort, or requiring wearing of awkward protective clothing;

k) Power sources that are inadequately protected or marked; l)

Inadequate ventilation causing personal discomfort or fatigue; and

m) Workspace too crowded or inefficiently organized. Organizational factors include such factors as internal communication with support organizations, the level of trust that is established between management and maintenance technicians, awareness and buy-in to management=s goals, union activities, etc. All can affect the quality of work – and therefore the scope for maintenance error. Following are some of the organizational factors that MEDA identifies as being potentially contributory to maintenance errors: a) Quality of support from technical organizations which is inconsistent, late or otherwise poor;

20-16

b) Company policies that are unfair or inconsistent in their application, inflexible in considering special circumstances, etc.; c) Company work processes, including inappropriate SOPs, inadequate work inspections, outdated manuals, etc.; d) Union action that becomes a distraction; and e) Corporate change (e.g. restructuring) creating uncertainty, relocations, lay-offs, demotions, etc. Leadership and supervision are tightly linked to organizational factors. Although supervisors do not normally perform the maintenance tasks, they can contribute to maintenance errors through poor planning, prioritizing and organizing of job tasks. Supervisors and management must provide a vision of where the maintenance function is headed and how it is going to get there; in their daily activities they must Awalk the talk@, i.e. their acts must match their words. Following are some areas where weaknesses in leadership and supervision can create a work environment conducive to maintenance errors: a) Inadequate planning or organization of tasks affecting availability of time or resources to complete work properly; b) Inadequate prioritization of work; c) Inadequate delegation or assignment of tasks; d) Unrealistic attitude or expectations leading to inadequate time to complete job; e) Excessive or inappropriate supervisory style, second-guessing technicians or failing to involve them in decisions affecting them; and f) Excessive or aimless meetings. Communication refers to any breakdown in (written or oral) communication that prevents the maintenance technician from getting the correct information regarding a maintenance task in a timely manner. Following are some MEDA examples of interfaces between employees where breakdowns in communication occur, thereby creating the potential for maintenance errors: a) Between departments — incomplete or vague written direction, incorrect routing of information, personality conflicts, or failure to pass timely information; b) Between mechanics — failure to communicate at all; miscommunication due to language barriers, use of slang or acronyms, etc.; failure to question when understanding is in doubt; or failure to offer suggestions when change is needed; c) Between shifts — inadequate turnovers due to poor (or rushed) verbal briefings, inadequate maintenance of records (job boards, check-off lists, etc.); d) Between maintenance crew and lead — when the lead fails to pass important information to the crew (including inadequate briefing at start of shift, or feedback on performance); the crew fails to report problems or opportunities to the lead; roles and responsibilities are unclear;

20-17

e) Between lead and management — when management fails to pass important information to the lead (including discussion of goals and plans, feedback on work completed, etc.); the lead fails to report problems or opportunities to management, etc.; and f) Between the flight crew and maintenance — vague or incomplete logbook write-up; late notification of defect; ACARS/data link not used, etc.

__________________

20-18

Appendix GLOSSARY OF TERMS AS USED IN THIS MANUAL MEANING

TERM Accident

An unintended event that causes death, injury, environmental or material damage. (See Annex 13 — Aircraft Accident and Incident Investigation for ICAO definition of an aircraft accident.)

Accident prevention

Accident prevention is the systematic application of practices and techniques aimed at the prevention (or at least a reduction) of accidental losses.

Accountability

Accountability calls those with responsibilities for meeting assigned objectives into account. Accountability may be held by those “responsible” or it may be assigned to someone else who is accountable for ensuring that subordinates have fulfilled their responsibilities.

Acceptable risk

That part of the assessed risk that is allowed to persist without further risk control action. Acceptable risk may be the residual risk after risk control measures are taken. (See Tolerable risk and ALARP)

ADREP

ICAO Accident/Incident Data Reporting Programme whereby States report to ICAO accidents and selected incidents in accordance with Annex 13.

ALARP

ALARP means that risk in a particular activity has been reduced to a level “As Low as Reasonably Practicable”, taking into account time, cost and difficulty of further reducing or eliminating the risk. (See Tolerable risk and Acceptable risk.)

Aggregate data

Data grouped according to some criterion and combined using mathematical or statistical methods (e.g. sum, count, average, standard deviation).

AIRS

Aircrew Incident Reporting System (developed by Airbus) to assist airlines in establishing their own confidential incident reporting system.

Air Carrier

An organization that undertakes — either directly, by lease or some other arrangement — to engage in air transportation.

AMO

Aircraft Maintenance Organization.

Analysis

The process of organizing facts to help establish validity and logic; to establish causal and contributory factors; and to support inferences and judgements (conclusions).

Analytical tool

An analytical tool is (usually) a software-based/computerized application used in one or more analytical methods, e.g. Excel.

A-1

APA

The Accident Prevention Adviser is designated by the organization to serve as an adviser to management on all aspects of safety on a full or part-time basis. May also be referred to as the flight safety officer or safety manager.

APM

Airport Manager.

ASRS

US Aviation Safety Reporting System collects, analyses and responds to voluntarily submitted aviation safety incident reports.

ATA

Air Transport Association of America, an association of airlines (United States)

ATC

Air Traffic Control. A service provided to promote the safe, orderly and expeditious flow of air traffic.

ATS

Air Traffic Services, including ATC, flight services, etc.

Audit

A form of inspection. (See Safety audit)

Auditor

A person trained and qualified to perform an audit.

Authority

Authority includes the power to commit resources, spend money and otherwise bring about change as an inherent part of line management.

Briefing

Briefings enhance the situational awareness of team members by communicating duties, standardizing activities, and ensuring that all team members understand the plan of action. Briefings are an integral part of SOPs.

BASIS

British Airways Safety Information System. A PC-based system, originally designed as a company incident reporting system which has become widely used for collecting and managing aviation safety information.

Bias

A tendency to apply a particular response regardless of the situation, e.g. partiality.

CAST

Commercial Aviation Safety Team (United States).

Cause

Something that precipitates an event or condition or is a reason for an action or condition.

Checklist

Checklists guard against vulnerabilities in human performance by providing a framework for verifying that all necessary actions are (or have been) carried out appropriately. Checklists are an integral part of SOPs.

CFIT

Controlled Flight Into Terrain. Collision with terrain or obstacles with a fully controllable aircraft.

CHIRP

UK Confidential Human Factors Incident Reporting Programme. A confidential reporting system which complements the UK Mandatory Occurrence Reporting system.

A-2

Contributory factor

An unsafe act or condition which was a necessary pre-condition for an accident or an incident.

Corrective action

Corrective action is a measure taken to reduce or eliminate any condition that may have an adverse effect on safety or quality.

Critical event

An event involving high risk.

CRM

Crew Resource Management.

Deductive reasoning

A “top down” approach of analysis logic. Deductive reasoning moves from a general proposition to establish how that proposition can explain more specific aspects. Used in fault tree analysis.

Defence analysis

Defence analysis is the process of assessing the adequacy of physical and/or administrative defences in place to protect people, property or the environment from specific hazards.

De-identified data

Data from which any identifying information that could be used to associate it with a particular flight, or flight crew has been removed.

Deviation

A deviation is an event triggered by a FDA system. It is considered as a departure from training and/or operating standards. (See Exceedances)

DFDR

Digital Flight Data Recorder. All current flight data recorders record in digital format, thus the term has been superseded by Flight Data Recorders (FDR).

EASA

European Aviation Safety Agency.

Environment

Environment includes everything external to a system which can affect or be affected by the system. It is the aggregate of all operational and ambient conditions and objects that can affect the development, operation, or maintenance of a system.

ECCAIRS

European Coordination Centre for Aviation Reporting Systems. A programme developed in Europe for collecting and analysing accident and incident reports with a view to the early detection of potentially hazardous situations.

Error

See Human Error.

ERP

Emergency Response Plans provide the basis for a systematic approach to managing an organization’s affairs in the aftermath of a significant unplanned event.

Exceedances

Exceedances represent occurrences in which pre-determined limits of aircraft parameters have been exceeded. They may be categorized at different levels based on the degree to which those limits were exceeded. Exceedances are normally tracked in FDA programmes. See Deviation.

FAA

United States Federal Aviation Administration.

A-3

Failure

A loss of function or malfunction of a system or part thereof.

FDA (ICAO term)

Flight Data Analysis programme: A proactive and non-punitive programme for gathering and analysing data which have been recorded in flight to improve flight crew performance, operating procedures, air traffic control procedures, air navigation services, or aircraft maintenance and design.

FDM (UK term)

Flight Data Monitoring. The UK and European term for FDA. A system capable of analysing recorded aircraft parameters, converting and processing the data to detect safety-related events.

FDR

Flight Data Recorder. Required recording equipment designed for post-crash analysis. See DFDR.

FSF

Flight Safety Foundation.

FOQA (US term)

FAA Flight Operational Quality Assurance Program. The US term for FDA as a proactive safety tool.

GAIN

Global Aviation Information Network. GAIN provides a cooperative framework for promoting and facilitating the voluntary collection and sharing of information to improve aviation safety.

Harm

An undesired result involving damage or injury.

Hazard

A hazard is any situation or condition which has the potential to cause injury or loss of life, property or environmental damage.

Hazard identification

The act of identifying unsafe conditions which create and/or facilitate unsafe acts. It is the process of determining what can go wrong, why and how.

Human error

An act that through ignorance, deficiency, or accident departs from or fails to achieve what should be done. It includes unintentional behaviour resulting in a lapse of memory or slip in performing an action, or an intentional behaviour resulting in a mistake.

IATA IFALPA

International Air Transport Association, a trade association of many larger airlines. International Federation of Airline Pilots Association.

IFATCA

International Federation of Air Traffic Controller Association.

ISASI

International Society of Air Safety Investigators.

Incident

An aviation occurrence that does not meet the injury and damage criteria to qualify it as an accident. (See Annex 13).

Inductive reasoning

Inductive reasoning is the process of moving from observations of specific behaviour to more generalized concepts. It is a “bottom up” logical analysis, inferring broader conclusions from specific known information.

A-4

Inspection

An inspection is the process of examining, checking or looking at a product or activity and comparing it with a standard. (See Audit)

Internal audit

An audit carried out by an Operator to evaluate its own performance.

Latent

An existing condition (which may not be presently apparent) and which may have an effect at a later time.

Liveware

Liveware is a term associated with the SHEL Model. Liveware represents the human element in a safety system.

LOSA

Line Operation Safety Audits are a method for gathering safety-related information by observing crew performance during routine operations. LOSA is a risk management tool for the proactive identification of hazards.

MEDA

Maintenance Error Decision Aid is a tool for analysing organizational factors which contribute to human errors in the maintenance domain.

Methodology

Methodology is an analytical approach or process involving a particular procedure or set of procedures. Methodologies may or may not be automated.

Mitigation

Steps taken to control or prevent a hazard from causing harm and to reduce the risk to an acceptable (or at least tolerable) level.

Non-compliance

The failure to fulfil a specific regulatory requirement.

Non-conformance

The failure to fulfil a specific Company requirement or standard.

Non-punitive

The absence of disciplinary action (regulatory, personnel, administrative) against an individual. Does not apply to flagrant or wilful breaches of regulations, procedures etc.

Occurrences

Accidents, incidents (both serious and minor) and events arising from some element(s) of the aviation system not performing to the expected level.

OFSH

Operators Flight Safety Handbook. The OFSH is a collaborative effort by GAIN to provide a single, easily referenced document providing necessary information for setting up and maintaining an SMS.

PEAT

Procedural Event Analysis Tool. PEAT is an investigative tool, useful in determining why an event occurred and whether a procedural deviation was involved.

Parameters

Used in the context of FDA. Parameters are the specific types of measured sensory data recorded by an FDR or QAR.

Probability

The likelihood that an event will occur. Probability may be expressed quantitatively (e.g. one chance in a million) or qualitatively (e.g. unlikely to occur).

A-5

Programme review

A systematic evaluation of the effectiveness of the programme elements to ensure a coherent and integrated accident prevention programme.

Public safety

The protection of life, health, property and the environment.

Quality assurance

A planned and systematic pattern of actions necessary to provide adequate confidence that an operation, service or product conforms to established requirements.

QAR

Quick Access Recorder. A secondary flight recorder with a removable recording medium for quick and easy access. It stores flight-recorded data in an expanded data frame, greatly increasing the resolution and accuracy of the ground analysis programme (e.g. FDA, FDM, FOQA).

Qualitative analysis

Those analytical processes that assess risk in a subjective, non-numerical manner.

Quantitative analysis Responsibility

Those analytical processes that apply mathematical methods to assess risk. A function specifically assigned to an individual, team or organizational section.

Residual risk

The risk remaining after risk control measures have been implemented.

Risk

Risk is an expression of the impact of an undesired event arising from a specified hazard in terms of severity and probability.

Risk assessment Risk control

Assessment of a system or component to establish the level of risk that exists. The application of measures to reduce or eliminate the assessed risks of a hazard.

Risk management

The process of identifying risks, assessing their implications, deciding on a course for mitigating the risk and evaluating the results.

Risk mitigation

See Mitigation.

Safety

Safety is an ideal state in which the risk of harm (to persons) or damage to property or the environment is reduced to an acceptable level through a continuing process of hazard identification and risk management.

Safety analysis

All associated analysis methods, processes and/or techniques to systematically evaluate safety-related risks.

Safety assessment

A systematic, comprehensive evaluation of a particular organizational element, process or system to show that the safety requirements are met. They can facilitate the identification of hazards.

Safety audit

Safety audits provide a systematic examination of selected processes and their results to verify compliance and standardization. They are conducted either by internal or external auditors and are an integral part of an SMS.

A-6

Safety culture

Safety culture consists of the shared beliefs, values, practices and attitudes of an organization towards safety. Corporate safety culture is the atmosphere created by management that shapes workers’ attitudes towards safety.

Safety deficiency

A safety deficiency is a hazard where the risk has been assessed as unacceptable and which warrants measures to reduce or eliminate the risk.

Safety management

Safety management is that part of the management function which determines its safety principles, policy and procedures and sets safety objectives and goals. Typically, safety management includes strategies detailing how these will be fulfilled; the role of individuals in respect to safety; the methods used to measure and record the level of safety in an organization; the tools available to monitor ongoing safety levels and safety issues; dissemination of safety lessons learned.

Safety Management System (SMS)

An SMS sets the framework for an organization to manage its safety interests. It systematically defines all the activities by which safety management is undertaken by an organization to secure an acceptable level of safety performance — one which as a minimum meets the provisions of regulatory safety requirements.

Safety monitoring

A systematic action conducted to detect changes affecting the safety system with the objective of identifying that acceptable safety levels are being met.

Safety oversight

Safety oversight is an integral part of an SMS through which the degree of conformance with State and company requirements can be determined. Safety oversight involves regular (if not continuous) monitoring of all aspects of an organisation’s safety-related activities. It also provides a source for hazard identification, validates the effectiveness of safety actions taken and provides continuing evaluation of safety performance.

Safety policy

Safety policy is a statement of the organization’s approach to safety management. Management’s adherence to the safety policy demonstrates their commitment to safety.

Safety promotion

Safety promotion is the means by which safety issues are communicated to ensure a culture of safe work practices within the organization.

Safety records

Safety records include all information about events, or hazards that are maintained as a basis for risk management and safety analysis and for demonstrating the effective operation of the safety management system.

Safety study

Safety studies are a formal analysis of a pervasive safety issue in the broadest possible context. They may be undertaken at the level of State administrations, airlines, manufacturers, academia, or professional and industry associations.

Safety survey

A safety survey is a systematic review of particular activities and facilities, conducted to obtain general safety information not otherwise available. Safety surveys typically involve checklists and informal confidential interviews, usually conducted independent of routine inspections and audits by government or company management.

A-7

SOP

Standard Operating Procedures provide guidance and specify a sequence of tasks for carrying out a job in a safe, efficient, logical and predictable manner. They unambiguously express what to do, when to do it, how and by whom, and what type of feedback is to be used to verify that necessary actions have been completed.

SDR

Service Difficulty Report. A system for reporting maintenance defects to the FAA (U.S.).

Severity

The potential consequences of a hazard.

SHEL Model

SHEL is a model depicting the multi-faceted nature and interdependencies of Human Factors including: Software, Hardware, Environment and Liveware.

STEADES

Safety Trend Evaluation Analysis and Data Exchange System — a safety data sharing programme of IATA.

Stakeholder

Any person or organization with a particular interest in aviation safety.

System

A combination (at any level of complexity) of physical components, procedures and human resources organized to perform a specific function.

System safety

The application of engineering and management principles, criteria and techniques to optimize safety within the constraints of operational effectiveness, time and cost throughout the life of the system.

Tolerable risk

Tolerable risks are those which may be undesirable, but once reduced to a level as low as reasonably practicable (ALARP) are acceptable. Further risk reduction is impracticable, the costs exceeding the improvement to be gained. (See ALARP)

Unacceptable risk

Any risk which cannot be tolerated is unacceptable. It is the part of the assessed risk that must be eliminated or controlled.

USOAP

ICAO Universal Safety Oversight Audit Programme.

Validation

Validation is the evaluation process for determining that the requirements specified are correct and complete and that they fully meet the users’ needs.

Verification

The evaluation of the results of a process to ensure the correctness and consistency with respect to the inputs and standards provided to that process.

___________________

A-8

BIBLIOGRAPHY

Accident/Incident Reporting (ADREP) Manual (Doc 9156), 1987 Airbus Safety Strategy Airport Services Manual, Part 7 — Airport Emergency Planning (Doc 9137) Airworthiness Manual (Doc 9760) Annex 6 — Operation of Aircraft Annex 8 — Airworthiness of Aircraft Annex 11 — Air Traffic Services Annex 13 — Aircraft Accident and Incident Investigation Annex 14 — Aerodromes Aviation Maintenance Human Factors (JAA JAR 145), CAP 716 CASA Aviation Safety Management: An Operator’s Guide to Building a Safety Program CASA — Safety Management Systems: Getting Started Flight Safety Foundation Aviation Safety: US Efforts to Implement Flight Operational Quality Assurance Programs, July – September 1998 Flight 2005: A Civil Aviation Safety Framework for Canada (TP 13521), Transport Canada Flight Attendant Manual Standard (TP 12295), Transport Canada Flight Attendant Training Standard (TP 12296), Transport Canada Flight Safety Managers Handbook – Airbus, March 1999(1) Global Aviation Information Network (GAIN) Helmreich, R.L., Klinect, J.R., & Wilhelm, JA (1999) “Models in Threat, Error and CRM in Flight Operations” in Proceedings of the Tenth International Symposium on Aviation Psychology (pp 677-682), Columbus, OH: The Ohio State University Human Factors Guidelines for Safety Audits Manual (Doc 9806) Human Factors Training Manual (Doc 9683) Human Performance and Safety Consultants Inc., 2001

1

“Introduction to Safety Management Systems” (TP 13739 E), Transport Canada, April 2001 Lautman, L.G. and P.L. Gallimore “Control of Crew-Caused Accidents” (reproduced in Flight Safety Foundation: Flight Safety Digest, October 1989) Line Operations Safety Audit (LOSA) Manual (Doc 9803), 2002 Management & Organization: Management’s Role in Safety (TP 12883), Transport Canada, November 1996 Manual of Aircraft Accident Investigation (Doc 6920) Manual of Aircraft Accident and Incident Investigation (Doc 9756) Manual of Investigations, Transportation Safety Board of Canada Operational Flight Safety Handbook (OFSH) Operational Flight Safety Handbook – Cabin Safety Compendium Operational Flight Safety Handbook (OFSH) (GAIN) Issue 1, June 2000 Preparation of an Operations Manual (Doc 9376) Procedures for Air Navigation Services — Air Traffic Management (PANS-ATM, Doc 4444) Reason, J., Collective Mistakes in Aviation: ‘The Last Great Frontier’, Flight Deck, Summer 1992, Issue 4 Report of the Twelfth Meeting of the Visual Aids Panel (Doc 9603) Risk Management & Decision Making (TP 13095), Transport Canada Risk Management: Guidelines for Decision Makers, CAN/CSA Q850, Canadian Standards Association “Safety Culture” (safety series No. 75 – NSAG) published by the International Atomic Energy Agency, Vienna, 1991. (Adapted and reproduced at Appendix 1 to Chapter 3 of Human Factors Guidelines for Safety Audits Manual (Doc 9806) Safety Management Systems for Air Traffic Management: A Guide to Implementation, CAP 730, UK CAA, 2002 Safety Management Systems for Commercial Air Transport Operations: A Guide to Implementation, CAP 712, UK CAA, May 2001 Safety Management Systems for Commercial Air Transport Operations — Safety Regulation Group (SRG) CAP 712, UK CAA, May 2001 Safety Management Systems for Flight Operations and Aircraft Maintenance Organizations (TP 13881), Transport Canada, 2002

2

Safety Management Systems: SRF Policy and Guidelines — Safety Regulation Group (SRG), UK CAA Safety Oversight Audit Manual (Doc 9735) Safety Oversight of the Civil Air Navigation System — A Framework (TP 13142), Transport Canada System Safety Handbook: Practices and Guidelines for Conducting System Safety Engineering and Management (FAA Dec 30, 2000) Towards an Integrated Approach for Ramp Safety Management by Hans Oude Egberink, University of Gronigenin Application of Psychology to the Aviation System (Vol I) Avebury Aviation, Aldershot, UK The Management of Safety on the Airport Ramp by Nick MacDonald and Ray Fuller in Aviation Psychology in Practice Avebury Technical, Aldershot, UK, 1994 “The Management of Safety — Guidance to Aerodromes and Air Traffic Service Units on the Development of Safety Management Systems”, Civil Aviation Authority, London, September 1998 Training Manual, Part E-1 — Cabin Attendants’ Safety Training Manual (Doc 7192) Wood, Richard H., Aviation Safety Programs: A Management Handbook, 3rd Edition, Englewood, Co.: Jeppesen, 2003

___________________

3