Data Communications (Data Comm) Operational Performance Metrics Plan Segment 1 – Phase 1 and Phase 2

February 22, 2012

Version 3.0

SIGNATURE PAGE

Quiu^M \ ) - J C M ^ O Karen Davis

Z-ZZ-ZQ17Date

Data Comm Metrics Team Lead

Sandra Anderson

Date

Manager, Air/Ground Data Communications Team

Federal Aviation Administration 800 Independence Avenue, SW Washington, D.C. 20591

Data Comm Operational Performance Metrics Plan

Page 1

Change History Version 0.06 0.10 0.12

Date 03/31/2010 05/05/2010 05/18/2010

0.2

07/09/2010

0.21

08/05/2010

0.30

08/26/2010

0.31

09/07/2010

1.0

09/17 – 9/28 /2010

1.1 1.2 1.3 2.0 2.1 3.0

1/13/2011 2/24/11 9/20/2011 11/18/11 2/17/12 2/22/12

Description of Changes Initial document Revised Following Team Input Additional Revisions, Edits Incorporated NextGen Systems Analysis Office Input & Additional Linkage Sections Incorporated Additional Input, Refined Metrics, Added Paragraph Describing Metrics Analysis Team Merged Section 5, Specific Metrics with Section 3, Metrics Categories; Changed Cost Effectiveness Category to Productivity Revisions incorporating team crosswalk review between metrics plan and summary matrix Final revisions from Leads and SME’s, updated IOC dates based on Implementation Schedule V11_07/22/2010 Separated Metrics out by phase Incorporated Metrics Team Comments Incorporated PM and Team comments Final revisions made; ready for PM signature Incorporated additional edits from PM Made final revisions; ready for PM signature

Data Comm Operational Performance Metrics Plan

Page 2

Table of Contents TABLES ......................................................................................................................................... 4 FIGURES ....................................................................................................................................... 4 EXECUTIVE SUMMARY .......................................................................................................... 5 SECTION 1.0

INTRODUCTION......................................................................................... 6

1.1 SCOPE ..............................................................................................................................................................6 1.2 ROLES AND RESPONSIBILITIES .........................................................................................................................7 1.3 DATA COMM ALIGNMENT WITH STRATEGIC PLANS AND REPORTS ..................................................................8 1.3.1 FAA Destination 2025 Alignment ..........................................................................................................8 1.3.2 NextGen Implementation Plan Alignment............................................................................................ 10 1.3.3 OMB Exhibit 300 Alignment ................................................................................................................ 11 1.4 NEXTGEN PERFORMANCE FRAMEWORK ........................................................................................................ 13 1.5 DATA COMM, SHARED BENEFITS, AND TRAJECTORY BASED OPERATIONS .................................................... 14

SECTION 2:

OPERATIONAL PERFORMANCE METRICS METHODOLOGY ...... 14

2.1 MECHANISM METRICS ................................................................................................................................... 17 2.1.1 Equipage and Usage ............................................................................................................................ 17 2.1.2 Traffic Management Initiatives (TMI) ................................................................................................. 18 2.2 USER IMPACT - AIRCRAFT THROUGHPUT AND EFFICIENCY METRICS ............................................................ 18 2.2.1 Delay Savings Metrics ......................................................................................................................... 18 2.2.2 Aircraft Throughput Metrics ................................................................................................................ 19 2.2.3 Flight Efficiency Metrics ..................................................................................................................... 20 2.3 PRODUCTIVITY METRICS................................................................................................................................ 21 2.4 SAFETY METRICS ........................................................................................................................................... 21 2.5 ENVIRONMENTAL METRICS ........................................................................................................................... 22 2.6 VARIABLES .................................................................................................................................................... 23 2.7 QUALITATIVE PERFORMANCE INDICATORS .................................................................................................... 23 2.8 BASELINING ................................................................................................................................................... 24

SECTION 3: 3.1 3.2 3.3 3.4

REPORTING .................................................................................................. 24

FAA EXECUTIVE LEVEL METRICS ................................................................................................................. 24 REPORTING .................................................................................................................................................... 25 PERIODIC REPORTS ........................................................................................................................................ 26 AD HOC REPORTS .......................................................................................................................................... 26

SECTION 4:

SEGMENT ONE PHASE ONE METRICS ................................................. 26

4.1 TOWER SPECIFIC METRICS ............................................................................................................................. 26 4.1.1 Tower Equipage and Usage Metrics .................................................................................................... 26 4.1.2 Tower Specific User Impact - Throughput and Efficiency Metrics ...................................................... 27 4.1.3 Tower Specific Productivity Metrics .................................................................................................... 28 4.1.4 Specific Safety Metrics ......................................................................................................................... 29

Data Comm Operational Performance Metrics Plan

Page 3

4.1.5

Tower Specific Environmental Metrics ................................................................................................ 30

SECTION 5:

SEGMENT ONE PHASE TWO METRICS ................................................ 30

5.1 EN ROUTE SPECIFIC METRICS ........................................................................................................................ 31 5.1.1 En Route Specific Mechanism Metrics................................................................................................. 31 5.1.2 En Route Specific User Impact – Throughput and Efficiency Metrics................................................. 31 5.1.3 En Route Specific Productivity Metrics ............................................................................................... 33 5.1.4 Specific Safety Metrics ......................................................................................................................... 34 5.1.5 En Route Specific Environmental Metrics ........................................................................................... 34

SECTION 6:

TIMELINE ...................................................................................................... 35

APPENDIX A: METRICS DATA SOURCES ......................................................................... 36 APPENDIX B: VARIABLES .................................................................................................... 38 APPENDIX C: ACRONYMS .................................................................................................... 40 APPENDIX D: DATA COMM OPERATIONAL PERFORMANCE METRICS MATRIX ....................................................................................................................................................... 42

Tables Table 1: FAA’s Destination 2025 Aspirations................................................................................ 9 Table 2: Data Comm OMB Performance Reference Model Mapping ......................................... 13 Table 3: Tower Equipage & Usage Metrics ................................................................................. 27 Table 4: Tower Throughput & Efficiency Metrics ....................................................................... 28 Table 5: Tower Productivity Metrics ............................................................................................ 29 Table 6: Safety Metrics for Tower and En Route ......................................................................... 30 Table 7: Tower Environmental Metrics ........................................................................................ 30 Table 8: En Route Equipage and Usage Metrics .......................................................................... 31 Table 9: En Route Throughput & Efficiency Metrics .................................................................. 33 Table 10: En Route Productivity Metrics ..................................................................................... 33 Table 11: Safety Metrics for Tower and En Route ....................................................................... 34 Table 12: En Route Environment Metrics .................................................................................... 35

Figures Figure 1: Data Comm Benefits Mechanisms ................................................................................ 11

Data Comm Operational Performance Metrics Plan

Page 4

Executive Summary The Federal Aviation Administration (FAA) has embarked on an initiative, known as the Next Generation Transportation System (NextGen), which will lead to a comprehensive overhaul of the National Airspace System (NAS). Data Communications (Data Comm) is a key transformational program within the NextGen effort. The Data Comm Program will implement data communication services that will increase controller productivity, contribute to improved flight times, reduce delays, and improve safety through the reduction of voice communication errors. Data Comm will enable NextGen advanced capabilities not possible using the current voice-based Air Traffic Control (ATC) system. The program consists of automation enhancements for the generation and exchange of air traffic control messages, and the communications data link between aircraft and ground automation. Data Comm will use a segmented, incremental approach to implement and enable these services. Segment One (S1) services will be implemented in two phases. Phase One will include the enhancement of Tower communications through the implementation of Tower Departure Clearance Service (DCL). Phase Two will include expanded DCL service and En Route services. Delivery of these services is dependent upon the availability of other NextGen systems and future releases of the En Route Automation Modernization (ERAM) system. This plan provides an overview of how the Data Comm Program will utilize quantitative measures to assess the operational performance of Data Comm services as they are implemented within the NextGen framework. The Operational Performance Metrics section of this plan is divided into sub-sections by phase and covers Segment One of the program. The subset of these metrics that are to be considered executive-level metrics are listed below: • • • • •

User Equipage Delays Airspace Throughput Controller Workload System Risk Event Rate (SRER)

This document will evolve as new information related to existing metrics becomes available or to add new metrics to address monitoring needs for future phases or segments. This plan will constitute as the “official set” of metrics used to evaluate the operational performance of the Data Comm service by our customers, stakeholders and the flying public.

Data Comm Operational Performance Metrics Plan

Page 5

Section 1.0 Introduction The FAA Air Traffic Organization (ATO) provides air traffic management services for aircraft operating within the NAS. Data Comm will significantly influence the delivery of these services. This Operational Performance Metrics Plan (OPMP) describes how the changes in communications services as well as the resulting changes in the NAS performance will be measured and analyzed. The OPMP defines what metrics, processes, and systems are required to support the capture and analysis of performance metrics over the course of the Data Comm Program’s operational life cycle. A critical requirement for development of the OPMP is that it is aligned with the strategic objectives of the FAA’s Destination 2025 (D2025) and the 2011 NextGen Implementation Plan. The focus of this Plan is to measure operational performance of Data Comm after implementation. The cost of implementing Data Comm and the tracking of the implementation schedule is outside the scope of this plan; assessment of cost and schedule performance will be documented in the Data Comm Metrics & Analysis Plan and performed as part of the program’s Earned Value Management (EVM) activities. While this plan provides a comprehensive overview, it is likely there will be additional metrics required in the future for Segment Two (S2) functionality. Fortunately, many robust databases that will allow for analyses beyond what are described in this plan already exist. The operational performance metrics generated will enable Data Comm Program Management to provide objective evidence to FAA, Department of Transportation (DOT), Office of Management and Budget (OMB), and system users that the Data Comm Program is achieving its operational performance goals. Additionally, metrics captured for Segment One will provide useful information for future Data Comm operational improvements and input into Segment Two requirements and design. 1.1 Scope The scope of this effort is to provide a comprehensive suite of operational performance metrics that meet the needs of Data Comm customers and stakeholders. The primary customers and stakeholders include but are not limited to: • Data Comm Program Management • FAA and DOT Executive Management • Other Federal Government: – Office of Management and Budget (OMB) – US Government Accountability Office (GAO) – Elected Officials: President and Congress • System Users: Airlines, General Aviation, Military Data Comm Operational Performance Metrics Plan

Page 6

• • •

Labor Organizations Industry Flying public

A metric is a quantitative measure of system performance. Data Comm metrics will enable analysts and managers to do the following: • • • • • •

Monitor program status against performance goals established with Business Case Analysis (BCA), OMB Exhibit 300, NextGen Implementation Plan and the D2025 Reconcile and validate the integrity of critical program data Monitor long-term trends in overall system performance Evaluate the performance and impact of program activities and enhancements Identify and support the need for future program enhancements Make informed and timely changes in Data Comm program implementation/operation

The operational metrics in this Plan address a wide range of needs, interests, and operational performance questions to support the goal of providing an optimized Data Comm service for customers, stakeholders, and the flying public. The operational performance metrics in this Plan provide a single source of metrics that reflect various overlapping and often consistent goals among these customers and stakeholders. 1.2 Roles and Responsibilities The Data Comm Program Control Lead is responsible for the operational performance metrics measurement and analysis process. The Program Control Lead will coordinate on a regular basis with the Program Manager and other team functional leads to establish designated metrics. The Metrics Team, managed by the Program Control Lead, is responsible for operational performance metrics analysis, development, monitoring, updating, and reporting. This Plan has been developed by the Data Comm Operational Performance Metrics team and includes a variety of participants including FAA personnel, aviation industry Subject Matter Experts (SME), and Data Comm support contractors. Important Team functions for the customer/user community are to assist in identifying potential metrics and analytical methods, to assist in setting priorities, and setting realistic program targets. The overall roles of team members can be divided into three broad categories, as outlined as follows: Development • Evaluate system capabilities and describe the benefit mechanisms • Identify potential metrics and analytical methods Data Comm Operational Performance Metrics Plan

Page 7

• • •

Identify or develop data sources Develop metric or analytical methodology Develop baselines as required

Management and Integration • Identify gaps in existing metrics or analytical capability suites • Identify and evaluate potential new metrics or capabilities • Validate and evaluate emerging metrics and capabilities • Approve metrics for inclusion in the Data Comm metrics suite • Assist in setting targets for approved metrics • Report results and status of metrics and analysis development activities Oversight • Identify potential operational metrics (including those already identified during Data Comm development) • Select metrics, set priorities • Assist in setting targets for approved metrics • Monitor progress towards targets • Scope Operational Performance Metrics Plan • Recommend program changes intended to improve program performance, based on the routine evaluation of the data The implementation of this Plan will require members of the program metrics team to conduct complex analyses. Roles of the Data Comm Program Metrics Analyst are outlined as follows: • • • • •

Querying a wide variety of data sources often using relational database techniques Analyzing data normalizing for many variables Interpreting results with thorough understanding of the NAS Keeping Program Manager informed of the results Reporting results to a wide variety of audiences

1.3 Data Comm Alignment with Strategic Plans and Reports FAA’s Destination 2025, NextGen Implementation Plan, the OMB Exhibit 300 provide guidance and define performance objectives and criteria to assess the benefits derived from FAA capital investments. Common to all of these documents is the need to evaluate the improvements in capacity, efficiency and access to the NAS in the near term and alignment with the NextGen endstate vision in the long term. This plan will be updated as new performance objectives or criteria are established by FAA or OMB. 1.3.1

FAA Destination 2025 Alignment

Data Comm Operational Performance Metrics Plan

Page 8

Destination 2025 Plan is the FAA’s strategic plan that describes a vision where there is an increase in airspace efficiency and safety around the globe. Costs are minimized for operators and passengers and there are no environmental impacts. The plan describes five aspirations or goals. Data Comm will indirectly contribute to the achievement of each of them, but most closely links to the aspiration "Aviation Access through Innovation" which is intended to enhance the flying experience of the traveling public and other users by improved access to and increased capacity of the nation’s aviation system. The Objective is to ensure airport and airspace capacity are more efficient, predictable, cost-effective and matched to public needs. The table below summarizes the aspirations of the Destination 2025 plan. Aspiration

Description

Move to the Next Level of Safety

Achieving the lowest possible accident rate and always improving safety, all users of our aviation system can arrive safely at their destinations. We will advance aviation safety worldwide.

Create a Workplace of Choice

Create a workplace of choice marked by integrity, fairness, diversity, accountability, safety and innovation. Our workforce will have the skills, abilities, and support systems required to achieve and sustain NextGen.

Deliver Aviation Access through Enhance the flying experience of the traveling public and other users by improved access to and increased Innovation capacity of the nation's aviation system. Ensure airport and airspace capacity are more efficient, predictable, cost-effective and matched to public. Sustain Our Future

Develop and operate an aviation system that reduces aviation's environmental and energy impacts to a level that does not constrain growth and is a model for sustainability.

Advance Global Collaboration

Achieve enhanced safety, efficiency, and sustainability of aviation around the world. Provide leadership in collaborative standard setting and creation of a seamless global aviation system.

Table 1: FAA’s Destination 2025 Aspirations

Data Comm Operational Performance Metrics Plan

Page 9

1.3.2

NextGen Implementation Plan Alignment

The NextGen Implementation Plan dated March 2011 outlines the FAA strategy to overhaul the airspace system in a continuous rollout of improvements that guide and track air traffic more precisely. Through collaboration with RTCA, FAA outlines its plans to deliver tangible nearterm benefits. The Data Comm Program responds to these challenges by: •

Increasing safety and capacity, saving time and fuel, and decreasing carbon emissions

•

Enabling Tailored Arrivals using optimized profiles with aircraft equipped with Future Air Navigation Systems (FANS) in high altitude airspace down to the runway

•

Providing pre-departure clearances

•

Providing routine and strategic information to flight crews and automate some routine tasks for both pilots and controllers

•

Sending clearances more efficiently to equipped aircraft when weather affects many flights

•

Proposing changes in trajectory or speed to eliminate conflicts via a data message

•

Managing reroutes and route structure changes resulting from changes in winds and weather

•

Negotiating final flight path information with proposed arrival time, sequencing, route, and runway assignments

•

Demonstrating reductions in emissions and fuel consumption.

•

Providing initial capabilities through globally harmonized Data Comm of FANs 1/A+ and through Aeronautical Telecommunications Network (ATN) for En Route

•

Providing conflict resolution advisories for more complex maneuvers with higher equipage

Figure 1 provides examples of the correlation between Data Comm capabilities, metrics, and goals. This Plan focuses on the metrics portion of this linkage while others develop the capabilities and applications.

Data Comm Operational Performance Metrics Plan

Page 10

Segment 1

Technology

Drives

Application

Brings About

Direct Impact Benefits

Metrics

-Tower: DCL enables reroutes - En Route: Routine Comm TFM reroutes

-VDL-2 DLAP -ERAM/TDLS links to DCNS -TDLS Software -D-ATIS storage & delivery -ERAM Automation

- More Efficient Traffic Management - Reduced Delay - Increased Capacity - Fewer Errors - Workload Reduction - Reduced Emissions

- System Usage, TMIs Used - Safety OEs, ODs, PDs, - Throughput, Efficiency, Delay - Productivity - Emissions

Success

Goals

- Reduce OEs, ODs, PDs - Reduce Delay by X - Increase Throughput X - Increase Operations per sector - Reduce CO2 by X Tons

Figure 1: Data Comm Benefits Mechanisms *See Appendix C for acronym listing

1.3.3

OMB Exhibit 300 Alignment

OMB’s Circular A-11 requires that Federal agencies submit to OMB and Congress an Exhibit 300 for each major capital investment or program as an appendix to their annual budgets. The Exhibit 300 provides an overall status of the program including cost, schedule, and performance. The Operational Performance section of the Exhibit 300 was modified significantly for BY13. At present, programs initiate the submission of operational performance data in the year prior to Data Comm Operational Performance Metrics Plan

Page 11

Initial Operating Capability (IOC). Since IOC is targeted for FY 2016, the Data Comm Program will need to submit operational performance data in 2015. Operational metrics are defined and statused in the Operational Performance Table (Part B, Section I.C.1.b) of the Exhibit 300. The template for this table is attached as Appendix E. Extracts from the Exhibit 300 are available to the public on the internet through the OMB IT Dashboard. The initial submission will only include baseline and targets values for selected operational performance metrics. In the following year—and each subsequent year—the actual value for each metric will be compared to the target value to determine a condition of “over the target” or “under the target” for table field “Measurement Condition.” The current requirement is for a program to report a minimum of five metrics field: two “Results-Specific” metrics and three “Activities/Technology-Specific” metrics. The results-specific metrics track operational performance and will be identified among those that are in this plan. A description of the metric will be provided in the “Metric Description” field. Results-Specific metrics are indicators of the effectiveness of the investment in delivering the desired service or support level and must: 1) be appropriate to the mission of the investment and its business owner or Customer, and 2) support the business case justification and could be the foundation of a quantitative approach to defining benefits in a cost-benefit analysis. The activities/technology-specific metrics track technical performance and are outside the scope of this plan. Prior to the Exhibit 300 changes for BY13 noted above, the Data Comm Program identified metrics under the BY12 requirements as listed in Table 2. For BY12 only four metrics were required. Mapping to OMB Performance Reference Model remains as a requirement for submission. These attributes are also listed in Table 2. When required, these metrics along with other metrics referenced in this plan will be reviewed for inclusion in the Exhibit 300 submission. Through the performance metrics specified in Table I.C.1.b, the Data Comm Program will demonstrate alignment with the FAA’s strategic goals and OMB requirements. To ensure compliance, this plan will be updated in response to any new OMB requirements for performance monitoring and assessment.

Data Comm Operational Performance Metrics Plan

Page 12

Measurement Area Customer Results

Measurement Category

Measurement Grouping

Potential OMB Exhibit 300 Operational Performance Metric (Description)

Accuracy of Service Operational errors associated with communications (System Risk Event Rate, Customer Benefit or Product Delivered SRER)

Mission and Services for Business Results Citizens

Air Transportation

Average Daily Airspace Capacity/Efficiency

Technology

Reliability and Availability

Accessibility

Percent of Data Comm targeted aircraft equipage achieved

Processes and Activities

Productivity

Productivity

Data Comm impact on controller workload

Table 2: Data Comm OMB Performance Reference Model Mapping

1.4 NextGen Performance Framework The FAA NextGen Systems Analysis Office has identified a NextGen Performance Framework, which is based on the International Civil Aviation Organization (ICAO) Air Traffic Management (ATM) Concept of Operations (CONOPS) service expectations. The following Key Performance Indicators (KPIs) are included: • • • • • • • • • • •

Access and Equity Capacity Productivity Efficiency Environment Flexibility Global Interoperability Predictability Participation Safety Security

The KPI’s of Productivity, Environment, and Safety are consistent with three of the metrics listed in Section 2. Others appear as subsets of those categories, while two: Global Interoperability and Security are currently outside the scope of this Data Comm metrics effort. The GAO was asked to review the FAA’s metrics for tracking the status of NextGen programs and implementation on NextGen capabilities, the reliability of those metrics, and any limitations. Data Comm Operational Performance Metrics Plan

Page 13

The GAO-10-629 report released in July 2010 identified the need to link individual program metrics and outcome-based performance to the NextGen long-term, higher level impact and benefits gained from the entire NextGen endeavor. The FAA has multiple efforts underway to develop such metrics but is in the early stage of development. The Data Comm Metrics Team will continue to leverage this development as it evolves program specific metrics and outcomes. 1.5

Data Comm, Shared Benefits, and Trajectory Based Operations

Data Comm is a technology enabler for other NextGen programs. As these NextGen programs are implemented the dependency on Data Comm increases. This creates an environment where the allocation of operational benefits to each separate service becomes more complex. When benefits are allocated across multiple services, it will likely mean that metrics will measure ‘application suites’ rather than individual applications or capabilities. One example where this will occur is with the implementation of Trajectory Based Operations (TBO). Future aircraft will be required to fly four-dimensional (4D) trajectories with the additional dimension being timed at specific locations or Required Time of Arrival (RTA) at specific fixes. Automation, both on flight decks and on the surface, will take known information such as aircraft flight plans, wind, and demand and calculate appropriate arrival times at fixes in order to orchestrate aircraft flows into congested locations such as airports or busy corridors. This will ensure all available capacity is used while avoiding too many aircraft in a congested area. It will be necessary for data communications to exist between the surface and aircraft in order to communicate the needed information and the calculation results establishing the trajectory criteria. In today’s system, controllers need to anticipate congestion and take steps to ensure that aircraft do not conflict with one another. This is achieved by controlling the speed of aircraft, adding maneuvers to mitigate conflict, and implementing Traffic Management Initiatives (TMIs) that reduce the number of aircraft approaching a congested area or, more likely a combination of all three. Certain TMIs can end up over-restricting traffic flows, which can cause available capacity to go unused creating additional delay. Trajectory Based Operations will reduce the need for TMIs while also reducing controller workload due to less need to intervene to mitigate potential conflict.

Section 2: Operational Performance Metrics Methodology To understand impacts from Data Comm implementation and use, it will be necessary to do indepth analysis comparing various components of NAS operations after implementation. It will be necessary to have enough time after implementation in order to have sufficient data to provide meaningful results. It takes approximately one year of data collection and analysis in order for meaningful results to be produced. Data Comm Operational Performance Metrics Plan

Page 14

Further, there needs to be enough time after implementation to allow for the ramp-up of the new system. For Data Comm, this ramp-up period is likely to be long due to the need for system users to equip their aircraft to take advantage of the service. This ramp-up period will likely result in the need for analyses over several years to determine performance for those aircraft equipped as well as system performance as equipage increases. Data Comm Segment One will be implemented in two phases. Phase One implementation will provide DCL service to FANS-equipped aircraft in control tower environment. Phase Two implementation will expand DCL service to Aeronautical Telecommunications Network (ATN) equipped aircraft as well as provide En Route Data Comm Services. The five metrics categories described below will apply to both phases of Segment One implementation, however the specific metrics for each phase, while similar, will have some differences in order to focus measures on the specific operations likely to be affected by Data Comm. Most of the metrics described in this Plan will be measured using the data sources identified. However, some will require modeling to obtain results. For example, fuel burn data is not available for direct measurement. However, Base of Aircraft Data Algorithms (BADA) are available that can closely approximate fuel burn using known parameters such as aircraft type, altitude, phase of flight, etc. Emissions, an environmental metric, will also be modeled to obtain results. During the time that Data Comm capabilities are added and aircraft equipped, there will be many other changes occurring in the NAS. This will complicate the analysis process since these other changes will need to be accounted for when trying to determine Data Comm performance. For the purpose of OMB Exhibit 300 reporting, the Data Comm Leadership Team will need to establish performance targets over the program lifecycle. Due to the complexity of the NAS, including the many variables that affect it, these targets will likely need to be stated as a percentage of change targeted rather than absolute values. This will allow variables to be identified and normalized following implementation. Normalization is explained in more detail in Appendix B. Many ongoing data collection efforts within the ATO and robust databases exist that can be used to obtain performance information. Many of these are listed in the appendices section of this Plan. The Data Comm Operational Performance Metrics Team will determine which of these data sources are required based on the metrics definition. Access to the majority of these databases is currently available. However, coordination will be required to gain access to several that are not currently available. The Team will also need to establish data collection and storage requirements for new Data Comm specific data.

Data Comm Operational Performance Metrics Plan

Page 15

The following is a list of criteria for operational performance metrics for Data Comm. The performance metrics have to meet these criteria to be included in the final list of metrics. 1. The metric must be defined and measurable, and have a specific outcome. 2. The metric must have a specific audience(s) or stakeholder(s) that is intended for and that will be the recipient of the output of the metric (e.g., OMB, ATO executives, airline(s), Data Comm Program). 3. Data and associated access to the data must exist for all metrics. 4. The outcome of the metric must be attributable to Data Comm services in some explicit way. In other words, if the metric result is clearly produced by the presence of a Data Comm service, then the metric should be included. For example, for Mechanism Measures, the number of aircraft that are Data Comm equipped, is clearly under the influence of Data Comm, and should be included in the performance metrics set. Five broad metrics categories will be used to measure Data Comm operational performance: • • • • •

Mechanism Metrics User Impact - Aircraft Efficiency and Throughput Productivity Metrics Safety Metrics Environmental Metrics

The sub-sections that follow provide a description of each of these metrics categories. Specific metrics that will be used to determine the performance of Data Comm Segment One Phase One capabilities are in Section 4, followed by the specific metrics for Phase Two En Route Services in Section 5. The expanded DCL services provided in Phase Two will be measured by the same set of metrics as Phase One however; the specific locations and flights measured will vary. For most categories, metrics can be established specific to En Route or Tower capabilities. In the safety category, individual metrics for Tower and En Route are similar and therefore are not separated by capability in this Plan. This Plan is a working document and will continue to be refined based on knowledge gained during the planning leading up to Initial Operating Capacity (IOC), and throughout the implementation of Data Comm.

Data Comm Operational Performance Metrics Plan

Page 16

2.1

Mechanism Metrics

For any improvement to the NAS to be realized, some change must occur in how the system is managed or what facilities are available. In the case of Data Comm, the change will be the use of digital data instead of voice for non-tactical communications as well as enabling new communications using data link technology. Data Comm will also provide a parallel communications channel in addition to the existing voice communications. Mechanism metrics are a measure of change to the NAS. Five mechanism metrics are listed in the Data Comm Summary Matrix (Appendix D). Generally, mechanism metrics do not necessarily reflect that operational benefits have occurred; only that operational change has occurred. It is necessary to measure these changes in order to correlate the other operational performance metrics with the mechanism changes. Below are two examples of how mechanism metrics can improve operations: •

After controllers and pilots have been trained and acclimated to the Data Comm system, it is anticipated that controller workload will be reduced by allowing routine communications to be handled by pre-canned text messages. Automating routine clearances, as well as establishing a new communications path will allow controllers to handle more planes, and thus allow for greater throughput and efficiency.

•

After controllers and pilots have been trained and acclimated to the Data Comm system, it is anticipated that additional off airway routing will be employed. This may allow more direct routing and more efficient use of available airspace, providing benefit for users and the FAA.

These changes will bring about a cascade of other changes such as work methods or flight path scheduling. The mechanism metrics identified below will help to quantify the amount of change that occurs as a result of Data Comm. 2.1.1

Equipage and Usage

In order for Data Comm to be used, the necessary equipment must be in place in FAA facilities and on the aircraft. Once equipment is in place, it must be actually used in order for benefits to be realized. These metrics measure the progress of Data Comm implementation. It is important to understand this progress when performing other operational performance measurements in order to focus on areas where improvement might be expected as well as correlating findings with results.

Data Comm Operational Performance Metrics Plan

Page 17

Individual aircraft operators, primarily airlines, will be very interested in these metrics since user equipage investments will be necessary to take advantage of Data Comm. Airline-specific analysis will be conducted and reported to airlines on an individual basis. 2.1.2

Traffic Management Initiatives (TMI)

It is anticipated that once Data Comm is implemented and traffic managers adjust to the capabilities provided, the number of traffic management initiatives will be reduced. In the interest of providing incentive for aircraft equipage, the FAA may provide priority handling for equipped aircraft. 2.2

User Impact - Aircraft Throughput and Efficiency Metrics

The User Impact category contains five metrics identified for Segment One services. The best metrics to measure user impact are throughput, often peak throughput, and flight efficiency as measured by time and/or distance flown. These can be measured using surveillance flight track data. Once Data Comm is implemented, the ability of the ATC system to handle more aircraft will be reflected by throughput metrics. It is expected that Data Comm improvements will allow flights to move through the NAS with less impedance and this will be reflected in the efficiency metrics. Throughput and efficiency measures normally must be analyzed together because an improvement in one metric can be at the expense of the other. For example, if throughput is reduced enough then aircraft can move through the system unimpeded. Conversely, if efficiency is sacrificed through holding or maneuvering, throughput can often be maximized. These metrics can be better measures than delay for the FAA to use in gauging its performance. 2.2.1

Delay Savings Metrics

Delay, a NextGen Efficiency KPI, is directly related to the throughput and efficiency metrics described above. As throughput decreases, delay increases, likewise as aircraft efficiency decreases causing increased flight times, delay increases. A delay metric is simply a time metric compared to an expected time. Using delay as a measure can be somewhat problematic for the FAA when measuring operational performance since delay is a comparison of some actual time measure, usually arrival time, compared with an expected time. The problem arises due to the expected time not usually being controlled by the FAA but rather by the system user through their schedule. Since delay is so commonly used by the flying public to measure how well the system is working, it is likely to continue to be used as a system measure. Delay can be a useful measure for small segments of flights since the comparison value is then minimum distance or unimpeded time. Data Comm Operational Performance Metrics Plan

Page 18

One measure involving delay, that is meaningful, is delay variability and is a NextGen Predictability KPI that is a subset of delay measurement. That is how much variation in delay occurs for a given flight from day to day. If overall efficiency improvement occurs then variability reduction will also likely occur. When variability is reduced scheduled Block Time savings will also likely occur resulting in user savings. 2.2.2

Aircraft Throughput Metrics

It is anticipated that Data Comm will improve capacity within the NAS. This will occur within center sectors because Data Comm will increase sector capacity by allowing controllers to handle more aircraft due to reduced communications workload as well as the ability to share communication responsibilities. It will also occur at airports as communications needed on airports becomes more efficient with the use of Data Comm. It is not possible to directly measure system capacity however, increased capacity will allow Airspace Throughput and/or Airport Throughput to increase during times that demand exceeds current capacity limits. For throughput measures to effectively reflect capacity increases, they must be done during peak periods while demand exceeds capacity. During non-peak periods when demand is less than capacity throughput only measures demand. Therefore, peak period throughput is often used as a capacity metric and is the NextGen Capacity KPI. For the purpose of measurement, the airspace can be segmented by sector, center, or any other definition that makes sense. An example might include a block of airspace that includes multiple centers where capacity is limited due to the coordination needed to use airspace effectively near boundaries. With reduced communications time needed due to Data Comm there may be opportunities to better use the airspace by improving throughput. Throughput can be measured using existing databases by counting the number of aircraft passing through sectors or passing through imaginary ‘gates’ during peak periods and by counting the maximum number of aircraft within a sector during peak periods. System throughput and sector capacity are closely related but not the same. Generally, as sector capacity increases the number of aircraft that can pass through a sector also increases due to the reduced constraints on the number of aircraft that can access the airspace. Another potential benefit of increased sector capacity is a reduction in the number of sectors needed. It is expected that with the implementation of DCL, the revised departure clearance capability will improve airport departure throughput, especially during periods of adverse weather requiring numerous reroute amendments prior to departure. Controllers will be able to deliver amended clearances more quickly with fewer communication errors, which will reduce or eliminate queues of aircraft waiting for clearances. Once aircraft queues are reduced, it is expected that Data Comm Operational Performance Metrics Plan

Page 19

departure throughput will improve during peak departure times. This will be very challenging to measure due to the variability between weather events. However, with enough time and large enough data sets it is anticipated that measureable analysis will be possible. In addition, the Data Comm Benefits Team is working with the Total Airspace and Airport Modeler (TAAM) has shown that DCL departure efficiencies can also improve the arrival rate at an impacted airport. Typically, during a weather impact event, arrival and departure demand is balanced by the Towers. If the departure rate goes down, the arrival rate goes down proportionally. This tactic avoids potential airport gridlock and gate congestion. TAAM model results have positively shown that DCL minimizes gate-out delay and ramp congestion associated with both departing and arriving aircraft. With DCL, both the departure and arrival rates can be increased. The greatest limitation of arrival throughput is runway capacity. Modeling suggests that airport arrival throughput will increase due to improvement in efficiencies once Data Comm is implemented in the Terminal Radar Approach Control (TRACON) airspace in the Segment Two timeframe. With any of these throughput metrics, care must be taken to analyze the associated efficiencies. For example, increases in airspace throughput could lead to greater aircraft holding downstream. Airport arrival throughput will be tracked in order to ensure that it is not negatively impacted by Data Comm. 2.2.3

Flight Efficiency Metrics

Flight Efficiency metrics include the following and are NextGen KPIs: • • •

Time flown Distance flown Fuel burned

The Flight Efficiency metrics above are interrelated, and wind is a factor. Distance measures are not generally affected by wind while wind does affect time measures. Fuel burn is closely associated with time; however, wind can affect how much fuel burns in a given amount of time. In addition, wind can affect distance flown when aircraft operators choose to alter routes due to wind. As with throughput metrics, efficiency metrics can be broken down by flight segments (including airport operations) to focus on the segment targeted by Data Comm capabilities.

Data Comm Operational Performance Metrics Plan

Page 20

In the En Route environment, it is anticipated that Data Comm implementation will cause improvements in efficiency due to improvement in the ability to more efficiently communicate routes, altitudes, and/or procedures. Time flown, distance flown, and fuel burned should all experience improvements. Fuel burn may improve if aircraft are authorized to fly at optimum altitudes, even if the time and distance flown do not improve. It is anticipated that these efficiency improvements will be greater during periods of adverse weather, particularly convective weather. Data Comm will allow for the communication of random, off airway routes thereby better utilizing available airspace minimizing system perturbation caused by weather. As noted above, this will be especially challenging to measure due to the variability of weather patterns. In the airport environment, it is anticipated that Data Comm will reduce the time required to issue clearances--especially amended clearances. As a result, time spent for ground operations will be reduced and a corresponding reduction of fuel burn will occur. 2.3 Productivity Metrics Productivity metrics measure the impact that Data Comm will have on operational costs. It is expected that Data Comm will reduce unit costs for the FAA to provide ATC services. Note that the cost of deploying the Data Comm system is outside the scope of this Plan. In air traffic control facilities it is the peak demand periods, often driven by unusual weather or events, that determine staffing needs. Data Comm will likely reduce the impact of these peak demands and could potentially reduce controller-staffing requirements when demand increases in the future. Data Comm—along with other NextGen Initiatives—will provide productivity benefits for air traffic controllers, pilots, and aircraft operators. The FAA is still determining how the changes in NextGen technology will affect the controller workload. However, Data Comm will likely contribute to FAA’s ability to administer the Air Traffic Controller Workforce plan. Several of the NextGen KPI productivity metrics relate to the costs associated with operations, flight times, miles flown and controllers per facility. 2.4

Safety Metrics

When the FAA implements any change to the NAS, it strives to ensure that the change does not adversely affect system safety. The FAA has a number of offices, which maintain and ensure very high levels of safety. Specifically, the ATO Office of Safety (AJS), which utilize established metrics to track safety trends. The Data Comm Program will continue to work closely with these safety offices to determine how Data Comm contributes to the overall safety of the NAS. Data Comm Operational Performance Metrics Plan

Page 21

Historically, voice communication errors have contributed to a significant percentage of operational errors (OEs). With the introduction of Data Comm Segment One, it is expected, and studies have indicated, that there should be a reduction in operational errors and deviations related to communications for both pilots and controllers. While Data Comm Segment One initially is not expected to influence the number of runway incursions near-term, Digital Taxi Instructions, available in Segment Two, are expected to have a measureable impact on reducing runway incursions. In the past, the primary metrics used to assess the safety of the NAS have been operational errors/deviations and runway incursions. The FAA has developed a new System Risk Event Rate (SRER) measure. It is designed to offer a better assessment of flight risks and to provide a true safety performance indicator of the overall Air Traffic Control system. The primary differences between the SRER metric and the previously used Operation Error metric is that it integrates pilot and controller performance data and quantifies the most serious risks. As part of the overall goal of gaining significant benefits from Data Comm use, improvements in system safety will be measured and reported as each segment of Data Comm becomes operational. Data Comm is an important component in the overall NextGen strategy and is expected to provide positive contributions to the Safety KPIs that measure accidents, operational errors/deviations, and runway incursions. 2.5

Environmental Metrics

With society’s increasing concern for the environment, there is increased pressure to measure the environmental impact as a result of system changes. It is expected that Data Comm will have a positive impact on the environment. Improving fuel burn efficiency, as described above, will result in reduced emissions. While actual fuel burn data is difficult to obtain, especially on a per flight basis, algorithms exist to calculate fuel burn and resulting emissions based on aircraft type, engines, altitude, speed, aircraft attitude (climbing, descending, and level), etc. Another environmental concern is noise footprint, the number of people on the ground exposed to aircraft noise. A flight profile noise footprint is measured using the Day-Night-Level (DNL) Noise Model owned by the FAA. However, it is quite labor intensive to use. Further, it is not anticipated that Data Comm Segment One capabilities will significantly influence the noise Data Comm Operational Performance Metrics Plan

Page 22

footprint. The Data Comm Metrics Team has determined noise metrics are not directly linked to Data Comm benefits and thus are not included in this Plan. The NextGen Environment KPIs include metrics as discussed above as well as some very high level metrics, such as global impact, that will be outside of the capabilities of the Data Comm Metrics Team to determine. 2.6

Variables

The NAS is a very complex system with many variables affecting performance. Some of these variables, such as weather change daily. Other variables such as types of aircraft or airport changes have a longer-term impact. Analysis of the metrics described above will need to account for the variables outside the actual operational performance change being measured as much as possible. Appendix B lists many of the variables that will need to be normalized. The variables list will be further defined as the plan evolves.

2.7

Qualitative Performance Indicators

The Data Comm Operational Performance Metrics Team will need to carefully track the mechanism measures described above in order to know where to look for improvements. Team members will also need to perform periodic site visits to observe Data Comm in use and learn from end-users what operational improvements have resulted from the implementation. Qualitative data will be collected by visiting field ATC facilities to observe Data Comm services in use. In addition, informal discussions and structured interviews will be conducted to obtain subjective feedback from ATC personnel on the Data Comm system. This analysis will include how the Data Comm system has affected the way ATC personnel do their jobs, what they like and do not like about the new system. This increased understanding will help the Metrics Team to better understand the operational data, and to focus metrics analysis on the specific operational changes that have occurred. The interactions with field personnel and the subjective data collected will also help to paint a more complete picture of Data Comm operational performance. Tracking these qualitative findings allow for the inclusion of user feedback in benefits reports, and will help provide information that is very meaningful to various stakeholders. As these focused analyses are completed, there will be many opportunities to prepare and provide presentations of the findings for internal FAA executives and external customers and stakeholders. Early data analyses while not statically valid will provide qualitative feedback that can be used to address implementation issues. Data Comm Operational Performance Metrics Plan

Page 23

2.8

Baselining

For NAS Operational Performance Metrics, i.e., the benefits case for Data Comm (not program performance), baselining is the determination of the conditions prior to any operational changes. Thus, the baseline condition is determined using the NAS operational data that dates before IOC. The baseline values for benefits, i.e., delay and workload, will be derived from this data set, i.e., the before IOC operational performance data. In the case of Data Comm, the NAS operational performance data that is needed for establishing baselines is being collected on a continuous basis. It does not require immediate analysis of the data to determine baseline values as this can be done any time. Further, when attempting to measure program changes in a system as complicated as the NAS, it is often necessary to wait until after the change occurs in order to gain information, inputs and insights to the specific changes to determine useful baseline values. This gives the analyst the information needed to normalize for variables that are often unforeseen prior to a system change.

Section 3: Reporting Tools and processes will be developed and implemented to provide reporting that satisfies a wide range of needs, interests and performance questions of Data Comm customers and stakeholders. As part of standard administrative processes within the FAA, performance metrics related to the Data Comm Program will be reported on a regular basis in the following publications: • • • •

FAA Dashboard NextGen Implementation Plan RTCA ATO Performance Assessment

Summaries and extractions of operational performance information are prepared by FAA executives, and reported to DOT management. As part of the Capital Planning and Investment Control (CPIC) process, Data Comm must report performance status to OMB on the Exhibit 300 (Ex. 300). Extracts from the Ex. 300 are published on the OMB IT Dashboard and are available for public viewing. Because of their auditing role, the GAO may evaluate the effectiveness of the Data Comm program management team based on the achievement of the operational performance goals associated with the investment. 3.1

FAA Executive Level Metrics

The FAA measures its performance based on a number of goals and metrics. The results of these performance measures are reported in the annual FAA Performance & Accountability Report (http://www.faa.gov/about/plans_reports/media/2010_par.pdf); the 2010 report is the most recent Data Comm Operational Performance Metrics Plan

Page 24

at this writing. There are three categories of executive level performance measures that Data Comm will likely impact. These categories are: •

Safety o Runway Incursions (Segment 2 Only) o Operational Errors

•

Capacity o Average Daily Airport Capacity o NAS On-Time Arrivals o Noise Exposure (Segment 2 Only) o Aviation Fuel Efficiency1

•

Organizational Excellence o Cost Control

There are existing processes in place that are used to gather the data needed to track and report on these metrics. However, the Data Comm Operational Performance Metrics Team will be focusing only on the portion of the NAS that Data Comm will be influencing performance. It is anticipated that the Data Comm Program Management Office and FAA executives will want to report these results if it is found that the use of Data Comm has any affect on the metrics outcomes. The OMB metrics represent a subset of the total suite of operational performance metrics, which will overlap with the FAA Performance and Accountability metrics. These metrics are discussed in Section 1.3.3. All metrics that the Data Comm Program will be tracking are detailed in sections 4 and 5 of this plan; the metrics may be reported to the executives are in italics. Note: Executive level metrics provide a high-level indication of program operations, and may not have performance targets or associated baselines established. 3.2

Reporting

Since the ATO is a performance based organization, the Data Comm Operational Performance Metrics Team will coordinate with FAA organizations responsible for performance tracking to develop and provide consistent and regularly available data streams for analysis. The results will

1

Data Comm will track Fuel Burn as a Throughput and Efficiency Metric, but is not considering it as an executive level metric.

Data Comm Operational Performance Metrics Plan

Page 25

normally feed into various ‘dashboards’ that are used for performance tracking and are normally updated at least monthly. Coordination will be needed between the Data Comm Operational Performance Metrics Team and those responsible for collecting dashboard data to ensure that data supplied is accurate and complete. 3.3

Periodic Reports

Periodic comprehensive reports are useful in that stakeholders can obtain whatever information is available as appropriate. The frequency of these reports can vary depending on the type of information available and the desired objective. Generally, annual metrics reporting will be conducted. However, there may be enough demand to warrant more frequent reporting in the early phase of Data Comm implementation. There will also be the need to satisfy the OMB Ex.300 annual reporting requirements. The Data Comm Program will need to work with FAA Office of the Chief Information Officer (AIO) and DOT to ensure that OMB reporting requirements for performance are achieved. The need for more frequent reporting—quarterly or monthly—may be required as the program evolves or requirements change. 3.4

Ad Hoc Reports

Data Comm will generate considerable demand to provide evidence that it is making a difference. FAA managers as well as system users, customers, and stakeholders will drive this demand. Depending on the audience, the determination will have to be made as to what analytical data and results are appropriate to share. As these focused analyses are completed, there may be many opportunities to prepare and provide presentations of these finding. This type of ad hoc reporting will be the responsibility of the Metrics Team and will be further defined based on the objective and type of data available.

Section 4: Segment One Phase One Metrics 4.1

Tower Specific Metrics

Data Comm Segment One Phase One will include DCL services via FANS. As Data Comm DCL is implemented in individual control towers, mechanism metrics will be needed in order to correlate other metrics with the actual changes. 4.1.1

Tower Equipage and Usage Metrics

Data Comm Operational Performance Metrics Plan

Page 26

Table 3 below describes metrics specific to tower equipage and usage. The italicized metrics in the following tables are considered for executive-level metrics reporting.

Metric Title FAA Equipage User Equipage

Data Comm Usage

Minutes of Comm Time Saved

TMI restriction, Numbers and Values

Tower Equipage and Usage Metrics Tower Specific Metric Metric Measurement Number of towers -% of total equipped and operational -% of planned Number of aircraft Data -% by airline Comm equipped -% of total fleet -Type/level of equipage Number of Data Comm - Number of aircraft messages sent logged into Data Comm system and number of sessions. -Number of Data Comm messages by airline -% of total clearances -Number of re-route clearances -Type of messages Calculated value -Calculation using above comparing known voice results comm times with Data Comm Departure TMI -Number at Data Comm restriction values airports -TMI restriction values -Data Comm TMI restriction exemptions

Data Source(s) Data Comm Program Airlines/ MITRE

ERAM/ TDLS

ERAM/ TDLS

NTML/ ETMS

Table 3: Tower Equipage & Usage Metrics *See Appendix C for acronym listing

4.1.2

Tower Specific User Impact - Throughput and Efficiency Metrics

With the use of DCL, there should be improvements in throughput and efficiency for departing aircraft. Since the time needed to communicate Instrument Flight Rules (IFR) clearances, especially reroute clearances will be reduced, there should be a reduction of the time needed for aircraft to wait for their clearance. This should result in delay reduction, fuel burn reduction, better airport movement predictability, and increased peak departure throughput. Data Comm Operational Performance Metrics Plan

Page 27

Metric Title Delays

Airspace Throughput

Efficiency

Fuel Burn

Tower Throughput & Efficiency Metrics Tower Specific Metrics Metric Measurement -Gate delay -Gate out time minus proposed time -Taxi time change -Average taxi time/delay before & after Data Comm -Minutes of -Taxi time/Delay variability (predictability variability before & after Data Comm metric) -Peak departure -30 min peak throughput change departure counts -Peak departure -Peak counts during throughput during reduced capacity weather events -Trends over time -Throughput changes by airport -Taxi Time -Taxi time trends -Rerouted flight gate comparing before delay and after DCL -Taxi and Gate time trends during weather events -Amount of fuel burned -Fuel burn calc based during ground operations on taxi time -Amount of fuel burned -Fuel burn calc during departure phase of during climb flight

Data Source(s)

ASPM/ Flight Track Data

ASPM/ Flight Track Data

ASPM/ Flight Track Data

ASPM/ Flight Track Data/ Fuel Burn BADA

Table 4: Tower Throughput & Efficiency Metrics *See Appendix C for acronym listing

Note: Departure delay is in italics since departure delay often leads to arrival delay, which is the actual executive level metric.

4.1.3 Tower Specific Productivity Metrics The communications workload for controllers will be reduced with the use of DCL. The amount of time savings per clearance issued is expected to be significant. This should result in a reduced need for additional clearance delivery position staffing in towers during peak periods when Data Comm Operational Performance Metrics Plan

Page 28

workload is high. While a reduction in workload may not reduce the number of personnel needed at an individual facility, it may allow for better use of time for other activities and influence the need for overtime during high workload events.

Tower Productivity Metrics Metric Title Tower Specific Metric Measurement Metrics Tower Controller -Clearance Delivery -Amount of time of Workload position hours before clearance and after DCL delivery/flight data positions are open by facility ATC Costs per Unit -Direct ATO employee -Operations per direct of Service cost per departure ATO employee -Cost by facility

Data Source(s) AJF A-7 - IT Technical Services Group (Cru-X Position Logs) AJF Finance/ OPSNET

Table 5: Tower Productivity Metrics *See Appendix C for acronym listing

4.1.4

Specific Safety Metrics

It is known that miscommunications have contributed to, or caused many operational errors or pilot deviations. Data Comm messages will be displayed on the pilot’s console and will remain available for visual verification, resulting in a reduction of controller and pilot miscommunications. It is possible that an incorrect data message could be sent. However, it is likely that such instances would be significantly less than today’s environment, where voicerelated misunderstandings are prevalent. Until Data Comm Segment Two capabilities become more widely available, there is likely to be little contribution to a reduction in runway incursions. Thus, the initial set of safety metrics will defer tracking runway incursions until the implementation of future Data Comm services. To maintain compatibility with current FAA Safety Metrics Offices, the Data Comm Metrics Team will continue to collaborate on communications related SRER OEs/ODs, and pilot deviations before and after Data Comm use. Due to the insignificant difference between Tower and En Route safety metrics, Table 6 below shows a combined set of Data Comm safety metrics.

Safety Metrics for Tower and En Route Data Comm Operational Performance Metrics Plan

Page 29

Metric Title System Risk Event Rate (SRER)

Sub Metrics -Errors by facility/airport -Comm related errors

Metrics Measurement -SRER count trends -Comm related SRER trends

Data Source(s) AJS-7, Office of Safety, Risk Reduction Information Directorate

Table 6: Safety Metrics for Tower and En Route *See Appendix C for acronym listing

4.1.5

Tower Specific Environmental Metrics

The implementation of DCL should result in fuel burn and emissions reductions. Actual emissions measurement is not practical. However, an approximation can be calculated based on aircraft and engine types and phase of flight, or in this case, taxi operations. Turbine engines are not very efficient at low altitudes so the emissions reductions with reduced ground operating time may be significant. Data Comm may reduce the need for aircraft to level off at intermediate altitudes, thereby reducing fuel consumption and emissions.

Metric Title Aircraft Emissions During Taxi Out

Aircraft Emissions During Climb

Tower Environmental Metrics Tower Specific Metrics Metrics Measurement -By airport -Trends of calculated values based on fuel burned -Reduced intermediate altitude operations

-Calculated values based on fuel burn

Data Source(s) ASPM/ Aircraft Performance Data/ Emission Algorithms ASPM/ Aircraft Performance Data/ Emission Algorithms/ Flight Track Data

Table 7: Tower Environmental Metrics *See Appendix C for acronym listing

Section 5: Segment One Phase Two Metrics As mentioned earlier, Data Comm Segment One will include DCL services via the ATN system as well as En Route services. The metrics for the DCL services will have already been established during Phase One as described above and will continue for Phase Two. Having gained the experience of using these metrics during Phase One, it is likely that additional refinement of them will have occurred and be applied during Phase Two. Data Comm Operational Performance Metrics Plan

Page 30

5.1

En Route Specific Metrics

5.1.1

En Route Specific Mechanism Metrics

Table 8 below describes metrics specific to En Route equipage and usage. The italicized metrics in all of the following tables are considered for executive-level metrics reporting.

Metric Title FAA Equipage

User Equipage

Data Comm Usage

Minutes of Comm time saved

TMI restriction, Numbers & Values

En Route Equipage and Usage Metrics En Route Specific Metric Measurement Metric Number of centers -% of total equipped and -% of planned operational -Number of sectors Number of aircraft -% by airline Data Comm equipped -% of total fleet - % equipage -Type/level of equipage Number of Data - Number of A/C logged Comm messages sent into Data Comm system and number of sessions. -Number of Data Comm messages -Messages by airline -% of total communications -Type of messages -Sector level usage Calculated value Calculation using above comparing known results voice comm times with Data Comm En Route TMI -TMI volume restrictions restriction values -TMI restriction value changes -Data Comm TMI restriction exemptions -MAP value changes

Data Source(s)

Data Comm Airlines/ MITRE

ERAM / TDLS

ERAM / TDLS

NTML/ ETMS

Table 8: En Route Equipage and Usage Metrics *See Appendix C for acronym listing

5.1.2

En Route Specific User Impact – Throughput and Efficiency Metrics

Data Comm Operational Performance Metrics Plan

Page 31

Data Comm will improve communications in the En Route environment by moving routine messages away from the voice channel and allowing for parallel communications with voice. This will increase sector capacity and should be measurable as increased peak throughput. Data Comm will also allow for the communication of routing information that is cumbersome to communicate by voice today. This will allow for better use of available airspace, should positively influence time and/or distances flown, and reduce fuel burn. Increasing sector capacity and improving efficiency, will allow more aircraft to fly at requested altitudes and benefit from improved fuel efficiency. In addition, with greater sector capacity and improved efficiency, aircraft will experience less average delay and less delay variability, which will allow average block times to be reduced.

Metric Title Delays

En Route Throughput & Efficiency Metrics En Route Specific Metric Measurement Metrics -Arrival delay -Actual arrival time sector/Center delay minus scheduled arrival time -Weather delay -Actual time during a weather event compared to unimpeded time -Delay variability (predictability metric)

Airspace Throughput

Efficiency

-Peak sector/Center throughput -Peak corridor throughput -Time/Distance flown: -Excess by sector -Excess by center -Excess by flight during weather -By airline -Time/distance at suboptimal altitude

-Delay variation minutes before & after Data Comm -Peak counts of aircraft passing through sectors/gates -Time/Distance flown trends -Time/distance flown vs. unimpeded/great circle route

Data Source(s)

ASPM/ Flight Track Data, PDARS

ASPM/ Flight Track Data, PDARS

ASPM/ Flight Track Data, PDARS

-Level time/distance flown at non-requested

Data Comm Operational Performance Metrics Plan

Page 32

Metric Title

Block Time

Fuel Burn

En Route Throughput & Efficiency Metrics En Route Specific Metric Measurement Metrics altitudes -Change by city pair -Gate to gate time -Change by airline trends by city pair -Scheduled block time trends -Change by sector -Calculated fuel burns -Change by center using flight parameters -Change by flight and BADA algorithms -By airline -Increased optimal altitude savings

Data Source(s)

ASPM

ETMS/ PDARS/ Fuel Burn BADA

Table 9: En Route Throughput & Efficiency Metrics *See Appendix C for acronym listing

5.1.3

En Route Specific Productivity Metrics

With increased communications capability provided by Data Comm, FAA productivity in providing ATC services should improve. As sector capacity improves each controller or controller team will be able to safely handle more operations. In addition, as capacity increases, the need to divide airspace into smaller sectors should be reduced.

Metric En Route Controller Workload

ATC Costs per Unit of Service (Flight/operation/etc.)

En Route Productivity Metrics En Route Specific Metric Measurement Metrics -Position hours by - Hours positions open facility (not combined) -R/D/A side hours -Weather event - Number of positions position hours open during weather - Controllers per flight - Total flights/number hour/mile of controllers -Direct ATO -Operations per direct employee cost per ATO employee flight -Cost by facility

Data Source(s) AJF A-7 - IT Technical Services Group (Cru-X Position Logs) / Flight Track Data AJF Finance/ OPSNET

Table 10: En Route Productivity Metrics *See Appendix C for acronym listing

Data Comm Operational Performance Metrics Plan

Page 33

5.1.4

Specific Safety Metrics

These safety metrics are the same as for Phase One repeated here in order to show a complete set of metrics for Phase Two. As above in DCL, it is likely that through experience of using these metrics during Phase One, experience will lead to refinements that will carry into Phase Two. It is known that miscommunications have contributed to, or caused many operational errors or pilot deviations. Data Comm messages will be displayed on the pilot’s console and will remain available for visual verification, resulting in a reduction of controller and pilot miscommunications. It is possible that an incorrect data message could be sent. However, it is likely that such instances would be significantly less than today’s environment, where voicerelated misunderstandings are prevalent. Until Data Comm Segment Two capabilities become more widely available, there is likely to be little contribution to a reduction in runway incursions. Thus, the initial set of safety metrics will defer tracking runway incursions until future Data Comm services are implemented. To maintain compatibility with current FAA Safety Metrics Offices (AJS), the Data Comm Metrics Team will continue to collaborate on communications related OEs/ODs, and pilot deviations before and after Data Comm use. Due to the insignificant difference between Tower and En Route safety metrics, Table 11 below shows a combined set of Data Comm safety metrics.

Metric Title System Risk Event Rate (SRER)

Safety Metrics for Tower and En Route Sub Metrics Metrics Measurement -Errors by -SRER count trends facility/airport -Comm related SRER -Comm related errors trends

Data Source(s) AJS-7, Office of Safety, Risk Reduction Information Directorate

Table 11: Safety Metrics for Tower and En Route *See Appendix C for acronym listing

5.1.5

En Route Specific Environmental Metrics

With improved efficiency resulting from Data Comm, it is reasonable to expect emission reductions will coincide with reductions in fuel consumption. Actual emissions measurement is not practical however, an approximation can be calculated based on aircraft and engine types and phase of flight, i.e. climb, cruise, descent. Turbine engines are more efficient at high altitudes so reduced flight time at sub-optimal altitudes should reduce fuel burn and emissions. Data Comm Operational Performance Metrics Plan

Page 34

Metric Title Aircraft Emissions During Flight

Aircraft Emissions During Climb/Descent

En Route Environment Metrics En Route Specific Metrics Metrics Measurement -By flight -Calculated values -By center/sector based on fuel burned -By airline

-Reduced intermediate altitude operations

-Level time/distance flown at nonrequested altitudes

-Increased use of ODPs

-Trend of numbers of ODPs used

Data Source(s) ASPM / Aircraft Performance Data/ Emissions Algorithms/ Flight Track Data ASPM/ Aircraft Performance Data/ Emissions Algorithms Flight Track Data

Table 12: En Route Environment Metrics *See Appendix C for acronym listing

Section 6: Timeline Figure 2 below summarizes the major metrics activities for FY12-FY18.

Develop Metrics Database DCIT Trials Establish DC Data Network Establish Safety Office Relationships Determine Surface Data Source Establish Environmental Office Relationship Establish Analysis Team Conduct Trend Analysis Establish Objectives

2012

2013

Measure/Report DCL

2014

2018

Figure 2: Data Comm Metrics Timeline – Future Activities

Data Comm Operational Performance Metrics Plan

Page 35

Appendix A: Metrics Data Sources Following is a list of existing data sources that will be used during the analyses of Data Comm metrics: •

Flight Track Data – Enhanced Traffic Management System (ETMS) and Center/TRACON Offload Data provide aircraft flight track data including altitude information and flight plan information. This data can be used to analyze entire individual flights or flight segments as needed.

•

ETMSC - The Enhanced Traffic Management System Counts is designed to provide information on traffic counts by airport or by city pair for various data groupings, such as aircraft type or by hour of the day. Information on oceanic flights, fractional ownership flights or business jet activity is also maintained. The daily data is available from January 2000 and is updated monthly.

•

ASQP - The Airline Service Quality Performance System (ASQP) contains data provided by the airlines by flight for airlines that carry at least 1% of all domestic passengers. Actual and scheduled time is available for gate departure and gate arrival. The airlines also provide the actual wheels-off time so that taxi-out time can be computed and wheelson time so that taxi-in time can be computed. In addition, the airlines provide causal data for all flights arriving 15 minutes past their scheduled arrival time. The data is available from June 2003 and is updated on a monthly basis.

•

NTML - The National Traffic Management Log. Contains information on traffic management initiatives.

•

OPSNET - The Operations Network (OPSNET) is the official source of historical NAS air traffic delays and operations. Daily data is available on a next-day basis. Monthly and annual counts are also available from the system by facility, state, region, service area, or nationally. Some ranking tables are also available. Data is available from January 1990 through the prior day. All historical operations are synchronized with ATADS (see below).

•

ASPM - Aviation System Performance Metrics (ASPM) provides information on individual flight performance and information on airport efficiency.

•

ATADS - The Air Traffic Activity Data System (ATADS) contains traffic counts for towered airports, TRACONS, Combined Center and Radar Approach Control (CERAPS), and centers. Instrument approach counts are also maintained. Daily, monthly and annual counts are available either by facility, state, region, service area or nationally. Data is available for the prior month by the 20th of each month. Operations are drawn from the OPSNET system, which is the current official source for this data.

Data Comm Operational Performance Metrics Plan

Page 36

•

ATO Safety Data – Offices within FAA do significant collection and tracking of data related to safety. Coordination will be needed to obtain safety related trend information associated with the implementation of Data Comm

•

BTS – DOT Bureau of Transportation Statistics

•

Operating Position Log Data – Will be useful for tracking trends on workload

•

IRIS – Integrated Reporting Information System. An FAA airspace and flight efficiency analysis tool

•

PDARS – The Performance Data Analysis and Reporting System utilizes radar flight track and other data from air traffic control facilities to enable monitoring, measurement, analysis, and graphical display of air traffic operational performance. PDARS provides a system- wide capability to monitor day-to-day operations of the National Airspace System (NAS) and to measure Air Traffic Control delivery of services.

The robust set of existing data sources listed above will be employed to analyze the specific Data Comm metrics. Furthermore, these data will also need to be obtained and stored specific to Data Comm. These data will need to be gathered by the program offices and/or the Data Comm system components and will include: • • •

Field utilization data Field observations Individual message data – Content – Time message sent – Time of acknowledgment – Type of response – Etc.

Database(s) will need to be developed or existing databases expanded to collect and store Data Comm specific data. This will require action on the part of the Data Comm Operational Performance metrics team prior to implementation.

Data Comm Operational Performance Metrics Plan

Page 37

Appendix B: Variables Appendix B lists many of the variables that must be considered and normalized when analyzing data and quantifying performance changes. The National Airspace System is a very complex environment with many varying factors that affect it every day. The following is a list of some of the variables affecting the movement of aircraft in the NAS. This section highlights the complexity of analyzing Data Comm improvements, as these variables must be normalized for as much a possible. During analysis attempting to quantify the performance of Data Comm system improvements, these variables will be considered. When comparing metrics before and after implementation or equipped to non-equipped aircraft, there will be variations caused by a few or perhaps several of these variables at once. Several techniques can be utilized to normalize for these variations. If there are only a few variables that might affect the results then it might be as simple as selecting time periods during which conditions were the same. Runway configuration is an example of this. Most airports have certain runway configurations that are most often used. An effective normalization might be to measure select periods using this standard configuration only. In other cases, when weather is a significant factor, it may be necessary to develop complex data reduction formulas to appropriately account for the confounding variables. Weather Low visibility Wind Snow & Ice Thunderstorms Airport Constraints Runway Configurations Runway conditions Construction Dedicated Runways Multi Use Runways Trafic Mix Airspace Contraints Terrain Noise Special Use Traffic Volume Data Comm Operational Performance Metrics Plan

Page 38

Workload Constraints Safety Related Requirements Land-and-Hold Short Line-Up-and-Wait Special separation requirements Equipment Location Outages Variations Automation Adaptation Support User Conditions Economic climate Airline mergers Hub changes Aircraft used Other The list above includes many variables however; others may be added as analysis begins.

Data Comm Operational Performance Metrics Plan

Page 39

Appendix C: Acronyms Acronym AIO AJF AJS ASPM ASQP ATC ATM ATN ATO ATADS BADA BCA BTS BY CERAPS CO2 CONOPS CPIC D2025 D-ATIS DCL DCNS DLAP DNL DOT ERAM ETMS ETMSC EVM FAA FANS FY GAO NextGen ICAO IFR IOC IRIS IT

Definition Office of the Chief Information Officer Air Traffic Organization – Finance Air Traffic Organization – Office of Safety

Aviation System Performance Metrics Airline Service Quality Performance System Air Traffic Control

Air Traffic Management Aeronautical Telecommunications Network Air Traffic Organization Air Traffic Activity Data System Base of Aircraft Data Algorithm Business Case Analysis DOT Bureau of Transportation Statistics Base Year Combined Center and Radar Approach Control Carbon Dioxide Concept of Operations Capital Planning and Investment Control Destination 2025 Digital-Automatic Terminal Information Service Departure Clearance Service Data Comm Network Services Data Link Applications Processor Day-Night-Level Department of Transportation En Route Automation Modernization Enhanced Traffic Management System Enhanced Traffic Management System Counts Earned Value Management Federal Aviation Administration

Future Air Navigation System Fiscal Year US Government Accountability Office Next Generation Transportation System International Civil Aviation Organization Instrument Flight Rules Initial Operating Capability Integrated Reporting Information System Information Technology

Data Comm Operational Performance Metrics Plan

Page 40

Acronym KPI MAP NAS NTML OD ODP OE OMB OPMP OPSNET OT PD PDARS RTA RTCA SME SRER TAAM TBO TDLS TFM TMI TRACON VDL-2

Definition Key Performance Indicator Monitor Alert Parameter National Airspace System National Traffic Management Log Operational Deviation Optimized Descent Profile Operational Error US Government Office of Management and Budget Operational Performance Metrics Plan Operations Network Operational Test Pilot Deviation Performance Data Analysis and Reporting System

Required Time of Arrival Aviation industry group; www.rtca.org Subject Matter Expert System Risk Event Rate Total Airspace and Airport Modeler Trajectory Based Operations Terminal Data Link Services Traffic Flow Management Traffic Management Initiatives Terminal Radar Approach Control VHF Data Link – Mode 2

Data Comm Operational Performance Metrics Plan

Page 41

Appendix D: Data Comm Operational Performance Metrics Matrix The attached matrices summarize the Operational Performance Metrics along with the target audience (customers) for the various Data Comm Program metrics for Segment 1 Phase 1 (S1P1) and Segment 1 Phase 2 (S1P2). This is a working document intended as a tool for use by the Data Comm Metrics Team and will be modified in conjunction with revision of the Plan.

Data Comm Operational Performance Metrics Plan

Page 42

Block time trends

Block Time

Measure throughput counts during peak periods

Peak Departure Throughput Trends ASPM Flight Track Data

2015

Average Daily Airspace Capacity/Efficiency

X

X

X

X

X

X

X

X

Data Comm Program

X

X

X

Page 43

Excess Distance Savings

Data Comm Operational Performance Metrics Plan

Airspace Capacity / Efficiency X

X

Average Daily Airport Capacity

*User Equipage is considered an Executive-level metric, even though it does not directly align with the categories in the FAA Performance and Accountability Report

Airspace Throughput

2015

2015

NAS On-Time Arrivals

ASPM

ASPM, Flight Track Data

2015

2015

X

Departure Delay Savings

Determine average block N/A times for selected city pairs to determine if there is any change due to Data Comm

Departure Gate and Taxi Delay Trends

NTML, ETMS

Data Comm Automation

Percent of Data Comm targeted aircraft equipage achieved

Arrival Delay Savings

Airspace & Airport Peak Throughput Trends

Delay trends for Comparison of gate and taxi segments of flights delay with and without DC. impacted by DC Use OOOI (Out/Off/On/In) data and, if available, surface surveillance data to measure changes in ground time

Delays

Count of restrictions and total Departure restriction values (i.e. severity Restriction Counts of restriction X duration, e.g. & Values 10 MIT for 2 hours is value 20)

Time savings for DCL reroute clearances

2015

2015

X

System Users

X

X

X

X

X

Data Comm Benefits

User Impact Measures

Number of Restrictions Restriction Values (severity)

Restriction changes

Minutes of Comm time saved

Number of A/C logged into Data Comm system and number of sessions. Messages communicated using Data Comm for different message types (clearances, amendments, altitude changes, etc.) Measure time to communicate DC messages and compare to average times for equivalent voice messages

Number of A/C logged in and messages communicated categorize by message type (i.e. Reroute, Initial Clearance, etc) Calculated Value Comparing Known Voice Comm Times with DC

Data Comm usage

Number of A/C logged into Data Comm system and number of DC Automation sessions. Number of DCL messages sent.

Percentage of targeted airline N/A fleet equipped and level of equipage

OMB 300 Metric Definition

Data Comm Aircraft Equipage

User equipage* Number of aircraft Data Comm equipped - % equipage

2015

Date Implemented

DC Accessibility & Usage

Airlines (MITRE)

Tech Ops/DC Program

FAA Equipage Number of facilities Percent of targeted facilities Number of Towers Data Comm that are Data Comm equipped equipped and equipped and and operational operational operational

Methodology/Metric

Data Source(s)

Description

Tower Specific Measurement

Metric Name

ATO Exec (Categories taken from FAA Performance & Accountability Report)

Data Comm Implementation Progress

Mechanism Measures

Category

OMB

Data Comm Metrics Summary Matrix (Segment 1 Phase 1)

X

X

X

X

X

X

NextGen Key Performance Indicator

Cost Control

Operational Errors & Deviations Per 100K Ops

Data Comm Workload Impact

Safety

Ground Operation Emissions Fuel Burn BADA

2015

2015

Calculated from fuel burn data, savings in distances or time flown

Operational errors associated with communications (System Risk Event Rate - SRER)

Data Comm impact on controller workload

X

X

X

X

X

X

X

Data Comm Program

X

X

X

X

X

X

X

X

X

X

X

Page 44

Data Comm Implementation Progress

Data Comm Operational Performance Metrics Plan

*User Equipage is considered an Executive-level metric, even though it does not directly align with the categories in the FAA Performance and Accountability Report

Reduced Emissions

AOV/AJS

2015

2015

OMB 300 Metric Definition Safety

Environmental Measures

Tower IFR departure SRER trends

Cru-X OPSNET

Cru-X

2015

2015

Date Implemented

Operational Errors & Deviations Per 100K Ops

Safety Metrics

System Risk Event Rate (SRER)

Tower Controller Costs

Clearance Delivery position hours before and after DCL

ETMS, PDARS, Fuel Burn BADA

ETMS, PDARS

Data Source(s) Data Comm Workload Impact

Monitor this existing ATO metric for changes correlating to implementation of Data Comm System Risk Event Compare total number of Rate changes comm related System Risk associated with Events (SRER) with and communications without Data Comm (measured by using the System Risk Event Rate metric SRER) Emissions Trends Derive reduction in emissions from savings in distances or time flown and associated fuel burn

Data Comm impact on controller workload, workload during weather

Fuel Burned During Taxi & Climb

Taxi & Gate times before and after DC

Tower Specific Measurement Cost Control

Cost per unit of FAA Cost per service flight/operation trends

Controller Position Hours

Controller Workload

Calculate savings in fuel based on savings in flight times and distances

Measure changes in flight distances/times

Methodology/Metric

Arrival Delay Savings

Productivity

Fuel burn trends, taxi and airborne

Flight travel distance trends

Description

System Users

Excess Distance Savings

Fuel Burn

Efficiency

Metric Name

ATO Exec (Categories taken from FAA Performance & Accountability Report)

Data Comm Benefits

User Impact Measures

Category

OMB

Data Comm Metrics Summary Matrix (Segment 1 Phase 1)

NextGen Key Performance Indicator

Departure Delay Savings

DC Accessibility & Usage

Average Daily Airport Capacity

NAS On-Time Arrivals

Data Comm Aircraft Equipage

Airspace Capacity / Efficiency

Methodology/Metric

Tower Specific Measurement

En Route Specific Measurement

Number of Restrictions Restriction Values (severity)

Delay trends for Comparison of gate and taxi Departure Gate and segments of flights delay with and without DC. Use Taxi Delay Trends impacted by DC OOOI (Out/Off/On/In) data and, if available, surface surveillance data to measure changes in ground time

Block time trends

Restriction changes

Delays

Block Time

ASPM Flight Track Data

ASPM

2015

2015

2015

2015

2015

Airspace Capacity / Efficiency X

X

X

X

X

X

X

X

X

X

Data Comm Program

X

X

X

Page 45

Excess Distance Savings

Data Comm Operational Performance Metrics Plan

Average Daily Airspace Capacity/Efficiency

X

Average Daily Airport Capacity

*User Equipage is considered an Executive-level metric, even though it does not directly align with the categories in the FAA Performance and Accountability Report

Measure throughput counts during peak periods

Change by City Pair, by airline, change in scheduled block times Peak Departure Airspace Peak Throughput Trends Throughput Trends

ASPM, Flight Track Data

NTML, ETMS

Data Comm Automation

Percent of Data Comm targeted aircraft equipage achieved

NAS On-Time Arrivals

Determine average block times N/A for selected city pairs to determine if there is any change due to Data Comm

En Route & Arrival Delay Trends including Weather Delays

Count of restrictions and total Departure En Route restriction values (i.e. severity Restriction Counts Restriction Counts of restriction X duration, e.g. 10 & Values & Values MIT for 2 hours is value 20)

Time savings for all types of enroute Data Comm messages

2015

2015

X

Departure Delay Savings

Airspace & Airport Peak Throughput Trends

Calculated Value Comparing Known Voice Comm Times with DC

Minutes of Comm time saved

Number of A/C logged into Data Comm system and number of sessions. DC Automation Number of Data Comm messages sent

Airlines (MITRE)

OMB 300 Metric Definition

Arrival Delay Savings

Airspace Throughput

Number of A/C logged into Data Comm system and number of sessions. Number of DCL messages sent.

Number of A/C logged into Data Comm system and number of sessions. Messages communicated using Data Comm for different message types (Transfer of Control, clearances, amendments, altitude changes, etc.).

Number of A/C logged in and messages communicated categorize by message type (i.e. Reroute, Initial Clearance, etc)

Data Comm usage

N/A

2015

Date Implemented

Data Comm Aircraft Equipage

Measure time to communicate Time savings for DC messages and compare to DCL reroute average times for equivalent clearances voice messages

N/A

Percentage of targeted airline fleet equipped and level of equipage

User equipage* Number of aircraft Data Comm equipped - % equipage

Tech Ops/DC Program

Data Source(s)

System Users

X

X

X

X

X

Data Comm Benefits

User Impact Measures

Description

ATO Exec (Categories taken from FAA Perform ance & Accountability Report)

DC Availability* & Usage

FAA Equipage Number of facilities Percent of targeted facilities Number of Towers Number of Centers Data Comm that are Data Comm equipped equipped and equipped and equipped and and operational operational operational operational

Metric Name

OMB

Data Comm Implementation Progress

Mechanism Measures

Category

Data Comm Metrics Summary Matrix (Segment 1 Phase 2)

X

X

X

X

X

X

NextGen Key Performance Indicator

Cost Control

Operational Errors & Deviations Per 100K Ops

Data Comm Workload Impact

Safety

Derive reduction in emissions Ground Operation from savings in distances or Emissions time flown and associated fuel burn

Airborne Emissions Fuel Burn BADA

2015

2015

Calculated from fuel burn data, savings in distances or time flown

Operational errors associated with communications (System Risk Event Rate - SRER)

Data Comm impact on controller workload.

X

X

X

X

X

X

Page 46

Departure Delay Savings

Data Comm Operational Performance Metrics Plan

*User Equipage is considered an Executive-level metric, even though it does not directly align with the categories in the FAA Performance and Accountability Report

Emissions Trends

AOV/AJS

2015

2015

Safety

Reduced Environmental Emissions Measures

N/A

Cru-X OPSNET

Cru-X

OMB 300 Metric Definition

Operational Errors & Deviations Per 100K Ops

Safety Metrics

System Risk Event Rate (SRER)

En Route Controller Costs

Clearance Delivery En Route Controller & A/D Side Position position hours before and after Time DCL

2015

2015

Date Implemented Data Comm Workload Impact

Monitor this existing ATO Tower Controller metric for changes correlating Costs to implementation of Data Comm System Risk Event Compare total number of N/A Rate changes comm related System Risk associated with Events (SRER) with and communications without Data Comm (measured by using the System Risk Event Rate metric SRER)

Data Comm impact on controller workload, workload during weather

ETMS, PDARS, Fuel Burn BADA

ETMS, PDARS

Data Source(s)

Cost Control

Cost per unit of FAA Cost per service flight/operation trends

Controller Position Hours

Controller Workload

En Route Specific Measurement

Taxi & Gate times Time/Distance before and after DC Flown, excess greater than unimpeded

Tower Specific Measurement

Calculate savings in fuel based Fuel Burned During Fuel Burned on savings in flight times and Taxi & Climb Airborne by sector, distances center, airline, altitude

Measure changes in flight distances/times

Methodology/Metric

System Users

X

Arrival Delay Savings

Productivity

Fuel burn trends, taxi and airborne

Flight travel distance trends

Description

ATO Exec (Categories taken from FAA Performance & Accountability Report)

Excess Distance Savings

Fuel Burn

Efficiency

Metric Name

OMB

Data Comm Program

X

X

X

X

X

X

Data Comm Benefits

User Impact Measures

Category

Data Comm Metrics Summary Matrix (Segment 1 Phase 2)

X

X

X

X

X

NextGen Key Performance Indicator

Data Comm Implementation Progress

DC Availability* & Usage

Average Daily Airport Capacity

NAS On-Time Arrivals

Data Comm Aircraft Equipage

Airspace Capacity / Efficiency

C FEA Performance Measurement Category Mapping

Data Comm Operational Performance Metrics Plan

B

Unit of Measure

A

Metric Description

D Baseline

E Target for the PY (2013)

Actual for the PY (2013)

F

Table C.1 Target for the CY (2014)

G Measurement Condition

H Reporting Frequency

I Most Recent Actual Results for CY (2014)

J

Page 47

Comment

K

Appendix E: Data Comm BY13 Exhibit 300 Operational Performance Table (Part B, Table I.C.1.b)