Reliability Measures and Risk Assessment Andrew Slone, Engineer, Reliability Performance Analysis, NERC EPRI Workshop, February 23, 2012

Overview • Reliability Measures    

2011 Risk Assessment of Reliability Performance Report 2012 State of Reliability Report Overview of ALR Metrics and Trends Severity Risk Index

• Future  Integrated Reliability Indicators

2

RELIABILITY | ACCOUNTABILITY

Reliability Improvement

Reliability Measures

? Reliability ConditionDriven Indicators

? ? ? ?

?

Organization? Effectiveness Metrics

? Standards/ StatuteDriven Metrics ?

?

System/EventDriven ? Metrics

Organizational Effectiveness 3

RELIABILITY | ACCOUNTABILITY

2011 Annual Report •

Annual Report “2011 Risk Assessment of Reliability Performance”   

Culminates a 3 year process to provide a view of risks to reliability Approved by the NERC Board of Trustees on August 4, 2011 Available at: http://www.nerc.com/docs/pc/rmwg/2011RMWG_Annual_Report.pdf

Condition Driven

Events Driven

Standards Driven

Integrated Measures and Analysis

State of Reliability Report 4

RELIABILITY | ACCOUNTABILITY

Data Source Integration and Analysis

Metrics

TADS Integration and Analysis

GADS

EVENTS

DADS Events Driven

Critical Infrastructure Protection 5

Condition Driven

Standards Dev & Prioritization

Standards Driven

Compliance

Events Analysis

RELIABILITY | ACCOUNTABILITY

2012 State of Reliability Report

• New in 2012 State of Reliability Report  Tighter integration of sources to ascertain historic Bulk Power System reliability  Focus on providing relevance for both industry and regulators  Additional Data Sources

6

RELIABILITY | ACCOUNTABILITY

Transition to 2012 State of Reliability Report

7

RELIABILITY | ACCOUNTABILITY

Metrics - Improving Trends ALR

Title

1-3

Planning Reserve Margin

1-4

BPS Transmission Related Events Resulting in Loss of Load

2-5

Disturbance Control Events Greater Than Most Severe Single Contingency

6-2

Energy Emergency Alert 3

6-3

Energy Emergency Alert 2

6-11

Automatic AC Transmission Outages Initiated by Failed Protection System Equipment

6-12

Automatic AC Transmission Outages Initiated by Human Error

8

RELIABILITY | ACCOUNTABILITY

Metrics – Inconclusive Trends ALR

Title

2-3

Activation of Under Frequency Load Shedding

2-4

Average Percent Non-Recovery of Disturbance Control Standard (DCS) Events

4-1

Automatic AC Transmission Outages Caused by Protection System Equipment-Related Misoperations

6-12

Automatic AC Transmission Outages Initiated by Human Error

6-14

Automatic AC Transmission Outages Initiated by Failed AC Circuit Equipment

9

RELIABILITY | ACCOUNTABILITY

Metrics – New/No Data ALR

Title

3-5

IROL/SOL Exceedance

6-1

Transmission Constraint Mitigation

6-15

Element Availability Percentage

6-16

Element Unavailability Percentage

1-5

System Voltage Performance

1-12

Interconnection Frequency Response

10

RELIABILITY | ACCOUNTABILITY

Severity Risk Key Concepts

11

RELIABILITY | ACCOUNTABILITY

Severity Risk Index (SRI)

• Focus on significant events and quantify the historic system performance using daily outage data  GADS and TADS daily outages, forced only  MW load loss and restoration duration from disturbance event reports  Could be considered similar to a daily, quarterly or yearly SAIDI metric for distribution systems  If everything was out of service, all load was unserved, the day’s score would be 1000  If no lines or units were out of service and no load was lost the day’s score would be 0  Normal “good days” will measure higher than 0 12

RELIABILITY | ACCOUNTABILITY

Severity Risk Index (SRI)

13

RELIABILITY | ACCOUNTABILITY

Integrated Reliability Indicators (IRI)

• Develop the conceptual model of reliability risks • Establish quantitative measures for evaluating the performance of the indicators • Support risk-informed decision making • May be able to correlate over the long term to predictive measures of reliability risks and if so… • Estimate the effectiveness of reliability risk reduction and/or mitigation

14

RELIABILITY | ACCOUNTABILITY

EDI Calculation DurationDays SRI EDI DurationDays • The EDI is derived from Severity Risk Index (SRI) • SRI = wT ∙(NT) + wG ∙(NG) + wL ∙(LLS) • Where:        15

SRI = severity risk index for a specified event, wL = weighting of load loss, LLS = normalized MW of Load Loss in percent, wT = weighting of transmission lines lost, NT = normalized number of transmission lines lost in percent, wG = weighting of generators lost, NG = normalized number of generators lost in percent RELIABILITY | ACCOUNTABILITY

SRI Calculation SRI

Gen T Weighted _ Loss MW _ LOSS 0.3 0.1 0.6 LLS Genloss _ base T loss _ base

• Load Loss and duration (for 80% of load restored)  Restoration Promptness Level (RPL) o RPL = 1, if restoration < 4 hrs o RPL = 2, if 4 =12 hours

 Load Loss Severity (LLS) = (RPL/3)*(Load Loss MW/Daily Peak Load)

16

RELIABILITY | ACCOUNTABILITY

SRI Base (Denominator) for Normalization

• Normalization Base  Use TADS inventory (200 kV and above) for Tloss_base o Yearly AC Circuits and Transformers inventory data o Average MVA rating for each voltage class (see next slide)

 Use unit MW Ratings in ESD as Gloss base o ESD – Electricity Supply and Demand database

• Aggregation level  Interconnection  NERC -Wide 17

RELIABILITY | ACCOUNTABILITY

SRI Base (Denominator) for Normalization (2)

• Average MVA capacity for AC Circuits o 765 kV → 3000 MVA o 500 kV → 2000 MVA o 345 kV → 1300 MVA o 230 kV → 700 MVA

• Example of AC circuit loss event in EDI: 100 – 230kV, 40 – 345 kV, 10– 500 kV, 3 – 765 kV: NT

18

100 700 40 1300 10 2000 3 2000 2708 700 1078 1300 344 2000 33 3000 151,000 4,084,000 0.037 RELIABILITY | ACCOUNTABILITY

Event Driven Index (EDI) - Example

• Using the combined risk index score SRI, the 2009 risk events summed to 506.117 points; the top five day’s SRI values were:  1/27/09 4.98 SRI points,  1/28/09 3.67 SRI points,  12/15/09 3.22 SRI points

7/1/09 3.70 SRI points 2/11/09 3.32 SRI points

• If EDIyear = success rate for BPS risk events, then EDI2009 = 100 x (365 x 1000 – 506.117)/(365 x 1000) ≈ 99.8613

19

RELIABILITY | ACCOUNTABILITY

Three-Tier Risk Measures

20

RELIABILITY | ACCOUNTABILITY

The Risk Control Reduction Cycle

• Find potential risks to reliability.

• Prioritize the risk clusters to find those risks which are the most severe.

Risk Cluster Identification

Prioritization

Intelligence and Analysis

Solve Problems • Solve the problems to eliminate potential risks to reliability.

21

Actionable Risk Control Steps

• Develop actionable risk control steps.

RELIABILITY | ACCOUNTABILITY

IRI Concept

Event Driven Index (EDI) Measures Risk from Major System Events

Uses the roster of reliability metrics developed by RMWG and approved by OC/PC

22

Condition Driven Index (CDI) Monitors Risk from Key Reliability Metrics

Based on event severity risk index (SRI) values and turns into an availability index

Standards/Statute Driven Index (SDI) Measures Risks from Severe Impact Violations

Identify a subset of standards that have highest impact to reliability and create an index of standards violations

RELIABILITY | ACCOUNTABILITY

IRI Concept (2) • BPS Integrated Reliability Index (IRI):  Event Driven Index (EDI): Based on event severity risk index (SRI) values  Condition Driven Index (CDI): Use a subset of metrics based on selection criteria  Standards/Statute Driven Index (SDI): Identify a subset of standards that have highest impact to reliability and create an index of standards violations

23

RELIABILITY | ACCOUNTABILITY

EDI Trends by Quarter

Event Driven Index (EDI)

100.00 99.95

99.90 99.85 99.80 99.75 99.70

24

RELIABILITY | ACCOUNTABILITY

Standards/Statute Driven Index (SDI) Proposed Approach

• Severe Reliability Impact Statement (RIS)  RIS indicates significance of impact on BPS

• • • •

25

Uses Violation Risk Factor (VRF) Demonstrates Violation Severity Level (VSL) Similar to daily SRI calculation Almost 4 year history violation history of 4,410 confirmed violations

RELIABILITY | ACCOUNTABILITY

SDI Calculation • SDI = (Total Compliance - ∑(wV∙NV/NR))/(Total Compliance) • Range from 0 to 100 • Total Compliance = 1.0 * Number of Days in a specific period • wV = weighting of a particular high risk req. violation • NV = number of violations for the selected high risk req. • NR = number of registered entities required to comply with the high risk req. 26

RELIABILITY | ACCOUNTABILITY

SDI Calculation Example • SDI = 100 ∙ (Total Compliance - ∑(wV∙NV/NR))/(Total Compliance) • SDI = 100 ∙ (1.0 ∙ Days in a Quarter - ∑(wV ∙ NV/NR)) / (1.0∙Days in a Quarter) • 1Q2009:  40 violations (PRC-005 R1)  RIS = Severe, VRF=High, VSL=Severe  A total of 335 TO, 828 GO registered entities  SDI = 100 x (1.0 x 90 - 0.038 x 40/(335+828))/(1.0 x 90) = 99.99

27

RELIABILITY | ACCOUNTABILITY

Possible SDI Trend by Quarter

28

RELIABILITY | ACCOUNTABILITY

Condition Driven Index (CDI)

• Uses metrics approved by OC/PC • Applies SMART criteria to rank the relative importance and weighting for performance • 5 trend ratings are determined per metric performance      29

Significant improvement Slight improvement Inconclusive Slight Deterioration Significant Deterioration RELIABILITY | ACCOUNTABILITY

Possible IRI Attributes • • • •

Scale from 0 to 100 100 would indicate a perfect performance Weighting should be developed and can be adjusted Aggregation at NERC, Interconnection and Region levels • Reporting period – targeting quarterly, reported at year end • Trial indicators until sufficient history is experienced

30

RELIABILITY | ACCOUNTABILITY

IRI Objectives

•Inform industry leaders •Increase transparency •Quantify action effectiveness of risk reduction •Meaningful bulk system performance trends •Guidance on improving system reliability •Support with risk‐informed decision making

31

RELIABILITY | ACCOUNTABILITY

Working with EIA and Sandia

• EIA’s and Sandia's statistical analysis experience and risk-control technologies  Leverage metric development experience for the Nuclear Regulatory Commission (NRC) and the National Aeronautics and Space Administration (NASA)  Statistical significance and confidence interval determination  Integrated strategy to apply science, technology, and engineering capabilities

32

RELIABILITY | ACCOUNTABILITY

IRI Future Steps Present • Severity Risk Index • Adequate Level of Reliability metrics • TADS • GADS • Event Analysis Database

33

IRI Process • Statistically link reliability reduction cause & effect • Link event, precursor indicators, and outage datasets to explore initiating events • Find top initiating events • Develop reliability index framework

Future • Use IRI to quantitatively measure BES reliability • Focus on improving top initiating event performance • Help focus industry on relevant reliability risks to improve BES reliability

RELIABILITY | ACCOUNTABILITY

Questions and Answers

34

RELIABILITY | ACCOUNTABILITY