Lessons from the Real World Webinar Series

Lessons from the Real World Webinar Series Forget about Big Data! (Start with small data, then work your way up) April 2, 2014 Agenda 1. The probl...
Author: Justin Lawson
0 downloads 2 Views 5MB Size
Lessons from the Real World Webinar Series

Forget about Big Data! (Start with small data, then work your way up) April 2, 2014

Agenda 1. The problem with starting big 2. Solution: Starting Small 3. Lessons learned at Centerpoint Energy

4. Lessons learned at Duke Energy

Today’s Presenters Jesse Berst Host & Moderator

SmartGridNews.com

Larsh Johnson

Aaron DeYonker

Chief Technology Officer Smart Grid Software & Services

Vice President of Products

eMeter, A Siemens Business

eMeter, A Siemens Business

Bill Bell Technology Director, Analytics & Data Services

CenterPoint Energy

Raiford Smith Director, Emerging Technology

Duke Energy

Larsh Johnson, Siemens

Name

Larsh Johnson

Background

Chief Technology Officer IC Smart Grid Software & Services– Siemens •

Founder of eMeter



Spearheads innovative energy and water solutions across Siemens and industry partner ecosystem



Previously co-founder and CTO for CellNet Data Systems; Director of Product Development at Interactive Communications; an Engineering Manager at Digital Optics Corporation; and an Electrical Engineer at Systems Control Inc



Holds a BS and MS in Mechanical Engineering from Stanford University and founding member of DOE’s GWAC (Gridwise Architecture Council)

Volume, velocity and variety of data increased dramatically within the last 20-30 years Evolution of data management within power industry 1980s

1990s

Until today

Future

Volume

Megabytes

Gigabytes

Terabytes

Petabytes

Velocity

~300 values/h, e.g. generated power

~1,000 values/min, e.g. CO2 Emission

>5,000 values/sec, e.g. AMI, PMUs

>10,000 values/ms, e.g. speed curves

Variety

Manually combined with other data once a month

Automatic transfer once a Transfer, connect and week, evaluation typically via process with other internal EXCEL and external data-sources

Real-time processing of structured and unstructured data (historical, actual and forecast)

Logic

Process Control

Plant Management

Fleet Management

Intelligent grids

Purpose

Operation Reports

Root cause analysis

Predictive diagnostics, advanced planning and forecasting

Real-time forecasting, real-time automated improvement and operation

Question

What happened last month?

What happened last week? Why did it happen?

© Siemens AG 2013 All rights reserved.

What shall we do? Source: E TI eMeter, A Siemens Business

Biggest impediments to Utility analytics efforts

Lack of trained personnel to utilize data analytics

35%

Lack of Hardware (Data Storage, computing capability)

24%

Lack of Software to manage large amount of data Lack of sufficient (budget constraints)

18%

funding

Low priority of needs in the utility

16%

6% Source: ZPRYME Smart Grid Utility and Executive Survey, 2013

© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

Aaron DeYonker, Siemens

Name

Aaron DeYonker

Background

Vice President of Products– eMeter, A Siemens Business •

Over 15 years in IT product development for eMeter



Currently manages the entire product portfolio and global roadmap for a leading Smart Grid software platform, geared towards more efficient use of our natural resources



Previously led product and program management teams for companies such as Microsoft and WebTV



Industry conference speaker, and achieved lead ‘Visionary’ status in Gartner Magic Quadrant 2 years in a row for eMeter’s enterprise software platform



Graduate of the Honors Program in the College of Literature, Sciences and Arts at the University of Michigan, Ann Arbor

Analytics – cutting through the hype

What specific business problems will it help solve ? How much time before I see any benefits ? Can I start small and extend gradually ?

© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

Advanced Analytics – 3 Building Blocks

Data Integration Platform

Analytical Models & algorithms

Business Workflow Integration

© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

Understanding Changes Weekly loading of a transformer – 2003 and 2011 Load in kW

Load profile 2003

Load profile 2011

200

100

0

-100

-200

-300 12:00

12:00

0:00

12:00

0:00

12:00

0:00

12:00

0:00

12:00

0:00

12:00

0:00

12:00

0:00

Source: LEW © Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

Quantifying Loss of Equipment Life

© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

Questions?

© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business

William Bell, CenterPoint Energy

Background

Name

William Bell

Technology Director, Analytics & Data Services– CenterPoint Energy •

Chaired the ERCOT Texas Test Plan Team subcommittee which developed the certification plans for the Texas Market



Responsible for the successful systems implementation of the Advanced Metering Project on a very constrained 6-month schedule mandated by the Public Utility Commission of Texas which went live in August 2009



Responsible for implementation of CenterPoint's Smart Grid Advanced Distribution Management System scheduled to be completed in late 2013 or early 2014.



Bachelor's of Business Administration Magna Cum Laude from the University of TX Arlington in 1979 and a law degree from the University of Houston in 1984

CenterPoint Energy (CNP)       

Electric Transmission & Distribution Natural Gas Distribution Competitive Natural Gas Sales & Services © 2012 CenterPoint Energy Proprietary and Confidential Information

Headquartered in Houston, Texas Serving 5.5 million electric & gas metered customers $22.8 billion in assets $7.5 billion in revenue More than 8,700 employees Over 135 years of service to our communities Electric transmission and distribution delivery business  Over 2.2 million metered customers in Houston area  17.3 GW peak demand  79 GW hours delivered annually  233 substations  3,739 miles of transmission  49,162 miles of distribution

Texas Electric Market Power Generating Companies Unregulated

Unregulated

 

Transmission and Distribution Utilities (TDUs)



Compete for customers Bill customers Issue disconnect, reconnect orders to TDUs

Regulated 

Own and maintain power lines



Deliver power to customers



Read meters

Step-down Substation

Restore power after outages 



Commercial Customer

Deliver data Residential Customer

© 2012 CenterPoint Energy Proprietary and Confidential Information

Analytics & Data Services Delivering Information you can “Trust” Enabling Efficient Data Driven Decision Making

© 2012 CenterPoint Energy Proprietary and Confidential Information

Why Analytics?? Analytics as a Discipline™ is taking that which we knew, that which we know now, and enrichment from other sources, and coalescing all that data into simple, actionable insight.™

© 2012 CenterPoint Energy Proprietary and Confidential Information

How to Define, Validate and Deliver Analytics?

Define

Deliver

Phase One

Phase Three

Validate Phase Two

© 2012 CenterPoint Energy Proprietary and Confidential Information

Progression of Analytics at CNP

Predictive Analytics Situational Awareness Reports

© 2012 CenterPoint Energy Proprietary and Confidential Information

Initial Progress 2012

Value

• Develop Analytics Delivery Value Model • “Analytics as a Discipline”™©

• Define and build Analytics Foundational Technologies • eMA (All things Meter); ISAS (Correlations); Streams (Real Time) • Basic Visualization (Google Earth); Tivoli (alerts and Geospatial Rendering); Foundations • Data Services (data movement); BOBJ (client interface)

Top 5

• Diversion Analytics (continues to evolve and improve) • Financial Unbilled Revenue Reporting (in production for almost 2 years) • Transformer Load Management (continues to involve into Equipment Load Management) • Meter Alert Trending (used in Diversion, Outage, Comms reliability and others) • Load Profile Flag (deemed unnecessary by clients)

© 2012 CenterPoint Energy Proprietary and Confidential Information

21

In 2013 The Team Successfully completed several Major Initiatives all in conjunction with the Strategic Plans of GMO, IG, EMO, BT, DVAL, TCC

Situational Awareness

• Provide real time situational awareness and correlations for Telecomms, Outage, Distribution Dispatching and Distribution Operations • (“Correlating data in real time to enable Operations to Affect the Outcome”) • Provide Instant Replay Capabilities for Training and Storm Preparedness • Provide Real Time Solutions to Identify Data Anomalies in support of Corporate Security • (“Eyes on the Horizon Threat Detection”)

• Enhanced Diversion Detection and Dispositioning combined with Usage • (“Stop the tax of energy theft in days rather than months or years”) • Enhanced Transformer Load Management, Connectivity and Predictive Loading also enabling Fuse Revenue & and Step Transformer Load Management • (“Protect the assets before they fail, enable preventative maintenance”) Asset Protection

Business Transformation

• Support for Business Transformation Initiatives, Right Crew, Right Place at Right Time, Proactive Resolution of Equipment Issues and Fleet Support • (Know Where Your Crews Are and Protect your Equipment”) • Financial and Regulatory Month End Revenue Estimation • (“Move from 90% estimation to a .01% estimation, Know your revenues” )

© 2012 CenterPoint Energy Proprietary and Confidential Information

22

Tamper / Diversion & Disposition Process Refine analysis

Manage evidence and back billing

Detection Identify premise

Generate work order

Disposition field investigation

Investigate

Reporting

Collect enrichment data

Support prosecution

Revenue Protection

Truck Roll No Problem Found

Investigate

Detection © 2012 CenterPoint Energy Proprietary and Confidential Information

Report / Learn

Disposition

Diversion Dashboards

© 2012 CenterPoint Energy Proprietary and Confidential Information

24

Single Transformer Report with Actual and Predicted Load + connectivity

© 2012 CenterPoint Energy Proprietary and Confidential Information

Financial & Regulatory Month End Revenue Estimation/Calculation

Maximizing Smart Meter Technology

• Daily Register Readings from each meter • 96 interval readings from each meter • Moved from 90+% estimation to .01% estimation

Data Driven Decision Making

• No longer dependent on monthly register reads • Weather Normalization can be more accurate supported by timely and comprehensive interval data rather than traditional sampling methods • Smart Meter interval data will be required in future rate cases for cost allocation and weather normalization

Predictive Analytics Benefits Financial Forecasts

• Enhanced understanding by rate class of recent trends in consumption • Enhanced ability to anticipate future loads • Our New Automated calculation has replaced arduous manual efforts with more accurate results

This solution went into production effective July 1, 2012. © 2012 CenterPoint Energy Proprietary and Confidential Information

Meter Outage Events (Views of Several Screens During the event)

Meter Outage Events Trouble Cases and Available Crews

28 © 2012 CenterPoint Energy Proprietary and Confidential Information

Real Time Situational Awareness Dashboard in Use Today

© 2012 CenterPoint Energy Proprietary and Confidential Information

Real Time Comms Dashboard in Use Today in Telecomms Control Room

© 2012 CenterPoint Energy Proprietary and Confidential Information

Real Time Situational Awareness Manages / Monitors System Performance Real time monitoring of WiMax communication system performance

• Application developed by our Analytics Data Services team • Leverages GIS and Google Earth

• Geospatial view of communication sites – Green – Up – Yellow – Either primary or secondary failed – Red – both primary and secondary failed

© 2012 CenterPoint Energy Proprietary and Confidential Information

Where We are Going 2014 and Beyond Refined Strategic Imperatives to align with new Vision 2015 and beyond •

A

D

S

2011-2012 2011 • • • • • •

• •

Drive to full automated orders with predicted “next best action” • Implement System of Systems Manager • Advanced Analytics correlations & predictions 2014 • Enhanced Situational Awareness with Weather and more • Establish and Operationalize Analytics Value Team • Interface with CVP and ADMS to support advanced analytics and Situational Awareness • Deploy FINREG initiative • Situational Awareness for GAS Ops, Fleet, DPD 2013 • Real Time Situational Awareness DVAL and Comms • Enhanced Diversion Detection with Usage • Deployed TLM/ELM • Financial and Regulatory Month end calculation • Operationalize Right Crew Right Place Initiatives

Refined Methodology “ Analytics as a Discipline Install Analytics Foundations and improve deliveries

Analytics Initiative Kicked Off Charter Established and Owner Named Mission Vision and Scope Defined Develop a solid Method for Analytics Investigate and Define Analytics Tools Operationalize the Top 5

© 2012 CenterPoint Energy Proprietary and Confidential Information

Benefits of ADS What the Analytics Teams do is take mountains of data and provide business value by extracting meaningful information and generating actionable tasks that:  Protect the Grid  Improve Communications performance  Reduce the back office cost of revenue collection with improved revenue estimations  Meeting Regulatory reporting requirements more efficiently and effectively  Protect the company and the Market from Diversion  Improve the quality of field work by issuing orders for the maintenance of equipment instead of rolling after outages  Developing a way for CNP to provide aggregated Demand Response as a resource to ERCOT and the Texas De-Regulated Market in the event generation capacity does not meet projected load  and many others. © 2012 CenterPoint Energy Proprietary and Confidential Information

CNP Analytics Technical Environment Overview

CNP Analytics Technical Environment Overview ERCOT EAI

NAESB

EnergyIP MDM

S O A

Data Collection Engine

Retailers Retailers Retailers (CRs) (CRs) (CRs)

Texas Market Systems

Legacy Systems CIS, GIS, BW,OAS, Comms, Mobile Data

SAP Data Services for Interfaces between Legacy and Analytics

EnergyIP Analytics Foundation IBM Pure Data for Operational Analytics ADS User Interface Portal

Workflow & Automation Interfaces, Tivoli, Streams, etc.

SMT

Take Out Points (TOPs) Cell Relays

Meters

Develop Data Foundation – Improve Data Delivery – Learn from Results © 2012 CenterPoint Energy Proprietary and Confidential Information

Department of Energy Disclaimer

© 2012 CenterPoint Energy Proprietary and Confidential Information

Questions?

Raiford Smith, Duke Energy

Name

Raiford Smith

Background

Director, Emerging Technology – Duke Energy •

Over 21 years in the industry: Technology – T&D, SmartGrid, IT and telecommunications; Marketing; Legal; Finance and Strategy



Over 12 years with Duke Energy



Previously with Enron, GenOn Energy, and Southern Company Energy Solutions



Holds a BS from University of Georgia, MBA University of Virginia and a JD from Charlotte School of Law

Big Data Analytics and Our Evolving Strategy Duke Energy’s Emerging Technology Organization Raiford Smith Director, Grid Emerging Technology

Emerging Technology Roles and Responsibilities • Emerging Technology is responsible for: – – – –

Technology development and testing New technology strategy, roadmap, risk and opportunity identification Lab/field testing of new technology Establish business value and formulation of initial business case development Transmission Emerging Technology

Grid Modernization

“Develop”

“Deploy”

Distribution Information Technology “Operate”

Duke Energy – Confidential and Proprietary Information

WHO

HOW

WHAT

Analytics in Utilities Today Operational Improvement Ingest Translate Context

Generation

Store

Earnings Enhancement

Business Transformation

Analytics

Business Intelligence

Validation Discover

Transmission

Distribution

Model

Train

Customer

Other

Unfortunately, most solutions are silo’d, involve some amount of proprietary equipment (and a LOT of middle-ware), and appear overly-focused on AMI use cases.

Current Analytics Framework Business Intelligence

Visualization / Discovery Custom Contextualization Enterprise Message Busses (Translation) Storage (Structured)

Storage (Structured)

Storage (Structured)

Data Collection

Data Collection

Data Collection

Data Collection

Data Collection

Security

Security

Security

Security

Security

Smart Assets

Smart Meter Transformer Line Sensor

X Protection & Control

Street Light

Distributed Energy Resources

Weather Sensor

Comm. Node

Most of our enterprise architecture is brittle and silo’d – designed to handle specific use cases rather than providing the necessary speed and flexibility to explore data.

Emerging Technology Analytics Framework Business Intelligence

S e c u r i t y

Smart Assets

Analytics at the Central Office

Visualization / Discovery Storage (Structured)

Storage (Unstructured)

CIM (Contextualization)

Analytics at the Substation

Field Message Busses (Translation)

Analytics at the Edge

Smart Meter Transformer Line Sensor

X Protection & Control

Street Light

Distributed Energy Resources

Weather Sensor

Comm. Node

Edge-based contextualization and interoperability with distributed applications are the keys to creating a rich analytics environment that can be mined for value page 43

Use Cases by Category and Vendor Coverage 30

25

Use Cases

20

Vendor Coverage

15

10

5

0

“Big data” appears to be a target rich environment. page 44

Is Skewed Data (or Data Availability) Leading to Skewed Use Cases? Devices Fully Representive?

Data Fully Represents Infrastructure?

Meters

Yes

Yes

Most familiar to all vendors

Communication Nodes

Yes

No

Some value overlaps with meter, EE, DR

Transformers

Yes

No

Missing nameplate information

Capacitor Banks

No

No

Limited information from Pi Historian

Reclosers

No

No

Limited information from Pi Historian

Line Sensor

Yes

No

Missing event and operations data

Asset Nameplate Data

No

No

No asset attribute data included

Customer Program Data

N/A

Yes

Duke program data for all customers.

Socioeconomic Data

N/A

No

Missing data on many customers

Weather Data

No

No

No McAlpine weather station data used

Category

Comments

A majority of the data available comes from AMI systems. Is this skewing our focus towards use cases that utilize AMI data?

Analytics Cost Reduction Opportunities Capital Spend by Major Business Function (2012) Customer + Other T&D Electric

3%

6%

O&M Spend by Major Business Function (2012) 100% 6%

9%

Other

23%

15%

Customer

10%

16%

T&D Electric

62%

60%

Generation (Non-Fuel)

22%

47%

Generation (Non-Fuel)

75%

47%

Duke Energy Corporation

Industry Set

Duke Energy Corporation

Industry Set

Based on spending, generation appears to be the biggest “target-rich” environment. Source: FERC Form 1; SNL; Accenture Customer function is only associated with O&M spend; Other category represents smaller business functions (e.g. corporate center) Industry Set represents 69 integrated investor owned utilities

Duke Energy’s Data Modeling and Analytics Initiative (DMAI) • Duke Energy needed to form a comprehensive “big data / analytics” strategy. • In 2013, industry experts who participated in the initiative received access to a cross-sectional dataset (1 week of comprehensive data) to review & analyze • Sample use cases were provided to “prime the pump” • Participants were asked to provide: – New use case ideas – Insights into what other types of data should be collected – Data warehouse, models and structures as well as data ingestion and extraction methods – Evaluation of the relative value of each use case provided

Duke Energy’s DMAI is an Ongoing, Collaborative Process Duke Stakeholders

Vendor Analytics

New Possibilities New Products & Services

Customer Analytics

Cyber Security

Emerging Technology

Customer Prospecting

Grid Modernization

Load & Weather Forecasting

Information Technology

Outage Management Grid Management

Load Forecasting

Vegetation Management

Meteorology

Revenue Protection Asset Management

Metering Services Supply Chain

Improved Customer Sat.

AT&T

Enhanced Operations

Example Tools Used by Vendors OSS DB

Languages

Analytics Platforms

Key Technology Discussions • RDBMS - NoSQL • OSS - Proprietary • Flexible Disaggregation for Discovery • Scalability, Warehousing

Frameworks

Context for Value Estimates Duke Energy Profile (2012) Value Code

Value Estimate

1-$

Thousands

2 - $$

Millions

3 - $$$

Tens of Millions

4 - $$$

Hundreds of Millions

5 - $$$$$

Billions

• • • • •

Market Cap: $45B Revenues: $19.6B Total Assets: $112B Electric Customers: 7.2M Total Franchise Capacity – 48,000 MW conventional – 1,700 MW renewables

• Estimates were rolled up into a “total value potential” metric

Value Estimates by Category Value Score

Std Dev

# Vendors Responses

Security

3.6

0.49

5

Limited information

Revenue Protection

3.5

1.009

13

Includes DG, retail competition and theft

Distribution Grid Analysis

3.4

0.881

11

Diverse examples of value cited

Load Forecasting

3.4

1.115

12

Asset Management Analysis

3.3

1.106

12

High score even within supporting data

Customer Analysis

3.1

1.062

15

Some value overlaps with meter, EE, DR

Demand Response

3.1

0.799

14

Meter Analytics

2.9

0.957

15

Outage Analysis

2.9

0.954

12

Energy Efficiency

2.9

0.828

13

Distributed Generation

2.8

0.916

9

Communications

2.5

0.5

6

Category

Comments

Most familiar to all vendors

Varying opinions on value of EE

DMAI - Lessons Learned • No one (including Duke Energy) is strong in all 3 critical analytics areas: technical, statistical, and industry knowledge • If data can be federated, significant value can be created. Similarly, silo’d data has numerous issues that limit potential value • Culture and process / organizational change can be just as daunting as the technology itself • Unstructured data helps discover new value and eliminates potential bias inherent in pre-determined structures • On the other hand, unstructured data is not suitable for every use case, nor does it solve every problem

Tools Technical Review – Insights - 1 • Digital sensors and appliances, low-cost communications and storage, demand for information, and the Internet of Things (IoT), have created an IT environment where “Big Data” is a problem. • Duke Energy may not have a Big Data problem yet; current state is that we have a lot of “small data” problems. • Each use case category evaluated contains silos of systems and data (and the categories themselves are silos). The greatest value will come when all data can be easily utilized for analysis.

• The majority of Big Data analytics vendors do not know our business, but they are learning fast.

Tools Technical Review – Insights - 2 • The strength of the Relational Database (RDBMS) is its schema, but this is also its greatest weakness since the schema is inflexible and difficult to change. • When questions about the data become more complex and cross silos, then complex RDBMS schemas break down. • The strength of Hadoop, NoSQL, and MapReduce are their abilities to store data in its original form in simple structures and process it with low-cost parallel computers. • Hadoop usage can reduce IT costs and can scale, runs on parallelized commodity hardware, and OSS has no license costs.

• IT Tools for ingestion and storage are becoming more commoditized and open; the Analytics and BI applications tend to be specialized by vendor and are not yet standardized.

page 54

Next Steps • Continue to embrace field device interoperability and distributed applications • Consolidate use cases with other Duke Energy and EPRI groups – Standardize categories – Identify and consolidate similar use cases • Continue vendor dialogue and look for partnership opportunities

• Develop requirements for Sandbox 2.0 • Define Duke Energy’s Big Data strategy and architecture

Questions?

Thank You! You will receive a link to download a copy of the slides to the email you used to register. Jesse Berst Host & Moderator

SmartGridNews.com [email protected]

Larsh Johnson CTO IC Smart Grid Software & Services

eMeter, A Siemens Business

Aaron DeYonker Vice President of Products

eMeter, A Siemens Business

[email protected]

[email protected]

Bill Bell

Raiford Smith

Technology Director, Analytics & Data Services

Director, Emerging Technology

CenterPoint Energy

[email protected]

[email protected]

Duke Energy