Lessons from the Real World Webinar Series
Forget about Big Data! (Start with small data, then work your way up) April 2, 2014
Agenda 1. The problem with starting big 2. Solution: Starting Small 3. Lessons learned at Centerpoint Energy
4. Lessons learned at Duke Energy
Today’s Presenters Jesse Berst Host & Moderator
SmartGridNews.com
Larsh Johnson
Aaron DeYonker
Chief Technology Officer Smart Grid Software & Services
Vice President of Products
eMeter, A Siemens Business
eMeter, A Siemens Business
Bill Bell Technology Director, Analytics & Data Services
CenterPoint Energy
Raiford Smith Director, Emerging Technology
Duke Energy
Larsh Johnson, Siemens
Name
Larsh Johnson
Background
Chief Technology Officer IC Smart Grid Software & Services– Siemens •
Founder of eMeter
•
Spearheads innovative energy and water solutions across Siemens and industry partner ecosystem
•
Previously co-founder and CTO for CellNet Data Systems; Director of Product Development at Interactive Communications; an Engineering Manager at Digital Optics Corporation; and an Electrical Engineer at Systems Control Inc
•
Holds a BS and MS in Mechanical Engineering from Stanford University and founding member of DOE’s GWAC (Gridwise Architecture Council)
Volume, velocity and variety of data increased dramatically within the last 20-30 years Evolution of data management within power industry 1980s
1990s
Until today
Future
Volume
Megabytes
Gigabytes
Terabytes
Petabytes
Velocity
~300 values/h, e.g. generated power
~1,000 values/min, e.g. CO2 Emission
>5,000 values/sec, e.g. AMI, PMUs
>10,000 values/ms, e.g. speed curves
Variety
Manually combined with other data once a month
Automatic transfer once a Transfer, connect and week, evaluation typically via process with other internal EXCEL and external data-sources
Real-time processing of structured and unstructured data (historical, actual and forecast)
Logic
Process Control
Plant Management
Fleet Management
Intelligent grids
Purpose
Operation Reports
Root cause analysis
Predictive diagnostics, advanced planning and forecasting
Real-time forecasting, real-time automated improvement and operation
Question
What happened last month?
What happened last week? Why did it happen?
© Siemens AG 2013 All rights reserved.
What shall we do? Source: E TI eMeter, A Siemens Business
Biggest impediments to Utility analytics efforts
Lack of trained personnel to utilize data analytics
35%
Lack of Hardware (Data Storage, computing capability)
24%
Lack of Software to manage large amount of data Lack of sufficient (budget constraints)
18%
funding
Low priority of needs in the utility
16%
6% Source: ZPRYME Smart Grid Utility and Executive Survey, 2013
© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
Aaron DeYonker, Siemens
Name
Aaron DeYonker
Background
Vice President of Products– eMeter, A Siemens Business •
Over 15 years in IT product development for eMeter
•
Currently manages the entire product portfolio and global roadmap for a leading Smart Grid software platform, geared towards more efficient use of our natural resources
•
Previously led product and program management teams for companies such as Microsoft and WebTV
•
Industry conference speaker, and achieved lead ‘Visionary’ status in Gartner Magic Quadrant 2 years in a row for eMeter’s enterprise software platform
•
Graduate of the Honors Program in the College of Literature, Sciences and Arts at the University of Michigan, Ann Arbor
Analytics – cutting through the hype
What specific business problems will it help solve ? How much time before I see any benefits ? Can I start small and extend gradually ?
© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
Advanced Analytics – 3 Building Blocks
Data Integration Platform
Analytical Models & algorithms
Business Workflow Integration
© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
Understanding Changes Weekly loading of a transformer – 2003 and 2011 Load in kW
Load profile 2003
Load profile 2011
200
100
0
-100
-200
-300 12:00
12:00
0:00
12:00
0:00
12:00
0:00
12:00
0:00
12:00
0:00
12:00
0:00
12:00
0:00
Source: LEW © Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
Quantifying Loss of Equipment Life
© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
Questions?
© Siemens AG 2013 All rights reserved. eMeter, A Siemens Business
William Bell, CenterPoint Energy
Background
Name
William Bell
Technology Director, Analytics & Data Services– CenterPoint Energy •
Chaired the ERCOT Texas Test Plan Team subcommittee which developed the certification plans for the Texas Market
•
Responsible for the successful systems implementation of the Advanced Metering Project on a very constrained 6-month schedule mandated by the Public Utility Commission of Texas which went live in August 2009
•
Responsible for implementation of CenterPoint's Smart Grid Advanced Distribution Management System scheduled to be completed in late 2013 or early 2014.
•
Bachelor's of Business Administration Magna Cum Laude from the University of TX Arlington in 1979 and a law degree from the University of Houston in 1984
CenterPoint Energy (CNP)
Electric Transmission & Distribution Natural Gas Distribution Competitive Natural Gas Sales & Services © 2012 CenterPoint Energy Proprietary and Confidential Information
Headquartered in Houston, Texas Serving 5.5 million electric & gas metered customers $22.8 billion in assets $7.5 billion in revenue More than 8,700 employees Over 135 years of service to our communities Electric transmission and distribution delivery business Over 2.2 million metered customers in Houston area 17.3 GW peak demand 79 GW hours delivered annually 233 substations 3,739 miles of transmission 49,162 miles of distribution
Texas Electric Market Power Generating Companies Unregulated
Unregulated
Transmission and Distribution Utilities (TDUs)
Compete for customers Bill customers Issue disconnect, reconnect orders to TDUs
Regulated
Own and maintain power lines
Deliver power to customers
Read meters
Step-down Substation
Restore power after outages
Commercial Customer
Deliver data Residential Customer
© 2012 CenterPoint Energy Proprietary and Confidential Information
Analytics & Data Services Delivering Information you can “Trust” Enabling Efficient Data Driven Decision Making
© 2012 CenterPoint Energy Proprietary and Confidential Information
Why Analytics?? Analytics as a Discipline™ is taking that which we knew, that which we know now, and enrichment from other sources, and coalescing all that data into simple, actionable insight.™
© 2012 CenterPoint Energy Proprietary and Confidential Information
How to Define, Validate and Deliver Analytics?
Define
Deliver
Phase One
Phase Three
Validate Phase Two
© 2012 CenterPoint Energy Proprietary and Confidential Information
Progression of Analytics at CNP
Predictive Analytics Situational Awareness Reports
© 2012 CenterPoint Energy Proprietary and Confidential Information
Initial Progress 2012
Value
• Develop Analytics Delivery Value Model • “Analytics as a Discipline”™©
• Define and build Analytics Foundational Technologies • eMA (All things Meter); ISAS (Correlations); Streams (Real Time) • Basic Visualization (Google Earth); Tivoli (alerts and Geospatial Rendering); Foundations • Data Services (data movement); BOBJ (client interface)
Top 5
• Diversion Analytics (continues to evolve and improve) • Financial Unbilled Revenue Reporting (in production for almost 2 years) • Transformer Load Management (continues to involve into Equipment Load Management) • Meter Alert Trending (used in Diversion, Outage, Comms reliability and others) • Load Profile Flag (deemed unnecessary by clients)
© 2012 CenterPoint Energy Proprietary and Confidential Information
21
In 2013 The Team Successfully completed several Major Initiatives all in conjunction with the Strategic Plans of GMO, IG, EMO, BT, DVAL, TCC
Situational Awareness
• Provide real time situational awareness and correlations for Telecomms, Outage, Distribution Dispatching and Distribution Operations • (“Correlating data in real time to enable Operations to Affect the Outcome”) • Provide Instant Replay Capabilities for Training and Storm Preparedness • Provide Real Time Solutions to Identify Data Anomalies in support of Corporate Security • (“Eyes on the Horizon Threat Detection”)
• Enhanced Diversion Detection and Dispositioning combined with Usage • (“Stop the tax of energy theft in days rather than months or years”) • Enhanced Transformer Load Management, Connectivity and Predictive Loading also enabling Fuse Revenue & and Step Transformer Load Management • (“Protect the assets before they fail, enable preventative maintenance”) Asset Protection
Business Transformation
• Support for Business Transformation Initiatives, Right Crew, Right Place at Right Time, Proactive Resolution of Equipment Issues and Fleet Support • (Know Where Your Crews Are and Protect your Equipment”) • Financial and Regulatory Month End Revenue Estimation • (“Move from 90% estimation to a .01% estimation, Know your revenues” )
© 2012 CenterPoint Energy Proprietary and Confidential Information
22
Tamper / Diversion & Disposition Process Refine analysis
Manage evidence and back billing
Detection Identify premise
Generate work order
Disposition field investigation
Investigate
Reporting
Collect enrichment data
Support prosecution
Revenue Protection
Truck Roll No Problem Found
Investigate
Detection © 2012 CenterPoint Energy Proprietary and Confidential Information
Report / Learn
Disposition
Diversion Dashboards
© 2012 CenterPoint Energy Proprietary and Confidential Information
24
Single Transformer Report with Actual and Predicted Load + connectivity
© 2012 CenterPoint Energy Proprietary and Confidential Information
Financial & Regulatory Month End Revenue Estimation/Calculation
Maximizing Smart Meter Technology
• Daily Register Readings from each meter • 96 interval readings from each meter • Moved from 90+% estimation to .01% estimation
Data Driven Decision Making
• No longer dependent on monthly register reads • Weather Normalization can be more accurate supported by timely and comprehensive interval data rather than traditional sampling methods • Smart Meter interval data will be required in future rate cases for cost allocation and weather normalization
Predictive Analytics Benefits Financial Forecasts
• Enhanced understanding by rate class of recent trends in consumption • Enhanced ability to anticipate future loads • Our New Automated calculation has replaced arduous manual efforts with more accurate results
This solution went into production effective July 1, 2012. © 2012 CenterPoint Energy Proprietary and Confidential Information
Meter Outage Events (Views of Several Screens During the event)
Meter Outage Events Trouble Cases and Available Crews
28 © 2012 CenterPoint Energy Proprietary and Confidential Information
Real Time Situational Awareness Dashboard in Use Today
© 2012 CenterPoint Energy Proprietary and Confidential Information
Real Time Comms Dashboard in Use Today in Telecomms Control Room
© 2012 CenterPoint Energy Proprietary and Confidential Information
Real Time Situational Awareness Manages / Monitors System Performance Real time monitoring of WiMax communication system performance
• Application developed by our Analytics Data Services team • Leverages GIS and Google Earth
• Geospatial view of communication sites – Green – Up – Yellow – Either primary or secondary failed – Red – both primary and secondary failed
© 2012 CenterPoint Energy Proprietary and Confidential Information
Where We are Going 2014 and Beyond Refined Strategic Imperatives to align with new Vision 2015 and beyond •
A
D
S
2011-2012 2011 • • • • • •
• •
Drive to full automated orders with predicted “next best action” • Implement System of Systems Manager • Advanced Analytics correlations & predictions 2014 • Enhanced Situational Awareness with Weather and more • Establish and Operationalize Analytics Value Team • Interface with CVP and ADMS to support advanced analytics and Situational Awareness • Deploy FINREG initiative • Situational Awareness for GAS Ops, Fleet, DPD 2013 • Real Time Situational Awareness DVAL and Comms • Enhanced Diversion Detection with Usage • Deployed TLM/ELM • Financial and Regulatory Month end calculation • Operationalize Right Crew Right Place Initiatives
Refined Methodology “ Analytics as a Discipline Install Analytics Foundations and improve deliveries
Analytics Initiative Kicked Off Charter Established and Owner Named Mission Vision and Scope Defined Develop a solid Method for Analytics Investigate and Define Analytics Tools Operationalize the Top 5
© 2012 CenterPoint Energy Proprietary and Confidential Information
Benefits of ADS What the Analytics Teams do is take mountains of data and provide business value by extracting meaningful information and generating actionable tasks that: Protect the Grid Improve Communications performance Reduce the back office cost of revenue collection with improved revenue estimations Meeting Regulatory reporting requirements more efficiently and effectively Protect the company and the Market from Diversion Improve the quality of field work by issuing orders for the maintenance of equipment instead of rolling after outages Developing a way for CNP to provide aggregated Demand Response as a resource to ERCOT and the Texas De-Regulated Market in the event generation capacity does not meet projected load and many others. © 2012 CenterPoint Energy Proprietary and Confidential Information
CNP Analytics Technical Environment Overview
CNP Analytics Technical Environment Overview ERCOT EAI
NAESB
EnergyIP MDM
S O A
Data Collection Engine
Retailers Retailers Retailers (CRs) (CRs) (CRs)
Texas Market Systems
Legacy Systems CIS, GIS, BW,OAS, Comms, Mobile Data
SAP Data Services for Interfaces between Legacy and Analytics
EnergyIP Analytics Foundation IBM Pure Data for Operational Analytics ADS User Interface Portal
Workflow & Automation Interfaces, Tivoli, Streams, etc.
SMT
Take Out Points (TOPs) Cell Relays
Meters
Develop Data Foundation – Improve Data Delivery – Learn from Results © 2012 CenterPoint Energy Proprietary and Confidential Information
Department of Energy Disclaimer
© 2012 CenterPoint Energy Proprietary and Confidential Information
Questions?
Raiford Smith, Duke Energy
Name
Raiford Smith
Background
Director, Emerging Technology – Duke Energy •
Over 21 years in the industry: Technology – T&D, SmartGrid, IT and telecommunications; Marketing; Legal; Finance and Strategy
•
Over 12 years with Duke Energy
•
Previously with Enron, GenOn Energy, and Southern Company Energy Solutions
•
Holds a BS from University of Georgia, MBA University of Virginia and a JD from Charlotte School of Law
Big Data Analytics and Our Evolving Strategy Duke Energy’s Emerging Technology Organization Raiford Smith Director, Grid Emerging Technology
Emerging Technology Roles and Responsibilities • Emerging Technology is responsible for: – – – –
Technology development and testing New technology strategy, roadmap, risk and opportunity identification Lab/field testing of new technology Establish business value and formulation of initial business case development Transmission Emerging Technology
Grid Modernization
“Develop”
“Deploy”
Distribution Information Technology “Operate”
Duke Energy – Confidential and Proprietary Information
WHO
HOW
WHAT
Analytics in Utilities Today Operational Improvement Ingest Translate Context
Generation
Store
Earnings Enhancement
Business Transformation
Analytics
Business Intelligence
Validation Discover
Transmission
Distribution
Model
Train
Customer
Other
Unfortunately, most solutions are silo’d, involve some amount of proprietary equipment (and a LOT of middle-ware), and appear overly-focused on AMI use cases.
Current Analytics Framework Business Intelligence
Visualization / Discovery Custom Contextualization Enterprise Message Busses (Translation) Storage (Structured)
Storage (Structured)
Storage (Structured)
Data Collection
Data Collection
Data Collection
Data Collection
Data Collection
Security
Security
Security
Security
Security
Smart Assets
Smart Meter Transformer Line Sensor
X Protection & Control
Street Light
Distributed Energy Resources
Weather Sensor
Comm. Node
Most of our enterprise architecture is brittle and silo’d – designed to handle specific use cases rather than providing the necessary speed and flexibility to explore data.
Emerging Technology Analytics Framework Business Intelligence
S e c u r i t y
Smart Assets
Analytics at the Central Office
Visualization / Discovery Storage (Structured)
Storage (Unstructured)
CIM (Contextualization)
Analytics at the Substation
Field Message Busses (Translation)
Analytics at the Edge
Smart Meter Transformer Line Sensor
X Protection & Control
Street Light
Distributed Energy Resources
Weather Sensor
Comm. Node
Edge-based contextualization and interoperability with distributed applications are the keys to creating a rich analytics environment that can be mined for value page 43
Use Cases by Category and Vendor Coverage 30
25
Use Cases
20
Vendor Coverage
15
10
5
0
“Big data” appears to be a target rich environment. page 44
Is Skewed Data (or Data Availability) Leading to Skewed Use Cases? Devices Fully Representive?
Data Fully Represents Infrastructure?
Meters
Yes
Yes
Most familiar to all vendors
Communication Nodes
Yes
No
Some value overlaps with meter, EE, DR
Transformers
Yes
No
Missing nameplate information
Capacitor Banks
No
No
Limited information from Pi Historian
Reclosers
No
No
Limited information from Pi Historian
Line Sensor
Yes
No
Missing event and operations data
Asset Nameplate Data
No
No
No asset attribute data included
Customer Program Data
N/A
Yes
Duke program data for all customers.
Socioeconomic Data
N/A
No
Missing data on many customers
Weather Data
No
No
No McAlpine weather station data used
Category
Comments
A majority of the data available comes from AMI systems. Is this skewing our focus towards use cases that utilize AMI data?
Analytics Cost Reduction Opportunities Capital Spend by Major Business Function (2012) Customer + Other T&D Electric
3%
6%
O&M Spend by Major Business Function (2012) 100% 6%
9%
Other
23%
15%
Customer
10%
16%
T&D Electric
62%
60%
Generation (Non-Fuel)
22%
47%
Generation (Non-Fuel)
75%
47%
Duke Energy Corporation
Industry Set
Duke Energy Corporation
Industry Set
Based on spending, generation appears to be the biggest “target-rich” environment. Source: FERC Form 1; SNL; Accenture Customer function is only associated with O&M spend; Other category represents smaller business functions (e.g. corporate center) Industry Set represents 69 integrated investor owned utilities
Duke Energy’s Data Modeling and Analytics Initiative (DMAI) • Duke Energy needed to form a comprehensive “big data / analytics” strategy. • In 2013, industry experts who participated in the initiative received access to a cross-sectional dataset (1 week of comprehensive data) to review & analyze • Sample use cases were provided to “prime the pump” • Participants were asked to provide: – New use case ideas – Insights into what other types of data should be collected – Data warehouse, models and structures as well as data ingestion and extraction methods – Evaluation of the relative value of each use case provided
Duke Energy’s DMAI is an Ongoing, Collaborative Process Duke Stakeholders
Vendor Analytics
New Possibilities New Products & Services
Customer Analytics
Cyber Security
Emerging Technology
Customer Prospecting
Grid Modernization
Load & Weather Forecasting
Information Technology
Outage Management Grid Management
Load Forecasting
Vegetation Management
Meteorology
Revenue Protection Asset Management
Metering Services Supply Chain
Improved Customer Sat.
AT&T
Enhanced Operations
Example Tools Used by Vendors OSS DB
Languages
Analytics Platforms
Key Technology Discussions • RDBMS - NoSQL • OSS - Proprietary • Flexible Disaggregation for Discovery • Scalability, Warehousing
Frameworks
Context for Value Estimates Duke Energy Profile (2012) Value Code
Value Estimate
1-$
Thousands
2 - $$
Millions
3 - $$$
Tens of Millions
4 - $$$
Hundreds of Millions
5 - $$$$$
Billions
• • • • •
Market Cap: $45B Revenues: $19.6B Total Assets: $112B Electric Customers: 7.2M Total Franchise Capacity – 48,000 MW conventional – 1,700 MW renewables
• Estimates were rolled up into a “total value potential” metric
Value Estimates by Category Value Score
Std Dev
# Vendors Responses
Security
3.6
0.49
5
Limited information
Revenue Protection
3.5
1.009
13
Includes DG, retail competition and theft
Distribution Grid Analysis
3.4
0.881
11
Diverse examples of value cited
Load Forecasting
3.4
1.115
12
Asset Management Analysis
3.3
1.106
12
High score even within supporting data
Customer Analysis
3.1
1.062
15
Some value overlaps with meter, EE, DR
Demand Response
3.1
0.799
14
Meter Analytics
2.9
0.957
15
Outage Analysis
2.9
0.954
12
Energy Efficiency
2.9
0.828
13
Distributed Generation
2.8
0.916
9
Communications
2.5
0.5
6
Category
Comments
Most familiar to all vendors
Varying opinions on value of EE
DMAI - Lessons Learned • No one (including Duke Energy) is strong in all 3 critical analytics areas: technical, statistical, and industry knowledge • If data can be federated, significant value can be created. Similarly, silo’d data has numerous issues that limit potential value • Culture and process / organizational change can be just as daunting as the technology itself • Unstructured data helps discover new value and eliminates potential bias inherent in pre-determined structures • On the other hand, unstructured data is not suitable for every use case, nor does it solve every problem
Tools Technical Review – Insights - 1 • Digital sensors and appliances, low-cost communications and storage, demand for information, and the Internet of Things (IoT), have created an IT environment where “Big Data” is a problem. • Duke Energy may not have a Big Data problem yet; current state is that we have a lot of “small data” problems. • Each use case category evaluated contains silos of systems and data (and the categories themselves are silos). The greatest value will come when all data can be easily utilized for analysis.
• The majority of Big Data analytics vendors do not know our business, but they are learning fast.
Tools Technical Review – Insights - 2 • The strength of the Relational Database (RDBMS) is its schema, but this is also its greatest weakness since the schema is inflexible and difficult to change. • When questions about the data become more complex and cross silos, then complex RDBMS schemas break down. • The strength of Hadoop, NoSQL, and MapReduce are their abilities to store data in its original form in simple structures and process it with low-cost parallel computers. • Hadoop usage can reduce IT costs and can scale, runs on parallelized commodity hardware, and OSS has no license costs.
• IT Tools for ingestion and storage are becoming more commoditized and open; the Analytics and BI applications tend to be specialized by vendor and are not yet standardized.
page 54
Next Steps • Continue to embrace field device interoperability and distributed applications • Consolidate use cases with other Duke Energy and EPRI groups – Standardize categories – Identify and consolidate similar use cases • Continue vendor dialogue and look for partnership opportunities
• Develop requirements for Sandbox 2.0 • Define Duke Energy’s Big Data strategy and architecture
Questions?
Thank You! You will receive a link to download a copy of the slides to the email you used to register. Jesse Berst Host & Moderator
SmartGridNews.com
[email protected]
Larsh Johnson CTO IC Smart Grid Software & Services
eMeter, A Siemens Business
Aaron DeYonker Vice President of Products
eMeter, A Siemens Business
[email protected]
[email protected]
Bill Bell
Raiford Smith
Technology Director, Analytics & Data Services
Director, Emerging Technology
CenterPoint Energy
[email protected]
[email protected]
Duke Energy