SCADA Master Planning 101 SCADA, in general, is the interface between the plant control system and the operators. It is the primary tool with which operators and managers try to coordinate and control the processes of their facilities. While SCADA was first introduced for large scale power distribution systems over 50 years ago, the first wave of water and wastewater SCADA systems were not implemented until the early 90s. As these original systems age, municipalities are finding themselves with components and software that is outdated, unsupported, and cumbersome to maintain. Enter the SCADA Master Plan! The term ‘SCADA Master Plan’ is a misnomer, as much more than just the SCADA system is being master planned. These projects typically refer to a re-evaluation and sometimes overhaul of the entire process control system including: • • • • • • • • • •

Programmable Logic Controller (PLC) Hardware PLC Software PLC Cabinets SCADA Software SCADA Servers Ethernet Switches Control Philosophies Reporting Alarming and Alarm Management Energy Management

Remote Communications

SCADA Design

Master Planning and Control System Design

Carollo has performed more than 50 SCADA Master Plans throughout the country and has developed a proven approach to conducting and implementing these challenging projects. The table below showcases Carollo’s experience in conducting SCADA Master Plans throughout the country.

Project Elements Control System Programming

Carollo Approach

Client Name/Project Name City of Yuma, AZ - East Mesa WTP City of Casa Grande, AZ - WRF Phase 3 Expansion SCADA St. George, UT - Quail Creek WTP Instrumentation Salt Lake City Public Utilities, UT - WRF Process Control Upgrade South Valley WRF, UT - Projects 4C and 4D SCADA Metropolitan Water District of Salt Lake & Sandy, UT - Point of the Mountain WTP SCADA

• • • • • • • • • • • • • • • • •

Weber Basin WCD, UT - Weber South WTP SCADA Weber Basin WCD, UT - Davis North WTP SCADA Magna Water Improvement District, UT - Barton Wellfield WTP Central Utah WCD, UT - Ashley Valley WTP Process Control System Upgrade Provo River Water Users Association, UT - SCADA System Contra Costa Water District, CA - SCADA WTP System Contra Costa Water District, CA - Randall-Bold WTP DCS Upgrade Contra Costa Water District, CA - Bollman WTP DCS Design-Build Project Contra Costa Water District, CA - Randall-Bold MPP Pump Station PLC/HMI Programming Project Cities of Mesa and Phoenix, AZ - Val Vista WTP Upgrade DCS City of Roseville, CA - Barton Road WTP Communications Upgrade Sacramento Regional County Sanitation District, CA - WWTP Boiler System Control City of Santa Maria, CA - Water Telemetry/SCADA System DCS City of Santa Rosa, CA - Laguna Wastewater Plant DCS Upgrade South Tahoe Public Utility District, CA - Effluent Pumping SCADA Upgrade Project City of Watsonville, CA - Recycle Water Facility Central Marin Sanitation Agency, CA - Wet Weather Improvement Project City of Arlington, TX – John F. Kubala WTP Expansion SCADA Programming City of Reedley, CA – WWTP Expansion SCADA Programming Collier County, FL – NCRWTP Control & Information System Replacement City of Roseville, CA – Pleasant Grove WWTP UV Conversion City of Roseville, CA – Dry Creek WWTP UV Conversion City of Merced, CA – 10-mgd Expansion SCADA System City of Fresno, CA – WWTP Expansion DCS

• • • • •

• • • • • • • • •

• • • • • • • • • • • •

• • • • • • • • • • • • • • • • • • • • • • • •

• • • • • • • • • •

Carollo’s approach is workshop intensive and focuses on the participation and buy-in of the client and the critical stakeholders, from operations staff to management. The workplan shown below is an example of the critical steps required for a typical SCADA master plan.

75th Street Wastewater Treatment Plant PROCESS AUTOMATION SYSTEM • STRATEGIC PLAN

TASK 0

MEETING/WORKSHOPS

DELIVERABLES

TASKS/WORK EFFORTS

Develop Detailed Scope

START/END DATES

Tech Memo No. 1 Scoping Summary

Develop Project Management Plan

TASKS 1-4 Investigation and Analysis of Existing Infrastructure

TASK 5 Investigation of Current and Emerging Technologies

TASK 7 Develop Alternatives and Cost Estimates

TASK 6 E,I&C Design Standards

TASK 10 Compile Tech Memos Into PAS Reports

• Scope, Schedule, And Budget • Project Team and Contact Information • Management Procedures (Communication, Reporting) • Quality Management Procedures • Project Risk Management & Response Plan • Decision Log

• Evaluate Existing Documentation • Site Tour and Control Cabinet Audit • Corrosion Assessment Forms • Verify Architecture and Risk Diagrams • Determine Upgrade Paths and Packages

• • • • •

Identify Needs and Desires Review Viable Technologies Describe Pros and Cons Identify Costs Coordination with Manufacturers

• Develop Criteria for Future PAS Upgrades • Prioritization of Alternatives to Balance Criticality, Risk, and ROI • Develop Realiable Cost Estimates Through Close Coordination with Vendors • Look at Implementation into CIP and Provide Justification

• Compile Surveys • Develop Hardware Components Standards • Develop Software Components Standards • Power Standards • Screen Standards • Control Panels Standards • Tagging Requirements • Control Philosophies • Networking Standards

• Compile Tech , Memos • Develop Executive Summary

Project Management Plan

Tech Memo No. 2 Existing Hardware & Software

Tech Memo No. 3 Current & Emerging Technologies

Tech Memo No. 4 Recommendations

E&IC Design Standards Documents

Draft PAS Report

Final PAS Report Scoping Workshop

Kick-off Meeting

Tech Memo No. 2 Review Workshop

9/17/12 • 10/3/12

9/18/12 • 10/10/12

10/11/12 • 11/30/12

Tech Memo No. 3 Review Workshop

10/12/12 • 1/11/13

Tech Memo No. 4 Review Workshop

E&IC Design Standards Workshop

1/7/13 • 2/22/13

2/25/13 • 3/29/13

4/1/13 • 5/3/13

Key aspects of nearly every SCADA Master Plan include: • Staff Interviews – Determine the current perception of the automation system

Process Automation System (PAS) User Survey Name: Department: Position:

Introduction: This survey has been prepared for management and staff to examine the utilization, strengths, and  shortcomings of the Boulder WWTP Process Automation System (PAS). The PAS system includes the SCADA  and PLCs that gathers and transmits, in real‐time, data that is used by operations and maintenance staff to  control the WWTP.  The PAS allows real‐time data to be summarized and stored for later retrieval. The data  can be used for analysis, planning, internal reports, and reports to outside agencies. The results of this  survey will be used to help identify and prioritize potential upgrades to the existing PAS system.

1 2 3 4 5

not at all not really no opinion somewhat yes

The questions relate to your  present and possible future utlization of the PAS. For each question below,  mark the box to the right that best fits your answer. Use the scale above to match your selection. 

QUESTIONS General 1 2 3 4 5 6

Do you currently interact with the PAS real‐time data? Do you currently use summarized or historical PAS data? Do you currently interact with the reporting  function of the PAS? Do you currently interact with the alarming of the PAS? Does the PAS system operate in timely fashion? Is the PAS system reliable?

Historical Data and Trending 7 8 9 10 11 12

Do you have access to all the historical data you need? Do you have access to all the real time trending data you need? Can you retrieve historical data easily? Is the retrieved historical data in a format that can be readily used? Can you retrieve trending data easily? Is the trending data in a format that can be readily used?

Reporting 13 Do you have access to all the reports you need? 14 Are reports easy to generate?

Process Automation 15 Do you feel that the WWTP process is adequately automated? Do you feel that there are plant processes that should be automated that are not  16 currently automated? 17 Do you feel that there are plant processes that require improved automation?

Operation and Maintenance 18 19 20 21

Is O&M data easily accessible? Is O&M data organized in a way that makes it easy to find what you need? Do you use the Online Operations Manual (OLM)? Is the OLM easy to access?

SCALE 1

2

3

4

5 NA

• Control Cabinet and Software Audits – Determine the current state of the automation system infrastructure The workplan shown below is an example of the critical steps required for a typical master plan.

PCM Panel Location: DAFT BLDG PCM Panel Tag: ACC‐DAF AREA CONTROL CENTER Process(es) Served: DAFT  Panel Dimensions: 3'X9'X6' Picture ‐ Panel External

General Comments:

Inspection Date: 1/7/13 Install Date: JAN. 2008 Condition*: 1 Inspection By: MKR

Picture ‐ Panel Internal

Good Condition ‐ no visible issues

*Condition 1 ‐ Very Good 2 ‐ Minor Defects 3 ‐ Needs Significant Maintenance 4 ‐ Requires Rehabilitation

Controller Model: RACK 0

Slot 4

Slot 5

IC694 4MDL930

Slot 3

SPACE CE

LRE 0 001

Slot 2

IC694 4ALG442

Slot 1

IC694 4ALG442

Slot 11

POWE WER SUPPLY  IC694 4PWR330

Slot 10

CENET MASTER DEVIC

Slot 9

CENET MASTER DEVIC

SPACE

Slot 8

SPACE CE

IC694MDL230

Slot 7

SPACE CE

Slot 9

Slot 6

SPACE CE

Slot 8

Slot 5

SPACE CE

Slot 7

SPACE CE

CPU 3 320

Slot 6

ETM 0 001

5 ‐ Requires Replacement (>50%)

RACK 2 Slot 4

IC694MDL230

Slot 3

POWE WER SUPPLY

Slot 2

IC694MDL230

Slot 1

GE RX3i Pac Systems

RACK 2 RACK 1 Component Models: UPS: APC ‐ Smart UPS (UPS‐DAF) DC Power Supply: PHX. CONT. QUINT POWER

SPD: Relays:

Slot 5

Slot 6

Slot 7

Slot 8

Slot 9

Slot 10

Slot 11

IC694MDL930

IC694MDL930

SPACE

SPACE

SPACE

SPACE

Slot 4

IC694MDL230

Slot 3

IC694MDL230

Slot 2

IC694MDL230

Slot 1

IC694ALG442

SPACE

SPACE

Slot 10 Slot 11

EXP. POWER SUPPLY  IC694PWR330

Controller Layout ‐ Include IO type, card model #, and spare capacity where possible

CUTLER HAMMER AEGIS 

(4) 24VDC 5A Quint‐PS‐100‐24DAC/24DC/S

Terminal Blocks: Relays: idec Switches: Hirschmann Mice‐MS20, MM3‐2FXM4/2TV1 HMI/OIT: Other: APC Redundant Switch (STS‐DAF‐1) Other: Accopian Power Monitor (PS‐DAF‐1) Other:

HMI/OIT: Optiplex 745/ Simatic 15" Flat Panel NEMA: 12 HVAC: Backpanel: Enclosure:  Circuit Breaker: SIEMENS SSY41 Other: T‐STAT A‐TEMNO (HOFFMAN)

• Develop and Prioritize Alternatives – Balance criticality, risk, and return on investment • I&C Design Standards – Develop a living document defining the automation system for staff and consultants • Cut-Over Plan – Develop a step-by-step plan for cut over to the new automation system including a process coordinated schedule, planned outages, testing, and start-up procedures.

Cabinet Tag

Component RX3i CPU RX3i Power Supply RX3i Ethernet Module RX3i DeviceNet Master RX3i Expansion Module RX3i Expansion Power  Supply RX3i Mixed Analog  Module RX3i DI Module RX3i Relay Module UPS DC Power Supply

ACC‐DAF AREA  CONTROL CENTER

Ethernet Switch

OIT  Thin Client SPD Fiber Patch Panel Circuit Breaker RX3i Back Plane Redundant Switch Thermostat Power Monitor

*Overall Panel Condition:  1. Very Good  2. Minor Defects  3. Needs Significant  Maintenance  4. Requires Rehabilitation  5. Requires Replacement

Model IC695CPU310 IC694PSA140 IC695ETM001 IC694DNM200 IC695LRE001

Overall Panel  Condition (1‐5)*

Active

IC694PWR330 IC694ALG442 IC694MDL230 IC694MDL930 APC smart UPS Phoenix Contact Quint‐PS‐ 100‐24VDC/(2) 5A, (2) 10A Hirschmann Mice‐MS20,  MM3‐2FXM4/2TV1,MM2‐ 2TX1 Siemens Simatic Flat Panel  1P 6AV7861‐2TB00‐1AA0 Dell Optiplex 745 Cutler Hammer Aegis  AGSHWCH12ON20XC Corning Lan Scape  Siemens SSY41 IC695CHS012BA APC Redundant Switch (STS‐ DAF‐1) A‐TEMNO (HOFFMAN) Accopian Power Monitor (PS‐ DAF‐1)

Component Life Cycle Active Active Active Active Active

Active Active Active Mature Mature 1

Remaining Service Life  (Years) Approx 10 years Approx 10 years Approx 10 years Approx 10 years Approx 10 years

Meets City  Standard (Y/N) Y Y Y Y Y

Approx 10 years

Y

Approx 10 years Approx 10 years Approx 10 years Approx 5 years

Y Y Y N

Approx 5 years

N

Approx 10 years

Y

Approx 10 years Approx 10 years

Y Y

Approx 10 years Approx 10 years Approx 10 years Approx 10 years

Y Y Y Y

Approx 10 years Approx 10 years

Y Y

Approx 10 years

Y

Active

Active Active Active Active Active Active Active Active Active

Why Carollo Carollo in teaming with Carollo Systems has the unique ability to perform both the SCADA master planning as well as implementation. This arrangement provides significant advantages to the Client by insuring coordination of the integrator into the design process, saving significant cost during design and implementation. Our experience has led to the development and refinement of several tools that have a proven track record for optimizing the decision making process through the planning process. These tools can be utilized and customized for individual SCADA master plans improving efficiency of the master planning process. Carollo is uniquely positioned to provide SCADA master planning to its clients because of their inherent knowledge of the facilities process and operation. Many SCADA master plans are conducted by an integrator who does not necessarily understand I&C design or the facility unique and interrelated processes. Carollo offers its clients the unique advantage of providing a multi-faceted, coordinated team that can provide a holistic approach to SCADA master planning by providing: • Process optimization by leading process engineers in the water and wastewater industry • Instrumentation and control design by leading I&C engineers specifically dedicated to the water and wastewater industry • Programming and implementation by leading integrators dedicated to the water and wastewater industry

The Future of SCADA SCADA and process automation is fast changing and continuously adjusting to the current technological advancements. There are several exciting developments in process automation including: • Tablets – Allows the operator to get away from the control room and into the facility for critical operations • Artificial Intelligence – Allow predictive control by allowing the automation system to analyze the plethora of control system variables and learn the most efficient method for control • Energy Management System (EMS) – Monitor the facilities energy consumption and carbon footprint down to each process. • Game Theory – Allow operators to compete to optimize process control by providing the interface for competition and incentives for performance

For more information concerning SCADA master planning please contact: Monte Richard [email protected] Carollo Systems, LLC is a SCADA systems programming and integration firm, 303.980.8260 (Ph) • Littleton, created to provide outstanding control systems and services for water and wastewater utilities. Through a flexible and innovative contracting approach,

CO