Plant-Floor and Enterprise Network Convergence Jerry Lucas, Cisco Systems
[email protected] Making Factory Automation Networks Secure
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Agenda Trend – Plant-Floor and Enterprise Network Convergence Rockwell Automation & Cisco Systems Alliance Converged Plantwide Ethernet Architectures Convergence-Ready Network Solutions Advantages of EtherNet/IP Manufacturing Security Overview Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
2
What you will learn Trends in Plant-Floor and Enterprise Network Convergence Technology enablers and business drivers Cultural and organizational convergence
How the Rockwell Automation and Cisco Systems Alliance are helping customers with their technology, network and cultural convergence Products/Services Education Series Webcasts Reference Architectures
A Layered Network Security Approach Overview Security Trends Defense in-depth: A Layered Approach
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
3
Convergence with the Internet of Things (IoT) Addresses Priorities Proprietary Networks
Ruggedized Infrastructure
Ethernet/IP Optimized for Industrial Applications
IoT Architectures
Connected Manufacturing – evolution of Operational Technology
Enterprise IT
Manufacturing Operations
Why converge networks? – Time to market – Security: perimeter no longer viable – Simplicity and Flexibility: maintenance and management – Problem resolution – Voice, video, data collaboration – Control of plant performance – Remote talent – Standards convergence … open systems – Future proofing
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Producing Tangible Business Benefits for Manufacturing Companies Best-in-Class Manufacturers Top 20%
• 8 hours of downtime per year (99.91% Uptime) • 11% total cost of ownership reduction for industrial network • 90% Overall Equipment Effectiveness (OEE) • +25% operating margin vs. corporate plan
67% Converged Industrial Ethernet Adoption Rate
Middle 50%
Bottom 30%
• Downtime: 36 hours/year • OEE: 80% • Downtime: 135 hours/year • OEE: 60%
33% Converged Industrial Ethernet Adoption Rate Source: Aberdeen Group 2012 Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Plant-Floor and Enterprise Convergence Trend – Technology Convergence
OEM
Enterprise-wide Systems
More Enterprise Integration Corporate Headquarters
Supplier
More Applications (control disciplines) Plant-wide Systems Receiving
Control Room
More Assets (things) Connected
Customer
More Collaboration Utilities
Material Handling
Processing
Batching/ Blending
Other Plant
Shipping
Packaging
Lower Total Cost of Ownership | Faster Time to Market | Better Asset Optimization | Broader Risk Management Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
6
Technology Convergence Changing Automation Networks Corporate Network Office Applications, Internetworking, Data Servers, Storage
Back-office Mainframes and Servers
Back-office Mainframes and Servers
Office Applications, Internetworking, Data Servers, Storage
Corporate Network
Control Network Gateway Human Machine Interface
PC Based Controllers
Programmable Logic Controllers
Motors, Drives, Actuators
Robotics
Control Network Device Level Network Ethernet
Traditional
Robotics Sensors and Other Input/Output Devices
Human Machine Interface
PC Based Controllers
Sensors and Other Input/Output Devices
Ethernet-based
Motors, Drives, Actuators
Programmable Logic Controllers
Ethernet
Automation equipment vendors are implementing Ethernet-based protocols as an replacement of traditional fieldbus networks Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Converged Plantwide Ethernet (CPwE) Enterprise Network Levels 4 - 5
Demilitarized Zone (DMZ)
Separation between Control & Enterprise Networks
Manufacturing Zone Level 3
Interconnection between Cell Zones, Server Farms, and DMZ
Cell Zone Levels 0-2
Network Connection for PLCs, HMIs, I/Os, & Drives Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Plant-wide Network Convergence Trend – Technology Convergence
Successful Plant-wide Network Convergence Requires Collaboration Simplification Innovation
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
9
Plant-Floor and Enterprise Requirements Similarities and Differences
Enterprise (IT) Requirements
So, what are the similarities and differences?
Plant-Floor (Industrial) Requirements Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
10
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements
Network Technology Standard IEEE 802.3 Ethernet and proprietary (non-standard) versions Standard IETF Internet Protocol (IPv6) and proprietary (non-standard) alternatives Industrial application layer protocols e.g. CIP, Modbus TCP
Local Area Network (LAN); smaller frames for control traffic Network availability Switch-Level and Device-Level Topologies Ring Topology is predominant for both, Redundant Star for switch topologies is emerging Standard IEEE, IEC and vendor specific Layer 2 resiliency protocols
Enterprise Requirements
Network Technology Standard IEEE 802.3 Ethernet Standard IETF Internet Protocol (IPv4 and IPv6) Standard application layer protocols – e.g. SIP, SNMP, DNS, RTP, SSH
Wide Area Network (WAN) and LAN; larger packets and frames Network availability Switch-Level topologies Redundant Star Topology is predominant Standard IEEE, IETF, and vendor specific Layer 2 and Layer 3 resiliency protocols
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
11
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements
Switches
Managed and Unmanaged Layer 2 is predominant
Traffic types
Information, control, safety, motion, time synchronization, energy management
Performance
Low Latency, Low Jitter Data Prioritization – QoS – Layer 2 & 3
IP Addressing
Static
Security Emerging: open by default, must close by configuration and architecture Inconsistent industrial security policies
Enterprise Requirements
Switches Managed Layer 2 and Layer 3
Traffic types
Voice, Video, Data
Performance Low Latency, Low Jitter Data Prioritization – QoS – Layer 3
IP Addressing
Dynamic
Security Pervasive Strong policies
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
12
Plant-Floor and Enterprise Requirements Similarities and Differences Plant-Floor Requirements
Wireless Autonomous – point solutions Mobile equipment (emerging) and personnel (prevalent)
Computing
Industrial Hardened Panel Mount Computers and Monitors Desktop Notebook 19” Rack Server Din Rail Mount
Virtualization
Emerging, becoming prevalent
Environment Plant-floor Control Room
Enterprise Requirements
Wireless Centrally managed and autonomous Mobile personnel – BYOD Guest access
Computing Desktop, Notebook Tablets 19” Rack Server and Blade Server Unified Computing Systems (UCS)
Virtualization
Widespread
Environment Data Center Data Communication Closet
IDF - Intermediate Distribution Frame Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
13
Plant-Floor and Enterprise Requirements Policies - Similarities and Differences Plant-Floor Network
Enterprise Network
24/7 Operations, High OEE
Protecting Intellectual Property and Company Assets
Availability Integrity Confidentiality
Confidentiality Integrity Availability
Converged Network of Data, Control, Information, Safety and Motion
Converged Network of Data, Voice and Video
Access Control
Strict Physical Access Simple Network Device Access
Strict Network Authentication and Access Policies
Implications of a Device Failure
Production is Down ($$’s/hour … or Worse)
Work-around or Wait
Threat Protection
Isolate Threat but Keep Operating
Shut Down Access to Detected Threat
Scheduled During Downtime
Automatically Pushed During Uptime
Focus Precedence of Priorities Types of Data Traffic
Upgrades
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
14
Plant-Floor and Enterprise Requirements Switching - Similarities and Differences Industrial Ethernet Switches Industrial hardened Panel or DIN mount Managed or unmanaged
IT Switches Campus, Data Center 19” rack mount – e.g. 1RU Managed
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
15
Plant-Floor and Enterprise Requirements Network Topology - Similarities and Differences Switch and Device-level Topologies
Controllers, Drives, and Distributed I/O
Cell/Area Zone
Redundant Star
Cisco Catalyst 3750 StackWise Switch Stack
Flex Links
Ring
Cisco Catalyst 3750 StackWise Switch Stack
Resilient Ethernet Protocol (REP)
Star/Bus Linear
Cisco Catalyst 3750 StackWise Switch Stack
Cisco Catalyst 2955 HMI
HMI Controller
Controllers
HMI
Controllers
HMI
Controllers, Drives, and Distributed I/O
Cell/Area Zone Cell/Area Zone
Controllers, Drives, and Distributed I/O
Cell/Area Zone
Controllers, Drives, and Distributed I/O
Cell/Area Zone
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
16
Plant-Floor and Enterprise Requirements Cisco Validated Design - Similarities and Differences Cisco Validated Designs (CVD) consist of systems and solutions that are designed, tested, and documented to facilitate and improve customer deployments. These designs incorporate a wide range of technologies and products into a portfolio of solutions that have been developed to address the business needs of our customers. Cisco Validated Designs are organized by solution areas and will list one, two or all three primary types of documents: Design Guides System Assurance Guides Application Deployment Guides
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
17
Plant-Floor and Enterprise Requirements Network Management - Similarities and Differences Plant Engineering
Information Technology
Cisco Network Assistant
FactoryTalk View, Faceplates
SNMP and IP sweeps • Establish early dialogue with your IT counterparts
Command Line Interface
RSLogix, Add-on Profile
Cisco Prime Device Manager
http://ab.rockwellautomation.com/Networks-andCommunications/Stratix-5700-Ethernet-Switches Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
18
Best Practices for Network, Technology, Organizational and Cultural Convergence IT and Plant-Floor Engineering collaboration and sharing of best practices on:
Standardization of design and technology System architecture design Protocols and services Service and support models Industrial Security Policy
Consult reference architectures, reference models and industry standards: Network Segmentation Network services Domains of Trust
An open, two-way dialogue is critical!
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
19
Rockwell Automation and Cisco Alliance Technology, Network, Cultural and Organizational Convergence Common Technology View: Achieve flexibility, visibility and efficiency through a single system architecture, using open, industry standard networking technologies, such as EtherNet/IP Converged Plantwide Ethernet Architectures: Plant-Floor focused reference architectures, comprised of Rockwell Automation and Cisco expertise, provide a foundation to successfully deploy the latest technologies optimized for both automation and IT professionals. Joint Product and Solution Collaboration: Stratix 5000 and 8000 families of Industrial Ethernet managed switches combine the best of both Rockwell Automation and Cisco to address IT and Plant-Floor priorities People and Process Optimization: Services and education to facilitate Plant-Floor and IT convergence, successful architecture deployment and efficient operations, so that critical resources can focus on increasing innovation and productivity
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
20
The Value in Bringing the Information Together Laboratory Information Management Systems Production Scheduling
Performan ce
Alarms/Events
HMIs
Quality Systems
Control Systems Data Historians Computerized Maintenance Management Systems Other Database Systems
You need robust Infrastructure Solutions to deliver the information fast, reliably and securely! Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Industrial Network Security Trends Established Industrial Security Standards
International Society of Automation
ISA/IEC-62443 (Formerly ISA-99) Industrial Automation and Control Systems (IACS) Security Defense-in-Depth IDMZ Deployment
National Institute of Standards and Technology
NIST 800-82 Industrial Control System (ICS) Security Defense-in-Depth IDMZ Deployment
Department of Homeland Security / Idaho National Lab
DHS INL/EXT-06-11478 Control Systems Cyber Security: Defense-in-Depth Strategies Defense-in-Depth IDMZ Deployment
A secure application depends on multiple layers of protection. Industrial security must be implemented as a system. Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Defense-in-Depth Multiple Layers to Protect the Network and Defend the Edge
Physical Security Network Security Computer Hardening Application Security Device Hardening Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
Strategic Alliance
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
24
STI – Solution Technology Integrator
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
25
Cisco and Rockwell Automation Alliance Cultural Convergence
Education Series Webcasts
What every IT professional should know about Plant-Floor Networking
What every Plant-Floor Engineer should know about working with IT
Industrial Ethernet: Introduction to Resiliency Fundamentals of Secure Remote Access for plant-floor Applications and Data Securing Architectures and Applications for Network Convergence IT-Ready EtherNet/IP Solutions
Available Online
http://www.ab.com/networks/architectures.html
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
26
Reference Architectures Converged Plantwide Ethernet Architectures
Rockwell Automation and Cisco Systems Collaboration Content relevant to both IT Network Engineers and Plant-Floor Control System Engineers Built on Technology and Industry Standards Recommendations and Design Guidance Documented configuration settings Cisco Validated Design “Future-ready” Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
27
Industry Standards Future-Ready Your Design
Technology
IEEE 802.3 - standard Ethernet 1588 - Precision Time Protocol (PTP)
IETF - standard Internet Protocol (IP) IEC - International Electrotechnical Commission ODVA - Common Industrial Protocol (CIP)
Manufacturing Purdue Reference Model for Control Hierarchy ISA-95 - Enterprise-Control System Integration ISA-99 – Industrial Automation and Control Systems (IACS) Security NIST 800-82 – Industrial Control System Security
Built on Industry Standards Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
28
Plant-wide Network Architectures Logical Model – Structure and Hierarchy
Logical Model Converged Plantwide Ethernet (CPwE)
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
29
Campus Network Model Structure and Hierarchy
Offers hierarchal modular topology
Building blocks
Fault domain (e.g. Layer 2 loops), broadcast domain, domains of trust (security)
Easier to grow, understand and troubleshoot Creates small domains - clear demarcations and segmentation Multi-tier switch model
Core
Distribution
Aggregates Distribution Switches Backbone of Network DMZ Connectivity
Distribution
Aggregates Access Switches Provides Layer 3 Services
Access
Aggregates Industrial Automation and Control System (IACS) Devices Provides Layer 2 Services
Core
Access
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
30
Logical Framework Converged Plantwide Ethernet (CPwE) Architectures Layer 3 Distribution Switch Layer 2 Access Switch
Catalyst 3750 StackWise Switch Stack
Layer 3 Building Block Rockwell Automation Stratix 8000 Layer 2 Access Switch
Drive
Layer 2 I/O BuildingHMIBlock Controller
Media & Connectors Cell/Area Zone #1 Redundant Star Topology Flex Links Resiliency
Level 2 HMI
HMI
Layer 2 I/O Drive Building Block Level 1 Controller
Controller
Cell/Area Zone #2 Ring Topology Resilient Ethernet Protocol (REP)
Cell/Area Zones Levels 0–2
Controller Drive
HMI
Layer 2 Building Block I/O Level 0 Drive
Cell/Area Zone #3 Bus/Star Topology
• The Cell/Area zone is a Layer 2 network for a functional area of the plant-floor. Key network considerations include: Structure and hierarchy using smaller Layer 2 building blocks Logical segmentation for traffic management and policy enforcement (e.g. QoS, Security) to accommodate time-sensitive applications Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
31
Logical Framework Converged Plantwide Ethernet (CPwE) Architectures Plant-Floor and Enterprise network convergence Plant engineer and IT network engineer collaboration Plant-wide EtherNet/IP Architectures Hierarchical segmentation
Scalability Resiliency Traffic management Policy enforcement
ERP, Email, Wide Area Network (WAN)
Enterprise Zone Levels 4 and 5 Demilitarized Zone (DMZ)
Patch Management Remote Gateway Services Application Mirror AV Server
Gbps Link for Failover Detection
Firewall (Standby)
Cisco ASA 5500
Firewall (Active)
Industrial Zone Site Operations and Control Level 3
FactoryTalk Application Servers
View Historian AssetCentre, Transaction Manager
Catalyst 6500/4500
FactoryTalk Services Platform
Remote Access Server
Directory Security/Audit
Data Servers
Plant Firewall: Inter-zone traffic segmentation ACLs, IPS and IDS VPN Services Portal and Terminal Server proxy
Cisco Catalyst Switch Catalyst 3750 StackWise Switch Stack
Network Services
DNS, DHCP, syslog server Network and security mgmt
Cell/Area Zones Levels 0–2
Security policies
Defense-in-depth
Secure remote access
Rockwell Automation Stratix 8000 Layer 2 Access Switch
Drive
HMI
Controller HMI
Controller HMI
I/O
Controller Cell/Area Zone #1 Redundant Star Topology Flex Links Resiliency
Drive
Drive
I/O I/O
Cell/Area Zone #2 Ring Topology Resilient Ethernet Protocol (REP)
I/O
Cell/Area Zone #3 Bus/Star Topology
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
32
Networking Design Considerations EtherNet/IP Considerations
Recommendations and guidance to help reduce Latency and Jitter, to help increase data Availability, Integrity and Confidentiality, and to help design and deploy a Robust, Secure and Future-Ready EtherNet/IP network infrastructure Robust Physical Layer Segmentation Resiliency Protocols and Redundant Topologies Time Synchronization Prioritization - Quality of Service (QoS) Multicast Management Convergence-Ready Solutions Security - Defense-in-Depth Scalable Secure Remote Access Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
33
Convergence-Ready Network Solutions Plant-wide Networks Partner Solution(s) e.g. OEM
Industrial Plant-wide Systems
Use of an industrial Ethernet protocol, such as EtherNet/IP, that fully utilizes standard Ethernet and IP as the industrial network infrastructure. Common network infrastructure devices – asset utilization Future-ready - sustainability
IP addressing schema: Class - address range, subnet, default gateway (routability) Implementation conventions – static/dynamic, hardware/software configurable, NAT/DNS (who manages?)
Use of industrial managed switches Network services such as loop prevention Integration between the network infrastructure and the control system – configuration, management, diagnostics/troubleshooting
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
34
Convergence-Ready Network Solutions Plant-wide Networks Use of Network Services
Segmentation Virtual LANs (VLANs) Structured hierarchy using Layer 2 and Layer 3 switching Topology
Data prioritization - quality of service (QoS) Availability – loop prevention, resilient topologies and protocols Multicast management Security stance
Physical access, port security, access control lists, FactoryTalk Security Alignment with emerging industrial automation and control system (IACS) security standards such as ISA-99 and NIST 800-82
Time Synchronization Services
IEEE 1588 Precision Time Protocol (PTP)
Grand Master, Boundary Clock, Transparent Clock
CIP Sync applications CIP Motion applications
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
35
EtherNet/IP Advantage Summary Single Network Technology for: Discrete Control, Process Control, Batch Control, Configuration, Information/Diagnostics, Safety Control, Time Synchronization, Motion Control and Energy Management Non industrial network traffic – Voice, Video and Data
Established – 300+ Vendors, over 5,000,000 nodes
ODVA: Cisco Systems and Rockwell Automation are principal members
Standard – IEEE 802.3 Ethernet and IETF TCP/IP Protocol Suite
IT friendly Future-ready – Sustainable; Industry Standards
Optimized Asset Utilization Common network infrastructure assets Common troubleshooting tools (assets) and skills/training (human assets) for Enterprise (IT) and Plant-Floor (Industrial) networks Reduces asset management requirements thus supporting lean initiatives
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
36
Additional Material Cisco and Rockwell Automation Alliance
Websites
http://www.ab.com/networks/architectures.html
Design Guides
Converged plant-wide Ethernet (CPwE)
Application Guides
Fiber Optic Infrastructure Application Guide
Education Series
http://www.ab.com/networks/architectures.html
Whitepapers Top 10 Recommendations for plant-wide EtherNet/IP Deployments Securing Manufacturing Computer and Controller Assets Production Software within Manufacturing Reference Architectures Achieving Secure Remote Access to plant-floor Applications and Data
Copyright © 2012 Rockwell Automation, Inc. All rights reserved.
37
Plant-wide Benefits of EtherNet/IP Seminar Making Factory Automation Networks Secure
Follow ROKAutomation on Facebook & Twitter. Connect with us on LinkedIn. www.rockwellautomation.com Copyright © 2012 Rockwell Automation, Inc. All rights reserved.