Analytics Driven IT Operations Management April 2016 Larry Smith Federal ITSM Segment Leader IBM Corp
[email protected] 301-803-3597
© 2016 IBM Corporation
Agenda • Big Data in IT Terms • Operations Challenges • Cognitive Operations Management • Summary
© 2016 IBM Corporation
Page 2
IT Big Data
© 2016 IBM Corporation
Page 3
IT Big Data Sources
© 2016 IBM Corporation
Page 4
Today’s IT Operations Challenges The number of IT challenges organizations face are increasing. The staff needed to resolve these issues is decreasing.
© 2016 IBM Corporation
Transportation
Finance
Airline canceled more than 700 flights and another 765 flights are delayed due to a software outage – Blamed ticketing partner while the real problem was on their end
Trading halted for half a day on the biggest US exchange for financial options following an outage caused by software problems
Broadband
Freight
Not surprisingly, many angry customers poured out their wrath via social networking after the largest video streaming company had a software outage for more than 20 hours
A leading freight company lost $120 million in revenue because IT was unaware that critical warning messages were associated with their key freight delivery application
Page 5
The ProblemIT Today’s
Operations Challenges
Why aren’t operations teams preventative today? Too much data to analyze manually Existing analytic techniques, such as standard thresholds, are not up to the task They cannot detect problems while they are emerging (before business impact) Set threshold too high, insufficient warning before total failure. Set threshold too low, too much noise, everything is ignored
If there is no ‘early detection’ before the outage, operations teams can only react while outage is already in effect and already losing money... © 2016 IBM Corporation
Page 6
Today’s IT Operations Challenges
Greater Visibility is Needed to Meet these Challenges What if these organizations had cognitive solutions that… 1. Continuously learned and understood their environments 2. Proactively alerted on emerging issues prior to service impacts 3. Provides quick-search for root cause analysis
© 2016 IBM Corporation
Page 7
The Era of Cognitive IT Operations
What is IT Operations Analytics (ITOA)? IT operations analytics solutions analyze terabytes of big data from your IT operations and turn it into relevant information and insights that you can act on immediately. These analytics solutions use cognitive computing capabilities to learn your IT systems behavior over time and provide early warnings of abnormal behavior.
© 2016 IBM Corporation
Page 8
Key IT Operations Data Types • Log files • Performance metrics • Events • Trouble tickets
IBM Offering for IT Operations Analytics Our Cognitive Capabilities
Continuously Learn Machine learning to establish and maintain thresholds across applications and resources
© 2016 IBM Corporation
Page 9
Anticipate and Adjust Detect emerging issues across services, proactively alert, and cognitively adjust to changes
Recommend for Action Provide expert advice for corrective actions and greater service assurance
Capabilities, Insights, and Benefits Continuously Learn
Capabilities: Learn behavior of applications and resources, Dynamically set and manage thresholds for monitoring data, Identify seasonal events Insights: True understanding of normal across the enterprise, Relationships across applications and resources, Patterns of seasonal activity Benefits: Significant savings in operational costs, Improved staff efficiency, Better service assurance
Anticipate and Adjust
Capabilities: Anomaly detection, Consolidation of anomalies, Correlation of related metrics, Forecasting Insights: Metrics shown to be historically anomalous around the same time, Potential for service degradations Benefits: Proactive notification of service degradations, Intelligent problem prioritization, Improved staff efficiency
Recommend Action
Capabilities: Natural Language Processing/Text Analytics, Direct connection to IBM Support knowledge base, Expert Advice Insights: Most problematic areas across the environment, Pattern recognition across unstructured data, Recommended actions for repair Benefits: Significant reduction in Mean Time to Repair, Cost savings from improved staff efficiency
© 2016 IBM Corporation
Page 10
© 2016 IBM Corporation
Page 11
Log Analytics
Log Collection Technology App Developer/ IT Ops Engineer
Application/system Push logs (Log File Agent, REST interface, Log stash)
Business Users
Application Components
Pull logs using remote monitoring (agent less option)
Log Analytics Server
Log Analytics - Search
12
Multi-variant Analytics Correlation of Multiple Metrics
Statistical models can discover mathematical relationships between metrics Internet Banking
Internet Banking A
Application
ESB
Java / WAS
AIX
RHEL
Oracle
Core Banking Application
Windows
z/OS
B
C
D
E
F
G
H
I
The extent this can be achieved depends on a number of factors, such as: range and type of data, availability of data, and stability of environment. Analytics falls back to a single metric if metrics are unrelated. © 2016 IBM Corporation
Page 14
Multi-variant Analytics
Example Scenario: Internet Banking Application Goal: Automatically learn normal mathematical relationships between metrics Web Response Time
Internet Banking
Anomaly Event
Business Impacted
WRT Bad
Application
Web Response Time
ESB
Java / WAS
AIX
RHEL
WRT Good
User Requests
Oracle
Core Banking Application
Time Early Warning
• Learns ‘Web Response Time’ has a normal causal
relationship with ‘User Requests’ - WRT gets slower as user load gets higher.
Windows
z/OS
• If this healthy historical relationship breaks down, say due to a memory leak, an anomaly is raised immediately • The problem is detected even while WRT service is “good”
Emerging problems can be detected even while service levels are good in absolute terms Page 15
© 2016 IBM Corporation
Visualization of “learned” behavior with Anomalies
© 2016 IBM Corporation
Page 16
IBM Operations Analytics – Predictive Insights Challenge: Reacting to performance thresholds is not enough. IT Staffs must become proactive to ensure mission critical applications never go down.
Performance Manager of Managers
IBM Watson Inside
Dynamically learns application and infrastructure behavior; manages and maintains thresholds for all performance KPIs
Anomaly Detection Alerting before potential issues become service impacting, enabling IT to shift from reactive to proactive
Forecasting Forecast anomalies and metrics to identify potential critical issues
Multivariate Analysis Discover related KPIs for deeper insight and faster mean time to repair
© 2016 IBM Corporation
Page 17
Available On-prem and as a Service
IBM Operations Analytics – Log Analysis Challenge: To diagnose service problems in applications, and the infrastructure supporting them, requires quickly analyzing incredible amounts of both structured and unstructured data Breadth of Searchable Data Search across all of your IT operational data to quickly resolve issues Expert Advice Any competitor can isolate problems. IBM helps clients quickly resolve them IBM Open Platform for Apache Hadoop IBM distribution of Apache Hadoop included for costefficient long-term data storage Insight Pack Ecosystem Library of “apps” that provide greater insight into domain specific data © 2016 IBM Corporation
Page 18
18
Log Analysis Service Desk Extension Challenge: In today’s dynamic environments, it is critical to meet Service Level Agreements. IT environments must respond to incidents, problems, and change requests rapidly to ensure client satisfaction.
Decision Support Leverage historical analysis to quickly identify the appropriate SME to assign tickets
Insights for CIOs Quickly identify problem hot spots and aid with strategic IT planning
Natural Language Processing/Text Analytics Powered by IBM text analytics to organize and categorize key information, improving MTTR
Supports Heterogeneous Environments Supports multiple service desk solutions, including IBM Control Desk, BMC Remedy, and ServiceNow
© 2016 IBM Corporation
Page 19
Analyst Feedback
In its initial IT Operations Analytics market share report, IDC named IBM one of three vendors that shaped the market in 2014. -- IDC, Worldwide IT Operations Analytics Software Market Shares, 2014: Special Report, Doc # US40619915
“IBM’s introduction of Trouble Ticket Analytics should be viewed as a game-changing opportunity for IT organizations seeking to bridge the ITSM/operations divide.”
“The most comprehensive operational analytics environment that we have seen to date is IBM’s Operational Analytics Portfolio.” © 2016 IBM Corporation
Page 20
IBM Operations Analytics Media Resources IBM Marketplace
http://www.ibm.com/marketplace/cloud/it-operations-analytics/us/en-us
Information Covering • Benefits • Featured Videos • Demo • Client Case Studies • Purchase Information • Webinars • Information Resources • Trial
IBM Operations Analytics Community Information Covering • Blogs • Documentation • How-Tos • Videos • Integrations (Insight and Medication Packs) • Forum http://developer.ibm.com/itoa/ © 2016 IBM Corporation
Page 21
Summary
© 2016 IBM Corporation
Page 22
Review: IBM Operations Analytics Offerings IBM Operations Analytics – Predictive Insights
IBM Operations Analytics – Log Analysis
Log Analysis Service Desk Extension
Dynamic Threshold Maintenance
Breadth of Searchable Data
Decision Support
Search across all of your IT operational data to quickly resolve issues
Leverage historical analysis to quickly identify the appropriate SME to assign tickets
Dynamically learns application and infrastructure behavior; manages thresholds dynamically
Anomaly Detection Alerting before potential issues become service impacting, enabling IT to shift from reactive to proactive
Forecasting Forecast anomalies and metrics to identify potential critical issues
Multivariate Analysis Discover related KPIs for deeper insight and faster mean time to repair
© 2016 IBM Corporation
Page 23
Expert Advice
Insights for CIOs
Any competitor can isolate problems. IBM helps clients quickly resolve them
Quickly identify problem hot spots and aid with strategic IT planning
Big Data Platform
Text Analytics
Built on top of the IBM Big Data Platform; industry-leading text analytics included
Insight Pack Ecosystem Library of “apps” that provide greater insight into domain specific data
Powered by IBM text analytics to organize and categorize key information, improving MTTR
Supports Heterogeneous Environments Supports multiple service desk solutions, including IBM Control Desk, BMC Remedy, and ServiceNow
IBM Differentiates with Breadth and Advanced Analytics Cognitive offering that can Continuously Learn, Anticipate and Adjust, and Recommend for Action Out of the Box analytics across all your IT Operational Data: Events, Logs, Documentation, Trouble Tickets Analytics that specialize in detecting emerging issues across your client’s IBM Middleware stack Analytics that specialize in isolating and resolving problems in your client’s IBM Middleware stack Analytics that learn and visualize relationships across your environment Natural Language Processing that detects patterns across unstructured data Configuration-less analytics that require no model building from IT staff Cognitive capabilities are powered by IBM Watson © 2016 IBM Corporation
Page 24
IBM Differentiates with Integration and Greater Out of the Box Value Single-vendor integration of ITOA with a broader IT Service Management portfolio (Application Performance Management, Netcool, Control Desk) Performance Manager of Managers that ingests, analyzes, and integrates data from all your performance management solutions Provider of an Apache Hadoop distribution with its ITOA solution Rich ecosystem of insight packs and plug-ins that are 100% IBM supported and offered at a fraction of competitor costs Simple pricing model (by Managed Server or Network Device) for all offerings that eliminates need to calculate and keep up with daily ingestion
© 2016 IBM Corporation
Page 25
Thanks!
© 2016 IBM Corporation
Page 26
Backup
© 2016 IBM Corporation
Page 27
Operations Analytics - Log Analysis Insight Pack Ecosystem Infrastructure
Middleware
Storage
-
-WAS -MQ -Integration Bus -More…
-IBM SAN Director -Tivoli Storage Manager -More…
Applications
Networks
Generic
-
-
-
-
Apache Hadoop Oracle Microsoft IIS More…
Oracle Siebel VMWare VSphere HP TeamSite More…
-
Cisco Nexus Cisco IOS Check Point Firewall More…
Over 50 Insight Packs Available http://developer.ibm.com/itoa/resources/ © 2016 IBM Corporation
Page 28
-
Syslog Windows OS Generic Annotator More…
Operations Analytics Log Analysis Breadth of Coverage Insight Packs for Middleware
Insight Packs for Infrastructure Apache Hadoop
IBM SAN Director
IBM MQ
Apache ActiveMQ F5 BIG-IP APM, GTM, LTM Microsoft Active Directory Microsoft Hyper-V Microsoft IIS Microsoft Exchange Server Microsoft SharePoint Microsoft SQL Server Oracle MySQL Oracle WebLogic Server Red Hat JBoss EAP SAP NetWeaver Application Server SAP HANA SAP Adaptive Service Enterprise
IBM DataPower IBM DB2 IBM Integration Bus IBM Pure Application System IBM Security Access Manager (WebSeal) IBM Security Policy Manager (TDP) IBM WAS
Insight Packs for Applications
IBM Spectrum Protect (TSM) Tivoli Storage Productivity Center IBM System Storage DS8000 IBM TS3500 NetApp ONTAP
Insight Packs for Networks Cisco Nexus
VMWare VSphere
Visit the ITOA Community for Insight Packs: http://developer.ibm.com/itoa/docs/loganalysis/insight-packs/available-packs/
© 2016 IBM Corporation
Insight Packs for Storage
Page 29
IBM AIX IBM HTTP Server HP LiveSite andTeamSite Oracle Database Oracle Siebel
Check Point Firewall Cisco ASA Network Cisco IOS Cisco UCS Manager
IBM Operations Analytics on Service Engage https://www.ibmsrviceengage.com/it-operations-analytics/learn Performance Management + Predictive Insights Learn how analytics adds value to IT Operations Explore capabilities with live demos Free trials of on-prem and SaaS Client success stories
© 2016 IBM Corporation
Page 30