Sponsored by Dell EMC

IDC Multi-Client Project

Executive Brief

SOFTWARE-DEFINED INFRASTRUCTURE 2016 Agile IT Infrastructures as the Basis for Digital Transformation

Introduction Enterprise competitiveness increasingly hinges on rapid availability of digital information and data. IT infrastructures evolved over time are reaching their limits when supporting departments with their IT needs. IT resources must be provided faster and more flexibly, and there is a need for new solution concepts. Software-defined infrastructure (SDI) is such a solution approach. It enables enterprises to enjoy IT efficiently and quickly as a service by separating hardware and software and focusing on extensive automation and orchestration. In this way the IT organization is able to supply the business with scalable agile resources for cloudand web-based services and native cloud workloads. At the same time, existing conventional backend systems and applications like databases and transaction and batch processing can continue to be used. Infrastructure investments are safeguarded. A software-defined infrastructure therefore becomes a key factor and core element of modern IT infrastructures. SDI is still new to IT organizations and datacenters, and this Executive Brief provides IT managers with an overview of the benefits and obstacles of SDI as perceived by enterprises. It supports IT organizations in assessing SDI for their own enterprises and supplies recommendations on how to address this new, but very pertinent subject. SDI provides significant benefits for datacenters. IDC predicts that by 2017, next-gen converged systems that are optimized for Flash and SDI will drive reductions of more than 30% in internal datacenter space and staff. (1) There is above-average expenditure in this segment. In 2015, $23 billion were spent worldwide on SDI software and hardware solutions, and market volume will grow to around €50 billion by 2019. The figures clearly indicate that this trend is picking up steam and thus deserves a spot on the agenda of every datacenter manager. This paper is based on primary research by IDC and takes various sources into account. It centers on a representative survey of 252 enterprises and organizations in Germany with more than 250 employees. It was conducted in March to better assess the current situation, motivations and challenges involved in the subject of software-defined infrastructures. The analysis was confined to enterprises that already seriously engaged in the subject of SDI. The statements and findings are not only relevant for German enterprises; many enterprises in other European countries are also focusing more strongly on the subject. The findings largely bear out the result of the IDC survey „Software Defined Infrastructure (SDI), Customer Survey Results 2015“ (IDC #259403), which explored what US enterprises have to say about SDI.

2

Defining Software-Defined Infrastructures Software-defined infrastructure (SDI) is an ongoing industry trend toward the homogenization of datacenter hardware resources and a shift in value away from customized hardware platforms towards software control, orchestration, and service delivery solutions. SDI solutions operate independently of any hardware-specific dependencies and are programmatically controlled. They can be based on vendor-specific and/or open source technology. Examples include: • Software-defined compute combining hypervisors such as vSphere or Hyper-V with software based virtual machine and/or cloud software such as OpenStack and/or standardized containers with automated, policy-based orchestration and control software. • Software-defined networking solutions such as VMware NSX, Cisco ACI/APIC, Juniper Contrail/Open Contrail, Nuage VSP, or Open Daylight SDN controller. • Software-defined storage solutions such as EMC Scale IO, ECS and Atmos, Ceph-based storage solutions from Red Hat, SUSE or Canonical or OpenStack Cinder and SWIFT. Software-defined infrastructure solutions include different product design points: • Addressable/Accessible via published APIs • Autonomous — able to operate independently • Self-contained, including underlying OS — i.e., a platform • Able to run independently on homogenous hardware — i.e., no custom hardware • Disaggregated control of underlying hardware • Extensible — programmatically extend features (1)

SDI Overview — Open, Elastic, Automation FIGURE 1

Platforms

Homogeneous Hardware

Elastic, Open

Value

Disconnect software from hardware

Speed, efficiency, agility

Automation

Beyond configuration & provisioning

Deliver Unified Units of IT

Policy Driven

Source: IDC Multi Client Study, Software-Defined Infrastructure in Germany 2016

3

Another important aspect is the relationship between cloud computing and SDI: SDI forges a link between internal (physical and virtualized) IT environments and external hosted or public cloud services.

Datacenter Transformation to SDI is Well Underway IDC believes that by 2019, almost half of the storage and server spending in Western Europe will be captured by innovation accelerators and Third Platform pillars. In particular, innovation accelerators accounted for 15% of the server and storage spending in Western Europe in 2015, excluding overlaps (i.e., multiple-counting) between each other. By 2019, that number will close in on 25%. IDC recommends that European IT decision makers should start preparing for the disruption. (2) IT infrastructure flexibilization and agility top the requirement specifications of many enterprises. Rigid IT resources yield unsatisfactory results for the digital transformation of enterprises and business units. Flexibilizing IT resources is a gradual process; many IT organizations have been driving the virtualization of servers, storage, and networks for some years. All 252 enterprises in Germany that took part in the survey have at least partially virtualized their server landscapes. However, the degree of virtualization varies dramatically. Around one-third of the enterprises have virtualized more than half their servers. This compares to 82% of enterprises in the US that have virtualized more than half of their servers. Most enterprises have also virtualized their storage environments and networks, so the basic IT requirements for SDI already exist. SDI now takes their virtualized IT landscapes to the next level: so after virtualization, software-defined compute, software-defined storage, software-defined networking, convergent systems, orchestration and automation via various systems and domains are the next logical steps in IT infrastructure development.

4

Key Challenges in the Datacenter FIGURE 2

Cloud Computing

Security and Compliance

Server and Storage Modernization

Data Migration

Operations Cost Savings 0%

10%

20%

30%

40%

50%

60%

Source: IDC Multi Client Study, Software Defined Infrastructure in Germany 2016

This represents a paradigmatic change for practically all enterprises: traditional hardware-focused infrastructures are equipped with components selected for the respective workload according to criteria such as efficiency, performance, failproof operation, costs and user convenience. SDI provides such resources that are no longer rigidly linked to a workload, but can be flexibly adapted to practically every business situation. Management tools therefore permit execution of resource- and data-sensitive application workloads, either in the enterprise‘s own datacenter or in the cloud. 50% of the IT managers interviewed in Germany cited cloud computing as the key challenge for datacenters in the coming 12 months. Software-defined infrastructure considerably simplifies the use of cloud computing for enterprises. SDI permits seamless automatic availability of IT resources; whether they come from a private, public or hybrid cloud is immaterial. On-premise resources can also be integrated via interfaces and standards. 18% of interviewees in Germany consider SDI to be a key enabler of cloud computing. They have realized that a modern IT infrastructure and IT architecture like SDI greatly facilitates the supply of IT resources via cloud services. This applies for another 40% of interviewees. SDI supports the cloud strategy of these enterprises. US enterprises place a much greater emphasis on SDI in the context of cloud computing. 70% see SDI as very important cloud enabler. This circumstance explains the different paces at which both cloud computing and SDI have become established and developed in the different regions. The following applies for all enterprises: transforming datacenters to SDI is a challenge. Introduction is highly complex. It requires stocktaking of the datacenter architecture and topology, and an analysis of all hardware and software components, not to mention putting all aspects of service delivery — all processes, roles, procedure, and responsibilities — on the test bench. Initially, building an SDI calls for a transparent picture of the current situation in the datacenter in order to proceed with changes to the architecture, processes, and solutions.

5

SDI Takes the Burden Off IT Organizations IT organizations benefit considerably from the automated processes SDI makes possible, which is one of SDI‘s strong points. Many manual tasks are eliminated by automatic, rule-based server provisioning, resulting in two major benefits. Time-consuming error-prone upgrades and patches by IT employees, inconsistent server configurations and the risk of poor application and workload quality, and high downtime due to server failures become a thing of the past. IT security is enhanced as well, since many manual potential error sources are avoided. US enterprises‘ Top 3 IT benefits are improved security management, higher IT staff productivity, and a lighter load for IT staff, leaving more time for additional tasks. So IT employees on both sides of the Atlantic agree on the benefits of SDI for IT.

SDI’s Top IT Goals FIGURE 2

Reduce The Complexity of Upgrades and Maintenance Improve IT Staff Productivity Better and Transparent Performance Utilization Improve Security Management

Better Allocate IT Staff 28% 29% 30% 31% 32% 33% 34% 35% 36% Source: IDC Multi Client Study, Software Defined Infrastructure in Germany 2016

SDI also simplifies storage provisioning. In hyper-convergent systems, storage is mapped directly at compute level and can be extended as required. Over-provisioning is therefore no longer necessary. Furthermore, IT staff can focus more on other subjects and plan their time better as unpredictable and often ad-hoc activities are eliminated. They have more time at their disposal for other tasks. This is a factor that should not be underestimated in achieving more efficiency and innovation.

6

Departments Demand Less Expensive and More Efficient Business Processes SDI effectively supports departments in achieving their targets more efficiently at a lower cost. Backed by an SDI, departments can duly access IT resources themselves. Time-consuming requesting and procurement processes are eliminated. Servers, storage, and platforms are instantly available. The required services can be selected from a catalog or app store. Costs are transparent and due to the high level of automation and flexibility of resource provisioning, less expensive than conventional procurement. Compliance and security are improved and departments can reduce their auditing expenses. In business terms, German enterprises are hoping for lower operative costs (35% of citations). This is realistic target. More efficient IT is cheaper per se and if business processes can be quickly and agilely mapped and adapted to the IT system, their cost is also reduced. Speed and agility are fast becoming a common competitiveness factor in many enterprises. And this is best achieved with a modern IT landscape and optimized workloads. So it is little wonder that US enterprises cite better scalability and greater agility of business processes most as a business advantage.

SDI is Used for a Wide Variety of Workloads and Application Areas All enterprises interviewed have already looked into the subject of SDI thoroughly. 25% of German enterprises have been using SDI for productive systems for more than a year. This shows how great the pressure is on IT and the need to support business processes with modern IT solutions and architectures. 40% of the German enterprises interviewed plan to deploy SDI for business-critical applications such as ERP, CRM and industry-specific applications in the next two years. These workloads demand particularly high standards of availability, failproof operation, and service quality. SDI is the basis for satisfying these demands and can be a way of reducing the cost of IT operations. As expected, US enterprises are ahead on deploying SDI for business-critical applications. The issues are similar to those in Germany, but citations are significantly higher.

Orchestrating IT Resources is at the Heart of SDI The orchestration of all SDI modules and services is a basic requirement for unlocking SDI’s full potential. For many enterprises, orchestration is an iterative process to master the complexity of SDI when restructuring or modifying the IT architecture and service delivery. Orchestration provides the technical basis for the modified architecture. Solutions on the market are developing rapidly. So for this reason it is essential that IT organizations record the status quo of their IT architecture in detail and harmonize planning for the further development of IT delivery with the roadmaps of the different solutions.

7

Open Source is part of most SDI implementations Open Source has long been a fixture in datacenters worldwide. Many solutions for software-defined compute, software-defined storage, software-defined networking, convergent systems and orchestration and automation via various systems and domains are Open Source or Open Source-based. As such, Open Source is almost always part of SDI architectures. The following applies for all (license-model) solutions: The various solution components for SDI are at different stages in their life cycle. While virtualization tools have been in use for many years, containers and OpenStack are still in their infancy. However, datacenters are willing to use these solutions because the pressure from departments is high. Enterprises are therefore caught between the poles of solution innovation and enterprise readiness. Containers enable an increase in application density by as much as 100 times higher than what virtualization delivers. They are more efficient and lightweight than virtual machines (VMs) in running multiple workloads and more easily transferrable from host to host and from cloud to cloud. European datacenter managers acknowledge these benefits of container technologies. They regard container technologies as enablers of digital transformation. Key digital enterprise objectives include making infrastructure cloud-ready (28%), modernizing test and development environments (27%), improving speed of deployment (26%), and supporting new application development (25%). (3)

Controlling Costs and Complexity It is all the more important with new issues like SDI to be aware of potential pitfalls and control risks. Investment in datacenter modernization needs to be planned with care, otherwise additional costs may arise that were not bargained for. 38% of interviewees stated that their SDI launch budgets were strapped. High levels of complexity came second on the list of obstacles (34% of citations). Introducing an SDI is indeed a complex task. Abstracting, virtualization, and orchestration based on additional layers and tiers, which have to be linked to existing solution components and workloads. However, the changed datacenter design reduces process complexity and maintenance and upgrade costs. And that is exactly the benefit of SDI.

8

Major Obstacles to SDI Introduction FIGURE 4

Limited Budget Too Complex Lacking Security and Compliance Lack of inhouse IT Skills/Staff LoB Concerns Lack of Inhouse Know How Solutions / Technology Immature Do not entire Know Vendor Landscape Insufficient Attention from IT Management 0%

5%

10%

15%

20%

25%

30%

35%

40%

Source: IDC Multi Client Study, Software Defined Infrastructure in Germany 2016

IDC Expects Above-Average Market Growth for SDI The SDI market consists of software and hardware from the software-defined compute, software-defined storage, software-defined networking and control software (orchestration and automation). In 2015, market volume worldwide totaled $23 billion. IDC expects a volume of more than $50 billion in 2019. Last year software accounted for 28% of the total volume. Software expenditure will rise and in 2019 should account for around 39%. Software-defined infrastructure is set to grow fastest in the infrastructure segment as a whole (CAGR 2014/2019: 23.8 %). (5)

Conclusion Software-defined infrastructure is a comprehensive concept for datacenter flexibilization and automation. Although still a new subject, many enterprises around the globe already deploy it to improve IT resource provisioning, which is a clear indication that existing IT infrastructures increasingly fall short of the speed and agility required to fully satisfy the departmental demands. IT departments are therefore called upon to fundamentally rethink and modernize their IT delivery models. Highly complex SDI influences the entire IT delivery setup, and should therefore be deemed a CIO issue.

9

IDC Recommendations Based on the survey results, IDC recommends the following to enterprises:

Start by Inventorizing Your IT Processes Isolated applications, a silo mentality in departments, shadow IT, and opaque IT procurement processes make IT operations in many organizations rigid and expensive. So start by thoroughly analyzing your entire IT setup. Also note that the decision to opt for an SDI is of fundamental importance for the IT organization and processes and is therefore a subject that must be addressed at CIO level.

Flexibilize Your IT Infrastructure In many enterprises, IT growth is uncontrolled and infrastructures rigid. As a result, business demands cannot be adequately addressed by central IT, and departments increasingly seek solutions elsewhere. Virtualization projects are a good and necessary step towards enhanced datacenter flexibility and cost efficiency. An SDI can considerably increase agility while improving need-based provisioning of IT resources and failproof operation.

Introduce Your SDI Step-by-Step Start with a thorough inventory. Solution silos abound not only in departments, but also in IT. SDI calls for a holistic appraisal. Discussion must focus on architecture aspects and the changes compared to a traditional datacenter topology. While many enterprises still maintain static datacenter environments, others have taken the first steps towards agility. Establishing an SDI always involves a mix of old and new worlds. Greenfield solutions are an absolute exception.

Avoid Vendor Lock-Ins SDI is based on openness and standards. SDI also holds a potential risk of dependency on providers, technologies, and architecture systems. There are various ways of avoiding vendor lock-ins. By adopting a multi-hypervisor strategy, you avoid the dominance of one product. Many enterprises also opt for Open Source, and all relevant providers have opened their solutions to standards via interfaces. You can choose between a set of different solution approaches to satisfy your flexibility, openness, and mission-critical requirements.

View Cloud Computing as the Main IT Delivery Model Cloud computing is the key challenge for many datacenters. Hybrid and multi clouds will win the day as delivery models. A software-defined infrastructure provides the basic architectural and IT framework for efficient cloud structures for IT, departments, and beyond the bounds of the enterprise. In doing so, you lay an important foundation stone in your enterprise‘s digital transformation.

10

User Recommendations for Users Interviewees were asked to give advice to other decision-makers when tackling the transformation of their datacenter according to software-defined infrastructure principles. Here are some of the answers, which are unedited to preserve their authenticity.

„Involve departments as soon as possible to better understand business requirements.“

„It is important to protect data because data security is going to be increasingly important in the coming years.“

„Security mechanisms shouldn‘t place more obstacles in the user‘s way to avoid them seeking alternatives.“

„Plan capacities with a generous but sure hand, open all interfaces, anticipate delays in implementation, reduce complexity.“

„We looked at the risks of SDI first, but found them to be non-critical.“

„Take on-board the right highly qualified providers for implementation.“

Sources: (1) IDC FutureScape: Worldwide Datacenter 2016 Predictions (#259808) (2) Software-Defined Infrastructure (SDI), Customer Survey Results 2015 (IDC #259403“) (3) Faster Than It looks: How Innovation Accelerators Impact Datacenters in Europe from Now to 2020 (IDC #EMEA41733416) (4) 2016 Container Technology Survey: European End-User Adoption Trends (EMEA41698316) (5) Worldwide Software-Defined Infrastructure, 2014–2019: Forecast Report (#US40903016)

11

DELL EMC

Customer Case: The Vatican Library Information about the customer In 2014, the Library launched an ambitious project to digitize 82,000 ancient manuscripts. This innovative initiative quickly generated eight WWW.DELLTECHNOLOGIES.COM

petabytes of data, which is expected to climb to 43 petabytes upon completion. Recognizing that the project‘s requirements would outpace traditional storage infrastructures, the Vatican Library worked with EMC to build a data lake based on EMC software-defined storage solutions. The Vatican Library now has the performance and efficiency to quickly pour digitized content into its data lake and enable advanced queries by researchers around the world. The library can also test new functionality, such as handwriting pattern recognition, and gains the speed and scalability to handle up to 40 million pages of digitized manuscripts.

Customer Requirements/Challenges • Digitize and manage 82,000 ancient manuscripts • Manage growth from 8 to 43 petabytes of digitized content

Description of the Solution/Approach At foundation of its data lake, the Vatican Library deployed 55 nodes of EMC Isilon scale-out storage as part of an EMC XtremIO all-flash storage infrastructure. Digitized content resides on Isilon for six months during its most active use period and is then moved to EMC Elastic Cloud Storage (ECS) for long-term storage and preservation. The library uses Hadoop Distributed File System (HDFS) to logically catalogue assets in the data lake, which enables online visitors to perform intelligent queries. In addition, the Vatican Library plans to deploy EMC ScaleIO software-defined storage to create a high-performance, scalable environment for testing new features of its digital catalogue. EMC ViPR Controller software automatically moves digitized content between the library‘s two datacenters. The library also replicates Isilon and XtremIO across the two datacenters.

12

With built-in support for Hadoop, ECS enables the library to store and manage its digitized catalogue regardless of the format in which content was originally created. Such flexibility is important because scanning and viewing technologies will likely change during the many years of the project. This future-proofs the library‘s data lake investments.

Project Highlights Doubled storage capacity with higher performance and lower cost Enabled universal access to digitized assets regardless of original format Delivered speed and scalability for pattern recognition of 40 million pages Supported advanced queries that open new research opportunities

Customer Quotes About the Project „ECS can accept new content into the data lake very quickly, which is critical because our project is advancing at a rapid pace. ECS also provides twice the storage capacity in a rack compared to traditional storage. This keeps cost and energy consumption low while still providing the high-performance and scalability we need. „ECS introduces a whole new philosophy that allows us to save anything in the data lake and use Hadoop to manage structured and unstructured content in a sensible way. ECS is a revolution for us because people around the world are able to query our catalogue and find information they might not expect to be there. It opens many new possibilities for researchers of ancient artifacts.“

„When you convert a manuscript to an image, there is no index to the words. So we run a pattern recognition program that looks at every pixel to analyze the handwriting and matches patterns to words that can then be queried. ScaleIO completes the pattern recognition analysis extremely fast. „This is important when you consider a manuscript may be 500 pages long and we have 82,000 manuscripts. In total, we will be analyzing more than 40 million pages. That makes ScaleIO a critical part of our project‘s success.“

“ECS is a revolution for us because people around the world are able to query our catalogue and find information they might not expect to be there. It opens many new possibilities for researchers of ancient artifacts.”

Luciano Ammenti, CIO of the Vatican Library

13

INTERVIEW WITH NIGEL MOULTON IDC spoke to Nigel Moulton, EMC Dell Chief Technology Officer EMEA, Converged Platforms & Solutions Division about the study „Software-Defined Infrastructure in Germany 2016“. IDC: Many IT departments are finding it

IDC: What approach do you

increasingly difficult to provide flexible IT

recommend to companies

resources quickly and inexpensively. Why is

that want greater flexibility

it important in your view that companies deal

and optimize their IT delivery?

with software-defined infrastructure?

Nigel Moulton: At Dell EMC

Nigel Moulton: Software-defined infrastructure

we recommend an approach

(SDI) provides the fastest, most economical way

that creates an optimum

of providing flexible IT resources that can be

environment for both the

consumed by many different departments in a

IT department and the user

company. SDI provides an environment where

of the system. Deploying

the user can self-select the compute/storage/

SDI (on either converged or hyper-converged

network environment that they want, and can

infrastructure) gives the IT department the optimum

have this automatically provisioned in a matter of

infrastructure on which to deploy SDI, while

minutes. The comparative cost can be shown to

learning the new skills required to support these

the user, so they can make an informed decision.

environments. A converged infrastructure system

This also allows the IT department to charge-

can be used to support and migrate more traditional

back different services that they offer, and hence

platform 2 applications, while hyper-converged

demonstrate accountability back to the business.

infrastructure is more likely to be used for some

IDC: What are the typical challenges that companies are facing when implementing a software-defined infrastructure?

NIGEL MOULTON

traditional platform 2 applications, but more likely platform 3 cloud native applications. Combining both infrastructure types with an SDI approach will yield the most cost-effective approach for

Nigel Moulton: Many IT departments are organized

the business while simultaneously providing the

in different silos: compute, network, storage, and

highest levels of agility for the user community.

virtualization, and work in isolation both from each other and also the application development teams. The processes that have evolved over time to support this have become very inefficient, and no longer support the overriding requirement of the business which is agility. When implementing SDI, automation replaces many of these processes and may require fewer IT personnel to implement the overall solution. This can present both an opportunity and a challenge as people within the IT department will need to acquire new skills to be able to implement these highly automated platforms.

14

Copyright Notice Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. For further details regarding this publication please contact: Katja Schmalen, Marketing Director, +49 69 90502-115 or [email protected]. Author: Matthias Zacher, Senior Consultant & Project Leader, IDC Copyright: IDC, 2016. Reproduction of this document is strictly prohibited without prior written consent.

IDC Central Europe GmbH Hanauer Landstr. 182 D 60314 Frankfurt • Germany T: +49 69 90502-0 F: +49 69 90502-100 E: [email protected] 16