Exadata X5: Working Smart with Oracle Exadata Database Machine Gurmeet Goindi Exadata Product Management Oracle
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Exadata Database Machine
• Scale-out, database optimized compute, networking, and storage hardware for fastest performance and lowest costs • Unique software and protocols enable fastest and most efficient OLTP, Analytics, and Consolidation • Delivered integrated, optimized, automated, and supported end-to-end to reduce operations costs
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
2
Thousands of Mission Critical Deployments Half are OLTP, Half Analytics • Petabyte Warehouses • Business Applications • SAP, Oracle, Siebel, PSFT, …
• Online Financial Trading • E-Commerce Sites • Massive DB Consolidation • Public SaaS Clouds • Oracle Fusion Apps, NetSuite, Salesforce, …
4 out of the 5 Largest Banks, Telecoms, Retailers Run Exadata Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
3
Exadata X5 is Sixth Generation DB Machine
DATA WAREHOUSING OLTP DATABASE AS A SERVICE DATABASE IN-MEMORY EXTREME FLASH Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
4
Exadata Extreme Performance Creates I.T. Economies of Scale
• Exadata is famous for running workloads faster FINANCE
HR
SALES
SUPPORT
CALL CENTER
SUPPLY CHAIN
• Exadata extreme performance also enables businesses to spend less by doing more: – Run more databases and users – Run more workloads and data – Run more analytics and transactions • Or run same workloads using less hardware to spend less
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
5
Exadata Hardware Spend Less by Doing More Database-Optimized Scale-Out Architecture • Fastest Compute • Fastest Networking • Fastest Storage • Fully Redundant
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
6
Exadata X5-2 Hardware Overview • Scale-Out Database Servers – Fastest (18 core) x86 compute processors (+50% cores*) – Largest memory capacity (+50% capacity*)
• Highest Capacity Internal Fabric (40 Gb/s InfiniBand) • Scale-Out Intelligent Storage Servers – Database offload to x86 processors accelerates analytics (+33% cores*) – Optimize for capacity with great performance using SMART TIERING of disk and flash (+50% flash*) – Optimize for I/O with NEW ALL-FLASH STORAGE * Compared to Exadata X4 generation Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
7
X5-2 High Capacity (HC) Storage Server 2X More Flash | Higher Performance Ultra-Fast PCIe Flash Cards
• 48 TB Disk | 6.4 TB PCIe Flash per Server – – – – –
State-of-the-art NVMe based PCIe flash Optimized Exadata InfiniBand I/O protocols Exadata storage offload – 16 cores/server 672 TB disk per Full Rack* 89.6 TB flash per Full Rack*
• Smart Flash Cache gives speed of flash with capacity of disk
Full Rack Rate*
Delta vs X4
140 GB/s
40%
OLTP Reads (8K)
4.14 M IOPS
55%
OLTP Writes (8K)
2.69 M IOPS
37%
Analytic Scans
Flash OLTP Latency 0.25 ms @ 1M IOPS
25%
* Full Rack : 8 DB servers, 14 HC storage servers Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
8
New X5-2 Extreme Flash (EF) Storage Server • Industry’s Highest I/O rates for OLTP and Analytics dramatically reduce cost per I/O – – – –
12.8 TB Ultra-Fast PCIe Flash per Server
State-of-the-art NVMe based PCIe flash Consistent, extremely low response times Exadata storage offload – 16 cores/server 179.2 TB flash per Full Rack*
• Ideal for I/O-intensive workloads – Deploy less servers to host huge I/O workloads
• Replaces High-Perf disk Storage Server
Full Rack Rate* Analytic Scans Random I/O (8K OLTP)
263 GB/s 4.14 M IOPS
Random 8K I/O Latency 0.25 ms @ 2M IOPS * Full Rack : 8 DB servers, 14 EF storage servers Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
9
Exadata is Much Faster than Fastest All-Flash Storage
4 3 2 1 0
0.6 Write
4.144
7X
4.144
3X
1.5
300
6X-Brick EMC XtremIO (Largest Config)
262
250 200
150
15X
100 50
Read
Analytic Scans
GB/sec
– 2X better I/O latency – Scales higher with more racks – Includes compute and networking tuned for flash speed – Many unique software features
5
IOPS (Millions)
• Exadata also has:
OLTP Reads/Writes Millions
• One Exadata Rack is 3X to 15X Faster than Fastest EMC All-Flash Array
Write
Read
1 Rack Exadata
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
18
0 6X-Brick EMC 1 Rack XtremIO Exadata (Largest Config)
10
X5-2 Database Server Faster Processors | More Cores | More Memory
• 18 Core Xeon Haswell EP
2-Socket, 36-Core X5 Server
– Fastest x86 processor ever shipped – 50% faster than IvyBridge (X4-2) processor
• Faster, larger memory (DDR4) – 256 GB base memory (8 x 32 GB DIMMs) – Expandable to 768 GB
Cores
X5 Capacity
X4 Capacity
Delta vs X4
36
24
50%
512 GB
50%
Memory 768 GB
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
11
Elastic Configurations Incrementally Scale Compute & Storage Achieve any Level of Performance with Minimum Hardware
Incrementally add DB or Storage Servers*
Start with
Database Server
Add Racks to Continue Scaling
Extreme Flash Storage High-Capacity Storage
Full Rack
Multi-Rack
2 Database Servers 3 Storage Servers * Can expand older machines with new servers Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
12
Elastically Optimize Exadata for Each Workload DB In-Memory Machine
Extreme Flash OLTP Machine
Data Warehousing Machine
Many DB Servers w/max DRAM), few Storage Servers
Equal DB Servers and Extreme Flash Storage Servers
Many HC Storage Servers, High Compute for Parallel SQL
576 DB Cores
396 DB Cores
512 Cores
13.3 TB RAM
8 TB RAM
90 TB Flash Cache
32 TB Flash
140 TB Flash
672 TB Storage
192 TB Disk • Speed of In-Memory DB • I/Os of PCIe Flash • Capacity of Disk for Archiving and Backups
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
13
Exadata X4-8 Elastic Configuration Large Scale OLTP, Consolidation, Database In-Memory
• 8-socket server version of Exadata, now with… • Elastic Compute Elastic
– 2 to 4 8-Socket DB servers per rack – Each with 120 Cores, up to 6TB RAM
• Elastic Storage – 3 to 14 X5-2 storage servers per rack
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
14
Exadata Hardware Generational Advances V1
V2
X2
X3
X4
X5
Sept 2008
Sept 2009
Sept 2010
Sept 2012
Nov 2013
Dec 2014
Xeon E5430 Q4, 2007 Intel GA
Xeon E5540 Q1, 2009 Intel GA
Xeon X5670 Q1, 2010 Intel GA
Xeon E5-2690 Q1, 2012 Intel GA
Xeon E5-2697v2 Q3, 2013 Intel GA
Xeon E5-2699 v3 Q3, 2014 Intel GA
Growth Factor
4X
Storage (TB) Flash (TB)
16 X
CPU (Cores)
4.5 X
Memory (GB)
24 X
Ethernet (GB/s)
50 X
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
15
Oracle Exadata Database Machine X5 Software Overview
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Exadata Software Spend Less by Doing More Unique Exadata Software Dramatically Improves • Database performance and consolidation • Network latency and throughput • Storage throughput, latency and capacity
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
17
Exadata X5-2 Software Overview • Fastest OLTP
• Fastest Analytics / Warehousing
– 4.14 million IOPS* – Smart Flash logging
• Most Efficient Consolidation – Workload prioritization from CPU to network to storage* – Near-zero overhead Virtual Machines*
• Maximum Availability – Optimizations for RAC, Data Guard, Backup and Flashback performance – Instant failover of In-Memory DB*
– SQL Offload to storage servers enables 263 GB/s analytics* – Pure-columnar Flash Cache* – JSON and XML analytics*
• Optimized Capacity – Smart Flash Cache gives speed of flash and capacity of disk – Hybrid Columnar Compression gives 10x size reduction for analytics and archive – Database Snapshots for test/dev*
* New or enhanced with Exadata X5 Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
18
New Exadata X5 Software Enhancements • Exafusion Direct-to-Wire OLTP Protocol • Pure Columnar Flash Cache • Smart Scan for JSON and XML • Oracle VM on Exadata • Smart Flash Cache Resource Management • Snapshots • In-Memory Fault Tolerance • HA, Management and Miscellaneous
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
19
OLTP: Exafusion Direct-to-Wire OLTP Protocol World’s First Database to InfiniBand Protocol
• InfiniBand has great throughput – But overheard of calling OS on every message limits small message rate
• Exafusion re-implements RAC Cache Fusion 10 GigE
– Bypasses networking software stack, interrupts, scheduling
InfiniBand
• Database directly calls InfiniBand hardware
InfiniBand Exafusion
3X Faster
8K OLTP Block Transfers/sec Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
20
Analytics: Exadata Pure Columnar Flash Cache World’s First Data Transforming Flash Cache Up to 5X Analytic Query Speedup • Hybrid Columnar Compression (HCC) balances OLTP and Analytics
Pure Columnar Data
– 10X compression, fast analytic scans, single I/O OLTP
Flash
• Smart Flash Cache now optimizes HCC data for pure analytic workloads • Automatically transforms HCC data to pure columnar format as it is loaded into flash cache
Flash Cache Population
Hybrid Columnar Data Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
21
Analytics: Smart Scan JSON and XML • Offloads rich set of JSON & XML operators – JSON: JSON_EXISTS, JSON_VALUE, JSON_QUERY, "IS JSON" and "IS NOT JSON" – XML: XMLExists, XMLCast(XMLQuery())
• 3x speedup in JSON analytic workloads select count(*) from pictures where json_value(photo, ‘$.tag’) like ‘%spain%’;
• Requires Oracle Database 12.1.0.2
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
22
Consolidation: Exadata Options Database 12c Multitenant
• VMs have good Isolation but poor efficiency and high management – Separate OS, memory, CPUs, and patching
More Isolation
More Efficiency
– Isolation without need to trust DBA, System Admin
Many DBs in one Server
• Database consolidation in a single OS is highly efficient but less isolated – DB Resource manager isolation adds no overhead; must trust admins to configure
VM
VM
VM
Virtual Machines
Dedicated DB Servers
• Best strategy is to combine VMs with database native consolidation – Multiple trusted DBs/PDBs in a VM
VM
VM
– Few VMs per server to limit overhead
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
23
Consolidation: High-Performance Virtualized Databases World’s First Virtualized Database Platform Using Ultra-Fast InfiniBand
FINANCE VM
SALES VM
• VMs provide CPU, memory, and OS isolation for consolidated workloads • Combine with Exadata network and I/O prioritization to achieve unique full-stack isolation • Oracle VM on Exadata delivers raw hardware performance – I/Os go directly to high-speed InfiniBand, bypassing Hypervisor
SUPPLY CHAIN VM
• Combine with Multitenant Database for highest consolidation density
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
24
Consolidation: Smart Flash Cache Resource Management • Flash cache usage in consolidated environments can be controlled by setting minimum or maximum flash cache usage per database FINANCE VM
SALES VM
ALTER IORMPLAN dbplan=( (name=sales, flashCacheMin = 100M, (name=finance, flashCacheLimit = 256M))
• Container database resource limits are specified directly – Pluggable database limits specified as percent of container DB
SUPPLY CHAIN VM
• I/O Resource management now automatically prioritizes flash OLTP I/Os over flash smart scan I/Os
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
25
Consolidation: Rapid Database Snapshots for Dev/Test • Fast space-efficient Snapshot database creation – Create sparse diskgroup on Exadata storage – Then create snapshot database or snapshot PDB that reads from base read-only copy of DB and writes to sparse diskgroup
Base DB
Sparse Sparse Snap Snap
CDB
• Integration with pluggable database enables creation of DB snapshot with single command create pluggable database johns_test from nightly_test_master create_file_dest='+SPARSE' snapshot copy;
• All Exadata features work on Snapshots (smart scans, smart flash cache, resource management …) • Requires Database 12.1.0.2 Bundle Patch Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
26
Management: Exadata I/O Analysis in AWR reports • Exadata Storage Server configuration and performance statistics are collected in Automatic Workload Repository (AWR) reports
• Three sections – Exadata Server Configuration – Exadata Server Health Report – Exadata Performance Statistics
• AWR Active Reports – Top Databases by IO requests and throughput – Performance drilldowns by database – Report outlier disks and cells
• Requires Oracle Database 12.1.0.2 Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
27
Management: Database Node Monitoring and Alerts • DBMCLI monitors, and manages DB nodes – Similar to CELLCLI for storage server
• Comprehensive monitoring and healing – Hard disks, battery, InfiniBand ports, ILOM, CPU, memory, temperature – CPU utilization, memory utilization, network interface throughput, file system utilization – Auto collect console history and alert on kernel panic – Threshold based alerts
• Alerts configured similar to storage server and delivered via E-mail and SNMP • Auto Service Request for component failures
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
28
Exadata with Maximum Availability Architecture (MAA) No Single Point of Failure Within Exadata Compute servers, DB servers, disks, flash, network, power
Log-based change replication with data consistency checking
Hardware Redundancy
Across Sites
Local standby for High-Availability Failover
Remote standby for Disaster Recovery
DATABASE IN-MEMORY
LAN
Redundant Systems Redundant Databases
Redundant Systems Redundant Databases DATABASE IN-MEMORY
Software Redundancy
DATABASE IN-MEMORY
Active clusters, Storage mirroring
Within a Site
Online patching, reconfiguration, expansion
WAN
Fastest RAC Node Failure Recovery | Deep ASM Mirroring Integration | Fastest Backup - RMAN Offload to Storage | Fastest Data Guard Redo Apply | Complete Failure Testing Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
29
High Availability: Additional Enhancements • Instant failure detection for database and storage servers – If a server disappears from both InfiniBand switches, declare it dead – No waiting for long heartbeat timeouts
• I/O latency capping – Disk and flash can exhibit occasional long latencies while performing internal maintenance or recovery operations – Exadata automatically redirects slow reads to a another cell – Writes redirected from slow disk or flash drive to a healthy flash device
• Eliminate false disk and flash failures – Many apparent drive failures are actually software lockups – Exadata automatically power cycles hung devices to clear software lockups
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
30
High Availability: In-Memory Fault Tolerance World’s First Fault Tolerant In-Memory Database Platform
• Similar to storage mirroring
• Duplicate In-Memory columns
on another node – Enabled per table/partition – Application transparent
Duplicate
Duplicate
Duplicate
• Downtime eliminated by using
Duplicate after node failure
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
31
Exadata for OLTP • Lowest latency IO and maximum IOPS with Smart Flash Cache • Fastest Redo Log Write with Smart Flash Logging • Unparalleled Data Integrity integrated end to end from database to storage • Smart network packet prioritization with Network Resource Management • Extreme Isolation from other workloads through CPU and IO resource management • Lowest overhead call into the network and maximum throughput with Exafusion • Instant death detection of storage servers and compute nodes without waiting for a heartbeat for extreme availability • Predictable performance with IO latency capping for reads and writes • Eliminate brownout after failure with In-memory fault tolerance • Prioritize rebalance of critical files • End to End Integrated simple diagnosis with AWR reports and Enterprise manager Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
32
Miscellaneous Enhancements • Storage Index now accelerates min() and max() queries with DB 12.1.0.2 – Especially useful for BI tools that want to quickly query column limits
• ACFS file system with 12.1.0.2 grid infrastructure – File locks, snapshots, faster small file operations
• Oracle Linux 6.6 update without reimaging entire system • FIPS 140-2 security compliance for smart scans (DB 11.2.0.4 and 12.1.0.2) • SNMP v3 support for more secure management • InfiniBand partitioning - non virtualized environments initially • Notification when disk rebalance completes (DB 12.1.0.2) • Assign databases to I/O resource management profiles to simplify configuration of large numbers of databases with DB 12.1.0.2.4
• X5-2 or X4-2 Cells now support up to 120,000 connections using active bonding Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
33
Exadata vs. Generic Hardware Benefit Gap is Growing, and will Continue to Grow
Application Benefits
• Software on Silicon • In-Memory Fault Tolerance • Direct-to-wire Protocol • Columnar Flash Cache • JSON and XML offload • Network Resource Management • Multitenant Aware Resource Mgmt • Compressed Flash Cache • IO Priorities • Prioritized File Recovery • Data Mining Offload
Hardware Improvements Common to Exadata & Generic Platforms
• Database Aware PCI Flash • Storage Indexes • Columnar Compression
• • • •
• Smart Scan • InfiniBand Scale-Out
2008
Faster Processors Faster, Cheaper Flash Larger Disk Drives Faster Network Interconnect 2014
Time Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
34
Spend Less by Paying Less for Software • Trusted Virtual Machine Partitions Pre-X5 Exadata
X5-2 Qtr Rack
– License DB software by Virtual Machine – not whole server – No extra fee for VM license, management tools or support
• Capacity-on-Demand Software Licensing – Start with 40% of CPUs enabled, turn on CPUs and license as needed – Fewer DB licenses minimum (16 cores) in Eighth Rack Exadata
• Run existing workload on less hardware to lower software costs – Software licenses are per processor and per disk (not per TB)
• All new Exadata software features are included in software license – Most new features apply to older generation machines – Transfer existing Database or Storage licenses from old Exadata to new Exadata X5 and pay zero for software on new X5 machine Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
35
Spend Less by Administrating Less • Save weeks to months of deployment time – Pre-assembled, pre-configured, pre-debugged – DB to disk
• Save tuning and debugging of components and interactions – Hundreds of engineer years testing, optimizing, and hardening DB, OS, drivers, firmware, network, and storage end-to-end
• Save troubleshooting and finger-pointing time – Oracle supports and is accountable for any issue – DB to Disk Oracle Experts Do the Work: • Save on DB and System patching: Platinum Support does it free Less Cost, • No need for separate management teams – DB Machine Admin Better Results, – Single end-to-end management tool (EM) Less Risk Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
36
Community Effect of Global Standard Reduces Risk • All Exadatas are the same, end-to-end – Only difference is number of servers
• Full-Stack performance, HA, and security identical to and pre-tested by: 1000s of Identical Configs Every Workload and Industry
– Oracle development, cloud, support – 1000s of Telecoms, Banks, Retailers, etc.
• You get: bank tested HA, government tested security, telecom tested scaling, etc.
• Common issues likely found by someone else Global Standard is Safer than Company Standard
• Unique-to-you issues minimized • Dedicated support team expert on single stack Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
37
Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
38