Understanding SSD Performance Using the SNIA SSS Performance Test Specification How to Use the PTS Results to Evaluate & Compare SSD Performance
An SSSI White Paper - January 2012 www.snia.org/forums/sssi/knowledge/education
About the Presenters Eden Kim is Chair of the SNIA Solid State Storage Technical Working Group and a member of the SNIA Solid State Storage Initiative Governing Board.
Doug Rollins joined Micron in 2009 as an applications engineer with the Enterprise SSD Products group. Prior to joining Micron, Mr. Rollins spent 13 years working in server system, network appliance, and storage platform/ data protection design and manufacture.
Mr. Kim was recognized in 2010 as the SNIA Outstanding Contributor for his work with the Solid State Storage Initiative and SSS Technical Working Group. Mr. Kim has been Chair of the SSS TWG since 2009 and has ushered the PTS through to publication.
Mr. Rollins is the named inventor in 13 U.S. patents and has been recognized by both the Storage Networking Industry Association (SNIA) and Intel Corporation for outstanding technical achievement. Mr. Rollins is an active member of several technical groups within SNIA including: The Solid State Storage Initiative and its Technical Working Group; Data Protection and Capacity Optimization; Marketing and Technical Development; Total Cost of Ownership; and the IO and Trace Tools Analysis. As co-chair of SNIA’s Solid State Storage Initiative’s Technical Working Group, Mr. Rollins was instrumental in the early development and validation of SNIA’s SSD Performance Test Specification.
Mr. Kim is CEO of Calypso Systems, Inc. which is the developer of the Calypso RTP / CTS SSD test platform. Calypso provides SSD Test and Measurement equipment and services to the solid state storage industry. Mr. Kim previously founded hard disk drive test companies Media Measurements, Inc., Swan Instruments, Inc. and acquired Scotts Valley Instruments. Mr. Kim received his BA/JD from the University of CA.
January 2012
Mr. Rollins earned his BA degree in mathematics from Humboldt State University.
1
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
2
Which SSD has the best Performance? …… it Depends…..
What Test Platform was used to test the drives? Was the test done at the File System or Device Level?
What Hardware/Software package was used?
Was the Drive Preconditioned? If so, how?
Was the HBA bandwidth sufficient?
Were the results taken at Steady State? How much data was written? What type of NAND Flash is it? Where was the data was written? What is the target workload? High Writes? High Reads? What data pattern was tested? Are there warranty life design issues?
January 2012
3
All NAND Flash SSDs Exhibit at least 3 Distinct Performance States:
“Fresh-Out-of-Box” (FOB) “Transition” “Steady State”
Steady State is the desirable test range.
January 2012
4
SSD Performance is HIGHLY Dependent on 3 Main Factors:
“Write History” “Measured Workload” “Hardware/Software Test Environment”
The exact same SSD can produce dramatically different results depending on these factors.
January 2012
5
Write History Previously written data may have more impact on performance than the measured IO command
Measured Workload The IO Access Pattern (Block Size / Read/Write Mix) can profoundly affect SSD performance - e.g. Large Block SEQ v Small Block RND
Hardware/Software H/W S/W should minimally affect measurements - is there sufficient bandwidth and host processing resources to generate the necessary IO loads? How much software overhead is there?
PTS Numbering Convention - IO Transfer Sizes and Alignment Reported in base 2 (eg RND 4KiB) Transfer Rates & Amounts Reported in base 10 (eg MB/s or TGBW)
January 2012
6
File System Test •
Specific File IO operations issued in the File System
•
IOs targeted at the Device traverse the SW/HW Stack
•
IOs are subject to cache, OS task switching & timing, driver fragmentation & coalescing
•
Original IO can be different at the Device level
•
Can lose 1:1 correspondence original IO & Physical Device IO
Synthetic Device Level Test • Applies a known and repeatable test stimulus • Targets Block IO Devices (not File System devices) • Uses Specified Test Workloads (Access Patterns, Data Pattern) • Specifies LBAs allowed to be used (ActiveRange, AR Amount) • Prescribes the Test Methodology
January 2012
7
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
8
IOs Traversing the SW/HW Stack - 7 Examples
NOTE: There will be a detailed webcast on How IOs Traverse the SW/HW Stack At a later date
January 2012
9
1. Cached File READ (or WRITE) I/O
NOTE: There is no R/W to SSD in this scenario
January 2012
10
2. Read-Ahead Caching I/O
Note: “Read-Ahead” caching I/O activity can occur: separately from or concurrently with application I/O operations.
January 2012
11
3. Write-Behind Caching I/O
Note “Write-Behind” caching I/O activity is occurs: separately from and/or concurrent with the application I/Os Note Cached data is subject to loss in the event of a power loss.
January 2012
12
4. Fragmented File I/O Operation
“Fragmented File” I/Os can occur for both READ and WRITE I/O operations The number of application I/Os may not correspond to device I/O operations to storage.
January 2012
13
5. Split I/O Commands for Large Data Transfers
“Split I/O Commands” due to large data transfer sizes can occur for both READ and WRITE I/O operations. The number of application I/Os may not correspond to device I/O operations to storage.
January 2012
14
6. Concurrent Sequential Access I/Os
RANDOM and SEQUENTIAL accesses are relative to where the respective I/O operations are performed/observed within the I/O stack. ACTUAL ACCESS PATTERNS impacted by Concurrent I/O operations “fragmented file” I/O operations Timing considerations and other factors
January 2012
15
7. File System MetaFile I/O Operations
Metafile I/O operations can introduce random access patterns to the device when interspersed with the concurrent I/O activity
January 2012
16
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
17
Creating a Standard: SNIA PTS-C & PTS-E Specifications
• Based on Synthetic Device Level Test • Standardized Preconditioning Methodology • Specified Test Workloads - Enterprise & Client • Test Hardware Specific • Test Software Agnostic - Tool Req’s Listed • Standardized PTS Report Format
January 2012
18
PTS rev 1.0 Performance Tests Test
Test Description
Purpose
Metric
WSAT
Continuous RND 4KiB W from FOB, No PC
FOB Performance Evolution over Time
IOPS
IOPS
Large & Small Block RND IOs at Steady State
Steady State IO Transfer Rate per second
IOPS
Throughput
Large Block SEQ R/W Data Transfer at Steady State
Steady State Bandwidth Speed
MB/Sec
Latency
AVE & MAX Response Times measured at a single OIO
Steady State IO Response Time Latency
mSec
WSAT
January 2012
IOPS
TP
19
LAT
Calypso Reference Test Platform (RTP 2.0) Hardware
January 2012
Software
Processor
Single Intel Xeon 5580W 3.2 Ghz 4 core
Operating System - Back End
CentOS 5.6
Motherboard
Intel 5520 HC
Test Software - Back End
CTS 6.5
RAM
12 GB ECC DDR3
Front End - GUI
Chrome Browser
HBA
6 Gb/s LSI 9212-4e-4i
Front End: OS, Database
Windows 7 / MySQL
20
MLC-A Test Report Summary Report Page – All Tests SNIA Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
SNIA SSS PTS Summary Report
MLC-A
DEVICE INFORMATION
Rev.
PTS 1.0
Page
1 of 26
Calypso Systems, Inc.
TEST HARDWARE PLATFORM
TEST SOFTWARE
REPORT DATE
SERIAL NO.
0000-0000-FFFF
SYSTEM
Calypso RTP 2.0
SYS OS
CENT OS 5.6
Report
06DEC11
FIRMWARE REV.
BFO1
Motherboard/cpu
Intel 5520HC / W5580
SW TOOL
Calypso CTS 6.5
Test Run
01NOV – 04DEC11
USER CAPACITY
MLC 256 GB
RAM
12GB ECC DDR3
SW Rev
1.19.10
Test Sponsor
Calypso
DEVICE INTERFACE
6 Gb/s SATA
Device Interface
LSA 9212-e 6Gb/s HBA
Release
Nov. 2011
Auditor
N/A
PTS 1.0 Report Format Summary Report Pages • Summary Report - All Tests
Testing Summary: Tests Run PTS-C 7.0
TEST
Purge
DP
OIO
WSAT - OPTIONAL
Security Erase
RND
TC 16 QD 2
PTS-C
TEST
Purge
DP
OIO
8.0
IOPS - REQUIRED
Security Erase
RND
TC 1 QD 8
DP
OIO
RND
TC 32 QD 32
PTS-C
TEST
Purge
9.0
THROUGHPUT REQUIRED
Security Erase
PTS-C
TEST
Purge
DP
OIO
10.0
LATENCY REQUIRED
Security Erase
RND
TC 1 QD 1
WIPC
WDPC
STEADY STATE
PC AR
TEST AR
AR AMT
SEGMENTS
WORKLOAD
TIME/GB
100%
100%
N/A
N/A
RND 4KiB W
24 Hrs 1.9 TB
WIPC
WDPC
STEADY STATE
PC AR
TEST AR
AR AMT
SEGMENTS
WORKLOAD
ROUNDS
100%
100%
16 GiB
2048
IOPS LOOP
2-6
WIPC
WDPC
STEADY STATE
PC AR
TEST AR
AR AMT
SEGMENTS
WORKLOAD
ROUNDS
100%
100%
16 GiB
2048
SEQ 1024KiB
1-5
WIPC
WDPC
STEADY STATE
PC AR
TEST AR
AR AMT
SEGMENTS
WORKLOAD
ROUNDS
100%
100%
16 GiB
2048
LAT LOOP
4–8
Test Sponsor – Special Notes ITEM
! ! January 2012 !
NOTATION
COMMENTS
21
Lists All Tests run on the Sample SSD Includes Key Set-up Information Generic Header Information applicable to All Tests
MLC-A Test Report Summary Report Page - IOPS SNIA Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
SNIA SSS PTS Summary Report
MLC-A
DEVICE INFORMATION
BFO1
FIRMWARE REV.
7 of 26
TEST SOFTWARE
REPORT DATE
SYS OS
CENT OS 5.6
Report
06DEC11
Intel 5520HC / W5580
SW TOOL
Calypso CTS 6.5
Test Run
01NOV – 04DEC11
Calypso RTP 2.0
SYSTEM Motherboard/cpu
PTS 1.0
Calypso Systems, Inc.
TEST HARDWARE PLATFORM
0000-0000-FFFF
SERIAL NO.
Rev. Page
USER CAPACITY
MLC 256 GB
RAM
12GB ECC DDR3
SW Rev
1.19.10
Test Sponsor
Calypso
DEVICE INTERFACE
6 Gb/s SATA
Device Interface
LSA 9212-e 6Gb/s HBA
Release
Nov. 2011
Auditor
N/A
PTS 1.0 Report Format Summary Report Pages • Summary Report - IOPS Tests
Test Description
Summary of Individual Test
Purpose
To measure RND IOPS matrix using different BS and R/W Mixes
Test Outline
PURGE, then apply preconditioning until Steady State is achieved according to the SNIA PTS
Preconditioning
PURGE followed by SNIA PTS prescribed WIPC & WDPC
Includes Summary Test Description
Test Set Up PTS-C
TEST
Purge
DP
OIO
8.0
IOPS - REQUIRED
Security Erase
RND
TC 1 QD 8
Includes Key Set-up Information
WIPC
WDPC
STEADY STATE
PC AR
TEST AR
AR AMT
SEGMENTS
WORKLOAD
ROUNDS
100%
100%
16 GiB
2048
IOPS LOOP
2-6
Select Performance Data RND 4KiB W
RND 4KiB R
RND 8KiB W
RND 8KiB R
3,147
29,876
1,584
21,723
Test Sponsor – Special Notes ITEM
January 2012
! !
NOTATION
COMMENTS
22
Select Performance Data Extracted
PTS 1.0 Report Format Individual Report Page Headers
Key Header Information • • • •
• Required on Each Individual Test Report Page • Lists the Settings pertaining to the reported Test
January 2012
23
Device & Test System DUT Preparation: PURGE & Preconditioning Test Loop Parameters: Data Pattern, OIO in QD & TC Steady State: Convergence, Rounds, PC AR, AR Amount
PTS-E
PTS: Tests Steps
1
Purge Set Conditions
2
5 6
January 2012
Secure Erase, Sanitize, Format Unit, other proprietary methods Set user selectable test parameters, such as Active Range, Data Pattern, Demand intensity
3
Pre-Condition
4
Run Until SS Collect Data
Generate Reports
Workload Independent Workload Dependent Reiterate loops until Steady State is reached, or run to a prescribed maximum number of loops Collect data from Steady State Measurement Window
Use standard report formats and include required and optional elements
24
PTS-E
PTS-E IOPS: Steady State “Rounds”
Purge WIPC: 2X SEQ 128KiB W
WDPC: IOPS “Loop” (7) R/W Mixes x (8) Block Sizes = 56 (1) Minute Tests 1 Round = 56 Minutes
January 2012
25
Steady State Convergence Plot – IOPS • Tracks Block Size IOPS by Rounds until Steady State • (8) IOPS Block Sizes: 0.5, 4, 8, 16, 32, 64, 128 & 1024 KiB Note Header Preconditioning & Test Loop Data • WIPC: 2X SEQ 128KiB • WDPC: Full IOPS Loop • Data Pattern: RND • OIO: 32 (TC 16 x QD 2) Reader can observe: • The Effects of RND IOs after SEQ Preconditioning • If all Block Sizes are evolving to Steady State • If the tracking Block Size variable is trending or fluctuating • The IOPS value of any Block Size at any Round
January 2012
26
Steady State Measurement Window – IOPS • Tracking Variable by Rounds until Steady State • IOPS Tracking Variable - RND 4KiB Note Header Steady State Information: • Steady State Convergence: Yes • Steady State Rounds: 14-18 • PC AR: 100% • AR Amount: 16GiB Reader can observe the: • Behavior of Tracking Variable over Convergence Rounds • Rounds constituting Steady State Measurement Window Note: Throughput and Latency Tracking Variables • PTS-C TP: SEQ 1024KiB in MB/s • PTS-E TP: SEQ 128KiB /1024KiB in MB/s • PTS-C & PTS-E LAT: RND 4KiB in mSec January 2012
27
SS Measurement Window Calculation IOPS Expands the 5 Round Steady State Measurement Window Note Steady State Determination Data: • 20% Max Data Excursion Bounds (2 dotted black lines) • Average IOPS of 5 Rounds (solid black line) • Least Mean Squares Linear Curve Fit (red line) • 10% Max Slope Excursion of Curved Fit (dashed black line) Reader can observe the: • “Quality” of Steady State Window • Scale of IOPS for Window Round Variation • Steady State Determination Calculations at bottom of page
January 2012
28
PTS Test Report IOPS: Test Set-Up Pages 1-3
MLC-A
Test Run Date:
11/14/2011 12:39 AM
Report Run Date:
Test Run Date:
11/21/2011 04:12 PM
11/14/2011 12:39 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB
NAND Type
MLC
Device I/F
6 Gb/s SATA
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
PTS-C 1.0
Page
1 of 6
REQUIRED: Data Pattern
Steady State
Serial No.
YES
Firmware Rev
%$"&
Rounds
1-5
Capacity
256 GB
PC AR
100%
NAND Type
MLC
2X SEQ/128KiB
Tester's Choice: OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Device I/F
Full IOPS Loop
Thread Count (TC)
1
AR Segments
2048
Test Platform
VENDOR: ABC CO.
6 Gb/s SATA
Purge
1220.98
3257.50
1640.21
819.63
410.45
203.45
98.33
17.62
2
1097.14 3253.88 %'(")*!+,%!
1371.77 %'(-!+,%!
817.42 %'(.!+,%!
395.00 95.98 %'(&/!+,%!176.20%'(01!+,%!
15.86 %'(/-!+,%!
3
1097.97
3253.88
1637.12
687.92
407.24
199.12
86.32
15.20
4
1097.97
2715.14
1638.55
816.18
343.34
200.55
86.34
17.17
5 3,500 1097.41
3254.83
1636.59
687.90
407.65
202.72
95.43
15.94
%'(&1.!+,%!
Test Run Date:
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Page
2 of 6
Steady State
RND
Tester's Choice:
1
3147.046087 3461.750696 2832.341478 3255.860085
5
3147.046087 3461.750696 2832.341478 '()*! +,-./0-!
Serial No.
YES
Firmware Rev
%$"&
Rounds
1-5
Capacity
256 GB
PC AR
100%
NAND Type
MLC 6 Gb/s SATA
8
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
3"12+,-./0-!
VENDOR: ABC CO.
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Convergence
OIO/Thread (QD)
3038.23209 &&"12+,-./0-!
Report Run Date:
11/21/2011 04:12 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
REQUIRED:
Full IOPS Loop
PTS-C 1.0
TEST SPONSOR
Data Pattern
2X SEQ/128KiB
Rev.
Purge Workload Independent
3 of 6
Steady State
RND
Tester's Choice:
Full IOPS Loop
PTS-C 1.0
Convergence
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Steady State Measurement Window %'(&"1-!+,%!
11/14/2011 12:39 AM
Client IOPS (REQUIRED) - Report Page
Test Loop Parameters
Security Erase Pre-Conditioning
Steady State Convergence Plot – All Block Sizes 1
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Convergence RND
11/21/2011 04:12 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Rev.
Report Run Date:
Client IOPS (REQUIRED) - Report Page
Client IOPS (REQUIRED) - Report Page
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Steady State Measurement Window – RND/4KiB *456-!
4,000
1
3257.50
2
3253.88
3
3253.88
4
2715.14
5
'()*!
+,-./0-!
&&"12+,-./0-!
3"12+,-./0-!
*456-!
3254.83
3,749
3,500
3,000 3,000
3,549
2,500
2,000
3,349 2,000
!"#
%$!"#
%$!"#
%$2,500
1,500
3,149
1,500 1,000
2,949
1,000 500
500
2,749
0 1
2
3
4
&'()*%
0 1
2
3
4
5
5
6
2,549 0
6
1
2
3
4
5
6
&'()*%
&'()*% Steady State Determination Data Average IOPS:
629.4 Measured Maximum Data Excursion:
Allowed Maximum Slope Excursion:
314.7 Measured Maximum Slope Excursion:
Least Squares Linear Fit Formula:
Steady State Convergence
January 2012
Steady State Window
29
3147.0
Allowed Maximum Data Excursion:
542.4 217.6 -54.407 * R + 3310.267
Steady State Measurement Calculations
PTS Test Report IOPS: Results Pages 4-6
MLC-A Test Run Date:
11/14/2011 12:39 AM
Report Run Date:
Test Run Date:
11/21/2011 04:12 PM
11/14/2011 12:39 AM
Device Under Test (DUT)
VENDOR: ABC CO. DUT Preparation
%####&####&""""
Serial No.
Purge
Firmware Rev
!"#$
Capacity
256 GB
NAND Type
MLC
Device I/F
6 Gb/s SATA
Test Platform
SSD MODEL NO: MLC-A 256 GB
Workload Independent
PTS-C 1.0
Page
4 of 6
Full IOPS Loop
11/21/2011 04:12 PM
Test Run Date:
11/14/2011 12:39 AM
Steady State
REQUIRED: RND
Tester's Choice:
Serial No.
VENDOR: ABC CO. DUT Preparation
!""""#""""#$$$$
Purge
Convergence
YES
Firmware Rev
%$"&
Rounds
1-5
Capacity
256 GB
PC AR
100%
NAND Type
MLC 6 Gb/s SATA
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
SSD MODEL NO: MLC-A 256 GB
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Data Pattern
Page
5 of 6
Steady State
VENDOR: ABC CO. Purge
Firmware Rev
%$"&
1-5
Capacity
256 GB
PC AR
100%
NAND Type
MLC 6 Gb/s SATA
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Serial No.
YES
Rounds
Workload Independent
6 of 6
Convergence RND
Tester's Choice:
Full IOPS Loop
PTS-C 1.0
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Client IOPS - ALL RW Mix & BS – 3D Columns
Client IOPS - ALL RW Mix & BS - 2D Plot
Client IOPS - ALL RW Mix & BS – Tabular Data
11/21/2011 04:12 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Convergence RND
Tester's Choice:
Full IOPS Loop
PTS-C 1.0
Device Under Test (DUT)
REQUIRED:
2X SEQ/128KiB
Rev.
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Report Run Date:
Client IOPS (REQUIRED) - Report Page
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
Rev.
Test Loop Parameters
Security Erase Pre-Conditioning
Report Run Date:
Client IOPS (REQUIRED) - Report Page
Client IOPS (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Read / Write Mix %
Block Size (KiB)
0/100
5/95
35/65
50/50
65/35
95/5
100/0
0.5
1,122.3
1,162.2
1,654.6
1,965.6
2,717.7
11,970.0
29,860.1
4
3,147.0
2,896.6
3,044.4
3,454.4
3,779.3
13,005.8
29,876.3
8
1,584.9
1,589.7
2,055.0
2,238.9
2,898.1
11,568.2
21,723.1
16
765.8
786.3
1,028.1
1,272.6
1,604.9
6,208.3
12,482.5
32
392.7
401.0
525.8
652.7
963.8
4,129.6
7,011.6
64
196.4
205.9
291.3
352.3
565.4
2,372.7
3,791.5
128
92.5
97.1
139.9
185.4
377.9
1,410.2
2,015.3
1024
16.4
16.5
23.3
27.3
90.8
191.4
266.7
"'&""!
(')(!
*('+(!
("'("!
+('*(!
)('(!
&""'"!
"'&""!
(')(!
*('+(!
("'("!
+('*(!
)('(!
&""'"!
100,000
30,000
10,000
25,000 1,000
()*+$
!"#
%$20,000
100
15,000 10,000 100/0
10
5,000 35/65 0
1 0.5
1
2
4
8
16
32
&'()*%$+,-%./+&0%
64
128
256
512
1024
0.5
65/35 4
8
16
32
64
0/100 128
,-./0$+&12$34&,5$
Tabular Data
January 2012
2D Plot
30
3D Bar Plot
1024
!"#$%&'$
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
31
SNIA PTS SSD Performance Test Comparison Products Tested
MLC-A 256 GB
Test Platform Hardware / Software
Calypso RTP 2.0 Intel 5520HC, Intel Xeon 5580 12GB ECC DDR3 RAM, LSI 6GB/s HBA
Calypso CTS 6.5 Cent OS 5.6
Tests Run
WSAT (FOB)
TP (Steady State)
MLC-B 160 GB
IOPS (Steady State)
SLC-A 100 GB
SLC-B 100 GB
LAT (Steady State)
Summary Performance - Selected Workload
MLC-A 256 GB
MLC-B 160 GB
SLC-A 100 GB
SLC-B 100 GB
FOB RND 4KiB W
56,896
20,364
39,092
55,677
Steady State RND 4KiB W
2,714
658
16,305
19,415
RND 4KiB R
29,876
38,087
93,707
46,365
RND 4KiB W
3,147
152
16,563
19,561
Throughput
SEQ 1024KiB R
417
264
514
437
MB/s
SEQ 1024KiB W
267
99
157
205
RND 4KiB R Ave
0.20
0.19
0.16
0.18
RND 4KiB W Ave
0.35
1.00
0.09
0.08
RND 4KiB R Max
1.60
1.43
6.26
0.39
RND 4KiB W Max
51.00
288.29
32.63
443.94
MLC-A
MLC-B
SLC-A
SLC-B
Test WSAT IOPS
Latency mSec (smaller value is better)
LINKS January 2012
32
SNIA PTS SSD Performance Test Comparison Products Tested
MLC-A 256 GB
Test Platform Hardware / Software
Calypso RTP 2.0 Intel 5520HC, Intel Xeon 5580 12GB ECC DDR3 RAM, LSI 6GB/s HBA
Calypso CTS 6.5 Cent OS 5.6
Tests Run
WSAT (FOB)
TP (Steady State)
MLC-B 160 GB
IOPS (Steady State)
SLC-A 100 GB
SLC-B 100 GB
LAT (Steady State)
Summary Performance - Selected Workload
MLC-A 256 GB
MLC-B 160 GB
SLC-A 100 GB
SLC-B 100 GB
FOB RND 4KiB W
56,896
20,364
39,092
55,677
Steady State RND 4KiB W
2,714
658
16,305
19,415
RND 4KiB R
29,876
38,087
93,707
46,365
RND 4KiB W
3,147
152
16,563
19,561
Throughput
SEQ 1024KiB R
417
264
514
437
MB/s
SEQ 1024KiB W
267
99
157
205
RND 4KiB R Ave
0.20
0.19
0.16
0.18
RND 4KiB W Ave
0.35
1.00
0.09
0.08
RND 4KiB R Max
1.60
1.43
6.26
0.39
RND 4KiB W Max
51.00
288.29
32.63
443.94
MLC-A
MLC-B
SLC-A
SLC-B
Test WSAT
Individual PTS Tests
IOPS
Latency mSec (smaller value is better)
LINKS January 2012
33
SNIA PTS SSD Performance Test Comparison Products Tested
MLC-A 256 GB
Test Platform Hardware / Software
Calypso RTP 2.0 Intel 5520HC, Intel Xeon 5580 12GB ECC DDR3 RAM, LSI 6GB/s HBA
Calypso CTS 6.5 Cent OS 5.6
Tests Run
WSAT (FOB)
TP (Steady State)
MLC-B 160 GB
IOPS (Steady State)
SLC-A 100 GB
SLC-B 100 GB
LAT (Steady State)
Summary Performance - Selected Workload
MLC-A 256 GB
MLC-B 160 GB
SLC-A 100 GB
SLC-B 100 GB
FOB RND 4KiB W
56,896
20,364
39,092
55,677
Steady State RND 4KiB W
2,714
658
16,305
19,415
RND 4KiB R
29,876
38,087
93,707
46,365
RND 4KiB W
3,147
152
16,563
19,561
Throughput
SEQ 1024KiB R
417
264
514
437
MB/s
SEQ 1024KiB W
267
99
157
205
RND 4KiB R Ave
0.20
0.19
0.16
0.18 0.08
Test WSAT IOPS
Latency mSec (smaller value is better)
LINKS January 2012
RND 4KiB W Ave
0.35
1.00
0.09
RND 4KiB R Max
1.60
1.43
6.26
0.39
RND 4KiB W Max
51.00
288.29
32.63
443.94
MLC-A
MLC-B
SLC-A
SLC-B
34
Link to Summary Report
Link to Full SNIA Report
PTS-C
MLC-A Summary Performance - Selected BS / RW
WSAT (Optional)
RND 4KiB
RND 8KiB
Block Size
FOB
56,896
Steady State
2,714
R/W Mix
IOPS
100% R 100% W
mSec (smaller value is better) January 2012
Setting
Parameter
Setting
PC AR
100%
QD / TC
2 / 16
Test AR
100%
DuraKon (T or GB)
20 Hr/18 TB
WIPC
NA -‐ FOB
Device I/F
6Gb/s SATAII
WDPC
NA -‐ FOB
Test System
RTP 2.0 / CTS 6.5
Parameter
Setting
Parameter
Setting
29,876
PC AR / Test AR
100% / 100%
QD / TC
8 / 1
3,147
AR Amt/Segments
16 GiB/2048
SS Rounds
1 -‐ 5
Device I/F
6Gb/s SATAII
No PC
WIPC:
SEQ128KiB + IOPS Loop
WDPC:
IOPS Loop
21,723
WIPC
100% W
1,585
WDPC
IOPS Loop
Test System
RTP 2.0 / CTS 6.5
R/W Mix
MB/Sec
Parameter
Setting
Parameter
Setting
PC AR / Test AR
100% / 100%
QD / TC
32 / 32
SEQ128KiB + SEQ 1024KiB
AR Amt/Segments
16 GiB/2048
SS Rounds
1 -‐ 5
SEQ 1024KiB
WIPC
2X SEQ 128KiB W + SEQ 1024KiB
Device I/F
6Gb/s SATAII
WDPC
SEQ 1024KiB
Test System
RTP 2.0 / CTS 6.5
417
SEQ 1024KiB 267
Block Size
R/W Mix
mSec
Parameter
Setting
Parameter
Setting
RND 4KiB AVE
100% R
0.20
PC AR / Test AR
100% / 100%
QD / TC
1 / 1
100% W
0.35
AR Amt/Segments
16 GiB/2048
SS Rounds
4 -‐ 8
100% R
1.60
WIPC
2X SEQ 128KiB W + LAT Loop
Device I/F
6Gb/s SATAII
100% W
51.00 35
WDPC
LAT Loop
Test System
RTP 2.0 / CTS 6.5
RND 4KiB MAX
FOB:
100% R
100% W
LATENCY
Parameter
2X SEQ 128KiB W + IOPS Loop
100% R
THROUGHPUT MB/Sec
IOPS
RND 4KiB 100% W
Block Size
IOPS TEST
FOB / Steady State
WIPC:
WDPC:
OIO = 1
PTS-C
MLC-B Summary Performance - Selected BS / RW
WSAT (Optional)
RND 4KiB
RND 8KiB
Block Size
THROUGHPUT MB/Sec
IOPS
FOB
20,364
Steady State
658
R/W Mix
IOPS
100% R 100% W
RND 4KiB 100% W
Block Size
IOPS TEST
FOB / Steady State
mSec (smaller value is better) January 2012
Setting
Parameter
Setting
PC AR
100%
QD / TC
2/ 16
Test AR
100%
DuraKon (T or GB)
12 Hr/250 GB
WIPC
NA -‐ FOB
Device I/F
6Gb/s SATAII
WDPC
NA -‐ FOB
Test System
RTP 2.0 / CTS 6.5
Parameter
Setting
Parameter
Setting
38,088
PC AR / Test AR
100% / 100%
QD / TC
32 / 1
152
AR Amt/Segments
16 GiB/2048
SS Rounds
9 -‐ 13
Device I/F
6Gb/s SATAII
100% R
22,979
WIPC
2X SEQ 128KiB W + IOPS Loop
100% W
151
WDPC
IOPS Loop
Test System
RTP 2.0 / CTS 6.5
R/W Mix
MB/Sec
Parameter
Setting
Parameter
Setting
100% R
PC AR / Test AR
100% / 100%
QD / TC
2 / 1
264
AR Amt/Segments
16 GiB/2048
SS Rounds
1 -‐ 5
WIPC
2X SEQ 128KiB W + SEQ 1024KiB
Device I/F
6Gb/s SATAII
WDPC
SEQ 1024KiB
Test System
RTP 2.0 / CTS 6.5
SEQ 1024KiB 100% W
LATENCY
Parameter
99
Block Size
R/W Mix
mSec
Parameter
Setting
Parameter
Setting
RND 4KiB AVE
100% R
0.19
PC AR / Test AR
100% / 100%
QD / TC
1 / 1
100% W
1.00
AR Amt/Segments
16 GiB/2048
SS Rounds
12 -‐ 16
100% R
1.43
WIPC
2X SEQ 128KiB W + LAT Loop
Device I/F
6Gb/s SATAII
WDPC
LAT Loop
Test System
RTP 2.0 / CTS 6.5
RND 4KiB MAX
100% W
288.29 36
PC AR: 16GiB 2048 Segments
PTS-E
SLC-A Summary Performance - Selected BS / RW
WSAT
IOPS TEST
RND 4KiB
RND 8KiB
Block Size
THROUGHPUT
SEQ 128KiB
MB/Sec
mSec (smaller value is better)
January 2012
FOB
39,092
Steady State
16,305
R/W Mix
IOPS
100% R 100% W
Parameter
Setting
Parameter
Setting
PC AR
100%
QD / TC
16 / 2
Test AR
100%
DuraKon (T or GB)
12 Hr / 2.9 TB
WIPC
NA -‐ FOB
Device I/F
6Gb/s SAS
WDPC
NA -‐ FOB
Test System
RTP 2.0 / CTS 6.5
Parameter
Setting
Parameter
Setting
93,707
PC AR
100%
QD / TC
16 / 2
16,563
Test AR
100%
SS Rounds
1 -‐ 5
100% R
50,301
WIPC
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% W
9,560
WDPC
IOPS Loop
Test System
RTP 2.0 / CTS 6.5
R/W Mix
MB/Sec
Parameter
Setting
Parameter
Setting
100% R
409
PC AR
100%
QD / TC
16 /2
100% W
145
Test AR
100%
SS Rounds
5 -‐ 9
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% R
514
WIPC
100% W
157
WDPC
SEQ 128./1024KiB
Test System
RTP 2.0 / CTS 6.5
Block Size
R/W Mix
mSec
Parameter
Setting
Parameter
Setting
RND 4KiB AVE
100% R
0.16
PC AR
100%
QD / TC
1 / 1
100% W
0.09
Test AR
100%
SS Rounds
3 -‐ 7
100% R
6.26
WIPC
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% W
32.63
WDPC
LAT Loop
Test System
RTP 2.0 / CTS 6.5
SEQ 1024KiB
LATENCY
IOPS
RND 4KiB 100% W
Block Size PTS-E TP 2 Block Sizes
FOB / Steady State
RND 4KiB MAX
37
PC AR: 100% LBA
PTS-E
SLC-B Summary Performance - Selected BS / RW
WSAT
RND 4KiB
RND 8KiB
Block Size
THROUGHPUT
SEQ 128KiB
MB/Sec
mSec (smaller value is better)
January 2012
FOB
55,677
Steady State
19,415
R/W Mix
IOPS
100% R
Parameter
Setting
Parameter
Setting
PC AR
100%
QD / TC
16 / 2
Test AR
100%
DuraKon (T or GB)
6 Hr / 1.9 TB
WIPC
NA -‐ FOB
Device I/F
6Gb/s SAS
WDPC
NA -‐ FOB
Test System
RTP 2.0 / CTS 6.5
Parameter
Setting
Parameter
Setting
46,365
PC AR
100%
QD / TC
16 / 2
100% W
19,561
Test AR
100%
SS Rounds
2 -‐ 6
100% R
32,259
WIPC
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% W
10,630
WDPC
IOPS Loop
Test System
RTP 2.0 / CTS 6.5
R/W Mix
MB/Sec
Parameter
Setting
Parameter
Setting
100% R
419
PC AR
100%
QD / TC
16 / 2
100% W
248
Test AR
100%
SS Rounds
4 -‐ 8
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% R
437
WIPC
100% W
205
WDPC
SEQ 128/1024KiB
Test System
RTP 2.0 / CTS 6.5
Block Size
R/W Mix
mSec
Parameter
Setting
Parameter
Setting
RND 4KiB AVE
100% R
0.18
PC AR
100%
QD / TC
1 / 1
100% W
0.08
Test AR
100%
SS Rounds
2 -‐ 6
100% R
0.39
WIPC
2X SEQ 128KiB W
Device I/F
6Gb/s SAS
100% W
443.94
WDPC
LAT Loop
Test System
RTP 2.0 / CTS 6.5
SEQ 1024KiB
LATENCY
IOPS
RND 4KiB 100% W
Block Size
IOPS TEST
FOB / Steady State
RND 4KiB MAX
38
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
39
Client WSAT MLC-A
WSAT Test Run Date:
11/17/2011 09:30 AM
Report Run Date:
11/22/2011 12:44 PM
RND 4KiB W from FOB
Client WSAT (OPTIONAL) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB
NAND Type
MLC
Device I/F
6 Gb/s SATA
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Test Loop Parameters
Security Erase Pre-Conditioning
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
N/A RND 4KiB
Rev.
PTS-C 1.0
Page
1 of 4
TEST SPONSOR
REQUIRED: Data Pattern Tester's Choice: OIO/Thread (QD) Thread Count (TC)
Steady State Convergence
RND
IOPS v TIME: • FOB Peak Drop: 60 Minutes • FOB to Steady State: 1000 Minutes • Peak IOPS: 56,896 • Steady State IOPS: 2,714
N/A
Rounds
N/A
PC AR
100%
2
AR AMOUNT
100%
16
AR Segments
N/A
Client IOPS (Linear) vs Time (Linear)
+""""!
*""""!
!"#
%$)""""!
IOPS v TGBW: • FOB Peak Drop: 250 TGBW • FOB to Steady State: 1400 TGBW
(""""!
'""""!
• SSD Capacity 256GB • Peak Drop in (1) Drive Fill • Steady State after (5) Drive Fills
&""""!
"! "!
'""!
)""!
+""!
,""! &"""! &'()%*+',-.)/0%
IOPS v Time
January 2012
&'""!
&)""!
&+""!
IOPS v TGBW
40
Note Header Information: • WIPC: None • WDPC: RND 4KiB • OIO: 32 (QD=2 / TC=16)
Client IOPS MLC-A
IOPS
Test Run Date:
11/14/2011 12:39 AM
Report Run Date:
11/21/2011 04:12 PM
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
VENDOR: ABC CO. DUT Preparation
!""""#""""#$$$$
Serial No. Firmware Rev
%$"&
Purge
256 GB MLC
Security Erase
Device I/F
6 Gb/s SATA
Workload Independent
Page
4 of 6
TEST SPONSOR Convergence
Data Pattern
RND
Tester's Choice:
Full IOPS Loop
YES
Rounds
1-5
PC AR
100% 16 GiB 2048
OIO/Thread (QD)
8
AR AMOUNT
Thread Count (TC)
1
AR Segments
Report Run Date:
11/21/2011 04:12 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
VENDOR: ABC CO. Purge
Firmware Rev
%$"&
Capacity
256 GB
NAND Type
MLC
Device I/F Test Platform
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Serial No.
Steady State
REQUIRED:
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
PTS-C 1.0
Test Loop Parameters
Pre-Conditioning
Capacity NAND Type
Test Platform
SSD MODEL NO: MLC-A 256 GB
Rev.
11/14/2011 12:39 AM
Client IOPS (REQUIRED) - Report Page
Client IOPS (REQUIRED) - Report Page
6 Gb/s SATA
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Convergence RND
Tester's Choice:
Full IOPS Loop
6 of 6
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
PTS-C 1.0
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Rev. Page
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Client IOPS - ALL RW Mix & BS – 3D Columns "'&""!
(')(!
*('+(!
("'("!
+('*(!
)('(!
&""'"!
Client IOPS - ALL RW Mix & BS – Tabular Data Read / Write Mix % 0/100
5/95
35/65
50/50
65/35
95/5
100/0
0.5
1,122.3
1,162.2
1,654.6
1,965.6
2,717.7
11,970.0
29,860.1
4
3,147.0
2,896.6
3,044.4
3,454.4
3,779.3
13,005.8
29,876.3
8
1,584.9
1,589.7
2,055.0
2,238.9
2,898.1
11,568.2
21,723.1
16
765.8
786.3
1,028.1
1,272.6
1,604.9
6,208.3
12,482.5
32
392.7
401.0
525.8
652.7
963.8
4,129.6
7,011.6
64
196.4
205.9
291.3
352.3
565.4
2,372.7
3,791.5
128
92.5
97.1
139.9
185.4
377.9
1,410.2
2,015.3
1024
16.4
16.5
23.3
27.3
90.8
191.4
266.7
30,000
25,000 20,000
()*+$
Block Size (KiB)
15,000 10,000
RND 4KiB IOPS 100% W: • •
January 2012
IOPS = 3,147 IOPS WSAT = 2,714 IOPS
100/0
Header Information: • • • • • •
5,000 35/65
WIPC: 2X SEQ 128KiB WDPC: IOPS Loop OIO: 8 (QD=8 / TC=1) SS Rounds: 1-5 AR Amount = 16GiB AR Segments = 2048
0 0.5
65/35 4
8
16
32
,-./0$+&12$34&,5$
41
64
0/100 128
1024
!"#$%&'$
Enterprise WSAT SLC-A
WSAT Test Run Date:
11/07/11 08:40 AM
Report Run Date:
Test Run Date:
11/14/2011 08:45 AM
11/07/11 08:40 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
!"#$
Capacity
100 GB
NAND Type
SLC
Purge
Format Unit Pre-Conditioning
Workload Independent
6 Gb/s SAS
Device I/F
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
%%%%&''''&""""
Firmware Rev
Test Platform
VENDOR: ABC CO.
RTP 2.0 CTS 6.5 Workload Dep.
None RND 4KiB
Rev.
PTS-E 1.0
Page
1 of 4
Test Loop Parameters REQUIRED: Data Pattern Tester's Choice: OIO/Thread (QD)
Convergence
RND
N/A
Serial No.
Rounds
N/A
Firmware Rev
%$&'
PC AR
100%
Capacity
100 GB
AR AMOUNT
100%
NAND Type
SLC
16
AR Segments
N/A
Device I/F
2
OPT:
N/A
Test Platform
Thread Count (TC)
VENDOR: ABC CO.
6 Gb/s SAS
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Purge
Format Unit
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
None RND 4KiB
Rev.
PTS-E 1.0
Page
3 of 4
Test Loop Parameters Data Pattern
Convergence
Tester's Choice:
N/A
Rounds
N/A
PC AR
100%
AR AMOUNT
100%
RND
OIO/Thread (QD)
16
AR Segments
N/A
Thread Count (TC)
2
OPT:
N/A
Enterprise IOPS (Linear) vs TGBW (Linear)
.')'''(
.#)###(
-')'''(
-#)###(
,')'''(
!"#
%$,#)###(
+')'''(
!#)###(
%')'''(
#( #(
'( '(
-'(
%''(
%-'(
*''( *-'( &'()%*+',-.)/0%
IOPS v Time
January 2012
+''(
+-'(
,''(
IOPS v TIME: • FOB Peak Drop: 30 Minutes • FOB to Steady State: 50 Minutes • Peak IOPS: 39,092 • Steady State IOPS: 16,305 IOPS v TGBW: • FOB Peak Drop: 70 GB • FOB to Steady State: 150 GB
+#)###( *#)###(
*')'''(
RND 4KiB W from FOB
TEST SPONSOR
REQUIRED:
Pre-Conditioning
Enterprise IOPS (Linear) vs Time (Linear)
!"#
%$11/14/2011 08:45 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Report Run Date:
Enterprise WSAT (REQUIRED) - Report Page
Enterprise WSAT (REQUIRED) - Report Page
*##(
,##(
.##( /##( !###( !*##( &'()*%+,-)./(01%23,405%6+78%
IOPS v TGBW
42
!,##(
!.##(
!/##(
*###(
• SSD Capacity 100GB • Peak Drop in (0.7) Drive Fills • Steady State after (1.5) Drive Fills Note Header Information: • WIPC: None • WDPC: RND 4KiB • OIO: 32 (QD=16 / TC=2)
Enterprise IOPS SLC-A
IOPS
Test Run Date:
11/02/2011 02:56 PM
Report Run Date:
11/14/2011 08:43 AM
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%%%%&''''&""""
Firmware Rev
!"#$
Capacity
100 GB
NAND Type
SLC
VENDOR: ABC CO. DUT Preparation Purge
4 of 6
Convergence
Data Pattern
RND
Tester's Choice:
2X SEQ/128KiB Full IOPS Loop
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
VENDOR: ABC CO. Purge
Firmware Rev
!"#$
Capacity
100 GB
NAND Type
SLC 6 Gb/s SAS
Device I/F Test Platform
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
%%%%&''''&""""
Serial No.
Steady State
Report Run Date:
11/14/2011 08:43 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
REQUIRED:
Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
PTS-E 1.0
Test Loop Parameters
Format Unit
Workload Independent
6 Gb/s SAS
Device I/F Test Platform
SSD MODEL NO: SLC-A 100 GB
Rev. Page
11/02/2011 02:56 PM
Enterprise IOPS (REQUIRED) - Report Page
Enterprise IOPS (REQUIRED) - Report Page
Test Loop Parameters
Format Unit
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
RND
Tester's Choice:
Full IOPS Loop
6 of 6
Convergence
Data Pattern
2X SEQ/128KiB
PTS-E 1.0
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
Enterprise IOPS - ALL RW Mix & BS – 3D Columns
'(%'')
Enterprise IOPS - ALL RW Mix & BS – Tabular Data
*(+*)
,*(-*)
*'(*')
-*(,*)))
+*(*)
%''(')
Read / Write Mix %
Block Size (KiB)
0/100
5/95
35/65
50/50
65/35
95/5
100/0
0.5
15,887.4
16,634.7
20,678.6
24,402.8
29,386.2
72,428.4
95,924.3
4
16,563.0
17,032.2
20,234.2
23,705.2
28,018.6
63,447.7
93,707.0
100,000
9,559.8
9,998.4
12,547.1
14,636.6
17,199.1
37,872.9
50,301.2
90,000
4,842.2
5,032.3
6,802.5
8,132.1
9,655.8
22,462.2
31,072.8
80,000
32
2,413.3
2,535.4
3,478.4
4,241.3
5,061.7
12,174.7
15,994.2
70,000
64
1,219.2
1,275.7
1,728.4
2,126.1
2,726.3
6,284.6
8,094.9
60,000
128
612.7
632.5
859.1
1,061.4
1,709.4
3,205.7
4,060.8
1024
74.8
78.0
103.6
126.7
202.7
398.8
514.6
()*+$
8 16
50,000 40,000 30,000 100/0
20,000
RND 4KiB IOPS 100% W: • •
January 2012
IOPS = 16,563 IOPS WSAT = 16,305 IOPS
Header Information: • • • • •
65/35
10,000
WIPC: 2X SEQ 128KiB WDPC: IOPS Loop OIO: 32 (QD=16 / TC=2) SS Rounds: 1-5 PC AR = 100%
0 0.5
35/65 4
8
16
32
,-./0$+&12$34&,5$
43
64
0/100 128
1024
!"#$%&'$
Client Throughput MLC-A
TP Test Run Date:
11/13/2011 10:24 AM
Report Run Date:
11/21/2011 04:03 PM
Client Throughput Test (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
+*&%
Capacity
256 GB
NAND Type
MLC
Purge
Security Erase
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB SEQ 1024KiB
PTS-C 1.0
Page
4 of 5
TEST SPONSOR Steady State Convergence
Data Pattern
Test Run Date:
RND
Tester's Choice:
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
32
AR AMOUNT
16 GiB
Thread Count (TC)
32
AR Segments
2048
Block Size (KiB) 1024
Serial No.
!"#$
Capacity
256 GB
NAND Type
MLC
267.2
6 Gb/s SATA
Device I/F Test Platform
VENDOR: ABC CO.
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
%####&####&""""
Firmware Rev
11/21/2011 04:03 PM
Purge
Test Loop Parameters
Security Erase
Workload Independent
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
SEQ 1024KiB
Tester's Choice:
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
32
AR AMOUNT
16 GiB
Thread Count (TC)
32
AR Segments
2048
SEQ 1024KiB R/W 100% R:
417 MB/Sec
100% W:
267 MB/Sec
+##%
100/0 416.9
'()*+,(-+.$/01234$
%&#$ )##%
(##%
!"#$
'##%
$##%
#% #,$##%
$##,#% 526$078$
January 2012
5 of 5
Convergence RND
*##%
TP Tables
PTS-C 1.0
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR
Client Throughput - ALL RW Mix & BS - 2D Plot
Read / Write Mix % 0/100
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Client Throughput - ALL RW Mix & BS – Tabular Data !"# $%# &'%&&%&&'&
11/13/2011 10:24 AM
Client Throughput Test (REQUIRED) - Report Page
Device Under Test (DUT)
Test Loop Parameters REQUIRED:
Pre-Conditioning
6 Gb/s SATA
Device I/F
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
(&&&&)&&&&)****
Firmware Rev
Test Platform
VENDOR: ABC CO.
Rev.
TP Pots
44
Note Header Information: • Device I/F: 6Gb/s SATA • OIO: 1024 (QD=32 / TC=32) • WIPC: SEQ 128KiB • WDPC: SEQ 1024KiB • SS Rounds: 1-5 • AR Amount: 16GiB • AR Segments: 2048
Enterprise Throughput SLC-A 1024KiB
TP Test Run Date:
12/04/2011 08:21 AM
Report Run Date:
12/04/2011 10:03 AM
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
)(*+
Capacity
100 GB
NAND Type
SLC 6 Gb/s SAS
Device I/F
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!'%%%%'((((
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Format Unit
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB SEQ 1024KiB
PTS-E 1.0
Page
4 of 5
Test Loop Parameters Data Pattern
Steady State
RND
Tester's Choice:
Serial No.
YES
Firmware Rev
%$&'
Rounds
4-8
Capacity
100 GB
PC AR
100%
NAND Type
SLC
OIO/Thread (QD)
16
AR AMOUNT
100%
Device I/F
Thread Count (TC)
2
AR Segments
N/A
Test Platform
Block Size (KiB) 1024
157.5
12/04/2011 10:03 AM
6 Gb/s SAS
SSD MODEL NO: SLC-A 100 GB
DUT Preparation Purge
Test Loop Parameters
Format Unit
Workload Independent
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
SEQ 1024KiB
Tester's Choice:
YES
Rounds
4-8
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
SEQ 1024KiB R/W 100% R:
514 MB/Sec
100% W:
157 MB/Sec
-##(
100/0
"!%$
514.3
&'()*+',*-$./0123$
+##(
*##(
)##(
!"#$
!##(
#( #.!##(
!##.#( 415$/67$
January 2012
5 of 5
Convergence RND
,##(
TP Tables
PTS-E 1.0
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR
Enterprise 1024KiB Throughput - ALL RW Mix & BS - 2D Plot
Read / Write Mix % 0/100
VENDOR: ABC CO.
!!!!"####"$$$$
Convergence
Enterprise 1024KiB Throughput - ALL RW Mix & BS – Tabular Data !"# "!$ %&!%%!%%&%
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
REQUIRED:
Pre-Conditioning
Rev.
12/04/2011 08:21 AM
Enterprise Throughput Test (REQUIRED) - Report Page
Enterprise Throughput Test (REQUIRED) - Report Page
TP Pots
45
Note Header Information: • Device I/F: 6Gb/s SAS • OIO: 32 (QD=16 / TC=2) • WIPC: SEQ 128KiB • WDPC: SEQ 1024KIB • SS Rounds: 4-8 • AR Amount: 100% • AR Segments: N/A
Enterprise Throughput SLC-A 128KiB
TP Test Run Date:
12/03/2011 04:32 PM
Report Run Date:
12/04/2011 10:04 AM
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
)(*+
Capacity
100 GB SLC
NAND Type
6 Gb/s SAS
Device I/F
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!'$$$$'((((
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Format Unit
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB SEQ 128KiB
PTS-E 1.0
Page
4 of 5
TEST SPONSOR
Test Loop Parameters
Steady State
REQUIRED:
Pre-Conditioning
Rev.
Data Pattern
RND
Tester's Choice:
Convergence
YES
Rounds
4-8
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
Block Size (KiB) 128
Device Under Test (DUT) Serial No.
%$&'
Capacity
100 GB
NAND Type
SLC 6 Gb/s SAS
Device I/F Test Platform
144.5
VENDOR: ABC CO.
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Firmware Rev
12/04/2011 10:04 AM
Purge
Test Loop Parameters
Format Unit
Workload Independent
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
SEQ 128KiB
Tester's Choice:
YES
Rounds
4-8
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
SEQ 128KiB R/W 100% R:
409 MB/Sec
100% W:
145 MB/Sec
-##(
100/0 409.3
,##(
'()*+,(-+.$/01234$
+##(
*##(
)##( !"#$ !##(
#( #.!##(
!##.#( 526$078$
January 2012
5 of 5
Convergence RND
"%&$
TP Tables
PTS-E 1.0
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR
Enterprise 128KiB Throughput - ALL RW Mix & BS - 2D Plot
Read / Write Mix % 0/100
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Enterprise 128KiB Throughput - ALL RW Mix & BS – Tabular Data !"# "$% $&!$$!$$&$
12/03/2011 04:32 PM
Enterprise Throughput Test (REQUIRED) - Report Page
Enterprise Throughput Test (REQUIRED) - Report Page
TP Pots
46
Note Header Information: • Device I/F: 6Gb/s SAS • OIO: 32 (QD=16 / TC=2) • WIPC: SEQ 128KiB • WDPC: SEQ 128KiB • SS Rounds: 4-8 • AR Amount: 100% • AR Segments: N/A
Client Latency MLC-A
LAT
11/11/2011
Test Run Date:
09:53 AM
11/15/2011 03:34 PM
Report Run Date:
Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB
NAND Type
MLC
Device I/F
6 Gb/s SATA
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB Full Latency Loop
Rev.
PTS-C 1.0
Page
5 of 6
Steady State
Serial No.
YES
Firmware Rev
%$"&
Rounds
4-8
Capacity
256 GB
PC AR
100%
NAND Type
MLC 6 Gb/s SATA
Tester's Choice: OIO/Thread (QD)
1
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
VENDOR: ABC CO.
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Convergence RND
Purge
Workload Independent
2X SEQ/128KiB
REQUIRED: Data Pattern
Steady State Convergence
RND
YES
Rounds
4-8
PC AR
100%
1
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Full Latency Loop
&#(!&$
!(-'$ &!()!$
*!(!$
!(*&$
!(&!$
3145$6478$
!(&*$
!(%!$ !('&$
!(*!$
!(*'$
!(''$
!('!$ !()!$
!(#!$
!()*$
'-(*+$
'!(!$ )!(!$ #!(!$ #(!*$
%&"'&$
!(!!$ #!!"!$
*$ ,$
."/$012$
%&"'&$
#(&-$
!(!$
!(&$
!(&$
#!!"!$
*$ +$
9:;1?5$6@198$
9:;1?5$6@198$
AVE Latency
AVE Latency: • 4KiB 100/0 R/W: 0.20 mSec • 4KiB 65/35 R/W: 0.30 mSec • 4KiB 0/100 R/W: 0.35 mSec
!"#!!$
!"#!!$
!()!$
&'(')$
&!(!)$
&!(!$
!(+!$
RND 0.5, 4, 8 KiB
MAX Latency: • 4KiB 100/0 R/W: 1.30 mSec • 4KiB 65/35 R/W: 50.02 mSec • 4KiB 0/100 R/W: 51.05 mSec
&)(,%$
%!(!$
!(,!$
3145$6478$
6 of 6
Client - MAX Latency vs BS and R/W Mix - 3D Plot
!(-!$
January 2012
PTS-C 1.0
Tester's Choice: OIO/Thread (QD)
&)(#*$
!()!$
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Client - AVE Latency vs BS and R/W Mix - 3D Plot
#(!!$
11/15/2011 03:34 PM
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
REQUIRED: Data Pattern
09:53 AM
Client Latency (REQUIRED) - Report Page
Test Loop Parameters
Security Erase Pre-Conditioning
11/11/2011
Test Run Date:
Client Latency (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
MAX Latency
47
."/$012$
Note Header Information: • OIO: 1 (QD=1 / TC=1) • WIPC: 2X SEQ 128KiB • WDPC: LAT Loop • SS Rounds: 4-8
Enterprise Latency SLC-A
LAT Test Run Date:
11/10/2011 05:07 AM
Report Run Date:
11/14/2011 08:47 AM
Enterprise Latency (REQUIRED) - Report Page
Device Under Test (DUT) Serial No.
%$&'
Capacity
100 GB
NAND Type
SLC
Device I/F
6 Gb/s SAS
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Data Pattern
11/10/2011 05:07 AM
Report Run Date:
11/14/2011 08:47 AM
Page
5 of 6
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
YES
Firmware Rev
%$&'
Rounds
3-7
Capacity
100 GB
PC AR
100%
NAND Type
SLC
OIO/Thread (QD)
1
AR AMOUNT
100%
Device I/F
Thread Count (TC)
1
AR Segments
N/A
Test Platform
VENDOR: ABC CO.
6 Gb/s SAS
Purge Workload Independent
6 of 6
Convergence RND
Tester's Choice:
Full LAT Loop
PTS-E 1.0
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev. Page
YES
Rounds
3-7
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
100%
Thread Count (TC)
1
AR Segments
N/A
Enterprise MAX Latency vs BS and R/W Mix - 3D Plot
-!(!!$
,!(*'$
,!(!!$
3145$6478$
!(#&$
!(!+$
!(#*$
!(#-$ !(!,$
!(#%$
!(#-$
!(#!$ !"#!!$
!(!&$
%&"'&$
!(!!$ !(&$
#!!"!$
*$ +$
9:;1?5$6@198$
AVE Latency
January 2012
')(%'$
%!(!!$
!(#'$
3145$6478$
!(#'$
!()!$ !(#&$
&%(,+$
+!(!!$
!()&$
."/$012$
&!(!!$
*%(+,$
'-(*%$
*!(!!$
'-(',$
'!(!!$ !"#!!$
)!(!!$ &(,!$
#!(!!$ !(!!$ !(&$
#!!"!$
*$ ,$
MAX Latency
48
AVE Latency: • 4KiB 100/0 R/W: 0.16 mSec • 4KiB 65/35 R/W: 0.14 mSec • 4KiB 0/100 R/W: 0.09 mSec MAX Latency: • 4KiB 100/0 R/W: 6.26 mSec • 4KiB 65/35 R/W: 46.78 mSec • 4KiB 0/100 R/W: 32.63 mSec
%&"'&$
#)(--$
%()%$
9:;1?5$6@198$
RND 0.5, 4, 8 KiB
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Enterprise AVE Latency vs BS and R/W Mix - 3D Plot
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Convergence RND
Tester's Choice:
Full LAT Loop
PTS-E 1.0
Steady State
REQUIRED:
2X SEQ/128KiB
Rev.
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
Test Run Date:
Enterprise Latency (REQUIRED) - Report Page
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
."/$012$
Note Header Information: • OIO: 1 (QD=1 / TC=1) • WIPC: 2X SEQ 128KiB • WDPC: LAT Loop • SS Rounds: 3-7
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
49
Comparing Client WSAT - IOPS v TIME
WSAT Test Run Date:
11/17/2011 09:30 AM
Report Run Date:
11/22/2011 12:44 PM
Test Run Date:
11/21/2011 06:35 AM
Client WSAT (OPTIONAL) - Report Page
Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB MLC
NAND Type
6 Gb/s SATA
Device I/F
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
REQUIRED: Data Pattern Tester's Choice: OIO/Thread (QD)
N/A RND 4KiB
Rev.
PTS-C 1.0
Page
1 of 4
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Thread Count (TC)
Device Under Test (DUT)
Steady State
RND
Serial No.
N/A
Firmware Rev
"&'(
Rounds
N/A
Capacity
160 GB
PC AR
100%
2
AR AMOUNT
100%
Device I/F
AR Segments
N/A
Test Platform
MLC
NAND Type
3 Gb/s SATA
Purge
Workload Independent
)""""!
+")"""!
!"#
%$!"#
%$,")"""!
(""""!
&")"""!
&""""!
$")"""!
"!
"!
MLC-A 256 GB
January 2012
1 of 4
REQUIRED: Data Pattern Tester's Choice: OIO/Thread (QD) Thread Count (TC)
&'""!
&)""!
&+""!
WSAT
Steady State
RND
Convergence
N/A
Rounds
N/A
PC AR
100%
2
AR AMOUNT
100%
16
AR Segments
N/A
FOB 4KiB SS 4KiB Peak Drop Steady State
IOPS v TIME MLC-A
MLC-B
55,896
20,364
2,714
658
60 Min
46 Min
1,000 Min
275 Min
*")"""!
'""""!
,""! &"""! &'()%*+',-.)/0%
PTS-C 1.0
Client IOPS (Linear) vs Time (Linear)
*""""!
+""!
N/A RND 4KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
-")"""!
)""!
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Convergence
16
VENDOR: ABC CO.
Client IOPS (Linear) vs Time (Linear)
'""!
11/24/2011 04:44 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
+""""!
"!
Report Run Date:
Client WSAT (OPTIONAL) - Report Page
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
"!
&""!
+""!
-""!
.""! $"""! &'()%*+',-.)/0%
MLC-B 160 GB
50
$&""!
$+""!
$-""!
NOTE: MLC-B x & y axis are scaled to match MLC-A for comparison
WSAT
Comparing Client WSAT - IOPS v TGBW Test Run Date:
11/21/2011 06:35 AM
Report Run Date:
11/24/2011 04:44 PM
Client WSAT (OPTIONAL) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
Purge
"&'(
Capacity
160 GB
3 Gb/s SATA
Device I/F
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
3 of 4
OIO/Thread (QD) Thread Count (TC)
WSAT
Steady State Convergence
N/A
Rounds
N/A
PC AR
100%
2
AR AMOUNT
100%
16
AR Segments
N/A
Tester's Choice:
RND 4KiB
PTS-C 1.0
RND
REQUIRED: Data Pattern
N/A
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
MLC
NAND Type
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Firmware Rev
Test Platform
VENDOR: ABC CO.
Client IOPS (Linear) vs Total Gigabytes Written (Linear)
FOB 4KiB SS 4KiB
-")"""!
Peak Drop
,")"""!
Steady State Peak Drop User Capacity Steady State User Capacity
+")"""!
!"#
%$*")"""!
IOPS v TGBW MLC-A
MLC-B
55,896
20,364
2,714
658
250 TGBW
38 TBBW
1,400 TGBW
80 TGBW
1 Drive Fill
.38 Drive Fills
5 Drive Fills
.8 Drive Fills
&")"""!
$")"""!
"! "!
&""!
+""!
-""!
.""!
$"""!
$&""!
$+""!
$-""!
$.""!
&"""!
NOTE: MLC-B x & y axis are scaled to match MLC-A for comparison
&'()*%+,-)./(01%23,405%6+78%
NOTE: MLC-B TGBW is limited by slower RND 4KiB W speed (1440 minute test)
MLC-A 256 GB
January 2012
MLC-B 160 GB
51
MLC-A interface is 6Gb/s MLC-B interface is 3Gb/s
Comparing Enterprise WSAT - IOPS v TIME
WSAT Test Run Date:
11/07/11 08:40 AM
Report Run Date:
Test Run Date:
11/14/2011 08:45 AM
11/02/2011 11:54 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
!"#$
Capacity
100 GB SLC
NAND Type
6 Gb/s SAS
Device I/F
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
%%%%&''''&""""
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
None RND 4KiB
Rev.
PTS-E 1.0
Page
1 of 4
Convergence
N/A
Serial No.
REQUIRED: Data Pattern
Rounds
N/A
Firmware Rev
$%&!
RND
PC AR
100%
Capacity
100 GB
Tester's Choice: OIO/Thread (QD)
AR AMOUNT
100%
16
AR Segments
N/A
Device I/F
2
OPT:
N/A
Test Platform
Thread Count (TC)
VENDOR: ABC CO.
SLC
NAND Type
6 Gb/s SATA
SSD MODEL NO: SLC-B 100 GB
DUT Preparation
!!!!"!!!!"####
Purge
Workload Independent
None RND 4KiB
Rev.
PTS-E 1.0
Page
1 of 4
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Convergence Rounds
N/A
RND
PC AR
100%
Tester's Choice: OIO/Thread (QD)
AR AMOUNT
100%
16
AR Segments
N/A
2
OPT:
N/A
Thread Count (TC)
WSAT
IOPS v TIME SLC-A
SLC-B
FOB 4KiB
39,092
55,677
SS 4KiB
16,305
19,415
N/A
REQUIRED: Data Pattern
Enterprise IOPS (Linear) vs Time (Linear)
Enterprise IOPS (Linear) vs Time (Linear) .')'''(
-&(&&&'
Peak Drop
30 Min
30 Min
-')'''(
,&(&&&'
Steady State
50 Min
100 Min
,')'''(
+&(&&&' !"#
%$!"#
%$11/14/2011 01:15 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
Report Run Date:
Enterprise WSAT (REQUIRED) - Report Page
Enterprise WSAT (REQUIRED) - Report Page
+')'''(
*&(&&&'
*')'''(
)&(&&&'
%')'''(
!&(&&&'
'(
&'
'(
-'(
%''(
%-'(
*''( *-'( &'()%*+',-.)/0%
+''(
+-'(
,''(
&'
,&'
!&&'
!,&'
)&&' ),&' &'()%*+',-.)/0%
*&&'
*,&'
+&&'
NOTE: SLC-A x & y axis are scaled to match SLC-B for comparison NOTE: SLC-B TIME is less due to faster RND 4KiB W speed (to X User Capacity)
SLC-A 100 GB
January 2012
SLC-B 100 GB
52
Comparing Enterprise WSAT - IOPS v TGBW
WSAT Test Run Date:
11/07/11 08:40 AM
Report Run Date:
11/14/2011 08:45 AM
Test Run Date:
11/02/2011 11:54 AM
Enterprise WSAT (REQUIRED) - Report Page
Device Under Test (DUT) Serial No.
%$&'
Capacity
100 GB SLC
NAND Type Device I/F
6 Gb/s SAS
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Format Unit
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
None RND 4KiB
Rev.
PTS-E 1.0
Page
3 of 4
TEST SPONSOR
Test Loop Parameters
Device Under Test (DUT) Convergence
N/A
Serial No.
Rounds
N/A
Firmware Rev
$%&!
PC AR
100%
Capacity
100 GB
AR AMOUNT
100%
OIO/Thread (QD)
16
AR Segments
N/A
Device I/F
Thread Count (TC)
2
OPT:
N/A
Test Platform
Data Pattern Tester's Choice:
VENDOR: ABC CO.
SLC
NAND Type
6 Gb/s SATA
SSD MODEL NO: SLC-B 100 GB
DUT Preparation
!!!!"!!!!"####
Purge Workload Independent
None RND 4KiB
Rev.
PTS-E 1.0
Page
3 of 4
TEST SPONSOR
Test Loop Parameters
Security Erase
Convergence
N/A
PC AR
100%
AR AMOUNT
100%
OIO/Thread (QD)
16
AR Segments
N/A
Thread Count (TC)
2
OPT:
N/A
Data Pattern Tester's Choice:
Enterprise IOPS (Linear) vs Total Gigabytes Written (Linear)
.#)###(
-&(&&&'
-#)###(
,&(&&&'
,#)###(
!"#
%$)&(&&&'
!#)###(
!&(&&&'
#( #(
*##(
,##(
.##( /##( !###( !*##( &'()*%+,-)./(01%23,405%6+78%
SLC-A 100 GB
January 2012
!,##(
!.##(
!/##(
*###(
SLC-B
FOB 4KiB
39,092
55,677
SS 4KiB
16,305
19,415
70 TGBW
200 TBBW
150 TGBW
600 TGBW
.7 Drive Fill
2 Drive Fills
1.5 Drive Fills
6 Drive Fills
Steady State Peak Drop User Capacity Steady State User Capacity
*&(&&&'
*#)###(
&' &'
)&&'
+&&'
-&&' .&&' !&&&' !)&&' &'()*%+,-)./(01%23,405%6+78%
SLC-B 100 GB
53
!+&&'
!-&&'
!.&&'
)&&&'
IOPS v TGBW SLC-A
Peak Drop
+&(&&&'
+#)###(
WSAT
N/A
Rounds RND
REQUIRED:
Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Enterprise IOPS (Linear) vs TGBW (Linear)
!"#
%$11/14/2011 01:15 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
RND
REQUIRED:
Pre-Conditioning
Report Run Date:
Enterprise WSAT (REQUIRED) - Report Page
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
NOTE: SLC-A x & y axis are scaled to match SLC-B for comparison
Comparing Client IOPS
IOPS Test Run Date:
11/14/2011 12:39 AM
Report Run Date:
11/21/2011 04:12 PM
Client IOPS (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
VENDOR: ABC CO. DUT Preparation
!""""#""""#$$$$ %####&####&""""
Serial No. No. Serial
Purge
Firmware Rev Rev Firmware
%$"& !"#$
Capacity Capacity
256 GB MLC
NAND Type Type NAND
6 Gb/s SATA
Device I/F I/F Device Test Platform Platform Test
SSD MODEL NO: MLC-A 256 GB
Security Erase Pre-Conditioning
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB Full IOPS Loop
Test Run Date:
11/17/2011 10:54 AM
PTS-C 1.0 6 of 6
TEST SPONSOR
(')(! (')(%
*('+(! *('+(%
Test Loop Parameters REQUIRED: Data Pattern RND
Steady State YES Convergence Rounds 1-5
Tester's Choice: OIO/Thread (QD) Thread Count (TC)
PC AR AR AMOUNT AR Segments
("'("! (#'(#%
+('*(! +('*(%
)('(! )('(%
11/21/2011 04:47 PM
8 1
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
100%
VENDOR: ABC CO. Purge
"&'(
Capacity
160 GB MLC
NAND Type
16 GiB
Device I/F
2048
Test Platform
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Serial No. Firmware Rev
3 Gb/s SATA
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
6 of 6
Convergence
YES
Rounds
9-13
RND
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
16 GiB
Thread Count (TC)
32
AR Segments
2048
Tester's Choice:
Full IOPS Loop
PTS-C 1.0
IOPS - ALL RW Mix & BS – 3D Columns
&""'"! $##'#%
")$""!
*)+*!
,*)-*!
*")*"!
-*),*!
+*)*!
Steady State
RND IOPS
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Client IOPS - ALL RW Mix & BS – 3D Columns "'&""! #'$##%
Report Run Date:
CLIENT IOPS (REQUIRED) - Report Page Rev. Page
$"")"!
30,000 70,000
MLC-A
MLC-B
0.5KiB 100% R
29,861
64,918
4KiB 100% R
29,876
38,087
4KiB 65:35 R/W
3,779
423
4KiB 100% W
3,147
152
128KiB 65:35
378
167
70,000 60,000
50,000 20,000
50,000
40,000 15,000
40,000
()*+$
()*+$
60,000 25,000
30,000 10,000 20,000 100/0
5,000 10,000
35/65
0 0.5
65/35 4
8
16
32
64
,-./0$+&12$34&,5$
MLC-A 256 GB
January 2012
128
0/100 1024
!"#$%&'$
NOTE:
MLC-A y-axis is scaled to match MLC-B for comparison
30,000 20,000 100/0 10,000
65/35
0 0.5
35/65 4
8
16
32
64
!"#$%&'$
Key Points: •
0/100 128
,-./0$+&12$34&,5$
1024
•
MLC-B 160 GB
54
•
MLC-B: • • MLC-A • •
Higher small block Reads Lower small block Writes. More balanced Block Size optimization Generally higher W performance
Writes have a disproportionately strong influence in any R/W mix performance (note 65/35 R/W comparisons)
Comparing Client IOPS
IOPS
Test Run Date:
11/14/2011 12:39 AM
Report Run Date:
Test Run Date:
11/21/2011 04:12 PM
11/17/2011 10:54 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%####&####&""""
Firmware Rev
!"#$
Capacity
256 GB
SSD MODEL NO: MLC-A 256 GB
DUT Preparation Purge
MLC
Device I/F
6 Gb/s SATA
Workload Independent
PTS-C 1.0 4 of 6
Full IOPS Loop
Steady State
Serial No.
%!!!!&''''&((((
Convergence
YES
Firmware Rev
!"#$
RND
Rounds
1-5
Capacity
160 GB
PC AR
100%
NAND Type
MLC
OIO/Thread (QD)
8
AR AMOUNT
16 GiB
Device I/F
3 Gb/s SATA
Thread Count (TC)
1
AR Segments
2048
Test Platform
REQUIRED: Tester's Choice:
VENDOR: ABC CO. DUT Preparation
5/95
65/35
50/50
35/65
4 of 6
TEST SPONSOR Convergence
YES
RND
Rounds
9-13
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
16 GiB
Thread Count (TC)
32
AR Segments
2048
Data Pattern
2X SEQ/128KiB Full IOPS Loop
Steady State
REQUIRED:
Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
PTS-C 1.0
Test Loop Parameters
Security Erase
Workload Independent
Rev. Page
Tester's Choice:
CLIENT IOPS - ALL RW Mix & BS – Tabular Data
Read / Write Mix % 0/100
SSD MODEL NO: MLC-B 160 GB
Purge
Client IOPS - ALL RW Mix & BS – Tabular Data Block Size (KiB)
11/21/2011 04:47 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
Rev. Page
Test Loop Parameters
Security Erase Pre-Conditioning
NAND Type
Test Platform
VENDOR: ABC CO.
Report Run Date:
CLIENT IOPS (REQUIRED) - Report Page
Client IOPS (REQUIRED) - Report Page
95/5
100/0
Read / Write Mix %
Block Size (KiB)
0/100
5/95
35/65
50/50
65/35
95/5
100/0
0.5
1,122.3
1,162.2
1,654.6
1,965.6
2,717.7
11,970.0
29,860.1
0.5
160.9
182.5
234.4
307.6
436.3
3,046.3
64,918.3
4
3,147.0
2,896.6
3,044.4
3,454.4
3,779.3
13,005.8
29,876.3
4
151.9
178.6
230.6
293.4
425.5
3,081.5
38,087.5
8
1,584.9
1,589.7
2,055.0
2,238.9
2,898.1
11,568.2
21,723.1
8
151.3
179.5
220.1
284.3
402.6
2,510.8
22,978.6
16
765.8
786.3
1,028.1
1,272.6
1,604.9
6,208.3
12,482.5
16
146.8
194.9
218.1
281.4
400.3
2,483.1
12,195.5
32
392.7
401.0
525.8
652.7
963.8
4,129.6
7,011.6
32
225.7
249.4
329.3
435.5
606.1
3,032.2
3,258.2
64
196.4
205.9
291.3
352.3
565.4
2,372.7
3,791.5
64
119.9
126.3
182.7
228.5
318.6
1,526.2
2,993.1
128
92.5
97.1
139.9
185.4
377.9
1,410.2
2,015.3
128
61.1
64.1
92.8
119.3
167.4
370.3
2,037.7
1024
16.4
16.5
23.3
27.3
90.8
191.4
266.7
1024
8.4
8.7
13.1
17.7
24.6
164.9
252.3
MLC-A 256 GB
January 2012
MLC-B 160 GB
55
Comparing Enterprise IOPS
IOPS Test Run Date:
11/02/2011 02:56 PM
Report Run Date:
11/14/2011 08:43 AM
Test Run Date:
Enterprise IOPS (REQUIRED) - Report Page
Device Under Test (DUT)
VENDOR: ABC CO. DUT Preparation
%%%%&''''&""""
Serial No.
Purge
Firmware Rev
!"#$
Capacity
100 GB
NAND Type
SLC 6 Gb/s SAS
Device I/F Test Platform
SSD MODEL NO: SLC-A 100 GB
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Page
6 of 6
Steady State
Firmware Rev
$%&!
1-5
Capacity
100 GB
PC AR
100%
NAND Type
SLC
OIO/Thread (QD)
16
AR AMOUNT
100%
Device I/F
Thread Count (TC)
2
AR Segments
N/A
Test Platform
,*(-*)
*'(*')
-*(,*)))
+*(*)
11/14/2011 04:17 PM
SSD MODEL NO: SLC-B 100 GB
DUT Preparation Purge
6 Gb/s SATA
Test Loop Parameters
Security Erase
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
RND
Tester's Choice:
Full IOPS Loop
6 of 6
Convergence
Data Pattern
2X SEQ/128KiB
PTS-E 1.0
YES
Rounds
2-6
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
%''(')
&'!&&(
100,000
100,000
90,000
90,000
80,000
80,000
70,000
70,000
60,000
60,000
50,000 40,000
)'%)(
*)'+)(
)&')&(
+)'*)(
%)')(
!&&'&(
SLC-A
SLC-B
0.5KiB 100% R
95,924
43,368
4KiB 100% R
93,707
46,365
4KiB 65:35 R/W
28,019
41,460
4KiB 100% W
16,563
19,561
128KiB 65:35
1,709
1,389
NOTE:
50,000
SLC-B y-axis is scaled to match SLC-A for comparison
40,000
30,000
Steady State
RND IOPS
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR
Enterprise IOPS - ALL RW Mix & BS – 3D Columns
()*+$
()*+$
*(+*)
VENDOR: ABC CO.
!!!!"!!!!"####
Serial No.
YES
Rounds
Enterprise IOPS - ALL RW Mix & BS – 3D Columns
'(%'')
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Convergence RND
Tester's Choice:
Full IOPS Loop
PTS-E 1.0
Device Under Test (DUT)
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev.
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
11/09/2011 11:35 AM
Enterprise IOPS (REQUIRED)
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
30,000 100/0
20,000
65/35
10,000 0 0.5
35/65 4
8
100/0
20,000
65/35
10,000
!"#$%&'$
0 0.5
16
32
64
,-./0$+&12$34&,5$
0/100 128
35/65 4
8
16
32
1024
64
!"#$%&'$
Key Points: •
0/100 128
1024
,-./0$+&12$34&,5$
•
SLC-A 100 GB
January 2012
SLC-B 100 GB
56
•
SLC-A: • • SLC-B • •
Higher small block Reads Lower small block Writes. More balanced Block Size optimization Generally higher W performance
Writes have a disproportionately strong influence in any R/W mix performance (note 65/35 R/W comparisons)
Comparing Enterprise IOPS
IOPS
Test Run Date:
11/02/2011 02:56 PM
Report Run Date:
11/14/2011 08:43 AM
Test Run Date:
Enterprise IOPS (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%%%%&''''&""""
Firmware Rev
!"#$
Capacity
100 GB
NAND Type
SLC
VENDOR: ABC CO. DUT Preparation Purge Workload Independent
PTS-E 1.0 4 of 6
TEST SPONSOR
Data Pattern
Full IOPS Loop
Steady State
RND
Tester's Choice:
Serial No.
!!!!"!!!!"####
Convergence
YES
Firmware Rev
$%&!
Rounds
1-5
Capacity
100 GB
PC AR
100%
NAND Type
SLC
OIO/Thread (QD)
16
AR AMOUNT
100%
Device I/F
Thread Count (TC)
2
AR Segments
N/A
Test Platform
5/95
35/65
50/50
65/35
11/14/2011 04:17 PM
SSD MODEL NO: SLC-B 100 GB
DUT Preparation Purge
4 of 6
TEST SPONSOR Convergence
Data Pattern
2X SEQ/128KiB Full IOPS Loop
Steady State
REQUIRED:
Pre-Conditioning
6 Gb/s SATA
PTS-E 1.0
Test Loop Parameters
Security Erase
Workload Independent
Rev. Page
RND
Tester's Choice:
YES
Rounds
2-6
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
Enterprise IOPS - ALL RW Mix & BS – Tabular Data
Read / Write Mix % 0/100
VENDOR: ABC CO.
RTP 2.0 CTS 6.5 Workload Dep.
Enterprise IOPS - ALL RW Mix & BS – Tabular Data Block Size (KiB)
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
REQUIRED:
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
Rev. Page
Test Loop Parameters
Format Unit Pre-Conditioning
6 Gb/s SAS
Device I/F Test Platform
SSD MODEL NO: SLC-A 100 GB
11/09/2011 11:35 AM
Enterprise IOPS (REQUIRED) - Report Page
95/5
100/0
Read / Write Mix %
Block Size (KiB)
0/100
5/95
35/65
50/50
65/35
95/5
100/0
0.5
15,887.4
16,634.7
20,678.6
24,402.8
29,386.2
72,428.4
95,924.3
0.5
13,255.7
13,581.2
15,581.3
18,393.7
21,115.5
40,004.8
43,368.3
4
16,563.0
17,032.2
20,234.2
23,705.2
28,018.6
63,447.7
93,707.0
4
19,560.6
20,238.4
23,886.3
26,641.2
29,827.0
41,460.1
46,365.3
8
9,559.8
9,998.4
12,547.1
14,636.6
17,199.1
37,872.9
50,301.2
8
10,630.3
11,033.1
13,806.4
15,780.9
18,244.6
27,803.9
32,259.3
16
4,842.2
5,032.3
6,802.5
8,132.1
9,655.8
22,462.2
31,072.8
16
5,620.5
5,824.9
7,451.9
8,651.6
10,174.7
15,772.3
18,309.1
32
2,413.3
2,535.4
3,478.4
4,241.3
5,061.7
12,174.7
15,994.2
32
2,872.2
3,002.7
3,862.5
4,513.2
5,377.6
8,337.0
9,472.3
64
1,219.2
1,275.7
1,728.4
2,126.1
2,726.3
6,284.6
8,094.9
64
1,461.8
1,515.4
1,962.1
2,293.9
2,751.4
4,304.9
4,829.3
128
612.7
632.5
859.1
1,061.4
1,709.4
3,205.7
4,060.8
128
735.7
763.7
987.4
1,148.8
1,389.0
2,187.9
2,439.4
1024
74.8
78.0
103.6
126.7
202.7
398.8
514.6
1024
92.5
95.9
124.1
144.6
173.1
277.2
307.6
SLC-A 100 GB
January 2012
SLC-B 100 GB
57
Comparing Client Throughput
TP Test Run Date:
11/13/2011 10:24 AM
Report Run Date:
11/21/2011 04:03 PM
Test Run Date:
Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB
NAND Type
MLC 6 Gb/s SATA
Device I/F
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge Workload Independent
SEQ 1024KiB
Page
5 of 5
Serial No.
YES
Firmware Rev
"&'(
Rounds
1-5
Capacity
160 GB
PC AR
100%
NAND Type
MLC
Tester's Choice: OIO/Thread (QD)
32
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
32
AR Segments
2048
Test Platform
VENDOR: ABC CO.
3 Gb/s SATA
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Convergence RND
Report Run Date:
11/21/2011 04:50 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
PTS-C 1.0
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
Rev.
11/15/2011 12:46 AM
Client Throughput Test (REQUIRED) - Report Page
Client Throughput Test (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Purge
Test Loop Parameters
Security Erase
Workload Independent
Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
SEQ 1024KiB
PTS-C 1.0 5 of 5
SEQ 1024KIB TP
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR Convergence RND
Tester's Choice:
YES
Rounds
1-5
PC AR
100%
OIO/Thread (QD)
2
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Client Throughput - ALL RW Mix & BS - 2D Plot
Client Throughput - ALL RW Mix & BS - 2D Plot
MLC-A
MLC-B
100% R
417 MB/s
264 MB/s
100% W
267 MB/s
99 MB/s
,""!
+""!
Steady State
+""!
*""!
)""!
(""!
&'()*+',*-"./0123"
'()*+,(-+.$/01234$
%&#$
!"#$
'""!
*""!
$""!
"!
"!
&"","! 526$078$
MLC-A 256 GB
January 2012
#$%"
•
&""!
&""!
",&""!
Key Points:
)""!
•
!!"
"-$""!
$""-"! 415"/67"
MLC-B 160 GB
58
MLC-A: • • MLC-B • •
DUT Interface 6Gb/s SATA Higher large block SEQ Reads & Writes DUT Interface 3 Gb/s SATA Slower large block SEQ Reads & Writes
Comparing Enterprise Throughput
TP Test Run Date:
12/04/2011 08:21 AM
Report Run Date:
12/04/2011 10:03 AM
Enterprise Throughput Test (REQUIRED) - Report Page SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%$&'
Capacity
100 GB SLC
NAND Type
6 Gb/s SAS
Device I/F
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge Workload Independent
SEQ 1024KiB
Page
5 of 5
Serial No.
YES
Firmware Rev
$%&!
Rounds
4-8
Capacity
100 GB
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Device I/F
Thread Count (TC)
2
AR Segments
N/A
Test Platform
Tester's Choice:
VENDOR: ABC CO.
Report Run Date:
SLC
NAND Type
6 Gb/s SATA
SSD MODEL NO: SLC-B 100 GB
DUT Preparation
!!!!"!!!!"####
Convergence RND
12/04/2011 10:43 AM
12/04/2011 11:48 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
PTS-E 1.0
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
Test Run Date:
SEQ 1024 KIB TP
Enterprise Throughput Test (REQUIRED) - Report Page Rev.
Purge Workload Independent
SEQ 1024KiB
PTS-E 1.0 5 of 5
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
RTP 2.0 CTS 6.5 Workload Dep.
Enterprise 1024KiB Throughput - ALL RW Mix & BS - 2D Plot
Test Loop Parameters
Security Erase Pre-Conditioning
Rev. Page
TEST SPONSOR Convergence
YES
Rounds
4-8
RND
PC AR
100%
OIO/Thread (QD)
16
AR AMOUNT
100%
Thread Count (TC)
2
AR Segments
N/A
Tester's Choice:
Enterprise 1024KiB Throughput - ALL RW Mix & BS - 2D Plot
-##(
Steady State SLC-A
SLC-B
100% R
514 MB/s
437 MB/s
100% W
157 MB/s
205 MB/s
,&&' "!%$
,##(
+&&'
+##(
*&&'
()*+,-).,/$012345$
&'()*+',*-$./0123$
%&'$
*##(
)##(
!"#$
!##(
• !"#$
(&&'
•
!&&'
#( #.!##(
!##.#( 415$/67$
SLC-A 100 GB
January 2012
Key Points:
)&&'
&' &-!&&'
!&&-&' 637$189$
SLC-B 100 GB
59
SLC-A: • • SLC-B • •
Higher large block SEQ Reads & Writes DUT Interface 6 Gb/s SAS Slower large block SEQ Reads & Writes DUT Interface 6 Gb/s SATA
Comparing Client Latency AVE
LAT
11/11/2011
Test Run Date:
09:53 AM
11/15/2011 03:34 PM
Report Run Date:
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%$"&
Capacity
256 GB
Device I/F Test Platform
MLC 6 Gb/s SATA
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
!""""#""""#$$$$
Firmware Rev
NAND Type
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB Full Latency Loop
Rev.
PTS-C 1.0
Page
5 of 6
TEST SPONSOR
Report Run Date:
11/22/2011 10:40 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
Test Loop Parameters
Security Erase Pre-Conditioning
11/16/2011 01:01 AM
Latency RND 4KiB AVE
Client Latency (REQUIRED) - Report Page
Client Latency (REQUIRED) - Report Page
Steady State
Serial No.
REQUIRED: Data Pattern
RND
Tester's Choice: OIO/Thread (QD)
PC AR
100%
1
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
YES
Firmware Rev
"&'(
Rounds
4-8
Capacity
160 GB
Client - AVE Latency vs BS and R/W Mix - 3D Plot
MLC 6 Gb/s SATA
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Convergence
NAND Type
VENDOR: ABC CO. Purge
Test Loop Parameters
Security Erase
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Data Pattern
2X SEQ/128KiB Full LAT Loop
PTS-C 1.0 5 of 6
Steady State
REQUIRED:
Pre-Conditioning
Rev. Page
TEST SPONSOR Convergence
YES
Rounds
12-16
RND
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
16 GiB
Thread Count (TC)
1
AR Segments
2048
Tester's Choice:
Client - AVE Latency vs BS and R/W Mix - 3D Plot
Steady State MLC-A
MLC-B
100% R AVE
0.20 mSec
0.19 mSec
65/35 R/W AVE
0.33 mSec
0.65 mSec
100% W AVE
0.35 mSec
1.00 mSec
#(*!$
#(*!$
#()!$
!(,'$
#()!$ #(!!$
#()!$
2034$5367$
!(+!$
3145$6478$
#(!!$
!(&*$
!(%!$
!('&$
!(*&$
!(%!$
!()!$
!(!!$
!()*$
%&"'&$ #!!"!$
*$ +$
!"#!!$
!(#!$
!(#-$
!())$
!(!!$
-".$/01$
!(&$
January 2012
%&"'&$
."/$012$ #!!"!$
*$ +$
9:;1?5$6@198$
89:;4$5?087$
MLC-A 256 GB
•
!(&,$
!('*$
!()!$
!()!$
!(&$
!(%&$
!(*!$
!"#!!$ !()!$
Key Points:
!(+!$
!(*'$
!(''$
!(*!$
!(,,$
#(!!$
MLC-B 160 GB
60
•
MLC-A: • • MLC-B • •
More stable & faster RND 4KiB AVE Latency Higher RND 0.5KiB 100% W AVE Latency Slower overall RND 4KiB AVE Latency Higher RND 8KiB 100% W AVE Latency
Comparing Client Latency MAX
LAT
11/11/2011
Test Run Date:
09:53 AM
11/15/2011 03:34 PM
Report Run Date:
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
!"#$
Capacity
256 GB
NAND Type Device I/F
MLC 6 Gb/s SATA
SSD MODEL NO: MLC-A 256 GB
DUT Preparation
%####&####&""""
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge
Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
2X SEQ/128KiB
Rev.
PTS-C 1.0
Page
6 of 6
Steady State
Serial No.
REQUIRED: Data Pattern
RND
Tester's Choice: OIO/Thread (QD)
PC AR
100%
1
AR AMOUNT
16 GiB
Device I/F
Thread Count (TC)
1
AR Segments
2048
Test Platform
Full Latency Loop
Report Run Date:
YES
Firmware Rev
"&'(
Rounds
4-8
Capacity
160 GB
NAND Type
VENDOR: ABC CO.
MLC 6 Gb/s SATA
SSD MODEL NO: MLC-B 160 GB
DUT Preparation
!""""#$$$$#%%%%
Convergence
Purge Workload Independent
Data Pattern
)&!(!!$
)&!(!$
)!!(!!$
3145$6478$
'!!(!$
3145$6478$
)!!(!$
Convergence
YES
Rounds
12-16
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
16 GiB
1
AR Segments
2048
&)(,%$
)+'(-%$
)++(),$
&!(!)$
%&"'&$
#(!*$
!(!$
!"#!!$
&'(')$
&!(!$ #(&-$
'-(*+$
!(&$
#!!"!$
*$
."/$012$
#&!(!!$
#(!*$
#(*'$
•
%&"'&$ *(--$
!(!!$ !(&$
#!!"!$
*$ +$
9:;1?5$6@198$
+$
January 2012
1.59 mSec
1.43 mSec
65/35 R/W MAX
50.02 mSec
283.21 mSec
100% W MAX
51.05 mSec
288.29 mSec
Key Points:
#!!(!!$
9:;1?5$6@198$
MLC-A 256 GB
MLC-B
)%,(-*$
&!(!!$
&!()!$
MLC-A
100% R MAX
)+'(,!$
Steady State
)+'()#$
!"#!!$ #!!(!$
Latency RND 4KiB MAX
Client - MAX Latency vs BS and R/W Mix - 3D Plot
'!!(!!$
#&!(!$
6 of 6
Thread Count (TC)
)+,()%$
&#(!&$
RND
Tester's Choice:
Full LAT Loop
PTS-C 1.0
Steady State
REQUIRED:
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Client - MAX Latency vs BS and R/W Mix - 3D Plot
&)(#*$
11/22/2011 10:40 AM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
11/16/2011 01:01 AM
Client Latency (REQUIRED) - Report Page
Client Latency (REQUIRED) - Report Page
MLC-B 160 GB
61
."/$012$
•
MLC-A: • • MLC-B • •
More stable & faster RND 4KiB MAX Latency Smaller RND 0.5KiB & 8KiB MAX Latency Slower overall RND 4KiB MAX Latency Higher RND 0.5KiB & 8KiB MAX Latency
Comparing Enterprise Latency AVE
LAT Test Run Date:
11/10/2011 05:07 AM
Report Run Date:
11/14/2011 08:47 AM
Test Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT) Serial No.
%%%%&''''&""""
Firmware Rev
!"#$
Capacity NAND Type Device I/F Test Platform
VENDOR: ABC CO.
SSD MODEL NO: SLC-A 100 GB
DUT Preparation Purge
SLC
Workload Independent
6 Gb/s SATA
RTP 2.0 CTS 6.5 Workload Dep.
Data Pattern
2X SEQ/128KiB
5 of 6
Convergence
YES
Rounds
3-7
RND
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
100%
Thread Count (TC)
1
AR Segments
N/A
Tester's Choice:
Full LAT Loop
Page
Steady State
REQUIRED:
Pre-Conditioning
100 GB
PTS-E 1.0
TEST SPONSOR
Test Loop Parameters
Format Unit
Rev.
Device Under Test (DUT) Serial No.
$%&!
Capacity
100 GB
NAND Type
SLC
Device I/F Test Platform
6 Gb/s SATA
SSD MODEL NO: SLC-B 100 GB
DUT Preparation Purge Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
!(#&$
!(!+$
!(#*$
!(#%$
!(#-$
!(#!$ !"#!!$
!(!&$
%&"'&$
!(!!$ !(&$
#!!"!$
*$
."/$012$
SLC-A 100 GB
January 2012
YES
Rounds
5-9
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
100%
Thread Count (TC)
1
AR Segments
N/A
!(#&$
!()&$
!(#,$ !(#%$
!(#%$
!(!+$
Steady State SLC-A
SLC-B
100% R AVE
0.16 mSec
0.18 mSec
65/35 R/W AVE
0.14 mSec
0.16 mSec
100% W AVE
0.09 mSec
0.08 mSec
!()#$ !(#)$
Key Points:
!(#+$
•
!(#!$ !"#!!$
!(!&$ %&"'&$
!(!!$ !(&$
#!!"!$
*$ +$
+$
9:;1?5$6@198$
Latency RND 4KiB AVE
!(#*$
!()!$
!(#'$
!(!,$
5 of 6
Convergence RND
Tester's Choice:
Full LAT Loop
PTS-E
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Enterprise AVE Latency vs BS and R/W Mix - 3D Plot
2034$5367$
!(#'$
12/04/2011 03:22 PM
Test Loop Parameters
Security Erase Pre-Conditioning
!()&$
!()!$
3145$6478$
VENDOR: ABC CO.
!!!!"!!!!"####
Firmware Rev
!()&$ !(#-$
Report Run Date:
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
Enterprise AVE Latency vs BS and R/W Mix - 3D Plot
!(#&$
12/04/2011 12:14 PM
Enterprise Latency (REQUIRED) - Report Page
Enterprise Latency (REQUIRED) - Report Page
89:;4$5?087$
SLC-B 100 GB
62
-".$/01$
SLC-A & SLC-B: • Comparable AVE Latency speeds
Comparing Enterprise Latency MAX
LAT Test Run Date:
11/10/2011 05:07 AM
Report Run Date:
11/14/2011 08:47 AM
Enterprise Latency (REQUIRED) - Report Page
Device Under Test (DUT) Serial No.
%$&'
Capacity
100 GB
NAND Type
SLC
Device I/F
6 Gb/s SATA
SSD MODEL NO: SLC-A 100 GB
DUT Preparation
!!!!"####"$$$$
Firmware Rev
Test Platform
VENDOR: ABC CO. Purge Workload Independent
RTP 2.0 CTS 6.5 Workload Dep.
Page
6 of 6
12/04/2011 12:14 PM
Report Run Date:
Serial No.
YES
Firmware Rev
Rounds
$%&!
3-7
Capacity
100 GB
PC AR
100%
NAND Type
OIO/Thread (QD)
1
AR AMOUNT
100%
Device I/F
Thread Count (TC)
1
AR Segments
N/A
Test Platform
VENDOR: ABC CO.
SLC 6 Gb/s SATA
SSD MODEL NO: SLC-B 100 GB
DUT Preparation
!!!!"!!!!"####
Convergence RND
Purge Workload Independent
Data Pattern
*&!(!!$
*&!(!!$
*!!(!!$
*!!(!!$
'&!(!!$
'&!(!!$
'!!(!!$
'-(*%$
#&!(!!$
+!(*'$
*%(,+$
#!!(!!$ &!(!!$
3145$6478$
3145$6478$
&!!(!!$
')(%'$
!"#!!$
'-('+$
%()%$
!(!!$
#)(--$
!(&$
#!!"!$
*$
**'(#)$
PC AR
100%
OIO/Thread (QD)
1
AR AMOUNT
100%
1
AR Segments
N/A
."/$012$
**'(,*$
SLC-A 100 GB
January 2012
SLC-A
SLC-B
6.26 mSec
0.39 mSec
65/35 R/W MAX
46.78 mSec
442.49 mSec
100% W MAX
32.63 mSec
443.94 mSec
100% R MAX
***(%'$
**)(*,$
**'(-)$
)&!(!!$
Key Points:
)!!(!!$ #&!(!!$
!"#!!$
•
!('*$ !(',$
&!(!!$
&(!!$
!(!!$ !(&$
%&"'&$
#!!"!$
*$
+$
9:;1?5$6@198$
5-9
Steady State
'!!(!!$
#!!(!!$ %&"'&$
&(+!$
Latency RND 4KiB MAX
Enterprise MAX Latency vs BS and R/W Mix - 3D Plot
&!!(!!$
&%(+,$
YES
Rounds
Thread Count (TC)
*&)(!*$
)!!(!!$
6 of 6
Convergence RND
Tester's Choice:
Full LAT Loop
PTS-E
Steady State
REQUIRED:
2X SEQ/128KiB
Rev. Page
TEST SPONSOR
Test Loop Parameters
Security Erase Pre-Conditioning
RTP 2.0 CTS 6.5 Workload Dep.
Enterprise MAX Latency vs BS and R/W Mix - 3D Plot
)&!(!!$
12/04/2011 03:22 PM
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS) Device Under Test (DUT)
Tester's Choice:
Full LAT Loop
PTS-E 1.0
Steady State
REQUIRED: Data Pattern
2X SEQ/128KiB
Rev.
TEST SPONSOR
Test Loop Parameters
Format Unit Pre-Conditioning
Test Run Date:
Enterprise Latency (REQUIRED) - Report Page
SNIA SSS TWG: Solid State Storage Performance Test Specification (PTS)
+$
9:;1?5$6@198$
SLC-B 100 GB
63
."/$012$
•
SLC-A: • • SLC-B • •
More stable & faster RND 4KiB MAX Latency Smaller RND 0.5KiB & 8KiB MAX Latency Slower overall RND 4KiB MAX Latency Much Higher RND 65/35 and 100% W MAX Latency
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
64
Basic Test Procedures Verified Test Environment. Do not bottleneck SSD performance. •
Hardware Bottlenecks - Anywhere within the data/control paths (HBA, data bus lanes, RAM CPU)
•
Software Influences - OS background tasks, application software, APIs and device drivers
•
Test Software Tools - Stimulus generators and Measurement tools can introduce performance overhead
Normalized Test Platform. Use the SAME platform to normalize the test environment across measurements. Calibration. Periodically calibrate using the same test stimulus / workload on a known device. Test Plan. A good test plan enumerates test objectives, test methodology and selection of tests. This includes: • • • January 2012
establishing the relevance of the test to the test objectives defining the test baseline, and prescribing the test procedures, number samples, test runs and statistical analysis employed. 65
SSD Specific Testing Purge Any SSD test should begin with a device Purge.
Preconditioning Precisely define the preconditioning regime.
Steady State Precisely define the preconditioning workload and steady state determination criteria.
Demand Intensity Map the target SSD on the test platform to determine the optimal OIO settings for the given test.
Block Size Sequencing Avoid Block Size Sequence / Cross Stimulus effects on the SSD performance.
Test Stimulus Workload Ensure the test workload is relevant to the characteristics of the targeted user workload. January 2012
66
SSD Test Best Practices Use Standardized Methodologies Benefit from industry, scientific and academic research in SSD tests and methodologies.
Reference Test Platform (RTP) Normalize the test environment and ensure repeatable and reproducible test results.
Standardized Tests Use of the RTP / PTS allows for easy comparison of performance between different SSD devices.
Standardized Reporting Ensures test standards compliance and disclosure of required test set-up conditions.
SNIA SSS PTS Uniform prescriptions for SSD testing allows valid comparison and understanding of SSD device performance.
January 2012
67
Download this deck at www.snia.org/forums/sssi/pts
1. Principles of NAND Flash SSD Performance 2. How IOs Traverse the S/W H/W Stack 3. PTS Client & Enterprise Test Specifications 4. Summary SSD Comparisons – www.snia.org/forums/sssi/pts 5. Using PTS Reports to Understand SSD Behavior 6. Using PTS Reports to Compare SSD Behavior 7. SSD Test Best Practices 8. Conclusion
January 2012
68
Conclusion NAND storage technology increases performance by orders of magnitude. SSD performance characteristics are considerably different from those of conventional spinning drives. NAND based SSDs are very “write history” sensitive requiring precise preconditioning & steady state SSD Test Environments can Adversely affect test results. New SSD Testing is Required for accurate, objective comparison. Use SNIA PTS & RTP for SSD Test & Measurement. End Users’ workloads are infinitely diverse. Knowing the attributes of the particular IO profile, end users can select those test results which best represents their workloads and disregard those less relevant. January 2012
69
SSD Client & Enterprise
PTS Comparison Data
Available at
www.snia.org/forums/sssi/pts
January 2012
70
