Disk-based storage in computers Memory/storage hierarchy Combining many technologies to balance costs/ benefits Recall the memory hierarchy and virtual memory lectures

198:211 Computer Architecture

Topics: System I/O Buses

Memory/storage hierarchies

An Example Memory Hierarchy L0:! Smaller,! faster,! and ! costlier! (per byte)! storage ! devices!

registers! CPU registers hold words retrieved from L1 cache.!

L1:!

on-chip L1! cache (SRAM)! L1 cache holds cache lines retrieved from the L2 cache memory.!

L2:!

off-chip L2! cache (SRAM)! L2 cache holds cache lines retrieved from main memory.!

L3:! Larger, ! slower, ! and ! cheaper ! (per byte)! storage! devices!

Performance

Capacity

Balancing performance with cost Small memories are fast but expensive Large memories are slow but cheap Exploit locality to get the best of both worlds locality = re-use/nearness of accesses allows most accesses to use small, fast memory

main memory! (DRAM)! Main memory holds disk ! blocks retrieved from local ! disks.!

L4:!

local secondary storage! (local disks)! Local disks hold files retrieved from disks on remote network servers.!

L5:!

remote secondary storage! (tapes, distributed file systems, Web servers)!

1

Disk-based storage in computers Memory/storage hierarchy Combining many technologies to balance costs/ benefits Recall the memory hierarchy and virtual memory lectures

What’s Inside A Disk Drive? Arm

Platters

Actuator

Persistence Storing data for lengthy periods of time

Electronics

DRAM/SRAM is “volatile”: contents lost if power lost Disks are “non-volatile”: contents survive power outages Disk are blocks access (read/write blocks) Conventional magnetic disks Newer: Solid state disks

Disk Electronics

Spindle

SCSI connector Image courtesy of Seagate Technology

Disk “Geometry” Just like a small computer – processor, memory, network interface Connect to disk

Disks contain platters, each with two surfaces Each surface organized in concentric rings called tracks Each track consists of sectors separated by gaps tracks! surface! track k!

Control processor

gaps!

Cache memory Control ASIC Connect to motor

spindle!

sectors!

2

Disk Geometry (Muliple-Platter View) Aligned tracks form a cylinder

Disk Structure Arm

Read/Write Head

cylinder k Upper Surface Platter

surface 0! platter 0!

Lower Surface

surface 1! surface 2! platter 1! surface 3! Cylinder

surface 4! platter 2! surface 5!

spindle!

Track Sector Actuator

Disk Operation (Single-Platter View) The disk surface ! spins at a fixed! rotational rate!

Disk Operation (Multi-Platter View)

The read/write head! is attached to the end! of the arm and flies over! the disk surface on! a thin cushion of air!

read/write heads ! move in unison! from cylinder to cylinder!

spindle!

spindle!

arm!

spindle!

spindle!

spindle!

By moving radially, the arm can position the read/write head over any track!

3

Disk Structure - top view of single platter

Disk Access

Surface organized into tracks Tracks divided into sectors

Head in position above a track

Disk Access

Rotation is counter-clockwise

Disk Access – Read

About to read blue sector

4

Disk Access – Read

After BLUE read

After BLUE read

After reading blue sector

Disk Access – Seek

After BLUE read

Disk Access – Read

Seek for RED

Seek to red’s track

Red request scheduled next

Disk Access – Rotational Latency

After BLUE read

Seek for RED

Rotational latency

Wait for red sector to rotate around

5

Disk Access – Read

After BLUE read

Seek for RED

Disk Access – Service Time Components

Rotational latency

After RED read

After BLUE read

Complete read of red

Disk Access Time

Average time to access a specific sector approximated by: Taccess = Tavg seek + Tavg rotation + Tavg transfer Seek time (Tavg seek) Time to position heads over cylinder containing target sector Typical Tavg seek = 3-5 ms Rotational latency (Tavg rotation) Time waiting for first bit of target sector to pass under r/w head Tavg rotation = 1/2 x 1/RPMs x 60 sec/1 min e.g., 3ms for 10,000 RPM disk

Transfer time (Tavg transfer) Time to read the bits in the target sector Tavg transfer = 1/RPM x 1/(avg # sectors/track) x 60 secs/1 min

Seek for RED

Rotational latency

After RED read

Seek Rotational Latency Data Transfer

Disk Access Time Example

Given:

Rotational rate = 7,200 RPM Average seek time = 5 ms Avg # sectors/track = 1000

Derived average time to access random sector: Tavg rotation = 1/2 x (60 secs/7200 RPM) x 1000 ms/sec = 4 ms Tavg transfer = 60/7200 RPM x 1/1000 secs/track x 1000 ms/sec = 0.008 ms Taccess = 5 ms + 4 ms + 0.008 ms = 9.008 ms Time to read sector: 0.008 ms

Important points: Access time dominated by seek time and rotational latency First bit in a sector is the most expensive, the rest are free SRAM access time is about 4 ns/doubleword, DRAM about 60 ns ~100,000 times longer to access a word on disk than in DRAM

e.g., 0.006ms for 10,000 RPM disk with 1,000 sectors/track given 512-byte sectors, ~85 MB/s data transfer rate

6

Disk Scheduling

Disk Scheduling

The operating system is responsible for using hardware efficiently — for the disk drives, this means having a fast access time and disk bandwidth. Access time has two major components Seek time is the time for the disk are to move the heads to the cylinder containing the desired sector. Rotational latency is the additional time waiting for the disk to rotate the desired sector to the disk head. Minimize seek time Seek time ≈ seek distance Disk bandwidth is the total number of bytes transferred, divided by the total time between the first request for service and the completion of the last transfer.

Several algorithms exist to schedule the servicing of disk I/O requests. We illustrate them with a request queue (0-199).

FCFS

98, 183, 37, 122, 14, 124, 65, 67 Head pointer 53

SSTF Illustration shows total head movement of 640 cylinders.!

Selects the request with the minimum seek time from the current head position. SSTF scheduling is a form of SJF scheduling; may cause starvation of some requests. Illustration shows total head movement of 236 cylinders.

7

SSTF (Cont.)

SCAN The disk arm starts at one end of the disk, and moves toward the other end, servicing requests until it gets to the other end of the disk, where the head movement is reversed and servicing continues. Sometimes called the elevator algorithm. Illustration shows total head movement of 208 cylinders.

SCAN (Cont.)

C-SCAN Provides a more uniform wait time than SCAN. The head moves from one end of the disk to the other. servicing requests as it goes. When it reaches the other end, however, it immediately returns to the beginning of the disk, without servicing any requests on the return trip. Treats the cylinders as a circular list that wraps around from the last cylinder to the first one.

8

C-SCAN (Cont.)

C-LOOK Version of C-SCAN Arm only goes as far as the last request in each direction, then reverses direction immediately, without first going all the way to the end of the disk.

C-LOOK (Cont.)

Selecting a Disk-Scheduling Algorithm SSTF is common and has a natural appeal SCAN and C-SCAN perform better for systems that place a heavy load on the disk. Performance depends on the number and types of requests. Requests for disk service can be influenced by the fileallocation method. The disk-scheduling algorithm should be written as a separate module of the operating system, allowing it to be replaced with a different algorithm if necessary. Either SSTF or LOOK is a reasonable choice for the default algorithm.

9

Solid state disks

Solid state disks An array of flash memory devices Emulates conventional hard disk drive or HDD No moving parts Consumes less power than HDD Small reads (< 4K) are 20x faster Average reads comparable to HDD reads Writes are still slow ½ x slower than HDD Capacity/cost (today) 0.15$/GB-HDD , 2-3$/GB-SSD

Solid State Drive The interface to the system looks like HDD Read, write and erase Memory consists of blocks Each block contains several pages Each page is 2K or 4K in size Unit of read/write are pages Need to erase before write!

I/O or input and output In addition to memory, data transfer needs to occur between CPU and Input output devices When reading from memory, a byte or several bytes can be transferred from memory to register using mov address, %eax or mov %eax, address! I/O devices also are sources or destinations for bytes of data I/O devices can be viewed just as memory I/O devices can be viewed as separate from memory

10

I/O programming There are two ways of addressing I/O devices Memory mapped I/O The address space is divided between memory and I/O devices Higher order addresses can refer to device Lower order addresses can refer to memory mov %eax, address will fetch data from I/O or memory based on the address E.g., memory range to from 0000 to BFFF I/O range from C000 to CFFF Device or memory selection based on address range Different devices can have different addresses in the I/O range

Memory mapped I/O Memory - I/O Bus

CPU

Main Memory

Disk

keyboard

Display

Network

Send or receive data to /from I/O device is a memory transfer instruction (mov) with the right address Main memory not selected when address is in I/O range Adv Uniformity of programming, same mov works for I/O and memory

Dis adv Memory address space is reduced

I/O mapped I/O Memory and I/O devices use distinct address spaces Isolated I/O Two separate instructions to address I/O devices A separate code or control signal based on the op code FFFF

will select memory or I/O IN for input MEM

0000

OUT for output 00FF

mov for memory access I/O

0000

Less flexible for programming

11

Interfacing with I/O Many devices, with varying speeds, complexity CPU/bus shared among all peripherals and memory CPU should be able to select a device and transfer data to the device Interpretation of data left to each device Unlike memory, device need to be ready before initiating transfer All of this handed by I/O module

I/O module Data

lines

Address

Control

Data transfer schemes data

To

device

status

I/O

logic

To

device

CPU selects device by means of address Data corresponds to instructions for device Each device has its own set of commands Status of device can be checked by reading status registers

There are two schemes Programmed data transfers CPU transfers data from I/O devices onto registers Useful for small data transfers Direct memory access or DMA Device or I/O module directly transfers data to memory Useful or large block transfers

12

Programmed I/O

Synchronous transfer

Programmed I/O can be further classified as Synchronous transfer Asynchronous transfer Interrupt driven transfer All of the above can be used to interface with different I/O devices Require special hardware features in the CPU

Simplest among three CPU and I/O speed match Transfer a byte, word, or double word Memory mapped mov %eax , 2 Address of device port is 2

I/O mapped mov $2, %edx out %eax, %edx

Similarly for Input device, Memory mapped: mov 3, %eax or I/O mapped mov $3, %edx in %edx, %eax

Asynchronous transfer

Interrupt driven I/O

I/O devices slower Instruct device to be ready Wait until device ready Device has status flag/register Busy waiting Waste of CPU resources

Processor need not wait for slow device Processor continues with other instructions Device interrupts processor when ready Interrupt Service Routine

Request device to get ready

READY

Yes

Issue data transfer command

No

CPU transfers word from device to register CPU writes word from register to memory

Request device

Fetch next instruction

Execute instruction

INT High

No

Yes

Call Interrupt Service routine

13

DMA or direct memory Bulk data transfers Direct device to memory transfer Memory bus is contention between CPU and DMA unit During DMA Either CPU is in hold state Or Cycle stealing CPU and DMA access in interleaved

System bus Request DMA device

Memory - I/O Bus Fetch next instruction

Main Memory

CPU

Execute instruction

INT High

Send R/W command

Starting address, #bytes

DMA interrupt

Several devices can be connected Single bus for all devices – cost sharing Added/removed without affecting others I/O devices can be connected to other devices following the same bus standard Disadvantages: Bus contention Speed of I/O devices determined by bus speed Bus speed determined by number of devices Slower devices impact others

keyboard

Display

Network

No

Yes

Characteristics of bus

Disk

A bus is a shared communication link Contains address bus, data bus Each bus is a set of wires Bus can transfer several bits between devices connected by bus Bus width determines the number of bits transferred in a cycle

Bus architecture Master issues command Bus Master

Data can go either way

Bus Slave

Any interaction consists of two steps 1. Issue command 2. transfer data

Master Initiates Issues command, starting address, #bytes

Slave Responds Sends or receives data as per command from master

14

Backplane bus

Computer buses Modern computers have several I/O devices Varying speeds A simple linear bus will not suffice Modern computers have hierarchical buses Bus is split into different segments CPU-Memory one bus CPU-I/O devices another bus CPU-cache – another bus

Two-bus systems

System bus- Memory and I/O

Single bus for memory and I/O Cheap Slow and bus becomes bottleneck

hierarchical-bus systems Processor-Memory bus

Bus Bridge

Backside cache

bus

L2 cache

Processor-Memory bus

Bus Bridge

I/O bus

I/O bus

Processor-memory traffic on one bus I/O devices connected by a bridge Bridge can connect to different kinds of buses Traffic is isolated I/O buses can provided expansion slots for devices

I/O bus

A single bus bridge connects to the processor-memory bus Other I/O buses connected to this bus bridge (tree) CPU-memory sees little contention Costly

15

Examples of buses

Examples of buses

ISA bus – Industry Standard bus Old technology 8 Mhz, < 1 byte transfer/cycle, bus B/W 5.3 MB/ sec (1 MB = 1048576 B) EISA bus – Extended ISA Old technology 8 Mhz, 4 byte transfer, bus B/W 32 Mb/sec PCI bus- Peripheral Component Interconnect Speeds up to 132 MB/s Bus speed of 33mhz, 4 Bytes/transfer PCI popularized Plug and Play

PCI-X extended PCI 133 MHz, 8 bytes/transfer, 1064 MB/sec or 1 GB/ sec Used to connect gigabit ethernet, high speed disks SCSI (Small Computer System Interface) Capable of handling internal/external peripherals Speed anywhere from 80 – 640 Mb/s Many types of SCSI

Parallel vs serial (point-to-point) bus

Fast SCSI Ultra SCSI Ultra wide SCSI

USB 1.0

Processor-Memory bus

CPI/IO

plug-and-play Full speed USB devices signal at 12Mb/s Low speed devices use a 1.5Mb/s subchannel. Up to 127 devices chained together

IO

Bus Bridge

I/O bus

Parallel bus Bus shared among devices Bus arbitration is slow Example: PCI, SCSI

IO

2.0 data rate of 480 mega bits per second

Serial I/O

Point to pint links connected directly to CPU Requires lots of additional high speed hardware Examples: SATA, USB, firewire

16

Firewire (apple) High speed serial port 400 mbps transfer rate 30 times faster than USB 1.0 plug-and-play

Intel Bus

North bridge and South bridge bus

http://www.testbench.in/pcie_sys_2.PNG

17

http://www.yourdictionary.com/images/computer/CHIPSET.GIF

18