Understanding Disk Technology and Storage Systems for Computer Architecture Implementation

COMP 740: Computer Architecture and Implementation Montek Singh Nov 28, 2016 Topic: Storage Systems (Disk Technology); I/O 1

Disk Systems: Characteristics Capacity [bytes] Mainframes typically have 3.7 GB of disk storage per MIPS PCs tend to require 5 MB of disk storage per MIPS Bandwidth (throughput) Bytes transferred per unit time Service rate Number of service requests satisfied per unit time Supercomputer I/O requests tend to involve large amounts of data; transaction processing systems tend to involve small ones Response time (latency) Time between start and completion of an event Cost [$/MB] 2

Historical Perspective 1956 IBM Ramac early 1970s Winchester For mainframe computers, proprietary interfaces Steady shrink in form factor: 27 in. to 14 in. Form factor and capacity drives market more than perf. 1970s developments 5.25 inch floppy disk form factor Emergence of industry standard disk interfaces Early 1980s: PCs and first generation workstations Mid 1980s: Client/server computing Centralized storage on file server Disk downsizing: 8 inch to 5.25 Mass market disk drives become a reality industry standards: SCSI, IPI, IDE 5.25 inch to 3.5 inch drives for PCs; End of proprietary interfaces 1990s: Laptops => 2.5 inch drives 2000s: 1.8 used in media players (1 microdrive didn t do as well) 2010s: Flash and solid-state drives (SSD) 3

Disk Figure of Merit: Areal Density Bits recorded along a track Metric is Bits Per Inch (BPI) Number of tracks per surface Metric is Tracks Per Inch (TPI) Disk metric is bit density per unit area Metric is Bits Per Square Inch: Areal Density = BPI x TPI Year Areal Density (kb/sq in) 17,100 130,000 1,340,000 10,000,000 1973 1979 1989 1997 2000 2006 2015 2 8 1,000,000 100,000 Areal Density 63 3,090 10,000 1,000 100 10 1 1965 1975 1985 1995 2005 2015 Year 4

Example: Parameters of A Single Disk Drive 1.8, 2.5, 3, 3.5, 5.25 3600-7200 60-120 8.33-16.66 32-4096 2000-4000 90000-150000 200-600 5-17.5 250000-1000000 170-9000 0.1-1.5 inches rpm rps ms B Diameter ("form factor") Speed of rotation Time of one revolution Sector size Tracks per inch Bits per inch Areal density Data rate MTBF Capacity Cost b/inch Mb/sq inch MBps hours MB $/MB 5

Technical Details (1) Platters 2-4mm thick, made of an aluminum alloy Typically 1-6 platters in a hard disk Both sides of each platter generally used One side of one platter dedicated to information to guide the servomechanisms that control speed of rotation and head movement Each recording surface has its own read-write head Only one operational at any time Data transfer to and from disk is bit-serial Rotation Speed of rotation carefully controlled by servomechanism E.g., 7200 rpm 1% or even 1 rpm All disk drives are synchronized in some disk arrays Angular positions are identical (within tolerance) at any time 6

Technical Details (2) Flying height Read/write heads do not touch rotating disk surface Careful aerodynamic design keeps small constant distance ( 0.2 ) Disk moves with approximate linear speed of 700 inches/sec 18m/s, 40 miles/hour Smaller flying height higher writing density Hence, platters and heads sealed hermetically in clean space Called Winchester drives for historical reasons Parking the heads Before disk is allowed to slow down and stop, heads are moved over an area of the disk not used for recording, where they are allowed to come into contact with the disk surface 7

Why Disks Are Bit-Serial High TPI value makes parallel access disks unworkable Reading heads move as rigid unit One of them (the one over the servo disk) supposedly defines radial (track) position Ensemble is not truly rigid, and various reasons (like thermal dilations) prevent all heads being positioned over same track simultaneously, repeatedly, and reliably Only one head is positioned accurately at a time: servo guides the assembly to approximate position of requested track, reading head does final positioning using a high frequency signal between data tracks Sector ID always contains track number for confirmation Portable disks use shock sensors to prevent overwriting of adjacent tracks caused by jarring of R/W head 8

Track Densities Until recently, only innermost track was recorded with maximum density All other tracks contained same number of sectors and bytes Recently, manufacturers are using bands of tracks Total number of tracks (100s-1000s) divided into bands or zones (4, 8, 16, ) each containing the same number of tracks Each band has innermost track recorded at maximum density, with other tracks having same capacity Greatest capacity gains occur with a small number of bands Two scheduling options Constant rotational speed, use buffer for speed matching Constant data rate, head spinning with rotational speed corresponding to recording density of zone 9

Sector Format Sector is the smallest unit of data that can be read or written Typically between 32B and 4KB, with 512B being a common size Format of sector ID: Identifies sector with information such as angular position and track # Has its own error correcting code (ECC) GAP: Permits electronics to process ECC information DATA and its ECC ID ECC GAP DATA ECC GAP Total sector size DATA is measure of formatted capacity of disk as fraction of nominal capacity Total sector size 10

Disk Miscellanea Smaller disks tend to be more cost-effective Smaller inertia, lower power consumption per megabyte, shorter seek distances, less vibration, less heat generated Heat generation proportional to N RPM2.8 D4.6 N = number of platters D = diameter RPM = rotational speed Disk access time = seek time + rotational latency + transfer time Typical values: 12-30 ms seek time, 8.3 ms rotational latency Disk growth rates Disk areal density doubling every three years Disk transfer rate doubling every five years Disk access time halving every ten years 11

Ensembles of Disks Key idea Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.) Used in mainframes for a long time RAID Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Independent Disks (redefined in 1992) 12

Improving Bandwidth with Disk Arrays Arrays of independent disk drives Similar to high-order interleaving in main memories Each file assigned to different disk drive Simultaneous access to files Load balancing issues File Stripeing/disk Stripeing/disk interleaving Single file distributed across array of disks Similar to low-order interleaving in main memories Each logical I/O request corresponds to a data Stripe Data Stripe divided into number of equal sized Stripe units Stripe units assigned to different disk units Two kinds of Stripeing depending on size of Stripe unit Fine-grained Stripeing: Stripe unit chosen to balance load Coarse-grained Stripeing: Larger Stripe unit 13

Improving Availability with Disk Arrays MTTF of large-system disks approaches 106 hours MTTF of PC-class disks approaches 150,000 hours But array of 1,000 PC-class disks has MTTF of only 150 hours All schemes to cope with low MTTF aim to fail soft Operation should be able to continue while repair is made Always depends on some form of redundancy Independent addressing Conventional (RAID-1) Fine-grained striping Coarse-grained striping RAID-0 RAID-1 RAID-4, RAID-5 No redundancy Replication Parity ECC RAID-3 RAID-2 14

RAID-0 Stripe 0 Stripe 1 Stripe 2 Stripe 3 Stripe 4 Stripe 5 Stripe 6 Stripe 7 Stripe 8 Stripe 9 Stripe 10 Stripe 11 Stripe 12 Stripe 13 Stripe 14 Stripe 15 Striped, non-redundant Parallel access to multiple disks Excellent data transfer rate (for small Stripes) Excellent I/O request processing rate (for large Stripes) Typically used for applications requiring high performance for non-critical data 15

RAID-1 Stripe 0 Stripe 0 Stripe 1 Stripe 1 Stripe 2 Stripe 2 Stripe 3 Stripe 3 Mirrored/replicated (most costly form of redundancy) I/O request rate: good for reads, fair for writes Data transfer rate: good for reads; writes slightly slower Read can be serviced by the disk with the shorter seek distance Write must be handled by both disks Typically used in system drives and critical files Banking, insurance data Web (e-commerce) servers 16

Combining RAID-0 and RAID-1 Stripe 0 Stripe 1 Stripe 2 Stripe 3 Stripe 0 Stripe 1 Stripe 2 Stripe 3 Stripe 4 Stripe 5 Stripe 6 Stripe 7 Stripe 4 Stripe 5 Stripe 6 Stripe 7 Stripe 8 Stripe 9 Stripe 10 Stripe 11 Stripe 8 Stripe 9 Stripe 10 Stripe 11 Stripe 12 Stripe 13 Stripe 14 Stripe 15 Stripe 12 Stripe 13 Stripe 14 Stripe 15 Can combine RAID-0 and RAID-1: Mirrored Stripes (RAID 0+1, or RAID 01) Example: picture above Striped Mirrors (RAID 1+0, or RAID 10) Data transfer rate: good for reads and writes Reliability: good Efficiency: poor (100% overhead in terms of disk utilization) 17

RAID-3 b0 b1 b2 b3 P(b) Fine-grained (bit) interleaving with parity E.g., parity = sum modulo 2 (XOR) of all bits Disks are synchronized, parity computed by disk controller When one disk fails (how do you know?) Data is recovered by subtracting all data in good disks from parity disk Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Hot spares used to reduce vulnerability in reduced mode Performance: Poor I/O request rate Excellent data transfer rate Typically used in large I/O request size applications, such as imaging or CAD 18

RAID-2 b0 b1 b2 b3 f0(b) f1(b) f2(b) Hamming codes capable of correcting two or more erasures E.g., single error-correcting, double error-detecting (SEC-DED) Problem with small writes (similar to DRAM cycle time) Poor I/O request rate Excellent data transfer rate 19

RAID-4 Blk 0 Blk 1 Blk 2 Blk 3 P(0-3) Blk 4 Blk 5 Blk 6 Blk 7 P(4-7) Blk 8 Blk 9 Blk 10 Blk 11 P(8-11) Blk 12 Blk 13 Blk 14 Blk 15 P(12-15) Coarse-grained Striping with parity Unlike RAID-3, not all disks need to be read on each write New parity computed by computing difference between old and new data Drawback: Like RAID-3, parity disk involved in every write; serializes small reads I/O request rate: excellent for reads, fair for writes Data transfer rate: good for reads, fair for writes 20

RAID-5 Blk 0 Blk 1 Blk 2 Blk 3 P(0-3) Blk 4 Blk 5 Blk 6 P(4-7) Blk 7 Blk 8 Blk 9 P(8-11) Blk 10 Blk 11 Blk 12 P(12-15) Blk 13 Blk 14 Blk 15 Key Idea: reduce load on parity disk Block-interleaved distributed parity Multiple writes can occur simultaneously Block 0 can be accessed in parallel with Block 5 First needs disks 1 and 5; second needs disks 2 and 4 I/O request rate: excellent for reads, good for writes Data transfer rate: good for reads, good for writes Typically used for high request rate, read-intensive data lookup 21

Removable Media Magnetic Tapes: Have been used in computer systems as long as disks Similar magnetic technology as disks have followed similar density improvements as disks Long Stripes wound on removable spools Benefits: can be essentially of unlimited length; removable Drawbacks: only offer sequential access; wear out Used for archival backups; but on the way out Optical Disks: CD s and DVD s CD-ROM/DVD-ROM: removable, inexpensive, random access CD-R/RW, DVD-R/RAM: writable/re-writable Becoming immensely popular: replacing floppy drives, tape drives Flash: flash cards, memory sticks, digital film, SSDs Microdrives: small portable hard disks (up to 1GB) 22

Interfacing I/O Subsystem to CPU Where do we connect I/O devices? Cache? Memory? Low-cost systems: I/O bus = memory bus! CPU and I/O compete for bus How does CPU address I/O devs.? Memory-mapped I/O Portions of address space assigned to I/O devices I/O opcodes (not very popular) How does CPU synch with I/O? Polling vs. interrupt-driven Real-time systems: hybrid Clock interrupts periodically CPU polls during interrupt 23

Bus: Different Options Bus width High-perf: separate address and data lines Low-cost: multiplex address and data lines Data width High-perf: wider is faster (e.g., 64 bits) Low-cost: narrower is cheaper (e.g., 8 bits) Transfer size High-perf: multiple words have less bus overhead Low-cost: single-word transfer is simpler Bus masters High-perf: multiple (requires arbitration) Low-cost: single master (simpler) Split transaction High-perf: Yes (request and reply in separate packets ) Low-cost: No (continuous connection is cheaper) Clocking High-perf: synchronous (though it is changing) Low-cost: asynchronous 24

Examples of Bus Standards How about SATA? IDE/Ultra ATA SCSI PCI PCI-X 32 or 64 bits 32 or 64 bits Data width 16 bits 8 or 16 bits Up to 100 MHz Clock rate 10-160 MHz 33-66 MHz 66-133 MHz # Bus masters 1 Multiple Multiple Multiple Peak Bandwidth 1066 MB/sec 200 MB/sec 320 MB/sec 533 MB/sec Clocking Async Async Sync Sync 25

Does I/O Performance Matter? The No Argument: There is always another process to run while one process waits for I/O to complete Counter-arguments: Above argument applies only if throughput is sole goal Interactive software and personal computers lay more emphasis on response time (=latency), not throughput Interactive multimedia, transaction processing Process switching actually increases I/O (paging traffic) Desktop/Mobile computing: only one person/computer fewer processes than in time-sharing Amdahl s Law: benefits of making CPU faster taper off if I/O becomes the bottleneck 26

Does CPU Performance Matter? Consequences of Moore s Law: Large, fast CPU s Small, cheap CPU s Main goal: keeping I/O devices busy, not keeping CPU busy bulk of the hardware cost is in I/O peripherals, not CPU Shift in focus: From computation to communication Reflected in change of terminology: 1960 s-80 s: Computing Revolution 1990 s-present: Information Age 27

Does Performance Matter? Performance is not the problem it once was! 15 years of doubling CPU speed every 18 months (x1000) Disks also have steadily become bigger and faster Most people would prefer more reliability than speed: Today s speeds come at a cost: frequent crashes Program crashes frustration Disk crashes hysteria Reliability, availability, dependability, etc. are the key terms Client-server model of computing has made reliability the key criterion for evaluation E.g., UNC-COMPSCI IMAP server (from 1990s to 2010 or so): 300 MHz or something, 99.9 % uptime 28