Freitas Storage Class Memory

IBM Almaden Research Center
Storage Class Memory:

Technology, Systems and
Applications
Rich Freitas
freitas@almaden.ibm.com
Nonvolatile Memory Seminar
Hot Chips Conference
August 22, 2010
Memorial Auditorium, Stanford University
2010 IBM Corporation
IBM Almaden Research center
Agenda
Technology
Disks
Flash
Phase Change Memory
Systems
Memory Systems
Storage Systems
Applications
Storage Class Memory, Technology, Systems and Applications

Rich Freitas, IBM Research
Hot Chips 8/2010
Definition of Storage Class Memory SCM
A new class of data storage/memory devices

many technologies compete to be the best SCM
SCM features:
Non-volatile
Short Access times (~ DRAM like )
Low cost per bit (more DISK like by 2020)
Solid state, no moving parts
SCM blurs the distinction between

MEMORY (fast, expensive, volatile ) and
STORAGE (slow, cheap, non-volatile)
3

Hot Chips 8/2010
System Targets for SCM
Megacenters
Billions!
Mobile
4
Desktop X

Datacenter
Hot Chips 8/2010
History of HDD is based on Areal Density Growth

Hot Chips 8/2010

Hot Chips 8/2010
-6.5%

Hot Chips 8/2010

Hot Chips 8/2010
Many device technologies considered for SCM

Phase Change RAM
most promising now (scaling)
Magnetic RAM
used today, but poor scaling and a space hog
Magnetic Racetrack
basic research, but very promising long term
Ferroelectric RAM
used today, but poor scaleability
Solid Electrolyte and resistive RAM (Memristor)
early development, promising
Organic, nano particle and polymeric RAM
bistable material plus

on-off switch
many different devices in this class, unlikely

Improved FLASH
still slow and poor write endurance
Generic SCM Array

9

Hot Chips 8/2010
Emerging Memory Technologies

FLASH
FRAM
Extension
Trap Storage
Ramtron
Saifun NROM
Fujitsu
MRAM
IBM
PCRAM
Ovonyx
RRAM
Solid
Electrolyte
Polymer/
Organic
IBM
Axon
Spansion
Infineon
Samsung
Infineon
BAE
Sharp
Freescale
Intel
Unity
TFE
Tower
STMicro
Spansion
TI
Philips
STMicro
Spansion
MEC
Infineon
Toshiba
STMicro
Samsung
Samsung
Zettacore
Macronix
Infineon
HP
Samsung
Samsung
NVE
Toshiba
NEC
Honeywell
Spansion
Hitachi
Toshiba
Macronix
Rohm
NEC
HP
Sony
Nano-xtal
Cypress
Fujitsu
Freescale
Matsushita
Matsushita
NEC
Roltronics
IBM
Macronix
Nanolayer
Infineon
Hitachi
Philips
Renesas
Oki
Samsung
Hynix
Hynix
Celis
Elpida
TSMC
Fujitsu
Seiko Epson
64Mb FRAM (Prototype)

0.13um 3.3V
10
4Mb MRAM (Product)

0.18um 3.3V

512Mb PRAM (Prototype)

0.1um 1.8V
Hot Chips 8/2010
4Mb C-RAM (Product)

0.25um 3.3V
Research interest
Papers presented at
Symposium on VLSI Technology
IEDM (Int. Electron Devices Meeting)
Number of papers
60
organic
50
solid electrolyte
RRAM
40
PCram
MRAM
30
FeRAM
SONOS Flash
20
nanocrystal Flash
Flash
10
0
2001
2002
2003
2004
2005
2006
2007
2008
Year
11

Hot Chips 8/2010
What is Flash?
gate
drain
source
Floating
Gate
oxide
control gate
e- e-
source
Flash
Memory
1
drain
Flash
Memory
0
Floating control gate

e- e- e- e- eGate
drain
source
12

Based on MOS transistor
Transistor gate is redesigned

Charge is placed or removed
near the gate
The threshold voltage Vth of the
transistor is shifted by the
presence of this charge
The threshold Voltage shift
detection enables non-volatile
memory function.
Hot Chips 8/2010
Feeds and Speeds for typical NAND Flash

NAND
Cell Size
4 F2
(2 F2 virtual x 2-bit
MLC)
Read Access Time
20-50 us
Read
15-25 MB/s
Write
5-8MB/sec
Erase
2ms
Start Up Time
Market Size (2007)
Applications
13
50-100 us
$14.2B
Multimedia

Hot Chips 8/2010
Representative NAND Flash Device

Interface: one or two bytes wide
Data accessed in pages
2112, 4224 or 8448 Bytes
B
U
F
Power circuits
Power
Circuits
Block 63
Charge Pumps
Clock drivers
Etc.
Block 1
Data erased in blocks

Block = 64 - 128 Pages
BCA
UHC
FE
Page
Page
Page
Page
Page
Page
Page
Page
Page
Page
Page
Page
Block 0
Transition to ONFI for some vendors
ONFI Open NAND Flash Interface

14

Hot Chips 8/2010
Representative Flash SSD Classes

250
100000
200
10000
150
Rd BW MB/s
Wt BW MB/s
1000
Rd IOPS
Wt IOPS
100
100
50
10
u sb
15
commercia l
en terprise

usb
commercial
Hot Chips 8/2010
enterprise
History of Phase-change memory

late 1960s Ovshinsky shows reversible electrical switching in disordered
semiconductors
early 1970s much research on mechanisms, but everything was too slow!
Crystalline phase
Electrical conductivity
Low resistance
High reflectivity
Amorphous phase
High resistance
Low reflectivity
[Neale:2001]
16

Hot Chips 8/2010
[Wuttig:2007]
Phase-change
RAM
Bit-line
PCRAM
programmable
resistor
Word
-line
Voltage
RESET pulse
Access device
(transistor, diode)
Potential headache:
Tmelt
High power/current
affects scaling!
temperature
SET
pulse
Tcryst
If crystallization is slow
time
17

Potential headache:
affects performance!
Hot Chips 8/2010
Almaden Research center

Paths IBM
to ultra-high
density memory
2F
add
2F
sub-lithographic fins
(N2 with 2-D)
starting
from standard
4F
N 1-D
(at IEDM 2005)
go to 3-D with
store M bits/cell
with 2M
demonstrated
L layers
multiple levels
demonstrated
(at IEDM 2007)
18

Hot Chips 8/2010
Industry SCM activities

SCM research in IBM
Intel/ST-Microelectronics
spun out Numonyx (FLASH &
PCM)
Samsung, Numonyx sample
PCM chips
IBM sub-litho PCM Alverstone PCM
Over 30 companies work on

SCM
including all major IT players
Samsung 512 Mbit PCM chip

19

Hot Chips 8/2010
Magnetic Racetrack Memory

MRAM alternatives
a 3-D shift
register
Data stored as pattern of magnetic domains in long

nanowire or racetrack of magnetic material.
Current pulses move domains along racetrack
Use deep trench to get many (10-100) bits per 4F2
IBM trench
DRAM
20

Magnetic Race Track Memory

S. Parkin (IBM), US patents 6,834,005
(2004) & 6,898,132 (2005)
Hot Chips 8/2010
Magnetic Racetrack Memory

Need deep trench with notches to pin
domains
Need sensitive sensors to read presence of
domains
Must insure a moderate current pulse moves
every domain one and only one notch
Basic physics of current-induced domain
motion being investigated
Promise (10-100 bits/F2) is enormous
but were still working on our basic understanding
of the physical phenomena
21

Hot Chips 8/2010
Memory/Storage Stack Latency Problem

Get data from TAPE (40s)
Century
1010
109
108
107
Month
Day
Hour
Second
22
Access DISK (5ms)
106
Time in ns
Human Scale
Year
Storage
105
10
Access FLASH (20 us)
SCM
103
102
Access PCM (100 1000 ns)

Get data from DRAM or PCM (60ns)
10
Get data from L2 cache (10ns)
Memory
CPU operations (1ns)

Hot Chips 8/2010
Architecture
CPU
I/O
Controller
Synchronous
Internal
DRAM
Memory
Controller
SCM
SCM
Storage
Controller
External
23
Hardware managed
Low overhead
Processor waits
Fast SCM, Not Flash
Cached or pooled memory
SCM
Asynchronous
Software managed
High overhead
Processor doesnt wait
Switch processes
Flash and slow SCM
Paging or storage
Disk

Hot Chips 8/2010
SCM in a large System
1980
Logic
Memory
Active Storage
CPU
RAM
DISK
CPU
2015
CPU
24
TAPE
FLASH
RAM
RAM
2008
Archival
SSD
...
SCM

TAPE
DISK
DISK
Hot Chips 8/2010
TAPE
CPU & Memory System Conceptual Alternatives

DRAM
DIMM
X86, PowerPC,
SPARC
PMC
CPU
VM
SCMC
SCM
Card
SCM
SCM
Controller
25

Hot Chips 8/2010
Input from the device cost crystal ball

$ 100,000.00
$ 10,000.00
$ 1,000.00
$ 100.00
$ 10.00
DRAM
DRAM-pr
FLASH
FLASH-pr
MRAM
SRAM
SCM
$ 1.00
$ 0.10
$ 0.01
1990
26
1995
2000
2005

2010
2015
Hot Chips 8/2010
2020
SCM: Generic Storage Design

CPU
MEMORY
SCM
SCM
SCM
DRAM
2
SCM
Control
SCM
SCM
SCM
SCM
27

SCM
Hot Chips 8/2010
SCM
28

Hot Chips 8/2010
Challenges with SCM

Asymmetric performance
Flash: writes much slower than reads
The fly in the ointment is write

endurance
Not as pronounced in other

technologies
Program/erase cycle
Issue for flash
Most are write-in-place
Data retention and Non-volatility
Its all relative
Use case dependent
Bad blocks
In many SCM technologies writes are

cumulatively destructive
For Flash it is the program/erase
cycle
Current commercial flash varieties
Single level cell (SLC) 105
Multi-level cell (MLC) 104
Coping strategy --> wear leveling

Typically hidden from applications by
infrastructure
Devices are shipped with bad blocks

Blocks wear out, etc.
29

Hot Chips 8/2010
Write and/or read endurance and life-time of SCM devices
In DRAM and disks (magnetic) there is no known wear out mechanism

In flash and many SCM technologies there are known wear out mechanisms
Tlife = Endurance Fill-Time
Fill-Time: time to write a memory unit (whats a data unit?)
Simple wear leveling each write is done to a new (empty) location
DRAM
Disk
256GB Flash
8 GB SCM
>1016
>1011
105 104
108
Wear leveled
Memory unit
1B
512 B
128 KB
256 GB
8 GB
Data unit
1B
512 B
128 KB
128 KB
128 B
Fill Time
100 ns
4 ms
2 ms
4000 s
500 s
Life Time
>31 yrs
>12 yrs
<4 min
>12 yrs
>190 yrs
Endurance
30

Hot Chips 8/2010
SCM impact on software

Operating systems
Extend state information kept about memory pages
New mechanisms to manage new resource
Enhanced to provide hints to other layers of software
Potential for direct involvement in managing caches and pools
Middle ware and applications evolutionary
Improved performance impact immediate full exploitation will
occur gradually
Little near term demand for non-volatility
Cost improvements will drive memory size
Memory size will drive larger and more complex data structures.
Reload time on a crash will be exacerbated
Users need for non-volatility, persistence, etc. will be driven by
these effects blurring of memory and storage
31

Hot Chips 8/2010
IBM QuickSilver Project SSD proof of concept

Ultra-fast storage
performance without
managing 1000s of disks.
SAN connected hosts
Demonstrated performance of
over 1 million IOPS using 40
SSDs.
Reduced $/IOPS, significantly
lower than traditional disk
storage farm.
Reduced floor space per IOPS
Improved energy efficiency for
high performance workloads.
Reduced number of storage
elements to manage
SAN
SVC
4 x 4Gbps
FC ports per node
Quick
Silver
Quick
Silver
SAN: Storage Area Network
32
SVC: San Volume Controller
32

Hot Chips 8/2010
Shift in Systems and Applications
Main Memory:
Storage:
Applications:
33
DRAM Disk Tape

Cost & power constrained
Paging not used
Only one type of memory:
volatile
DRAM SCM Disk Tape

Much larger memory space for
same power and cost
Paging viable
Memory pools: different speeds,
some persistent
Fast boot and hibernate
Active data on disk

Inactive data on tape
SANs in heavy use
Active data on SCM

Inactive data on disk/tape
DAS ??
Compute centric
Focus on hiding disk
latency
Data centric comes to fore

Focus on efficient memory use and
exploiting persistence
Fast, persistent metadata

Hot Chips 8/2010
Performance increase of
230% by automatic movement
of 3% of the application's
data to SSD
60000
50000
Throughput (IO/s)
40000
230%
IOPS
Improveme
nt
30000
20000
10000
0
0 :0 0 2 :0 0 4 :0 0 6 :0 0 8 :0 0 1 0 :0 01 2 :0 01 4 :0 01 6 :0 0 1 8 :0
T im
34

Hot Chips 8/2010
SCM Design Triangle

Speed
Memory-type
Uses
Storage-type
Uses
Cost/bit
(Write) Endurance
Power!
35

Hot Chips 8/2010
Summary
Storage Class Memory is a new class of data
storage/memory technology many technologies
are competing to be the best SCM
SCM blurs the distinction between memory and
storage
SCM will impact the design of computer systems
and applications
Flash, which has may SCM characteristics, is
available now and various SCMs are in the wings.
EasyTier like software will foster exploitation of
Flash and SCM
36

Hot Chips 8/2010
References
FAST2010 Tutorial
T2: Freitas and Chiu, Solid State Storage: Technology,
Design and Applications
http://www.usenix.org/events/fast10/tutorials
IBM Journal of Research and Development

Special issue on storage
http://www.research.ibm.com/journal/rd52-45.html
Four papers related to SCM
Questions?
37

Hot Chips 8/2010

Freitas Storage Class Memory

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Freitas Storage Class Memory

Caricato da

Copyright:

Formati disponibili

IBM Almaden Research Center

Storage Class Memory:

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Definition of Storage Class Memory SCM

A new class of data storage/memory devices

SCM blurs the distinction between

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

System Targets for SCM

Storage Class Memory, Technology, Systems and Applications

2010 IBM Corporation

IBM Almaden Research center

History of HDD is based on Areal Density Growth

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Many device technologies considered for SCM

bistable material plus

many different devices in this class, unlikely

Generic SCM Array

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Emerging Memory Technologies

64Mb FRAM (Prototype)

4Mb MRAM (Product)

Storage Class Memory, Technology, Systems and Applications

512Mb PRAM (Prototype)

Hot Chips 8/2010

4Mb C-RAM (Product)

2010 IBM Corporation

IBM Almaden Research center

Storage Class Memory, Technology, Systems and Applications

Hot Chips 8/2010

2010 IBM Corporation

IBM Almaden Research center

Floating control gate

Storage Class Memory, Technology, Systems and Applications

Based on MOS transistor

Transistor gate is redesigned

2010 IBM Corporation

IBM Almaden Research center

Feeds and Speeds for typical NAND Flash