Course Title & Code: Big Data Storage and Management (CDS502)

Week 10:
* Advanced Storage Management
- Evolution of Storage Technology
- Redundant Array of Independent Disk (RAID)
- Intelligent Storage Systems

Course Lecturers:
Dr. Mohd. Adib Haji Omar
Dr. Chew XinYing
 15/11/2019 (W10)
We l e a d

• 6.15pm – 7.30pm
• Advanced Storage Technology
• Evolution of Storage Technology
• Redundant Array of Independent Disk (RAID)
• Intelligent Storage Systems

• 7.30pm – 8.00pm
• Bio Break & Maghrib Prayer
• 8.00pm – 9.00pm
• NoSQL Data Modelling
• Conceptual Techniques
• General Techniques
• Hierarchical Techniques

• 9.00pm – 9.15pm
• Q & A / Discussion
 Advanced Storage Technology
We l e a d

Intelligent Storage System

• Optimized I/O processing capability
• Improving performance, scalability and availability
• Consists of four main components
• Front end, cache, back end and physical disks
 Advanced Storage Technology
We l e a d

• Front end
• Interface between the storage system and the host
• Consists of two components
• Front-end ports
• Front-end controllers
 Advanced Storage Technology
We l e a d

• Cache
• Temporary storage to improve the I/O requests
 Advanced Storage Technology
We l e a d

• Cache Structure
• Organized into pages, the smallest unit of cache allocation
• Contains data store and tag RAM
• Data store – holds the data
• Tag RAM – location of the data in the data store and disk
• Dirty bit flag
 Advanced Storage Technology
We l e a d

• Read operation with cache

• Cache read hit
• Request from the host, the data is in the cache
• Data sent to the host without disk operation
 Advanced Storage Technology
We l e a d

• Read operation with cache

• Cache read miss
• Request from the host, the data is not in the cache
• Retrieve from the disk, put in the cache and send to the host
• Prefetch or read-ahead algorithm when read requests are sequential
 Advanced Storage Technology
We l e a d

Write operation with cache

• Write-back cache
• Data is written in the cache and sends acknowledgement to the host
• Update later to the disk
• Write-through cache
• Data is written in the cache and immediately written to the disk
• Low risk of data loss but the respond is slower
• Write aside size
• Bypassing the cache due to large data size

Cache Implementation
• Dedicated
• Specified memory location for read and write
• Global
• Read and write operation on any available addresses in the cache
 Advanced Storage Technology
We l e a d

Cache management
• Least recently used (LRU)
• Data is replaced if not used frequently
• Most recently used (MRU)
• Data is replaced if used frequently

• Flushing – committing the data to the disk

• Idle : continuously occuring
• High watermark : activated when cache utilization is high
• Forced : large I/O burst
 Advanced Storage Technology
We l e a d

Cache data protection

• Cache mirroring
• Need two independent memory card/cache
• Need to ensure cache coherency
• Cache vaulting
• Temporary storage into vault drives
• Restore the contents when the power is restored
 Advanced Storage Technology
We l e a d

• Back end
• Interface between the cache and the physical disk
• Consists of two components
• Back-end ports
• Back-end controllers
 Advanced Storage Technology
We l e a d

• Physical Disk
• Connected to the back end storage controller
• Persistent data storage
• Variety types of disks
• FC, SATA, flash drives etc
 Advanced Storage Technology
We l e a d

Storage Provisioning
• Traditional
• Physical disks with RAID capability
• Logical units are created from the RAID sets
• Each unit is assigned a logical unit number (LUN)
 Advanced Storage Technology
We l e a d

Storage Provisioning
• Virtual
• LUN for virtual storage is call thin LUN
• Shared pool of storage
 Advanced Storage Technology
We l e a d

Types of Intelligent Storage Systems

• High-end storage systems
• Large storage
• Large cache
• Fault tolerance
• Multiple front end ports
• Multiple back end controllers
• Scalable
• Connectivity
• Performance
• Storage
• Handling concurrent I/Os
• Replication support
• Mission critical applications
 Advanced Storage Technology
We l e a d

Types of Intelligent Storage Systems

• Midrange storage systems
• Meeting SME requirements
• Less storage capacity
• Smaller cache
• Fewer front end ports
• Fewer back end controllers
• Replication support
 Advanced Storage Technology
We l e a d

Benefits of Intelligent Storage System

• Increased storage capacity
• Improved I/O performance
• Easier storage management
• Improved data availability
• Improved scalability and flexibility
• Improved business continuity
• Improved security and access control
 Summary (Key Takeaways)
We l e a d

• Key Features of Intelligent Storage System

• Types of Intelligent Storage System

• Benefits of Intelligent Storage System

Thank You

Prepared & Presented by:

Dr. Mohd. Adib Haji Omar
Dr. Chew XinYing
School of Computer Sciences