EMC ViPR Data

Services
Storage Services at Cloud Scale: EMC
Roadmap and Strategy
Download this slide
VIRTUALIZE http://ouo.io/sedyi
EVERYTHING
COMPROMISE
NOTHING

Copyright 2014 EMC Corporation. All rights reserved. 1

Agenda
ASD Organization
Market Drivers
EMC ViPR Overview
EMC ViPR Data Services
EMC Object Strategy Overview
EMC ViPR Object Data Service Deep Dive
ViPR HDFS Data Service Overview
Roadmap

Copyright 2014 EMC Corporation. All rights reserved. 2

 Advanced Software
Division

Copyright 2014 EMC Corporation. All rights reserved. 3

Advanced Software Division Overview
ASD builds cloud storage and management
software optimized for the cloud and software-
defined data center
Product portfolio:
Amitabh Srivastava Cloud Data Services
ViPR Data Services
President, ASD
Atmos
Centera
Automation and Monitoring
ViPR Control Services (ViPR Controller)
SRM Suite
Network Management
Software Assurance Suite

Copyright 2014 EMC Corporation. All rights reserved. 4

 Market Overview &
Trends

Copyright 2014 EMC Corporation. All rights reserved. 5

The Big Data Economy
More data sources, richer content, longer utility

40
ZB

Source: IDC 2012 Digital Universe Study

Copyright 2014 EMC Corporation. All rights reserved. 6

2012-2016 Workload Growth
Enterprise IT must adapt to support mixed workloads

Copyright 2014 EMC Corporation. All rights reserved. 7

IT Storage Infrastructure Must Adapt
INEFFICIENT COMPLEX DEVELOPMENT
SCALE MANAGEMENT RISK
Unconstrained growth, Siloed management for Legacy infrastructure not
stranded investments physical, virtual and cloud suited for new Web,
driving up fixed costs creating complexity mobile and cloud apps

Copyright 2014 EMC Corporation. All rights reserved. 8

Big Data Storage Requirements
In-place analytics and protection of all data types

Data Unification:
Big Data storage must support structured, semi-
structured, and unstructured data types.

40 ZB
In-Place Analytics:
Analytics, compute workloads need to execute
where the data live.
Data Compliance:
More sources of data, more volume, velocity, etc.
exacerbate compliance and long-term retention
requirements

Copyright 2014 EMC Corporation. All rights reserved. 9

Why Software-Defined?
Software-defined accelerates innovation and agility

Software enables easier and faster development

of new capabilities
Tightly coupling software and hardware hinders
innovation
Software updates are possible, but dont always go
smoothly
Software-defined brings innovation to multiple
platforms
New applications and services on existing hardware
adds value
Targeting multiple hardware platforms increases value
of software

Copyright 2014 EMC Corporation. All rights reserved. 10

 EMC ViPR
EMC ViPR and ViPR Data Services
Overview

Copyright 2014 EMC Corporation. All rights reserved. 11

What is Software-defined Storage?
Software-defined storage abstracts, and pools storage resources and
automates resource delivery and management with intelligent, policy-
driven software

SDS Characteristics: SDS must feature:

Its Software Automation
Decouples control plane Extensibility
from data plane Openness
Support for multiple storage
platforms
Programmable APIs

Copyright 2014 EMC Corporation. All rights reserved. 12

EMC ViPR - Software-Defined Storage
Software Control Plane AND Data Plane
EMC ViPR Platform
OBJECT HDFS
ViPR
Data Services

BLOCK FILE
ViPR Provisioning Self-Service Automation
Reporting
Controller

Third-Party
VMAX VNX Isilon VPLEX

Commodity
Centera Atmos XtremIO

Copyright 2014 EMC Corporation. All rights reserved. 13

EMC ViPR Software Defined Storage

API
API Interface
Interface REST, CAS,

Index
Metadata
Policy Management
Storage
Storage Engine
Engine Data Protection
Replication Heterogonous storage
Unified
Scale-Out
Persistence Layer Extensible
Enterprise Grade
Disk Management

Copyright 2014 EMC Corporation. All rights reserved. 14

ViPR Data Services
Data Services that Span Arrays and Support Hybrid Data Types

Storage services at cloud scale

Built in software
Layered over both traditional and new storage
devices
Object and HDFS data services in 2013
Many more to follow, at regular intervals
Unified platform
Data services can be used as different semantic
views on the same data e.g. Object on File, HDFS
on Object

Copyright 2014 EMC Corporation. All rights reserved. 15

ViPR Data Services: Architecture
OBJECT HDFS KEY VALUE 3rd PARTY

ViPR
OBJECT HDFS KEY-VALUE 3rd
rd PARTY

Data Path GEO-SCALE INDEX, METADATA, TRANSACTIONS

GEO SCALE INDEX, METADATA, TRANSACTIONS

ViPR Distributed Infrastructure Elastic Volumes

Control Path Device Drivers Migration

Third-Party Commodity
VMAX VNX Isilon VPLEX Centera Atmos XtremIO

Copyright 2014 EMC Corporation. All rights reserved. 16

EMC ViPR Software-defined Storage
ViPR Data Services meet the new demands of Big Data

ViPR unifies storage

Define data services in software, execute
across heterogeneous infrastructure

40 ZB ViPR facilitates analytics

Use existing storage as a Big Data repository
ViPR simplifies compliance
Simpler tiering, compliance capabilities,
persistence layer flexibility

Copyright 2014 EMC Corporation. All rights reserved. 17

 EMC Object Strategy

Copyright 2014 EMC Corporation. All rights reserved. 18

Object Architecture Evolution

Copyright 2014 EMC Corporation. All rights reserved. 19

 Todays Product Offerings
Purpose-built object appliance Software-defined platform-ViPR 1.0
Centera / Atmos

Single system to buy and grow Apply multiple data services across
Node-based, scale-out arrays
architecture Support hybrid data types (eg:
Homogeneous object Object/file, HDFS/Object)

Copyright 2014 EMC Corporation. All rights reserved. 20

 ViPR Object Data
Service Deep Dive

Copyright 2014 EMC Corporation. All rights reserved. 21

ViPR Object Data Service
Similar to Object model popularized by
Amazon S3
API support: Amazon S3, OpenStack Swift,
OBJECT EMC Atmos
Future API support: EMC Centera CAS
Extensions
Byte-range updates, atomic append, rich ACLs
Object on File
Access a set of objects as files, directly on the
underlying file storage device, with native file system
performance

Copyright 2014 EMC Corporation. All rights reserved. 22

Configuring Object Storage
Leverages ViPR Controller provisioning

OBJECT
VDI EXCHANGE DATABASE

VIRTUAL STORAGE POOLS

VIRTUAL ARRAY VIRTUAL ARRAY

3rd Party 3rd Party

VMAX VNX VNX
Isilon
40K 7500 5500

Copyright 2014 EMC Corporation. All rights reserved. 23

Configuring Object Storage
Supports physical segregation

OBJECT OBJECT

VIRTUAL STORAGE POOLS

VIRTUAL ARRAY VIRTUAL ARRAY

Commodity Commodity Commodity 3rd Party

VNX
Isilon
5500

Copyright 2014 EMC Corporation. All rights reserved. 24

Within the Object Store
Buckets provide logical segregation

OBJECT

App App App

1 2 N

Data is distributed and

VIRTUAL ARRAY
intermingled across the storage
Buckets grow and shrink on
Commodity Commodity Commodity
demand

Copyright 2014 EMC Corporation. All rights reserved. 25

How it works
Initial processing
Scalable number of data
OBJECT nodes process object requests
Scale-out architecture with
full failover between nodes
Virtualized for ease of
deployment
VIRTUAL ARRAY

3rd Party
VNX
Isilon
5500

Copyright 2014 EMC Corporation. All rights reserved. 26

How it works
Writes to containers
Data stored in append-only containers
OBJECT Distributed index to track data
locations
Write first updates container
Then updates the index
Success if index update succeeds
VIRTUAL ARRAY

3rd Party
VNX
Isilon
5500

Copyright 2014 EMC Corporation. All rights reserved. 27

ViPR Object on File Capability
Extend Object capabilities to file-based storage

A bucket of Objects can be dynamically

toggled, via API calls, between different
access modes:
Object mode: Full Object API. No File access.
File mode: Native Read/Write access to Files,
directly on the underlying device, via NFS
mount. No Object access.
Dual mode: Read access as either Object or
File.

Copyright 2014 EMC Corporation. All rights reserved. 28

Object on File: Canonical Example
1
Next generation video sharing Web application
uses higher latency Object storage to store and
expose uploaded video content

OBJECT HDFS KEY VALUE 3rd PARTY

ViPR
Data Path
GEO SCALE INDEX, METADATA, TRANSACTIONS

Traditional video processing apps use lower

latency File storage to curate video content
2
Copyright 2014 EMC Corporation. All rights reserved. 29
Summary
ViPR provides storage services at cloud scale
Built in software
Layered over both traditional and new storage devices
ViPR Object data service available today
HDFS 2H 2013, many more to follow, at regular intervals
Unified platform
Data services can be used as different semantic views on
the same data e.g. Object on File, HDFS on Object

Copyright 2014 EMC Corporation. All rights reserved. 30

 Feature Roadmap
ViPR Object Data Service

Copyright 2014 EMC Corporation. All rights reserved. 31

Non-Disclosure and Disclaimer
The information on the following slides is EMC Confidential and
must not be shared with unauthorized parties without prior
written consent of EMC.
EMC makes no representation and undertakes no obligations
with regard to product planning information, anticipated product
characteristics, performance specifications, or anticipated
release dates (collectively, Roadmap Information).
Roadmap Information is provided by EMC as an accommodation
to the recipient solely for purposes of discussion and without
intending to be bound thereby.
ROADMAP CAN CHANGE. This information is EMC Confidential and must not be shared with unauthorized parties without prior written consent of
EMC. EMC makes no representation and undertakes no obligations with regard to product planning information, anticipated product
characteristics, performance specifications, or anticipated release dates (collectively, Roadmap Information). Roadmap Information is provided
by EMC as2014
Copyright an accommodation to the
EMC Corporation. All rights recipient solely for purposes of discussion and without intending to be bound thereby.
reserved. 32
EMC Object: Choice & Flexibility
Purpose-built appliance Software-defined platform
(ViPR v.Next ) (ViPR 2.0)

Single system to buy and grow Choice of hardware

Node-based, scale-out architecture Support low-cost commodity
Homogeneous object Support multiple data types (eg:
Support multiple data types (eg: Object/file, HDFS/Object)
Object/file, HDFS/Object)

Copyright 2014 EMC Corporation. All rights reserved. 33

ViPR Object Data Service Roadmap Overview
Geo Replication
Geo Distribution
Compliance
Commodity hardware support

Copyright 2014 EMC Corporation. All rights reserved. 34

Geo Replication
Increase data protection, access and performance

Data replicated across sites

optimizes access and reduces
latency.
Improved data access
Site 1 Site 2 Increased data protection
Reduced storage overhead
Site 3

Copyright 2014 EMC Corporation. All rights reserved. 35

 Geo Distribution
Anywhere access to content

Single namespace
https://accesspoint.yourcompany.com
Access all data through a namespace
Distribute and replicate across multiple
geographically distributed sites
Distribute index, meta-data, and data
Active/Active architecture
Write to and read from any location

Copyright 2014 EMC Corporation. All rights reserved. 36

Compliance
Compliance archiving and long-term retention

Enabled at object or bucket level

Time-based retention enforcement
Immutability and checksum
Platform lockdown
Privileged delete
SEC 17a-4f & HIPAA

Copyright 2014 EMC Corporation. All rights reserved. 37

Object on Commodity Hardware
Apply and manage full-featured Object capabilities

HDFS KEY VALUE

Running on top commodity

white boxes
OBJECT
3rd PARTY End to end commodity H/W
bootstrap, setup, rolling
VIRTUAL ARRAY upgrades, and configuration
management.
Enabling Object, HDFS, Key-
Commodity Commodity Commodity
value, native file in the future

Copyright 2014 EMC Corporation. All rights reserved. 38

Object on Commodity Hardware
Apply and manage full-featured Object capabilities

OBJECT

EMC CENTERA EMC ATMOS

Today: Multiple object platforms

VIRTUAL ARRAY
Vision: Platform semantics
become bucket attributes
Commodity Commodity Commodity Application capture on a bucket
basis

Copyright 2014 EMC Corporation. All rights reserved. 39

Object on Commodity
Protected writes on commodity
Write updates multiple
OBJECT containers
Stored on different nodes/drives
Then updates the index
Success if index update succeeds

VIRTUAL ARRAY

Commodity Commodity Commodity

Copyright 2014 EMC Corporation. All rights reserved. 40

Object on Commodity
How ViPR efficiently utilizes commodity
Open containers accept data
OBJECT Full containers are immutable
Transform full containers
Bulk operation for efficiency
Bulk re-protection for efficiency

Full Open
VIRTUAL ARRAY

Commodity Commodity Commodity

Copyright 2014 EMC Corporation. All rights reserved. 41

Object on Commodity
A common index for unified Object storage

OBJECT

Centera Atmos

VIRTUAL ARRAY
Extend indexing for all object types
Commodity Commodity Commodity Flexible metadata search
First class citizen or for analytics set

Copyright 2014 EMC Corporation. All rights reserved. 42

Object Investments & Roadmap
EMC Syncplicity integration
SEC 17a-4f compliance for
REST data
GeoDrive Windows update
SDK updates
Watch4Net integration
EMC World announcement
Customer EAP
Copyright 2014 EMC Corporation. All rights reserved.
Gen3 Flex-240 Hardware Gen3 Flex-360 Hardware
GeoDrive Windows updates IPv6
Atmos CDP updates Envelope encryption
Tech refresh
GeoDrive Windows updates
Atmos CDP updates
Increase Object Count Geo-distribution
Improve Self-Healing namespace
Enhanced Reporting HDFS
Serviceability
Geo-Replication & Distribution
S3, Atmos, Swift APIs Compliance
Isilon, VNX, Third Party arrays Bring Your Own Commodity
Object Data Service Centera SDK API, Co-existence
HDFS Data Service (v1.1) & Transformation (Centera
arrays)
43