Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Friday, November 9, 12
Marty Weiner
Yashh Nelapati
Orodruin, Mordor
The Shire
Pinterest is . . .
An online pinboard to organize and
share what inspires you.
Scaling Pinterest
Friday, November 9, 12
Friday, November 9, 12
Friday, November 9, 12
Friday, November 9, 12
Relationships
Marty Weiner
Grayskull, Eternia
Scaling Pinterest
Friday, November 9, 12
Relationships
Marty Weiner
Grayskull, Eternia
Yashh Nelapati
Gotham City
Scaling Pinterest
Friday, November 9, 12
Mar 2010
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2011
Jan 2011
Jan 2012
Jan 2012
Mar 2010
Mar 2010
Scaling Pinterest
Friday, November 9, 12
RackSpace
1 small Web Engine
Jan 2011
1 small MySQL DB
Jan 2012
1 Engineer
Jan 2011
Jan 2012
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2011
Jan 2012
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2012
Mar 2010
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2011
Jan 2011
Jan 2012
Jan 2012
Mar 2010
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2012
Jan 2012
Lesson Learned #1
It will fail. Keep it simple.
Scaling Pinterest
Friday, November 9, 12
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2011
Jan 2012
Mar 2010
Scaling Pinterest
Friday, November 9, 12
/ Day
Amazon EC2 + S3Page+Views
Akamai,
ELB
Jan 2011
Jan 2012
Mar 2010
Scaling Pinterest
Friday, November 9, 12
Jan 2011
Jan 2012
Oct 2012
Mar 2010
Scaling Pinterest
Friday, November 9, 12
/ Day
Amazon EC2 + S3Page+Views
Edge
Cast, Akamai, Level 3
Jan 2011
40 Engineers
Jan 2012
Oct 2012
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Why MySQL?
Scaling Pinterest
Friday, November 9, 12
Extremely mature
Well known and well liked
Rarely catastrophic loss of data
Response time to request rate increases linearly
Very good software support - XtraBackup, Innotop, Maatkit
Solid active community
Very good support from Percona
Free
Why Memcache?
Scaling Pinterest
Friday, November 9, 12
Extremely mature
Very good performance
Well known and well liked
Never crashes, and few failure modes
Free
Why Redis?
Scaling Pinterest
Friday, November 9, 12
Clustering
vs
Sharding
Scaling Pinterest
Friday, November 9, 12
Clustering
Sharding
Scaling Pinterest
Friday, November 9, 12
Clustering
Sharding
Scaling Pinterest
Friday, November 9, 12
Why Clustering?
Scaling Pinterest
Friday, November 9, 12
source: thereifixedit.com
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Cluster Manager
Scaling Pinterest
Friday, November 9, 12
Lesson Learned #2
Clustering is scary.
Scaling Pinterest
Friday, November 9, 12
Why Sharding?
Scaling Pinterest
Friday, November 9, 12
When to shard?
Scaling Pinterest
Friday, November 9, 12
Our Transition
1 DB + Foreign Keys + Joins
1 DB + Denormalized + Cache
1 DB + Read slaves + Cache
Several functionally sharded DBs + Read slaves + Cache
ID sharded DBs + Backup slaves + Cache
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
How we sharded
Scaling Pinterest
Friday, November 9, 12
db00001
db00002
.......
db00512
db00513
db00514
.......
db01024
db03072
db03073
.......
db03583
db03584
db03585
.......
db04096
High Availability
db00001
db00002
.......
db00512
db00513
db00514
.......
db01024
db03072
db03073
.......
db03583
db03584
db03585
.......
db04096
db00001
db00002
.......
db00256
db00001
db00002
.......
db00512
db00257
db00258
.......
db00512
ID Structure
64 bits
Shard ID
Type
Local ID
Scaling Pinterest
Friday, November 9, 12
Lookup Structure
{sharddb001a:
sharddb002b:
sharddb003a:
...
sharddb008b:
sharddb003a
Scaling Pinterest
Friday, November 9, 12
(
1, 512),
( 513, 1024),
(1025, 1536),
(3585, 4096)}
DB01025
users
users
ser-data
user_has_boards
ser-data
boards
ser-data
ID Structure
Scaling Pinterest
Friday, November 9, 12
Scaling Pinterest
Friday, November 9, 12
Local ID
Full ID
Full ID (+ timestamp)
Loading a Page
Scaling Pinterest
Friday, November 9, 12
Scripting
Scaling Pinterest
Friday, November 9, 12
In The Works
Scaling Pinterest
Friday, November 9, 12
Connection limits
Isolation of functionality
Isolation of access (security)
Lesson Learned #3
Keep it fun.
Scaling Pinterest
Friday, November 9, 12
NEED ENGIES
jobs@pinterest.com
Scaling Pinterest
Friday, November 9, 12
Questions?
marty@pinterest.com
Scaling Pinterest
Friday, November 9, 12
yashh@pinterest.com