What, Why and How

Ensure system can handle traffic Positive member experience and company reputation Identify scalability, bottlenecks, infrastructure limits Establishing baseline performance metrics Performance regression to reveal system performance before and after changes Everyone wants to be faster, human nature Save $$$

Bing 2 sec delay in search, rev per user down 4% Google 400 ms .6% drop in searches per user Yahoo 400 ms delay full page onload, traffic 5-9% drop Mozilla created IE landing page, 2.2 sec faster 15% increase in downloads (conversions) Shopzilla 7 sec to 3, 12 increase in revenue User metric levels take a long time to return back to normal, once the member experience is degraded Site speed consistently piques as an issue by focus groups and BizRate surveys

 Performance - used to validate speed , stability, reliability and scalability of current system
Load testing measuring response times, throughput rates Endurance testing system behavior under higher load over long periods Stress Testing determine system behavior under higher loads Capacity Testing - used to plan for future growth, how much does your current system support now

 Yslow and Page Speed

Written by Steve Souders browser plugins that benchmark, analyzes web page performance results We use Yslow to collect data on HL, Gilt and Rue, events#all page every 30 minutes

ShowSlow
Written and Maintained by Sergey Chernyshev open source tool that helps monitor various website performance metrics over time, aggregates data collected for Yslow, Pagespeed We use it for benchmarking over time how our site performs Page Size, Load Time, Number of Requests, Ranking from Yslow http://10.0.200.52/showslow_1.1.3/details/?url=http%3A%2F%2Fwww.hautelook.com%2Fevent s%23all

XHProf
http://xhprof.hautelook.net Profiler that we use for measuring performance of our API v3 and V4 Implemented in v3/v4 continuous webservices automation Used to gauge improvements in our API and what effect changes have on API response times

Stopwatch
Old school method for gather perceived load times

 Jmeter
Open source Java app used for testing web applications Desktop application used for simulating, measuring server load and performance testing HTTP, SOAP, JDBC, LDAP Does not process HTML, execute JavaScript or render static content Monitors size of HTTP request for Events, Catalog and Product pages

 Collects and represents HTTP traffic initiated by the browser

 Site Speed Report offered by GA

Aggregates data from all users vs. 1 machine Records Page load time by browser, geographic area and page Records execution speed of events, user interactions 1 year performance events /all pages

 3 months all pages

 HauteLook onload time significantly higher than competition Over 300 HTTP requests 80% of data downloaded was images Over 2/3 of these requests are images Stale Dynamics no longer being used Still potential blocking on downloading and executing some scripts FB and Twitter blocking page onload event from firing

HauteLook Full Page Download Time *13 - 18 s

Competition #1 7-9s

Competiton #2 8-9s

Perceived Download Time

Default Landing Page size

12 - 17 s
*4.7 - 5.8 MB

6-8s
1.6 - 2.1 MB

5-6 s
1.7 MB

* Download times and page size data was gathered over 2 days

HauteLook Number of Images Total Size of Images Total HTTP requests Javascript files 225 5 MB 365 421k

Competitor 1 66 1.5 MB 87 259k

Competitor 2 55 <= 1 MB 89 730k

HTML/Text

19.4k

158k

19k

Competitor 1

HauteLook

Competitor 2

13

V3
events 3.7 s 5.9 sec

member/member_id/order s 2.99 sec cart/item 3.4 sec checkout .5 sec 1.48s

V4 events 242 ms to 1.22 sec member/member_id/order s 2.2 sec members/{member_id}/ca rt 3.6 sec members/member_id/che ckout .45 sec 1.5 sec

 Increased image compression from 80 to 60

decreased page size 30% onload speed largely unchanged

Changed events page to not depend on the promo widgets

In cases where site waits for promos to load, we no longer prevent the site from rendering

Clean up dynamics tables

Amount of data returned in HTTP request decreased by 20%

Lazy Loading of Ending Soon events

Loads New Events first, then ending soon no change in performance, complicated the code not implemented

Loading FB and twitter after page load event fires Reduce number of HTTP requests
Each requests contains additional network payload Many requests spend time in Waiting state 300 requests * 6 ms = 1800 ms 150 requests * 6 ms = 900 ms saved

 Reporting inconsistencies running performance tools from different machines Results during manual measurement varied greatly from day to day depending on # events Method initially used to collect wasnt showing accurate onload times, showing cached results and initial request Firefox doesnt exit cleanly, using too much RAM and quits Performance environment and methods used to obtain performance results must be consistent
Dont Leave Firebug Net panel open

Obtain real world measurements outside the network GA

 Async and Defer Attributes Prevents Parser and Resource Blocking Async download script and continue parsing page, execute when available Defer dont execute js code until page is downloaded and finished parsing Scripts with out these attributes are fetched and downloaded immediately FB like , google analytics, jquery for later use Addresses actual and perceived page loading speed App Cache Meant to caches resource defined by developers Good for offline browsing, speed (locally cached) and reduced server load Allows developers to specify which files to cache using a manifest, simple test file
3 parts CACHE, Networks and Fallback CACHE list resources to be cache Network define which resources are only available online, connection is necessary Fallback what to show when user is offline and resource isnt cache i.e. show offline message

Safari Desktop unlimited; Android, Firefox Desktop, Mobile Safari 10MB; Chrome 5MB, Opera 50MB default Cons: Appcaches do not expire, manifest file must be updated Cant cache across different domains unless content is served using https If one resource in manifest is not available no resources will be cached FF ask for permission when using appcache

 Sprites One large image, instead of having 10 different images for buttons with defaults and rollover states, put all those images onto 1 image laid out on a grid use the background-position CSS property to define exact position of the image on the master to be displayed When page is loaded only one master image is loaded and CSS would place the image from the master images into place Reduces the amount of HTTP requests(we have over 300) 3rd party scripts Omniture, GA, tellapart, adservices conversion, social sharing widgets Tracking pixels, analytics need to be async
quant.js

79 entries in dynamic_pixels tables for various types of tracking new member signup, checkout. Each pixel requires a request Events page contains 21 requests for 3rd party tracking, analytics and social 130 KB Removing Dynamics pixels from page did not have any noticeable improvement in performance

Performance cornerstone goal for Tech Improvements to the API service benefits all platforms Async and Defer are very beneficial for JS heavy sites Too many HTTP requests > 300 Amazon 150, Nordstrom (Womens) 77, Gilt 80, Rue 80 Requests account for a lot of network overhead with packet headers, content and separate browser requests Many requests spend at least some time in Waiting State
Cutting amount of requests in half could shave at least1 sec

Too many images, consider sprites Gilt 40-70, Amazon 70-90, Nordstrom 20-30, Rue 45-60, HauteLook 170-225 Why do we need more images than Amazon? Removing Dynamics Pixel wont help much, having them load async

 www.stevesouders.com www.webdirections.org/blog/get-offline www.softwareishard.com/blog/firebug/firebug-net-paneltimings/

www.browserscope.org www.performance-testing.org www.appcachefacts.info/ www.html5rocks.com