THE “SHIFT LEFT” STRATEGY

OVERCOMING THE DOWNWARD SPIRAL OF DOOM

Darian Rashid
darian@clearskytestautomation.com
2
 Late?
Too many defects?
Not enough content delivered?

How did we get here?

3
LETS REWIND

Requirement Design Development Test Deploy

First time we
Create Test Cases validate
requirements

Development has most likely overrun its time

Test capacity is constrained
Not enough people and time to run System, Integration and Regression effectively

4
5
 Fix easy issues first
“Cannot replicate”
Test “Coverage”

Technical debt mountain

How many issues can be
differed?
How many issues from
previous releases continue to
be ignored?

6
AMBIGUITY IN REQUIREMENTS? WELL, YES AND NO…

• Ambiguous requirements
• Test cases created after development has begun
Leads to missed expectations Best case: Spend time arguing that
and rework the delivery met the requirements
Discovered well into Worst Case: Implement CRs during
development cycle testing
Each reworked item could:
Impact delivery schedule
Incur fines
7
BUT..

“We just finished up

our sprints. Now we’re
gonna test.”
 THE CODEBASE

9
THE CODEBASE
Brittle code
Deeply coupled,
interconnected web
Usually new features on top of
legacy code
Refactored last in the Regan era

10
 THE TEAM

11
12
13
EMERGENCY PATCHES
Haven’t accounted for these in the release
Are already overcommitted for the next release
Cannot drop deliverables

RESULT: A compressed time frame for the next release

14
THE “DOWNWARD SPIRAL OF DOOM”
Ambiguous
Production requirements &
issues & over-
patches commitment &
technical debt

Compressed
Low quality
release EACH development
timeframe
RELEASE
WORSE
THAN THE
LAST
Compressed
Rework &
Q/A
brittle code
timeframe

Rushed delivery
& slashed test
matrix

15
16
THE PARADIGM SHIFT
Customer
Requirements
CONTAIN ISSUES

Create
Create Develop Code/
Create Test Development Run Manual
Customer Execute Tests, Prove Delivery
Specifications Design Tests
Requirements Daily
Documents

Objectively measurable
test specifications

PROVE SOFTWARE DELIVERY BY:

1. Creating test specifications BEFORE development begins
2. Execute ALL test cases DAILY, even during development
3. Measure test coverage against requirements DAILY
 CONVENTIONAL GHERKIN
Create Adult Customer Profile with Preferred Phone as Home GIVEN I am at the Home page
Phone WHEN I click on the “Register Now” Link
THEN I will be taken to the Register screen
1.Launch the eCustomer Application AND I will see the following:
Title First Name Middle Name Last Name … Answer
2.Click on ‘or register’ link
(blank) (blank) (blank) (blank) … (blank)

3.Enter the following Mandatory Details: WHEN I enter the following:

(a) enter the valid 10 digit Home phone number First Name Middle Name Last Name … Answer
(b)Preferred Phone: Home
Miss Bethany Jones … 9021
(c)Preferred Contact Method: Email
(d)Enter the other details given in the default demographics
section of test case initialization table THEN I will be taken to the Contact Information Screen
AND I should see the following
What other details? Which details First Name Middle Name Last Name … Answer
are valid details? Which are invalid? Miss Bethany Jones … 9021

What should I see when I click on this
link? How do I know which page I’m
taken to? How do I know that data
loaded that page correctly. If this page
isn’t right, the rest of the test cannot
proceed.
Each cell is a test case. The number of
output values to check against will
depend on the test case.
COMPARE LEFT AND RIGHT
TEST SPECIFICATIONS

Represent the behaviors of the user and the system

Created in a simple, human-readable manner

Objectively measurable
Used as the primary customer and internal acceptance criteria
Become the requirements for the development organization
Proves delivery

19
PROVE DELIVERY Create Customer
Requirements
• Creating
OBJECTIVELY
MEASURABLE test
specifications Create Test
Prove Delivery Specifications in
Gherkin
• Execute EVERY test
cases EVERY day
• Measure test Manual
coverage against Testing

requirements DAILY

Run Regression Create

Tests Multiple Development
END RESULTS Times Daily Design Documents
• Maintain quality
• Accurately show %
delivery Test Specs. Pass. Feature Develop Code
• Foresee Risk complete, add to Unit
regression suite. Testing
Execute EVERY Test EVERY Day?

21
BUT…
Average Test Execution Time: 2 minutes

Number of Tests 1 VM

1000 8.3 Hours

3000 25 Hours
5000 41.6 Hours
10,000 83.3 Hours

22
 THE ANSWER

HYPERSCALE ON THE CLOUD

23
MAXIMIZE TEST THROUGHPUT THROUGH MASSIVE-
SCALE PARALLEL TESTING USING THE CLOUD

CLEAR
SKY
CORE

24 RUNNERS
SYSTEM CAN’T TAKE THE HEAT? KEEP SCALING!

CLEAR
LOAD
SKY
BALANCER
CORE

25 RUNNERS
THE DEMO
THE CONCLUSION

Create unambiguous test specifications, used as requirements

Run EVERY test EVERY day to create a safety net
Use hyperscaled automation on the cloud to scale
Test case
Instances under test

KEEP QUALITY CONSTANT DAILY

27