Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Fail fast
20% time Project mobility
Testing won't get done unless it is unavoidable Test is everyone's job Actual testers are rare, scarcity creates clarity
The result The SET role Engineering Productivity not Testing
Engineering Productivity
A separate organization beholden to Google, not to Product Areas
"To accelerate Google" Tools from IDEs, compilers, build, test Testing, release engineering, engineering education
Product area leads negotiate with Eng Prod leads for test engineering Testers are assigned to product areas but report through Eng Prod Testers responsible for their own fate: hiring, promotion, reviews
Testing roles
SWE Feature developers code reviews, TDD, unit tests, acceptance tests own quality for every piece of code they touch
Test scenario developers risk analysis, end-to-end test scripts, quality management
Testing roles
Release and iterate Make gathering feedback part of the process Don't be afraid to take chances Fail fast, correct as you go
Release channels Canary (daily build) Developer (weekly build) Test (best weekly build) --> Dogfood Beta (latest, most stable test channel build) --> Release
Types of tests
Google is a melting pot of testers from various companies, all using a different term for a different type of test We standardized:
Types of tests
Small tests
Almost always automated Run in a mocked/faked environment Exercise code within a single function or module Focus on functional issues such as data validity and error conditions Execute within seconds Mostly written by SWEs
Does this code do what it is supposed to do?
Types of tests
Medium tests
Usually automated Run in a fake or real environment Test the interaction between functions that call each other directly Focus on functional issues such as data validity and error conditions Execute within a few seconds Written by SETs and SWEs
Do a set of near neighbor functions interoperate the way they are supposed to?
Types of tests
Large tests
Can be automated or manual Run in an actual production environment Test end-to-end functionality Can take minutes to hours to run Written by TEs and SETs
Does the product operate the way a user would expect and produce the desired results. Given all this, an enormous test should be self explanatory!
Automation vs manual
Automate when
Results can be checked by a machine No human judgement or cleverness is required Repetitive So important it can't be left to chance
Manual otherwise
Shared code
The secret?
"Committers" Similar to the open source idea of a person with "commit" status on a project At Google committers have language-specific "readabilities"
Code reviews All code must be reviewed and approved by a person with readability You earn your readability by a track record of writing good code
Platform dependencies
Don't deal with them, minimize them All engineers (dev and test) use the same OS on their desktop as we use in our data centers Local = Test = Staging = Production
Test Certified
Level 1 Set up test coverage bundles Set up a continuous build Classify your tests as Small, Medium, and Large Identify nondeterministic tests Create a smoke test suite Level 2 No releases with red tests Require smoke test suite to pass before a submit Incremental coverage by all tests >= 50% Incremental coverage by small tests >= 10% At least one feature tested by an integration test Level 3 Require tests for all nontrivial changes Incremental coverage by small tests >= 50% New significant features are tested by integration tests
Level 4 Automate running of smoke tests before submitting new code Smoke tests should take less than 30 minutes to run No nondeterministic tests Total test coverage should be at least 40% Test coverage from small tests alone should be at least 25% All significant features are tested by integration tests Level 5 Add a test for each non-trivial bug fix Actively use available analysis tools Total test coverage should be at least 60% Test coverage from small tests alone should be at least 40%
SET - test developer coding on behalf of the developers TE - user developer coding on behalf of the user TEs must be engineers to remain first class citizens
TEs progress up the job ladder, some become product managers, some become UX designers, some SETs, some SWEs ...
ACC
Simple and powerful model for analyzing web apps Think of the product in terms of
Verbs (capabilities)
ACC: Attributes
Adjectives and adverbs Ask yourself o Why are we building this? o Why is this interesting to users? Examples from Chrome o Fast o Secure o Stable
o Elegant
ACC: Components
ACC: Capabilities
Examples from Chrome Load webpages Sync preferences Run web apps
Quality Bots