Welcome to the 2013

This webinar is presented by Bj Rollison and he will discuss:

API Testing: The heart of functional testing

Join us on Twitter! @esconfs #esconfs

www.eurostarconferences.com

Overview
What is API Testing? Why should we do API testing? Whats the difference between Unit and API Tests? How to we approach API testing? What types of bugs will we find?

@esconfs #esconfs

www.eurostarconferences.com

What Is API Testing?

API Application Programming Interface
Implementation hidden from calling program

Component & integration levels

Individual API End-to-End scenarios

API Testing testing below the UI

Programmatically simulate data or control flow scenarios Focus on functionality; not on behavior or customer experience
@esconfs #esconfs
www.eurostarconferences.com

http://www.faqs.org/faqs/ software-eng/testingfaq/section-14.html

What Is API Testing?

Functional testing where it lives! Separation of business logic and look and feel
Core business logic functionality is here! Customer experience and behavior is here!

Cloud Services

@esconfs #esconfs

www.eurostarconferences.com

Why Should We Do API Testing

Agile software development
Team approach testing engaged early

Drive testing upstream

Testing in parallel with development, not after the fact

Automated test cost reduction

Stable API interfaces == automation stability

Reduced business costs

Reduced build breaks (pre-check-in suite)

Refocus testing objectives at system level or through GUI

Software behavior and customer experience
@esconfs #esconfs
www.eurostarconferences.com

When Is API Testing A Good Investment

Business logic is encapsulated in classes rather than within forms No graphical user interface headless testing External APIs used by 3rd party developers Internal APIs used by 1st party application developers

@esconfs #esconfs

www.eurostarconferences.com

Unit Testing vs. API Testing

Unit Testing
Owned by development Limited in scope Mocked dependencies Does it work by itself Usually ran before check-in
@esconfs #esconfs

API Testing
Owned by test Broader in scope Mocked and real dependencies Does it play well with others Pre-check-in suites Ran after build is created
www.eurostarconferences.com

Approaches To API Testing

Its not either black box or white box; its both black and white!

@esconfs #esconfs

www.eurostarconferences.com

Testing an API as a Black Box

INPUT PARAMETERS
lpLocaleName [in, optional] Pointer to a locale name, or one of the predefined variants LCType [in] The locale information to retrieve. lpLCData [out, optional] Pointer to a buffer in which this function retrieves the requested locale information. cchData [in] Size, in characters, of the data buffer indicated by lpLCData.

OUTPUT VALUE
Returns the number of characters retrieved in the locale data buffer if successful and cchData is a nonzero value. The function returns 0 if it does not succeed. Error codes ERROR_INSUFFICIENT_BUFFER ERROR_INVALID_FLAGS ERROR_INVALID_PARAMETER

int GetLocaleInfoEx( _In_opt_ LPCWSTR lpLocaleName, _In_ LCTYPE LCType, _Out_opt_ LPWSTR lpLCData, _In_ int cchData );

API testing starts as soon as the API interface is specd out!

@esconfs #esconfs

www.eurostarconferences.com

Testing an API as a Black Box

Input argument values Boundary values, equivalent set samples Parameter combinations Combinatorial (n-wise) testing Error Validation Exception handling, and output data w/error codes Properties
Getters and setters

Scenario testing Sequence of API calls, E-2-E scenario testing

@esconfs #esconfs
www.eurostarconferences.com

Testing an API as a White Box

BOOL KillProcessByName(char *szProcessToKill) { HANDLE hProcessSnap; HANDLE hProcess; PROCESSENTRY32 pe32; DWORD dwPriorityClass; hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); if(hProcessSnap == INVALID_HANDLE_VALUE){ return( FALSE ); } pe32.dwSize = sizeof(PROCESSENTRY32); if (!Process32First(hProcessSnap, &pe32)) { CloseHandle(hProcessSnap); return( FALSE ); } do{ if(!strcmp(pe32.szExeFile,szProcessToKill)){ hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID); TerminateProcess(hProcess,0); CloseHandle(hProcess); } }while(Process32Next(hProcessSnap,&pe32)); ..

@esconfs #esconfs

www.eurostarconferences.com

Testing an API as a White Box

Code Review
Early and effective bug detection

Code coverage analysis

Structural testing, identify holes

Fault injection
Break points, resource constraints, system failures

Traces
Trace statements useful for debugging

@esconfs #esconfs

www.eurostarconferences.com

API Test Design Principles

Robust automated test cases 5 stages of API test design
Setup
Initialize data, create objects, start services, etc.

Execution
Steps to exercise API or scenario, also logging

Verification
Oracles to evaluate execution outcome

Reporting
Pass, Fail, or Blocked

Clean up
Pre-test state
@esconfs #esconfs
www.eurostarconferences.com

What Types Of Bugs Will API Testing Find

@esconfs #esconfs

Fails to handle error conditions gracefully Missing or duplicate functionality Unused flags Not implemented errors Stress Reliability Security Multi-threading issues Inconsistent error handling Performance Improper errors/warnings to caller Incorrect handling of valid argument values
www.eurostarconferences.com

Key Points
API testing benefits both the development and testing organization. API testing is both a black box and white box activity Finds some types of bugs more efficiently; butcant find all bugs! Doesnt evaluate look and feel Can change focus of GUI automation Requires testers with additional skills Not for everyone!
@esconfs #esconfs
www.eurostarconferences.com

Questions or Comments
For more thoughts on API testing see
www.testingmentor.com/imtesty

If you have questions or comments

Email Bj.Rollison@TestingMentor.com Twitter - @TestingMentor

@esconfs #esconfs

www.eurostarconferences.com

Thanks for listening!

Go to www.eurostarconferences.com for more information

@esconfs #esconfs

www.eurostarconferences.com