Harman / Becker Automotive Systems Test Automation Reference Guide

Harman Mid-System Test Automation

Framework Architecture

Reference Guide
Version 0.1
Harman / Becker Automotive Systems Test Automation Reference Guide

Abstract: This document specifies the system architecture for the Harman Mid Platform test
framework Environment. This will provide an automated support for framework design,
configuration, test execution and results analysis.

This document contains confidential information intended solely for Harman / Becker Automotive
Systems

Table of Contents

1. Introduction.......................................................................................................1
1.1 Purpose....................................................................................................................1
1.2 Scope........................................................................................................................1
1.3 Intended Audience....................................................................................................1
1.4 Revision History........................................................................................................1

2. Architectural Goals and Constraints..................................................................3

3. Overview..........................................................................................................4
3.1 Generic Components................................................................................................4
3.2 Basic Model of Operation – Script-driven Test Execution.........................................4

4. Harman Mid-Platform Test Framework Applications and Services....................5

5. Test Methodology..............................................................................................7

6. Harman Mid-Platform Test Framework Architecture.........................................8

7. Graphical User Interface.................................................................................10

8. Types of Testing .............................................................................................12

9. Test Monitoring and Reporting........................................................................13

9.1 Monitoring...............................................................................................................13
9.2 Reporting................................................................................................................13
Harman / Becker Automotive Systems Test Automation Reference Guide

1. Introduction
1.1 Purpose

This document provides a comprehensive architectural overview of the Mid Platform

Test framework for automation, using a number of different architectural views to depict
different aspects of the system. It is intended to capture and convey the significant
architectural decisions which have been made on the system. This document explains
how each module in the framework will work and how to automate test scripts. This
document is currently designated as a draft. It is being released in order to solicit
feedback.

Automation of test activities has become a must for the organizations that want to
reduce product cycle times in order to match ever -shortening Internet times in a market.
Automated testing techniques leverage tools to dramatically reduce the amount of time
required in executing test cases. Also, the tools ensure that the tests are maintainable
and usable even when the products are undergoing rapid changes in order to meet
changing market requirements and technology.

This test automation framework helps us to run functional, system and regression
testing. It implements test automation practices along with a user -friendly tool
developed in Python, GTK for Linux platforms.

1.2 Scope

This document is the reference guide for designing the actual test framework for Media,
Bluetooth and Software download features. Testing team can use this to automate using
scripts. This document will be the reference for developing test framework GUI
(frontend) also.

1.3 Intended Audience

This document can be used as a reference manual by the testing team.

1.4 Revision History

Copyright © 2009 Harman/Becker Automotive Systems

All rights reserved. No part of this document may be reproduced, stored in a retrieval
system or transmitted in any for or by any means without the prior written permission of
the publisher
Harman / Becker Automotive Systems Test Automation Reference Guide

Version Date Author Description

0.1 05-Nov-2009 Guruprasad T S Initial Draft
Harman / Becker Automotive Systems Test Automation Reference Guide

2. Architectural Goals and Constraints

The primary architectural goal is to produce a system that enables efficient, consistent,
and accurate testing. In order to achieve this goal, the Harman Mid Platform test
framework should adhere to the following principles:

1. Maximize the use of easily attainable open source (Ex: Python, GTK) or Harman-owned
technology.
2. Implement HMI-independent interfaces between components.
3. Facilitate efficient code porting through deliberate design.
4. Minimize the required test “footprint” on constrained devices.
5. Specify an extendable architecture.
6. Identify mandatory services for each component.
Harman / Becker Automotive Systems Test Automation Reference Guide

3. Overview
3.1 Generic Components
The generic test framework consists of the following components:
• Scripts for automation
• Frontend GUI
• Script execution engine
• Client layer
• Service layer
• Logger

3.2 Basic Model of Operation – Script-driven Test Execution

With the components in place, the basic model of operation for test configuration and execution
is as follows:

Based on the test plan, a set of scripts are developed. These scripts are loaded onto the
console. For each individual test case, framework executes the script so as to invoke
commands performing the following functions:

1. Connect the test framework with the platform services.

2. Select the scripts for execution.
3. Start the test execution.
4. At test completion, gather and interpret the results.
Harman / Becker Automotive Systems Test Automation Reference Guide

4. Harman Mid-Platform Test Framework Applications

and Services
The primary intended application of the Test Framework is automation and accuracy of
test result, by which improve the quality of the product. In this primary application, the
Test Framework will be driven by formally released scripts that define exactly the
configurations and tests for a given use case.

The application is made possible by the use of Python, GTK and the flexible script-driven
execution environment within the framework. In support of all of these applications, this
environment provides the following generic services:

Bluetooth Services:
This test framework provides services that allow to automate possible Bluetooth
use cases with the Bluetooth platform service against Bluetooth enabled devices (Mobile
phone, PDA, etc.). We could able to connect, dial and automate A2DP and AVRCP
profiles.

Media Services:
This test framework provides services that allow to automate playback, stop,
pause, resume, navigation of tracks etc with the Media platform service against auxiliary
devices (USB stick, iPod, SD card etc.)

Results Analysis Services:

At the conclusion of a test run, this framework service supports the gathering of results
and subsequent analysis for pass/fail determination. Logger will generates service signal
logs. We could able to extract the test result.
Harman / Becker Automotive Systems Test Automation Reference Guide
Harman / Becker Automotive Systems Test Automation Reference Guide

5. Test Methodology
All the test cases/ scripts will be there in the test repository which is a part of the test framework.
We just have to select the test cases to run. These test cases will run one by one and generate
the results. The Independent design and test processes ensure unbiased, repeatable results.

The diagram, as shown below, shows the connectivity between different elements of an
automated test environment. Test Scripts running can connect to a Target using TCP/IP
Protocol.

Configuration of target and test equipments, test actions, assertion statements, comparison of
results, etc. as required by a test case are scripted using a standard scripting language like
Python. The framework provides a template for scripting test cases and defining test
configuration files. Common functions are available for scripting message outputs to test logs,
action, assertion and comparison statements in a test case. Currently the framework supports
Python test scripts.

Test scripts can be grouped together at various levels for testing either a functional block or a
subsystem or the target as a whole. The entire testware (test scripts, test configuration files,
drivers, stubs, test plans, setup diagrams, libraries, etc.) is accessible and maintainable from
GUI. User can select test cases and execute them. Test execution results are logged in files
and test reports are generated. Test logs and reports are well formatted and linked to each
other. Summary of test case execution and result is stored from which one can create statistical
test reports for analysis and decision - making.

Figure 1: Test Automation methodology

Harman / Becker Automotive Systems Test Automation Reference Guide

6. Harman Mid-Platform Test Framework Architecture

Figure 2: Harman Mid Platform Test Framework Architecture

The above diagram gives the idea of the components in the framework design. Test
framework solution consists of

• Test scripts containing the test cases and scenario logic to be executed
• Test Framework Interface which exposes control path to the user interface
• Test Configuration module to help select tests to be executed
• Script execution engine to run the selected test scripts.
• Logger module to log the test results and test execution logs
• Client layer which provides the interface for service layer.
• D-Bus interface (Bindings) which provide the D-Bus service interface for
platform services
• Logger backend to store the logs. Backend can be a simple file or an
elaborate SQL database (in the future)
• UI facilitates user to run test.
Harman / Becker Automotive Systems Test Automation Reference Guide

When this test tool is invoked, GUI and Test Server processes are started. GUI client makes a
TCP connection to the platform for sending requests and receiving status/logs. The GUI and the
test server processes are active as long as the tool is running. However, the test server invokes
the test engine process only when the test server needs to execute a test batch. After the test
batch execution is over, the test engine exits.

Solution entails developments only on the Linux host, no modifications are necessary on the
target. This architecture is a script driven automation system. We will be passing the arguments
for the APIs via scripts (XML). We can select the test cases with the help of GUI. These scripts
get processed in the script execution engine, which will invoke respective services (Bluetooth/
Media etc) and run. The test result will be logged in the logger.
Harman / Becker Automotive Systems Test Automation Reference Guide

7. Graphical User Interface

GUI component is responsible for interaction with user. It is a front -end of the tool. User can
select, execute and control execution of test cases from the GUI. User can also view test
execution status, logs and reports.
Harman / Becker Automotive Systems Test Automation Reference Guide

Figure 3: Frontend GUI

Harman / Becker Automotive Systems Test Automation Reference Guide

8. Types of Testing

Following are the types of testing we can automate using this framework.

• Functional Testing

• Stress testing

• User interaction Testing

• Regression Testing

• Performance Testing

• Interface Testing
Harman / Becker Automotive Systems Test Automation Reference Guide

9. Test Monitoring and Reporting

9.1 Monitoring
We can select the test cases to be executed using UI and can execute. During execution
we can monitor the execution status in a monitoring command window. Which will shows
the execution logs step by step.

9.2 Reporting
Once a test session has been completed we can then generate a test report that
contains the desired executions, verdicts and logs for each one. For each test case
execution it will generate the logs and finally after the execution of that particular test
case it generates the execution result whether pass/ fail/ inconclusive.
Harman / Becker Automotive Systems Test Automation Reference Guide