VectorCAST for PIC controller

Core Technologies Division

Analysis Report Vectorcast

Vector CAST Analysis Report

Author: Easter Raj S

VectorCAST tool support for 33FJ128MC804 PIC controller - feasibility analysis

Copyright Notice
This document contains proprietary information of HCL Technologies Ltd. No part of this document may be reproduced, stored, copied, or transmitted in any form or by means of electronic, mechanical, photocopying or otherwise, without the express consent of HCL Technologies. This document is intended for internal circulation only and not meant for external distribution.

VectorCAST for PIC controller

Summary
Vector Cast/C++ is an integrated software test solution that significantly reduces the time, effort, and cost associated with testing C/C++ software components necessary for validating safety- and mission-critical embedded systems. A one month Vector cast license was procured from Vector cast to evaluate the product for the 33FJ128MC804 PIC controller Firmware and the details are in this report.

VectorCAST for PIC controller

Contents
1 RELEASE CONTROL RECORD............................................................................................4 2 DEFINITIONS............................................................................................................................5 3 REFERENCE DOCUMENT.....................................................................................................6 4 PURPOSE....................................................................................................................................6 5 SCOPE OF THE ANALYSIS....................................................................................................6 5.1 PROBLEM DESCRIPTION...............................................................................................................6 5.2 TYPE OF ANALYSIS.....................................................................................................................6 5.3 ANALYSIS CRITERIA, ACCEPTANCE LEVEL.....................................................................................6 5.4 TOOLS USED .............................................................................................................................7 6 ANALYSIS APPROACH ASSUMPTIONS.............................................................................7 7 ANALYSIS APPROACH...........................................................................................................7 7.1 INSTALLATION PROCEDURE (VECTORCAST ON THE LICENSE SERVER)..............................................7 7.2 BUILDING ENVIRONMENT.............................................................................................................8 7.3 EXECUTING THE APPLICATION FOR CODE COVERAGE ........................................................................8 7.4 CREATING THE TEST CASES .......................................................................................................21 8 RESULT VALIDATION..........................................................................................................23 9 OBSERVATION ......................................................................................................................23 10 SUMMARY / CONCLUSIONS /RECOMMENDATIONS................................................23

Tables
TABLE 1 : RELEASE CONTROL RECORD...........................................................................4 TABLE 2 : DEFINITIONS...........................................................................................................5

VectorCAST for PIC controller

1 Release Control Record

Table 1 : Release Control Record Version No.

Release Authors Easter Raj S Easter Raj S

Reason for Change

Date of Issue

0.1 0.2

Initial Version 21 Sep 2010 Updated for compiler 12 Jan 2012 template customization

VectorCAST for PIC controller

2 Definitions
Table 2 : Definitions

Term / Abbreviation UUT Unit Under Test

Definition

VectorCAST for PIC controller

3 Reference Document
1. Vector Cast getting started document (vcast_getting_started.pdf) 2. Vector Cast tool usage (vecchelp.pdf)

4 Purpose
The purpose of this document is to describe the Vector CAST tool feasibility analysis for 33FJ128MC804 PIC controller Firmware unit testing and code coverage. This document describes the analysis performed and the result of the analysis.

5 Scope of the Analysis

The objective of this analysis is to understand the Vectorcast C/C++ tool, have a handson experience with the tool and to analyze the VectorCAST feasibility for 33FJ128MC804 PIC controller on windows platform. The scope also includes analyzing the difference between using Vectorcast C/C++ for unit testing the 33FJ128MC804 PIC controller firmware and performing manual unit testing (Effort analysis is limited to perform branch coverage testing and not for functional testing). 5.1 Problem Description Generally, software-component testing requires generating one line of test code (in the form of stubs, drivers, and test data) for each line of application code to be tested. The necessity to create this "disposable" test software is the main reason manual component testing is so expensive and inefficient. Test software not only has to be written but also has to be debugged to ensure that it performs as expected. 5.2 Type of Analysis Vector cast will be installed in a Windows machine where the code coverage report and test cases will be generated for the 33FJ128MC804 PIC controller Oscillator configuration code and UART receive functionalities. Test cases will be executed and the test execution report will be generated. Also the various features available in Vector cast will be tested / experimented. 5.3 Analysis Criteria, Acceptance Level The following were the acceptance criteria Software usability.

VectorCAST for PIC controller Time taken to create and execute the unit test cases compared to manual unit testing. Accuracy and completeness of the basic path and the branches identified with a sample source code.

5.4

Tools used The following tools were used Windows OS PC PIC Module Controller Microchip MP LAB and C30 Compiler VectorCAST C/C++

6 Analysis Approach Assumptions

NA

7 Analysis Approach
7.1 Installation Procedure (VectorCAST on the License Server)

1. Insert the VectorCAST distribution CD into the CD drive, or unzip the VectorCAST distribution file (vcast.win32.version.zip). 2. Double-click on the Setup.exe icon. The VectorCAST software displays a welcome page. 3. Click Next and select an installation mode: Typical: Select this option if you will be running VectorCAST on the License Server. All VectorCAST and FLEXlm files will be installed in a specified directory (next step) on the local computer. FLEXlm Server Only: Select this option if you will NOT be running VectorCAST on the License Server. 4. Click Next and select an installation directory. The default directory (C:\VCAST) is recommended. Note: The directory path you use must not include spaces or international accent marks or diacritics. 5. Click Next. 6. Select the program folder. "VectorCAST" is the default program folder. 7. Click Next. VectorCAST begins installing the licensed software.

VectorCAST for PIC controller

installing_VectorCAS T_51.pdf

7.2

Building Environment 1. Under User Variables: Variable Name: LM_LICENSE_FILE Variable Value: C:\FLEXlm\license.dat 2. Under User Variables: Variable Name: VCAST_MICROCHIP_INSTALL_DIR Variable Value: C:\Program Files\Microchip\MPLAB C 30 3. Under System Variables: Variable Name: VECTORCAST__DIR Variable Value: C:\VCAST\

7.3

Executing the application for Code coverage 1. Copy the content of attached file and append into compiler list data file (C_TEMPLATE.DATA) in the vector CAST installation path C:\VCAST\DATA.

Custom izedCom pilerS etting.txt

2. Verify that the Vector Cast License is valid. (License .dat file to be verified for date) 3. Open the FLEXlm Server

VectorCAST for PIC controller

4. Start the FlexLM Server

5. Verify that the Server Started Successfully 6. Start the VectorCast Application

VectorCAST for PIC controller

7. Make sure that the config file is copied in to the working folder.

CCAST_.CFG

8. Setup Working Directory e.g.: C:\vcast_tutorial\My_804_SPI-DAC_P 9. From Menu bar select -> File and open New c/c++ Environment 10. Automatically this configuration is loaded with the Config file

VectorCAST for PIC controller

11. All other configurations will be automatically loaded, if we place this config file in working directory. 12. By clicking Next , we see the following, and Enter a Project name say: My_Test123 13. Click Next and select the first option Traditional Unit Testing 14. Click Next and select Coverage Type: None 15. Click Next and Set the Current working folders for source and headers

VectorCAST for PIC controller

16. Click Next and Select the Main file to be as UUT

17. Click Next and again Next , we see the following summary

18. Press Build, It starts building the project

VectorCAST for PIC controller

19. Once built successfully, will load the individual functions of main and are populated such a way

VectorCAST for PIC controller 20. If we need to Test and Cover the function OSCCONFIG, right click and Insert TestCase. And on Coverage tab select Branch coverage

21. Right click on the Test Case and select Execute with Debug and also with coverage.

VectorCAST for PIC controller

22. Just press OK and continue. It opens MPLAB IDE by itself. Select FileImport

23. By this time, there will be a .cof file created in the working folder just import that

VectorCAST for PIC controller

24. Also will see the below screen

25. Just ignore it, DONT abort or close , Make sure that the Debugger is selected for Simulator

VectorCAST for PIC controller

26. Run the project in MPLAB IDE

VectorCAST for PIC controller 27. It displays the following message and select option NO

28. Now we can see that the control stops at a location in the file.

29. Now close the MPLAB IDE, It will display the following message

VectorCAST for PIC controller

30. May select Yes /No , select Yes 31. In the VectorCast IDE, we may see that the tests executed successfully

VectorCAST for PIC controller 32. By clicking on the check box before the test case, we may see the code covered and red shows not covered and Green shows covered. Also may see that the Braches are covered as 46%

33. Now may select the TestViewFull Report

VectorCAST for PIC controller 34. Will generate the report in the .html format.

FullReport1.htm l

7.4

Creating the Test cases

Once the environment is created the Vector cast will do the complexity analysis and will provide the details regarding the number of test cases for each function in the module. Also the details regarding the flow of each basic path will be generated by Vector cast which will be taken as an input for assigning input values for variables for test case execution The following procedure was followed for creating and executing test cases o In Vector cast, in the test case tree window, select the function for which the test case need to be inserted and then right click, and click Insert Test case o Double click on the newly inserted test case. The test case sheet will be opened in the MDI window. o The values for required variables will be entered to execute the required test path and click the execute button to execute the test cases Report generated by verifying the Test case
FullReport2.htm l

VectorCAST for PIC controller 7.4.1.1 Effort analysis

Using VectorCast tool:

Sl No 1 2 3 4 5 6

Step Building the environment Executing the Code Coverage for complete test stub Design the test cases for each basic path Develop Test Cases/ each test case Verify the correctness of the test stub/ test case Execute the test cases and generate the report / test case

Approx effort involved 10 min 2 Hrs 1 Hr 30 Mnts 30 Mnts 2 Hrs

Total time 6 Hrs 10Mnts

Using Manual Testing approach

Sl No 1 2 3 4 5 6 7 8

Step Construct the flow graph from the source code or flow charts. Complete Test Stub Identify independent paths for complete test stub. Calculate Cyclomatic Complexity, V(G). Design the test cases for each basic path

Approx effort involved 4Hrs 1 hr -1 Hr 1 Hrs 2 hrs 2 hrs 1 Hr

Update the source code and add the required stubs and driver code for each test case testable through UART (serial connection) Verify the correctness of the stubs and the driver code Execute the Test cases /each test case Prepare the Execution report/test case Total time 12 Hrs

VectorCAST for PIC controller

8 Result Validation
The Vector CAST tool unit testing effort comparison with manual testing is given in below.
Effort in Hrs for Unit testing using Vector Cast 6 hrs 10 min Effort in Hrs for Unit testing Manually 12 hrs

9 Observation
Vector CAST supports fully for lower version of PIC controller and hence the batch execution of test case could feasible. Since Vector CAST is customized for 33FJ128MC804, it could not load the *.cof file directly. Hence, each test case has to be executed with Debug option. This might take little more time for test case execution.

10 Summary / Conclusions /Recommendations

It was observed that using vector cast, the unit testing time was reduced considerably. This analysis was done based on UUT stubbing approach. Vector cast (with minimal customization) is feasible for 33FJ128MC804 PIC modules for unit testing because of the following reasons Microchip C30 Support (with minimal support). Reduced effort and time. Detailed Code Coverage. Debugging the Stub Code generated is not required. Accurate due to automation. A detailed report is generated.