ExtentReport Review

(by: Rogelio CAGS Caga-anan Jr.)

Comparison between
extentreports-java-2.031.jar
and
extentreports-java-2.4.jar

Pseudocode
// verifyIssueWorkflow Test
parent = report.startTest("verifyIssueWorkflow","verify that the Issue workflow
functions as expected");

// verifyAmendWorkflow Test
parent = report.startTest("verifyAmendFlow","verify that the amend workflow
functions as expected");

// create child report - Login

child = report.startTest("Login");
// log child steps results
child.log(LogStatus.PASS,"type username","OK");
child.log(LogStatus.PASS,"type password","OK");
child.log(LogStatus.PASS,"click login","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Login

child = report.startTest("Login");
// log child steps results
child.log(LogStatus.PASS,"type username","OK");
child.log(LogStatus.PASS,"type password","OK");
child.log(LogStatus.PASS,"click login","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Goto Zone

child = report.startTest("Goto Zone");
// log child steps results
child.log(LogStatus.PASS,"select Zone","OK");
child.log(LogStatus.PASS,"click login","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Goto Zone

child = report.startTest("Goto Zone");
// log child steps results
child.log(LogStatus.PASS,"select Zone","OK");
child.log(LogStatus.PASS,"click login","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Select Trade Application

child = report.startTest("Select Trade Application");
// log child steps results
child.log(LogStatus.PASS,"select Application","OK");
child.log(LogStatus.PASS,"click OK","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Select Trade Application

child = report.startTest("Select Trade Application");
// log child steps results
child.log(LogStatus.PASS,"select Application","OK");
child.log(LogStatus.PASS,"click OK","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Logout

child = report.startTest("Logout");
// log child steps results
child.log(LogStatus.PASS,"click Exit","OK");
child.log(LogStatus.PASS,"click Logout","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// create child report - Logout

child = report.startTest("Logout");
// log child steps results
child.log(LogStatus.PASS,"click Exit","OK");
child.log(LogStatus.PASS,"click Logout","OK");
child.log(LogStatus.PASS,"verify landing page","OK");
// append this child
parent.appendChild(child);

// based on the logStatus of the child steps, log status as parent test status
parent.log(LogStatus.PASS,"verify Issue Workflow","PASSED");

// based on the logStatus of the child steps, log status as parent test status
parent.log(LogStatus.PASS,"verify Issue Workflow","PASSED");

report.endTest(parent);

report.endTest(parent);

Psuedo code is meant to represent

Each Parent node has

1 call to startTest()
2 Parent nodes
- parent = report.startTest()
- verifyIssueWorkflowTest
AND
- verifyAmendWorkflowTest
1 log representing the status of
the test
Each parent has 4 child nodes and associated steps- parent.log
- Login
(LogStatus.PASS,Pass)
Each child node is represented by a call
- type username
to startTest
- type password
- child = report.startTest()
- click login
- verify landing page
- Goto Zone
Each associated step is represented by a
- select zone
log
- click ok
- child.log(LogStatus.PASS,type
- verify landing page
Each child node is appended to parent
username,OK);
- Select Trade Application
- parent.appendChild(child);
- select Application
- click ok
- verify landing page
- Logout
- click Exit
- click Logout
- verify landing page

Code Structure using

@Test (priority=1)
TestNG

public void verifyIssueWorkflow(){

TestNG records this as a Test,

corresponding to the count in Test
View in v2.031

parent = report.startTest("verifyIssueWorkflow", "verify that the Issue workflow functions

as expected");

Test View in v2.04 counts all

report.startTest() as a Test in Test
View

// insert child node and logs

child = report.startTest(Login);
child.log(LogStatus.PASS,type username,OK);
:
// append child
parent.appendChild(child);

Total test count is based on the

number of report.startTest() calls,
hence there is always an additional
// insert more child node and logs as needed
1 test counted which is the parent
= report.startTest() call instead of
// must do a parent.log to show details in summary view
only counting child =
if (testStatus){
parent.log(LogStatus.PASS, "verifyIssueWorkflow", "Status:
PASSED");
report.startTest()
calls

} else {
Test View in v2.04 the status of the
// need to log this to parent if at least 1 test step failed
parent.log(LogStatus.FAIL, "verifyIssueWorkflow", "Status:parent
FAILED");
is based on the last
}
RunStatus as seen in the Quick Test
// close the report
report.endTest(parent);

Summary and not the no. of parent

= report.startTest() calls
TestNG records this as the status of the Test,

// force fail the test if any test step fails

corresponding to the parent.log() in v2.031
if (!testStatus){
Assert.fail();
}
}

ing the code Using extentreports-java-2.031.jar

3. Test view
reflects correct
test count graph
and matches
with Quick Test
Summary

1. Quick Test Summary reflects

correct status of parent
(verifyIssueWorkflow &
verifyAmendFlow) based on
parent.logStatus()

Reflects 28
steps instead
of 26
Reflects 2
Failed steps
instead of 1

In Step View, all child nodes are taken as

a step, hence the extra 2 steps that
could be brought by calling parent =
report.startTest() & appendChild() for
verifyIssueWorkflow and
verifyAmendFlow

4. Pass
Percentage
reflects correct
value based on
Test View

2. Quick Test Summary reflects

correct runStatus, based on the
last logStatus of the child node

Expanding verifyIssueWorkflow
Status taken from
parent.logStatus()
This line is counted as 1 step possible
side effect when calling - parent =
report.startTest()

Child Node representing Login

Child logStatus -> Login Steps

Child Node status is

taken from last
runStatus
There are 13 steps for
verifyIssueWorkflow
Login has 4
Goto Zone has 3
Select Trade Application
has 3
Logout has 3
Steps View counts 14,
including the additional
step mentioned above

Expanding verifyAmendFlow
Status taken from
parent.logStatus()
This line is counted as 1 step as a
side effect of calling - parent =
report.startTest()
For v2.031 & v2.04, it
would be more accurate
if these are the only
items counted as test
steps
Which means there
would be:
13 steps, 12 passed, 1
failed
Should be the same for
verifyIssueWorkflow

There are 13 steps for

verifyAmendFlow
Login has 4
Goto Zone has 3
Select Trade Application
has 3
Logout has 3
Steps View counts 14,
including the additional
step mentioned above

Child Node status is

taken from last
runStatus

ing the code using extentreports-java-2.04.jar

Test View reflects

the no. of child
nodes instead of
the no. of parent
nodes

Steps View
values is now the
same as the
Quick Test View.
Still has 28 steps
total instead of
26

1. In Quick Test
Summary, only
LogStatus used is
reflected

ding the parent nodes verifyIssueWorkflow and verifyAmendFlow,

Reflected in Test View as 10 tests
8 passed, 2 failed

For v2.04, it would be more

accurate if only these items
are counted as tests.
Which means there would
be:
8 tests total
7 passed
1 failed

Count in Steps View for v2.04 is the same as that in v2.031

Comparing TestNG result with

ExtentReport using v2.031
1. Quick Test Summary matches TestNG @Test status
2. Total time reported by ExtentReport and TestNG are
similar
3. Status of TestNG test class match with the count in
Test View
4. Additional Run status indicates status of last
child.log() status

3
2
1

Comparing TestNG result with

ExtentReport using v2.04
1. RunStatus matches TestNG @Test status
2. Total time reported by ExtentReport and TestNG are
similar
3. Status of TestNG test class does not match with the
count in Test View

3
2
1

Findings

Test View using v2.031 is correct based on the pseudo code

provided as it counts the actual top level test (parent node)
Test View using v2.04 is correct based on definition of a test
because parent node based on the pseudo code represents
a test scenario (sequence of tests) rather than a single test
In both versions, the Steps View does not count the steps
based on the pseudo code as an additional step is counted
possibly as a side effect of the parent = report.startTest()
and the parent.appendChild() methods.
Changes have been implemented in v2.04
[Feature] Show Child test's step counts in Quick Test Summary
#44
[Feature] Quick Test Summary - Show only Headers which have
at least 1 count #45

Conclusions

Both versions, depending on intended reporting schema,

can be used independently. End users would benefit greatly
if the reporting schema is configurable

Schema 1 : Parent node is counted as test (v2.031)

Schema 2 : Child node is counted as a test (v2.04)

Depending on user preferences, user might expect that test

framework used (ex. TestNG or Junit) would have the same report
results as ExtentReports

Revisit Test Step count to reflect the actual test steps without
adding the additional step that could be brought about by parent
= report.startTest() and appendChild() methods