Test Report For STeP-IN Testers' Arena 2014 Contest 10

STeP-IN Testers Arena 2014 Contest 10
Ravisuriya | ravisuriya1@gmail.com | @testingGarage 1

Task
Task statement given by STeP-IN is as below.
Read the document Testing Techniques and select any ten techniques (total) from these three categories:
Evaluation based techniques; Coverage based techniques; and Problem based techcniques.
Find a bug in an open source software for each of the testing techniques you selected.

Product Tested
1. Notepad++ v6.5.4
Feature Tested: RegEx Search
URL: http://sourceforge.net/projects/notepad-plus/
2. Notepad++ Plugin RegEx Helper 0.0.2.2
Feature Tested: RegEx Search
URL: http://sourceforge.net/projects/nppregexhelper/
3. Greenshot 1.1.6 32 bit
Feature Tested:
URL: http://sourceforge.net/projects/greenshot/

Test Environment
Windows Ultimate 7 SP1 32 Bit
.Net Framework 4

In this document, I consider the bug as
the behavior of the product which is troubling me to what I want to accomplish
the behavior of the product which did not help me in accomplish
the behavior which is confusing me and I see there is no much help about it to solve

In simple, anything that annoys me Im considering as a bug. I will be giving the perspective from the technical end and
user end for each bug. This can be seen as test investigation but not for all the bugs.

Report Format
#Bug-ID
Title
Product & Technique
Description



#Bug-1

Title
RegEx Pattern Matching :: The matching literal expression is not highlighted when it is line break.

Description
Product: Notepad++
Test Technique: Problem Based Technique Output Constraint

1. The text file had no printable characters but just one newline character. That is two empty lines.
2. I passed the expression [^ ] to RegEx Helper plugin keeping Ignore Spaces checkbox unchecked.

I notice the two matches for this expression because this plugin ignores line breaks. But it is not highlighted in the
text file. I selected the two entries in Matches section of Plugin. The matched extraction is not highlighted i.e. the
two white spaces is not highlighted.

This is a problem because the plugin is not highlighting each extracted pattern in this context. As per the claim and
usual behavior, the plugin has to highlight the matching and extracted expression.

Test Investigation

The other problem with this behavior is, the difference between white space and a new line character. Not having a
white space but just the new line character, still the RegEx Engine of this plugin show above said result. If new line
and white space are different, then this output looks as a problem. While I say this, in Regular Expression, the
problem can be from either side i.e. how I construct the regular expression and how the engine executes it. When I
isolate this on investigation I see, the output in this case is misguiding to me while the expression I constructed is
right for my context of data. Cross verification of the result, using Notepad++ inbuilt RegEx search, these two lines
are highlighted.

Refer the below screenshots 1a and 1b. It shows the extracted pattern which is not highlighted and the one which is
highlighted.

Snap-1a


Snap-1b

#Bug-2

Title
Extracted Pattern Count :: Total count of the extracted pattern is not computed or not shown on moving to the next
matched literal.

Description
Product: Notepad++
Test Technique: Problem Based Technique Computation Constraint

1. I had the below text in the file
andaman
arunchal
2. I used the expression {^ ] and did a search.

I noticed the message -- Find: Found the 1
st
occurrence from the top. The end of the document has been
reached., at bottom of the Find dialog. Clicking on button Find Next, I did not notice the message. On completing
all the 20 matching literals in the text, I noticed the same message again.

In the RegEx search, I expect to see the count of matching literals as I move further looking at them. This helps to
see the matching count and its position. In case of Notepad++ Find, I have to use Mark feature to see total number
of matching counts. And, the highlighting from Mark feature makes highlighting using one color throughout the
matched patterns. This does not help much in case of file having huge data. With this, I see there is no computation
happening on each move for the next matched literal here.

When using the RegEx search, I can navigate only in front (down) direction. This is one level of technical constraint
but this do not stop from providing the count and position reference. If it is happening, then displaying the
reference position of matched literal is simple and need to be just pushed to UI.


Refer the below screenshot Snap-2a.

Snap-2a

#Bug-3
Title
Pattern Extraction :: Notepad++ claims to handle the regular expression with POSIX standards. But Im unable to use
the POSIX extended command options.

Description
Product: Notepad++
Test Technique: Problem Based Technique Input Constraint

1. Using the POSIX extended options in Search feature, the message it is considered as text.
That is, -i (q|Q)[^u] will be treated as regular expression text and not as text with the command.

The help section do not indicate what is the constraint on input. Passing the command options, it is not
accepted. This is logical for this context because the context is Text Editor and not the shell. In this context the
provided syntax is valid construct but not valid for context of Notepad++. I use command options often in my
practice, when I pass the same to RegEx engine of NotePad++, it was not executed giving me a surprise shock.

I expected the help file to indicate this and as well the RegEx engine showing appropriate message for not
considering the commands in the input text.


#Bug-4

Title
Regular Expression Search :: Notepad++ help file indicates the regex search operation is two way while it is one way.

Description
Product: Notepad++
Test Technique: Coverage Based Technique Specification-based Testing

1. I browsed the help file of Notepad++ to refer the assistance provided for RegEx search.
2. The help file mentions, the RegEx search is two directional i.e. Up and Down.

The search in Notepad++ for RegEx is one way i.e. Down. The radio button Up is disabled on choosing option
Regular Expression for search mode. This specification statement is misguiding and technically the RegEx execution
goes in a flow of one direction.

I expect, the help file not to misguide with such feature available specification for the product. Refer the below
screenshot Snap-4a.

Snap-4a



#Bug-5

Title
Plugin Launch :: The hotkey Ctrl + F12 do not open the plugin on Lenovo 78CN24WW.

Description
Product: RegEx Helper
Test Technique: Coverage Based Technique Menu Tour

1. I used the hotkey Ctrl + F12 to launch the RegEx Helper.
2. I did not see the plugin UI getting launched.

Using Ctrl + F12, the brightness of my laptop screen got brighter because F12 serves to brighten my screen. I tried
using the keys Alt, Shift, and Windows with F12, but did not see the Plugin getting launched instead brightness of
screen got changed. Just tried using F12 and noticed the same behavior.

Refer the below screen shot Snap-5a.

Snap-5a



#Bug-6

Title
UI :: The label text of Ignoring Space is confusing though it is consistent and functions what it claims.

Description
Product: RegEx Helper
Test Technique: Evaluation Based Technique Consistent with purpose.

1. I opened the plugin UI.
2. Taking a tour of the plugin at UI level for first, I got confused with a checkbox.
3. The checkbox with Ignore Spaces (use \x20 to match) confused me until I tested an Regular Expression with
space.

The label says,
I can ignore spaces (white space)
But which I want to match them, I have to use \x20

Though it is consistent with its purpose with its functionality but it is not consistent when it want communicates
what it is. Because, it brings in the confusion by adding the ignore and as well match as a label text.

I got confused here for a while as I started to question this and interpreted in multiple ways
To ignore space, should I use \x20 ?
To match the ignored space, should I use \x20?
To match all spaces, should I use \x20?
Should I use this only when I want to ignore spaces and not a space?
As this, I spent time in learning it which could have been avoided if the UI label was better.

I expect the label to be simple and straight forward in what it does than also having instructions for
accomplishing what it does not do. Refer the screenshot Snap-6a.

Snap-6a



#Bug-7

Title
Expression Execution :: The message shown does not indicate me what is incorrect though it identifies it is incorrect.

Description
Product: Notepad++
Test Technique: Coverage Based Technique Path Testing.

1. I used the regular expression ([([(Fir|1)])])st .
2. The RegEx engine of Notepad++ said me, Find: Invalid regular expression.

The message what Im shown is valid but it is not helping me to know what is invalid from the RegEx Engines state
identification. If one is used to debug and practicing consistently with RegEx, it may be easy to identify what is the
problem. Otherwise, the message does not help much except saying what is entered is not with right syntax.

Test Investigation

([([(Fir|1)])])st this expression is invalid because of the parenthesis which is highlighted with Turquoise color. If
one goes as per the programming languages parenthesis or general finite automata syntax pattern, identifying this
could be time consuming.

If the message said, The Regular Expression has unmatched parenthesis. from the execution state of engine, it will
be useful and very much needed. If the engine is identifying it is incorrect, then it hints, the engine know the reason
as well which is not just pulled out and shown on UI.

Refer the screenshot Snap-7a.

Snap-7a



#Bug-8

Title
Image Upload :: Canceling the uploading of image shows an error dialog which has no clue what it is.

Description
Product: Greenshot
Test Technique: Coverage Based Technique Statement and Branch Coverage Testing.

1. I configured to upload the captured image to Imgur.
2. During the upload, I canceled the upload.
3. On canceling the image uploading, I noticed a dialog saying there was an error while uploading an image.

The condition that I created was, brought in queue of image uploading and canceled the one which was getting
uploaded. While I did this, the dialog came up. The dialog says, it is an error, but the dialog is not consistent with
how an error dialog should as per Microsoft guidelines. I expect to see the message which is clear and simple in
knowing what it is.

Test Investigation

Looking at the source code of Greenshot i.e. ImgurUtil.cs file available at http://bit.ly/1d4Q3eE, I notice the idea
here was to print exception to a dialog without mentioning what kind of dialog it is. But, the exception or error was
not written in language which can be understood easily the cause and what has to be done.

The below screenshot Snap-8a shows the conditional branch having the exception object ex printed as it is. Hence I
see the message as in the dialog.

Snap-8a

The below screenshot Snap-8b, shows the error dialog displayed.

Snap-8b

There is scope to improve the error handling and showing them in Greenshot. From the code it is evident that any
exception or error will appear as this but just change of message i.e. with the message coming from object ex.



#Bug-9

Title
Image Upload :: Uploading of image fails after my anonymous user credit is exhausted and this is not communicated to
me.

Description
Product: Greenshot
Test Technique: Coverage Based Technique Configuration Coverage.

1. I configured to upload the image as anonymous user with Imgur.
2. As per the Imgur API 2.0, the credit for anonymous user account was 424.
3. The credit value was deducted when I upload the image and view the uploaded image. I did not try for deleting
the image during this session.
4. Once the credit value got exhausted, the product started to show message An error occurred while uploading
to Imgur: The remote server returned an error (403) Forbidden.

This message did not say what the problem is except HTTP status code 403. I expected the image which I captured to
get uploaded as quickly as possible.

Test Investigation

The HTTP Status Code 403 says, the server is no longer accepting the request instead it is refusing. Investigating
further, I noticed the credit value was exhausted and I had no clue about it anywhere in Imgur History dialog nor in
this dialog. As the result, I will tend to see the same dialog for next 24 hours if I dont sign with my user account
with Imgur. Anyways, with the signed in user account, the same behavior will be observed once the credit limit is
exhausted.

Showing the details of credit limit on UI and message indicating in simple and understandable language will help me
here.

Below screenshot Snap-9a shows the error dialog displayed when the credit value is exhausted.

Snap-9a


#Bug-10

Title
Image Capture :: Greenshot stops responding and goes to hang state when the capture wait time grows while capturing
is continued successively.

Description
Product: Greenshot
Test Technique: Coverage Based Technique State Based Testing.

1. I moved the wait before capture time to 10000 milliseconds i.e. 10 seconds.
2. That is, for one initiating a capture, product will wait for 10 seconds to snap the screen.
3. In these 10 seconds of wait time, I continued to make continued successive capture.
4. Noticed, the product going into non-responsive state and my screen turning to black.
5. After a while it showed message saying Greenshot is not responding.

I expected, the product to make me wait until the capture starts for previous initiation. A message saying you to
complete the initiated capture before starting a fresh capture could have helped me here.

Test Investigation

I have not investigated the state transition here as I do not have the programming environment setup and profiling
support on my test machine. Looking at the source of the capture and wait time, it shows the state event activity is
delayed.

Refer the below screenshot Snap-10a.

Snap-10a



Log Trace

AppName=Greenshot
AppPath=C:\Program Files\Greenshot\Greenshot.exe
ReportDescription=A problem caused this program to stop interacting with Windows.

UI[3]=Greenshot is not responding
UI[4]=Windows can check online for a solution. If you close the program, you might lose information.
UI[5]=Check for a solution and close the program
UI[6]=Check for a solution and close the program
UI[7]=Close the program

Test Report For STeP-IN Testers' Arena 2014 Contest 10

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Test Report For STeP-IN Testers' Arena 2014 Contest 10

Caricato da

Copyright:

STeP-IN Testers Arena 2014 Contest 10

Ravisuriya | ravisuriya1@gmail.com | @testingGarage 1

Potrebbero piacerti anche