Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
The purpose of this document is to list some tools that help QAC members to
increase testing productivity, save time, and help improve our QAC department.
1. TestCafe studio
Test Cafe is a Node. js end-to-end free and open source automation tool which
is used to test web applications. It works on all popular environments such as
Windows, MacOS, and Linux.
Pros
● Fast Browser is controlled from inside a web page. This makes the test run
inside a browser as fast as your browser can render a page with no extra
network requests.
● Stable to Execution.
Because a test is executed inside a browser, the network latency effects are
reduced. Unlike WebDriver you won't hit stale element exceptions, or element not
interactable exceptions, as from within a web browser all DOM elements are
accessible.
Cons
● Magic. Browsers executed in TestCafe are not aware that they run in test mode.
So at some edges automation control can be broken. It's also quite hard to debug
possible issues, as the tester doesn't know how actually a web page is parsed to
inject automation scripts.
● No Browser Control. Because TestCafe does not control the browser, the tester
can't actually automate all users actions. For instance, TestCafe can't open new
tabs or open a new browser window in incognito mode. There can also be some
issues running tests on 3rd party servers or inside iframes.
Summary
TestCafe is a good option to start for cross browser testing. And the reason to
use TestCafe with CodeceptJS: if there is an edge case or issue, tests can be
switched to WebDriver easily. As all helpers in CodeceptJS share the same
syntax.
● Cucumber integration
● and others
2. Selenium
Selenium is a portable framework for testing web applications. Selenium
provides a playback tool for authoring functional tests without the need to learn a
test scripting language (Selenium IDE). ... The tests can then run against most
modern web browsers. Selenium runs on Windows, Linux, and macOS.
Pros
● Free
a. Since it’s portable across all platforms and often doesn’t require learning
new languages, Selenium allows for an unmatched flexibility
a. With Selenium, it’s possible to test native, hybrid, or web mobile apps, but an
additional software is required. There are two main options – Appium and
Selendroid. They are both based on Selenium so that developers already versed
in it can apply the same principles when testing mobile apps.
b. Both tools are open source and have great community support. The main
difference is that Appium supports iOS, Android, and Windows devices, while
Selendroid focused exclusively on Android. Selendroid also bundles with Appium
so that when testing for Android versions 2.3 through 4.3, the program will
automatically switch to Selendroid.
● Wide range of supported languages, platforms, and browsers
The flexibility that Selenium provides is almost unmatched in the test automation
world. First, we have ten supported languages, among which are Java, Ruby,
C#, PHP, JavaScript, Perl, and R – all among today’s most used programming
languages. It’s also the only mainstream tool that covers Linux testing. For a
detailed and updated list of supported browser versions, refer to this page.
Selenium can be extended beyond its standard functionality with a wide range of
plugins. Appium and Selendroid are two of them. On this page, you will see
third-party plugins that are somewhat officially endorsed by Selenium, but you
can find more unsupported ones on Github. Be sure to also google “Selenium
plugins <tool name>” and you may find an extension for easy integration with
your favorite programming product, such as Jenkins or Eclipse.
These plugins exist not only for WebDriver but also for Grid and Selenium IDE.
The latter ones are especially extensive, able to strengthen the functionally-weak
tool and make it production-ready.
Cons
● Steep learning curve. Selenium doesn't allow for codeless testing. A good grasp
of one of the programming languages is needed which narrows down the pool of
people that can be engaged in the testing process. Many companies, especially
startups, tend to employ their best coders to write product features and engage
less skilled people in automated tests. This won’t work with Selenium.
● TestNG creates two types of reports upon test execution: detailed and summary.
The summary provides simple passed/failed data; while detailed reports have
logs, errors, test groups, etc.
● JUnit uses HTML to generate simple reports in Selenium with indicators “failed”
and “succeeded.”
● Extent Library is the most complex option: It creates test summaries, includes
screenshots, generates pie charts, and so on.
● Allure creates beautiful reports with graphs, a timeline, and categorized test
results – all on a handy dashboard.
3. Jenkins
Jenkins is a continuous integration (CI) tool for testing codebase changes in real
time. Jenkins lets detect and resolve codebase problems quickly and automate
build testing. An open-source program, it was forked from the Hudson CI tool.
Jenkins runs on servers in a container and supports version control tools such as
Perforce, Git, and Subversion.
Pros
● Jenkins is open source and free. Jenkins is free to download and the source
code is also available. This has spawned a growing community of developers
who are actively helping each other and contributing to the Jenkins project. This
ensures that you get better and more stable versions each year.
● Scheduling Test Automation. Using Jenkins, developers can spend less
time and effort on testing or managing test automation. Since Jenkins is free,
easy to use and user-friendly, developers are able to get up and running in very
little time. Following installation, testers can schedule and run test automation
cases when they’re ready and then dedicate their time to other tasks.
● Summaries and Result Trends. Jenkins provides both test summaries and
information on result trends. On the test result trend graph, users can see what
test results look like over time and gain more insight on how to move forward.
Test summaries give specific information on the number of tests, how long they
took to execute and other information that can play a role in total productivity.
Summary
Jenkins is a great choice if you’re ready to jump into test automation. It works
well with the CI methodology and helps your team complete the testing process
more quickly. There’s a reason why Jenkins is one of the most popular tools
available. It packs everything that companies need to support test automation
into one highly functional tool.
Automating testing empowers your team and allows them to be more confident in
the work that they do because they can quickly see the results. Jenkins makes it
easy to execute this type of testing. Jenkins extensive feature set provides
companies with a tremendous advantage. When you couple that with the fact
that Jenkins continues to evolve to keep up with the industry, it’s a no-brainer to
go this route.
This blog explored what Jenkins is, how it fits in with the Continuous Integration
and the reasons you should consider using Jenkins to support test automation.
We hope that this information inspires you to explore the best tools for your
upcoming projects. We know that using the right tools can make the development
process go more smoothly.
● Configure the MAven project by adding a Maven plugin into the Jenkins testing
tool.
Summary
Jenkins is an effective tool for testers. It is capable of implementing continuous
integration for developers and implements testing by linking automation tools like
Selenium.
4. Postman
Postman is a robust toolchain that allows API developers to share, test,
document, and monitor their APIs. With this tool, testers can easily integrate the
tests into the build automation. Postman is extremely valuable for functional
testers and developers.
Pros
● User-friendliness. With a simple interface, testers can quickly create test suites
by filling in templates. Postman also provides code snippets, which support script
creation with examples of validations for response time, response code, etc.
● Accessibility. Postman users can access their files seamlessly by logging into
their account on a device with the Postman application installed or Postman
browser extension.
Cons
● Limited testing area. While Postman is ideal for RESTful API tests, it is not well
designed for SOAP APIs and other APIs.
● Low script reusability. Postman users are unable to reuse their pre-written scripts
or add more requests. This means testers have to create new test scripts over
and over for each project.
● Constrained integration. While APIs enable the Agile process, the tool itself does
not support much in integration capabilities. It becomes an obstacle in connecting
Postman with the existing systems and collaborating within the team.
5. Ranorex
Ranorex lets testers set up and run test automation scripts. It has graphical user
interface (GUI) object recognition, automatic timeout handling, dynamic web
elements identification, and page object mapping built in. It also lets testers to
test web apps on every major web browser and platform.
Pros
● Multiplatform Application
With Ranorex, testers can run automated tests for web and desktop applications
as well as mobile apps, while competing products such as Selenium and Katalon
Studio don’t support desktop testing, and Watir aims at web testing only.
Given its user-friendly UI on top of the code layer, Ranorex is considered one of
the most approachable test automation tools on the market. In addition, Ranorex
Help Center offers a variety of resources to ease the first steps of working with
the framework:
● Ranorex Demo Application leads through the process of creating, recording, and
analyzing your automated software tests. Accompanied by detailed instructions,
the app helps you get familiar with the Ranorex Studio user interface.
Ranorex Studio does a good job in facilitating the workflow with their tool,
compared to other providers. In particular, Selenium courses and tutorials are
supplied by third parties.
● The path editor — a tool for specifying and editing tracked and identified
UI-elements for RanoreXPath
● Ranorex Spy — a major functionality that makes Ranorex unique from other test
automation tools. It explores and analyzes the application under test capturing
necessary elements.
However, some users claim that the support provides only standard answers and
the only way to resolve complex issues is to allow the Ranorex support team to
connect directly to the user’s machine. This may be forbidden by corporate data
protection policies.
2. Meanwhile, non-technical testers are able to create codeless test cases or reuse
existing core modules;
3. Then, using the comprehensive XML-based test report, project owners and
managers can review test results and check the project progress.
Every test run in Ranorex Studio ends with an automatic report that provides
details on test execution including visual screenshots for validation. This makes
the workflow much easier than Selenium’s, where the tester have to build a
separate reporting tool to plug into the tester’s test solutions. TestComplete also
offers test reports, but they contain results only on current test runs. Ranorex
reports are more comprehensive, as they provide Pie Charts with both current
and previous test runs.
Cons
● Paid license
● No macOS support
Although Ranorex now supports web testing of Mac apps integrating with
Selenium WebDriver, the framework still can’t be launched on macOS. Ranorex
is built on .NET, which doesn’t run on macOS unless using the cross-platform
Mono project. However, Mono isn’t supported by Ranorex. In addition, Ranorex
needs access to certain parts of the OS that Mac does not allow. Locking down
their systems, macOS hinders many Ranorex features that work on Windows
systems.
● Small community
While free tools such as Selenium and Watir are backed by a massive
community, the Ranorex user base is much smaller. iDataLabs research shows
that Selenium and Watir are used by 24,725 and 1,376 companies respectively,
while only 954 companies are applying Ranorex. As a result, it might be difficult
to find solutions to your problems and pieces of advice you’re seeking on the
web. Users believe that the Ranorex team should work on organizing the
community to share more plugins, problem-solving, etc.
● Unstable releases
While being one of the most popular test automation providers, Ranorex
frequently releases new versions. This, in turn, takes up time for updating the
existing suites of automated tests. While new releases isn’t something to
complain about, oftentimes it turns out that the new features are still unstable
and contain bugs. That’s why it might be necessary to wait for the second update
of a new version before making the change.
Ranorex is an effective GUI test automation tool that helps counteract challenges
you may encounter while shifting towards agile software development. This
framework accommodates different skillsets and ensures productive team
collaboration while being integrable and innovative.
That said, Ranorex has its drawbacks too, so it makes sense to define your
priorities while choosing a test automation tool for your team. Whether your
project has budget constraints or a lot of junior testers aboard, whether you
develop primarily on macOS devices or look for strong image recognition
features — all these and many other specifications will help you decide if
Ranorex is the framework you should go for.
6. Xray
is the #1 Manual & Automated Test Management App for QA. It’s a full-featured
tool that lives inside and seamlessly integrates with Jira. Its aim is to help
companies improve the quality of their products through effective and efficient
testing.
Features:
● Traceability between requirements, tests, defects, executions
● Test environments
● Integrates with test automation frameworks (Selenium, JUnit, Nunit, Robot, ...)
● Built-in reports
● Testpad
● Testpad is a simpler and more accessible manual test tool that prioritises
pragmatism over process. Instead of managing cases one at a time, it uses
checklist-inspired test plans that can be adapted to a wide range of
styles including Exploratory testing, the manual side of Agile, syntax highlighted
BDD, and even traditional test case management.
Key features:
● Guest testers, invited by email, who don’t need accounts
● Practitest
Other Features:
● A vast array of third-party integrations with common bug trackers, automation
tools, and robust API for the rest.
● Reuse tests and correlate results across different releases and products.
● Unique hierarchical filter trees - organize everything and find anything quickly.
● Never work twice - with anti-bug duplicates, permutations, step parameters and
call to test
7. TestRail
● TestRail is your source for scalable, customizable, web-based test case
management. Set up in just minutes with our cloud-based/SaaS solution, or
install on TestRail your own server.
● Efficiently manage manual and automated test cases, plans, and runs.
● Get real-time insights into testing progress with informative dashboards, metrics,
and activity reports.
● Boost efficiency with milestones, personal to-do lists, and email notifications.
● Document test cases with screenshots and expected results. Use the flexible
built-in templates or create your own custom templates.
8. TestMonitor
TestMonitor is an end-to-end test management tool for every organization. A
simple, intuitive approach to testing. Whether you’re implementing enterprise
software, need QA, building a quality app or just need a helping hand in your test
project, TestMonitor has you covered.
Features
● Requirement and risk-based testing.
● Advanced test case design capable of supporting thousands of cases.
● Third-party integrations featuring Jira, DevOps, and Slack. REST API included.
9. AppliTools:
Applitools is an automated testing tool which automatically validates the look and
feels and user experience of the apps and sites. It is designed in such a way that
it easily integrates with the existing tests instead of requiring to create a new test.
Features:
● It is one of the qa tools which allows cross browser test in various devices
10. TestComplete:
TestComplete is an automated test management tool which helps to increase
efficiency and reduce the cost of the testing process. It's very easy-to-use
interface helps QA teams to implement an automation solution in very less
amount of time.
Features:
● It is one of the qa tools that supports multiple scripting languages
11. Ghostlab
Ghostlab is a Mac based testing app that allows test out responsive design
across a variety of devices and browsers. It is a tool for synchronized browser
testing. It synchronizes scrolls, clicks, reloads and form input across all
connected clients to test a full user experience.
Features:
● No setup required for installation
● This mobile testing tool can Synchronised clicks, events and scrolls
12. TestCaseLab
TestCaseLab is a tool for manual QA engineers that allows them to follow testing
activities, creating test cases, categorizing, gathering them in test plans and
starting test runs.
Pros
● User-friendly Test Case Management Tool
The platform has an intuitive UI that allows users to easily create and organize
Test Cases, Test Plans, and Test Runs. For new users, video tutorials and blogs
are available to help you get started.
● Seamless Collaboration
Collaboration can be a challenge if you are using excel sheets for test case
management. Sharing the sheet with your team members may still involve a lot
of work. Worse, they may end up working on a different version of an excel
sheet. With TestCaseLab, you can easily work with other members of your team.
You’ll get a notification when someone edits the same test case.
● Extended Functionalities
TestCaseLab allows its users to easily link their bug-tracking system with the
platform, which speeds up the testing process. You do not need to do your work
twice. The platform also integrates with Redmine, JIRA Atlassian, Pivotaltracker,
YouTrack, Asana, and Trello.
Cons
● "Attaching files (images) to test cases is a little problem."
● "Users sometimes get confused between test cases and test runs."
● "Missing links to test plans. Inconsistent test cases ordering, changes based on."
TestCaseLab Alternatives:
● Testrail
TestRail helps you manage and track your software testing efforts and organize
your QA department. Its intuitive web-based user interface makes it easy to
create test cases, manage test runs and coordinate your entire testing process.
● Zephyr
Manage all aspects of software quality; integrate with JIRA and various test
tools, foster collaboration and gain real-time visibility.
● TestLink
● TestLodge
Traditional test management software is complicated. We’ve given you only the
essential tools to manage your test cases in a simple and efficient way.
● qTest Management
● Other tools:
● Infrastructure
○ Release Management
○ Source Control
● Code Reviews
● Testing
○ Automated Testing & Continuous Integration
○ Usability Testing
○ Load Testing
○ Availability Monitoring
○ Business Analytics
○ Exception Handling
○ Log Monitoring
○ Performance Monitoring
● Customer Support
○ Help Desks
○ Status Pages
INFRASTRUCTURE
● Release Management
○ Deploy
○ dploy
○ Capistrano
● Source Control
○ Beanstalk
○ BitBucket
○ GitHub
CODE REVIEWS
○ Code Climate
○ Hound
○ Pull Review
○ CodeReview
○ Crucible
○ Review Board
TESTING
○ circleci
○ Codeship
○ Semaphore
○ Travis
○ BugHerd
○ FogBugz
○ Jira
○ Redmine
○ Sifter
○ BrowserStack
○ Duo
○ Litmus
○ W3C Validator
○ xScope
● Usability Testing
○ CrazyEgg
○ Silverback
○ UserTesting
● Load Testing
○ Blitz
○ Load Impact
● Availability Monitoring
○ pagerduty
○ Pingdom
● Business Analytics
○ Baremetrics
○ KISSMetrics
○ Mixpanel
● Exception Handling
○ bugsnag
○ Honeybadger
○ Sentry
● Log Monitoring
○ LogEntries
○ Loggly
○ Papertrail
● Performance Monitoring
○ AppSignal
○ Datadog
○ Instrumental
○ New Relic
○ Scout
○ Skylight
○ ArmorHub
○ BugCrowd
○ Tinfoil
CUSTOMER SUPPORT
● Help Desks
○ Desk
○ Groove
○ Help Scout
○ Snappy
● Status Pages
○ StatusPage.io