Beyond Continuous Testing with

Artificial Intelligence
By Wolfgang Platz, Founder and Chief Strategy Officer
 Tricentis | Beyond Continuous Testing with AI | 1

From the golden robots of Hephaestus, to Dr. Frankenstein’s monster, to Hal 9000, we’ve been fascinated by the
idea of artificial intelligence (AI) for centuries. Creating artificial self-directing intelligence is the #2 dream of
mankind—second only to being able to fly like a bird. However, while the Wright brothers were off the ground in
1903, AI hasn’t been ready to take off until very recently. AI requires data + computing power + algorithms. We’ve
had the algorithms for a long time; now, big data and colossal computing power have brought us past the tipping
point.
The feasibility of AI couldn’t come at a better time. Today, “Digital Disruption” is forcing enterprises to innovate at
lightning speed. We need to dedicate resources to creating new sources of customer value while continuously
increasing operational agility. Otherwise, we risk waking up one day to find out—like Nokia—that even though we
did nothing “wrong,” we somehow lost.
The speed of digital disruption is already staggering, and it’s only going to increase. To put this into some very
concrete terms, consider that:

• There are 7.4 billion people in the world

• 4.5 billion have regular access to a toilet

• 5.5 billion own a mobile phone

All of a sudden, a huge number of people jumped from a very provincial lifestyle straight into digital times—
creating a tremendous demand for more, and more innovative, software.
Anyone responsible for producing software knows that the traditional ways of developing and delivering software
aren’t adequate for meeting this new demand. Not long ago, most companies were releasing software annually, bi-
annually, or quarterly. Now, iterations commonly last 2 weeks or less. While delivery cycle time is decreasing, the
technical complexity required to deliver a positive user experience and maintain a competitive edge is increasing—
as is the rate at which we need to introduce compelling innovations.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 2

In terms of software testing, these competing forces create a gap. We’ve turned to Continuous Testing to bridge
the gap that we’re facing today. But how do we test when, over time, these trends continue and the gap widens?
We will inevitably need to go beyond Continuous Testing. We will need additional assistance to be able to deliver a
positive user experience given such high delivery speeds and technical complexity.
We’ve already undergone quite a journey to arrive at Continuous Testing. “Classical” testing was designed for
software delivery cycles that span months (or sometimes even a year). Agile has made 2-week development
iterations the norm—but now even more is needed to meet today’s voracious demand for software. Attempts to
further accelerate the process exposed a chasm between Development, Test, and Operations. That chasm
needed to be bridged with DevOps and Continuous Testing in order to move beyond that acceleration plateau.
Today, the vast majority of organizations are talking about Continuous Testing and trying to implement it.
Nevertheless, when we look into the future, it’s clear that even Continuous Testing will not be sufficient. We need
help.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 3

We need “Digital Testing” to achieve further acceleration and meet the quality needs of a future driven by IoT,
robotics, and quantum computing. AI, imitating intelligent human behavior for machine learning and predictive
analytics, can help us get there.

What is AI?
Before we look more closely at how AI can take software testing to the next level, let’s take a step back and review
what AI really means.
Forrester defines AI as
“A system, built through coding, business rules, and increasingly self-learning capabilities, that is able to supplement
human cognition and activities and interacts with humans natural, but also understands the environment, solves human
problems, and performs human tasks.“
I think the key points in that definition are self-learning (it needs to have learning capabilities) and human cognition
(which is achieved by understanding the environment and context).

Another interesting definition is that AI is

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 4

“A field of study that gives computers the ability to learn without being explicitly programmed.”

One of the key points of AI is that you do not need to explicitly program algorithms. Algorithms are certainly used,
but they’re not designed for an explicit solution of a distinct problem. The machines can learn, and they use data to
do so.
Any discussion of deep machine learning must involve the term “neural networks.” The idea is that you have a large
set of input data, and this input data leads to a narrower set of outputs. To process that data, you use neurons,
which are in between the input and the output. These neurons are assessment points; parameters that help
classify the input.
In a neural network, you don’t have just one layer; you have multiple ones. They need to parameterize themselves,
which means they learn in order to provide the right output results from the inputs.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 5

To make this concept more tangible, let’s consider the example of optical facial recognition. The input layer photos
are sent to a machine, assesses these different pixel areas in the photo to determine if it’s a face:

All the layers in the neural networks are parameterizing themselves—learning how to make the correct
assessment.
In summary, let’s conclude that:

1. If it isn’t learning, it’s not AI

2. If you need to explicitly program the transition from input to output, it’s not learning—and therefore not AI

How Can AI be Applied?

As a rule of thumb, any complex task that a human can solve without consciously making a decision is a candidate
for AI.
The most prominent example of AI is image recognition capabilities. If you look out the window, you instantly
recognize what’s there without having to think about every little detail and consciously analyzing all the patterns.
It’s really an incredibly complex task, but you can do it instantly. It’s a huge challenge for a machine, but it is feasible
with AI. The results of what AI can deliver in this respect are already astonishing. Since the machine is applying
intelligence to a very specific problem, this application falls under the category of “narrow AI.”

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 6

Voice recognition is another example of this. You hear what people say and you immediately understand what
they’re talking about. You don’t need to decompose the pattern of speech; you just know it. This is another area
where AI is already achieving amazing results. Siri, Alexa, Google, and other voice recognition capabilities are
fantastic. The advancements over the past couple years are breathtaking.
More sophisticated (general) applications of AI build on these elementary capabilities. For example, autonomous
driving builds on AI-based visual recognition.
Here’s a more expansive look at the various areas where AI is being used and advanced:

• Human emotions: Analyzing facial features, reacting to human emotions, identifying criminals
• Natural language processing: Contextual understanding, generation, translation, transcription
• Visual and image processing: Visual emotions, activity recognitions, object recognition
• Audio: Voice recognition, sound recognition, intonation
• Negotiation: Negotiate and understand negotiation processes, consider cultural factors
• Robotics: Robot learning, multi-robot systems, multi-legged walking, embodied cognition
• Probabilistic reasoning: Common sense knowledge, reasoning about actions, non-monotonic reasoning

Applying AI to Software Testing

To meet the challenges presented by accelerating delivery speed with increasing technical complexity, we need to
follow a very simple imperative:
Test smarter, not harder.
As I mentioned earlier, we’re fast approaching a time when Continuous Testing will be unable to keep pace with
shrinking delivery cycles times, increasing technical complexity, and accelerating rates of change. We’re already
starting to use basic forms of AI, but we need to continue the testing evolution in order to achieve the efficiency
needed for “Digital Testing” of robotics, IoT, and so forth.
For example, consider how image recognition can be used to bring UI testing to the next level so that dynamic UI
controls (e.g., for responsive sites) can be automatically recognized in all their various shapes and forms.
We can already recognize UI controls from the human perspective, beyond plain template matching. The UI’s pixel
structures can be interpreted with the goal of recognizing image patterns and identification information such as
text. For example, edge detection can be used to identify the following control as a button and OCR/ GDI layers can
be used to identify its text.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 7

Using AI, the software testing tool can learn to identify controls during the scanning and test execution process—
independent of the control size, color, text alignment, etc.. A learning approach is applied: images are generalized
by adding new image patterns and adapting existing image patterns during the scan and test execution process.

Each control knows its context on the graphical interface via anchor identification. Control properties are extracted
from the image pattern, and can in turn be used to constrain automation of certain controls (for example, a
scrollbar) during test execution. This eliminates the need to recognize them from their technical implementation
(ID, etc.).

The end result of this learning? It enables repeatable and stable test execution, even on responsive UIs. This will
be the next chapter of UI test automation.

Not AI—But Does That Matter?

Given AI’s buzzword status, it’s all-too-commonly applied to innovations that are incredibly exciting and valuable—
but are not AI because they don’t involve self-learning.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 8

One of the most prominent examples of this is when IBM Deep Blue beat Gary Kasparov in 1997. It passes the
Turing test1, but it’s not AI. Why? Because it’s explicit programming without learning. The next logical move is
determined by algorithms, but it’s not a self-learning system.
Another achievement commonly mislabeled as “AI” is General Electric’s project where jet engine logs are sent to a
data hub every second and continuously analyzed to determine whether the engines need a sanity check. This
innovation has been able to reduce the need for engine checks by over 60%. Since engine checks are incredibly
costly, this will have a significant impact on the bottom line. Again, this is an incredibly innovative and valuable
computing feat, but it’s not AI.
The trend of labelling any breakthrough algorithmic innovation “AI” applies to the software testing industry as well.
“Self-healing” technologies fall into this category. One of the most interesting self-healing technologies in software
testing is technology that updates broken (out-of-sync) test cases at runtime. There are typically multiple ways to
identify a control (such as a button) that you’re trying to automate. Each control is usually identified by a technical
property, such as an ID, a name, etc.). In many cases, multiple sets of properties are redundantly unique to a
control. For example, assume 2 of the 3 identifiers being used to identify a particular control can no longer locate
it. If just 1 of the 3 identifiers is still current, self-healing technology can be used to identify the control, as well as
update the other 2 identifiers.
I don’t dispute the value of this technology at all. In fact, it’s something we’ve developed at Tricentis, and we have
seen first-hand how it can save testers from having to constantly update test cases as applications evolve. Yet,
despite the “self” in its title, self-healing technology is driven by advanced algorithms, not self-learning. It’s not AI. It’s
what I call “cognitive testing.” Nevertheless, it is a prime example of the type of innovation we need to satisfy the
demands of “digital testing” beyond Continuous Testing.

1
The Turing Test was designed to be a rudimentary way of determining whether or not a computer counts as "intelligent." The test is carried out as a
sort of imitation game. On one side of a computer screen sits a human judge, whose job is to chat to some mysterious conversation partners on the
other side. Most of those conversation partners will be humans; one will be a chatbot, created for the sole purpose of tricking the judge into thinking
that it is the real human.

www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved

v
 Tricentis | Beyond Continuous Testing with AI | 9

Wrap-Up
Unquestionably, we need to learn how to work smarter, not harder. To ensure quality in an era where software will
be processing an unimaginable number of data points in real time—for example, both figuratively driving the
internet of things and literally driving “self-driving” cars—we need all the help we can get! Innovations labelled “AI”
might get all the attention, but we’ll need to extend beyond AI in order to excel. In terms of software testing, some
of the non-AI / “cognitive testing” inventions that I expect to be critical include automated test portfolio
optimization, self-adjusting risk assessment, automated defect diagnosis, and smart environment provisioning.
The bottom line: AI is perfectly poised to advance software testing by automating tasks that involve self-learning
and traditionally required human cognition. We can and must continue to dream about AI and make it a reality—
but we can’t afford to dismiss the role of smart testing technologies that lack the trendy “AI” label.

Learn more about AI and the future of testing

Watch "The Future of Testing" on-demand.

About Tricentis
Tricentis is recognized as having the industry’s most innovative software testing technologies—transforming testing from a roadblock to a catalyst for
innovation. Our Continuous Testing platform is architected to simplify testing of modern enterprise applications. Through risk-based testing, scriptless
end-to-end test automation, and the industry’s most extensive technology support, Tricentis breaks through the barriers experienced with
conventional software testing methods. Tricentis has earned “leader” status in all three top analyst reports. This honor is based on our proven track
record of helping 600+ Global 2000 companies achieve test automation rates of over 90%—increasing risk coverage while accelerating testing to keep
pace with Agile and DevOps.

www.tricentis.com
© 2018 Tricentis GmbH. All Rights Reserved
www.tricentis.com © 2018 Tricentis GmbH. All Rights Reserved