For: Security &

Risk Professionals

Stop Billions In Fraud Losses With

Machine Learning
by Andras Cser, April 6, 2015

Key Takeaways
Legacy Fraud Management Mechanisms Fail In Todays Economy
In an economy that will continue to see new forms of electronic payments, you cant
adapt the risk scoring models or author new static rules of legacy fraud solutions fast
enough to keep up with evolving fraud methods, particularly mobile payment fraud.
Machine Learning Models Reduce The Frequency Of Model Updates
Machine learning models improve their accuracy autonomously based on transactional
data, navigational data, and analyst and investigator decisions. While they take time to
burn in, long term they cost 30% to 50% less to operate than legacy models.
Keep Vendors Honest And Compare Their Results With Existing Scoring
Machine learning is a relatively new field in fraud management. While many of its
algorithms have been in use for some time, unsupervised machine learning is a new
area. Be sure to compare the risk scores vendors generate using machine learning with
your existing scores to avoid any regression.

Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USA

Tel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.com

For Security & Risk Professionals

April 6, 2015

Stop Billions In Fraud Losses With Machine

Learning
Machine Learning Is The Only Way To Keep Up With Resourceful
Fraudsters Committing Cross-Channel Fraud
by Andras Cser
with Stephanie Balaouras and Jennie Duong

Why Read This Report

Security and risk (S&R) professionals specializing in fraud find it increasingly difficult to develop new
behavioral patterns and models to detect the telltale signs of cybercriminal activity across commerce
channels particularly mobile. This is exacerbated by many firms inability to find or afford enough fraud
data scientists. Thus, when S&R pros must adapt their own fraud models, its a slow and inefficient process.
The alternative is to wait for vendors to update the models in their commercial solutions. Both scenarios
leave some businesses vulnerable to significant fraud losses for an extended period. In addition, with legacy
models there is no way for S&R pros to know when the accuracy of the model has deteriorated. This is why
there is so much excitement at the prospect of applying machine learning methods, algorithms, and models
to fraud management. The hope is that machine learning will drastically reduce model update cycle times,
which will not only improve fraud detection but give fraud analysts and investigators more time to focus
their efforts on investigating suspicious transactions. In this report, we examine the promised benefits of
machine learning and provide an overview of the vendors incorporating it in their technology.

Table Of Contents

Notes & Resources

2 Legacy Fraud Detection Methods Will Cost

Firms Billions In Losses

Forrester interviewed 12 vendor and

user companies: Accertify, ACI, BAE, CA
Technologies, Feedzai, IBM, Kaspersky
Lab, RSA, SAS, Skytree, ThreatMetrix, and
WorldPay.

3 Machine Learning Overcomes The

Limitations Of Legacy Methods
8 Case Studies In Machine Learning And Fraud
11 Navigate The Vendor Landscape
recommendations

14 Track Machine Learning Efficiency In Vendor

Proof-Of-Concept Demos
WHAT IT MEANS

Related Research Documents

Big Data In Fraud Management: Variety
Leads To Value And Improved Customer
Experience
The Forrester Wave: Enterprise Fraud
Management, Q1 2013

15 Machine Learning Will Extend To All Types Of

Transaction Fraud
16 Supplemental Material

2015, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available
resources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar,
and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To
purchase reprints of this document, please email clientsupport@forrester.com. For additional information, go to www.forrester.com.

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

Legacy Fraud Detection Methods Will Cost Firms Billions In Losses

According to several studies, eCommerce fraud loss as a percentage of revenue ranges between
.85% and .9%.1 Forrester estimates 2014 online retail sales of $294 billion in the US, CD$38 billion
in Canada, and $5.7 billion in Mexico; given this, we can estimate that 2014 online fraud losses
in just North America to be approximately $2.7 billion.2 Without an improvement in the fraud
loss rate, North American eCommerce fraud losses will increase to approximately $4.2 billion by
2018. However, reducing fraud losses will be challenging. According to Forresters North American
Consumer Technographics Financial Services Survey, 2014, 72% of US online adults use at least
one form of electronic payment regularly. In an economy that will see continued adoption of
multiple cashless payment options, its very difficult to detect and prevent fraud across multiple
channels, including web, mobile, phone, in-person, and kiosk, and across payment types. Legacy
fraud techniques and methods are failing S&R pros because these techniques:

Cant detect fraud quickly enough given an avalanche of customer data. Mobile devices,

cross-channel interactions, and customers social media activity generate an enormous amount
of data. Using legacy tools to identify fraud trends while protecting against new and emerging
fraud patterns is difficult, if not impossible, in an environment of fast-streaming, nonpersisted,
and real-time data in huge volumes from a variety of sources such as clickstreams,
geolocation information from mobile devices, Facebook, and Twitter posts.3 Fraud management
solutions and tools that require extensive, supervised training cant keep up with these new
fraud patterns.

Have yet to adapt fraud patterns to account for the popularity of mobile commerce. Many

legacy enterprise fraud management (EFM) platforms are only suites for traditional payment
fraud detection and cant easily integrate location, IP address, and social network data not to
mention sensor data such as acceleration and application use patterns churned out by mobile
devices. Baseline differences from these data sources often indicate fraudulent activity.

Cant easily identify new cross-channel fraud patterns. Cybercriminals love to use the mobile

channel to perpetrate fraud. The telephone/call center, in-person, and even ATMs are also still
popular channels. In fact, fraudsters often take advantage of multiple channels to commit moresophisticated and -complex schemes.4 For example, a fraudster can sign up for online access,
change the mailing address of the account online, then order an ATM card to drain the account.
Thus, actions that may look innocuous on one channel suddenly become fraudulent if observed
in the context of multiple channels.5 To tackle cross-channel fraud, legacy fraud management
techniques (such as statistical models and rule sets) wont suffice: They are slow, hard to retrain
and maintain, and produce high false positive rates.

Require the work of expensive, hard-to-find data scientists to be successful. Forrester

interviewed a small, regional bank that said that its top concern regarding EFM implementation
was the complete lack of data scientist employees and the prohibitively high labor costs of

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

such skilled workers. Only the largest banks can afford to employ armies of data scientists to
continually refine statistical models and maintain rule sets in EFM solutions. To curb fraud
losses and control the cost of manual fraud identification, analysis, and investigation, small
and regional banks need to find alternative, less labor-intensive but equally effective, fraud
management solutions.
Machine Learning Overcomes The Limitations Of Legacy Methods
Machine learning is a type of artificial intelligence (AI) that gives computers the ability to learn
without human programming.6 Machine learning focuses on the development of computer programs
that can teach themselves to grow and change when exposed to new data. Machine learning works by
building automated analytical models using an iterative mathematical algorithm that learns from its
mistakes in previous iterations using new data supplied.7 Heres whats different about machine
learning compared with traditional fraud detection methods. Machine learning solutions:

Support real-time decision-making. Ten years ago, most banks and eCommerce firms were

able to live with nightly batch fraud screening processes. One of the biggest changes in EFM
is the shift toward real-time decision-making and interdiction (less than a 1-second response
time). Today, because of increased fraudulent activity and competition and faster (often
instantaneous) bank transfers, this is no longer viable. The norm is real-time. Fortunately,
machine learning algorithms coupled with powerful hardware architectures can support this
requirement. Many algorithms that were once unfeasible to implement on old hardware, such as
ensemble models and complex, neural networks coupled with in-memory processing, became
operationalized weapons in the arsenal for fighting fraud.

Dont rely on static rules, the manual adjustment of model thresholds, or blacklists. Machine

learning is not about maintaining rule sets for score adjustments or business decisions. It doesnt
rely on continuous human programming to manually adjust thresholds in statistical models or
to maintain whitelists and blacklists. It also doesnt rely on fuzzy matching using a predefined
algorithm, or the simulation of what-if scenarios.

Require much less training than statistical models. Older models that are statistical-based

systems require extensive supervised training. This requires fraud analysts to feed six months
of training data into the system in order to tune the statistical model to identify known,
confirmed fraud. This is a slow, expensive, skill-intensive, and rigid process. Machine learning
models are ideal for unsupervised training. In this case, analysts feed production data into the
system and allow it to learn and adapt continuously as it identifies anomalies and incorporates
analyst and investigator feedback from confirmed fraud cases.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

Require less maintenance. Older, rules-based systems are very high maintenance. Legacy

fraud management systems often have 100 to 200 static rules. They require extensive and costly
manual maintenance, such as tuning of thresholds, and, based on our interviews, have a 30%
to 40% slower throughput at the same false positive rates than comparable machine learning
systems. Forresters interviewees said that fraudsters immediately find rule thresholds, such as
$300 of funds available immediately after depositing a check, and exploit them quickly, thus
making rules less effective. Machine learning solutions dont require rules and thus have a lower
maintenance cost than systems with explicit rules.

Feed on large data sets and improve their accuracy with time. The more data you feed to a

machine learning algorithm, the better it becomes. Thus, big data makes machine learning not
only possible but also more reliable than legacy models that could consider only much smaller
data sets. In fact, the availability of rich contextual data from mobile operating systems and
applications only serves to improve EFM systems powered with machine learning: The system
can identify subtle fraud patterns based on geolocation, device usage patterns, application
network traffic, and other contextual data.

Learn from analyst and investigator feedback. Machine learning models can learn from

analyst and investigator feedback and knowledge of prior transactions. If an investigator marks
a case fraudulent, the machine learning can leverage that information in reviewing subsequent
transactions with similar parameters to improve decisioning. In the first year after deployment
of Accertifys solution, European airline easyJet cut the percentage of fraud loss on revenue by
29%, and in the second year by 39%.8

Advanced Techniques Improve Detection And Automate New Pattern Identification

Machine learning differs from legacy statistical and rules-based models in a number of other ways.9
Some features and functions help to improve detection and automate the identification of new fraud
patterns at a lower cost. Machine learning:

Can automatically identify fraud patterns faster in transaction streams. Financial crime and

compliance applications of machine learning most often relate to pattern classification in which
the goal is to divide data into groups that convey some concept of reputational, regulatory, or
financial risk. Machine learning systems can identify relationships and causality between input
variables (such as IP address) and an output variable (such as fraud or not) much faster than
legacy methods can. Identifying patterns in transaction streams allows EFM solutions with
machine learning to flag transactions that are suspicious based on trending. An example trend
might include a fraudster trying to withdraw the following amounts in order $1,000, $500,
$300, $100 to establish the fraud limits of the ATM.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

Allows for classification and grouping of transactions. With classification, the model

separates transactions into two or more classifications, such as fraudulent transaction,

transaction to review, and legitimate transaction. Then, the algorithm produces a model that
makes the call on unseen transactions and places them into the above classifications. Regression
predicts continuous outputs (e.g., How much is the predicted loss for a transaction?).
Clustering identifies previously unknown groups based on data only by looking at transactions
and then attempts to put unknown transactions into the identified groups. The result?
Classification allows EFM tools to predict if a particular transaction will be fraudulent or not
based on its attributes like dollar amount and location.

Can operate supervised or unsupervised. Supervised learning means that the system attempts

to identify data elements on its own that have been labeled fraudulent by data scientists training
the solution. This iterative process requires some manual tuning of the model by a data scientist.
Unsupervised learning means that no one gives labels or indicators to the model, allowing
the model to find structure automatically in the input. Classification and regression can be
both supervised and unsupervised, while clustering is typically unsupervised. The benefit of
unsupervised algorithms is that they dont require extensive human labor for training and are
less costly to maintain.

Identify predictor features automatically. Machine learning algorithms and methods can

identify predictor features automatically. For example, for a North American regional bank,
contextual authentication machine learning has in the past identified the following predictor
features automatically: 1) time the user has been known as a customer to the bank; 2) number
of transactions in the past 10 minutes; 3) number of times the user is transferring money to
the destination account; and 4) how typical this geolocation is for the user (deduced from the
IP address). An EFM solution employing this method will have a lower cost of finding features
than data scientists manually trying to identify predictor features.

Random Forests, Deep Learning Algorithms Advance In Machine Learning

Machine learning algorithms matter. While regression methods and Bayes predictors have been
around for a long time, S&R professionals dont consider them robust enough for machine learning.
Here is a quick overview and the pros and cons of the most common machine learning algorithm
solutions for fraud management:10

Random forests are very well understood and fast to learn and score. Random forests

are based on decision trees and are one of the oldest machine learning technologies.11 Data
scientists usually use them in credit and fraud scoring among many other applications such as
medical diagnosis. Pros of the method include: 1) It can handle missing data; 2) its analytics is
robust enough to resist the skewing of outliers; 3) it requires little tuning; and 4) its fast to train

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

and score. Challenges are really few and easy to overcome, but they are: 1) One cannot easily
interpret decisions as rules or single trees; 2) the input requires labeled data; and 3) it handles a
high number of features poorly.

Deep learning or neural networks provide excellent predictive power but are a black box.

Neural networks (or more recently called deep learning systems) are complex nonlinear
models with very large numbers of parameters. The prediction equations are generally sums of
exponentials. The equations behind neural nets tend to be so complex that they are effectively
a black box. The highest performance fraud and credit scoring tools routinely employ some
type of neural network. The benefits of neural networks include: 1) They have the ability to
represent complex patterns; 2) they can provide great predictive power; and 3) they are parallel.
The downsides are: 1) They have difficulty handling different input types; 2) they cant handle
missing data values; 3) theyre slow; 4) they require extensive tuning and retuning; and 5) they
are hard to interpret.

Support vector machines (SVMs) decide which population a transaction belongs to. SVMs

assume that the outcome it predicts is binary, so repeated use is required when predicting
multiple outcomes. Good data preparation is essential. Data scientists use them in credit card
fraud management and credit scoring. Benefits of SVMs include: 1) They have the ability to
detect nonlinear patterns; 2) theyre effective with many features (e.g., high dimensionality); 3)
they have good predictive power; and 4) theyre not as prone to overfitting as neural networks.
Challenges include: 1) SVMs cant handle missing values; 2) theyre hard to scale; and 3) it is
hard to find optimal kernels.12

Clustering algorithms perform well in unsupervised learning and group observations.

Clustering algorithms include KMeans, KMediods, and Kohonen (self-organizing) maps all
based on the KMeans algorithm. Data scientists use them for supervised learning in an iterative
fashion to segment data in very large sets. Benefits of clustering algorithms include: 1) They
have a high tolerance to missing data and outliers; 2) they have good predictive powers; 3) they
offer fairly easy graphical representation; and 4) can reduce dimensionality quickly. Challenges
of clustering algorithms are: 1) They have an inability to handle different types of input; 2) they
require tuning; 3) theyre hard to interpret; and 4) KMeans requires specification of the number
of clusters.

Machine Learning Improves Fraud Management Performance

Good fraud management is becoming a competitive differentiator among banks, insurance
companies, and healthcare organizations. More and more customers understand, look for, and
demand robust security and fraud management practices at a bank. In light of the recent credit card
breaches at Home Depot, Neiman Marcus, Target, and others, S&R professionals are increasingly
paying attention to and becoming concerned about solid fraud management practices that keep the

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

customer experience intact. Your firm/s customers, who are rightfully worried about their online
security and privacy, are more and more likely to go to a competitor if you cant protect them. In
order to find the tell-tale signs of fraudulent activity in the enormous data sets collected from
mobile devices, phone conversations, and transactions conducted across many different channels,
S&R professionals require machine learning algorithms because they:

Can catch behavior changes and patterns as they occur. You can only defend against threats

you can identify. Machine learning helps by automatically identifying changed behaviors and
detecting new patterns long before an army of human analysts (or in fact older, typology-based
algorithms) could. Machine learning allows for quick and effective ingestion of large data sets
and allows for checking out different data segmentation types and what-if scenarios quickly
and efficiently. Many times, machine learning can identify counterintuitive results that human
analysts cant. Machine learning can also identify patterns that senior management can use to
make strategic decisions such as reorganization or a shift in go-to-market-strategy.

Allow you to measure the accuracy of your fraud model. Machine learning methods allow

you to measure how accurately you can predict the outcome of your classification. This is very
important, especially with real-time interdiction when you may have to block transactions
if you have a high confidence of a transaction being fraudulent. This can greatly contribute
to reducing false negative rates and ensures that you dont overlook the truly high risk and
fraudulent transactions. Machine learning applied in iterations can also greatly improve data
quality and fill out missing feature values in records.

Improve the performance and scale over rules-based or manual methods. Vendors Forrester
interviewed for this report said that machine learning algorithms, especially when combined
with each other, tend to perform 30% to 40% faster at equal false positive rates than equivalent
rules-based and other AI systems. Given the recent drop in prices and explosion of computing
power, many machine learning algorithms that have been previously cost-prohibitive from a
computing perspective have become commonly available and thus significantly refined and
more accurate.

Require no programming and tuning with unsupervised machine learning. Unsupervised

machine learning (available with decision trees and random forests) goes a long way in those
scenarios where a fraud team has no access to previous training truth data or when there
is a lack of data scientists. Since most machine learning tools built into commercial fraud
management platforms require no programming, they allow for a much lower cost and faster
implementation of robust, industrial-strength EFM.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

Case Studies In Machine Learning And Fraud

The following are three enterprise case studies demonstrating the best practices and benefits of
machine learning in fraud management:

Global financial services companies monitor trading using self-organizing maps. Over the

past decade, over $15 billion in losses have been through rogue trading activity. BAE Systems
developed and implemented the NetReveal Unauthorized Trading detection solution to address
this at a number of companies; NetReveal uses unsupervised learning in the form of selforganizing maps. BAE used unsupervised SOM (self-organizing maps) as neural network
methods to detect unauthorized trading. Maps show traders activity features like volume of
trades, most frequently traded products, and other key risk indicators. It uses a neural network
to automatically cluster traders exhibiting similar behaviors. This allows the solution to identify
significant changes in trader behavior (e.g., a front-office trader behaving like a back-office
trader). The solution has been proven to identify unauthorized trading before significant losses
occur and predicts trader misbehavior for about six months giving ample time to the bank to
avoid actual losses (see Figure 1).

A US credit issuer uses high-performance models to cover cross-channel fraud. A top US

credit card issuer with 50 million accounts with a multilayered authorization system struggled
to improve fraud detection without affecting customer experience. The issuer had both in-house
and multiple third-party commercial fraud risk scoring engines. The growth in EMV smart
card adoption led to fraud shifting to card not present transactions on alternative channels
and online banking. Existing fraud risk scores were colliding, and this in turn resulted in a
poor customer experience. A team of three full-time equivalents (FTEs) chose Feedzai random
forest, SVM, and other machine learning models to define model features, extract training
samples, and test performance in three weeks. The issuer improved detection rates by more
than 40%, for a $125 million increase in savings. The solution detected that 68% of the issuers
fraud was cross-channel, and its fraud models achieved a 20 millisecond response time at 10,000
transactions per second.

A US bank improves anti-money-laundering (AML) efforts with hybrid models. A US

bank, seeing high false positives in AML alerts, was frustrated with its inefficient AML
investigation. The bank deployed SAS AML solution and applied a hybrid model of supervised
and unsupervised logistic regression and decision trees. The hybrid model allowed the bank to
simplify and reduce its filing workload of suspicious activity reports (SARs) using a transparent
and auditable operational process that was also easily explainable to auditors. The solution
produced $1 million savings in AML investigations in the first year, reduced the number
of work items by 46%, and allows for autotriage of work items in queue in an objective and
repeatable manner.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

Stop Billions In Fraud Losses With Machine Learning

Figure 1 Example Of Kohonen Map Interface

Bar chart of average
trader activity in
cluster

OTHER

OTHER

OTHER

OTHER

OTHER

OTHER

120912

OTHER

SALES

OTHER

OTHER

OTHER

OTHER

Cluster of traders with

similar behavior

SALES

OTHER

OTHER

OTHER

OTHER

SALES

SALES

SALES

SALES

OTHER

OTHER

SALES

OTHER

TRADING

SALES

SALES

OTHER

TRADING

TRADING

SALES

SALES

SALES

TRADING

TRADING

Source: Forrester Research, Inc. Unauthorized reproduction or distribution prohibited.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

10

Stop Billions In Fraud Losses With Machine Learning

Prepare For The Speed Bumps: Supervised Machine Learning Can Be Tough To Set Up
Although effective, machine learning methods arent the end-all-be-all of solutions. If youre
considering using machine learning to improve fraud management, you should be aware that:

You may struggle to set up supervised machine learning. Supervised machine learning can be
hard to set up because it requires reliable training data and time. Over-fitting the model (seeing
patterns and signals where there are none) can be a problem as well. Neural networks can
require a fair bit of computing resources to solve complex problems.

Machine learning solutions require a burn in time. Since models improve as more data is fed

to them, it takes time (a few months at least) for the model to reach its target accuracy, as the
model needs to capture fraud patterns hidden in the data. S&R pros can avoid this by using parallel
and legacy methods for risk-scoring during this silent period. Unsupervised learning algorithms
usually require a lot of data before they can be as accurate as supervised learning algorithms.

Missing and outlier data or too many features can mislead some algorithms. Especially with

unsupervised training, and neural networks and support vector machines, missing feature
values or incorrectly captured, erroneous feature values can cause the model to go astray, which
can require manual intervention and retraining. When the number of dimensions of the model
increases, the models size becomes large and sparse. This scarcity is problematic for any method
that requires statistical significance. So, with greater volumes of data, results can be more
difficult to correlate and irrelevant features can impede the model. Machine learning algorithms
can also produce spurious and unstable results.

Easy of interpretation can suffer, leading to hard-to-explain fraud decisions. Machine

learning methods dont always produce clear-cut explanations as rules-based systems or

Bayesian predictors. The logic that the learning algorithm identifies is hard to explain, which
causes natural tension between accuracy and interpretability. As a result, even though the model
may be very accurate, the score may be hard to explain, which causes a perception of black-box
decision-making. Thus, fraud management professionals may face difficulties explaining risk
scores to affected customers, regulators, and auditors.

Machine learning requires big data expertise and tools. The percentage of transactions that

are fraudulent in a given payment activity or online banking system is small. Given that most
transactions are genuine, the identification of the patterns associated with fraud using machine
learning requires the analysis of very large sets of data. Big data analytics involves technologies
that facilitate the economic and rapid analysis of these data sets. Traditional statistical methods
cant handle big data really well. This mandates that the company deploys and maintains a tall
technology stack, consisting of many systems to pull in from and process data, including DBMS,
Hadoop jobs, NoSQL data stores, Python, R, Weka, and streaming systems.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

11

Stop Billions In Fraud Losses With Machine Learning

Navigate The Vendor Landscape

Vendors increasingly incorporate machine learning, especially unsupervised machine learning, into
their offerings. Every vendor has to compete in multiple geographies where some of their clients
are reluctant to provide training data for tuning the vendors EFM model so machine learning
algorithms increasingly substitute training data. Forresters interviews with vendors indicate a
dynamic landscape in which:

Accertify uses deterministic and nondeterministic algorithms. Through its fraud solution,

clients that opt in can leverage positive and negative experiences across all participating
Accertify clients. Accertify Index leverages a cross-client statistical model to provide a score
to indicate the correlation of one or more data elements to a positive or negative experience
by one or more clients. Accertify also creates custom machine learning models for clients,
leveraging the many custom variables that clients send that are distinct to their business to
provide improved and customized decisioning. Medium-to-large eCommerce companies use
the vendors solution.

ACI tightly integrates its payment risk management offering with payment solutions. As

part of the Universal Payments Framework, and building on ReDs existing models, ACIs
solutions enable machine learning to use transactional and peripheral data from the entire
transaction life cycle, but ACIs machine learning algorithms do not substitute for training data.
The solution then creates and fine-tunes the detection triggers, which improves response times
and accuracy. ACI has invested heavily in analytics research to further automate the creation of
rules and modeling capabilities. Medium-to-large banks and financial services companies use
the vendors solution.

BAE offers an integrated solution with a broad selection of models. The NetReveal machine

learning is a fully integrated solution that contains configurable analytics. The solution offers
transparent models for feature relevance and dimensionality reduction, logistical regression,
frequency profiling and outlier analysis, text mining and sentiment analysis, as well as rule
induction, neural networks, stochastic gradient-based algorithms, and unsupervised clustering.
The single view of the customer allows investigation not just at the level of the individual, but
also around the company he/she keeps. Future plans include automating online learning,
as offline approaches are increasingly less effective; automated model drift detection and
correction; and improved visualization tools.

CA Technologies adds machine learning to 3D Secure implementation. CA Risk Analytics

uses machine learning to automatically create features from inputs in the current transaction
and the behavior distillates on users. These features are used to power the neural network model
suite that produces risk scores for making real-time operational decisions to stop authentication
fraud. CA Technologies neural networks extend functionality of CA Risk Analytics to examine
patterns across multiple issuers.13 CA Technologies uses machine learning in its risk-based

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

12

Stop Billions In Fraud Losses With Machine Learning

authentication and 3D Secure processing solutions. It plans to use machine learning to: 1)
augment Risk Analytics with models for multiple international regions and 2) use real-time data
from multiple issuers for fraud risk scoring.

Feedzai scales based on industry standard big data Hadoop and NoSQL/Cassandra

platforms. Feedzais Fraud Prevention Platform is a self-serve, end-to-end, big data modeling
environment. The solution links online and offline behavior patterns to increase profile accuracy
and maintains individual baselines of expected behavior regardless of channel. It also allows
for white-box, model-based continuous machine learning: It continuously rebuilds behavioral
profiles and continuously updates fraud patterns in real time.14 The vendors plans include:
1) easier and faster feature design using a point-and-click data science framework; 2) deep
learning anti-fraud models that are closer to artificial intelligence and mimic how the human
brain works; and 3) cloud API improvements.

IBM connects fraud management, security, and marketing BI with machine learning.

IBM launched a new Counter Fraud initiative in March 2014.15 Based on SPSS, the solution
incorporates standard and proprietary machine learning and other types of algorithms and
can integrate with R algorithms.16 It offers anomaly detection, and entity analytics algorithms
establish relations between entities. IBM plans to: 1) invest in developing new algorithms; 2)
integrate information security, identity and access management, and AML; 3) use Watson and
other artificial intelligence techniques to process data; 4) apply machine learning algorithms to
large, in-memory data sets and evaluate models against those faster; and 5) use the cloud as a
delivery and computing platform.

Kaspersky Lab plans to use machine learning for biometric analysis against fraud. Kaspersky
Lab uses machine learning techniques to identify global trends in malware, fraud in social
networks, phishing campaigns, and fraudster behavior analysis. Kaspersky Labs access to
advanced threat research provides a foundation for selecting which data to analyze and which
machine learning models to apply to a given problem. Machine learning will take a central role
in the future Kaspersky Fraud Preventions behavioral biometric analysis capabilities, including
navigation, mouse clicks, and historical behavior.

RSA uses Bayesian models in its risk engine across the board. RSA hosted and on-premises

Adaptive Authentication solutions risk model is self-learning it learns from case resolution
as well as genuine or failed authentication feedback and online live shared fraud intelligence
data. The risk engine modifies its risk predictions based on case investigation results and
authentication feedback then automatically updates the risk model to catch fraudulent activities
that were missed or genuine activities that were wrongly flagged. RSA plans to: 1) enhance its
device identification with machine learning; 2) enhance the use link analysis; 3) allow customers
to use custom predictor features; and 4) use machine learning to create unsupervised algorithms
for behavior anomaly detection.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

13

Stop Billions In Fraud Losses With Machine Learning

SAS combines and autoselects multiple models in its solution for the best fraud detection.

SAS Data Mining, SAS Visual Scenario Designer, and SAS Model Manager all work using
multiple approaches to look for the best fraud detection lift, and then the technology
recommends the best approach as opposed to a human bias that might adversely affect the
results. In addition to SAS many machine learning methodologies, the capabilities supporting
the model life cycle within the SAS solutions support data driven detection. Using its big data
analytics platform, SAS plans to: 1) use ML tools in its new enterprise fraud management
models; 2) make analytics more accessible to all business users by using visual interfaces; and 3)
improve performance by using in-memory processing.

Skytree develops high-performance machine learning algorithms using big data. Skytrees

specialty is creating high-performance and scalable machine learning (KMeans, SVM)

algorithms.17 The vendors solutions are used for fraud and credit risk scoring of customers as
well as pricing and churn analysis. The solution can identify the point of compromise by using
big data sources and providing real-time insight into patterns. At a credit card network, Skytree
helped the client move from annual model updates to daily or real-time model updates on nofrills Linux x86 hardware. Skytrees focused on allowing analysts to create models rather than
forcing them to write software.

ThreatMetrix provides web fraud with machine learning in its cloud platform. In its

largely banking- and eCommerce-targeted solution, ThreatMetrix collects malware-related

information and information on device data, login, user name, address, phone number, email,
product purchased, amount, and quantity. The solution uses machine learning algorithms on
anonymized data to uniquely identify devices and to generate trust scores based on global
behavior patterns. This year, ThreatMetrix is expanding this platform to include both supervised
and unsupervised learning and create machine-learning-based real-time rules. API and userinterface-based analyst feedback refines the algorithms. ThreatMetrix TrustDefender Platform
will combine integrated end-point intelligence, anonymized identity and context intelligence,
visualization, case management, workflow, and machine learning.

WorldPay collects data from its payment gateway, fraud screening, and acquirers. Models

comprise sector-built profiles, global velocity controls, global split cross-reference controls,
global fraud data pooling, custom rules, merchant and sector-built rules all of these alongside
ongoing merchant review/chargeback reviews. While RiskGuardian today does not offer all of
the mathematical algorithm capability as may be technically defined for a machine learning
system, WorldPay is planning to partner with a machine learning algorithmic-based services
provider to run proof of concepts around the business case for implementing machine learning
in its solutions in the first half of 2015.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

14

Stop Billions In Fraud Losses With Machine Learning

R E C O M M E N D AT I O N S

Track Machine Learning Efficiency In Vendor Proof-Of-Concept Demos

Finding the right vendor to convert your legacy EFM portfolio to one powered by machine learning
is no easy task there are a lot of misconceptions and exaggerated vendor claims. Forresters
conversations with EFM practitioners highlight that S&R pros should:

Demand proof that the vendor can do unsupervised learning fast. Especially if your

fraud team lacks training data, data scientists, and statistician talent, the solution needs to
be able to perform self-learning on existing offline data and streaming online data, and you
must be able to deploy it in production in no more than four to five months. If you already
invested in supervised machine learning, try using it in parallel first with unsupervised
machine learning tools, and then shift your fraud management portfolio toward
unsupervised methods; if they produce the same results as supervised algorithms, you will
save on supervised machine learning training costs.

Ask for visibility into machine learning algorithms. The whole point of using machine

learning is to be able to avoid black box fraud risk scoring. No matter how complex, a
machine learning algorithm should be understandable and customizable to your analysts
and data scientists. While vendors will offer their extensive professional services for model
creation and updates, its much less costly to quickly become operationally self-sufficient in
this area.

Insist on free proofs of concept to get a taste of implementation complexity. Many

vendors dont offer free proofs of concept in EFM with machine learning algorithms. They
claim that their solution has been proven to work with many previous clients, and therefore,
proof of concept is unnecessary. In many instances, this behavior implies that extensive
customization and coding is required to get the solution off the ground. Any vendor
claiming to have a working, shrink-wrapped EFM with machine learning platform should
be able to stand up a simple proof-of-concept environment in two to three weeks.

Pool data within and across banks. Machine learning algorithms feed on data. S&R and

fraud management professionals need to work with network security, marketing, finance,
and other departments within the company to create an integrated data warehouse that
can feed and improve machine learning-based fraud management. Smaller eCommerce
companies and banks can also benefit from sharing anonymized and encrypted transaction
information, whitelists, and blacklists with each other to build higher quality input data for
machine learning algorithms.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

15

Stop Billions In Fraud Losses With Machine Learning

W h at I t M eans

Machine Learning Will Extend To All Types Of Transaction Fraud

Retail banking already uses many of the above machine learning techniques to prevent credit card
fraud and is slowly extending to other forms of transactional fraud such as payments and checks.
The larger volume of data from the online and mobile channel is stimulating demand for online,
real-time, adaptive learning solutions. In the future, we expect that machine learning will:

Expand into corporate banking, wealth management, and investment trading. The risks
here are so large that banks are nervous about being left behind. Unlike retail banking,
the levels of known fraud incidents are low in number (although high in value); hence,
supervised techniques are in use and the focus is on unsupervised approaches where the
goal is to find interesting patterns in the data without knowing in advance what to look for.

Become a tool for fraudsters too. Fraudsters are employing technologically sophisticated

systems: They are using distributed systems, internal knowledge, big data, and even machine
learning to detect weak spots and to discover ways to maximize their attacks. Old-school
defenses are not the safer bet; in fact, they are anything but. Security teams and dedicated
fraud teams that want to defend themselves against fraud need to have a solution that is
better than their industry average, because fraud flows to the weakest points. In addition,
your teams need to constantly evolve their solutions because fraud patterns change quickly
and you need to have solutions that abstract away the complexity of their software antifraud
solutions (arm fraud analysts with power of data science).

Migrate to the cloud for faster adoption and greater effectiveness. The on-premises EFM

transactional monitoring systems of the past live in a semivacuum: They can only learn from
transactions, fraud, and analyst decisions that affect the firm where they are implemented.
Cloud-based EFM solutions of the future can share machine learning algorithm parameters,
best practices, and hotlists and inform all solution clients proactively of new fraudulent
activity without the need for a model update. Collaborating (instead of competing)
on security and fraud by sharing data securely and privately will benefit all banks and
eCommerce companies.

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

16

Stop Billions In Fraud Losses With Machine Learning

Supplemental Material
Companies Interviewed For This Report
Accertify

Kaspersky Lab

ACI

RSA

BAE Systems

SAS

CA Technologies

Skytree

Feedzai

ThreatMetrix

IBM

WorldPay

Endnotes
Sources: 2014 LexisNexis True Cost of Fraud Study, LexisNexis, August 2014 (http://www.lexisnexis.
com/risk/downloads/assets/true-cost-fraud-2014.pdf) and 2014-2015 Online Fraud Management
Benchmark Study, CyberSource, 2014 (http://www.cybersource.com/resources/collateral/Resource_Center/
whitepapers_and_reports/CYBS-Fraud-Benchmark-Report.pdf).

For more information about the projected US and Latin American eCommerce market, see the US
eCommerce Forecast: 2013 To 2018, Forrester report and see the Latin America eCommerce Forecast,
2014 To 2019 Forrester report.

Big data will only increase the effectiveness of fraud management and regulatory compliance and in turn,
directly improved the overall customer experience. For more information, see the Big Data In Fraud
Management: Variety Leads To Value And Improved Customer Experience Forrester report.

Companies continue to lose money due to fraud issues such as chargebacks, uncoverable transfers, and
time-intensive investigation on fraudulent transactions. For more information, see the Market Overview:
Fraud Management Solutions Forrester report.

For example, consider this scheme. Fraudster: 1) steals a persons bank account number and other
personally identifiable information; 2) calls the bank contact center and establishes an online presence for
the victims account; 3) changes the address on the account online; 4) calls into the call center and orders a
new ATM/ debit card; and 5) empties the account at an ATM. If S&R and fraud management professionals
only look at the phone channel, this fraud scheme is not detectable.

Source: Machine learning, WhatIs.com (http://whatis.techtarget.com/definition/machine-learning).

Source: COS 511: Theoretical Machine LearningMachine learning, Wikipedia Princeton University
(http://www.cs.princeton.edu/courses/archive/spr08/cos511/scribe_notes/0204.pdfhttp://en.wikipedia.org/
wiki/Machine_learning).

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

For Security & Risk Professionals

17

Stop Billions In Fraud Losses With Machine Learning

Source: How Accertify helped easyJet use fraud screening to cut their fraud loss on revenue by 39%,
Accertify (http://www.accertify.com/Publications/Case-Studies/easyJet1/).

Source: Rich Caruana and Alexandru Niculescu-Mizil, An Empirical Comparison of Supervised Learning
Algorithms, Cornell University, Department of Computer Science (http://www.cs.cornell.edu/~caruana/
ctp/ct.papers/caruana.icml06.pdf).

For a more detailed overview of machine learning algorithms, visit the following URL. Source: Basics of
Machine Learning, The University of Edinburgh School of Informatics (http://homepages.inf.ed.ac.uk/
vlavrenk/iaml.html).

10

For more information on decision trees, visit the following URL. Source: Decision Trees, MindTools
(http://www.mindtools.com/dectree.html).

11

Kernel methods owe their name to the use of kernel functions, which enable them to operate in a highdimensional, implicit feature space without ever computing the coordinates of the data in that space, but
rather by simply computing the inner products between the images of all pairs of data in the feature space.
Source: Thomas Hofmann, Bernhard Scholkopf, and Alexander Smola, Kernal Methods In Machine
Learning, Institute of Mathematical Statistics, 2008 (http://www.kernel-machines.org/publications/
pdfs/0701907.pdf

12

The sharing of data depends on how the contract between the vendor and the issuer is structured.

13

A white-box model means that the vendor exposes and documents how the model works and how model
variables impact the algorithm, and this documentation is available to the customer organization.

14

For more information, see the Quick Take: IBM Announces A Streamlined Fraud Management Portfolio
Forrester report.

15

Source: The R Project for Statistical Computing (www.r-project.org).

16

The vendors claims that its models are one to two orders of magnitude faster than competitors models.

17

2015, Forrester Research, Inc. Reproduction Prohibited

April 6, 2015

About Forrester
A global research and advisory firm, Forrester inspires leaders,
informs better decisions, and helps the worlds top companies turn
the complexity of change into business advantage. Our researchbased insight and objective advice enable IT professionals to
lead more successfully within IT and extend their impact beyond
the traditional IT organization. Tailored to your individual role, our
resources allow you to focus on important business issues
margin, speed, growth first, technology second.
for more information
To find out how Forrester Research can help you be successful every day, please
contact the office nearest you, or visit us at www.forrester.com. For a complete list
of worldwide locations, visit www.forrester.com/about.
Client support
For information on hard-copy or electronic reprints, please contact Client Support
at +1 866.367.7378, +1 617.613.5730, or clientsupport@forrester.com. We offer
quantity discounts and special pricing for academic and nonprofit institutions.

Forrester Focuses On
Security & Risk Professionals
To help your firm capitalize on new business opportunities safely,
you must ensure proper governance oversight to manage risk while
optimizing security processes and technologies for future flexibility.
Forresters subject-matter expertise and deep understanding of your
role will help you create forward-thinking strategies; weigh opportunity
against risk; justify decisions; and optimize your individual, team, and
corporate performance.

Forrester Research (Nasdaq: FORR) is a global research and advisory firm serving professionals in 13 key roles across three distinct client
segments. Our clients face progressively complex business and technology decisions every day. To help them understand, strategize, and act
upon opportunities brought by change, Forrester provides proprietary research, consumer and business data, custom consulting, events and
online communities, and peer-to-peer executive programs. We guide leaders in business technology, marketing and strategy, and the technology
industry through independent fact-based insight, ensuring their business success today and tomorrow.
120912