Profit from AI and Machine

Learning
An Ovum white paper for Dataiku

Publication Date: 29 October 2018

Tony Baer
Profit from AI and Machine Learning

Summary
Catalyst
Innovation is never a matter of technology alone. Getting stakeholders on the same page and making
the process repeatable are essential to making any form of innovation sustainable beyond that first
success. This is often termed the "softer" side of technology innovation. But with the perfect storm of
data availability, the proliferation of algorithms and modeling frameworks (most of them open source),
and the ability of the cloud to make leading-edge infrastructure for artificial intelligence (AI), such as
GPUs and FPGAs, readily consumable, the technology of AI is threatening to progress faster than the
ability of organizations to mobilize their people and processes.

Given that AI has percolated up from the lab to the boardroom, Ovum partnered with Dataiku to
survey just over a dozen early adopter organizations with AI projects in production to identify best
practices for mobilizing people and making processes – and success – repeatable.

Ovum view
Data science is the starting point for AI, as the scientific method is essential to putting the project
lifecycle on a sound footing. There is no single recipe for whether it is more effective for AI initiatives
to originate from C-suite vision or percolate from the operating units, but either way, the key to making
success repeatable is for "transformational" thinkers to look beyond their own line organizations
(departments) to how their innovative approaches could apply elsewhere in the organization. Growing
and locating data science teams that lead AI projects is a challenge; while ideally, these teams should
be collocated in the operating units, the de facto reality is that the shortage of qualified data scientists
and data engineers tends to concentrate their presence near headquarters. While college and
university programs are turning out more qualified practitioners, the reality is that getting the skills out
to where they are needed will require data science teams at or near central or regional headquarters
to actively embrace "train the trainer" approaches (where trainers teach others to train) to broaden
knowledge and skills. And, if your organization decides to take the plunge and develop its own AI
projects, keep in mind that the project lifecycle will involve a superset of steps and more moving parts
compared to traditional data science and analytics projects. Be agile and pay huge attention to data
quality and relevance, because AI models are extremely hungry for data. And be sure to manage with
the expectation for change, because AI models are moving targets.

Key messages
▪ The scientific method of data science for comparing, testing, and evaluating models is the
starting point for AI.
▪ The biggest challenge is getting scarce expertise to where it is needed; beyond reliance on
the higher education system, enterprises should incorporate collaboration, knowledge
sharing, and "train the trainer" objectives into the missions of data science teams to broaden
expertise throughout the organization.
▪ The lifecycle of AI projects includes a superset of tasks and more complexity and change
compared to traditional data science and analytics projects.

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 2

Profit from AI and Machine Learning

Research scope
Qualitative research to identify best practices
Ovum and Dataiku conducted a jointly sponsored qualitative survey designed to identify the best
practices for enterprises implementing AI projects, encompassing machine learning (ML) and/or deep
learning (DL). Because the study was focused on best practices, we limited its scope to a select
sample of Global 2000 organizations with the most experience with AI projects in production. This
does not count implementation of applications or tools that embed AI (usually ML) under the covers.

Most organizations surveyed had dozens to hundreds of AI projects in production, and dozens of data
scientists (in some cases up to 50 or 60) on their staffs. Those projects could range from chatbots to
more ambitious pricing, product, or customer segmentation strategy efforts. Most of the AI projects in
production involved ML, not DL. Overall, we gained insights from organizations with collective
experience of over a century of staff years in AI.
The deep-dive discussions – each of which were 45–90 minutes in length – were conducted on an off-
the-record basis. Our goal wasn’t to find out what AI projects were being conducted; instead, it was to
learn how those projects were being managed from the people and process perspective. Throughout
this paper, we have interspersed several choice quotations to provide color.

Respondent selection
We interviewed respondents in senior management roles who understood the big picture and had
strategic roles formulating policy for AI project staffing and implementation. We also interviewed an
executive consultant whose engagements help large organizations create data mining, big data, and
ML programs, and a senior data scientist based in a regional Center of Excellence (CoE) for a global
enterprise, who provided an excellent counterpoint to the executive-level views. The makeup of our
research panel is shown in Table 1.
Table 1: Research panel
Respondent Industry
Vice President, High Risk Computation Banking
Vice President and Data Science Lead Banking
Chief Data Scientist Banking
Senior Director, Data Science Insurance
Director, Data Science & Machine Learning Insurance
Senior Data Scientist Insurance
Global Chief Data Officer Business (HR) services provider
Chief Data Scientist Oil and gas
Executive Director, Analytics Travel services
Chief Data Scientist Aerospace
Senior Manager, Data Science & Global Services Aerospace
Vice President, Data Science Media and entertainment
Executive Consultant, Data Science and Predictive Independent data science consulting firm
Modeling

Source: Ovum

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 3

Profit from AI and Machine Learning

Core assumptions
Working definition of AI
As a working definition, we differentiated AI projects from data science projects as follows. Data
science projects use scientific method to develop the right models, and the models that are deployed
do not evolve until people make changes. By contrast, we view AI as data science projects that
develop models that self-learn (through supervised or unsupervised processes) and evolve based on
the inflow of new data. As such, data science projects involve static models, whereas AI projects
utilize models that are dynamic.

Note: Our definition of AI encompasses ML and DL. Note that across the organizations that we
interviewed, ML encompassed all of the projects in production; as for DL, a few organizations
regarded it as an area for future investigation.

Data science is a prerequisite for AI

While this research focuses on AI, there is heavy mention of data and data science. We believe that
AI requires the discipline of data science to develop and maintain the models for the following
reasons:
▪ AI projects require the same scientific methods for thoroughly vetting, testing, and comparing
results for candidate models and data sets that data science projects involve.
▪ AI projects consume lots of data; any problems with the data (e.g. poor quality, poor fit) will
magnify themselves when fed to ML or DL models that, by their natures, are likely to change
over time.
▪ AI projects involve a superset of steps and processes compared to data science projects, as
our research validates (and is discussed later in this report).

The AI decision
Impetus for AI: Executive vision versus bottom-up inspiration
Observations
"AI is when you want to get budget. Machine learning is when you need to hire people…" – Chief Data
Scientist, European retail bank
So where does the idea and inspiration for AI projects begin? Does it begin with an executive vision or
an idea that emerges from innovative minds down in the trenches in the operating units?

It shouldn't be surprising that, given our sample was weighted toward senior executives, in most (but
not all) cases the impetus came from the C-suite. The following is a sampling of responses that we
heard:

▪ There were specific initiatives championed by the CEO for strategic goals such as
containment, improving customer support, or taking a leap ahead with risk management.

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 4

Profit from AI and Machine Learning

▪ The CEO of a large aerospace firm viewed AI as a strategic future competitive differentiator in
its industry.
▪ One of the largest operating units (marketing) identified the need to apply ML to optimizing
customer engagement as the company expanded from being a cable TV provider to adding
broadcast networks, studios, and content. The changing nature of its business changed the
scope of its engagement with its customer base.
Top-down
The advantage of an initiative starting from the C-suite is that, when there is effective leadership, it
provides a very effective means for aligning the rest of the organization. And, if a strategic initiative
sponsored by the top succeeds, it can be an extremely effective scenario for establishing credibility
and securing buy-in from the rest of the organization. The downside is when the C-suite gets a case
of "shiny new object" syndrome, where they read about popular innovations such as big data, open
source, or AI without developing a sound business case for adoption.
Bottom-up
The benefit of an initiative percolating upward from the trenches is that the chances are it will more
likely be a response to a tangible business or operational issue. With the bar lowered to tactical
projects, such as implementing a chatbot to improve the efficiency of customer service, getting that
first win can be a much simpler challenge compared to a corporate strategic initiative: the goals are
simpler, and in all likelihood, there are fewer constituencies involved. The drawback is that bottom-up
initiatives can easily become siloed and disjointed.
Best practices
"When looking to spread the word [on AI], look for 'transformational talent'…" – Chief Data Scientist,
Oil and Gas services company

We concluded that there was no single argument that a top-down or bottom-up approach to
embracing AI was more effective. It will depend on the culture of the organization: does the
organization tend to take its orders from the top and/or is there established precedent for empowering
the working level? (The most functional organizations will have combinations of both.)

The challenge is not getting that first win, but making those wins sustainable: can success be
repeated? When executives take the lead, the best success will come from an approach designed to
plant seeds of new thinking; the executive has a vision and challenges the rest of the organization to
start thinking differently. When innovation for AI begins at the bottom, the best chances for repeated
success stem from the talent. A respondent cited the need to have "transformational talent" that could
think beyond the challenges of their own operating unit and evangelize (and share) with the rest of the
organization.

When to use AI
Observations
While the purpose of this research did not focus on what AI projects were being developed or run in
production, we asked respondents about what types of projects merited AI. They gave us the
following scenarios:

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 5

Profit from AI and Machine Learning

▪ Complexity and scale: Problems that involved too many features and/or data variety for
humans to understand without help.
▪ Prescriptive solutions: Going beyond predictive analytics to provide prescriptive answers that
can deliver tangible benefits, such as suggestions of when to maintain a machine, when to
take a specific action to prevent customer churn, or how to prevent an interaction from turning
negative.
▪ A problem domain is dynamic: If the problem involves too many moving parts, such as if the
entry of a new competitor changes the nature of competition or a new product changes how
the customer base should be segmented. In such cases, traditional approaches to building
models based on static rules will not be able to stay relevant without significant human effort.
As such, AI automates the tasks of changing the rules that would otherwise fall to humans.
▪ The stakes are sufficient to justify new approach: Organizations viewed AI as highly strategic,
such as the aerospace firm or the media firm that saw AI as the means for understanding how
to interact with customers now that the scope of its business had broadened.
Best practices
When evaluating whether to employ AI, there is one best practice that should be obvious: like any
technology or methodology that is being applied for the first time, choose a use case that would
represent "low hanging fruit": a project that responds to a well-defined, well-known problem that will
yield tangible results. Then determine if the solution involves the criteria described above (e.g. it
requires a prescriptive approach or involves a problem domain that is too complex or dynamic for
traditional data science modeling approaches). Above all, make sure that the project team will
address all the essential tasks in the project lifecycle. As we note later in this report, AI projects will
require additional tasks and responsibilities compared to traditional data science projects.

The people equation

Where are the data science teams?
Observations
"It is hard for data scientists to understand the business if they live in a bubble…" – Chief Data
Scientist, Global Banking firm

"The world is moving too quickly to have a single organization that filters everything…" – Chief
Analytics Officer, Insurance
This is a scenario where we found a stark disconnect between where data science teams (and data
scientists) tend to be found, and where they are needed. Respondents were unanimous in their belief
that data science teams should reside in the business units, rather than at headquarters level. The
rationale is that relying on such a centralized model in the long run will pose a bottleneck to innovation
with AI.

The challenge, however, is that given the relatively scarce supply of data scientists, we are in a
"seller's" market, in which data scientists can pick and choose where they want to go, and in most
cases, it will be cities or metropolitan regions that attract them in critical mass. The result is that even

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 6

Profit from AI and Machine Learning

if an organization requires data science teams to be situated in a local business unit, the supply of
these professionals will tend to be near headquarters or academic/research centers.

The data science organizations took several forms, including:

▪ Headquarters-based data science teams that are the logical descendants of teams
conducting data mining.
▪ Large or regional business units with leading-edge talent and imagination.
▪ Centers of Excellence (CoEs). These organizations are based at global and/or regional
headquarters, typically being selective in their choices of projects that could range from proofs
of concepts to minimum viable projects, and assistance to local business units for delivering
specific projects. CoEs tend to exist at an early stage in the development of the talent pool, to
provide talent and expertise not available at the local level.
▪ Internal consulting groups. Often an expansion of the CoE, these groups tend to have more
hands-on relationships with business units, which contract with them to help deliver projects
utilizing AI.
Figure 1: Evolution of the data science organization

Early Early majority Mainstream

Center of excellence
Innovative
local
business • New discipline
unit • Small talent pool
• Elite practitioners
gravitating to talent
centers
• PoCs, MVPs, early
wins
Internal consulting
group
• More hands-on
• Contract with LOBs
• "Train the trainer" to broaden
skills to regional units
Internal consulting
group + groups
embedded in LOBs
• Data science teams
established in LOBs
• Talent broadens out to the
regions
• Internal consulting group for
"exceptional" projects

Source: Ovum

Best practices
The ultimate goal is transferring and cultivating knowledge where it is needed – at the local business
unit level. Just as we believe that strong executive leadership should take a "plant the seeds"
approach, we also believe that making the use of AI sustainable will need a "train the trainer" model
that actively spreads knowledge and cultivates expertise across the organization. We view the
evolution of the data science practice as a journey, as shown in Figure 1. It will begin either at the
local level, where an innovative team with leading-edge talent develops practices that go viral, or with
a centrally located CoE that acts as an incubator. From there, the trajectory will find the CoE
broadening into a more hands-on internal consulting unit that spurs development of local talent
through collaboration and knowledge transfer (aided by collaboration tools and services readily
accessible from the cloud). Ultimately, the internal consulting group's job is either to put itself out of
business by being the knowledge and practice transfer agent, or to perform a diminished role as it fills
the gap between local data science teams that are increasingly self-reliant in skills (although not in
knowledge). In all scenarios, collaboration and communication across teams and organizations is

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 7

Profit from AI and Machine Learning

essential to the success of any enterprise-level AI initiative; collaboration is essential for sharing best
practices, and even more importantly, lessons learned from real-life experience.
Figure 2: AI project key roles*

Project
owner Business liaison
(sponsor) (communicator)

Data Project Visualization

engineer manager expert

Domain
Data
expert
scientist
(SME)

"The biggest headache is the handshake

*Not discrete jobs/positions between development and deployment."
− Executive Consultant

Source: Ovum

Who is the project team?

Observations
"The biggest headache is the handshake between development and deployment." – Executive
Consultant

The makeup of a project team undertaking AI projects is quite similar to that of a team that tackles
data science projects. Figure 2 shows roles, not individual jobs or job titles, with dedicated staff shown
in red. At a minimum, the project should have a manager, a data scientist, and a domain (subject
matter) expert. Additionally, it will draw on the talents of data engineers, who understand the
architecture of clusters and the physical and logical layout of data stores; visualization experts, who
can make the results understandable to a business audience; and the people who interface with the
business – the project owner or sponsor, who is ultimately responsible for getting the project
approved, and the business liaison, who works with stakeholders (customers) to ensure that the
project aligns with their needs. In many cases, people will assume multiple roles. For instance, the
business liaison may also be the visualization expert, as they are tasked with communicating the
results. The project owner may or may not double as the project manager. And, while the data
scientist and the data engineer are likely to be separate roles, there is an overlap of responsibilities
when it comes to preparing and cleansing data.
Best practices
While all relationships are critical (e.g. the project must have people who can work effectively with the
business), the relationship between the data scientist and the data engineer is especially critical to the
successful execution of the project. The pairing of these roles is essential to ensure that models
developed on laptops or in other sandboxes get deployed effectively in production – and survive in

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 8

Profit from AI and Machine Learning

translation. Additionally, the success of a project is often related to the amount of bandwidth that the
data scientist has for performing data science (e.g. developing, testing, validating, and selecting
models and data sets). We saw different approaches to data scientist and data engineer staffing. In
one extreme case (an aerospace firm with massively scaled projects), the optimal staffing goal was
having two data engineers for every data scientist, but in most cases, such a staffing model will not be
practical. A best practice that we identified was setting realistic goals: having the data engineers build
automated data pipelines so that the data scientists would only have to spend half their job wrangling
data.

We also saw the emergence of a new sub-specialty among data engineers that was a hybrid of data
engineer and data scientist: the ML or AI engineer. Several organizations that we spoke with cited the
need for a data engineer who also has domain knowledge of the behavior of AI (at this point, mostly
ML) algorithms. We have also seen the need for such a specialty cited in external surveys (e.g.
O’Reilly's The State of Enterprise Machine Learning Adoption) and job boards from sources such as
Indeed.com. Given that the dynamic nature of AI models can make deployment and performance
more challenging, the need for such a hybrid specialty makes sense.

It goes without saying that collaboration is essential; innovations such as AI never succeed if the
spark and the insights stay trapped in the mind of the data scientist or data engineer. With the state of
the art in AI disciplines including ML and DL actively developing, sharing best practices with
colleagues across the field (inside and outside the enterprise) is essential. Beyond the practitioner
world, close relationships are essential between data scientists and data engineers to ensure that
models survive translation into deployment, and with the business, to ensure models remain aligned
with competitive and operational goals. There is no single recipe for the informal or formal means by
which exchange of thought and knowledge occurs, but contact must be ongoing rather than episodic.
Collaboration-oriented tools and technologies can play important supporting roles where collaboration
is ingrained in the culture.

Keeping the process real

Process approach
Observations
Our survey did not focus on identifying the best project management methodology for data science
projects. In the data mining field alone, there are numerous models, although the cross-industry
standard process for data mining (CRISP-DM) is one of the best known. Our respondents provided a
number of procedural recommendations, such as that data scientists should meet at least once a
week with the project owner, and at least biweekly with fellow data scientists to get peer review or
input. We also found some outliers. For instance, one of the aerospace firms in our sample indicated
the use of a modified waterfall model: once the ML model emerged from testing and evaluation, there
would be a freeze as it entered production, the rationale being the safety-critical nature of its products
(aircraft) and the complexity of the supply chain (while increasingly, much or most of the value is in
software, not in physical parts, there are still significant lead times required for suppliers). Another
outlier, presented by an oil and gas firm, was to rotate data scientists every six to eight weeks

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 9

Profit from AI and Machine Learning

between projects to get fresh views. Our take is that continuity is probably more appropriate for most
AI projects.
Best practices
Stay agile
Agility was the common thread. By nature, ML and DL models will be dynamic; they will change based
on the data that is fed to them. Continuous improvement and continuous integration practices
borrowed from the DevOps/DataOps communities are the most effective ways to keep models in
sync. An adaptation of this practice, as developed by PerformaMetrics (one of the participants in the
study) provides a good example of how lifecycle management can be made more agile (see the
dotted lines in Figure 3).
Serve the customer
This is a truism that applies to all projects, AI included. The first yardstick for gauging the success and
alignment of the project with business needs is ensuring that the customer's needs and expectations
for value are being served. Note that there is a subtle distinction between customer expectations for
value and outcome. It is always essential to deliver value but, given that drift in the data and model is
inevitable, there is the possibility that the outcome may not always match what was initially expected.
In these situations, there is the need to check alignment of the data and model with the business
problem. If there are disconnects, it is essential to assess whether the problem is attributable to

▪ poor data or poorly performing models, or both

▪ changing realities, where disruptive events have upended the competitive landscape,
customer expectations, or operational realities (e.g. that cyber threats have mutated,
changing the attack surface).

Determining whether drift is due to technical issues or changes in the underlying reality is essential to
ensuring the project aligns with customer needs.

Keeping AI projects aligned

Observations
Compared to traditional data science projects, keeping AI projects aligned with customer needs is
more challenging (but to the customer, potentially more rewarding) because of the dynamic nature of
the model – by design, AI models learn from the data. Model drift is an issue unique to AI projects. By
comparison, data drift can afflict all analytic projects – even those with static models that don't learn.
But because AI models are designed to learn, the impact of data drift can be magnified compared to
traditional data science analytics projects.
So, after asking the most basic question – Is the customer happy? – it is essential to track alignment
by asking questions such as:

▪ Are the predictions well aligned to the data, and are they meeting customer expectations?
▪ Is the model under- or over-predicting trends found in the data?
▪ How well is the model performing, from a technical perspective (is it meeting SLAs?) and a
business perspective?

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 10

Profit from AI and Machine Learning

Best practices
Because models cannot yet explain themselves, model accountability is a work in progress.
Consequently, the best ways to maintain accountability require tracking at a more rudimentary level:
track the inputs and the outputs that resulted. In this case, it involves tracking the data that feeds the
model. An important best practice that we identified is time stamping data, either by data set or
through time slicing. In an ideal world, it would also be nice to time stamp changes to the model, but
today, this approach remains problematic, because it would not be practical to use the check-
in/check-out process that is often employed with software development to models that change
continually. Today, the most practical approach would be to use break points or similar mechanisms
to snapshot the model at different points in time. However, when it comes to addressing this
challenge, we believe that automation will eventually generate the comprehensive audit trail that
model accountability needs. Watch this space.
Figure 3: Project lifecycle management for AI projects

Source: PerformaMetrics

How AI projects compare to conventional data science projects

Observations
General responses
"The development phase is the same, but the production phase is different…" – Executive Consultant

Our panel gave us a mix of responses when asked how the lifecycles of AI projects compare to
traditional data science projects. On one hand, there were similarities:
▪ The project management tasks are similar.
▪ The scientific method of data science is a mandatory prerequisite for developing and
deploying AI models.
▪ Data is king: AI and data science projects need both the right data (data that is pertinent for
the problem) and to get the data right (data quality is just as important as ever).

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 11

Profit from AI and Machine Learning

▪ Deployment is a major obstacle – getting models that are proven in the sandbox (on the
laptop) to survive intact when they get deployed (and inevitably modified) to run at scale on
clusters.
▪ There is always competition in comparing and testing hypotheses, models, and data sets.
Differences
But there are also stark differences that boiled down to the realization that AI projects are supersets of
data science projects: they involve more tasks and have more moving parts. Some of the additional
tasks in AI projects include:

▪ tuning hyperparameters
▪ labeling data
▪ training the model
▪ checking the "decisions" made by models.

Then there is the difference in the modeling processes. Conventional data science projects involve
static models: once the model emerges from the scientific process of testing and comparing
hypotheses and data, it won't change until a human modifies or replaces it. By contrast, AI models are
dynamic. They require the same scientific rigor as static data science models during the development
phase, but once they are deployed, they will evolve as they learn, as data keeps getting ingested.

Note: Today, modeling is largely a manual process, where the data scientist creates and tests the
model, and then refines it for deployment. In the future, this process, too, may be wholly or partially
automated through the use of ML or DL during development.
Best practices
Manage for change
Change is the lifeblood of AI. The goal of AI is building models that change through learning. As such,
managing the lifecycle of an AI project is an exercise in managing for change. But classical "change
impact management" is a process that would be too heavyweight to be used every time a model
changes; instead, such a top-down approach should only be employed at certain watershed
moments, such as at the outset of creating the project, or at a breakpoint in midlife where there might
be the need for significant midcourse correction. As we noted above, agile practices that apply
continuous development and integration approaches to modeling will be best suited to dealing with
the moving targets of models and data inflows. And because you need data to constantly nourish the
model, it makes sense to create data pipelines to perform all the transformation and preparation
instead of repeatedly running the same batch jobs.
Data is still king
This is true of any analytic project, but AI projects are much hungrier for data because the models rely
on them for the learning process: the more data, the more they learn. As they consume data, the
models evolve in a learning process that may be supervised or unsupervised. That means that the
impact of using the wrong data, or having data that is poorly prepared, can be much greater
compared to static data science models. And that is also why data scientists will never be able to
totally free themselves of the burdens of wrestling with the data. The best practice was having

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 12

Profit from AI and Machine Learning

sufficient data engineering support to ensure that no more than half the data scientist's job involves
wrangling data.

Takeaways
Data science is the starting point for AI
AI projects require the same scientific discipline for thoroughly vetting, testing, and comparing results
for candidate models and data sets as data science projects.

Getting skills where they are needed is the biggest people

obstacle
For global organizations, the tightness of the data scientist (and data engineer) market means that
talent will not often be available in the locations where it is needed. While skilled practitioners today
can dictate their terms, locating themselves near headquarters or other magnet metropolitan areas,
skills are increasingly needed at the local operating units that may be sited in regions lacking the right
talent pool. "Train the trainer" approaches are critical for talent at headquarters or centrally located
CoEs to spread skills and expertise, while collaboration is essential for sharing the lessons learned.

The AI project lifecycle is a superset of conventional data

science and analytics projects
Compared to traditional analytics and data science projects, AI projects involve more of just about
everything: data, tasks, and opportunities for projects to drift off track. While the models for
conventional projects are static (and only changed by a human), those for AI projects are dynamic.
That requires additional tasks such as tuning hyperparameters, labeling data, training the model, and
checking the "decisions" made by models. AI projects are far more ravenous for data, so any
problems with data quality or appropriateness (is the data right for the problem?) become magnified.
The bottom line is to expect more tasks, and the need to manage against moving targets.

Acknowledgement
This research was cosponsored by Ovum and Dataiku. Dataiku is one of an emerging breed of tools
that help organizations scale data science and AI from insight to deployment, automating key steps in
the development and scaling of models to supporting collaboration across all stakeholders through
sharing of knowledge.

The Dataiku solution helps teams scale self-service with cleansing, wrangling, and troubleshooting of
data sets; automating model feature generation; integrating models to applications through APIs;
conducting side-by-side comparisons of the performance of multiple algorithms; and supporting
operationalization with automated deployment capabilities. While tools on their own are not the
solution to collaboration, they are key enablers for organizations that make the commitment for
integrating data science people and processes into the business.

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 13

Profit from AI and Machine Learning

Appendix
Author
Tony Baer, Principal Analyst, Information Management

tony.baer@ovum.com

Ovum Consulting
We hope that this analysis will help you make informed and imaginative business decisions. If you
have further requirements, Ovum's consulting team may be able to help you. For more information
about Ovum's consulting capabilities, please contact us directly at consulting@ovum.com.

Copyright notice and disclaimer

The contents of this product are protected by international copyright laws, database rights and other
intellectual property rights. The owner of these rights is Informa Telecoms and Media Limited, our
affiliates or other third party licensors. All product and company names and logos contained within or
appearing on this product are the trademarks, service marks or trading names of their respective
owners, including Informa Telecoms and Media Limited. This product may not be copied, reproduced,
distributed or transmitted in any form or by any means without the prior permission of Informa
Telecoms and Media Limited.

Whilst reasonable efforts have been made to ensure that the information and content of this product
was correct as at the date of first publication, neither Informa Telecoms and Media Limited nor any
person engaged or employed by Informa Telecoms and Media Limited accepts any liability for any
errors, omissions or other inaccuracies. Readers should independently verify any facts and figures as
no liability can be accepted in this regard – readers assume full responsibility and risk accordingly for
their use of such information and content.

Any views and/or opinions expressed in this product by individual authors or contributors are their
personal views and/or opinions and do not necessarily reflect the views and/or opinions of Informa
Telecoms and Media Limited.

© 2018 Ovum. All rights reserved. Unauthorized reproduction prohibited. Page 14

CONTACT US
www.ovum.com
analystsupport@ovum.com

INTERNATIONAL OFFICES
Beijing
Dubai
Hong Kong
Hyderabad
Johannesburg
London
Melbourne
New York
San Francisco
Sao Paulo
Tokyo