The Current and Future Uses

of Big Data
Douglas Stridsberg
For submission to the 2014 Project Firefly Emerging Leaders.

Introduction
Today, we constantly and inevitably leave trails of data when browsing,
shopping, communicating and searching online. Companies increasingly
churn out exhaust data as a by-product from customers using their
services.i The term Big Data was born around the turn of the last decade
to describe the exponential growth in the availability and size of enormous,
largely unstructured sets of data. Consumers trails of data and companies
exhaust data are both examples of data falling into this category, and it is
the analysis and use of this data that this essay will investigate. Is Big Data
just a hype a marketing fad that will die out with time, or is there real
value to analyzing this data in search of competitive advantage and
productivity enhancement? I will begin by examining the current trends
concerning Big Data and move onto whether or not it has real, long-term
use in a business context.
Trends in Big Data
The three major, identifiable trends in Big Data today are centered on the
use rather than the collection of data. Firstly, we observe that companies are
starting to derive revenue from the selling of Big Data to others in a preprocessed form.ii Previously, companies have kept the data to themselves
because they did not see a value in it beyond their internal usage; they have
now realized the value of their particular data sets, either lying in the fact
that it is difficult for others to obtain or in the fact that their processing
and analysis of the data is superior to others. One such firm is General
Electric, who has begun placing sensors on the engines and apparatus it
manufactures to gather data it can then use to streamline servicing and
other processes.iii Other examples include grocery stores that are beginning
to sell consumer habit data the value of their data lies in the sheer
volume of it and the fact that everyone has to, at some point, enter their
stores and purchase their products. I believe the monetary value of Big
Data will only increase as more of it is collected in increasingly relevant
areas of every-day life.

Secondly, we see that an ever-increasing amount of data has led to major

advances in the data visualization market.iv As we collect more data, it
becomes increasingly important to visualize and present it in order to fully
understand it. Big Data analysis is no longer an obscure occupation of a
specialist management consultancy the data is reaching, and needs to
reach, ever larger audiences and so naturally the appearance and
presentation of it is growing in importance. Firms like Tableau and SAS
Institute are reaping the benefits from this trend through their industry
leading software Tableau Reader and SAS Visual Analytics respectively.
Better data visualization is making Big Data accessible to the masses and
I believe it will continue doing so as competition heats up and the demand
grows.
Lastly, but perhaps most interestingly, is the growing applications of
machine learning to aid the analysis of Big Data.v Machine learning is the
concept behind computers learning to identify patterns or to apply
algorithms on data. Algorithms have previously been heavily applied to
Big Data, and whilst they can successfully pick out patterns, the power of
machine learning (which stems from how a human brain functions) is
needed to critically evaluate these patterns. Companies like Netflix, who
as early as in 2006 announced the Netflix Progress Prize of $1m to whoever
could improve their movie recommendation algorithm,vi are realizing the
enormous benefits machine learning can bring to the industry. I am certain
that machine learning will continue to provide benefits to Big Data
analysis in the long-term.
Is Big Data a key tool?
In light of the examples above along with numerous other success stories,
I argue that Big Data indeed is a key tool in staying ahead of your
competition, but only if it is analyzed correctly and its limitations are
understood. Hadoop, a popular open-source data processing software, is
increasingly being used to process Big Data, and in some instances it is
being used without much thought or planning as a one-stop-solution to
a firms analysis problems. This is obviously very bad practice to fully
and properly analyze our Big Data we must look beyond Hadoop and other
quick fixes and see the entire picture.vii As more data is collected, we also
face the problem of handling the sheer volume of it: what part of the data
is useful? What part should be ignored? I argue that firms must align their
Big Data projects with specific business goalsviii for them to be successful.
More data does not necessarily bring you closer to achieving these goals;
rather, one should adopt a quality over quantity-approach to truly use
Big Data correctly.

The multiple successful case studies of Big Data usage that exist speak
further in favor of Big Data being key in gaining competitive advantage.
The first such case study comes from Wal-Mart, who in-house developed
a search engine platform, Polaris, to use machine learning and synonym
mining on its Big Data in order to provide better search results for
customers.ix The platform yielded 10-15% better purchase completion
rates after being introduced, which for a company the size of Wal-Mart
means billions of dollars.x The second case study is from Macys, who
implemented near-real-time pricing on its millions of products using a
system developed by SAS Institute.xi The pricing is based on demand and
inventory and is said to have increased margins by a substantial amount.xii
Both these cases show how important Big Data has been and how it has
been applied to very different areas this speaks volumes, I believe, about
the future.
Conclusions
Big Data is still in its infancy and is in constant evolution. As technology
evolves and best practices are further developed, I believe we will become
increasingly better at using the data more correctly and more efficiently.
However, as more and more sources of data emerge (such as from the
Internet of Things), it becomes vitally important to understand what
data you are analyzing and where it comes from, otherwise no correct and
sound conclusions can be drawn. If we cannot handle the increasing
volume of data we collect, we will suffer from information overload and
will be unable to use it for any concrete purpose. I argue that we are merely
at the beginning of the road to unlocking the true potential of Big Data in
competition and business intelligence. It is thus with great excitement that
I look forward to what the future brings to the field of data analysis.
i

McKinsey Global Institute, 2011.

CenturyLinkVoice, 2014.
iii
CenturyLinkVoice, 2014.
iv
CenturyLinkVoice, 2014.
v
CenturyLinkVoice, 2014.
vi
BellKor, n.d.
vii
Henschen, 2013.
viii
Pramanick, 2013.
ix
Laskowski, 2013.
x
Laskowski, 2013.
xi
Laskowski, 2013.
xii
Laskowski, 2013.
ii