Big Data Analytics for Supply Chain Management
Jens Leveling¹, Matthias Edelbrock², Boris Otto³
¹ Software Engineering, Fraunhofer-Institute for Material Flow and Logistics IML, Dortmund, Germany
² Audi-Endowed Chair of Supply Net Order Management, Technical University of Dortmund, Dortmund, Germany
³ Information Management and Engineering, Fraunhofer-Institute for Material Flow and Logistics IML, Dortmund,
Germany
(jens.leveling@iml.fraunhofer.de, matthias.edelbrock@tu-dortmund.de, boris.otto@iml.fraunhofer.de)
Abstract - A high number of business cases are
characterized by an expanded complexity. This is based on
increased collaboration between companies, customers and
governmental organizations on one hand and more
individual products and services on the other hand. Due to
that, companies are planning to address these issues with Big
Data solutions. This paper deals with Big Data solutions
focusing on Supply Chains, which represents a key discipline
for handling the increased collaboration next to vast
amounts of exchanged data. Today, the main focus lays on
optimizing Supply Chain Visibility to handle complexity and
to support decision making for handling risks and
interruptions along supply chains. Therefore, Big Data
concepts and technologies will play a key role. This paper
describes the current skituation, actual solutions and
presents exemplary use-cases for illustration. A classification
regarding the area of application and potential benefits
arising from Big Data Analytics are also given. Furthermore,
this paper outlines general technologies to show capabilities
of Big Data analytics.
Keywords—supply chain management, supply chain
visibility, supply chain risk management, big data
I. INTRODUCTION
Nowadays, many companies are planning to extend
the existing IT-Infrastructure with Big Data solutions.
This is needed due to the challenges result on the on-
going globalization, increasing data volumes and
customer needs in more individual products and service
configurations. The globalization enables acquisition of
new customers and requires new and changing business
models due to changing market requirements. New
suppliers and distributors have to be integrated into the
supply chain. Additionally, the supply chain operations
and processes have to be constructed for global
application by considering new product variants. The
products and offered solutions themselves are more and
more custom-designed as a result of adapted
requirements related to region specifics and competitors’
offerings. This economic competition leads to short
product life-cycles, not only in the mobile phone and IT
sector.
Next to the direct operational aspects, the focus of
each company lays more and more in the company’s
context and the supply chain environments. Both are not
only influenced by the market itself, but also by risks or
bottlenecks that might occur along the supply chains or
within production lines. For example, supply chain risk
management is in focus of many companies, but is limited
978-1-4799-6410-9/14/$31.00 ©2014 IEEE
supported by IT systems. Disturbances are treated
reactively in most cases. An early identification of
upcoming or future risks is typically missing. According
to this, the required reaction time for almost all actions is
not available. Even the transparency of the first level of
the supply chain – the direct suppliers and customers - is
not always given. All these issues are in the focus of
companies with different emphasises due to individual
business strategies, manufactured products and offered
services. Therefore, data becomes more important and it is
turning to be one main part of new business models in the
future as it has been shown by [1].
Big Data solutions have to be adapted to the
company’s requirements and environment. Instead, data
of every enterprise is individual due to different semantic
meanings and structure changings not only between
different companies, but also within departments and
divisions. Big Data activities are starting often by setting
up a (Big) Data strategy for the entire company.
Therefore, the value drivers and targets, which have to be
archived with the new IT solutions, should be clarified
[2].For these issues a general classification of (Big) Data
strategy targets for companies and a potential usage based
on the categorization are illustrated by two exemplary use
cases as shown by [2].
This paper is structured as follows: In the next chapter
big data is defined and a general description of Big Data
technologies and data sources is given. In the 3rd section
related solutions for Supply Chain Management (SCM)
are characterized. The 4th chapter shows categorization in
terms of the area of application and shows exemplary use-
cases of big data analytics. Finally, a conclusion of the
explained approach and an outlook of future work are
given.
II. BIG DATA
A. Big Data term definition
Big data describes a way of collecting, managing and
analysing large amounts of data. Therefore, big data is
mostly referenced with the three Vs as described by [3, 4].
They are defined as volume, velocity and variety.
The volume characterizes the vast amounts of data
stored within the IT infrastructure. In general, storing a
large amount of data already represents a big issue for the
data storage itself. One of the main challenges for IT
infrastructures dealing with big data is to ensure the
availability of storage space and an efficient accessibility.
918
 Proceedings of the 2014 IEEE IEEM
Velocity describes the large amounts of data that
arrive in real-time irregularly. The fast arriving data has to
be handled, if a further usage is planned.
The last one – variety – relates to the various structure
of information handled within the big data environment.
The stored data can either be based on a structure (data
type) or consist of unstructured information. The three Vs
including their challenges for IT infrastructures and big
data architectures are considered (presented) in the next
sections. The value of big data is often described as the 4th
V. This is related to the fact that the data needs to be
processed and analysed for further usage.
Handling large amounts of data requires both,
concepts for managing these datasets and concepts for
processing the amounts of data. In this section, actual
concepts for both challenges are described.
B. Data Sources
Mainly data sources are divided into internal and
external data. Internal data is mostly available in business
IT-systems and databases, for example from an ERP-
system. The internal communication of production
systems is also available as data streams, for example
from radio-frequency identification (RFID) devices.
External data sources are also available as data
streams, for example from Social Media like Facebook
and as Datasets from data portals. Both are described in
this subchapter. Social Media sources are mostly
unstructured and the data semantic is various and
frequently changing. Social Media data is mostly not
directly accessible. For example, Twitter only provides a
data stream that contains a limited amount of tweets.
Therefore, companies like DataSift offer purchasable
streams of social media data [5]. On the other hand
external sources like search engine trends are also less
structured, but the provided APIs, like Google Search
API, are mostly free to use. Some external data sources
can be searched and captured via technologies like
Sematic Web (WS) as shown in [6, 7 and 8] or Linked
Open Data (LOD) as described in [1].
Open Data sources, like Eurostat
(epp.eurostat.ec.europa.eu), are free for commercial and
non-commercial use. For example they contain statistics,
geo and political information about regions and countries.
These sources can be accessed through open data portals
like the European Union Open Data Portal, which
represents a catalogue for further open data sources [10].
Next to the open data portals, there are many other
platforms that offer closed data. Closed data means that
datasets have to be purchased or licenced for access.
Example platforms are:
Factual, is an open data platform founded in 2009.
Factual offers different data services like data mapping or
ad targeting. All services are based on their global
location database containing data for more than 65 million
local businesses and places of interest spread all over the
world.
Microsoft Azure Data Market is a cloud computing
infrastructure. Close data and open data can be accessed
through this data market platform. The datasets are
provided by many different companies and organizations.
919
Data.com is a service platform focussing on Business
Data management like contacts and company profiles.
The service offers access to millions of company profiles
and allows current updates on the customer data.
C. Data Management and Analytics
New architectural designs are required due to various
structures and a high amount of data. Conventional
architectural components, like SQL databases, are no
longer able to handle these amounts of data. Additionally,
existing enterprise architectures, which focus on the
business processes themselves, are often designed without
any data-centred characteristic. However, the modern
enterprise architecture has to focus on data to enable big
data analytics [3].
Data are mostly structured too individually to store
them within the given structures of relational databases.
Therefore, several new database concepts have been
developed to store and manage individually structured
data. These concepts are generally known as NoSQL (Not
only SQL). These databases have no or limited schema
restrictions compared to relational concepts. The benefit is
to apply an update of data structures without database
table rewrites [5]. Indeed, the NoSQL-Databases focus
lies on distributed accessibility and scalability. For supply
chain modelling, NoSQL Graph Databases are very
attractive as a result of their close relation to applications
like transport schedule optimization, navigation systems
or social networks [12]. The data model of these databases
organizes the data within a graph structure, comparable to
road networks. The information in Graph Databases is
stored within the nodes. The additional information is
mostly defined as properties to describe the node [12].
Finally, a link organizes the connection between different
database entries.
The right data management concept for handling big
data is linked to the data analytic approach. Large
distributed data storages are required by batch analytics.
In general, batch analytics are working with distributed
tasks on large data storages to search and extract
information [5]. Batch analytics are executed and
managed through large scale processing frameworks.
Today, the Apache Hadoop framework is widely used for
batch analytics. It consists of two main components: The
Hadoop File Systems (HDFS) for managing vast amounts
of data and a MapReduce based framework for executing
the analytics as characterized by [4, 11].
A challenge within the use of data analytics is to
analyse irregular arriving data streams next to existing
large data sets. Data Streams are often characterized as
real-time data. Therefore, the Lambda Architecture has
been designed to enable Data Stream analytics in addition
to the batch analytics as described by [3, 13]. Data
Streams can be processed by Technologies like Complex
Event Processing (CEP) as shown by [3, 14]. CEPs are for
example used to integration and processing events from
RFID devices, barcode readers and sensors.
 Proceedings of the 2014 IEEE IEEM
III. RELATED WORK
A. Big Data for Supply Chain Management
Nowadays, in SCM the optimization of the supply
chain visibility (SCV) is one of the core developments.
The SCV as shown by [15] is a complex problem due to
the interaction between involved people, processes,
technologies and information flows. The key target of
SCV is to show current activities and involvements along
a supply chain. This can be used to collect information for
decision makers [16] in many cases, for example, if an
interruption occurs.
Today, available data sets are often outdated, but
actual information is required for decision making [12, 16,
17 and 18]. This issue has to be addressed by new IT
solutions for SCM. Some Big Data based solutions for
Supply Chain Management are presented below.
In [9] LOD and SW are used to increase SCV. Here,
business transactions are evaluated and analysed to collect
external data and to describe the supply chain network.
Furthermore, existing supply chain data is used. The
analytic results are compared with the existing data to
update the supply chain network information. Afterwards,
the up to date supply chain information is used to search
for upcoming risks and interruptions for all sites and
transport routes within the supply chain. For this solution
an ontology network as described by [19] is needed. The
Background is that in a Supply Chain many companies are
exchanging data, but every company has an individual
semantic understanding of the given information.
The RiskVis framework as developed by [16]
combines available solutions for visualization of supply
chains based on Google Maps and Baidu Maps with
analysing approaches for internal and external data.
Therefore, an analysing module extracts problems and
disruptions. Additionally, the existing information about
how to handle unplanned disruptions, along with the
supply chain network configuration, is available for the
system through accommodated interfaces. Furthermore,
the data for the risk extraction is stored next to the
analytics module on a server cluster. The analysis is
focused on real-time data to provide a real-time view of
the supply chain for risk monitoring to enable both:
Reducing complexity and providing necessary
information for decision makers [16]. Heretofore an
interruption forecast or early risk identification is not
designed.
B. Industrial Awareness of Big Data Solutions
As of 2011 there has been already a high interest or
new solutions to increase the supply chain visibility based
on big data analytics. But the existing security policies
stopped the implementation and integration of different
data sources. The industry spotted high capabilities for
solutions based on HDFS and MapReduce in 2011.
The situation about dealing with large amounts of data
has not been changed. Existing data is often stored in
large files like web logs or in complex XML documents.
All those files are transformed into existing structured
data as they are demanded by conventional databases. The
transformation is necessary, if a further usage is planned.
920
The mentioned transformation is often too time intensive
and the stored data is limited for later usage. The
distributed processing is needed to extract required
information shortly from large datasets. This is not yet
implemented due to the restricted accessibility of different
internal data sources. Furthermore, security polices mostly
complicate the integration of external data in existing
enterprise IT infrastructures.
Current solutions for increasing the SCV are
optimizing the collaboration within the supply chain.
Additionally, 44% of enterprises improved the internal
visibility and 40% are optimizing their operations to
improve monitoring, usability or efficiency as the [15]
reports. This leads to reconsidering security policies to
enable the access to different data sources for analytic
methods. The understanding that data is valuable has
highly increased throughout the industry. But the valuable
information needs to be processed.
Following the Gartner survey [20], nowadays around
26-28% of manufacturing companies and retailers invest
in big data solutions. In the transportation sector only 20%
of the asked companies have already invested, but with
50% there is the highest value of planned invests within
the next two years. The problem addressed with big data
(summarized over all industries) is about 32% in
improving risk management.
IV. CATEGORIZATION AND POTENTIAL BENEFITS
A. Classification
A categorization regarding the analysed type of data
delivers seven potential scenarios. These scenarios use
both, structured and unstructured data, and can be further
categorized in click stream-, social media-, server-log-,
sensor-, location-, text-, and video-/audio-data [21].
The Application of Big Data Analytics can contribute
to different targets and provide benefits in various fields.
The following sections can be divided into operational
efficiency, customer experience and new business models.
Big Data Analytics can enable new business models and
generate new revenue streams. Operational efficiency
based on Big Data capabilities uses data for better
decision-making and improvement in terms of process
quality, performance or resource consumption. The
section of customer experience is mainly assigned to
marketing and e.g. focusing on more precise customer
assessment, which also supports a company’s SCM [2,
22]. By focusing on logistics and supply chains,
Automatic identification and data capture (Auto-ID)
technologies like RFID and bar codes are widely used to
track handling units and the transported goods. Therefore,
many read-points have to be shared along the supply
chain.
Figure 1 shows the overarching model of data
categorization. Data in the outer circle is of higher
fuzziness, volume and change frequency. A wider
distance to the circle´s center implicates less control and
increasing ambiguity of both the data and its source. The
inner circle consists mostly of data that is owned by the
company and administrated within the companies IT
infrastructure, e.g. ERP systems.
 Proceedings of the 2014 IEEE IEEM
Fig. 1. Overarching model of data categorization
Measuring the benefits of Big Data is possible with
specific IT related Key Performance Indicators. These
KPIs refer to qualitative and quantitative aspects, and can
either be time-, quality- or financially oriented. The five
highest recommended KPIs are lead-time reduction,
increase of customer satisfaction, decrease of product
costs and time to market and contribution to sales increase
[22].
Both categorizations show, that there is a high
interlinking from an organisational point of view. The
application of Big Data Analytics for SCM affects other
departments, e.g. marketing and sales. For that reason,
KPIs have to consider various facets for a holistic
evaluation of use cases.
B. Potential Benefits
A Capgemini study [22] identifies machine-to-
machine-communication (M2M) as the issue with the
most gained importance in the last year. M2M data is
derived from sensors attached on machines or products,
which can collect various types of data like temperature,
usage data or positioning data. In 2013 11% of the
surveyed companies quoted, that they already use data
generated from machine-to-machine communications.
This increased significantly as in 2014 23% of the asked
companies use M2M data. Additionally, 12% of the
companies are in the status of implementation and another
13% plan to implement. M2M enables automatic
information exchange between different objects, e.g.
vending machines, cameras, transport vehicles, containers
and their corresponding database. Possible use cases can
contain activities like monitoring areas and machines,
increasing the maintenance of facilities and an automatic
ordering if demand is recognized. The use case of
automatic ordering can be fully automated up to a self-
distribution of necessary goods. Regarding to the above-
mentioned classification, machine-to-machine
communication will enable new business models and has
the potential to highly increase operational efficiency.
The goal of the smaRTI research project [23] was to
increase the intelligence of the material flow. Therefore,
the identification and localization of handling units, like
palettes, is achieved through the usage of Auto-ID
technologies. A wide spreading of read points, like
barcode or RFID reader, leads to an increasing
921
transparency of freight deliveries between companies.
When a handling unit is detected, the read points are
generating events, which are available in real-time and
enabling better transparency and sped up processes. This
is achieved due to planning optimization of deliveries and
supported detection of bottlenecks and risks. Latencies
can also be avoided or at least reduced. The project
finished 2013 with a pilot implementation for a candy
product supply chain with 18 read points shared between
the manufacturer, a supermarket and handling unit
pooling operator. The implementation focused in tracking
2500 handling units, which are transported in a loop
between the three companies. Around 90.000 read events
had been created. The outcome of the pilot
implementation was a complete transparent handling unit
flow between the three participants. All companies see a
benefit by a roll out of the smaRTI results on a complete
supply chain, not only for one product. Every handling
unit, and the transported goods could be directly located
as soon they reached a read point location. The next step
is to analyze the smaRTI data besides the business
processes data to detect not only bottlenecks or risks along
the supply chain, but also to find action within the
process, which can be optimized or excluded. An example
would be some manual quality measurement for counting
the received goods and comparing them with the expected
numbers. In summary the smaRTI approach can decrease
the loose of handlings units by 50% and can reduce the
goods in stock up to 10% in the consumer goods domain.
Use cases with regard to anticipatory shipping
intention – as described in [24] - will speed up delivery
times of goods and increase the utilization ratio of
distribution capacities. One use case example represents
DHLs volume forecast with predictive capacity utilization
and planning. The parcel volume analytics helps to
improve the prediction accuracy of expected parcels and
freight within their network. This is realized by
correlating data from different sources and with different
degree of privacy protection. Some input data could be the
company’s internal historic shipment data and external
events, public holidays, weather condition and forecast,
Google search terms, and the shopping behavior of online
customers.
Another example [25] is Amazons US Patent for
Anticipatory shipping from December 2013. The aim of
this patent is to ship goods prior to the customers’ order to
reduce delivery time. A prediction of upcoming orders is
the key element of the patent. This patent enables several
applications. First, a shipment is transported to a
destination area without knowing the complete delivery
address. During the shipment to the specific geographical
location, this address will be completed depending on
placed orders in the meantime. This optimized distribution
improves the lead time as well as the customer satisfaction
and can help to increase sales. Secondly, Amazon tries to
match some of the goods while they are already in transit
to a specific geographical location with current customer
orders. During the transport, the goods are added to
orders, which have been placed – as expected - by a
specific customer in this area. The motivation is to use the
disadvantage of lower-cost transportation (non-expedited
delivery) for buffering the speculatively selected items. If
one of the items in transit fits to a customers demand, the
 Proceedings of the 2014 IEEE IEEM
item will be delivered within a short delivery time. This
results in reduced transportation costs for both, Amazon
and the customer. The shopping process itself has major
influence on the forecast data generated for shipment and
delivery. Some information influencing the forecast could
be the specific web pages viewed and duration of views,
links hovered over with the mouse arrow and duration of
hovering, shopping cart, wish lists and the relatedness to
previous purchases, e.g. in the case of new product
releases, where no historical buying patterns from similar
customers exist.
V. CONCLUSION AND OUTLOOK
The supply chain management requires new solutions
due to the increasing complexity. Existing solutions and
IT Systems are not able to address this problem.
Therefore, new solutions have been developed in the last
years. As the use cases show, there are already systems
that not only allow a better handling of the vast amounts
of data, but also show how Big Data Analytics can
influence business models. This can also lead to new
business models as shown by the Amazon patent.
In this paper, we presented ongoing research work
about developing solutions for increasing the supply chain
visibility based on a data source classification and
potential benefits. The expectation for the future is that
various companies from different industries will form Big
Data ecosystems for gaining new business models and
providing new services to customers. This will lead to
even more rapidly increasing complexity for the SCM.
Depending on this, our future work will be about
designing solutions and IT systems including various
external sources. Therefore, we need the classification of
data sources to design a rule catalog for data cleansing -
including data quality measurement - and a terminology
for both, data cleansing and data analytics. The smaRTI
project shows that using Auto-ID technologies to track
handling units can optimize the material flow
transparency. Here the next step is to analyze these data
with business process and external data, like weather
information, to detect further risks and bottlenecks along
supply chains. Our next research work will focus on this.
REFERENCES
[1] Smart Service Welt Working Group, “Smart Service Welt:
Recommendations for the Strategic Initiative Web-based
Services for Businesses”, acatech, Berlin, 2014
[2] M. Jeseke, M. Grüner, F. Wieß, "BIG DATA IN
LOGISTICS: A DHL perspective on how to move beyond
the hype", DHL Customer Solutions & Innovation, 12.2013
[3] S. Robak, B. Franczyk, M. Robak, “Applying big data and
linked data concepts in supply chains management,” 2013
Federated Conference on Computer Science and
Information Systems (FedCSIS), pp. 1215 – 1221, 201
[4] A. Katal, M. Wazid, R. H. Goudar, “big data: Issues,
Challenges, Tools and Good Practices”, IEEE Sixth
International Conference on Contemporary Computing
(IC3), pp. 404-409 , 08.2013
[5] S. Ghemawat, H. Gobioff, S.T. Leung, “The Google File
System”, ACM SIGOPS Operating Systems Review, ACM,
pp. 29-43, 08.2003
[6] T. Berners-Lee, et al., “W3C Semantic Web Activity”
[Online]. Available: http://www.w3.org/2001/sw, 2001
922
[7] S. K. Rajapaksha, et al., “Internal Structure and Semantic
Web Link Structure Based Ontology Ranking”, ICIAFS
2008. 4th International Conference on Information and
Automation for Sustainability, pp. 86-90, 2008
[8] D. Pfisterer, et al., “SPITFIRE: Toward a Semantic Web of
Things”, Communications Magazine, IEEE (Volume: 49,
Issue: 11 ), pp. 40-48, 2009
[9] W. Hofman, “supply chain visibility with Linked Open
Data for supply chain Risk Analysis”, Workshop on IT
Innovations Enabling Seamless and Secure supply chains,
pp. 20-31, 08.2011
[10] supply chain Council (SCC), supply chain Operations
Refernce (SCOR) Overview Version 10.0, 2010,
https://supply-chain.org/f/Web-Scor-Overview.pdf
[11] K. Bakshi, “Considerations for big data: architecture and
Approach “, Aerospace Conference, 2012 IEEE, ISBN 978-
1-4577-0556-4, pp 1 – 7, 2012
[12] C. Sooriaarachchi, T. Gunawardena, B. Kulasuriya, T.
Dayaratne, “A study into the capabilities of NoSQL
databases in handling a highly heterogeneous tree”, IEEE
6th International Conference on Information and
Automation for Sustainability (ICIAfS), ISBN 978-1-4673-
1976-8 , pp. 106-111 ,2012
[13] N. Marz and J. Warren, “Big data. Principles and best
practices of scalable realtime data systems”. Manning
Publications, MEAP Edition, Manning Early Access
Program big data version 7, 2012.
[14] J. Xingyi, “Efficient Complex Event Processing over RFID
Data Stream”, Seventh IEEE/ACIS International
Conference on Computer and Information Science, ISBN
978-0-7695-3131-1, pp. 75-81, 2008
[15] B. Heaney, “supply chain visibility – A Critical Strategy to
Optimize Cost and Service”, [Online]
http://aberdeen.com/Aberdeen-Library/8509/RA-supply-
chain-visibility.aspx, 2013
[16] R. Siow Mong, Z. Wang, et.al., ”RiskVis: supply chain
Visualization with Risk management and Real - time
Monitoring”, IEEE International Conference on
Automation Science (CASE), 207-212, 2013
[17] M. Barratt, and A. Oke, “Antecedents of supply chain
visibility in retail supply chains: A resource-based theory
perspective,” Journal of Operations management, vol. 25,
pp. 1217-1233, 2007
[18] J. L. Griffiths, A. Phelan, K. A. Osman, and A. Furness,
“Using item attendant information and communications
technologies to improve supply chain visibility,” Agile
Manufacturing, 2007. ICAM 2007, pp. 172 – 180, 2007
[19] P. Haase, D1.1.5 –Updated version of the networked
ontology model, EU F6 NeOn: Lifecycle support for
networked ontologies, 2009
[20] L. Kart, N. Heudecker, F. Buytendijk, „Survey Analysis:
big data Adoption in 2013 Shows Substance Behind the
Hype“, Gartner, 2013
[21] BITKOM-Arbeitskreis Big Data, "Big-Data-Technologien
– Wissen für Entscheider", BITKOM, 2014
[22] Capgemini Deutschland Holding GmbH, "Studie IT-Trends
2014: IT-Kompetenz im Management steigt"(in German),
Capgemini Deutschland Holding GmbH, 2014
[23] Effizienzcluster Management GmbH, Smart Reusable
Transport Items (smaRTI) - Intelligent Material Flow,
(online) http://www.effizienzcluster.de/en/leitthemen_
projekte/projekt.php?proPid=5, 2014
[24] N. Bubner, Ni. Bubner, R. Helbig, M. Jeske, „LOGISTICS
TREND RADAR: Delivering insight today. Creating value
tomorrow!”, DHL Customer Solutions & Innovation, 2014
[25] J. R. Spiegel, M. T. McKenna, G. S. Lakshman, P. G.
Nordstrom, “Amazon US Patent Anticipatory Shipping”,
Amazon Technologies Inc., 12.2013