CARLSON SCHOOL SUPPLY CHAIN & OPERATIONS INSIGHTS

Vol. 25, No. 3 – May 2014

www.carlsonschool.umn.edu/supply-chain-operations

Predicting demand forecastability for inventory planning

The benefit of an inventory planning system depends on the accuracy of the forecasting model used. The
new Q-Metric measures the “forecastability” of historical demand data and can be used to estimate the
benefits of a forecasting system over a simple reorder point system.

Is my crystal ball any good?

Most retailers, distributors, and manufacturers invest

significant resources in developing better demand
forecasts for inventory planning. Unfortunately, much of
this time and money is wasted because the demand for
many items is “unforecastable.” Managers need to know
if their forecasting “crystal ball” is working and they need
to know which items are forecastable and which are not.
A reliable measure of demand forecastability can help
managers evaluate the performance of a forecasting
system and select the right type of planning and control
system to use to manage items stored in inventory.

Several highly touted metrics for estimating the

forecastability of demand are widely used by well-
managed firms such as General Mills and Honeywell. These include time series metrics (e.g., coefficient
of variation - CV) and forecast accuracy metrics (e.g., mean absolute percent error - MAPE). But are
these metrics useful for measuring forecastability in practice?

When is my crystal ball any good?

In a recent study, a research team led by Professor Arthur Hill in the Supply Chain and Operations
Department at the University of Minnesota’s Carlson School of Management determined that previously
developed measures of forecastability are not effective. They developed the new “Q-Metric” that
reliably measures the economic advantage of using a forecasting system over a simple reorder point
(pull) system. The Q-Metric is defined as the ratio of the standard deviation of demand and the
standard deviation of forecast error for double exponential smoothing (a simple forecasting method).
When the Q-Metric is greater than one, the variability of the demand is higher than that of the forecast
error for a simple forecasting model, and the potential benefit of the forecasting model is high. For
example, a Q-Metric of 110% means that the inventory carrying cost for a reorder point (pull) system
will likely be 10% more than for a forecasting-based system. Conversely, when the Q-Metric is less than
one, the variability of the demand is less than that of the forecast error and the potential benefit of the
forecasting model is low. For example, a Q-Metric of 90% means that the inventory carrying cost for a
reorder point (pull) system will be likely be 10% less than for a forecasting-based system.

Copyright © 2014 Professor Arthur V. Hill, Carlson School of Management, University of Minnesota
The researchers analyzed the Q-Metric on a database of historical demand with thousands of items from
a Fortune 100 firm. They de-seasonalized the demand for each item and divided it into an estimation
sample (the earlier half of the time series demand data) and a holdout sample (the latter half of the
demand data). Three major surprises emerged from this analysis.

Surprise 1. The first major surprise was that the Q-Metric for most items was less than one, which
means that most items were not forecastable. This was quite interesting given that the firm was
spending millions of dollars to create statistical forecasts and then even more money in having analysts
review and tamper with these forecasts.

Surprise 2. The second major surprise was that highly touted time series metrics (e.g., CV) and forecast
accuracy metrics (e.g., MAPE) were not good predictors of the Q-Metric in the holdout sample. (Note:
Many time series metrics were considered including CV, skewness, kurtosis, t-statistic for trend, first-
order sample autocorrelation, runs test statistic, Durbin Watson statistic, and approximate entropy.
Many forecast accuracy metrics were also considered including MAPE, mean absolute scaled error
(MASE), Theil’s U, percent variation explained, noise-to-signal ratio, and demand-forecast correlation.)

Surprise 3. The third major surprise was that the Q-Metric in the holdout sample could be accurately
predicted with just three variables measured in the estimation sample. The research team validated this
model for predicting the Q-Metric on four additional datasets with a wide variety of products including
batteries, hand tools, tires, and Department of Defense repair parts.

Practical advice for using crystal balls

 Stop believing the conventional wisdom that time series metrics such as the coefficient of variation
(CV) measure the forecastability of the demand history for an item.
 Stop using flawed forecast accuracy metrics such as the mean absolute percent error (MAPE). The
MASE is better and the Q-Metric is even better.
 Use the new Q-Metric to measure forecastability. The Q-Metric is directly related to the economics
of a forecasting system for inventory planning and is a good measure of the economic value of a
forecasting system vis-à-vis a reorder point (pull) system.
 Use the predicted Q-Metric to triage items. For items that are not forecastable (i.e., Q < 1), stop
wasting resources developing sophisticated forecasting systems. Instead, use a reorder point (pull)
system and focus on reducing leadtimes, stabilizing demand, and improving supply chain visibility.

Investing time and money in developing forecasts for unforecastable items adds noise and nervousness
to the system and adds no value to anyone except the forecasting software vendor.

Reference: Hill, A.V., W. Zhang, and G.F. Burch (2014). “Forecasting the Forecastability Quotient for Inventory
Management,” under second review by the International Journal of Forecasting.

Contact: Arthur V. Hill, Associate Dean for MBA Programs and John and Nancy Lindahl Professor, Carlson School of
Management, University of Minnesota, Supply Chain and Operations Department, 321 19th Avenue South,
Minneapolis, MN 55455-0413. Voice 612-624-4015. Email ahill@umn.edu.

Revised April 24, 2014

Copyright © 2014 Professor Arthur V. Hill, Carlson School of Management, University of Minnesota Page 2