:

Time Series Models

and Forecasting and Forecasting
Time Series &Forecasting
Time Time--Series data Series data are data which are measured over Time Time--Series data Series data are data which are measured over
time. In most applications the period between
measurements is uniform.
Time Time- -Series Series is a is an ordered sequence of values of a
variable at equally spaced time intervals.
Time Series &Forecasting
Applications:
The usage of time series models is twofold:
Obtain an understanding of the underlying forces and structure that Obtain an understanding of the underlying forces and structure that
produced the observed data
Fit a model and proceed to forecasting, monitoring or even feedback
and feedforward control.
Time Series Analysis is used for many applications such as:
Economic Forecasting
Sales Forecasting
Budgetary Analysis
Workload Projections
Utility Studies
Census Analysis
Budgetary Analysis
Stock Market Analysis
Yield Projections
Process and Quality
Control
Inventory Studies
Census Analysis
and many, many more...
Time-Series Plot
1000
A time time--series plot series plot is a two-dimensional plot of the time series.
The vertical axis measures the variable of interest and the
horizontal axis corresponds to the time periods.
200
300
400
500
600
700
800
900
1000
$

x

1
,
0
0
0
0
100
Months
(Figure 1) (Figure 1)
Forecasting Horizons
Long Term
- 5+ years into the future
- R&D, plant location, product planning - R&D, plant location, product planning
- Principally judgement-based
Medium Term
- 1 season to 2 years
- Aggregate planning, capacity planning, sales forecasts
- Mixture of quantitative methods and judgement
Short Term Short Term
- 1 day to 1 year, less than 1 season
- Demand forecasting, staffing levels, purchasing, inventory levels
- Quantitative methods
Short TermForecasting:
Needs and Uses
Scheduling existing resources
- How many employees do we need and when? - How many employees do we need and when?
- How much product should we make in anticipation of demand?
Acquiring additional resources
- When are we going to run out of capacity?
- How many more people will we need?
- How large will our back-orders be?
Determining what resources are needed Determining what resources are needed
- What kind of machines will we require?
- Which services are growing in demand? declining?
- What kind of people should we be hiring?
Types of Forecasting Models
Types of Forecasts
- Qualitative --- based on experience, judgement, knowledge;
- Quantitative --- based on data, statistics;
Methods of Forecasting
- Naive Methods --- eye-balling the numbers;
- Formal Methods --- systematically reduce forecasting errors;
time series models (e.g. exponential smoothing);
causal models (e.g. regression).
- Focus here on Time Series Models - Focus here on Time Series Models
Assumptions of Time Series Models
- There is information about the past;
- This information can be quantified in the form of data;
- The pattern of the past will continue into the future.
Forecasting Examples
Examples from student projects:
- Demand for tellers in a bank;
- Traffic on major communication switch; - Traffic on major communication switch;
- Demand for liquor in bar;
- Demand for frozen foods in local grocery warehouse.
Example from Industry: American Hospital Supply Corp.
- 70,000 items;
- 25 stocking locations;
- Store 3 years of data (63 million data points); - Store 3 years of data (63 million data points);
- Update forecasts monthly;
- 21 million forecast updates per year.
Simple Moving Average
Forecast F
t
is average of n previous observations or
actuals D
t
:
1
( ) F D D D = + + +
Note that the n past observations are equally weighted.
1 1 1
1
1
( )
1
t t t t n
t
t i
i t n
F D D D
n
F D
n
+ +
+
= +
= + + +
=


Issues with moving average forecasts:
- All n past observations treated equally;
- Observations older than n are not included at all;
- Requires that n past observations be retained;
- Problem when 1000's of items are being forecast.
Simple Moving Average
Include n most recent observations
Weight equally
Ignore older observations Ignore older observations
weight
1/n
today
1 2
3
... n
Moving Average
Internet Unicycle Sales
n = 3
200
250
300
350
400
450
U
n
i
t
s
n = 3
0
50
100
150
Apr-01 Sep-02 Jan-04 May-05 Oct-06 Feb-08 Jul-09 Nov-10 Apr-12 Aug-13
Month
Example:
Moving Average
Forecasting
Moving Average Models (Example)
Months Crimes n = 3 errors
Moving Average
Actual
Forecast
50 48
51 35 #N/A #N/A
52 49 #N/A #N/A
53 44 44.00 #N/A
54 61 42.67 #N/A
55 68 51.33 6.33
56 82 57.67 8.21
0
10
20
30
40
50
60
70
80
90
50 51 52 53 54 55 56 57 58
C
r
i
m
e
s
Forecast
57 71 70.33 10.59
58 50 73.67 9.13
59 67.67 12.32
Months
Weighted Moving Average
Forecast F
t
is weighted average of n previous
observations or actuals D
t
:
1 1 2 1 1 t t t n t n
F w D w D w D
+ +
= + + +
Note that the n past observations are unequally weighted.
1 1
1
1
,
w h ere 1, ,
t
t t i i
i t n
n
k i j
k
F w D
w w w i j
+ +
= +
=
=
= > <

Note that the n past observations are unequally weighted.

Issues with weighted moving average forecasts:
- Decreasing weights given to older observations;
- Observations older than n are not included at all;
- Requires that n past observations be retained;
- Problem when 1000's of items are being forecast.
Weighted Moving Average
Include n past observations
Weight recent observations much more heavily
than very old observations: than very old observations:
weight
Decreasing weight given
to older observations
n n

( 1) n n

today

1 n

n
Example:
Weighted Moving
Average Forecasting
Weighted Moving Average
(Example)
Actual wMA(n=3)
Month Burglaries 100.00% =SUM(C4:C6) All weights should add-up exactly to 1
To determine best
weights and
Month Burglaries 100.00% =SUM(C4:C6) All weights should add-up exactly to 1
42 88 0.1 The further away from the forecast period
43 44 0.3 weights: the lower is the weight
44 60 0.6 Most recent observation has the highest weight
45 56 58.0 =B5*$C$6+B4*$C$5+B3*$C$4
46 70 56.0 =B6*$C$6+B5*$C$5+B4*$C$4
47 91 64.8 =B7*$C$6+B6*$C$5+B5*$C$4
48 54 81.2 :
49 60 66.7 :
50 48 61.3 :
51 35 52.2 :
52 49 41.4 :
53 44 44.7 :
weights and
period (n) we can
use forecast
accuracy.
MSE = Mean
Square Error is a
good measure for
53 44 44.7 :
54 61 44.6 :
55 68 54.7 :
56 82 63.5 :
57 71 75.7 :
58 50 74.0
59 59.5 Preliminary forecasted number of burglaries
MSE = 256.3 =SUMXMY2(B7:B20,C7:C20)/COUNT(B7:B20)
RMSE = 16.01 =SQRT(C22)
good measure for
forecast accuracy.
RMSE = is the
square root of the
MSE.
Weighted Moving Average
(Example)
Best weights for a given n (in this
Actual wMA(k=3)
Month Burglaries 100.00%
Best weights for a given n (in this
case 3) is determined by
minimizing RMSE.
Same procedure could be applied
to models with different ns and
the one with lowest RMSE could
be considered as the model with
Month Burglaries 100.00%
42 88 0.0285
43 44 0.2093
44 60 0.7622
45 56 57.5
46 70 56.5
47 91 66.8
48 54 85.6
49 60 62.2
50 48 59.6
51 35 50.7
52 49 38.4
be considered as the model with
best forecasting period.
53 44 46.0
54 61 44.8
55 68 57.1
56 82 65.8
57 71 78.5
58 50 73.2
59 55.3
MSE = 250.6
RMSE = 15.83
Exponential Smoothing: Concept
Include all past observations
Weight recent observations much more heavily
than very old observations: than very old observations:
weight
Decreasing weight given
to older observations
0 1 < <

2
(1 )
(1 )
today

(1 )
3
(1 )
Exponential Smoothing: Math
| |
2
1 2
1 2
(1 ) (1 )
(1 ) (1 )
t t t t
t t t t
F D D D
F D D D




= + + +
= + + +

1
(1 )
t t t
F D F

= +
| |
1 2 t t t t
Exponential Smoothing: Math
2
1 2
1
(1 ) (1 )
(1 )
t t t t
t t t
F D D D
F D F

= + + +
= +

Thus, new forecast is weighted sum of old forecast and actual
demand
Notes:
- Only 2 values (D
t
and F
t-1
) are required, compared with n for moving
average
- Parameter determined empirically (whatever works best)
1 t t t
- Rule of thumb: < 0.5
- Typically, = 0.2 or = 0.3 works well
Forecast for k periods into future is:
t k t
F F
+
=
Exponential Smoothing
450
Internet Unicycle Sales (1000's)
150
200
250
300
350
400
450
U
n
i
t
s
a = 0.2
0
50
100
150
Jan-03 May-04 Sep-05 Feb-07 Jun-08 Nov-09 Mar-11 Aug-12
Month
Example:
Exponential Smoothing
Complicating Factors
Simple Exponential Smoothing works well Simple Exponential Smoothing works well
with data that is moving sideways
(stationary)
Must be adapted for data series which
exhibit a definite trend
Must be further adapted for data series
which exhibit seasonal patterns
Must be further adapted for data series
which exhibit seasonal patterns
Holts Method:
Double Exponential Smoothing
What happens when there is a definite trend?
A trendy clothing boutique has had the following sales
over the past 6 months: over the past 6 months:
1 2 3 4 5 6
510 512 528 530 542 552
530
540
550
560
D
e
m
a
n
d
Actual
Forecast
480
490
500
510
520
530
1 2 3 4 5 6 7 8 9 10
Month
D
e
m
a
n
d
Forecast
Holts Method:
Double Exponential Smoothing
Ideas behind smoothing with trend:
- ``De-trend'' time-series by separating base from trend effects
- Smooth base in usual manner using - Smooth base in usual manner using
- Smooth trend forecasts in usual manner using
Smooth the base forecast B
t
Smooth the trend forecast T
t
1 1
(1 )( )
t t t t
B D B T

= + +
( ) (1 ) T B B T

= +
Forecast k periods into future F
t+k
with base and trend
1 1
( ) (1 )
t t t t
T B B T

= +
t k t t
F B kT
+
= +
ES with Trend
Internet Unicycle Sales (1000's)
200
250
300
350
400
450
U
n
i
t
s
a = 0.2, b = 0.4
0
50
100
150
Jan-03 May-04 Sep-05 Feb-07 Jun-08 Nov-09 Mar-11 Aug-12
Month
Example:
Exponential Smoothing
with Trend
Winters Method:
Exponential Smoothing
w/ Trend and Seasonality
Ideas behind smoothing with trend and seasonality:
- De-trend: and de-seasonalizetime-series by separating base from - De-trend: and de-seasonalizetime-series by separating base from
trend and seasonality effects
- Smooth base in usual manner using
- Smooth trend forecasts in usual manner using
- Smooth seasonality forecasts using g
Assume m seasons in a cycle
- 12 months in a year - 12 months in a year
- 4 quarters in a month
- 3 months in a quarter
- et cetera
Winters Method:
Exponential Smoothing
w/ Trend and Seasonality
Smooth the base forecast B
t
Smooth the trend forecast T
t
1 1
(1 )( )
t
t t t
t m
D
B B T
S

= + +
1 1
( ) (1 )
t t t t
T B B T

= +
Smooth the seasonality forecast S
t
(1 )
t
t t m
t
D
S S
B


= +
Winters Method:
Exponential Smoothing
w/ Trend and Seasonality
Forecast F
t
with trend and seasonality
Smooth the trend forecast T
t
1 1
( )
t k t t t k m
F B kT S
+ +
= +
1 1
( ) (1 )
t t t t
T B B T

= +
Smooth the seasonality forecast S
t
(1 )
t
t t m
t
D
S S
B


= +
ES with Trend and Seasonality
Internet Unicycle Sales (1000's)
200
250
300
350
400
450
500
U
n
i
t
s
a = 0.2, b = 0.4, g = 0.6
0
50
100
150
200
Jan-03 May-04 Sep-05 Feb-07 Jun-08 Nov-09 Mar-11 Aug-12
Month
Example:
Exponential Smoothing Exponential Smoothing
with
Trend and Seasonality
Forecasting Performance
Mean Forecast Error (MFE or Bias): Measures
average deviation of forecast from actuals.
How good is the forecast?
average deviation of forecast from actuals.
Mean Absolute Deviation (MAD): Measures
average absolute deviation of forecast from
actuals.
Mean Absolute Percentage Error (MAPE):
Measures absolute error as a percentage of the Measures absolute error as a percentage of the
forecast.
Standard Squared Error (MSE): Measures
variance of forecast error
Forecasting Performance Measures
1
( )
n
t t
MFE D F
n
=

1
( )
t t
t
MFE D F
n
=
=

1
1
n
t t
t
MAD D F
n
=
=

100
n
t t
D F
MAPE

=

1
t t
t
t
MAPE
n D
=
=

2
1
1
( )
n
t t
t
MSE D F
n
=
=

Mean Forecast Error (MFE or Bias)

1
( )
n
t t
MFE D F
n
=

Want MFE to be as close to zero as possible --

minimum bias
A large positive (negative) MFE means that the
forecast is undershooting (overshooting) the actual
1
( )
t t
t
MFE D F
n
=
=

forecast is undershooting (overshooting) the actual

observations
Note that zero MFE does not imply that forecasts are
perfect (no error) -- only that mean is on target
Also called forecast BIAS
Mean Absolute Deviation (MAD)
1
n
t t
MAD D F
n
=

Measures absolute error

Positive and negative errors thus do not cancel out (as
with MFE)
1
t t
t
MAD D F
n
=
=

Want MAD to be as small as possible

No way to know if MAD error is large or small in
relation to the actual data
Mean Absolute Percentage Error
(MAPE)

Same as MAD, except ...

Measures deviation as a percentage of actual data
1
100
n
t t
t
t
D F
MAPE
n D
=

=

Measures deviation as a percentage of actual data
Mean Squared Error (MSE)
2
1
( )
n
MSE D F =

Measures squared forecast error -- error variance

Recognizes that large errors are disproportionately more
expensive than small errors
2
1
1
( )
t t
t
MSE D F
n
=
=

expensive than small errors

But is not as easily interpreted as MAD, MAPE -- not as
intuitive
Components of Time Series Data
Trend Component Trend Component
Seasonal Component
Cyclical Component
Random Component
Components of Time Series Data
A linear trend linear trend is any long-term increase or decrease
in a time series in which the rate of change is
relatively constant. relatively constant.
A seasonal component seasonal component is a pattern that is repeated
throughout a time series and has a recurrence
period of at most one year.
A cyclical component cyclical component is a pattern within the time
series that repeats itself throughout the time series
and has a recurrence period of more than one year. and has a recurrence period of more than one year.
The random component random component refers to changes in the
time-series data that are unpredictable and cannot
be associated with the trend, seasonal, or cyclical
components.
Trend-Based Forecasting
Techniques
LINEAR TREND MODEL LINEAR TREND MODEL LINEAR TREND MODEL LINEAR TREND MODEL
where:
y
t
= Value of trend at time t
0 1 t t
y t = + +
y
t
= Value of trend at time t

0
= Intercept of the trend line

1
= Slope of the trend line
t = Time (t = 1, 2, . . . )
Linear Trend Model (Example)
Year t Sales
Taft Ice Cream Sales
Year t Sales
1991 1 $300,000
1992 2 $295,000
1993 3 $330,000
1994 4 $345,000
1995 5 $320,000
1996 6 $370,000 1996 6 $370,000
1997 7 $380,000
1998 8 $400,000
1999 9 $385,000
2000 10 $430,000
Linear Trend Model
(Example, Contd.) (Example, Contd.)
$500,000
$200,000
$250,000
$300,000
$350,000
$400,000
$450,000
S
a
l
e
s
$0
$50,000
$100,000
$150,000
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
Year
Linear Trend Model
(Example, Contd.) (Example, Contd.)
LEAST SQUARES EQUATIONS LEAST SQUARES EQUATIONS
t y

1 2
2
t
t
t y
t y
n
b
t y
t
n

0 1
t
y t
b b
n n
=

where:
n = Number of periods in the time series
t = Time period independent variable
y
t
= Dependent variable at time t
n n
Linear Trend Model
(Example, Contd.) (Example, Contd.)
Taft Linear Trend Model
$500,000
y = 14576t + 277333
$150,000
$200,000
$250,000
$300,000
$350,000
$400,000
$450,000
$500,000
S
a
l
e
s
$0
$50,000
$100,000
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
Year
Linear Trend Model
- Forecasting -
Trend Projection: Trend Projection:
Forecasting Period t = 11:
277, 333.33 14, 575.76( )
t
F t = +
277, 333.33 14, 575.76(11)
$437, 666.69
t
F = +
Nonlinear Trend Models
2
y t = + +
2
0 1 t t
y t = + +