Arima Model 735t5w

ARIMA Modelling and Forecasting

By: Amar Kumar

Introduction • Stationarity of the time series • AR and MA Models • ARIMA Models (Auto-Regressive Integrated Moving Average) • Diagnostic Checks

• Forecast using ARIMA models

Stationarity of the time series • If an AR model is not stationary  previous values of the error term will have a non-declining effect on the current value of the dependent variable.  The coefficients on the MA process would not converge to zero as the lag length increases. • For an AR model to be stationary, the coefficients on the corresponding MA process decline with lag length, converging on 0.

Stationary Series (1/2) <= homoscedasticity

How to deal with non-stationarity: Model the differences of the and not the actual term. x(t) – x(t-1) = ARMA (p , q)

Stationary Series (2/2) 1.

2.

3.

Dickey Fuller Test of Stationarity X(t) = Rho * X(t‐1) + Er(t) Rho- introduced coefficient

 X(t) ‐ X(t‐1) = (Rho ‐ 1) X(t ‐ 1) + Er(t) Ho: (Rho – 1)=0 If null hypothesis is rejected, we’ll get a stationary time series. *Also called unit root test

Auto-Regressive Time Series Model x(t) = alpha * x(t – 1) + error (t) AR(1) Mean reverting behaviour

Moving Average Time Series Model x(t) = beta * error(t-1) + error (t) MA(1)  Noisy pattern

AR Signature

MA Signature

AR or MA? It depends! • Whether a series displays AR or MA behavior often depends on the extent to which it has been differenced. • An “underdifferenced” series has an AR signature (positive autocorrelation) • After one or more orders of differencing, the autocorrelation will become more negative and an MA signature will emerge • Don’t go too far: if series already has zero or negative autocorrelation at lag 1, don’t difference again

ARIMA Model Box-Jenkins Methodology

• A series which needs to be differenced to be made stationary is an “integrated” (I) series • Lags of the stationarized series are called “autoregressive” (AR) • Lags of the forecast errors are called “moving average” (MA) • ARIMA(p,d,q) p = the number of autoregressive d = the number of nonseasonal differences q = the number of moving-average

The ARIMA “filtering box”

Diagnostic Checks • With this approach we only test for autocorrelation Ljung-Box statistic. • If there is evidence of autocorrelation, we need to go back to the identification stage and respecify the model, by adding more lags.

• A criticism of this approach is that it fails to identify if the model is too big or overparameterised, it only tells us if it is too small.

Parsimonious Model • The aim of this type of modelling is to produce a model that is parsimonious, or as small as possible, whilst ing the diagnostic checks. • A parsimonious model is desirable because including irrelevant lags in the model increases the coefficient standard errors and therefore reduces the t-statistics. • Models that incorporate large numbers of lags, tend not to forecast well, as they fit data specific features, explaining much of the noise or random features in the data.

Measuring Forecast Accuracy • To determine how accurate a forecast is, the simplest method is to plot the forecast against the actual values as a direct comparison • In addition it may be worthwhile to compare the turning points, this is particularly important in finance. • There are a number of methods to determine accuracy of the forecast, often more than one is included in a set of results.

Mean Squared Error (MSE) • The MSE statistic can be defined as: T 1 2 MSE  ( y  f )  t s t ,s T  (T1  1) t T1

T  total sample size T1  first out - of - sample forecast observatio n f t ,s  s - step - ahead forecasts made at time t yt  s  the actual value at time t

MSE Example Steps Ahead 1 2 3

Forecast

Actual

0.1 0.25 0.5

0.15 0.20 0.40

Squared Error 0.0025 0.0025 0.01

MSE  (0.0025  0.0025  0.01) / 3  0.005

Forecast Accuracy • There are a number of other measures used and these include: – Mean Absolute Error – Mean Average Prediction Error – Chows test for predictive failure – Theil’s U-statistic (where the forecast is compared to that of a benchmark model) – Root Mean Square Error ( the square root of the MSE)

Application of ARIMA model library(tseries) ; library(graphics) ; data(Airengers) ; par(mfrow=c(2,1)) plot(Airengers) ; plot(log(Airengers)) ; plot(diff(log(Airengers))) adf.test(diff(log(Airengers)), alternative="stationary", k=0) acf(log(Airengers)) ; acf(diff(log(Airengers))) pacf(diff(log(Airengers))) fit <- arima(Airengers, order=c(0, 1, 1), list(order=c(0, 1, 0), period = 12)) fore <- predict(fit, n.ahead=24) U <- fore$pred + 2*fore$se L <- fore$pred - 2*fore$se ts.plot(Airengers, fore$pred, U, L, col=c(1, 2, 4, 4), lty=c(1, 1, 2, 2)) legend("topleft", c("Actual", "Forecast", "Error Bounds (95% prediction interval)"), col=c(1, 2, 4),lty=c(1, 1, 2)) res <- residuals(fit) ; acf(res) ; pacf(res) qqnorm(residuals(fit)) ; qqline(residuals(fit)) adf.test(fit$residuals, alternative ="stationary") tsdiag(fit) Box.test(resid(fit), 12, type="Ljung", fitdf = 1)

Conclusion • When using ARIMA models, whether the series is stationary or not determine how stable it is. • Box-Jenkins methodology is part art and part science. • Forecasting of time series is an important measure of how well a model works • There are many measures of how accurate a forecast is, usually a number is calculated to determine if the forecast is acceptable, although all have faults.