Non-Stationary Models

This file will cover two non-stationary models

  • Autoregressive integrated moving average (ARIMA) models
  • ARUMA

Essentially, ARIMA are ARMA models extended to have roots on the unit circle. Technically, ARIMA models are a special case of ARUMA model. In this class, we will focus on the using ARUMA to model seasonality.

ARIMA(p,d,q)

An ARIMA model is described by the equation

$$ \phi \left( B \right) \left( 1 - B \right)^d X_t = \theta \left( B \right) a_t $$

where

  • All the roots of $\phi \left( B \right)$ and $\theta \left( B \right)$ are outside the unit circle
  • $d$ is a non-negative integer
  • Differences of $\left( 1 - B \right)^d X_t$ satisfy a stationary model (ARMA)

The $\left( 1 - B \right)^d$ factor of the equation makes the model non-stationary

Properties of ARIMA

  • The roots on the unit circle dominate the behavior of the realization
  • The autocorrelations are defined to have a magnitude of 1 $\left( \rho_k = 1 \right)$
    • These are sometimes refered to as "extended autocorrelations" and denoted with $\rho^*_k$
  • The variance of ARIMA is not well defined.
    • The root on the unit circle causes the variance to be infinite.

ARIMA(0,1,0)

Let $Y_t$ be defined as

$$ Y_t = (1-B) X_t $$

Then

$$Y_t = a_t$$

Thus,

$$ X_t = X_{t-1} + a_t $$

Observations from ARIMA(0,1,0)

  • The value at time $t$ is equal to the value at time $t-1$ plus random noise.
  • At each time $t$, the process is equally likely to move up or down i.e. a random walk.

Additional Comments

Sample autocorrelations that are slowly descreasing in magnitude are a sign that an ARIMA model is appropriate.

ARUMA(p,d,q)

An ARUMA model is described by the equation

$$ \phi \left( B \right) \left( 1 - B \right)^d \left( 1- B^s \right) X_t = \theta \left( B \right) a_t $$

where

  • All the conditions from ARIMA are met
  • $s$ is a non-negative integer

ARUMA is an generalization of ARIMA that includes a term or term(s) for seasonality.

Seasonality

Some examples of seasonality are quarterly and monthly. The factors for quarterly seasonality is shown. Notice that these factors have roots on the unit circle, making them non-stationary components.

$$ \left( 1 - B^4 \right) = (1-B) (1+ B) \left( 1 + B^2 \right) $$

Extensions

ARUMA really is a more general model than the combination of ARIMA with seasonality. It includes any model with additional non-stationary factors that do not form seasonal or integrated forms.

Notation

The orders of autoregressive, integration, and moving average are denoted with p, d, and q respectively. So an ARIMA model is described with ARIMA(p,d,q). There is no generalization for seasonality or other non-stationary factors. So the factor orders of ARUMA are denoted in the same way as ARIMA.

References

  • [1] W. Woodward and B. Salder, "ARIMA Models and Seasonality", SMU, 2019