Autoregressive models are expressed as dependence on lagged values. These types of models are generally good are modeling stationary processes that move forward in time.
First order AR models AR(1) and second order AR models AR(2) are discussed first. Then the details from these based models are used to discuss the generalizations to pth order AR models AR(p).
The first order autoregressive model AR(1) is expressed as
$$ X_t = \beta + \phi X_{t-1} + a_t $$where $\beta = (1-\phi)\mu$
This model is sometimes written in zero-mean form as
$$ X_t = \phi X_{t-1} + a_t $$The interpretation of AR(1) is "the value of the process at time $t$ depends on the value of the process at $t-1$ plus a random noise component."
There are several properties that depend on the value of $\phi$ and the sign of $\phi$.
Realizations with $\left| \phi \right| = 1$, look similar to realizations that are encountered in practice. These are a special case of ARIMA realizations. Realizations with $\left| \phi \right| > 1$, do not looks like real realizations in practice.
The characteristic equation of an AR(1) model can be found by using backshift notation to factor the model. Shown here with the zero mean form AR(1).
$$ X_t - \phi X_{t-1} = a_t $$Noting that $B X_t = X_{t-1}$ and setting $a_t$ to 0
$$ (1-\phi B)X_t = 0 $$The characteristic equation is
$$ (1-\phi B) = 0 $$and the root is
$$ root = \frac{1}{\phi} $$Since AR(1) is stationary if and only if $\left| \phi \right| < 1$, then an AR(1) model will be stationary if the root of the characteristic equation is $\left| \frac{1}{\phi} \right| > 1$.
The first order autoregressive model AR(1) is expressed as
$$ X_t = \beta + \phi_1 X_{t-1} + \phi_2 X_{t-2} + a_t $$where $\beta = (1 -\phi_1 - \phi_2) + \mu$
The zero-mean for of the model is written as
$$ x_t - \phi_1 B X_{t} - \phi_2 B^2 X_{t} = a_t $$Which is typically written in one of the following forms:
The interpretation of AR(2) is "the value at time $t$ is a linear combination of two values at the two previous time periods plus a random noise component.
Starting from the zero-mean model, we have
$$ x_t - \phi_1 B X_{t} - \phi_2 B^2 X_{t} = a_t $$Then setting $a_t$ to 0, we have the characteristic equation
$$ 1 - \phi_1 B - \phi_2 B^2 = 0 $$There will either be two real roots or the roots will be a complex conjugate pair.
The beahvior of the AR(2) model is based on the roots of the characteristic equation.
Where $f_0$ is given by
$$ f_0 = \frac{1}{2\pi}cos^{-1} \left( \frac{\phi_1}{2 \sqrt{-\phi_2}} \right) $$An AR(p) model is expressed as
$$ X_t = \beta + \phi_1 X_{t-1} + \phi_2 X_{t-2} + ... + \phi_2 X_{t-p} + a_t $$where $\beta = (1 -\phi_1 - \phi_2 - ... - \phi_p) + \mu$
The zero-mean for of the model is written as
$$ x_t - \phi_1 B X_{t} - \phi_2 B^2 X_{t} - \phi_p B^p X_{t} = a_t $$AR(p) models reflect a contribution of AR(1) and AR(2) contributions.
Roots that are close to the unit circle will be the dominate behavior.
High order AR models are summerized in a factor table to help present the behaviors that are present in the model.
Example
The model is
$$ X_t - 1.95 X_{t-1} - 1.85 X_{t-2} - 0.855 X_{t-3} = a_t $$factors to
$$ (1 - 0.95B) (1-B + 0.9B^2) $$Then the table is constructed from the factors
| Factor | Root(s) | Abs Reciprocal | System Frequency |
|---|---|---|---|
| 1- 0.95 $B$ | $1.053$ | $0.95$ | $0$ |
| 1 - $B$ + 0.95 $B^2$ | 0.556 $\pm$ 0.896i | $0.95$ | $0.16$ |
Notes from the table: