Causality and Invertibility

Causality and Invertibility

For an ARMA(p,q) process $\varphi (B)X_t=\theta (B)W_t$ driven by white noise $W_t\sim \text{WN}(0,{\sigma }_w^2)$, two parallel structural conditions on the AR and MA polynomials govern when the model defines a usable time series. The first is causality: the process is causal if $X_t$ can be written as a convergent infinite sum of past shocks,
$X_t={\sum }_{j=0}^{\infty }{\psi }_j{W}_{t-j},{\sum }_j|{\psi }_j|<\infty ,$
so that today’s value is a stable linear function of the noise from time $t$ backward. The condition for causality is that all roots of the polynomial $\varphi (z)=1-{\varphi }_{1}z-\cdots -{\varphi }_p{z}^p$ lie strictly outside the unit circle in the complex plane, and on this condition the weights ${\psi }_j$ are determined by the algebraic identity $\theta (B)=\varphi (B)\psi (B)$. Causality is the property that makes the model usable for forecasting: it expresses today’s value in terms of quantities that are in principle observable (or recoverable from observations), rather than requiring access to the future.

The second is invertibility: the process is invertible if the white noise $W_t$ can be recovered as a convergent infinite sum of past observations of the process,
$W_t=X_t-{\sum }_{j=1}^{\infty }{\pi }_j{X}_{t-j},{\sum }_j|{\pi }_j|<\infty ,$
so that the unobservable shock can be reconstructed from the observed history. The condition for invertibility is that all roots of the polynomial $\theta (z)=1+{\theta }_{1}z+\cdots +{\theta }_q{z}^q$ lie strictly outside the unit circle, with the weights ${\pi }_j$ determined by the dual identity $\varphi (B)=\theta (B)\pi (B)$. Invertibility is what makes the model’s parameters identifiable from the observed series alone, since two MA processes whose root locations are reciprocals of each other produce identical second-order distributions and cannot otherwise be distinguished.

Standard fitting routines impose both conditions automatically by parameterizing the optimization in terms of partial autocorrelations rather than raw polynomial coefficients. After fitting, the polynomial roots can be extracted numerically (via np.roots, R’s polyroot, or built-in summary methods) and any root close to the unit circle is a warning sign: either the model is fragile to small data perturbations, or a missed differencing step has produced a near-unit-root pattern that ARIMA differencing should have removed.

Causal vs. Stationary $\infty$) representation in past noise, not just second-order time-invariance. A causal process is automatically stationary; a stationary process need not be causal. There is also a parallel solution involving future shocks $W_{t+1},W_{t+2},\dots$, which is mathematically valid but useless for forecasting. The convention everywhere is to use the causal version.

For finite-order AR and ARMA processes, “causal” and “stationary” coincide as soon as the AR roots lie outside the unit circle. But “causal” is the stronger statement: it asserts a particular MA(

Intuition

The unit-circle root conditions are doing two jobs at once. First, they make the formal power-series inverses $\varphi (B{)}^{-1}$ and $\theta (B{)}^{-1}$ converge, so the recursive equations $\varphi (B)X_t=\theta (B)W_t$ admit honest infinite-sum solutions. Second, among the multiple solutions that could satisfy the equation (one expressed in past noise, one in future noise, and a continuum of mixed alternatives in the non-causal case), the unit-circle conditions pick out the unique solution that is both causal (usable for forecasting from observed history) and uniquely identifiable from the data.

For a pure MA(q) specifically, invertibility is a parameter-identifiability condition rather than a structural-existence one: any MA(q) is automatically stationary, but two MA(q) processes whose root sets are related by reciprocation produce the same second-order distribution, and under Gaussianity the two are observationally indistinguishable. The convention of choosing the version with all MA roots outside the unit circle is what gives standard software a unique answer to report. Without invertibility, every MA fit would have $2^q$ equally good candidates, none of them preferable on data alone.

Properties

(Power-Series Inversion) Roots outside the unit circle make polynomial inverses converge.

If $\varphi (z)={\prod }_j(1-z/z_j)$ has all roots $|z_j|>1$, the inverse can be expanded as a convergent geometric-series product, $\varphi (B{)}^{-1}={\sum }_{k\ge 0}{c}_k{B}^k$, and the inversion produces a stable operator that can be applied to any bounded time series. Roots inside the unit circle would force the series for the inverse to involve negative powers of $B$ (forward shifts), giving a non-causal solution depending on future noise; roots exactly on the unit circle would make the series diverge entirely, killing both stationarity and any chance of inversion. The unit-circle conditions are therefore the precise dividing line between operators that admit clean, convergent inverses and those that do not.

(MA($\infty$) Weights via Recursion) Causal ARMA forecasting reduces to a coefficient recursion.

For a causal ARMA(p, q), the weights ${\psi }_j$ in the MA($\infty$) representation satisfy the recursion
${\psi }_0=1,{\psi }_j={\theta }_j+{\sum }_{k=1}^p{\varphi }_k{\psi }_{j-k},j\ge 1,$
with the convention ${\theta }_j=0$ for $j>q$ and ${\psi }_j=0$ for $j<0$. This recursion is what software computes when asked for the impulse response of an ARMA model: the trajectory of $\{{\psi }_j\}$ shows how a single unit shock at time $0$ propagates forward through the dynamics. It also gives the optimal $h$-step-ahead forecast variance directly as ${\sigma }_w^2{\sum }_{j=0}^{h-1}{\psi }_j^2$, which grows with the horizon at a rate set by how slowly the ${\psi }_j$ decay.

(AR($\infty$) Weights via Recursion) Invertibility makes today’s shock reconstructible from past observations.

The dual recursion gives the weights ${\pi }_j$ in the AR($\infty$) representation:
${\pi }_0=1,{\pi }_j=-{\theta }_j+{\sum }_{k=1}^q{\theta }_k{\pi }_{j-k}-{\varphi }_j,j\ge 1,$
with ${\varphi }_j=0$ for $j>p$. The $\pi$-weights express today’s shock as a function of past observations, $W_t=X_t-{\sum }_{j\ge 1}{\pi }_j{X}_{t-j}$, which makes one-step-ahead forecasting trivially ${\hat{X}}_{t+1}={\sum }_{j\ge 1}{\pi }_j{X}_{t+1-j}$ in terms of observable history. Combined with the $\psi$-weights, this gives the entire infinite-history apparatus needed for ARMA forecasting and residual diagnostics.

(Identifiability under Invertibility) Each MA equivalence class contains a unique invertible representative.

For any non-invertible MA(q), there exists a unique invertible MA(q) with the same autocovariance and a rescaled noise variance. Concretely, replacing each MA root $z$ inside the unit circle by its reciprocal $1/z$ outside the unit circle, and multiplying ${\sigma }_w^2$ by ${\prod }_{|z_j|<1}|z_j{|}^2$, produces the invertible twin. Standard fitting routines automatically restrict the optimization to the invertible region (equivalently, to the unique representative) so that the maximum-likelihood estimate is well-defined and not arbitrary.

(Causal-Invertible Region for ARMA(1,1)) The square $|\varphi |<1$ and $|\theta |<1$ delineates the well-posed parameter space.

For an ARMA(1, 1), the AR polynomial $1-\varphi z$ has its single root at $1/\varphi$ and the MA polynomial $1+\theta z$ has its single root at $-1/\theta$, so the roots-outside-the-unit-circle requirement reduces to the pair of inequalities $|\varphi |<1$ and $|\theta |<1$. Inside this open square (with the additional caveat $\varphi \ne -\theta$ to avoid a common-factor cancellation that would make the model degenerate to white noise), the ARMA(1, 1) is well-posed, and its parameter estimates from arima are uniquely identifiable from data.

(Why Software Stays in the Causal-Invertible Region) The likelihood is multi-modal across reciprocal-root equivalence classes.

All major implementations (R arima, Python statsmodels.tsa.ARIMA, MATLAB arima) restrict their optimizers to the causal-invertible region by reparameterizing the polynomial coefficients in terms of partial autocorrelations, which automatically respect the unit-circle constraints. This avoids the problem that the raw-coefficient likelihood has many equally good local maxima (one in each equivalence class) and is multi-modal across those classes. Restricting to the unique causal-invertible representative gives the optimizer a single global maximum to find.

Examples

AR(1) causality check

Is $X_t=1.2X_{t-1}+W_t$ causal?

Solution $\varphi (z)=1-1.2z$, with the single root $z=1/1.2\approx 0.833$, which is inside the unit circle. So the process is not causal, and in fact also not stationary. Trying to invert $1-1.2B$ as a power series gives ${\sum }_{k\ge 0}1.2^k{B}^k$, in which the coefficients $|1.2^k|\to \infty$, so the series diverges. The shock $W_t$ never decays in its influence, and $\text{Var}(X_t)$ grows without bound, making forecasts useless. There is a parallel non-causal stationary solution involving $W_{t+1},W_{t+2},\dots$ for the same $\varphi =1.2$, but that solution requires future noise and cannot be implemented from observations.

The AR polynomial is

MA(1) invertibility flip

Show that $X_t=W_t+4W_{t-1}$ with $W_t\sim N(0,1)$ has the same autocovariance as the invertible $Y_t=V_t+0.25V_{t-1}$ with $V_t\sim N(0,16)$.

Solution $X$: the variance is ${\gamma }_X(0)=(1+16)\cdot 1=17$, the lag-one covariance is ${\gamma }_X(1)=4\cdot 1=4$, and all higher lags are zero. For $Y$: the variance is ${\gamma }_Y(0)=(1+0.0625)\cdot 16=17$, the lag-one covariance is ${\gamma }_Y(1)=0.25\cdot 16=4$, again with zeros past lag one. So the autocovariances match. The MA root for $X$ is $z=-1/4=-0.25$, inside the unit circle and therefore non-invertible, while the MA root for $Y$ is $z=-4$, outside and therefore invertible. The flip rule is to replace the inside-the-unit-circle root by its reciprocal outside the unit circle, then rescale the noise variance by the squared modulus $|z{|}^2=16$. Software fits will report the $Y$ parametrization because it is the unique invertible representative.

For

ARMA(2,1) causality and invertibility

Verify whether $X_t=0.4X_{t-1}+0.45X_{t-2}+W_t+0.5W_{t-1}$ is causal and invertible.

Solution $\varphi (z)=1-0.4z-0.45z^2=0$ rearranges as $0.45z^2+0.4z-1=0$, with roots $z=(-0.4\pm \sqrt{0.16+1.8})/(0.9)=(-0.4\pm 1.4)/0.9$, giving $z_1=1.111$ and $z_2=-2$. Both moduli exceed $1$, so the process is causal. The MA polynomial $\theta (z)=1+0.5z=0$ has the single root $z=-2$, modulus $2>1$, so the process is invertible. Both conditions hold, and the model is well-posed. The caveat is that $z_1=1.111$ is uncomfortably close to the unit circle: the variance and forecast standard errors will be large, and a small change in the data could push the root inside the unit circle, breaking stationarity.

The AR polynomial