Stationarity

Stationarity

A stochastic process $\{X_t\}$ is stationary when its probabilistic structure does not depend on the absolute placement of time. Two sharpenings of this idea are used in practice, differing in how much of the structure is required to be time-invariant.

Strict stationarity. The process is strictly stationary when the joint distribution of any finite collection of values is invariant under arbitrary time shifts: for every $k\ge 1$, every $h\in \mathbb{Z}$, and every choice of times $t_1,\dots ,t_k$,
$(X_{t_1},\dots ,X_{t_k})\stackrel{d}{=}(X_{t_1+h},\dots ,X_{t_k+h}).$
This is the strong condition: not just the mean, not just the covariance, but the entire joint distribution must look the same after shifting. Strict stationarity implies that every individual $X_t$ has the same distribution, every pair $(X_s,X_t)$ has the same joint distribution as any other pair separated by the same lag, and so on for all finite-dimensional marginals.

Weak stationarity. The process is weakly stationary (also second-order stationary or wide-sense stationary) when only the first two moments are time-invariant:
$\mathbb{E}[X_t]=\mu \text{ for all }t,\mathrm{Cov}(X_s,X_t)=\gamma (|s-t|)\text{ for all }s,t.$
The mean is a constant rather than a function of $t$, and the autocovariance depends only on the lag $h=|s-t|$ rather than on the absolute times. Higher moments and the joint distribution itself are unconstrained. This is the working definition used throughout time-series analysis, because it is the minimal assumption needed for the autocorrelation function $\rho (h)=\gamma (h)/\gamma (0)$ to be well-defined and the sample autocovariance to estimate it consistently.

Strict implies weak (when second moments exist), but weak does not imply strict in general. The exception is the Gaussian case: a Gaussian process is fully determined by its mean and covariance, so weak stationarity forces the entire joint distribution to be shift-invariant, and the two notions coincide.

"Stationary" Without Qualifier weakly stationary. That is the convention in Shumway-Stoffer, in statsmodels, and in nearly every applied time-series source. Strict stationarity is invoked when the argument needs joint distributions (ergodic theorems, mixing conditions, non-Gaussian limit theorems); weak stationarity is invoked everywhere else.

In practice, “stationary” almost always means

Intuition

Stationarity is the assumption that lets a single observed path of a stochastic process tell us about the underlying distribution. Without some form of time invariance, every time point would have its own mean, variance, and dependence structure, and a single realization would give exactly one data point per time index, not enough to estimate anything. Stationarity says “the rules of the process are the same at every $t$,” so observations at many different times can be pooled to estimate properties of those rules. This is the time-series analogue of the i.i.d. assumption in classical statistics: not that observations are independent, but that they come from the same data-generating mechanism applied at every step.

The two-tier weak/strict split exists because most of what time-series analysis actually does only needs the first two moments. Estimating the autocorrelation, fitting an ARMA, computing a periodogram, running the Kalman filter: all of these rely on the mean and covariance, not on higher moments. So the weaker assumption suffices, and weakening it is a real win because more processes satisfy weak stationarity than strict stationarity. The strict version becomes essential only when the analysis hinges on the joint distribution itself, for example when proving central limit theorems for non-linear functionals of the series, or when reasoning about ergodicity beyond the mean.

Properties

(Strict Implies Weak) If $\{X_t\}$ is strictly stationary and has finite second moments, it is weakly stationary.

If the joint distribution of $(X_s,X_t)$ depends only on $|s-t|$, then so does any function of it, including $\mathrm{Cov}(X_s,X_t)$. The mean $\mathbb{E}[X_t]$ is the same for every $t$ because all marginal distributions agree. The converse fails because matching just the first two moments leaves higher moments free to vary with time. The implication does require finite second moments: a strictly stationary process with infinite variance (e.g., one driven by Cauchy innovations) is strictly but not weakly stationary, because the autocovariance is not even defined.

(Gaussian Equivalence) A Gaussian process is strictly stationary if and only if it is weakly stationary.

A Gaussian process is determined entirely by its mean function and covariance function, so once those are time-invariant, the entire joint distribution is time-invariant too. This is the only large family of processes for which the two notions coincide universally. It is why much of Gaussian-process theory and Kalman-filter theory does not need to distinguish between the two: for Gaussian models, “stationary” is unambiguous.

(Mean is Constant) Under weak stationarity, $\mathbb{E}[X_t]=\mu$ for every $t$.

This is half of the definition. The practical consequence is that the sample mean $\bar{X}=\frac{1}{T}{\sum }_{t=1}^T{X}_t$ is an unbiased estimator of $\mu$, regardless of the dependence structure of the series. The variance of $\bar{X}$ does depend on the dependence (through the autocovariances), but the mean is unbiased even under serial correlation.

(Autocovariance Depends Only on Lag) Under weak stationarity, $\gamma (s,t)=\gamma (|s-t|)$.

The autocovariance is a function of one variable (the lag) rather than two (the absolute times). This is what makes the function plottable and estimable: a single sample autocovariance $\hat{\gamma }(h)$ is computed from all $T-|h|$ pairs of observations separated by lag $h$, pooling information across the time axis. Without lag-only dependence, each $(s,t)$ would be its own quantity to estimate, and there would be no way to estimate it from one realization.

(Sample Estimators Are Consistent Under Ergodicity) When the process is stationary and ergodic, sample moments converge to population moments.

Ergodicity is the additional condition that allows time averages to substitute for ensemble averages. Under stationarity alone, $\bar{X}$ is unbiased but might not converge: the realization could persist forever in some atypical region. Under ergodicity, $\bar{X}\stackrel{p}{\to }\mu$ as $T\to \infty$, and similarly $\hat{\gamma }(h)\stackrel{p}{\to }\gamma (h)$. A standard sufficient condition for ergodicity in the mean is that the autocovariance decays to zero, $\gamma (h)\to 0$ as $h\to \infty$, which is satisfied by all causal ARMA processes.

(Differencing as a Stationarity Tool) Many non-stationary processes become stationary after differencing.

The random walk is the canonical example: $\{x_t\}$ is non-stationary (variance grows with $t$), but the first difference $\nabla x_t=x_t-x_{t-1}$ is white noise, which is strictly stationary. More generally, if $X_t={\mu }_t+Y_t$ with ${\mu }_t$ a polynomial of degree $k$ in $t$ and $Y_t$ stationary, then ${\nabla }^k{X}_t$ is stationary. This is the algebraic basis for the integration order $d$ in ARIMA(p, d, q) models: non-stationary series are differenced until a stationary residual remains, then ARMA structure is fit to that residual.

(Increment Stationarity is Weaker Than Stationarity Itself) A process can have stationary increments without being stationary.

Brownian motion is the canonical example: $B_t-B_s\sim \mathcal{N}(0,t-s)$ depends only on $t-s$, so the increments are strictly stationary, yet $\mathrm{Var}(B_t)=t$ is not constant in $t$, so $B_t$ itself is not stationary. The same dichotomy holds for the random walk. A useful operational test: if differencing the process makes it stationary, the original was integrated of order one, with stationary increments but non-stationary levels.

Examples

White noise is strictly stationary

Let $\{W_t\}$ be i.i.d. with mean $0$ and variance ${\sigma }^2$. Verify both weak and strict stationarity.

The mean is $\mathbb{E}[W_t]=0$ for every $t$, so it is constant. The autocovariance is $\gamma (s,t)={\sigma }^2$ if $s=t$ and $0$ otherwise, which depends on $|s-t|$ alone. So $\{W_t\}$ is weakly stationary. For strict stationarity, the joint distribution of any $(W_{t_1},\dots ,W_{t_k})$ is a product of marginals (by independence), and each marginal is the same regardless of $t$, so the joint distribution is shift-invariant. White noise is therefore both strictly and weakly stationary; the strongest form of stationarity is automatic when the variables are i.i.d.

Random walk is not stationary

Let $x_t={\sum }_{j=1}^t{w}_j$ with $w_j$ i.i.d. mean zero. Show that $\{x_t\}$ is not weakly stationary.

The mean function $\mathbb{E}[x_t]=0$ is constant (so the first condition holds), but the autocovariance is $\gamma (s,t)={\sigma }^2\min (s,t)$, which depends on $\min (s,t)$ rather than only on $|s-t|$. Specifically $\gamma (1,2)={\sigma }^2$ but $\gamma (101,102)=101{\sigma }^2$, even though both pairs have lag $1$. The second condition of weak stationarity fails, so the random walk is not stationary. The differenced process $\nabla x_t=w_t$ is white noise, however, so the random walk is integrated of order one: non-stationary in levels, stationary in increments.

Linear trend is not stationary

Let $X_t={\beta }_0+{\beta }_{1}t+Y_t$ with ${\beta }_1\ne 0$ and $Y_t$ stationary. Show that $\{X_t\}$ is not stationary, but that one differencing fixes it.

The mean is $\mathbb{E}[X_t]={\beta }_0+{\beta }_{1}t+\mathbb{E}[Y_t]$, which is linear in $t$ rather than constant. So the first condition of weak stationarity fails immediately. Differencing once: $\nabla X_t={\beta }_1+\nabla Y_t$. The constant ${\beta }_1$ has constant mean ${\beta }_1$, and $\nabla Y_t$ is stationary because differencing is a fixed linear filter on a stationary input. So $\nabla X_t$ has constant mean ${\beta }_1$ and lag-only autocovariance, which is the definition of weakly stationary. This is the deterministic-trend version of the more general result that polynomial trends of degree $k$ are removed by $k$ differences.

Weakly stationary but not strictly stationary

Construct a process that satisfies the weak conditions but not the strict ones.

Let $X_t\sim N(0,1)$ for even $t$ and $X_t\sim \text{Laplace}(0,1/\sqrt{2})$ for odd $t$, all independent. Both distributions have mean $0$ and variance $1$, so $\mathbb{E}[X_t]=0$ and $\mathrm{Cov}(X_s,X_t)=1$ if $s=t$ and $0$ otherwise, which is weakly stationary. But the marginal distribution of $X_1$ is Laplace while the marginal of $X_2$ is Gaussian, so the joint distribution of $(X_1)$ differs from that of $(X_2)$, and the process is not strictly stationary. This kind of example shows that weak stationarity is a genuine relaxation: it really does allow the higher-moment structure to vary in time.

See Also

• 4.4 Random Walks: canonical non-stationary example; differencing recovers stationarity.
• 4.6 Brownian Motion: non-stationary process with strictly stationary increments.
• 2.1. Time Series: applied treatment of stationarity inside time-series analysis.
• 2.2.2 Autocovariance Function: the lag-only function whose existence requires weak stationarity.
• 2.5.6 ARIMA(p,d,q) Process: differencing integer order $d$ to bring a non-stationary series to stationarity.
• 3.10 Gaussian Processes: stationarity for GPs corresponds to using a stationary kernel like RBF or Matérn.