State Space Models

State-space models split the data-generating process for a time series into two pieces: a hidden state $x_t$ that evolves over time according to its own dynamics, and noisy observations $y_t$ that depend on the state through a measurement equation. This separation matters whenever the analyst needs to handle missing data, separate process noise from measurement noise, model time-varying parameters, or jointly model multiple coupled series. The classical ARMA family treats the observed series as one stochastic process; the state-space framework treats it as a noisy view of a deeper one.

The framework subsumes nearly every linear time-series model: ARMA, ARIMA, VAR, regression with time-varying coefficients, local-level and local-linear-trend models, and unobserved-components models all admit state-space representations. The unification has practical benefits beyond conceptual cleanliness: once a model is in state-space form, the same recursive inference algorithm (the Kalman filter) computes its likelihood, its forecasts, and its missing-data interpolations.

Prerequisites

• 2.1. Time Series: the observation side of the model
• 2.2.2 Autocovariance Function: for residual diagnostics on the innovations
• 4.7 Stationarity: state-space models do not require stationary observations, but the state often is
• 6.2 Multivariate Gaussian Distribution: needed to read the Gaussian state-space derivations

Contents

• 2.6.1 Vector Autoregression: the multivariate AR model. A $k$-dimensional series where each component is a linear function of the past of all components. The simplest state-space model in which the state and the observation coincide.
• 2.6.2 Dynamic Linear Model: the general linear Gaussian state-space framework. A hidden Markov state with linear transition and linear noisy observation; encompasses ARMA, ARIMA, VAR, time-varying regression, and local-trend models as special cases.

Where the inference happens

The Bayesian inference algorithms for state-space models, filtering (online posterior of the state given data so far) and smoothing (offline posterior using the entire record), live in the Bayesian Statistics treatment because the algorithms are general Bayesian inference machinery rather than time-series specific:

• 3.12.1 Kalman Filter: the forward recursion that computes $p(x_t\mid y_{1:t})$ in $O(p^3)$ per step.
• 3.12.2 Kalman Smoother: the backward pass that refines each state estimate using future observations, computing $p(x_t\mid y_{1:T})$.

This split is deliberate. A Dynamic Linear Model is a time-series construct (a way of writing down what the data-generating process looks like), so it lives here. The Kalman filter is a Bayesian inference algorithm (a closed-form sequential Gaussian update applicable to many problems beyond time series, including robotics state estimation, GP regression in time, and recursive least squares), so it lives in Bayesian Statistics. The two folders link to each other; the math is written down only once.

Recommended path

Read 2.6.1 Vector Autoregression first to see the multivariate AR generalization, then 2.6.2 Dynamic Linear Model for the full state-space framework. With the model side in hand, follow the link out to 3.12 Kalman Filtering for the inference algorithms. A reader interested only in the algorithms can skip the time-series notes and start at 3.12 Kalman Filtering directly; a reader interested only in the time-series modeling can stop after the DLM note.