Kalman Smoother

Kalman Smoother

The Kalman smoother computes the posterior $p(x_t\mid y_{1:T})=\mathcal{N}(x_t^T,P_t^T)$ for every $t=1,\dots ,T$: the smoother distribution of each state given the entire observation record, including observations after time $t$. This is the right Bayesian quantity when all the data is in hand and the goal is the best possible estimate at every time point retrospectively, as opposed to the filter distribution $p(x_t\mid y_{1:t})$ which uses only data up through $t$.

The standard implementation is the Rauch-Tung-Striebel (RTS) algorithm, a backward pass that runs after the forward Kalman filter and refines the filter’s estimates using future observations. The recursion starts at the last time step (where smoother and filter agree, $x_T^T$ from the filter) and walks backward, updating each $x_t^T$ using both the filter result $x_t^t$ and the next step’s smoother result $x_{t+1}^T$:
$x_t^T=x_t^t+J_t(x_{t+1}^T-x_{t+1}^t),P_t^T=P_t^t+J_t(P_{t+1}^T-P_{t+1}^t)J_t^{\top },$
where the smoother gain is
$J_t=P_t^t{\Phi }^{\top }(P_{t+1}^t{)}^{-1}.$
The quantity $x_{t+1}^T-x_{t+1}^t$ is the smoother correction (the change in the next state’s estimate once the future was incorporated), and $J_t$ propagates that correction back to time $t$ through the dynamics.

Because the backward pass needs the filter’s stored $(x_t^t,P_t^t,x_t^{t-1},P_t^{t-1})$ at every $t$, the smoother runs in $O(T{p}^3)$ time and $O(T{p}^2)$ memory: the time matches the filter, but the memory grows linearly in $T$ because the entire filter trajectory must be saved. The output is a sequence of Gaussian posteriors $\{(x_t^T,P_t^T){\}}_{t=1}^T$ with $P_t^T⪯P_t^t$ at every $t$, since adding future data can only reduce uncertainty.

Smoother is Strictly Offline $x_{t+1}^T$ before it can compute $x_t^T$, which means the smoother cannot produce its output for time $t$ until after time $T$. Use the filter for any application that needs estimates as the data arrives. The smoother is appropriate only when the entire record is available before the analysis begins.

The RTS recursion needs

Intuition

The filter and the smoother answer different questions about the same model. The filter asks “what is my best guess for $x_t$ given everything I have seen so far?”, a question whose answer can change as new data arrives. The smoother asks “what is my best guess for $x_t$ given the entire record?”, a question whose answer is fixed once the record is complete. The smoother is more accurate at every $t<T$ because it uses strictly more information; the filter is more useful at every $t=T$ because the smoother’s answer is not yet computable in real time.

The backward-pass recipe makes the information flow explicit. The forward filter passes information from past to present along the dynamics: by the end of the forward pass, every observation through time $t$ has been absorbed into the filter estimate $x_t^t$. The backward smoother then passes information from future to present along the time-reversed dynamics, so that observations after time $t$ refine the estimate at $t$. The smoother gain $J_t=P_t^t{\Phi }^{\top }(P_{t+1}^t{)}^{-1}$ is the time-reversed analogue of the Kalman gain: it weighs how much the prediction error at $t+1$ should be propagated back to correct the estimate at $t$. When the dynamics are highly informative ($\Phi P_t^t{\Phi }^{\top }$ is much larger than $Q$), the smoother changes the filter estimate substantially; when the dynamics are weak, the smoother barely updates the filter result.

Properties

(Backward Recursion After Forward Filter) The smoother needs the entire filter trajectory in memory.

Each backward step at time $t$ requires the filter outputs $(x_t^t,P_t^t)$ and the prediction $(x_{t+1}^t,P_{t+1}^t)$, plus the next smoother output $(x_{t+1}^T,P_{t+1}^T)$ from the previous backward step. Standard implementations run the filter forward, store all intermediate quantities, then run the backward pass. Memory cost is $O(T{p}^2)$, manageable for typical state-space problems but prohibitive for very long records with high-dimensional states. Streaming variants (fixed-lag smoother) trade some accuracy for $O(L{p}^2)$ memory by only smoothing $L$ steps back from the current time.

(Smoother Variance is Always Smaller) $P_t^T⪯P_t^t$ in the matrix-inequality sense.

Adding the future observations $y_{t+1},\dots ,y_T$ to the conditioning information can only sharpen the posterior. The smoother variance equals the filter variance only at the last time point ($t=T$, where there is no future to add) and gets strictly smaller for $t<T$ when the dynamics are informative enough that future observations carry information about $x_t$. This monotonicity is the formal Bayesian statement of “more data, less uncertainty,” propagated through the dynamics by the smoother gain.

(Smoother Gain as Time-Reversed Filter Gain) $J_t=P_t^t{\Phi }^{\top }(P_{t+1}^t{)}^{-1}$ is the Kalman gain for the reversed dynamics.

The Kalman filter’s gain $K_t$ has the form (prior covariance)(observation Jacobian)/(observation noise covariance). The smoother’s gain $J_t$ has the structurally identical form (filter covariance)(reverse-dynamics Jacobian)/(forward-prediction covariance). The smoother is the Kalman filter run backward through time, where the “observation” at the new step is the smoothed estimate from the future step. This duality is why RTS is so cheap: it is a second Kalman filter, not a fundamentally new algorithm.

(Equality at $t=T$) The smoother and filter produce identical estimates at the final time point.

At $t=T$ there are no observations after the current time, so the smoother distribution $p(x_T\mid y_{1:T})$ coincides with the filter distribution. The backward recursion is initialized at $x_T^T=x_T^T$ (the filter result, which is also the smoother result) and produces non-trivial corrections only for $t<T$. This makes the smoother appear to “fix” past estimates as new data comes in, while leaving the most recent estimate alone, an information-flow pattern that matches the dynamics of the underlying Bayesian update.

(EM Algorithm E-Step) The smoother is the E-step of EM for state-space model parameter estimation.

When the model parameters $\Phi ,Q,R$ are unknown, fitting them by maximum likelihood requires the EM algorithm. The E-step requires the posterior $p(x_t\mid y_{1:T},{\theta }_{\text{current}})$ for every $t$, exactly what the smoother computes, including the cross-time covariances $P_{t,t-1}^T$ that the EM update equations need. The M-step then updates $\Phi ,Q,R$ by solving the Gaussian likelihood-maximization given the smoothed sufficient statistics. EM iteration is the standard way to learn state-space parameters when the structure but not the values of $\Phi ,Q,R$ are known.

(Missing Data Interpolation) The smoother fills gaps by propagating information through the dynamics from both directions.

When some $y_t$ are missing, the filter handles them by skipping the update step at those times, leaving large posterior variance there. The smoother then uses observations both before and after the gap to interpolate, which can dramatically reduce the variance at the missing time points. This is the canonical use case for the smoother in scientific applications: a partial record (e.g., 40% missing biomarker measurements after day 35 in the bone-marrow transplant example) becomes a complete inferred trajectory with honest uncertainty bands.

(Cost is Linear in $T$) Backward pass is $O(T{p}^3)$, same as forward filter.

Each backward step inverts a $p\times p$ matrix and does a constant number of $p\times p$ matrix products, for $O(p^3)$ per step. Total $O(T{p}^3)$, identical to the filter’s cost. So running the full filter-smoother pair takes about twice as long as just the filter, a small price for the much more accurate state estimates that the smoother provides at every $t<T$.

Examples

Smoothing missing observations in a biomarker time series

Three blood markers (white-blood-cell count, platelets, hematocrit) are measured daily on a patient after a bone-marrow transplant, but $\sim 40\%$ of measurements are missing, especially after day 35. How does the smoother differ from the filter on the missing days?

The information flow that the smoother adds

The filter handles missing days by carrying the prior forward (no update step), so the filter variance at day 50 (well into the missing-heavy region) is large because no observation has anchored the state for many steps. The smoother runs backward from day 100 (or wherever the record ends) and finds that observations on day 60, 70, and 90 do constrain what the state could have been at day 50. Information from those future observations propagates backward through the dynamics: the day-90 observation updates the smoother estimate at day 80, which updates day 70, and so on. By the time the backward pass reaches day 50, the variance has shrunk substantially compared to the filter’s. The smoother estimate is therefore the right tool for retrospective interpolation, while the filter estimate is what you would have known on day 50 in real time.

Filter vs. smoother on a noisy 2D trajectory

You record a vehicle’s GPS position every second for 100 seconds. The position is the state, the GPS reading is the observation, and the dynamics are constant velocity with small acceleration noise. Compare the filter’s and smoother’s outputs visually.

What you'd see $P_t^T$) are roughly half the width of the filter's bands at every $t<T$, and the smoothed and filtered estimates agree exactly only at $t=T$.

The filter trajectory looks jittery: every time a noisy GPS reading arrives, the estimate jumps slightly toward it, then the prediction step pushes the estimate forward by the current velocity guess. The early estimates are particularly bad because the velocity has not been pinned down yet. The smoother trajectory looks smooth, because the future observations have effectively “averaged out” the early measurement noise, the velocity is well-estimated everywhere, and the trajectory matches the true path much more closely. The variance bands (

EM for unknown state-space parameters

You believe a series follows a state-space model but do not know the dynamics matrix $\Phi$ or the noise covariances $Q,R$. How does EM use the smoother to estimate them?

The EM-with-smoother loop $(\Phi ,Q,R)$ to reasonable starting values. E-step: run the Kalman filter and smoother with the current parameters to compute $p(x_t\mid y_{1:T})$ at every $t$, plus the cross-time covariances $P_{t,t-1}^T$. M-step: use the smoothed sufficient statistics to maximize the expected complete-data log-likelihood, which has closed-form Gaussian solutions for $\Phi ,Q,R$ in terms of the smoothed quantities. Iterate until convergence. The smoother is essential because the M-step needs the retrospective posterior, not the filter's online posterior; the latter would systematically underestimate state variability and bias the parameter estimates downward.

Initialize

See Also

• 3.12 Kalman Filtering: section index, decision rule for filter vs. smoother
• 3.12.1 Kalman Filter: the forward pass that the smoother runs after
• 3.11.5 EM Algorithm: uses the smoother as its E-step for state-space parameter learning
• 2.6.2 Dynamic Linear Model: the model the smoother operates on, in time-series language
• 6.2 Multivariate Gaussian Distribution: the Gaussian conditioning identity behind both passes