Time-Lagged Regression

Time-Lagged Regression

Time-lagged regression (also called distributed-lag regression, or finite impulse-response regression) explains an output time series $y_t$ as a weighted sum of present, past, and possibly future values of an input series $x_t$:
$y_t={\sum }_{r=-m_{-}}^{m_{+}}{\beta }_r{x}_{t-r}+v_t,$
where $\{{\beta }_r\}$ are lag coefficients, $m_{-},m_{+}\ge 0$ are the maximum future and past lags considered, and $v_t$ is a noise term. With $m_{-}=0$ (no future lags) the model is causal; with both $m_{\pm }>0$ it is non-causal and useful only for offline analysis. The infinite version $y_t={\sum }_r{\beta }_r{x}_{t-r}+v_t$ is the population analogue, but in practice only finitely many lags are estimated.

Estimation. With $L=m_{-}+m_{+}+1$ lag coefficients, build the design matrix $\mathbf{X}$ whose row at time $t$ is $(x_{t+m_{-}},x_{t+m_{-}-1},\dots ,x_{t-m_{+}})$, and run OLS:
$\hat{\mathbf{\beta }}=({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }\mathbf{y}.$

When the lag count is large (often the case in neuroscience, economics, audio), ${\mathbf{X}}^{\top }\mathbf{X}$ becomes ill-conditioned and OLS overfits. Standard remedy: ridge regression or lasso, with the regularization strength chosen by cross-validation.

Connection to filtering. The fitted ${\hat{\beta }}_r$ are the weights of a finite linear filter applied to $x_t$ (see 2.3.4 Linear Filters and Frequency Response); the frequency response $\hat{A}(\omega )={\sum }_r{\hat{\beta }}_r{e}^{-2\pi ir\omega }$ describes which frequencies of $x$ drive $y$.

Correlated Predictor Lags $\mathbf{X}$ in a distributed-lag model are far from orthogonal; adjacent columns can have correlation above $0.9$. OLS standard errors will be large and individual ${\hat{\beta }}_r$ may flip sign run-to-run, even when the sum ${\sum }_r{\hat{\beta }}_r$ is stable. Inference on individual lag coefficients is unreliable; inference on aggregate impulse-response shape is what to trust.

Lagged versions of the same series are heavily correlated. The columns of

Intuition

Time-lagged regression is the simplest way to model ”$x$ drives $y$ with some delay” without committing to a specific dynamical mechanism. Instead of writing a parametric model like an AR or ARMA, the analyst lets the data choose the impulse response: how strongly an impulse in $x$ at time $t=0$ shows up in $y$ at times $t=0,1,2,\dots$. The weights ${\beta }_r$ are exactly that impulse response, and their full sequence describes both the shape and the delay of the input-to-output coupling.

The classic example in time-series textbooks is the SOI–Recruitment fish-population data: a climate index ($x$) drives a fish-population variable ($y$) with a delay of several months, and the distributed-lag fit shows precisely how the response builds and dissipates. The neuroscience example covered in lecture is spectro-temporal receptive field (STRF) estimation: $x$ is a sound spectrogram and $y$ is the firing rate of an auditory neuron, with the fitted $\beta$ surface describing which sound features at which past times drive the neuron’s response.

Properties

(Equivalence to Filtering) The fitted model can be rewritten ${\hat{y}}_t=(\hat{\beta }\ast x{)}_t$, a discrete convolution. The frequency response is the DTFT of $\hat{\beta }$.

This is why time-lagged regression is sometimes called finite-impulse-response (FIR) filter identification: from the engineer’s point of view, we are estimating the filter that best maps $x$ to $y$. The frequency response $\hat{A}(\omega )$ is informative diagnostically: a low-pass filter (high gain at low $\omega$, low gain at high $\omega$) means the coupling is dominated by slow components; a band-pass filter localizes the coupling to a specific frequency range.

(OLS Optimality on Whitened Inputs) If $x_t$ is white noise and $v_t$ is white noise independent of $x$, then ${\hat{\beta }}_r={\hat{\gamma }}_{xy}(r)/{\hat{\gamma }}_{xx}(0)$, so the OLS estimate equals the rescaled sample cross-covariance.

For non-white inputs the OLS estimate is the cross-covariance after pre-whitening both series with the same filter. This is the basis of the standard distributed-lag identification procedure: fit an ARMA to $x$ to get its innovations, apply the same filter to $y$, then read off lag coefficients from the cross-correlation of the whitened series.

(Lag Count vs. Sample Size) For OLS to be well-conditioned, the lag count $L$ should satisfy $L\ll n$, typically $L\le n/10$ as a rule of thumb.

When $L$ is comparable to $n$, the design matrix has fewer rows than effective columns, the estimator overfits dramatically, and out-of-sample prediction error explodes. Ridge regression with a CV-tuned penalty extends usable $L$ to several hundred even with $n$ in the thousands, and is the standard remedy in modern STRF and finance applications.

(Causal vs. Non-Causal Identification) Including future lags ($m_{-}>0$) helps detect whether $y$ leads $x$ (significant negative-lag coefficients) or whether the assumed direction of causality is wrong.

A clean causal coupling shows non-zero ${\hat{\beta }}_r$ at $r\ge 0$ and noise at $r<0$; a wrong-direction model shows the mirror pattern. This is a powerful sanity check that pure ARMA modeling lacks. (Causality here is in the engineering sense, the temporal direction of the linear coupling, not Granger causality, which is a related but distinct test.)

(Residual Diagnostics) The residuals ${\hat{v}}_t=y_t-{\hat{y}}_t$ should be white. If they are not, the lag count is too short, or there is missing endogenous structure that needs separate ARMA modeling.

A two-stage workflow: (i) fit the distributed-lag model, (ii) fit an ARMA to the residuals to capture any leftover serial correlation. Combined this gives a transfer function (or ARMAX) model: $y_t={\sum }_r{\beta }_r{x}_{t-r}+\text{ARMA noise}$.

(Standard-Error Inflation under Serial Correlation) OLS standard errors are wrong when $v_t$ is autocorrelated; HAC standard errors (Newey-West) or GLS with an estimated noise covariance are needed for valid inference.

The point estimates ${\hat{\beta }}_r$ remain unbiased, but their advertised significance can be wildly wrong if computed naively. In econometric practice, HAC corrections are standard for distributed-lag models; in neuroscience, the inference is often visual (compare to the noise distribution from circularly-shifted permutation tests).

Examples

Climate-driven fish recruitment

Let $x_t$ be the monthly Southern Oscillation Index and $y_t$ be monthly fish recruitment. A distributed-lag fit with $L=24$ monthly lags shows ${\hat{\beta }}_r$ peaking at $r\approx 5$ months and decaying smoothly out to $r\approx 12$.

Interpret. $r=12$ says some climate effects accumulate over the better part of a year. The shape is a smooth, unimodal impulse response, typical of a low-pass coupling. The fitted frequency response $\hat{A}(\omega )$ would show a peak near $\omega =0$, confirming that low-frequency (slow) variations in $x$ drive low-frequency (slow) variations in $y$.

The peak at lag 5 says climate conditions drive fish populations with about a five-month delay, a plausible biological lead time for ENSO-related sea-surface temperature changes to propagate through the food chain to recruitment. The decay tail to

Spectro-temporal receptive field (STRF)

An auditory neuron’s firing rate $y_t$ is regressed on a sound spectrogram $x_{t,f}$ across $F=32$ frequency channels and $L=50$ time lags ($\sim 100$ ms at $f_s=500$ Hz).

How is the model parameterized and fit? $F\times L=1600$ regressors per output sample. With a typical recording of $n=10^5$ samples, OLS is feasible but overfits without regularization. Ridge regression with regularization $\lambda$ chosen by 5-fold CV is standard. The fitted coefficient surface ${\hat{\beta }}_{r,f}$ is the spectro-temporal receptive field: a 2-D plot of frequency vs. lag that shows the neuron's preferred sound feature. Auditory cortical neurons typically show a band-limited preferred frequency (rows of $\hat{\beta }$ peak at one or two channels) and a $\sim 30$ ms response delay (columns of $\hat{\beta }$ peak at $r\approx 15$).

The full design has

Diagnosing direction with non-causal lags

You suspect $x$ leads $y$ but want to be sure. Fit a distributed-lag model with $m_{-}=10$, $m_{+}=10$, a 21-lag fit including 10 future and 10 past lags.

What pattern confirms the causal direction? ${\hat{\beta }}_r$ for $r\ge 0$ (past values of $x$ predict $y$) with ${\hat{\beta }}_r\approx 0$ for $r<0$ (future values of $x$ do not) is consistent with $x\to y$ causality. The opposite pattern (significant negative-lag coefficients) would mean $y\to x$: the variables are coupled in the reverse direction, and the model setup was wrong. A symmetric pattern (both significant) suggests bidirectional coupling or a common driver, neither of which the simple distributed-lag model can disentangle.

Significant