Time-series analysis studies sequences of observations indexed by time, where the temporal ordering carries information that cross-sectional methods cannot exploit. The defining problems are estimating dependence structure from a single observed realization, decomposing a series into trend / seasonal / stochastic components, fitting parametric models that capture the dependence parsimoniously, and forecasting future values with calibrated uncertainty. The tools below are the standard classical toolkit, supplemented by neural-network methods for non-linear and high-dimensional cases.
Prerequisites
- Stochastic processes: the underlying probability framework
- Random walks: canonical non-stationary baseline
- Stationarity: weak vs. strict, the central regularity assumption
Contents
- 2.1. Time Series: the framework: time series as stochastic process, trend, seasonality, stationarity, white noise, ergodicity.
- 2.2 Measures of Dependence: mean, autocovariance, autocorrelation, cross-covariance, cross-correlation; the lag-domain language for serial dependence.
- 2.3 Spectral Analysis: DFT, periodogram, spectral density, linear filters, time-frequency analysis; the frequency-domain alternative to the lag domain.
- 2.4 Time Series Regression: time-series-specific regression (sinusoidal model, Nyquist limit, distributed-lag regression, rolling-origin CV); the general regression machinery (OLS, ridge/lasso, ordinary CV) lives in ML regression models.
- 2.5 ARIMA Models: backshift operator, AR, MA, ARMA, ARIMA, causality and invertibility, partial autocorrelation; classical Box-Jenkins methodology.
- 2.6 State Space Models: VAR, dynamic linear models; the framework for hidden-state inference. Filtering and smoothing algorithms (Kalman) live in 3.12 Kalman Filtering under Bayesian Statistics.
- 2.7 Neural Sequence Models for Time Series: bridge to CNNs, RNNs, and Transformers in the deep-learning section, with notes on when neural methods beat classical ones.
Recommended path
Start with 2.1. Time Series to anchor the framework, then 2.2 Measures of Dependence for the lag-domain language. From there the two main streams diverge: 2.3 Spectral Analysis develops the frequency-domain machinery for periodic and signal-processing applications, while 2.4 Time Series Regression develops the regression-based machinery for trend, lag, and exogenous-driver modeling. The two converge in 2.5 ARIMA Models, which is the central parametric framework for stationary serial dependence and the canonical first-fit for any new time-series problem. 2.6 State Space Models generalizes ARIMA to handle hidden states, missing data, and time-varying parameters; the inference algorithms it uses are general Bayesian machinery and live in 3.12 Kalman Filtering. 2.7 Neural Sequence Models for Time Series is the off-ramp to neural methods when the linear toolkit underperforms.