Spectral Density

Spectral Density

For a stationary time series $\{X_t\}$ whose autocovariance is absolutely summable (${\sum }_h|\gamma (h)|<\infty$), the spectral density $f(\omega )$ is the discrete-time Fourier transform of the autocovariance:
$f(\omega )={\sum }_{h=-\infty }^{\infty }\gamma (h)e^{-2\pi ih\omega },\omega \in [-\frac{1}{2},\frac{1}{2}].$
The autocovariance is recovered by the inverse transform $\gamma (h)={\int }_{-1/2}^{1/2}{e}^{2\pi ih\omega }f(\omega )d\omega$. So the autocovariance and the spectral density are the time-domain and frequency-domain faces of the same information about the process; neither contains anything the other lacks.

$f(\omega )$ is the population quantity that the periodogram estimates from data. It says how the variance of the process is distributed across frequencies, playing the role of a probability density for the energy of the process rather than for its values. At lag $h=0$, the inverse transform recovers the total variance as the integral of the spectral density over all frequencies,
$\gamma (0)=\mathrm{Var}(X_t)={\int }_{-1/2}^{1/2}f(\omega )d\omega ,$
so reading off the proportion ${\int }_a^{b}f(\omega )d\omega /\gamma (0)$ tells you the fraction of the process’s variance carried by frequencies in $[a,b]$.

For real-valued processes, $f(\omega )=f(-\omega )$ (because $\gamma (h)$ is real and even), so plots conventionally show only $\omega \in [0,1/2]$. The Nyquist frequency $\omega =1/2$ is the highest oscillation a discretely sampled process can express.

Time-Domain vs. Frequency-Domain Equivalence

The autocovariance and the spectral density are Fourier-pair representations of the same information about a stationary process. The right choice between them is pragmatic: lag-based diagnostics (ACF/PACF for ARMA model identification) are easier in the time domain, while periodic structure (oscillations, hidden frequencies, harmonic peaks) is easier to read in the frequency domain.

Intuition

If autocovariance $\gamma (h)$ tells you how strongly the process at time $t$ co-varies with the process at time $t+h$, the spectral density rearranges that same information by frequency: it tells you how much of the process’s total variance is contributed by sinusoidal fluctuations at each rate $\omega$. A process whose energy concentrates near $\omega =0$ moves slowly and has long memory; one whose energy concentrates near $\omega =1/2$ alternates rapidly. White noise, with no preferred timescale, has flat spectral density: equal power at every frequency, the structural reason behind the analogy with white light.

The non-negativity of $f(\omega )$ is the frequency-domain reflection of the positive semi-definiteness of $\gamma$. Both are different statements of the same constraint: a real autocovariance function whose Fourier transform went negative could be used to construct a linear combination ${\sum }_i{a}_i{X}_{t_i}$ with negative variance, which is impossible. So the constraint that makes a function a valid autocovariance in the time domain is exactly the constraint that makes its Fourier transform a valid spectral density in the frequency domain.

Properties

(Non-Negativity) $f(\omega )\ge 0$ for every $\omega$, by construction from a positive semi-definite $\gamma$.

A negative spectral density would imply a linear combination of the process with negative variance, which contradicts the positive semi-definiteness of $\gamma$. Equivalently, every valid spectral density is the squared magnitude of some transfer function applied to white noise, a direct consequence of the linear-filter identity below.

(Symmetry) $f(\omega )=f(-\omega )$ for real-valued processes.

This follows from $\gamma (h)=\gamma (-h)$. The frequency band $[0,1/2]$ contains all the information; values for $\omega \in [-1/2,0)$ are mirror images of values in $[0,1/2]$.

(Variance Decomposition) The integral of the spectral density over all frequencies equals the process variance.

Setting $h=0$ in the inverse transform gives $\gamma (0)={\int }_{-1/2}^{1/2}f(\omega )d\omega$, which is the spectral analogue of the identity that variance equals the total mass of any density. Reading off proportions of the integral over a frequency band quantifies how much of the process’s variability is carried by oscillations in that band: the basis of spectral-bandwidth analyses (“80% of the power is below $\omega =0.05$”).

(White Noise) For $\{W_t\}\sim \mathrm{WN}(0,{\sigma }^2)$, the spectral density is the constant ${\sigma }^2$.

Substituting the white-noise autocovariance ($\gamma (0)={\sigma }^2$, $\gamma (h)=0$ for $h\ne 0$) into the definition of $f(\omega )$ collapses the sum to its single non-zero term: $f_W(\omega )={\sigma }^2$ at every $\omega$. Constant spectrum is the defining frequency-domain feature of white noise; any deviation from a flat spectrum is structure that an ARMA or other parametric model could in principle capture.

(MA(1) Spectral Density) For $X_t=W_t+\theta W_{t-1}$, the spectrum is a polynomial in $\cos$.

Plugging the MA(1) autocovariance into the definition gives
$f_X(\omega )={\sigma }^2(1+{\theta }^2+2\theta \cos 2\pi \omega ).$
The shape decreases monotonically from $\omega =0$ to $\omega =1/2$ when $\theta >0$, giving low-pass behavior, with power concentrated at low frequencies because positive lag-1 correlation makes realizations look smoothed. With $\theta <0$ the shape increases, giving high-pass behavior with power at $\omega =1/2$ from the alternating structure.

(AR(1) Spectral Density) For $X_t=\varphi X_{t-1}+W_t$ with $|\varphi |<1$, the spectrum is the reciprocal of a polynomial.

By the linear-filter identity below applied to the AR(1) inverted into MA($\infty$) form, the spectrum becomes
$f_X(\omega )=\frac{{\sigma }^2}{|1-\varphi e^{-2\pi i\omega }{|}^2}=\frac{{\sigma }^2}{1-2\varphi \cos (2\pi \omega )+{\varphi }^2}.$
This is unimodal with a peak at $\omega =0$ for $\varphi >0$ (long memory, slow drift) and at $\omega =1/2$ for $\varphi <0$ (rapid alternation). The closer $|\varphi |$ is to $1$, the sharper the peak: the AR(1) is a one-pole linear filter applied to white noise, and the pole at $z=1/\varphi$ creates the spectral concentration.

(Linear Filter Identity) Filtering a stationary process scales its spectrum by the squared magnitude of the filter’s frequency response.

If $Y_t={\sum }_j{a}_j{X}_{t-j}$ is the output of a linear filter with frequency response $A(\omega )={\sum }_j{a}_j{e}^{-2\pi ij\omega }$ applied to a stationary input $X_t$, then
$f_Y(\omega )=|A(\omega ){|}^2{f}_X(\omega ).$
This is the central computational tool of spectral analysis: convolution in the time domain becomes multiplication in the frequency domain. It is why ARMA spectra have closed-form expressions, why filter design is much easier in the frequency domain, and why audio engineering, telecommunications, and seismology all live in the spectral world.

Examples

Periodic process with random phase

For $X_t=U_1\cos (2\pi {\omega }_{0}t)+U_2\sin (2\pi {\omega }_{0}t)$ with $U_1,U_2$ uncorrelated, mean zero, variance ${\sigma }^2$, where does the energy live?

Solution $\gamma (h)=\mathbb{E}[X_t{X}_{t+h}]={\sigma }^2\cos (2\pi {\omega }_{0}h),$ using $\mathbb{E}[U_i{U}_j]={\sigma }^{2}1\{i=j\}$ and the trig identity $\cos \alpha \cos \beta +\sin \alpha \sin \beta =\cos (\alpha -\beta )$. Writing the cosine in complex form, $\gamma (h)=\frac{{\sigma }^2}{2}{e}^{-2\pi i{\omega }_{0}h}+\frac{{\sigma }^2}{2}{e}^{2\pi i{\omega }_{0}h},$ which has the form of an inverse Fourier transform with mass ${\sigma }^2/2$ concentrated at $\pm {\omega }_0$. This is a discrete spectral measure: the process is a single tone, all its energy lives at exactly one frequency, and there is no spectral density, only a spectral measure with two atoms. This is why the more general spectral distribution function exists alongside the density: it accommodates processes whose energy is concentrated at points rather than spread over intervals.

Computing the autocovariance,

Spectrum of an MA(2) low-pass filter

What is the spectrum of $X_t=\frac{1}{3}(W_t+W_{t-1}+W_{t-2})$, the running mean of three white-noise terms?

Solution $a=(1/3,1/3,1/3)$ is $A(\omega )=\frac{1}{3}(1+e^{-2\pi i\omega }+e^{-4\pi i\omega }).$ Factoring out $e^{-2\pi i\omega }$ gives $A(\omega )=e^{-2\pi i\omega }\cdot \frac{1}{3}(1+2\cos 2\pi \omega )$, so $|A(\omega ){|}^2=\frac{1}{9}(1+2\cos 2\pi \omega {)}^2.$ At $\omega =0$ the gain is $1$; at $\omega =1/2$ it is $1/9$; at $\omega =1/3$ the factor $(1+2\cos 2\pi \omega )$ vanishes, producing a spectral zero. Multiplying by the white-noise spectrum ${\sigma }^2$ gives the output spectrum $f_X(\omega )=\frac{{\sigma }^2}{9}(1+2\cos 2\pi \omega {)}^2$, preserving slow variation while killing the highest frequencies. A moving average is a low-pass filter, the simplest smoother in time-series practice.

The frequency response of the filter