Time-Frequency Analysis

Time-Frequency Analysis (STFT and Spectrogram)

The short-time Fourier transform (STFT) computes a frequency snapshot for each time by sliding a length-$L$ window function $w_t$ across the data and taking a DFT of each windowed segment:
$d(\tau ,\omega )=\frac{1}{\sqrt{L}}{\sum }_{t=1}^L{w}_t{x}_{\tau +t}{e}^{-2\pi it\omega },$
where $\tau$ slides across the record (typically in steps smaller than $L$ so that windows overlap). The squared modulus $S(\tau ,\omega )=|d(\tau ,\omega ){|}^2$ is the spectrogram: a 2-D function of time and frequency, conventionally displayed as a heat map with time on the horizontal axis and frequency on the vertical axis.

The motivation is that the standard DFT and spectral density assume the spectrum is constant in time. For non-stationary signals (speech, music, EEG, seismic events, financial volatility regimes) the analyst’s question is not “what frequencies are present” but “when is each frequency present.” The STFT trades a single global spectrum for a sequence of local spectra, one per window, recovering the time information that the global DFT discards.

The trade-off is governed by the time-frequency uncertainty principle (Heisenberg-Gabor): a window of length $L$ samples gives frequency resolution $\Delta \omega \sim 1/L$ and time resolution $\Delta \tau \sim L$, so the product $\Delta \tau \cdot \Delta \omega \gtrsim 1$ is bounded below by a constant. Sharper time localization forces blurrier frequency resolution and vice versa, and the choice of $L$ is the central design knob.

Window Choice Matters scipy.signal.spectrogram) is a Tukey or Hanning window, never a rectangle.

A rectangular window has sharp edges that produce strong spectral leakage (its Fourier transform is a sinc, with large side-lobes). Tapered windows like Hanning, Hamming, and Gaussian smooth the edges and dramatically reduce leakage at the cost of slightly broader main lobes. The default in production tools (

Intuition

The DFT of an entire record asks “how much of frequency $\omega$ is in the data, summed over the whole history?” If a $440$ Hz tone plays for the first second of a $10$-second clip and silence follows, the DFT registers $440$ Hz as moderately strong but loses the fact that it was confined to the start. The STFT fixes this by chopping the record into overlapping windows, taking a DFT of each, and arranging the results into a 2-D image: time on one axis, frequency on the other, color showing power. The $440$ Hz tone now appears as a bright horizontal stripe confined to the first second.

The unavoidable trade-off is that a window narrow enough to localize the tone in time is too short to nail down its frequency precisely. This is not a defect of the algorithm; it is a mathematical theorem (Heisenberg-Gabor) that any time-frequency representation built from a single window obeys $\Delta \tau \cdot \Delta \omega \gtrsim 1$. For signals where the frequency content evolves at very different rates in different bands (low frequencies change slowly, high frequencies change quickly), wavelet transforms replace the fixed window with a frequency-dependent window, but that is a different note.

Properties

(Inheriting from the DFT) Each fixed-time slice of the spectrogram is the periodogram of the windowed segment.

So everything that’s true of the periodogram holds within each STFT column: the same ${\chi }^2$ noise distribution at each frequency bin, the same need for smoothing across nearby bins, the same sensitivity to mean and trend within the window. Welch’s method (averaging periodograms across multiple windows) is essentially an STFT with no time axis displayed.

(Window Length Trade-Off) Frequency resolution $\Delta \omega \approx 1/L$; time resolution $\Delta \tau \approx L$.

A window of $L=256$ samples at sampling rate $f_s=1000$ Hz spans $0.256$ s of time and resolves frequencies down to $1000/256\approx 3.9$ Hz. Halving $L$ doubles the time resolution but doubles the frequency bin width. The right $L$ depends entirely on the time scale at which the spectral content is changing: short windows for fast transients, long windows for slowly varying tones.

(Hop Size and Overlap) Adjacent windows are typically separated by a hop $H<L$, giving overlap $(L-H)/L$.

The default overlap is $50\%$ ($H=L/2$). Smaller hops give a smoother spectrogram in the time axis at higher computational cost; hops $\ge L$ leave gaps the window’s smoothing cannot fill. The combination of overlap and a tapered window provides time-resolution finer than the window length itself by interpolating between adjacent windowed estimates.

(Window Choice) Tapered windows suppress leakage at the cost of broader main lobes.

The Hanning window $w_t=\frac{1}{2}(1-\cos (2\pi t/L))$ is the default-of-defaults: $-31$ dB side-lobes versus $-13$ dB for a rectangular window, with only modest main-lobe broadening. The Kaiser window has a tunable parameter $\beta$ that lets the user dial the side-lobe / main-lobe trade-off. Choice of window matters most when looking for weak frequency components near strong ones, since side-lobes from the strong component can mask the weak one entirely.

(Reconstruction) Overlapping windows that satisfy the constant-overlap-add condition allow exact inverse-STFT.

If ${\sum }_{k}w(t-kH)=c$ for some constant $c$, the original signal can be exactly recovered from the STFT by inverting each windowed segment and summing. This makes the STFT a basis for filtering operations performed in the time-frequency plane: zero out spectrogram cells that contain noise, then inverse-STFT to reconstruct a cleaned signal. The technique underlies modern noise reduction (spectral subtraction) and time-stretching (phase vocoder).

(Multitaper Extension) Replacing one window with several orthogonal Slepian tapers reduces variance at fixed time-frequency resolution.

The multitaper method uses $K$ orthogonal Slepian tapers, computes the spectrogram with each, and averages the results. This reduces variance by a factor of $K$ at fixed time-frequency resolution, a significant variance reduction at no cost in resolving power. Implemented in MNE-Python’s tfr_multitaper and nitime.algorithms.multi_taper_psd. Standard in neuroscience and seismology where the signal-to-noise per spectrogram bin is low.

Examples

Chirp signal

For $x_t=\cos (2\pi t(0.1+0.001t))$ for $t=1,\dots ,1000$ (a sinusoid whose frequency rises linearly from $0.1$ to $1.1$ over the record), what does the spectrogram look like, and how does window length change it?

Solution $L=64$ the spectrogram shows a smooth diagonal ridge from $\omega \approx 0.1$ at the start to $\omega \approx 1.1$ at the end (with the part above $0.5$ aliasing back into the visible band). The ridge is narrow vertically (good frequency resolution from the relatively long window) but somewhat smeared horizontally (the frequency shifts noticeably within a single window). With $L=16$ the ridge is sharper in time but blocky in frequency. The DFT of the entire record, by contrast, shows a wide blob spanning $0.1$ to $0.5$, so the time-locality of the chirp is invisible without the time axis.

With

Detecting a transient burst

An EEG channel contains a brief $40$ Hz “gamma burst” in a single $200$-ms window of an otherwise quiet $10$-second recording, sampled at $f_s=1000$ Hz. What window length should you use?

Solution $200$ ms transient at $40$ Hz, an STFT with $L=256$ samples ($256$ ms, about the burst duration) has frequency resolution $\Delta \omega \approx 1000/256\approx 4$ Hz, fine enough to localize the $40$ Hz peak. Hop $H=32$ samples gives more than $7\times$ overlap, so the burst's onset is visible to within $\sim 30$ ms. The spectrogram shows a small bright island near $(t,\omega )=(\text{burst time},40)$ Hz against a dark background. Shortening the window to $L=64$ blurs the frequency localization; lengthening to $L=1024$ sharpens frequency but smears the burst across several seconds, making the transient look like a steady tone.

For a