Linear Filters and Frequency Response

Linear Filter and Frequency Response

A linear time-invariant filter transforms an input series $\{x_t\}$ into an output series $\{y_t\}$ by a fixed weighted combination of past, present, and possibly future values:
$y_t={\sum }_{j=-\infty }^{\infty }{a}_j{x}_{t-j}=(a\ast x{)}_t,{\sum }_j|a_j|<\infty .$
The coefficients $\{a_j\}$ are the filter weights (also called the impulse response: they are exactly what comes out when the input is a unit impulse $x_t=1\{t=0\}$). The filter is causal if $a_j=0$ for all $j<0$, meaning the output depends only on present and past inputs and can be implemented online; non-causal filters require the entire input record before producing output and so are restricted to offline analysis.

The frequency response is the discrete-time Fourier transform of the weights:
$A(\omega )={\sum }_{j=-\infty }^{\infty }{a}_j{e}^{-2\pi ij\omega },\omega \in [-\frac{1}{2},\frac{1}{2}].$
Its magnitude $|A(\omega )|$ is the gain at frequency $\omega$ (how much each frequency is amplified or attenuated by the filter), and its argument $\arg A(\omega )$ is the phase shift (how much that frequency component is delayed in time).

The central spectral identity is that filtering a stationary input scales its spectrum by the squared gain:
$f_y(\omega )=|A(\omega ){|}^2{f}_x(\omega ).$
This converts the time-domain operation $(a\ast x{)}_t$ into a frequency-domain product, which is far easier to work with both analytically and computationally.

Convolution-Multiplication Duality convolution in the time domain becomes multiplication in the frequency domain. The DTFT of $(a\ast x)$ equals $A(\omega )\hat{x}(\omega )$, and applying expectations to a stationary process gives the spectral identity $f_y=|A{|}^2{f}_x$. Designing a filter to amplify or suppress a specific frequency band is therefore done in the frequency domain (decide what $|A(\omega ){|}^2$ should look like) and inverse-transformed to recover the weights $a_j$ that implement it.

The whole reason linear filters are tractable is that

Intuition

A linear filter is a recipe for re-weighting a time series: smoothing, sharpening, delaying, or differencing. Once the recipe is described in the frequency domain (“keep frequencies below $0.1$, kill frequencies above $0.4$”), its action on any input is just point-wise multiplication of spectra. Convolution is unwieldy because every output value depends on a window of input values; multiplication is trivial because each frequency is handled independently. This duality is what makes spectral analysis the natural language for any system whose response is linear and time-invariant: amplifiers, electrical circuits, audio processing, the cochlea, and most signal-processing pipelines.

The spectral identity $f_y(\omega )=|A(\omega ){|}^2{f}_x(\omega )$ has the same shape as the variance rule for scalar random variables, $\mathrm{Var}(aX)=a^2\mathrm{Var}(X)$. The filter’s gain plays the role of $a$, and squaring is the price of dropping phase information when going from amplitudes to power. Once the spectrum is recognized as a frequency-by-frequency variance, the two are visibly the same identity.

Properties

(Convolution-Multiplication) The DTFT of a convolution is the product of the DTFTs.

The identity $\widehat{a\ast x}(\omega )=A(\omega )\hat{x}(\omega )$, applied to expectations over a stationary process, yields the spectral identity $f_y=|A{|}^2{f}_x$. It also explains why FFT-based convolution beats direct convolution for long filters: an $O(n^2)$ time-domain operation becomes an $O(n\log n)$ frequency-domain product plus two FFTs, with the crossover happening at filter lengths around $n\approx 64$ in typical libraries.

(Cascade) Two filters in series have combined frequency response equal to the product of the individual responses.

If filter $A$ is followed by filter $B$, the cascade has frequency response $B(\omega )A(\omega )$ and the output spectrum is $f_z(\omega )=|B(\omega ){|}^2|A(\omega ){|}^2{f}_x(\omega )$. Cascading is how complex filters are built from simple primitives: pre-emphasis, equalization, decimation pipelines all use this composition rule.

(Causality and Stability) Stability requires the weights to be absolutely summable; causality requires no negative-index weights.

A filter with ${\sum }_j|a_j|<\infty$ is stable (bounded inputs produce bounded outputs); a filter with $a_j=0$ for $j<0$ is causal. Stability is required for the frequency response to even be defined as a convergent sum. Causality is required for online application. Many useful filters (zero-phase, ideal low-pass) are non-causal in their natural form and must be approximated causally for real-time use.

(Differencing as a Filter) The difference operator is a high-pass filter with zero gain at zero frequency.

The difference operator $\nabla x_t=x_t-x_{t-1}$ has weights $a_0=1,a_1=-1$ and frequency response $A(\omega )=1-e^{-2\pi i\omega }$, giving
$|A(\omega ){|}^2=4{\sin }^2(\pi \omega ).$
The gain is zero at $\omega =0$ (kills constant trends) and reaches its maximum of $4$ at $\omega =1/2$ (amplifies the highest frequencies). This is why differencing a signal with a linear trend produces a stationary residual, but also why over-differencing inflates high-frequency noise, a fact that matters for ARIMA modeling.

(Moving Average as Low-Pass) The simple moving average smooths by averaging over a window.

For weights $a_j=1/L$ for $j=0,\dots ,L-1$, the frequency response works out to
$|A(\omega )|=\frac{1}{L}\left|\frac{\sin (\pi L\omega )}{\sin (\pi \omega )}\right|,$
peaking at $|A(0)|=1$ and zero at $\omega =k/L$ for $k=1,\dots ,L-1$. Increasing $L$ narrows the pass-band around $\omega =0$ (more aggressive smoothing) but also slows the filter’s response to changes in the underlying signal. The phase factor $e^{-i\pi (L-1)\omega }$ encodes a delay of $(L-1)/2$ samples, the cost of asymmetric averaging over only past observations.

(Zero-Phase Filtering) Applying a filter forward and then backward eliminates phase distortion.

Running a filter $A$ once forward and once in reverse (filtfilt in scipy) yields a zero-phase response with squared gain $|A(\omega ){|}^4$. This eliminates the phase distortion of asymmetric filters at the cost of doubling the gain shape and making the operation strictly non-causal. Standard practice for offline cleaning (EEG, audio post-production, climate data) where exact temporal alignment matters more than real-time operation.

Examples

First-difference of an AR(1) process

For $X_t=0.9X_{t-1}+W_t$ with $W_t\sim \mathrm{WN}(0,1)$, what is the spectrum of $Y_t=\nabla X_t$?

Solution $f_X(\omega )=1/(1-1.8\cos 2\pi \omega +0.81)$. The differencing filter has $|A(\omega ){|}^2=4{\sin }^2(\pi \omega )$. Multiplying these gives $f_Y(\omega )=\frac{4{\sin }^2(\pi \omega )}{1-1.8\cos 2\pi \omega +0.81}.$ At $\omega =0$, $f_Y\to 0$, since differencing kills the constant component that the AR(1) builds up at low frequency. At $\omega =1/2$, $f_Y=4/(1+1.8+0.81)\approx 1.10$, larger than $f_X(1/2)=1/3.61\approx 0.28$, so high-frequency noise is amplified four-fold. The output spectrum is bowl-shaped: low frequencies suppressed, high frequencies enhanced.

The AR(1) spectrum is

Filtering away a 60Hz line-noise component

An EEG signal $x_t$ sampled at $f_s=500$ Hz contains a strong $60$ Hz line-noise contamination. Sketch a notch filter that passes everything else and predict the output spectrum.

Solution ${\omega }_0=60/500=0.12$. A notch filter with frequency response approximately $|A(\omega ){|}^2=\{\begin{array}{ll}0& |\omega -{\omega }_0|<\delta \\ 1& \text{otherwise}\end{array}$ for small bandwidth $\delta$ kills the spike. The output spectrum is $f_y(\omega )=|A(\omega ){|}^2{f}_x(\omega )$, identical to $f_x$ except for a thin trough at $\pm 0.12$ that removes the line noise. The corresponding filter weights $a_j$ come from the inverse DFT of $A(\omega )$ and have many non-zero entries (sharp frequency responses require long filters), but the design lives entirely in the frequency domain.

In normalized units the offending frequency is