Integrated Brownian Motion

Integrated Brownian Motion

Integrated Brownian Motion (IBM) is the stochastic process $I(t)={\int }_0^t{W}_{s}ds$ where $W_s$ is standard Brownian motion on $[0,\infty )$. Because integration of a continuous path produces a differentiable path, IBM sample paths are once-differentiable ($C^1$) — smoother than Brownian motion but rougher than analytic functions. As a Gaussian process, IBM has mean zero and covariance kernel

$$K_I(s,t)=\frac{1}{2}\max (s,t)\min (s,t{)}^2-\frac{1}{6}\min (s,t{)}^3=\frac{1}{6}\min (s,t{)}^2(3\max (s,t)-\min (s,t)).$$

The kernel enforces $I(0)=0$ and $I^{\prime }(0)=0$; in regression, these constraints are released by adding a linear term:

$$f(t)={\beta }_0+{\beta }_{1}t+\tau I(t),{\beta }_0,{\beta }_1\stackrel{\text{iid}}{\sim }\mathcal{N}(0,C),C\to \infty ,$$

which gives the combined kernel $K(s,t)=C(1+st)+{\tau }^2{K}_I(s,t)$.

Headline result: under this prior, the posterior mean from GP regression with noisy observations is the natural cubic spline interpolant of the data — the same smoother used in frequentist penalized splines. This connects Bayesian nonparametric regression directly to a classical smoothing method.

Discrete approximation: placing $\mathcal{N}(0,{\tau }^2)$ priors on the coefficients of a ReLU basis $\{\text{ReLU}(x-j){\}}_{j=1}^{m-1}$ converges (as the grid density grows) to the scaled-IBM-plus-linear-term GP. This is the link between high-dimensional Bayesian ridge regression and GP regression.

Prerequisites

• 3.10 Gaussian Processes — parent index.
• 3.10.1 Kernels — positive semi-definiteness and kernel algebra.
• 3.10.2 GP Regression — the inference machinery that produces the cubic-spline posterior.
• 4.6 Brownian Motion — the underlying process that IBM integrates.

Intuition

Brownian motion samples are continuous but nowhere differentiable — a very rough prior. For most regression problems, this is too rough: we expect the unknown function to have at least continuous first derivatives. Integrating Brownian motion once produces a smoother process. Because $I(t)={\int }_0^t{W}_{s}ds$ and $W_s$ is continuous, the fundamental theorem of calculus gives $I^{\prime }(t)=W_t$, so $I$ is differentiable with Brownian-motion derivative. The derivative is continuous but rough; $I$ itself is $C^1$.

The appearance of IBM in Bayesian nonparametric regression is not an aesthetic choice — it emerges as the limit of high-dimensional linear regression with a ReLU basis. Consider the regression $y_i={\beta }_0+{\beta }_1{x}_i+{\sum }_{j=2}^m{\beta }_j\text{ReLU}(x_i-(j-1))+{ϵ}_i$ with priors ${\beta }_j\stackrel{\text{iid}}{\sim }\mathcal{N}(0,{\tau }^2)$ for $j\ge 2$. Writing the regression function as $f(x)$, the covariance between $f(u)$ and $f(v)$ under the prior equals a sum of products of ReLU functions, which under a dense-grid limit converges to the scaled-IBM-plus-linear-term kernel. The IBM kernel is therefore what the “infinite ReLU basis with ridge prior” reduces to.

The cubic-spline connection is the other half of the story. Frequentist smoothing splines find $\hat{f}$ minimizing ${\sum }_i(y_i-f(x_i){)}^2+\lambda \int (f^{\prime \prime }(t){)}^{2}dt$ — quadratic loss plus penalty on the integrated squared second derivative. The minimizer is a natural cubic spline: a piecewise cubic function with continuous second derivatives, linear outside the data range, and cubic between knots. The Bayesian GP with the scaled-IBM-plus-linear-term kernel gives exactly the same posterior mean, with the ridge-penalty $\lambda$ replaced by the noise-to-signal ratio ${\sigma }^2/{\tau }^2=1/{\gamma }^2$.

Properties

(Once-Differentiable Samples) Sample paths are $C^1$ but not $C^2$.

Because $I(t)={\int }_0^t{W}_{s}ds$ and $W_s$ is continuous a.s., $I$ is differentiable a.s. with derivative $I^{\prime }(t)=W_t$. So IBM samples are $C^1$. But since $W$ is not differentiable, $I$ is not $C^2$ — the second derivative does not exist in any classical sense. This smoothness class matches the minimum needed for most physical regression problems (continuous function, continuous slope) without assuming analyticity, which would overpenalize sharp features. The posterior mean inherits this smoothness: cubic splines are $C^2$ as deterministic functions, and posterior samples are $C^1$ but not $C^2$.

(IBM Kernel Derivation) Double-integral of the Brownian motion kernel.

The IBM kernel is computed by exchanging integration and expectation:

$$K_I(s,t)=\text{Cov}\left({\int }_0^s{W}_{u}du,{\int }_0^t{W}_{v}dv\right)={\int }_0^s{\int }_0^t\text{Cov}(W_u,W_v)dvdu={\int }_0^s{\int }_0^t\min (u,v)dvdu.$$

Assume $s\le t$ and split the inner integral at $v=u$:

$${\int }_0^t\min (u,v)dv={\int }_0^{u}vdv+{\int }_u^{t}udv=\frac{u^2}{2}+u(t-u)=ut-\frac{u^2}{2}.$$

Then

$$K_I(s,t)={\int }_0^s\left(ut-\frac{u^2}{2}\right)du=\frac{s^{2}t}{2}-\frac{s^3}{6}\text{for }s\le t.$$

Symmetrizing, $K_I(s,t)=\frac{1}{2}\max (s,t)\min (s,t{)}^2-\frac{1}{6}\min (s,t{)}^3$. Positive semi-definiteness inherits from the Brownian motion kernel via the integral representation.

(Scaled IBM Plus Linear Term) Releases the $f(0)=f^{\prime }(0)=0$ constraint.

The bare IBM kernel pins $f(0)=0$ (since $K_I(0,0)=0$) and $f^{\prime }(0)=0$ (from $K_I$‘s cubic-in-$s$ behavior at small $s$). For generic regression where no special boundary condition applies, add a linear term:

$$f(t)={\beta }_0+{\beta }_{1}t+\tau I(t),{\beta }_0,{\beta }_1\stackrel{\text{iid}}{\sim }\mathcal{N}(0,C).$$

Independence of ${\beta }_0,{\beta }_1$, and $\{I(t)\}$ gives the additive kernel

$$K(s,t)=C(1+st)+{\tau }^2{K}_I(s,t).$$

With $C\to \infty$, the linear component becomes an improper flat prior on intercept and slope, handled in the posterior via the Sherman-Morrison-Woodbury identity. The ratio $\gamma =\tau /\sigma$ is the only identifiable hyperparameter when $C\to \infty$.

(Cubic-Spline Posterior Mean) Posterior under IBM+linear kernel is a natural cubic spline.

For noisy observations $y_i=f(x_i)+{ϵ}_i$ with scaled-IBM-plus-linear-term prior and $C\to \infty$, the posterior mean

$$\hat{f}(x)=(1,x)({\mathbf{X}}^{\top }{\mathbf{A}}_{\gamma }^{-1}\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }{\mathbf{A}}_{\gamma }^{-1}\mathbf{y}+{\gamma }^2{\mathbf{k}}_I(x{)}^{\top }({\mathbf{A}}_{\gamma }^{-1}-{\mathbf{A}}_{\gamma }^{-1}\mathbf{X}({\mathbf{X}}^{\top }{\mathbf{A}}_{\gamma }^{-1}\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }{\mathbf{A}}_{\gamma }^{-1})\mathbf{y}$$

with ${\mathbf{A}}_{\gamma }=\mathbf{I}+{\gamma }^2{\mathbf{K}}_I$ is a natural cubic spline: piecewise cubic on the intervals defined by the $x_i$, twice continuously differentiable at the knots, and linear outside the data range. The ratio $\gamma =\tau /\sigma$ controls smoothness: small $\gamma$ gives near-linear fits (ridge-shrunk toward the OLS line), large $\gamma$ gives near-interpolating fits (overfitting the data). This is exactly the effect of the smoothing-spline penalty parameter $\lambda$, with the identification $\lambda =1/{\gamma }^2$.

(Simpson-Rule Integration) Posterior mean for $\int f$ gives Simpson’s rule.

Just as the Brownian motion prior induces the trapezoidal integration rule for $\mathbb{E}[{\int }_0^{1}f(x)dx\mid \mathbf{y}]$, the scaled-IBM-plus-linear-term prior induces a version of Simpson’s rule. This is not a coincidence: classical quadrature rules can be viewed as posterior-mean estimators under specific GP priors over the integrand. The Brownian motion prior assumes only continuity, so it commits to the weakest (trapezoidal) rule. The IBM prior assumes differentiability, so it commits to the stronger (Simpson) rule that exploits smoothness. The connection between GP priors and quadrature is explored in the Bayesian-numerics literature (Diaconis 1988, Hennig-Osborne-Kersting 2022).

(Discrete ReLU-Basis Approximation) Finite ReLU basis $\to$ IBM in the dense-grid limit.

The discrete model $y_i={\beta }_0+{\beta }_1{x}_i+{\sum }_{j=2}^m{\beta }_j\text{ReLU}(x_i-(j-1))+{ϵ}_i$ with ${\beta }_j\stackrel{\text{iid}}{\sim }\mathcal{N}(0,{\tau }^2)$ for $j\ge 2$ converges, as $m\to \infty$ with $m/M$ fixed, to the scaled-IBM-plus-linear-term GP on $[0,M]$. The derivation uses the explicit Brownian motion representation $B(t)\approx {\sum }_{i=1}^N{\alpha }_{i}1\{t\ge t_i\}$ with ${\alpha }_i\stackrel{\text{iid}}{\sim }\mathcal{N}(0,M/N)$, which when integrated gives $I(t)\approx {\sum }_i{\alpha }_i(t-t_i{)}_{+}$ — exactly the ReLU basis with ${\beta }_j=\tau {\alpha }_{j-1}$. This is the concrete way Bayesian ridge regression with a ReLU basis is GP regression with an IBM-plus-linear-term kernel.

(Higher-Order Integrated Brownian Motion) Integrating $k$ times gives $C^k$ sample paths.

Define $I^{(k)}(t)={\int }_0^t{\int }_0^{s_1}\cdots {\int }_0^{s_{k-1}}{W}_{s_k}d{s}_k\cdots d{s}_1$ — the $k$-times-integrated Brownian motion. Sample paths of $I^{(k)}$ are $C^k$ (but not $C^{k+1}$), and the posterior mean for GP regression with the $k$-integrated kernel plus a polynomial linear term is a spline of degree $2k+1$. The $k=0$ case is Brownian motion (linear-spline posterior), $k=1$ is IBM (cubic-spline posterior), $k=2$ gives quintic-spline posteriors, and so on. Higher-order IBM is appropriate when the regression function is believed to have higher derivatives than just the first; the cost is that more observations are needed to escape the prior’s smoothness constraint.

Derivation

Derivation of the IBM kernel from the Brownian motion kernel, carried out for $s\le t$ (the general form follows by symmetry).

Step 1: Set up the double integral. By definition,

$$K_I(s,t)=\text{Cov}\left({\int }_0^s{W}_{u}du,{\int }_0^t{W}_{v}dv\right).$$

Since $W$ has mean zero, the covariance equals the expectation of the product, and by Fubini’s theorem the expectation commutes with the integrals:

$$K_I(s,t)=\mathbb{E}\left[{\int }_0^s{\int }_0^t{W}_u{W}_{v}dvdu\right]={\int }_0^s{\int }_0^t\mathbb{E}[W_u{W}_v]dvdu={\int }_0^s{\int }_0^t\min (u,v)dvdu.$$

Step 2: Evaluate the inner integral. For fixed $u$, split $[0,t]$ at $v=u$:

$${\int }_0^t\min (u,v)dv={\int }_0^{u}vdv+{\int }_u^{t}udv=\frac{u^2}{2}+u(t-u)=ut-\frac{u^2}{2}.$$

Step 3: Evaluate the outer integral. Substitute and integrate over $u\in [0,s]$:

$$K_I(s,t)={\int }_0^s\left(ut-\frac{u^2}{2}\right)du={\left[\frac{u^{2}t}{2}-\frac{u^3}{6}\right]}_0^s=\frac{s^{2}t}{2}-\frac{s^3}{6}.$$

Step 4: Symmetrize. For $s\le t$, we have $\min (s,t)=s$ and $\max (s,t)=t$, so

$$K_I(s,t)=\frac{1}{2}\max (s,t)\min (s,t{)}^2-\frac{1}{6}\min (s,t{)}^3.$$

By symmetry in $s$ and $t$, this formula holds for all $s,t\ge 0$.

Examples

(Easy) Variance of IBM at time $t$.

Compute $\text{Var}(I(t))$.

Solution. $\text{Var}(I(t))=K_I(t,t)=\frac{1}{2}t\cdot t^2-\frac{1}{6}{t}^3=\frac{t^3}{3}$. Variance grows cubically with time, versus linearly for Brownian motion. This matches intuition: integrating smooths short-term fluctuations but accumulates long-term drift.

(Medium) Verify the IBM kernel via the ReLU-basis approximation.

Show that the ReLU-basis prior $f(x)={\sum }_{j=1}^N{\alpha }_j(x-t_j{)}_{+}$ with $t_j=jM/N$ and ${\alpha }_j\stackrel{\text{iid}}{\sim }\mathcal{N}(0,M/N)$ converges, as $N\to \infty$, to $\tau I(x)$ with $\tau =1$.

Solution. Compute the covariance:

$$\text{Cov}(f(u),f(v))=\sum _{j=1}^N\frac{M}{N}(u-t_j{)}_{+}(v-t_j{)}_{+}.$$

This is a Riemann sum for ${\int }_0^M(u-s{)}_{+}(v-s{)}_{+}ds$ with step size $M/N$. For $0\le s\le \min (u,v)$, both factors are $(u-s)$ and $(v-s)$; for $s>\min (u,v)$ one factor is zero. Evaluating the integral assuming $u\le v$:

$${\int }_0^u(u-s)(v-s)ds={\int }_0^u(uv-(u+v)s+s^2)ds=u^{2}v-\frac{(u+v)u^2}{2}+\frac{u^3}{3}.$$

Simplify: $u^{2}v-\frac{u^3}{2}-\frac{u^{2}v}{2}+\frac{u^3}{3}=\frac{u^{2}v}{2}-\frac{u^3}{6}=K_I(u,v)$ for $u\le v$. Therefore the Riemann sum converges to $K_I(u,v)$, confirming the ReLU-basis prior limits to IBM.

(Hard) Natural cubic spline as posterior mean.

Take $n=3$ data points $(x_i,y_i)=(0,0),(1,2),(2,1)$ with noise $\sigma =0.1$ and scaled-IBM-plus-linear-term prior with $\gamma =1$. Describe the posterior mean $\hat{f}(x)$ qualitatively.

Solution. The posterior mean is a natural cubic spline through the noisy observations. Key qualitative features:

• On $[0,2]$, $\hat{f}$ is piecewise cubic, with two cubic pieces joined at $x=1$ with continuous first and second derivatives.
• Outside $[0,2]$, $\hat{f}$ is linear (this is the “natural” boundary condition, equivalent to ${\hat{f}}^{\prime \prime }(0)={\hat{f}}^{\prime \prime }(2)=0$).
• Because $\sigma =0.1$ is small relative to the signal, $\hat{f}$ nearly interpolates: $\hat{f}(0)\approx 0$, $\hat{f}(1)\approx 2$, $\hat{f}(2)\approx 1$.
• Between knots, $\hat{f}$ smoothly interpolates with a rise from $(0,0)$ to $(1,2)$ and a fall from $(1,2)$ to $(2,1)$, with continuous slope at $x=1$.

Larger $\sigma$ (or smaller $\gamma$) would push $\hat{f}$ toward the ridge-shrunk OLS line through the three points, sacrificing interpolation for smoothness. Smaller $\sigma$ (or larger $\gamma$) would push $\hat{f}$ toward exact interpolation with possible overshoot between knots.

See Also

• 3.10 Gaussian Processes — parent index.
• 3.10.1 Kernels — the IBM kernel sits in the kernel taxonomy here.
• 3.10.2 GP Regression — regression formulas with the IBM kernel.
• 4.6 Brownian Motion — the underlying process that IBM integrates.
• 3.8.1 Bayesian Linear Regression — parametric ReLU-basis cousin that limits to IBM.