2.2c.8 Student's t Distribution

Student's t Distribution

A continuous random variable $T\in \mathbb{R}$ follows the Student’s $t$ distribution with $\nu >0$ degrees of freedom, written $T\sim t_{\nu }$, if it can be expressed as
$T=\frac{Z}{\sqrt{U/\nu }},$
where $Z\sim \mathcal{N}(0,1)$ is normally distributed, $U\sim {\mathcal{X}}_{\nu }^2$ is Chi-Square distributed, and $Z\perp U$.

Equivalently, $T$ is the distribution of a standardized normal random variable when the variance is estimated from data rather than known. For small $\nu$, $t_{\nu }$ has heavier tails than $\mathcal{N}(0,1)$, such that as $\nu \to \infty$, $t_{\nu }\to \mathcal{N}(0,1)$.

Its probability density function is
$f_T(t)=\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\sqrt{\nu \pi }\Gamma \left(\frac{\nu }{2}\right)}{\left(1+\frac{t^2}{\nu }\right)}^{-\frac{\nu +1}{2}},t\in \mathbb{R}.$

The cumulative distribution function of $t_{\nu }$ has no closed-form expression in general, but can be written in terms of the regularized incomplete Beta function $I_x(a,b)$ as
$F_T(t)=1-\frac{1}{2}{I}_{\frac{\nu }{\nu +t^2}}\left(\frac{\nu }{2},\frac{1}{2}\right).$

A key characteristic of $t_{\nu }$ is robustness to outliers relative to the normal due to heavier tails, especially when $\nu$ is small. The tails decay polynomially as $|t{|}^{-(\nu +1)}$, in contrast to the exponential decay of the Gaussian.

Special cases:

• If $\nu =1$, then $T$ follows the Cauchy distribution, which has no finite mean or variance.

• If $\nu \to \infty$, then $t_{\nu }\to \mathcal{N}(0,1)$, and in practice $t_{\nu }$ is nearly indistinguishable from the standard normal for $\nu \gtrsim 30$.

Standard t-statistic $X_1,\dots ,X_n\stackrel{\text{i.i.d.}}{\sim }\mathcal{N}(\mu ,{\sigma }^2)$ with unknown ${\sigma }^2$, then $\frac{\bar{X}-\mu }{S/\sqrt{n}}\sim t_{n-1},$ where $S^2=\frac{1}{n-1}{\sum }_{i=1}^n(X_i-\bar{X}{)}^2$. This is the foundation of the one-sample t-test and the construction of confidence intervals for the mean when the population variance is unknown.

If

Intuition

The Student’s $t$ distribution arises from a simple question: what happens when you standardize a normal random variable but have to estimate the standard deviation from data? If $\sigma$ were known, the standardized variable $\frac{\bar{X}-\mu }{\sigma /\sqrt{n}}$ would be exactly $\mathcal{N}(0,1)$. But replacing $\sigma$ with the sample standard deviation $S$ introduces extra randomness in the denominator, which fattens the tails—extreme values become more probable because $S$ can be unusually small.

With few observations ($\nu$ small), $S$ is a noisy estimate and the tails are substantially heavier. As $\nu$ grows, $S$ becomes a reliable estimate of $\sigma$, and $t_{\nu }$ collapses to $\mathcal{N}(0,1)$. This is why the $t$-distribution is central to inference with small samples: it correctly accounts for the uncertainty in estimating the spread.

Properties

(Mean) $\mathbb{E}[T\sim t_{\nu }]=0$ for $\nu >1$. (Undefined for $\nu \le 1$.)

(Variance) $\text{Var}[T\sim t_{\nu }]=\frac{\nu }{\nu -2}$ for $\nu >2$. (Infinite for $1<\nu \le 2$; undefined for $\nu \le 1$.)

As $\nu$ grows, the variance decreases toward $1$, recovering the standard normal variance. For small $\nu$, the inflated variance reflects the additional uncertainty from estimating ${\sigma }^2$.

(Symmetry) The density $f_T(t)$ is symmetric about $0$, so $T$ and $-T$ are identically distributed and all odd moments vanish (when they exist).

(Kurtosis) $\text{Kurt}[T\sim t_{\nu }]=\frac{6}{\nu -4}$ for $\nu >4$. (Infinite for $2<\nu \le 4$.)

The excess kurtosis is always positive (leptokurtic), confirming heavier tails than the Gaussian. As $\nu \to \infty$, the kurtosis tends to $0$.

(Scaling) If $T\sim t_{\nu }$, then $T^2\sim F(1,\nu )$, where $F(1,\nu )$ is the F-distribution with $1$ and $\nu$ degrees of freedom.

(Sum) The sum of independent $t$-distributed random variables does not follow a $t$-distribution. This is unlike the Gaussian, which is closed under addition.

(Gaussian Limit) For any fixed $t$, $f_{t_{\nu }}(t)\to f_{\mathcal{N}(0,1)}(t)$ as $\nu \to \infty$, since $U/\nu \stackrel{p}{\to }1$ by the law of large numbers, so $T=Z/\sqrt{U/\nu }\stackrel{d}{\to }Z$.

Examples

Confidence Interval for the Mean

A researcher measures the reaction time (in ms) of $n=8$ participants: $\{312,298,325,340,305,290,318,330\}$. Assume the reaction times are normally distributed with unknown mean $\mu$ and unknown variance ${\sigma }^2$.

Construct a $95\%$ confidence interval for $\mu$. First, compute the sample statistics: $\bar{X}=\frac{1}{8}{\sum }_{i=1}^8{X}_i=314.75,S^2=\frac{1}{7}{\sum }_{i=1}^8(X_i-\bar{X}{)}^2=295.07,S\approx \mathrm{17.18.}$ Since $\sigma$ is unknown, we use the $t$-distribution with $\nu =n-1=7$ degrees of freedom. The $95\%$ confidence interval is $\bar{X}\pm t_{7,0.025}\cdot \frac{S}{\sqrt{n}}=314.75\pm 2.365\cdot \frac{17.18}{\sqrt{8}}=314.75\pm 14.37,$ giving the interval $[300.38,329.12]$. Had we incorrectly used $z_{0.025}=1.96$ (the normal critical value), the interval would have been $[302.84,326.66]$—narrower, and thus overly confident for such a small sample.

One-Sample t-Test

A coffee machine is calibrated to dispense ${\mu }_0=250$ mL per cup. A quality inspector draws $n=12$ cups and measures a sample mean of $\bar{X}=243$ mL with sample standard deviation $S=10$ mL.

At significance level $\alpha =0.05$, is there evidence that the machine is under-dispensing? We test $H_0:\mu =250$ against $H_1:\mu <250$ (one-sided). The test statistic is $t=\frac{\bar{X}-{\mu }_0}{S/\sqrt{n}}=\frac{243-250}{10/\sqrt{12}}=\frac{-7}{2.887}\approx -\mathrm{2.425.}$ Under $H_0$, $t\sim t_{11}$. The critical value for a one-sided test at $\alpha =0.05$ with $11$ degrees of freedom is $t_{11,0.05}\approx -1.796$. Since $-2.425<-1.796$, we reject $H_0$. There is statistically significant evidence at the $5\%$ level that the machine is dispensing less than $250$ mL.