1.3.2 Confidence Intervals

Confidence Interval

A confidence interval at level $1-\alpha$ for a parameter $\theta$ is a random interval $[L(X),U(X)]$, constructed from the data $X$, such that
${\mathbb{P}}_{\theta }(L(X)\le \theta \le U(X))=1-\alpha$
for all values of $\theta$. The quantity $1-\alpha$ is the confidence level (commonly $0.95$ or $0.99$).

For the sample mean ${\bar{X}}_n$ of i.i.d. observations from a population with mean $\mu$, a $(1-\alpha )$ confidence interval based on the normal approximation is
${\bar{X}}_n\pm z_{\alpha /2}\cdot \text{SE}({\bar{X}}_n),$
where $z_{\alpha /2}$ is the critical value such that $\mathbb{P}(Z>z_{\alpha /2})=\alpha /2$ for $Z\sim \mathcal{N}(0,1)$. Common values: $z_{0.025}=1.96$ (95%), $z_{0.005}=2.576$ (99%).

When the population standard deviation is unknown and $n$ is small, the normal quantile $z_{\alpha /2}$ is replaced by the Student’s $t$ quantile $t_{\alpha /2,n-1}$:
${\bar{X}}_n\pm t_{\alpha /2,n-1}\cdot \frac{s}{\sqrt{n}}.$

Frequentist Interpretation $95\%$ confidence interval does not mean "$\theta$ falls in this interval with probability $0.95$." The parameter $\theta$ is a fixed (unknown) constant—it either lies in the interval or it does not. The correct interpretation is: if we repeated the experiment many times and constructed an interval each time, $95\%$ of those intervals would contain the true $\theta$.

A

Intuition

A confidence interval translates the uncertainty captured by the standard error into a concrete range of plausible values for the unknown parameter. The wider the interval, the less precise our estimate; the narrower, the more informative the data.

The construction relies on the sampling distribution of the estimator. By the CLT, ${\bar{X}}_n$ is approximately normal for large $n$, so the interval ${\bar{X}}_n\pm z_{\alpha /2}\cdot \text{SE}$ captures the central $(1-\alpha )$ mass of that distribution. For small samples where the CLT approximation is rough, the $t$-distribution—which has heavier tails—provides wider and more honest intervals.

Properties

(Width) The width of the interval is $2z_{\alpha /2}\cdot \text{SE}=2z_{\alpha /2}\cdot \sigma /\sqrt{n}$. It shrinks as $O(1/\sqrt{n})$.

(Duality with Hypothesis Testing) A parameter value ${\theta }_0$ lies inside the $(1-\alpha )$ confidence interval if and only if the hypothesis test of $H_0:\theta ={\theta }_0$ at level $\alpha$ fails to reject.

(Bootstrap Confidence Intervals) When the sampling distribution of ${\hat{\Theta }}_n$ is unknown or non-normal, the bootstrap can construct confidence intervals without relying on the CLT—using either the percentile method (take the $\alpha /2$ and $1-\alpha /2$ quantiles of the bootstrap distribution) or the bootstrap-$t$ method.

(Bayesian Analogue) The Bayesian counterpart is the credible interval, derived from the posterior $p(\theta \mid X)$. Unlike confidence intervals, credible intervals can be interpreted as probability statements about $\theta$ (conditional on the observed data and the prior).

Examples

Estimating a Population Mean

A sample of $n=25$ light bulbs has mean lifetime $\bar{x}=1200$ hours with sample standard deviation $s=100$ hours. Construct a $95\%$ confidence interval for the true mean lifetime $\mu$.

Solution $n=25$ is moderate and $\sigma$ is unknown, use the $t$-distribution with $\nu =24$ degrees of freedom. The critical value is $t_{0.025,24}\approx 2.064$. The interval is $1200\pm 2.064\cdot \frac{100}{\sqrt{25}}=1200\pm 2.064\cdot 20=1200\pm 41.3,$ giving $[1158.7,1241.3]$. We are $95\%$ confident that the true mean lifetime lies in this range.

Since

Proportion Confidence Interval

In a survey of $n=500$ voters, $\hat{p}=0.58$ support a policy. Construct a $99\%$ confidence interval for the true proportion $p$.

Solution $\widehat{\text{SE}}(\hat{p})=\sqrt{0.58\cdot 0.42/500}\approx 0.0221$. With $z_{0.005}=2.576$, $0.58\pm 2.576\cdot 0.0221=0.58\pm 0.057,$ giving $[0.523,0.637]$. Even at the $99\%$ level, the interval excludes $0.5$, providing strong evidence that the majority supports the policy.

The estimated standard error is