2.2c.9 Beta Distribution

Beta Distribution

A continuous random variable $X\in [0,1]$ follows the Beta distribution $X\sim \text{Beta}(\alpha ,\beta )$ with shape parameters $\alpha >0$ and $\beta >0$, defined by the probability density function
$f_X(x)=\frac{x^{\alpha -1}(1-x{)}^{\beta -1}}{B(\alpha ,\beta )},x\in [0,1],$
where B(\alpha,\beta) = \dfrac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}={\Large\int}\limits_{0}^{1} \enspace t^{\alpha-1}(1-t)^{\beta - 1}dt is the Beta function, which serves as a normalizing constant ensuring the sum-to-one condition.

The Beta distribution models random proportions, probabilities, and rates—any quantity naturally bounded between $0$ and $1$. Because its two shape parameters $\alpha$ and $\beta$ control the density’s shape independently, the Beta family can produce symmetric, left-skewed, right-skewed, U-shaped, and uniform densities, making it exceptionally flexible.

Special cases:

• If $\alpha =\beta =1$, then $X\sim \text{Uniform}(0,1)$, since $f_X(x)=1$ for all $x\in [0,1]$.
• If $\alpha =1$ and $\beta >1$, the density is strictly decreasing on $[0,1]$.
• If $\alpha >1$ and $\beta =1$, the density is strictly increasing on $[0,1]$.

The cumulative distribution function of the Beta distribution is
$F_X(x)=I_x(\alpha ,\beta )=\frac{1}{B(\alpha ,\beta )}{\int }_0^x{t}^{\alpha -1}(1-t{)}^{\beta -1}dt,$
where $I_x(\alpha ,\beta )$ is the regularized incomplete Beta function, which has no closed-form expression in general.

A key characteristic of the Beta distribution is shape control: $\alpha$ governs how much mass is pulled toward $1$, and $\beta$ governs how much mass is pulled toward $0$. When both $\alpha ,\beta >1$, the density is unimodal with mode at $\frac{\alpha -1}{\alpha +\beta -2}$; when both $\alpha ,\beta <1$, the density is U-shaped, concentrating mass near $0$ and $1$.

Bayesian Interpretation conjugate prior for the Bernoulli and Binomial likelihoods. If $\Theta \sim \text{Beta}(\alpha ,\beta )$ is a prior belief about a probability parameter and $X_1,\dots ,X_n\mid \Theta \stackrel{\text{i.i.d.}}{\sim }\text{Ber}(\Theta )$, then the posterior is $\Theta \mid X_{1:n}\sim \text{Beta}\left(\alpha +{\sum }_{i=1}^n{X}_i,\beta +n-{\sum }_{i=1}^n{X}_i\right).$ The parameters $\alpha$ and $\beta$ can thus be interpreted as pseudo-counts: $\alpha -1$ prior successes and $\beta -1$ prior failures.

In Bayesian inference, the Beta distribution is the

Intuition

The Beta distribution is best understood as a distribution over probabilities. Suppose you are trying to estimate the true probability $\theta$ that a coin lands heads. Before flipping the coin, you might express your uncertainty about $\theta$ using a $\text{Beta}(\alpha ,\beta )$ prior. After observing $s$ heads and $f$ tails, your updated belief is $\text{Beta}(\alpha +s,\beta +f)$. Each success shifts mass toward $1$; each failure shifts mass toward $0$. As the number of observations grows, the posterior concentrates around the true $\theta$.

This makes the Beta distribution central to Bayesian estimation and to any setting where one needs to reason about unknown proportions—click-through rates, disease prevalence, batting averages, and so on.

Properties

(Mean) $\mathbb{E}[X\sim \text{Beta}(\alpha ,\beta )]=\frac{\alpha }{\alpha +\beta }$.

(Variance) $\text{Var}[X\sim \text{Beta}(\alpha ,\beta )]=\frac{\alpha \beta }{(\alpha +\beta {)}^2(\alpha +\beta +1)}$.

As $\alpha +\beta$ grows, the variance shrinks, concentrating the distribution more tightly around its mean. This reflects greater certainty—in a Bayesian context, more data yields a more concentrated posterior.

(Mode) If $\alpha ,\beta >1$, $\text{mode}(X)=\frac{\alpha -1}{\alpha +\beta -2}$.

(Symmetry) If $X\sim \text{Beta}(\alpha ,\beta )$, then $1-X\sim \text{Beta}(\beta ,\alpha )$.

This means swapping $\alpha$ and $\beta$ reflects the density about $x=1/2$. In particular, if $\alpha =\beta$, then $X$ is symmetric around $1/2$.

(Reflection of the Gamma) If $Y_1\sim \text{Gamma}(\alpha ,1)$ and $Y_2\sim \text{Gamma}(\beta ,1)$ are independent, then

$\frac{Y_1}{Y_1+Y_2}\sim \text{Beta}(\alpha ,\beta ).$
This construction gives a concrete way to generate Beta random variables from Gamma random variables, and reveals the Beta as measuring the relative share of one Gamma process against the total.

(Order Statistics) If $U_{(k)}$ is the $k$-th order statistic of $n$ i.i.d. $\text{Uniform}(0,1)$ random variables, then

$U_{(k)}\sim \text{Beta}(k,n-k+1).$

(MGF) The moment generating function of $X\sim \text{Beta}(\alpha ,\beta )$ has no simple closed form, but the $r$-th raw moment is given by

$\mathbb{E}[X^r]=\frac{B(\alpha +r,\beta )}{B(\alpha ,\beta )}={\prod }_{j=0}^{r-1}\frac{\alpha +j}{\alpha +\beta +j}.$

Examples

Coin Fairness

Suppose you want to determine whether a coin is fair. You start with no prior information, so you set $\Theta \sim \text{Beta}(1,1)=\text{Uniform}(0,1)$. You flip the coin $10$ times and observe $7$ heads and $3$ tails.

What is the posterior distribution of $\Theta$, and what is the MAP estimate? By conjugacy, the posterior is $\Theta \mid \text{data}\sim \text{Beta}(1+7,1+3)=\text{Beta}(8,4).$ The MAP estimate is the mode of the posterior: ${\hat{\theta }}_{\text{MAP}}=\frac{8-1}{8+4-2}=\frac{7}{10}=0.7,$ which coincides with the sample proportion. The posterior mean, however, is $\mathbb{E}[\Theta \mid \text{data}]=\frac{8}{8+4}=\frac{2}{3}\approx 0.667,$ which is shrunk toward $1/2$ relative to the MLE, reflecting the influence of the uniform prior.

Order Statistic Distribution

Let $U_1,U_2,U_3{\sim }_{\text{iid}}\text{Uniform}(0,1)$. Define $M=\text{median}(U_1,U_2,U_3)=U_{(2)}$.

What is the distribution and expected value of $M$? By the order statistic property, the $k$-th order statistic of $n$ i.i.d. $\text{Uniform}(0,1)$ random variables is $\text{Beta}(k,n-k+1)$. Here $k=2$ and $n=3$, so $M\sim \text{Beta}(2,2).$ Since $\alpha =\beta =2$, the distribution is symmetric about $1/2$ (by the symmetry property). Its expected value is $\mathbb{E}[M]=\frac{2}{2+2}=\frac{1}{2},$ and its variance is $\text{Var}(M)=\frac{2\cdot 2}{4^2\cdot 5}=\frac{1}{20}$. The median of three uniform draws is, on average, exactly $1/2$, but with less variability than any single draw (which has variance $1/12$).