Beta-Binomial Distribution

A discrete random variable $X \in {0, 1, \dots, n}$ follows the Beta-Binomial distribution $X \sim BetaBin (n, α, β)$ with $n \in N$ trials and shape parameters $α > 0$ , $β > 0$ , defined by the probability mass function
$p_{X} (x) = (x n) \frac{B ( x + α , n - x + β )}{B ( α , β )},$
where $B (α, β) = \frac{Γ ( α ) Γ ( β )}{Γ ( α + β )}$ is the Beta function.

The Beta-Binomial distribution is a compound distribution: it arises from a Binomial experiment in which the success probability $p$ is not fixed, but is itself a random variable drawn from a $Beta (α, β)$ prior. Formally, if
$Θ \sim Beta (α, β), X ∣ Θ \sim Bin (n, Θ),$
then the marginal distribution of $X$ (obtained by integrating out $Θ$ ) is $BetaBin (n, α, β)$ . This can be verified by computing
$p_{X} (x) = \int_{0}^{1} (x n) θ^{x} (1 - θ)^{n - x} \cdot \frac{θ ^{α - 1} ( 1 - θ ) ^{β - 1}}{B ( α , β )} d θ = (x n) \frac{B ( x + α , n - x + β )}{B ( α , β )} .$

A key characteristic of the Beta-Binomial is overdispersion: because the success probability varies across trials (in the sense that it is drawn once from a Beta and then held fixed), the variance of $X$ exceeds that of a Binomial with the same mean. This makes the Beta-Binomial appropriate whenever count data exhibits more variability than the Binomial alone can explain.

Special cases:

If $α = β = 1$ , the success probability is $Uniform (0, 1)$ , and $p_{X} (x) = \frac{1}{n + 1}$ for all $x$ —a discrete uniform over ${0, \dots, n}$ .

As $α + β \to \infty$ with $\frac{α}{α + β} = p$ held constant, the Beta prior concentrates at $p$ and $BetaBin (n, α, β) \to Bin (n, p)$ .

Exchangeability, not Independence $n$ underlying Bernoulli trials are exchangeable but not independent. Conditional on $Θ = θ$ they are i.i.d., but marginally (after integrating out $Θ$ ) they are positively correlated: observing a success makes the next trial more likely to succeed, because it shifts the posterior of $Θ$ upward. This is a manifestation of Bayes' rule in action.

In the Beta-Binomial model, the

Intuition

The Beta-Binomial answers a natural question: what happens when you run a Binomial experiment but don’t know the true success probability? In a standard Binomial, $p$ is fixed and known—every trial has the same chance of success. But in many real situations, $p$ is uncertain or varies across groups. The Beta-Binomial handles this by placing a Beta prior on $p$ and averaging over all possible values.

Think of it as a two-stage process. First, nature draws a success probability $θ$ from $Beta (α, β)$ . Then, $n$ Bernoulli trials are conducted with that $θ$ . Because $θ$ is random, the resulting count $X$ has more spread than a plain Binomial—sometimes $θ$ is high and you see many successes, sometimes it is low and you see few. This extra variability is what makes the Beta-Binomial useful for modeling heterogeneous populations, where different subgroups may have different underlying rates.

Properties

(Mean) $E [X \sim BetaBin (n, α, β)] = \frac{n α}{α + β}$ .

This is the same as $n \cdot E [Θ]$ , which follows from the law of iterated expectation: $E [X] = E [E [X ∣ Θ]] = E [n Θ] = n \cdot \frac{α}{α + β}$ .

(Variance) $Var [X \sim BetaBin (n, α, β)] = \frac{n α β ( α + β + n )}{( α + β ) ^{2} ( α + β + 1 )}$ .

By the law of total variance, $Var (X) = E [Var (X ∣ Θ)] + Var (E [X ∣ Θ])$ . The first term gives the average within-group binomial variance; the second captures the between-group variability due to the random $Θ$ . The result can be rewritten as
$Var (X) = n p (1 - p) (1 + \frac{( n - 1 ) ρ}{1}), p = \frac{α}{α + β}, ρ = \frac{1}{α + β + 1},$
where $ρ$ is the intraclass correlation. Since $ρ > 0$ , the variance always exceeds the Binomial variance $n p (1 - p)$ .

(Overdispersion Factor) The ratio $Var (X) / [n p (1 - p)] = 1 + (n - 1) ρ > 1$ for $n \geq 2$ , confirming that the Beta-Binomial is overdispersed relative to the Binomial.

(Binomial Limit) As $α + β \to \infty$ with $α / (α + β) = p$ fixed, the Beta prior concentrates at $p$ , $ρ \to 0$ , and $BetaBin (n, α, β) \to Bin (n, p)$ .

(Symmetry) $BetaBin (n, α, β)$ satisfies $p_{X} (x) = p_{X} (n - x)$ if and only if $α = β$ , mirroring the symmetry of the $Beta (α, α)$ prior about $1/2$ .

(Marginal Likelihood) In Bayesian inference, the normalizing constant of the posterior $Θ ∣ X$ is exactly the Beta-Binomial PMF. That is, $p_{X} (x) = (x n) \frac{B ( x + α , n - x + β )}{B ( α , β )}$ serves as the marginal likelihood (or evidence) used for Bayesian model comparison.

Examples

Defective Items Across Factories

A company operates many factories, each producing $n = 20$ items per batch. The defect rate $θ$ varies from factory to factory according to $Θ \sim Beta (2, 18)$ , so that the average defect rate is $E [Θ] = 2/20 = 0.1$ . Let $X$ be the number of defective items in a randomly chosen batch.

What is the distribution of $X$ , and how does its variance compare to a Binomial with $p = 0.1$ ? Since $Θ \sim Beta (2, 18)$ and $X ∣ Θ \sim Bin (20, Θ)$ , we have $X \sim BetaBin (20, 2, 18)$ . The mean is $E [X] = \frac{20 \cdot 2}{2 + 18} = 2,$ which matches $Bin (20, 0.1)$ . However, the variance is $Var (X) = \frac{20 \cdot 2 \cdot 18 \cdot ( 20 + 20 )}{2 0 ^{2} \cdot 21} = \frac{20 \cdot 2 \cdot 18 \cdot 40}{400 \cdot 21} = \frac{14400}{8400} \approx 3.43,$ while $Var [Bin (20, 0.1)] = 20 \cdot 0.1 \cdot 0.9 = 1.8$ . The Beta-Binomial variance is nearly twice as large, reflecting the factory-to-factory heterogeneity in defect rates. The overdispersion factor is $3.43/1.8 \approx 1.90$ .

Bayesian Prediction

You are estimating the probability $θ$ that a website visitor clicks a particular ad. Your prior is $Θ \sim Beta (3, 7)$ , reflecting a belief that the click-through rate is around $30%$ . You observe $n = 50$ visitors and $x = 8$ clicks.

What is the posterior distribution of $Θ$ , and what is the predictive probability that the next visitor clicks? By Beta-Bernoulli conjugacy, the posterior is $Θ ∣ X = 8 \sim Beta (3 + 8, 7 + 50 - 8) = Beta (11, 49) .$ The posterior mean is $\frac{11}{11 + 49} = \frac{11}{60} \approx 0.183$ , pulled from the prior mean of $0.3$ toward the observed rate of $8/50 = 0.16$ by the weight of the data.

The posterior predictive probability that the next visitor clicks is simply the posterior mean of $Θ$ :
$P (X_{51} = 1 ∣ X = 8) = E [Θ ∣ X = 8] = \frac{11}{60} \approx 0.183.$
More generally, the predictive distribution for $Y$ new successes in $m$ future trials is $Y ∣ X = 8 \sim BetaBin (m, 11, 49)$ , which is the Beta-Binomial with the updated posterior parameters.

Project Alexandria

Explorer

2.2b.10 Beta-Binomial Distribution

Intuition

Properties

(Mean) $E [X \sim BetaBin (n, α, β)] = \frac{n α}{α + β}$ .

(Variance) $Var [X \sim BetaBin (n, α, β)] = \frac{n α β ( α + β + n )}{( α + β ) ^{2} ( α + β + 1 )}$ .

(Overdispersion Factor) The ratio $Var (X) / [n p (1 - p)] = 1 + (n - 1) ρ > 1$ for $n \geq 2$ , confirming that the Beta-Binomial is overdispersed relative to the Binomial.

(Binomial Limit) As $α + β \to \infty$ with $α / (α + β) = p$ fixed, the Beta prior concentrates at $p$ , $ρ \to 0$ , and $BetaBin (n, α, β) \to Bin (n, p)$ .

(Symmetry) $BetaBin (n, α, β)$ satisfies $p_{X} (x) = p_{X} (n - x)$ if and only if $α = β$ , mirroring the symmetry of the $Beta (α, α)$ prior about $1/2$ .

(Marginal Likelihood) In Bayesian inference, the normalizing constant of the posterior $Θ ∣ X$ is exactly the Beta-Binomial PMF. That is, $p_{X} (x) = (x n) \frac{B ( x + α , n - x + β )}{B ( α , β )}$ serves as the marginal likelihood (or evidence) used for Bayesian model comparison.

Examples

Graph View

Table of Contents

Project Alexandria

Explorer

2.2b.10 Beta-Binomial Distribution

Intuition

Properties

(Mean) E[X∼BetaBin(n,α,β)]=α+βnα​.

(Variance) Var[X∼BetaBin(n,α,β)]=(α+β)2(α+β+1)nαβ(α+β+n)​.

(Overdispersion Factor) The ratio Var(X)/[np(1−p)]=1+(n−1)ρ>1 for n≥2, confirming that the Beta-Binomial is overdispersed relative to the Binomial.

(Binomial Limit) As α+β→∞ with α/(α+β)=p fixed, the Beta prior concentrates at p, ρ→0, and BetaBin(n,α,β)→Bin(n,p).

(Symmetry) BetaBin(n,α,β) satisfies pX​(x)=pX​(n−x) if and only if α=β, mirroring the symmetry of the Beta(α,α) prior about 1/2.

(Marginal Likelihood) In Bayesian inference, the normalizing constant of the posterior Θ∣X is exactly the Beta-Binomial PMF. That is, pX​(x)=(xn​)B(α,β)B(x+α,n−x+β)​ serves as the marginal likelihood (or evidence) used for Bayesian model comparison.

Examples

Graph View

Table of Contents

(Mean) $E [X \sim BetaBin (n, α, β)] = \frac{n α}{α + β}$ .

(Variance) $Var [X \sim BetaBin (n, α, β)] = \frac{n α β ( α + β + n )}{( α + β ) ^{2} ( α + β + 1 )}$ .

(Overdispersion Factor) The ratio $Var (X) / [n p (1 - p)] = 1 + (n - 1) ρ > 1$ for $n \geq 2$ , confirming that the Beta-Binomial is overdispersed relative to the Binomial.

(Binomial Limit) As $α + β \to \infty$ with $α / (α + β) = p$ fixed, the Beta prior concentrates at $p$ , $ρ \to 0$ , and $BetaBin (n, α, β) \to Bin (n, p)$ .

(Symmetry) $BetaBin (n, α, β)$ satisfies $p_{X} (x) = p_{X} (n - x)$ if and only if $α = β$ , mirroring the symmetry of the $Beta (α, α)$ prior about $1/2$ .

(Marginal Likelihood) In Bayesian inference, the normalizing constant of the posterior $Θ ∣ X$ is exactly the Beta-Binomial PMF. That is, $p_{X} (x) = (x n) \frac{B ( x + α , n - x + β )}{B ( α , β )}$ serves as the marginal likelihood (or evidence) used for Bayesian model comparison.