A conjugate family is a prior distribution such that the posterior, obtained by Bayes’ rule with a particular likelihood, lies in the same parametric family as the prior. Conjugate priors deliver closed-form posterior updates, so Bayesian inference reduces to tracking a few hyperparameters across observations instead of evaluating intractable integrals. They are the first tool to reach for whenever a model’s likelihood admits a conjugate partner.

The practical shape of every conjugate update is the same: the prior’s hyperparameters decompose into “prior counts” (successes, trials, sum-of-squares, effective sample size), and the posterior’s hyperparameters equal the prior’s plus the corresponding data counts. This makes the prior interpretable as “pseudo-data” whose influence is diluted as real data accumulate. When enough data is seen, the posterior is dominated by the likelihood and the prior washes out (see Bayesian Framework’s (Prior Washing Out) property).

Prerequisites

• 3.1 Bayesian Framework — prior, likelihood, posterior, Bayes’ rule.
• 1.2.1 Maximum Likelihood Estimation — frequentist counterpart referenced in each conjugate example.

Contents

• 3.5.1 Beta-Binomial Model — coin-bias parameter, the canonical conjugate pair.
• 3.5.2 Normal-Normal Model — Gaussian mean with known variance; hierarchical-models workhorse.
• 3.5.3 Gamma-Poisson Model — Poisson rate; canonical count-data prior.
• 3.5.4 Dirichlet-Multinomial Model — categorical probabilities; multivariate extension of Beta-Binomial.

Shared Structure

Every conjugate pair in this subfolder follows the same template. The conjugate prior is chosen so that its density has the same kernel (the $\theta$-dependent part) as the likelihood, viewed as a function of $\theta$. Multiplying prior by likelihood then produces another density in the same family with updated hyperparameters. For the Beta-Binomial, both prior and likelihood are products of ${\theta }^a(1-\theta {)}^b$, so the posterior is $\text{Beta}(\alpha +y,\beta +n-y)$. For the Gamma-Poisson, both are products of ${\lambda }^a{e}^{-b\lambda }$, so the posterior is $\text{Gamma}(\alpha +\sum y_i,\beta +n)$. Every example in this folder has the structure

$\text{posterior hyperparameters}=\text{prior hyperparameters}+\text{data summaries}.$

This is the mechanical reason Bayesian inference under conjugate priors is fast and tractable — the “sum” is a vector of sufficient statistics, and updating the posterior requires only accumulating those statistics as data arrive.

When Conjugacy Fails

Conjugate priors cover an important but narrow set of models. Logistic regression, Poisson regression with non-log link, Gaussian processes with non-Gaussian noise, and almost all hierarchical models with more than two levels have no conjugate prior. The standard toolkit once conjugacy fails:

1. Gibbs sampling with conditionally conjugate updates — the full joint is not conjugate but each full conditional is. See 3.11.2 Gibbs Sampling.
2. Data augmentation — introduce latent variables that restore conjugacy of the conditional updates. See probit regression in the Gibbs note.
3. Laplace approximation — approximate the posterior by a Gaussian around the MAP. See 3.9 Laplace Approximation.
4. Metropolis-Hastings — direct sampling without any conjugacy; applies to arbitrary posteriors.

See Also

• 3. Bayesian Statistics — parent index.
• 3.6 Hierarchical Models — conjugate priors at the hyperprior level give conditionally conjugate hierarchies, the natural Gibbs target.
• 3.7 Marginal Likelihood — conjugate pairs admit closed-form evidence, enabling hyperparameter tuning via empirical Bayes.