Gamma-Poisson Model

Gamma-Poisson Conjugacy

Let $\lambda >0$ be an unknown Poisson rate and let $y_1,\dots ,y_n$ be i.i.d. Poisson counts. The Gamma-Poisson conjugate pair is
$\lambda \sim \text{Gamma}(\alpha ,\beta ),y_i\mid \lambda \stackrel{\text{iid}}{\sim }\text{Poisson}(\lambda ),$
which produces the closed-form posterior
$\lambda \mid y\sim \text{Gamma}\left(\alpha +{\sum }_{i=1}^n{y}_i,\beta +n\right).$
The prior hyperparameters update by adding the total event count $\sum y_i$ to the shape $\alpha$ and the total exposure $n$ to the rate $\beta$. The prior is interpretable as having previously observed $\alpha$ events over $\beta$ units of exposure (shape/rate parametrization: mean $\alpha /\beta$, variance $\alpha /{\beta }^2$).

Exposure-weighted form. When observation $i$ has exposure $t_i$ (time, area, population) and $y_i\mid \lambda \sim \text{Poisson}(\lambda t_i)$, the posterior becomes $\text{Gamma}(\alpha +\sum y_i,\beta +\sum t_i)$. Total events over total exposure.

Posterior summaries:

• Posterior mean: $\mathbb{E}[\lambda \mid y]=\frac{\alpha +\sum y_i}{\beta +n}$ — a weighted average of the prior mean $\alpha /\beta$ and the MLE $\bar{y}$ with weights $\beta /(\beta +n)$ and $n/(\beta +n)$.
• Posterior mode (MAP): $\frac{\alpha +\sum y_i-1}{\beta +n}$ when $\alpha +\sum y_i\ge 1$; equals the MLE $\bar{y}$ under the improper prior $\text{Gamma}(1,0)$.
• Posterior variance: $\frac{\alpha +\sum y_i}{(\beta +n{)}^2}$; shrinks like $O(1/n)$.

Common prior choices:

• $\text{Gamma}(1,0)$ (improper, flat on $\lambda$): posterior is $\text{Gamma}(1+\sum y_i,n)$; MAP equals MLE.
• $\text{Gamma}(0.5,0)$ (improper Jeffreys prior): $p(\lambda )\propto {\lambda }^{-1/2}$ from Fisher information.
• $\text{Gamma}(\alpha ,\beta )$ with small $\alpha ,\beta$: weakly informative; mean $\alpha /\beta$, effective exposure $\beta$.

Posterior predictive for a new count $\tilde{y}$ over unit exposure:
$\tilde{y}\mid y\sim \text{NegBin}\left(\alpha +\sum y_i,\frac{\beta +n}{\beta +n+1}\right),$
a Negative Binomial (also called Gamma-Poisson mixture) with variance strictly larger than the plug-in Poisson prediction. The extra variance is exactly the amount required to account for remaining uncertainty about $\lambda$, and is the Bayesian account of why raw Poisson models overstate confidence.

Prerequisites

• 3.1 Bayesian Framework — prior, likelihood, posterior, Bayes’ rule.
• 3.5 Conjugate Families — the general conjugacy pattern this note instantiates.
• 2.2b.9 Poisson Distribution — the likelihood.
• 2.2c.6 Gamma Distribution — the conjugate prior family.

Intuition

Count data from a Poisson process carries information about its rate only through the sum of counts and the total exposure. The Poisson likelihood, written as a function of $\lambda$, is proportional to ${\lambda }^{\sum y_i}{e}^{-n\lambda }$ — a ${\lambda }^a{e}^{-b\lambda }$ kernel. The Gamma density has exactly that kernel (${\lambda }^{\alpha -1}{e}^{-\beta \lambda }$), so multiplying prior by likelihood adds exponents and stays Gamma. Mechanically, the prior contributes $\alpha$ pseudo-events over $\beta$ pseudo-exposure; the data contribute $\sum y_i$ real events over $n$ real units of exposure; the posterior just accumulates both.

The Negative Binomial posterior predictive reveals why Poisson regression models so often underfit dispersion. A frequentist plug-in predicts $\tilde{y}\sim \text{Poisson}(\hat{\lambda })$, which has variance equal to its mean. The Bayesian account, marginalizing over posterior uncertainty about $\lambda$, predicts $\tilde{y}$ with variance strictly greater than its mean — exactly the overdispersion that a practicing analyst sees in real count data (insurance claims, goals per soccer match, radioactive decay counts observed under rate uncertainty). Negative-Binomial regression is, at bottom, Bayesian Gamma-Poisson with the rate-uncertainty baked in.

Properties

(Posterior Mean as Shrinkage) The posterior mean is a convex combination of the prior mean and the MLE, weighted by effective sample sizes.

$\mathbb{E}[\lambda \mid y]=w\frac{\alpha }{\beta }+(1-w)\bar{y},w=\frac{\beta }{\beta +n}.$
The prior’s effective exposure is $\beta$, the data’s exposure is $n$. When $n\ll \beta$, the posterior sits near the prior mean; when $n\gg \beta$, it sits at the MLE. This is the count-data analog of 3.5.2 Normal-Normal Model’s precision-weighted averaging.

(Posterior Predictive is Negative Binomial, Not Poisson) Predictive distribution inflates variance to account for rate uncertainty.

$\tilde{y}\mid y\sim \text{NegBin}\left(r,p\right),r=\alpha +\sum y_i,p=\frac{\beta +n}{\beta +n+1},$
with mean $r(1-p)/p=(\alpha +\sum y_i)/(\beta +n)=\mathbb{E}[\lambda \mid y]$ and variance $r(1-p)/p^2$, which exceeds the mean. As $n\to \infty$ and the posterior concentrates at $\hat{\lambda }$, the Negative Binomial converges to $\text{Poisson}(\hat{\lambda })$ and variance inflation vanishes.

(Sequential Updating) Gamma-Poisson updates are associative in the $(\alpha ,\beta )$ parametrization.

After seeing $y^{(1)}$ of size $n_1$ and then $y^{(2)}$ of size $n_2$, the posterior is $\text{Gamma}(\alpha +\sum y^{(1)}+\sum y^{(2)},\beta +n_1+n_2)$, identical to processing all $n_1+n_2$ observations at once. The same collapse holds for exposure-weighted counts. See 3.1 Bayesian Framework’s (Sequential Updating) property.

(Bernstein-von Mises) As $n\to \infty$, the posterior converges to the Gaussian around the MLE with variance equal to the inverse Fisher information.

The Gamma posterior $\text{Gamma}({\alpha }_n,{\beta }_n)$ has mean ${\alpha }_n/{\beta }_n\to \bar{y}$ and variance ${\alpha }_n/{\beta }_n^2\to \bar{y}/n$. The Fisher information for a Poisson rate is $I(\lambda )=1/\lambda$, so the inverse-information variance is $\lambda /n\approx \bar{y}/n$, matching the posterior variance. The Gamma shape grows linearly with $n$, so the distribution tightens to Gaussian by the central-limit argument applied inside the Gamma family.

(Jeffreys Prior is Gamma$(1/2,0)$) The improper prior $p(\lambda )\propto {\lambda }^{-1/2}$ is the Jeffreys (Fisher-information-induced) prior.

For a Poisson likelihood, $I(\lambda )=1/\lambda$, so the Jeffreys prior $p(\lambda )\propto \sqrt{I(\lambda )}={\lambda }^{-1/2}$ is improper Gamma with $\alpha =1/2,\beta =0$. The posterior $\text{Gamma}(1/2+\sum y_i,n)$ is proper whenever at least one observation exceeds zero. This prior is reparametrization-invariant: the Jeffreys prior for $\lambda$ yields the same inference as the Jeffreys prior for $\sqrt{\lambda }$.

(Exposure Scales Linearly) When observation $i$ has known exposure $t_i$ and $y_i\mid \lambda \sim \text{Poisson}(\lambda t_i)$, posteriors accumulate both events and exposure.

$\lambda \mid y\sim \text{Gamma}\left(\alpha +{\sum }_{i=1}^n{y}_i,\beta +{\sum }_{i=1}^n{t}_i\right).$
This is the workhorse form in epidemiology (incidence rates per person-year), quality control (defects per item-hour), and particle-physics counting experiments (events per unit luminosity). The posterior mean is “total events over total exposure,” with priors as pseudo-events over pseudo-exposure.

(Sum of Counts is Sufficient) Only $\sum y_i$ and $n$ (or $\sum t_i$) matter for inference.

By the factorization criterion, $T(y)=(\sum y_i,n)$ is a sufficient statistic for $\lambda$. Two datasets with the same sum and same exposure yield identical posteriors regardless of the individual counts. This is the Poisson-family version of the sufficiency that made the Beta-Binomial collapse to $(y,n)$.

Examples

Example 1: Storefront Traffic Rate

A retailer wants to estimate the hourly customer arrival rate $\lambda$ at a new store. A prior elicited from an experienced operations manager is $\lambda \sim \text{Gamma}(10,2)$, which has prior mean $10/2=5$ customers per hour and prior standard deviation $\sqrt{10}/2\approx 1.58$. Over one full day (8 hours), the store logs $y=48$ arrivals.

Posterior: $\lambda \mid y\sim \text{Gamma}(10+48,2+8)=\text{Gamma}(58,10)$.
Posterior mean: $58/10=5.8$ customers/hour.
Posterior std: $\sqrt{58}/10\approx 0.762$.
95% equal-tail credible interval (Gamma quantiles): $[4.40,7.42]$.

The prior’s effective exposure is $\beta =2$ hours. After 8 hours of data, the posterior mean has shifted from the prior mean of 5.0 toward the observed rate of 48/8 = 6.0, landing at 5.8.

Posterior predictive for next hour’s count: $\tilde{y}\mid y\sim \text{NegBin}(58,10/11)$, with mean $58/10=5.8$ and variance $58\cdot 1/11\cdot 11/10=5.8\cdot 1.1=6.38$ — larger than $5.8$ by exactly the factor $1+1/({\beta }_n)=11/10$.

Example 2: Defects per Unit (Exposure-Weighted)

A manufacturing line records defects per batch, where batches differ in size. Batches have sizes $t_1,\dots ,t_5=(100,120,95,140,110)$ units and observed defect counts $y_1,\dots ,y_5=(3,4,2,5,2)$. Using a weakly informative prior $\text{Gamma}(1,50)$ (prior mean $0.02$ defects per unit with low confidence):

Total events: $\sum y_i=16$.
Total exposure: $\sum t_i=565$.
Posterior: $\lambda \mid y\sim \text{Gamma}(1+16,50+565)=\text{Gamma}(17,615)$.
Posterior mean: $17/615\approx 0.0276$ defects per unit.
MLE by comparison: $16/565\approx 0.0283$.

The prior’s effective exposure of 50 units nudges the posterior mean a hair toward the prior mean of 0.02, but the 565 units of real data dominate. A 95% credible interval on $\lambda$ is approximately $[0.0161,0.0421]$.

Example 3: Conjugate Prior as Pseudo-Data

A Bayesian analyst asserts prior $\text{Gamma}(\alpha ,\beta )$ without justification. Challenged to explain what this prior “means,” the analyst re-expresses it as: “Before seeing data, I behave as though I had previously seen $\alpha$ events in $\beta$ units of exposure.”

Verification: If an improper flat prior $\text{Gamma}(0,0)$ were updated with imaginary data of $\alpha$ events and $\beta$ exposure, the posterior would be $\text{Gamma}(0+\alpha ,0+\beta )=\text{Gamma}(\alpha ,\beta )$ — precisely the stated prior.

Hyperparameter elicitation consequence: if an expert is willing to say “my prior rate is around 3 events per hour, and I am about as confident as if I had watched 5 hours of data,” then $\alpha /\beta =3$ and $\beta =5$, giving $\alpha =15$ and $\beta =5$. This pseudo-data framing is often easier to elicit than a raw $(\alpha ,\beta )$ pair.

Derivation

Goal: derive the $\text{Gamma}(\alpha +\sum y_i,\beta +n)$ posterior.

The Gamma prior density is
$p(\lambda )=\frac{{\beta }^{\alpha }}{\Gamma (\alpha )}{\lambda }^{\alpha -1}{e}^{-\beta \lambda }\propto {\lambda }^{\alpha -1}{e}^{-\beta \lambda }.$
The Poisson likelihood for $n$ i.i.d. observations is
$p(y\mid \lambda )={\prod }_{i=1}^n\frac{e^{-\lambda }{\lambda }^{y_i}}{y_i!}\propto {\lambda }^{\sum y_i}{e}^{-n\lambda }.$
Multiplying prior and likelihood:
$p(\lambda \mid y)\propto {\lambda }^{\alpha -1}{e}^{-\beta \lambda }\cdot {\lambda }^{\sum y_i}{e}^{-n\lambda }={\lambda }^{(\alpha +\sum y_i)-1}{e}^{-(\beta +n)\lambda },$
the kernel of $\text{Gamma}(\alpha +\sum y_i,\beta +n)$. The normalizing constant follows automatically. For exposure-weighted observations, the likelihood becomes ${\prod }_i(\lambda t_i{)}^{y_i}{e}^{-\lambda t_i}/y_i!\propto {\lambda }^{\sum y_i}{e}^{-\lambda \sum t_i}$, and the posterior hyperparameters become $(\alpha +\sum y_i,\beta +\sum t_i)$.

Posterior predictive: for new $\tilde{y}$ with unit exposure,
$p(\tilde{y}\mid y)={\int }_0^{\infty }\frac{e^{-\lambda }{\lambda }^{\tilde{y}}}{\tilde{y}!}p(\lambda \mid y)d\lambda .$
With $p(\lambda \mid y)=\text{Gamma}({\alpha }_n,{\beta }_n)$, the integral is a Gamma-Poisson mixture, closed-form as $\text{NegBin}({\alpha }_n,{\beta }_n/({\beta }_n+1))$ in the standard stopping-time parametrization.

See Also

• 3.5 Conjugate Families — parent index.
• 3.5.1 Beta-Binomial Model — canonical conjugate-pair template.
• 3.5.2 Normal-Normal Model — continuous analog with precision addition.
• 3.5.4 Dirichlet-Multinomial Model — categorical extension of Beta-Binomial.
• 3.6 Hierarchical Models — Gamma-Poisson hierarchies are the natural model for count data across units with varying exposure (e.g., disease incidence across counties).
• 2.2b.9 Poisson Distribution — the likelihood.
• 2.2c.6 Gamma Distribution — the prior.