2.2c.6 Gamma Distribution

Gamma Distribution

A continuous random variable $X\in (0,\infty )$ follows the Gamma distribution $X\sim \mathrm{Gamma}(\alpha ,\beta )$ with shape $\alpha >0$ and rate $\beta >0$ (equivalently, scale $\theta =1/\beta$), defined for all $x>0$ by the probability density function
$f_X(x)=\frac{{\beta }^{\alpha }}{\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-\beta x},$
where $\Gamma (\alpha )$ is the Gamma function, defined by the integral
$\Gamma (\alpha )={\int }_0^{\infty }{t}^{\alpha -1}{e}^{-t}dt,\alpha >0.$
The Gamma function satisfies $\Gamma (\alpha +1)=\alpha \Gamma (\alpha )$, and for positive integers $\Gamma (n)=(n-1)!$, making it a continuous extension of the factorial.

The Gamma distribution models nonnegative, right-skewed quantities such as waiting times and accumulated amounts. A key interpretation: if events arrive as a Poisson process with rate $\lambda$, then the waiting time to the $\alpha$-th event is $\mathrm{Gamma}(\alpha ,\lambda )$ (for integer $\alpha$).

Special cases:

• If $\alpha =1$, then $X\sim \mathrm{Exp}(\beta )$ (exponential).
• If $\alpha =k/2$ and $\beta =1/2$, then $X\sim {\mathcal{X}}_k^2$ (chi-square).

The cumulative distribution function of the Gamma distribution is
$F_X(x)=\frac{\gamma (\alpha ,\beta x)}{\Gamma (\alpha )},$
where $\gamma (\alpha ,z)$ is the lower incomplete gamma function.

A key characteristic of the Gamma family is shape control: $\alpha$ governs skewness/peakedness and $\beta$ sets the time/scale. For $\alpha >1$, the density has a mode at $(\alpha -1)/\beta$; for $0<\alpha \le 1$, it is decreasing from $+\infty$ at $0$.

Rate vs. Scale Parameterization scale $\theta >0$ instead of rate $\beta$. Then, $X\sim \mathrm{Gamma}(\alpha ,\theta )⟺f_X(x)=\frac{1}{\Gamma (\alpha ){\theta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\theta },$ with $\theta =1/\beta$. Check which convention a class/text uses.

Some sources use

Intuition

The Gamma distribution generalizes the Exponential from “time until the first event” to “time until the $\alpha$-th event” in a Poisson process. If you are waiting for a single event at rate $\lambda$, the wait is $\text{Exp}(\lambda )$. If you need $\alpha$ events to occur, the total wait is $\text{Gamma}(\alpha ,\lambda )$—the sum of $\alpha$ independent exponential waiting times.

Beyond waiting times, the Gamma is a flexible family for modeling any nonnegative, right-skewed quantity. Small $\alpha$ produces a sharply peaked density near $0$ with a long tail; large $\alpha$ yields a bell-shaped density shifted away from $0$ that increasingly resembles a Gaussian (by the central limit theorem, since the Gamma with integer $\alpha$ is a sum of $\alpha$ i.i.d. exponentials).

Properties

(Mean) $\mathbb{E}[X\sim \mathrm{Gamma}(\alpha ,\beta )]=\frac{\alpha }{\beta }$.

(Variance) $\mathrm{Var}[X\sim \mathrm{Gamma}(\alpha ,\beta )]=\frac{\alpha }{{\beta }^2}$.

(Mode) If $\alpha >1$, $\mathrm{mode}(X)=\frac{\alpha -1}{\beta }$. (If $0<\alpha \le 1$, the mode is at $0$.)

(MGF) For $t<\beta$,

$M_X(t)=\mathbb{E}[e^{tX}]={\left(\frac{\beta }{\beta -t}\right)}^{\alpha }.$

(Sum) If $X_i\stackrel{\text{i.i.d.}}{\sim }\mathrm{Gamma}({\alpha }_i,\beta )$ are independent with a common rate $\beta$, then

${\sum }_{i=1}^n{X}_i\sim \mathrm{Gamma}\left({\sum }_{i=1}^n{\alpha }_i,\beta \right).$
This follows directly from the MGF: the product of individual MGFs yields a Gamma MGF with summed shape parameters.

(Scaling) If $X\sim \mathrm{Gamma}(\alpha ,\beta )$ and $c>0$, then $cX\sim \mathrm{Gamma}\left(\alpha ,\beta /c\right)$ in rate form. Equivalently, in scale form: $cX\sim \mathrm{Gamma}(\alpha ,c\theta )$.

(Memorylessness?) Only the $\alpha =1$ case (exponential) is memoryless; for $\alpha \ne 1$, the Gamma is not memoryless.

(Poisson Process Waiting Time) If $N(t)$ is Poisson with rate $\lambda$ and $T_k$ is the waiting time to the $k$-th arrival, then $T_k\sim \mathrm{Gamma}(k,\lambda )$.

(Conjugacy with Poisson) If $X_1,\dots ,X_n\mid \lambda \stackrel{\text{i.i.d.}}{\sim }\mathrm{Poisson}(\lambda )$ and $\lambda \sim \mathrm{Gamma}(\alpha ,\beta )$ (rate form), then

$\lambda \mid X_{1:n}\sim \mathrm{Gamma}\left(\alpha +{\sum }_{i=1}^n{X}_i,\beta +n\right).$
This makes the Gamma the conjugate prior for the Poisson likelihood, analogous to how the Beta is conjugate to the Bernoulli.

(Beta Connection) If $Y_1\sim \mathrm{Gamma}(\alpha ,1)$ and $Y_2\sim \mathrm{Gamma}(\beta ,1)$ are independent, then $\frac{Y_1}{Y_1+Y_2}\sim \text{Beta}(\alpha ,\beta )$ and $Y_1+Y_2\sim \mathrm{Gamma}(\alpha +\beta ,1)$, with the ratio and the sum being independent.

Examples

Erlang Waiting Time

Customers arrive at a service desk according to a Poisson process with rate $\lambda =3$ per hour. The manager wants to know the distribution of the time until the $5$th customer arrives.

What is the distribution, expected value, and standard deviation of the waiting time? $\lambda =3$, the waiting time to the $5$th arrival is $T_5\sim \mathrm{Gamma}(5,3)$. The expected waiting time is $\mathbb{E}[T_5]=\frac{\alpha }{\beta }=\frac{5}{3}\approx 1.67\text{ hours},$ and the variance is $\mathrm{Var}(T_5)=\frac{\alpha }{{\beta }^2}=\frac{5}{9}\approx 0.556,$ giving a standard deviation of $\sqrt{5/9}\approx 0.745$ hours. Since $\alpha =5$ is an integer, this is also an Erlang distribution: the sum of $5$ independent $\mathrm{Exp}(3)$ waiting times.

Since the arrival process is Poisson with rate

Bayesian Estimation of a Poisson Rate

A scientist counts radioactive decays and observes $x_1=4,x_2=7,x_3=5$ decays in three one-second intervals. She assumes $X_i\mid \lambda \stackrel{\text{i.i.d.}}{\sim }\mathrm{Poisson}(\lambda )$ with prior $\lambda \sim \mathrm{Gamma}(2,1)$.

What is the posterior distribution of $\lambda$, and what is the Bayes estimator under squared error loss? By the conjugacy property, the posterior is $\lambda \mid X_{1:3}\sim \mathrm{Gamma}\left(2+(4+7+5),1+3\right)=\mathrm{Gamma}(18,4).$ The Bayes estimator under squared error loss is the posterior mean: ${\hat{\lambda }}_{\text{Bayes}}=\mathbb{E}[\lambda \mid X_{1:3}]=\frac{18}{4}=\mathrm{4.5.}$ For comparison, the MLE is $\bar{X}=\frac{4+7+5}{3}\approx 5.33$. The posterior mean is shrunk toward the prior mean of $2/1=2$, reflecting the influence of the prior. The posterior variance is $18/16=1.125$, so a $95\%$ credible interval can be obtained from the Gamma quantiles.