2.2b.5 Geometric Distribution

Geometric Distribution

A discrete random variable $X\ge 1$ follows the Geometric distribution $X\sim \text{Geom}(p)$ if it represents the number of trials required to obtain the first success in a sequence of independent Bernoulli trials. Formally, the Geometric distribution of a random variable is given by the PMF,
$p_X(x)=(1-p{)}^{x-1}p,$
where each trial has a probability $p$ of success.

The geometric distribution is the only discrete distribution that is memoryless, meaning that the probability of needing more than $m$ additional trials to get the first success is independent of the $k$ trials that have already been conducted. Formally,
$\mathbb{P}[X>k+m\mid X>k]=\mathbb{P}[X>m],$
showing that whether or not you start fresh or you have already had $k$ failures, the chance of needing more than $m$ additional trials remains the same.

Intuition

I honestly cannot help you if you don’t understand the PMF for the geometric distribution. There is a $p^1$ factor, which indicates that there is only a single success. It is self-descriptive.

However, when I first learned about the memoryless property (i.e. right now), I was confused. So let’s get through it together. We have $k$ as the number of initial (failed) trials, and $m$ as the sequential number of trials to get a single success. Then,

\mathbb{P}[X > k+m \mid X > k] &= \frac{\mathbb{P}[X > k+m \cap X > k]}{\mathbb{P}[X > k]} \\ &= \frac{\mathbb{P}[X > k+m]}{\mathbb{P}[X > k]} \\ &= \frac{(1-p)^{k+m}}{(1-p)^k} \\ &= (1-p)^m \\ &= \mathbb{P}[X > m]. \end{align}$$ where the tail probability of the Geometric random variable is given by $\mathbb{P}[X > x]=(1-p)^x$. # Properties #### (*Mean*) $\mathbb{E}[X \sim \text{Geom}(p)]=\frac{1}{p}$. We start by setting up our equation for solving the expectation using the [[2.6b.2 Tail Sum Formula|tail sum formula]] by saying that

\begin{equation}
\begin{aligned}
\mathbb{P}[X \geq i] &= \sum^{\infty}{j=i}\mathbb{P}[X=j] \
&=\sum^{\infty}{j=i}p(1-p)^{j-1} \
&=\frac{p}{1-p}\sum^{\infty}_{j=i}(1-p)^{j} \
\end{aligned}
\end{equation}

Using the definition of the infinite geometric sum $S_n=\sum^{\infty}_{i=0}a_ir^i=\frac{a_1}{1-r}$ with $a_1=(1-p)^i$ and $r=1-p$ (a is not $1$ since geometric series start at $j=1$, and it is not $1-p$ since $j=i$, we are led to,

\begin{equation}
\begin{aligned}
\mathbb{P}[X \geq i] &= \frac{p}{1-p}\sum^{\infty}_{j=i}(1-p)^{j} \
&= \frac{p}{1-p}\frac{(1-p)^i}{1-(1-p)} \
&= (1-p)^{i-1}
\end{aligned}
\end{equation}

This means that for large $n$, the probability that we need $i$ trials or more before the first success is determined by $\mathbb{P}[X \geq i] = (1-p)^{i-1}$. Then the probability that we will get a success in less than $i$ trials is $\mathbb{P}[X < i] = 1 - \mathbb{P}[X \geq i]$. Let's put this in perspective using a biased coin with a probability of heads $p=\frac{1}{3}$. The probability that we need 1 toss or more before the first success is 1. That we will need 2 or more is $1-p=\frac{2}{3}$. The probability that we will need 3 or more tosses before a success is $(1-p)^2=\frac{4}{9}$. The probability that we will get a successful trial before 3 tosses (so in the first 2 tosses) is $1-\frac{4}{9}=\frac{5}{9}$. Looking at the outcome space for 3 trials or success, we can see why: $$\Upomega=\set{TTT, TTH, TH, H}$$ Here, $\mathbb{P}[X=3]=\frac{1}{3}(\frac{2}{3})^2=\frac{4}{27}$. This is confirmed by $\mathbb{P}[X=3]=\mathbb{P}[TTH]$. Therefore, $\mathbb{P}[X < 3]=\mathbb{P}[TH]+\mathbb{P}[H]=\frac{1}{3}\frac{2}{3}+\frac{1}{3}=\frac{5}{9}$, which is equal to $1 - (1-p)^2=\frac{5}{9}$. We now calculate the expectation of the geometric distribution using the [[2.6b.2 Tail Sum Formula|tail sum formula]] and the previously introduced infinite geometric sum as follows:

\begin{equation}
\begin{aligned}
\mathbb{E}[X] &= \sum^{\infty}{i=1}\mathbb{P}[X \geq i] \
&= \sum^{\infty}{i=1}(1-p)^{i-1} \
&= \frac{(1-p)^{1-1}}{1-(1-p)} \
&= \frac{1}{p}
\end{aligned}
\end{equation}

Therefore, $\mathbb{E}[X]=\frac{1}{p}$ for a geometric distribution. #### (*Variance*) $\text{Var}[X \sim \text{Geom}(x)]=\frac{1-p}{p^2}$ We already know that $\mathbb{E}[X]=\frac{1}{p}$. Therefore, using $Var[X]=\mathbb{E}[X^2]-E[X]^2$, we only need to find $E[X^2]$. We proceed as follows:

\begin{equation}
\begin{aligned}
\mathbb{E}[X^2] &= \mathbb{E}[X(X-1)]+\mathbb{E}[X]-\mathbb{E}[X]^2
\end{aligned}
\end{equation}

# Examples >[!example] Class Enrollment > >[!example] Laundry Time > >To avoid doing laundry too often, Marcus creates a system. Each night, he labels one of his shirts as the "dirtiest." In the morning, he randomly picks one of his clean shirts to wear. If he happens to wear the dirtiest one, he throws it in the dirty pile at the end of the day (shirts in the dirty pile aren’t reused). Once all his shirts end up dirty, Marcus does laundry. That night, he again designates one shirt as the dirtiest, and the cycle begins anew. > >>[!example]- If Marcus has $n$ shirts, what is the expected number of days that transpire between laundry events? >>Observe that the cycle can be broken into $n$ phases. During phase $i$: >>1. Marcus labels one of the remaining $n-i+1$ shirts as the dirtiest. >>2. In the morning, he picks one of those $n-i+1$ shirts. >>3. >>The number of days it takes for Marcus to throw each shirt in the dirty pile can be expressed as the sum $D=\sum^{n}_{i=1}S_{i}$, where $S_{i} \sim \text{Geom}\left( \dfrac{1}{n-i+1} \right)$. This probability works solely because once Marcus labels a shirt as the "dirtiest," no other shirt can claim this position before the "dirtiest" is thrown in the dirty pile.