1.3 Conditional Probability

Conditional Probability

For two events $A,B\subseteq \Omega$ in the same probability space, the law of conditional probability states that the probability of $B$ given that $A$ has happened is
$\mathbb{P}[B\mid A]=\frac{\mathbb{P}[A\cap B]}{\mathbb{P}[A]}.$
This implies that the probability of $A$ is nonzero ($\mathbb{P}[A]>0$).

Non-Associativity $\mathbb{P}[B\mid A]\ne \mathbb{P}[A\mid B]$, unless $\mathbb{P}[A]=\mathbb{P}[B]$.

Conditional probabilities are non-associative. This means that generally,

Intuition

The joint probability $\mathbb{P}(A\cap B)=\mathbb{P}(B\cap A)$ is the probability that both $A$ and $B$ happen. It is restricted to the sample points that occur in both events. In a Venn diagram, it can be represented as the overlap between two circles, $A$ and $B$:

\begin{document}
\begin{tikzpicture}
  % Define circles for A and B
  \draw[thick] (0,0) circle (2cm) node[left] {\(A\)};
  \draw[thick] (2.5,0) circle (2cm) node[right] {\(B\)};
 
  % Label A ∩ B
  \begin{scope}
    \clip (0,0) circle (2cm);
    \fill[gray, opacity=0.15] (2.5,0) circle (2cm);
  \end{scope}
  \node at (1.25,0) {\(A \cap B\)};
 
  % Label sample space
  \draw[thick] (-3,-2.5) rectangle (6,2.5) node[below left] {\(\Omega\)};
\end{tikzpicture}
 
\end{document}

By dividing the joint probability by the probability of the event that has already happened we are “zooming in” on the world where $A$ happened and renormalizing so that those remaining outcomes sum to 1.  In effect, we:

1. Throw out all outcomes where A didn’t occur.
2. Rescale the probabilities of the leftover outcomes (i.e., $A\cap B$, $A\cap \bar{B}$) by $1/\mathbb{P}(A)$ so they total 1.
   Essentially, $A$ becomes our new sample space.

Examples

Boys and Girls

A company is holding a dinner for working mothers with at least one son.

Jackson, a mother with two children, is invited. What is the probability that both her children are boys? $\Omega =\{(b,b),(b,g),(g,b),(g,g)\}$. Let $B_i$ be the probability that her $i$-th child is a boy. We need to find $\mathbb{P}(B_1\cap B_2\mid B_1\cup B_2)$. We have: $\mathbb{P}(B_1\cap B_2\mid B_1\cup B_2)=\frac{\mathbb{P}([B_1\cap B_2]\cap [B_1\cup B_2])}{\mathbb{P}(B_1\cup B_2)}=\frac{\mathbb{P}(B_1\cap B_2)}{\mathbb{P}(B_1\cup B_2)}=\frac{1/4}{1/3}=\frac{1}{3},$ since $B_1\cap B_2=\{(b,b)\}$ and $B_1\cup B_2=\{(b,b),(b,g),(g,b)\}$.

We have the following sample space:

A few days before the party, you are taking a walk and see your new colleague, Ms. Parker, who is known to have two children.

If you see her walking with one of her children, and that child is a boy, what is the probability that both children are boys? $B_1$. Then, we are solving for $\mathbb{P}(B_1\cap B_2\mid B_1)=\mathbb{P}(B_2\mid B_1)=\mathbb{P}(B_2)=\frac{1}{2},$ since whether the observed child is a boy is independent of her other child being a boy.

Note the slight distinction in the two questions. Here, we know that one of her children is a boy. Label the observed child as

Scratched CDs

You just bought 3 used movie CDs from Blockbuster! Unfortunately for you, one of the CDs is scratched on one side, while a second one is scratched on both sides. In a fit of rage, you shove the CDs back in the bag and shuffle it.

Later in the day, you take out one of the CDs, and observe that it has a scratched side. What is the probability that the other side of the CD will be scratched too? $\Omega =\{(S,S),(S,C)\}$, and therefore the probability is $3/6=1/2$, since there are 3 clean sides and 3 scratched sides.

Here, it is easy to make a mistake here and say that the sample space is

However, we are recording each of the 6 sides here, rather than recording each CD. Therefore, the conditional sample space is $\Omega =\{S{S}_1,S{S}_2,S{C}_S\}$, (i.e., the two sides of the double-scratched CD and the scratched side of the CD with a single scratched side). In $2/3$ of cases, the CD with both sides scratched is selected.

As a sanity check, consider $S_i$ being an indicator of whether the $i$-th side is scratched. Then,

\mathbb{P}(S_{2} = 1 \mid S_{1} = 1) &= \frac{\mathbb{P}(S_{1} = 1 \cap S_{2} = 1)}{\mathbb{P}(S_{1}=1)} \
&= \frac{1/3}{1/2} \
&= 2/3.
\end{align}$$

Monty Hall Problem

You are a contestant on a game show and can choose to open one of 3 doors, two of them containing a goat, and one of them containing a brand new car. You choose one of the doors, and Monty Hall opens another one, knowing that he’s opening a door with a goat behind it.

Should you switch, or should you keep your original choice? $A,B,C$ be the events that the car is behind door A, B, or C. Then $\mathbb{P}[A]=\mathbb{P}[B]=\mathbb{P}[C]=\frac{1}{3}$. Let's say you pick door A. This means that $\mathbb{P}[B\cup C]=\frac{2}{3}$. If Monty opens door C, then $\mathbb{P}[C]=0$, and so $\mathbb{P}[B]=\frac{2}{3}$. In most cases, it is therefore better to switch.

Let

What is the sample space here? Consider a triple $(A,B,C)$, which represents:

1. A: The location of the prize.
2. B: The initial door opened by the contestant.
3. C: The door opened by Monty.

Note that some triples are not possible, e.g. $(A,B,A)$, since Monty cannot open the door that contains the car or the door that the contestant has picked. It turns out that there are 12 different permutations of triples that can be made. These include:

$$\begin{array}{r}(A,B,C)\\ (A,C,B)\\ (B,C,A)\\ (B,A,C)\\ (C,B,A)\\ (C,A,B)\\ (A,A,B)\\ (A,A,C)\\ (B,B,A)\\ (B,B,C)\\ (C,C,A)\\ (C,C,B)\end{array}$$

We note that there are only two events that could happen: $E_1=$ Monty has one door to pick (as in the first 6 triples), and $E_2=$ Monty has two doors to pick (as in the last 6 triples). Then:

$$\begin{array}{r}\mathbb{P}[E_1]=\frac{1}{3}\times \frac{1}{3}\times 1=\frac{1}{9}\\ \mathbb{P}[E_2]=\frac{1}{3}\times \frac{1}{3}\times \frac{1}{2}=\frac{1}{18}\end{array}$$

Note that in the first equation we multiply by 1 because Monty has 1 door to choose, and in the second equation we multiply by $\frac{1}{2}$ because he can choose between 2 doors.

Let’s suppose the contestant wants to switch doors. Now, let $W$ be the event that the contestant wins. Which sample points $(A,B,C)$ are in W? Since we are interested in the event that the contestant has to win, the only way this could happen is if they didn’t initially pick the prize door (that is, the first 6 triples). Therefore, the contestant has a $\mathbb{P}[W]=6\times \frac{1}{9}=\frac{2}{3}$ chance of winning.