Conditional probability

A conditional probability is expressed as $P(E\vert F)$ for any two events $E,F \in {\cal F}$ and is called the ``probability of $E$, given $F$.'' Its definition is

$$P(E\vert F) = { P(E \cap F) \over P(F) }.$$ (9.4)

Two events, $E$ and $F$, are called independent if and only if $P(E \cap F) = P(E) P(F)$; otherwise, they are called dependent. An important and sometimes misleading concept is conditional independence. Consider some third event, $G \in {\cal F}$. It might be the case that $E$ and $F$ are dependent, but when $G$ is given, they become independent. Thus, $P(E \cap F) \not = P(E) P(F)$; however, $P(E \cap F \vert G) = P(E\vert G) P(F\vert G)$. Such examples occur frequently in practice. For example, $E$ might indicate someone's height, and $F$ is their reading level. These will generally be dependent events because children are generally shorter and have a lower reading level. If we are given the person's age as an event $G$, then height is no longer important. It seems intuitive that there should be no correlation between height and reading level once the age is given.

The definition of conditional probability, (9.4), imposes the constraint that

$$P(E \cap F) = P(F) P(E\vert F) = P(E) P(F\vert E) ,$$ (9.5)

which nicely relates $P(E\vert F)$ to $P(F\vert E)$. This results in Bayes' rule, which is a convenient way to swap $E$ and $F$:

$$P(F\vert E) = {P(E\vert F)P(F) \over P(E)} .$$ (9.6)

The probability distribution, $P(F)$, is referred to as the prior, and $P(F\vert E)$ is the posterior. These terms indicate that the probabilities come before and after $E$ is considered, respectively.

If all probabilities are conditioned on some event, $G \in {\cal F}$, then conditional Bayes' rule arises, which only differs from (9.6) by placing the condition $G$ on all probabilities:

$$P(F\vert E,G) = {P(E\vert F,G)P(F\vert G) \over P(E\vert G)} .$$ (9.7)