Probability space

A probability space is a three-tuple, $(S,{\cal F},P)$, in which the three components are

1. Sample space: A nonempty set $S$ called the sample space, which represents all possible outcomes.
2. Event space: A collection ${\cal F}$ of subsets of $S$, called the event space. If $S$ is discrete, then usually ${\cal F} = {\rm pow}(S)$. If $S$ is continuous, then ${\cal F}$ is usually a sigma-algebra on $S$, as defined in Section 5.1.3.
3. Probability function: A function, $P: {\cal F}\rightarrow {\mathbb{R}}$, that assigns probabilities to the events in ${\cal F}$. This will sometimes be referred to as a probability distribution over $S$.

The probability function, $P$, must satisfy several basic axioms:

1. $P(E) \geq 0$ for all $E \in {\cal F}$.
2. $P(S) = 1$.
3. $P(E \cup F) = P(E) + P(F)$ if $E \cap F = \emptyset$, for all $E,F \in {\cal F}$.

If $S$ is discrete, then the definition of $P$ over all of ${\cal F}$ can be inferred from its definition on single elements of $S$ by using the axioms. It is common in this case to write $P(s)$ for some $s \in S$, which is slightly abusive because $s$ is not an event. It technically should be $P(\{s\})$ for some $\{s\} \in {\cal F}$.

Example 9..4 (Tossing a Die)   Consider tossing a six-sided cube or die that has numbers $1$ to $6$ painted on its sides. When the die comes to rest, it will always show one number. In this case, $S = \{1,2,3,4,5,6\}$ is the sample space. The event space is ${\rm pow}(S)$, which is all $2^6$ subsets of $S$. Suppose that the probability function is assigned to indicate that all numbers are equally likely. For any individual $s \in S$, $P(\{s\}) = 1/6$. The events include all subsets so that any probability statement can be formulated. For example, what is the probability that an even number is obtained? The event $E = \{2,4,6\}$ has probability $P(E) = 1/2$ of occurring. $\blacksquare$

The third probability axiom looks similar to the last axiom in the definition of a measure space in Section 5.1.3. In fact, $P$ is technically a special kind of measure space as mentioned in Example 5.12. If $S$ is continuous, however, this measure cannot be captured by defining probabilities over the singleton sets. The probabilities of singleton sets are usually zero. Instead, a probability density function, $p: S \rightarrow {\mathbb{R}}$, is used to define the probability measure. The probability function, $P$, for any event $E \in {\cal F}$ can then be determined via integration:

$$P(E) = \int_E p(x) dx ,$$ (9.3)

in which $x \in E$ is the variable of integration. Intuitively, $P$ indicates the total probability mass that accumulates over $E$.