Stochastic Processes and Financial Mathematics
(part one)

2.3 Infinite $\mathrm{\Omega }$

So far we focused on finite sample spaces of the form $\mathrm{\Omega }=\{x_1,x_2,\dots x_n\}$. In such a case we would normally take $\mathcal{F}=\mathcal{P}(\mathrm{\Omega })$, which is also a finite set. Since $\mathcal{F}$ contains every subset of $\mathrm{\Omega }$, any $\sigma$-field on $\mathrm{\Omega }$ is a sub-$\sigma$-field of $\mathcal{F}$. We have seen how it is possible to construct other $\sigma$-fields on $\mathrm{\Omega }$ too.

In this case we can define a probability measure on $\mathrm{\Omega }$ by choosing a finite sequence $a_1,a_2,\dots ,a_n$ such that each $a_i\in [0,1]$ and $\sum _1^n{a}_i=1$. We set $\mathbb{P}[x_i]=a_i$. This naturally extends to defining $\mathbb{P}[A]$ for any subset $A\subseteq \mathrm{\Omega }$, by setting

$$\begin{array}{}\text{(2.3)}& \mathbb{P}[A]=\sum _{\{i;x_i\in A\}}\mathbb{P}[x_i]=\sum _{\{i;x_i\in A\}}{a}_i.\end{array}$$

It is hopefully obvious (and tedious to check) that, with this definition, $\mathbb{P}$ is a probability measure. Consequently $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ is a probability space.

All our experiments with (finitely many tosses/throws of) dice and coins fit into this category of examples. In fact, if our experiment has countably many outcomes, say, $\mathrm{\Omega }=\{x_1,x_2,\dots \}$ we can still work in much the same way, and the sum in (2.3) will become an infinite series that sums to $1$.

However, the theory of stochastic processes, as well as most sophisticated examples of stochastic models, require an uncountable sample space. In such cases, we can’t use (2.3), because there is no such thing as an uncountable sum.

An example with uncountable $\mathrm{\Omega }$

We now flex our muscles a bit, and look at an example where $\mathrm{\Omega }$ is uncountable. We toss a coin infinitely many times, then $\mathrm{\Omega }=\{H,T{\}}^{\mathbb{N}}$, meaning that we write an outcome as a sequence $\omega ={\omega }_1,{\omega }_2,\dots$ where ${\omega }_i\in \{H,T\}$. The set $\mathrm{\Omega }$ is uncountable.

We define the random variables $X_n(\omega )={\omega }_n$, so as $X_n$ represents the result ($H$ or $T$) of the $n^{th}$ throw. We take

$$\mathcal{F}=\sigma (X_1,X_2,\dots )$$

i.e. $\mathcal{F}$ is the smallest $\sigma$-field with respect to which all the $X_n$ are random variables. Then

$$\begin{array}{rcl}\sigma (X_1)& =& \{\mathrm{\varnothing },\{H\star \star \star \dots \},\{T\star \star \star \dots \},\mathrm{\Omega }\}\\ \sigma (X_1,X_2)& =& \sigma (\{HH\star \star \dots \},\{TH\star \star \dots \},\{HT\star \star \dots \},\{TT\star \star \dots \})\\ & =& \{\mathrm{\varnothing },\{HH\star \star \dots \},\{TH\star \star \dots \},\{HT\star \star \dots \},\{TT\star \star \dots \},\\ & & \{H\star \star \star \dots \},\{T\star \star \star \dots \},\{\star H\star \star \dots \},\{\star T\star \star \dots \},\left\{\begin{array}{c}HH\star \star \dots \\ TT\star \star \dots \end{array}\right\},\left\{\begin{array}{c}HT\star \star \dots \\ TH\star \star \dots \end{array}\right\},\\ & & \{HH\star \star \dots {\}}^c,\{TH\star \star \dots {\}}^c,\{HT\star \star \dots {\}}^c,\{TT\star \star \dots {\}}^c,\mathrm{\Omega }\},\end{array}$$

where $\star$ means that it can take on either $H$ or $T$, so $\{H\star \star \star \dots \}=\{\omega :{\omega }_1=H\}$.

With the information available to us in $\sigma (X_1,X_2)$, we can distinguish between $\omega$’s where the first or second outcomes are different. But if two $\omega$’s have the same first and second outcomes, they fall into exactly the same subset(s) of $\sigma (X_1,X_2)$. Consequently, if a random variable depends on anything more than the first and second outcomes, it will not be $\sigma (X_1,X_2)$ measurable.

It is not immediately clear if we can define a probability measure on $\mathcal{F}$! Since $\mathrm{\Omega }$ is uncountable, we cannot use the idea of (2.3) and define $\mathbb{P}$ in terms of $\mathbb{P}[\omega ]$ for each individual $\omega \in \mathrm{\Omega }$. Equation (2.3) simply would not make sense; there is no such thing as an uncountable sum.

To define a probability measure in this case requires a significant amount of machinery from measure theory. It is outside of the scope of this course. For our purposes, whenever we need to use an infinite $\mathrm{\Omega }$ you will be given a probability measure and some of its helpful properties. For example, in this case there exists a probability measure $\mathbb{P}:\mathcal{F}\to [0,1]$ such that

• • The $X_n$ are independent random variables.

• • $\mathbb{P}[X_n=H]=\mathbb{P}[X_n=T]=\frac{1}{2}$ for all $n\in \mathbb{N}$.

From this, you can work with $\mathbb{P}$ without having to know how $\mathbb{P}$ was constructed. You don’t even need to know exactly which subsets of $\mathrm{\Omega }$ are in $\mathcal{F}$, because Proposition 2.2.6 gives you access to plenty of random variables.

Almost surely

In the example from Section 2.3 we used $\mathrm{\Omega }=\{H,T{\}}^{\mathbb{N}}$, which is the set of all sequences made up of $H$s and $T$s. Our probability measure was independent, fair, coin tosses and we used the random variable $X_n$ for the $n^{th}$ toss.

Let’s examine this example a bit. First let us note that, for any individual sequence ${\omega }_1,{\omega }_2,\dots$ of heads and tails, by independence

$$\mathbb{P}[X_1={\omega }_1,X_2={\omega }_2,\dots ]=\frac{1}{2}\cdot \frac{1}{2}\cdot \frac{1}{2}\dots =0.$$

So every element of $\mathrm{\Omega }$ has probability zero. This is not a problem – if we take enough elements of $\mathrm{\Omega }$ together then we do get non-zero probabilities, for example

$$\mathbb{P}[X_1=H]=\mathbb{P}[\omega \in \mathrm{\Omega }\text{ such that }{\omega }_1=H]=\frac{1}{2}$$

which is not surprising.

The probability that we never throw a head is

$$\mathbb{P}[\text{for all }n,X_n=T]=\frac{1}{2}\cdot \frac{1}{2}\dots =0$$

which means that the probability that we eventually throw a head is

$$\mathbb{P}[\text{for some }n,X_n=H]=1-\mathbb{P}[\text{for all }n,X_n=T]=1.$$

So, the event $\{\text{for some }n,X_n=H\}$ has probability $1$, but is not equal to the whole sample space $\mathrm{\Omega }$. To handle this situation we have a piece of terminology.

So, we would say that ‘almost surely, our coin will eventually throw a head’. We might say that ‘$Y\le 1$’ almost surely, to mean that $\mathbb{P}[Y\le 1]=1$. This piece of terminology will be used very frequently from now on. We might sometimes say that an event ‘almost always’ happens, with the same meaning.

For another example, suppose that we define $q_n^H$ and $q_n^T$ to be the proportion of heads and, respectively, tails in the first $n$ coin tosses $X_1,X_2,\dots ,X_n$. Formally, this means that

$$q_n^H=\frac{1}{n}\sum _{i=1}^n\text{𝟙}\{X_i=H\}\text{and}{q}_n^T=\frac{1}{n}\sum _{i=1}^n\text{𝟙}\{X_i=T\}.$$

Here $\text{𝟙}\{X_i=H\}$ is equal to $1$ is $X_i=H$, and equal to zero otherwise; similarly for $\text{𝟙}\{X_i=T\}$. We will think a bit more about random variables of this type in the next section. Of course $q_n^H+q_n^T=1$.

The random variables $\text{𝟙}\{X_i=H\}$ are i.i.d. with $\mathbb{E}[\text{𝟙}\{X_i=H\}]=\frac{1}{2}$, hence by Theorem 1.1.1 we have $\mathbb{P}[q_n^H\to \frac{1}{2}\text{ as }n\to \mathrm{\infty }]=1,$ and by the same argument we have also $\mathbb{P}[q_n^T\to \frac{1}{2}\text{ as }n\to \mathrm{\infty }]=1$. In words, this means that in the long run half our tosses will be tails and half will be heads (which makes sense - our coin is fair). That is, the event

$$E=\{\underset{n\to \mathrm{\infty }}{lim}{q}_n^H=\frac{1}{2}\text{ and }\underset{n\to \mathrm{\infty }}{lim}{q}_n^T=\frac{1}{2}\}$$

occurs almost surely.

There are many many examples of sequences (e.g. $HHTHHTHHT\dots$) that don’t have $q_n^T\to \frac{1}{2}$ and $q_n^H\to \frac{1}{2}$. We might think of the event $E$ as being only a ‘small’ subset of $\mathrm{\Omega }$, but it has probability one.