last updated: May 9, 2024

Probability with Measure

5.4 Independence

In this subsection we consider the meaning of independence for infinite sequences of events and random variables. A useful heuristic is ‘independence means multiply’. Recall that two events \(A_{1}, A_{2} \in {\cal F}\) are independent if

\[ \P [A_{1} \cap A_{2}] = \P [A_{1}]\P [A_{2}].\]

For three events we would use \(\P [A_1\cap A_2\cap A_3]=\P [A_1]\P [A_2]\P [A_3]\) and so on.

For many applications, we want to discuss independence of infinitely many events, or to be precise a sequence \((A_{n})\) of events with \(A_{n} \in {\cal F}\) for all \(\nN \). The definition of independence is extended from the finite case by considering all finite subsets of the sequence. Formally:

  • Definition 5.4.1 We say that the events in the sequence \((A_{n})\) are independent if the finite set \(\{A_{i_{1}}, A_{i_{2}}, \ldots , A_{i_{m}}\}\) is independent for all finite subsets \(\{i_{1}, i_{2}, \ldots , i_{m}\}\) of the natural numbers, i.e.

    \[\P [A_{i_{1}} \cap A_{i_{2}} \cap \cdots , A_{i_{m}}] = \P [A_{i_{1}}]\P [A_{i_{2}}] \cdots \P [A_{i_{m}}].\]

Two random variables \(X\) and \(Y\) are said to be independent if \(\P [X \in A, Y \in B] = \P [X \in A]\P [Y \in B]\) for all \(A, B \in {\cal B}(\R )\). This idea is extended to three or more random variables in the same way as above. For an infinite sequence of random variables \((X_{n})\), we say that the \(X_{n}\) are independent if every finite subset \(X_{i_{1}}, X_{i_{2}}, \ldots , X_{i_{m}}\) of random variables is independent, i.e.

\[ \P [X_{i_{1}} \in A_{i_{1}}, X_{i_{2}} \in A_{i_{2}}, \ldots , X_{i_{m}} \in A_{i_{m}}] = \P [X_{i_{1}} \in A_{i_{1}}] \P [X_{i_{2}} \in A_{i_{2}}] \cdots \P [X_{i_{m}} \in A_{i_{m}}]\]

for all \(A_{i_{1}}, A_{i_{2}}, \ldots , A_{i_{m}} \in {\cal B}(\R )\) and for all finite \(\{i_{1}, i_{2}, \ldots , i_{m}\} \sw \N \).

We often want to consider random variables in \(\R ^d\), where \(d\in \N \). Let us consider the case \(d=2\). A random variable in \(\R ^2\) \(Z = (X,Y)\) is a measurable function from \((\Omega , {\cal F})\) to \((\R ^{2}, {\cal B}(\R ^{2}))\) where \({\cal B}(\R ^{2})\) is the product \(\sigma \)-field introduced in Section 1.9. The law of \(Z\) is the function \(p_Z(A)=\P [Z\in A]\) where \(A\in \mc {B}(\R ^2)\). The joint law of \(X\) and \(Y\) is \(p_{Z}(A \times B) = \P [X \in A, Y \in B]\) for \(A, B \in {\cal B}(\R )\), and the marginal laws of \(X\) and \(Y\) are \(p_X(A)=\P [X\in A]\) and \(p_Y(B)=\P [Y\in B]\). From the definitions above, we have that \(X\) and \(Y\) are independent if and only if

\[ p_{Z}(A \times B) = p_{X}(A)p_{Y}(B),\]

i.e. if the joint law factorises as the product of the two marginals. The same ideas extend to \(\R ^3\) with e.g. \(W=(X,Y,Z)\) and so on.

  • Theorem 5.4.2 Let \(X\) and \(Y\) be random variables.

    • 1. If \(X\) and \(Y\) are independent and \(f,g:\R \to \R \) are measurable functions then \(f(X)\) and \(g(Y)\) are independent.

    • 2. If \(X,Y\in \Lone \) with \(XY\in \Lone \) then \(\E [XY] = \E [X]\E [Y].\)

    • 3. The following two conditions are equivalent:

      • (a) \(X\) and \(Y\) are independent;

      • (b) \(\E [f(X)g(Y)]=\E [f(X)]\E [g(Y)]\) for all bounded measurable functions \(f,g:\R \to \R \).

Proof: The first part is left for you in Exercise 5.8.

\((\star )\) The proof of the second and third parts uses the result of a tricky exercise and also Fubini’s theorem, so we will view them as off-syllabus but we will cover them within lectures. For the second part,

\[ \E [XY] = \int _{\R ^{2}}xy\,p_{Z}(dx, dy) = \left (\int _{\R }x\,p_{X}(dx)\right )\left (\int _{\R }y\,p_{Y}(dy)\right ) = \E [X]\E [Y]\]

Here, the first equality is the two-dimensional version of Problem 5.14, and we have used Fubini’s theorem (from Section 4.9.1) in the second equality to write the integral over \(\R ^{2}\) as a repeated integral.

For the final part, recall that bounded random variables are in \(\Lone \), so combining parts 1 and 2 gives that (a)\(\Rightarrow \)(b). To see that (b)\(\Rightarrow \)(a), take measurable sets \(A,B\in \mc {B}(\R )\) and set \(f=\1_A\) and \(g=\1_B\). Then we have \(\E [f(X)g(Y)]=\P [X\in A,Y\in B]\) and \(\E [f(X)]=\P [X\in A]\), \(\E [g(Y)]=\P [Y\in B]\), so (b) gives \(\P [X\in A,Y\in B]=\P [X\in A]\P [Y\in B]\).   ∎

Regarding part 2 of Theorem 5.4.2, note that dependent random variables \(X\) and \(Y\) can also satisfy \(\E [XY]=\E [X]\E [Y]\). See Exercise 5.9 for an example of this.