Stochastic Processes and Financial Mathematics
(part one)

Chapter A Solutions to exercises (part one)

Chapter 1

1.1 At time $1$ we would have $10 (1 + r)$ in cash and $5$ units of stock. This gives the value of our assets at time $1$ as $10 (1 + r) + 5 S_{1}$ .
1.2
- (a) Assuming we don’t buy or sell anything at time $0$ , the value of our portfolio at time $1$ will be $x (1 + r) + y S_{1}$ , where $S_{1}$ is as in the solution to 1.1. Since we need to be certain of paying off our debt, we should assume a worst case scenario for $S_{1}$ , that is $S_{1} = s d$ . So, we are certain to pay off our debt if and only if
  
  $x (1 + r) + y s d \geq K .$
- (b) Since certainty requires that we cover the worst case scenario of our stock performance, and $d < 1 + r$ , our best strategy is to exchange all our assets for cash at time $t = 0$ . This gives us $x + y s$ in cash at time $0$ . At time $1$ we would then have $(1 + r) (x + y s)$ in cash at time $0$ and could pay our debt providing that
  
  $(1 + r) (x + y s) \geq K .$
1.3
- (a) We borrow $s$ cash at time $t = 0$ and use it to buy one stock. Then wait until time $t = 1$ , at which point we will a stock worth $S_{1}$ , where $S_{1}$ is as in 1.1. We sell our stock, which gives us $S_{1}$ in cash. Since $S_{1} \geq s d > s (1 + r)$ , we repay or debt (plus interest) which costs us $s (1 + r)$ . We then have
  
  $S_{1} - s (1 + r) > 0$
  
  in cash. This is an arbitrage.
- (b) We perform the same strategy as in (a), but with the roles of cash an stock swapped: We borrow $1$ stock at time $t = 0$ and then sell it for $s$ in cash. Then wait until time $t = 1$ , at which point we will have $s (1 + r)$ in cash. Since the price of a stock is now $S_{1}$ , where $S_{1}$ is as in 1.1, and $S_{1} \leq u s < (1 + r) s$ , we now buy one stock costing $S_{1}$ and repay the stockbroker, leaving us with
  
  $s (1 + r) - S_{1} > 0$
  
  in cash. This is an arbitrage.
1.4 We can calculate, using integration by parts, that

$\begin{array}{r} E [X] = \int_{- \infty}^{\infty} x f_{X} (x) d x = \int_{0}^{\infty} x λ e^{- λ x} d x = \frac{1}{λ} . \end{array}$ Similarly, we can calculate that

$E [X^{2}] = \int_{- \infty}^{\infty} x^{2} f_{X} (x) d x = \int_{0}^{\infty} x^{2} λ e^{- λ x} d x = \frac{2}{λ^{2}}$

This gives that

$var (X) = E [X^{2}] - E [X]^{2} = \frac{1}{λ^{2}} .$
1.5 We can calculate

$E [X_{n}] = 0 \cdot P [X_{n} = 0] + n^{2} P [X_{n} = n^{2}] = \frac{n^{2}}{n} = n \to \infty$

as $n \to \infty$ . Also,

$P [| X_{n} | > 0] = P [X_{n} = n^{2}] = \frac{1}{n} \to 0$

as $n \to \infty$ .
1.6 We have

$\begin{aligned} P [Y \leq y] & = P [\frac{X - μ}{σ} \leq y] \\ = P [X \leq μ + y σ] \\ = \int_{- \infty}^{μ + y σ} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{(x - μ)^{2}}{2 σ^{2}}} d x . \end{aligned}$ We want to turn the above integral into the distribution function of the standard normal. To do so, substitute $z = \frac{x - μ}{σ}$ , and we obtain

$\begin{aligned} P [Y \leq y] & = \int_{- \infty}^{y} \frac{1}{\sqrt{2 π σ^{2}}} e^{- \frac{z^{2}}{2}} σ d z \\ = \int_{- \infty}^{y} \frac{1}{\sqrt{2 π}} e^{- \frac{z^{2}}{2}} d z . \end{aligned}$ Hence, $Y$ has the distribution function of $N (0, 1)$ , which means $Y \sim N (0, 1)$ .
1.7 For $p \neq 1$ ,

$\int_{1}^{\infty} x^{- p} d x = {[\frac{x^{- p + 1}}{- p + 1}]}_{x = 1}^{\infty}$

and we obtain a finite answer only if $- p + 1 < 0$ . When $p = 1$ , we have

$\int_{1}^{\infty} x^{- 1} d x = {[\log x]}_{x = 1}^{\infty} = \infty,$

so we conclude that $\int_{1}^{\infty} x^{- p} d x$ is finite if and only if $p > 1$ .
1.8 As $n \to \infty$ ,
- • $e^{- n} \to 0$ because $e > 1$ .
- • $\sin (\frac{n π}{2})$ oscillates through $0, 1, 0, - 1$ and does not converge.
- • $\frac{\cos (n π)}{n}$ converges to zero by the sandwich rule since $| \frac{\cos (n π)}{n} | \leq \frac{1}{n} \to 0$ .
- • $\sum_{i = 1}^{n} 2^{- i} = 1 - 2^{- n}$ is a geometric series, and $1 - 2^{- n} \to 1 - 0 = 1$ .
- • $\sum_{i = 1}^{n} \frac{1}{i}$ tends to infinity, because
  
  $\overset{⏞}{\frac{1}{2}} + \overset{⏞}{\frac{1}{3} + \frac{1}{4}} + \overset{⏞}{\frac{1}{5} + \frac{1}{6} + \frac{1}{7} + \frac{1}{8}} + \overset{⏞}{\frac{1}{9} + \frac{1}{10} + + \frac{1}{11} + \frac{1}{12} + \frac{1}{13} + \frac{1}{14} + \frac{1}{15} + \frac{1}{16}} + \dots .$
  
  each term contained in a $\overset{⏞}{\cdot}$ is greater than or equal to $\frac{1}{2}$ .
1.9 Recall that $\sum_{n = 1}^{\infty} n^{- 2} < \infty$ . Define $n_{1} = 1$ and then

$n_{r + 1} = inf {k \in N; k > n_{r} and | x_{k} | \leq r^{- 2}} .$

Then for all $r$ we have $| x_{n_{r}} | \leq r^{- 2}$ . Hence,

$\sum_{r = 1}^{\infty} | x_{n_{r}} | \leq \sum_{r = 1}^{\infty} r^{- 2} < \infty .$

Chapter 2

2.1 The result of rolling a pair of dice can be written $ω = (ω_{1}, ω_{2})$ where $ω_{1}, ω_{2} \in {1, 2, 3, 4, 5, 6}$ . So a suitable $Ω$ would be

$Ω = {(ω_{1}, ω_{2}); ω_{1}, ω_{2} \in {1, 2, 3, 4, 5, 6}} .$

Of course other choices are possible too. Since our choice of $Ω$ is finite, a suitable $σ$ -field is $F = P (Ω)$ .
2.2
- (a) Consider the case of $F$ . We need to check the three properties in the definition of a $σ$ -field.
  - 1. We have $\emptyset \in F$ so the first property holds automatically.
  - 2. For the second we check compliments: $Ω ∖ Ω = \emptyset$ , $Ω ∖ {1} = {2, 3}$ , $Ω ∖ {2, 3} = {1}$ and $Ω ∖ Ω = \emptyset$ ; in all cases we obtain an element of $F$ .
  - 3. For the third, we check unions. We have ${1} \cup {2, 3} = Ω$ . Including $\emptyset$ into a union doesn’t change anything; including $Ω$ into a union results in $Ω$ . This covers all possible cases, in each case resulting in an element of $F$ .
  So, $F$ is a $σ$ -field. Since $F^{'}$ is just $F$ with the roles of $1$ and $2$ swapped, by symmetry $F^{'}$ is also a $σ$ -field.
- (b) We have ${1} \in F \cup F^{'}$ and ${2} \in F \cup F^{'}$ , but ${1} \cup {2} = {1, 2}$ and ${1, 2} \notin F \cup F^{'}$ . Hence $F \cup F^{'}$ is not closed under unions; it fails property 3 of the definition of a $σ$ -field.
  
  However, $F \cap F^{'}$ is the intersection of two $σ$ -fields, so is automatically itself a $σ$ -field. (Alternatively, note that $F \cap F^{'} = {\emptyset, Ω}$ , and check the definition.)
2.3
- (a) Let $A_{m}$ be the event that the sequence $(X_{n})$ contains precisely $m$ heads. Let $A_{m, k}$ be the event that we see precisely $m$ heads during the first $k$ tosses and, from then on, only tails. Then,
  
  $\begin{aligned} P [A_{m, k}] & = P [m heads in X_{1}, \dots, X_{k}, no heads in X_{k + 1}, X_{k + 2}, \dots] \\ = P [m heads in X_{1}, \dots, X_{k},] P [no heads in X_{k + 1}, X_{k + 2}, \dots] \\ = P [m heads in X_{1}, \dots, X_{k},] \times 0 \\ = 0. \end{aligned}$ If $A_{m}$ occurs then at least one of $A_{m, 1}, A_{m, 2}, \dots$ occurs. Hence,
  
  $P [A_{m}] \leq \sum_{k = 0}^{\infty} P [A_{m, k}] = 0$
  
  .
- (b) Let $A$ be the event that the sequence $(X_{n})$ contains finitely many heads. Then, if $A$ occurs, precisely one of $A_{1}, A_{2}, \dots$ occurs. Hence,
  
  $P [A] = \sum_{m = 0}^{\infty} P [A_{m}] = 0.$
  
  That is, the probability of having only finite many heads is zero. Hence, almost surely, the sequence $(X_{n})$ contains infinitely many heads.
  
  By symmetry (exchange the roles of heads and tails) almost surely the sequence $(X_{n})$ also contains infinitely many tails.

2.4
- (a) $F_{1}$ contains the information of whether a $6$ was rolled.
- (b) $F_{2}$ gives the information of which value was rolled if a $1$ , $2$ or $3$ was rolled, but if the roll was in ${4, 5, 6}$ then $F_{1}$ cannot tell the difference between these values.
- (c) $F_{3}$ contains the information of which of the sets ${1, 2}$ , ${3, 4}$ and ${5, 6}$ contains the value rolled, but gives no more information than that.
2.5
- (a) The values taken by $X_{1}$ are
  
  $X_{1} (ω) = {\begin{cases} 0 & if ω = 3, \\ 1 & if ω = 2, 4, \\ 4 & if ω = 1, 5. \end{cases}$
  
  Hence, the pre-images are $X_{1}^{- 1} (0) = {3}$ , $X_{1}^{- 1} (1) = {2, 4}$ and $X_{1}^{- 1} (4) = {1, 5}$ . These are all elements of $G_{1}$ , so $X_{1} \in m G_{1}$ . However, they are not all elements of $G_{2}$ (in fact, none of them are!) so $X_{1} \notin m G_{2}$ .
- (b) Define
  
  $X_{2} (ω) = {\begin{cases} 0 & if ω = 1, 2, \\ 1 & otherwise . \end{cases}$
  
  Then the pre-images are $X_{2}^{- 1} (0) = {1, 2}$ and $X_{2}^{- 1} (1) = {3, 4, 5}$ . Hence $X_{2} \in m G_{2}$ .
  
  It isn’t possible to use unions/intersections/complements to make the set ${3, 4, 5}$ out of ${1, 5}, {2, 4}, {3}$ (one way to see this is note that as soon as we tried to include $5$ we would also have to include $1$ , meaning that we can’t make ${3, 4, 5}$ ). Hence $X_{2} \notin m G_{1}$ .
2.6 We have $X (T T) = 0$ , $X (H T) = X (T H) = 1$ and $X (H H) = 2$ . This gives

$σ (X) = {\emptyset, {T T}, {T H, H T}, {H H}, {T T, T H, H T}, {T T, H H}, {T H, H T, H H}, Ω} .$

To work this out, you could note that $X^{- 1} (i)$ must be in $σ (X)$ for $i = 0, 1, 2$ , then also include $\emptyset$ and $Ω$ , then keep adding unions and complements of the events until find you have added them all. (Of course, this only works because $σ (X)$ is finite!)
2.7 Since sums, divisions and products of random variables are random variables, $X^{2} + 1$ is a random variable. Since $X^{2} + 1$ is non-zero, we have that $\frac{1}{X^{2} + 1}$ is a random variable, and using products again, $\frac{X}{X^{2} + 1}$ is a random variable.

For $\sin (X)$ , recall that for any $x \in R$ we have

$\sin (x) = \sum_{n = 0}^{\infty} \frac{(- 1)^{n}}{(2 n + 1)!} x^{2 n + 1} .$

Since $X$ is a random variable so is $\frac{(- 1)^{n}}{(2 n + 1)!} X^{2 n + 1}$ . Since limits of random variables are also random variables, so is

$\sin X = lim_{N \to \infty} \sum_{n = 0}^{N} \frac{(- 1)^{n}}{(2 n + 1)!} X^{2 n + 1} .$
2.8 Since $X \geq 0$ we have $E [X^{p}] = E [| X |^{p}]$ for all $p$ .

$E [X] = \int_{1}^{\infty} x^{1} 2 x^{- 3} d x = 2 \int_{1}^{\infty} x^{- 2} d x = 2 {[\frac{x^{- 1}}{- 1}]}_{1}^{\infty} = 2 < \infty$

so $X \in L^{1}$ , but

$E [X^{2}] = \int_{1}^{\infty} x^{2} 2 x^{- 3} d x = 2 \int_{1}^{\infty} x^{- 1} d x = 2 {[\log x]}_{1}^{\infty} = \infty$

so $X \notin L^{2}$ .
2.9 Let $1 \leq p \leq q < \infty$ and $X \in L^{q}$ . Our plan is to generalize the argument that we used to prove (2.7).

First, we condition to see that

$| X |^{p} = | X |^{p} 𝟙 {| X | \leq 1} + | X |^{p} 𝟙 {| X | > 1} .$

Hence,

$\begin{aligned} E [| X |^{p}] & = E [| X |^{p} 𝟙 {| X | \leq 1} + | X |^{p} 𝟙 {| X | > 1}] \\ \leq E [1 + | X |^{q}] \\ = 1 + E | X |^{q}] < \infty \end{aligned}$ Here we use that, if $| X | \leq 1$ then $| X |^{p} \leq 1$ , and if $| X | > 1$ then since $p \leq q$ we have $| X |^{p} < | X |^{q}$ . So $X \in L^{p}$ .
2.10 First, we condition to see that

$\begin{matrix} (A.1) & X = X 𝟙_{{X < a}} + X 𝟙_{{X \geq a}} . \end{matrix}$

From (A.1) since $X \geq 0$ we have

$\begin{aligned} X & \geq X 𝟙_{{X \geq a}} \\ \geq a 𝟙 {X \geq a} . \end{aligned}$ This second inequality follows by looking at two cases: if $X < a$ then both sides are zero, but if $X \geq a$ then we can use that $X \geq a$ . Using monotonicity of $E$ , we then have

$E [X] \geq E [a 𝟙_{{X \geq a}}] = a E [1_{{X \geq a}}] = a P [X \geq a] .$

Dividing through by $a$ finishes the proof.

This result is known as Markov’s inequality.
2.11 Since $X \geq 0$ , Markov’s inequality (from 2.10) gives us that

$P [X \geq a] \leq \frac{1}{a} E [X] = 0$

for any $a > 0$ . Hence,

$\begin{aligned} P [X > 0] & = P [X > 1] + P [1 \geq X > 0] \\ = P [X > 1] + P [⋃_{n = 1}^{\infty} {\frac{1}{n} \geq X > \frac{1}{n + 1}}] \\ = P [X > 1] + \sum_{n = 1}^{\infty} P [\frac{1}{n} \geq X > \frac{1}{n + 1}] \\ \leq P [X \geq 1] + \sum_{n = 1}^{\infty} P [X > \frac{1}{n + 1}] \\ = 0. \end{aligned}$

Chapter 3

3.1 We have $S_{2} = X_{1} + X_{2}$ so

$\begin{aligned} E [S_{2} | σ (X_{1})] & = E [X_{1} | σ (X_{1})] + E [X_{2} | σ (X_{1})] \\ = X_{1} + E [X_{2}] \\ = X_{1} . \end{aligned}$ Here, we use the linearity of conditional expectation, followed by the fact that $X_{1}$ is $σ (X_{1})$ measurable and $X_{2}$ is independent of $σ (X_{1})$ with $E [X_{2}] = 0$ .

Secondly, $S_{2}^{2} = X_{1}^{2} + 2 X_{1} X_{2} + X_{2}^{2}$ so

$\begin{aligned} E [S_{2}^{2}] & = E [X_{1}^{2} | σ (X_{1})] + 2 E [X_{1} X_{2} | σ (X_{1})] + E [X_{2}^{2} | σ (X_{1})] \\ = X_{1}^{2} + 2 X_{1} E [X_{2} | σ (X_{1})] + E [X_{2}^{2}] \\ = X_{1}^{2} + 1 \end{aligned}$ Here, we again use linearity of conditional expectation to deduce the first line. To deduce the second line, we use that $X_{1}^{2}$ and $X_{1}$ are $σ (X_{1})$ measurable (using the taking out what is known rule for the middle term), whereas $X_{2}$ is independent of $σ (X_{1})$ . The final line comes from $E [X_{2} | σ (X_{1})] = 0$ (which we already knew from above) and that $E [X_{2}^{2}] = 1$ .
3.2 For $i \leq n$ , we have $X_{i} \in m F_{n}$ , so $S_{n} = X_{1} + X_{2} + \dots + X_{n}$ is also $F_{n}$ measurable. For each $i$ we have $| X_{i} | \leq 2$ so $| S_{n} | \leq 2 n$ , hence $S_{n} \in L^{1}$ . Lastly,

$\begin{aligned} E [S_{n + 1} | F_{n}] & = E [X_{1} + \dots + X_{n} | F_{n}] + E [X_{n + 1} | F_{n}] \\ = (X_{1} + \dots + X_{n}) + E [X_{n + 1}] \\ = S_{n} . \end{aligned}$ Here, we use linearity of conditional expectation in the first line. To deduce the second, we use that $X_{i} \in m F_{n}$ for $i \leq n$ and that $X_{n + 1}$ is independent of $F_{n}$ in the second. For the final line we note that $E [X_{n + 1}] = \frac{1}{3} 2 + \frac{2}{3} (- 1) = 0$ .
3.3
- (a) Since $X_{i} \in m F_{n}$ for all $i \leq n$ , we have $M_{n} = X_{1} X_{2} \dots X_{n} \in m F_{n}$ . Since $| X_{i} | \leq c$ for all $i$ , we have $| M_{n} | \leq c^{n} < \infty$ , so $M_{n}$ is bounded and hence $M_{n} \in L^{1}$ for all $n$ . Lastly,
  
  $\begin{aligned} E [M_{n + 1} | F_{n}] & = E [X_{1} X_{2} \dots X_{n} X_{n + 1} | F_{n}] \\ = X_{1} \dots X_{n} E [X_{n + 1} | F_{n}] \\ = X_{1} \dots X_{n} E [X_{n + 1}] \\ = X_{1} \dots X_{n} \\ = M_{n} . \end{aligned}$ Here, to deduce the second line we use that $X_{i} \in m F_{n}$ for $i \leq n$ . To deduce the third line we use that $X_{n + 1}$ is independent of $F_{n}$ and then to deduce the forth line we use that $E [X_{n + 1}] = 1$ .
- (b) By definition of conditional expectation (Theorem 3.1.1), we have $M_{n} \in L^{1}$ and $M_{n} \in m G_{n}$ for all $n$ . It remains only to check that
  
  $\begin{aligned} E [M_{n + 1} | G_{n}] & = E [E [Z | G_{n + 1}] | G_{n}] \\ = E [Z | G_{n}] \\ = M_{n} . \end{aligned}$ Here, to get from the first to the second line we use the tower property.
3.4 Note that the first and second conditions in Definition 3.3.3 are the same for super/sub-martingales as for martingales. Hence, $M$ satisfies these conditions. For the third condition, we have that $E [M_{n + 1} | F_{n}] \leq M_{n}$ (because $M$ is a supermartingale) and $E [M_{n + 1} | F_{n}] \geq M_{n}$ (because $M$ is a submartingale. Hence $E [M_{n + 1} | F_{n}] = M_{n}$ , so $M$ is a martingale.
3.5
- (a) If $m = n$ then $E [M_{n} | F_{n}] = M_{n}$ because $M_{n}$ is $F_{n}$ measurable. For $m > n$ , we have
  
  $E [M_{m} | F_{n}] = E [E [M_{m} | F_{m - 1}] | F_{n}] = E [M_{m - 1} | F_{n}] .$
  
  Here we use the tower property to deduce the first equality (note that $m - 1 \geq n$ so $F_{m - 1} \supseteq F_{n}$ ) and the fact that $(M_{n})$ is a martingale to deduce the second inequality (i.e. $E [M_{m} | F_{m - 1}] = M_{m - 1}$ ). Iterating, from $m, m - 1 \dots, n + 1$ we obtain that
  
  $E [M_{m} | F_{n}] = E [M_{n + 1} | F_{n}]$
  
  and the martingale property then gives that this is equal to $M_{n}$ , as required.
- (b) If $(M_{n})$ is a submartingale then $E [M_{m} | F_{n}] \geq M_{n}$ , whereas if $(M_{n})$ is a supermartingale then $E [M_{m} | F_{n}] \leq M_{n}$ .
3.6 We have $(M_{n + 1} - M_{n})^{2} = M_{n + 1}^{2} - 2 M_{n + 1} M_{n} + M_{n}^{2}$ so

$\begin{aligned} E [(M_{n + 1} - M_{n})^{2} | F_{n}] & = E [M_{n + 1}^{2} | F_{n}] - 2 E [M_{n + 1} M_{n} | F_{n}] + E [M_{n}^{2} | F_{n}] \\ = E [M_{n + 1}^{2} | F_{n}] - 2 M_{n} E [M_{n + 1} | F_{n}] + M_{n}^{2} \\ = E [M_{n + 1}^{2} | F_{n}] - 2 M_{n}^{2} + M_{n}^{2} \\ = E [M_{n + 1}^{2} | F_{n}] - M_{n}^{2} \end{aligned}$ as required. Here, we use the taking out what is known rule (since $M_{n} \in m F_{n}$ ) and the martingale property of $(M_{n})$ .

It follows that $E [M_{n + 1}^{2} | F_{n}] - M_{n}^{2} = E [(M_{n + 1} - M_{n})^{2} | F_{n}] \geq 0$ . We have $M_{n}^{2} \in m F_{n}$ and since $M_{n} \in L^{2}$ we have $M_{n}^{2} \in L^{1}$ . Hence $(M_{n}^{2})$ is a submartingale.
3.7 Using that $E [X_{n + 1} | F_{n}] = a X_{n} + b X_{n - 1}$ we can calculate

$E [S_{n + 1} | F_{n}] = E [α X_{n + 1} + X_{n} | F_{n}] = α (a X_{n} + b X_{n - 1}) + X_{n} = (α a + 1) X_{n} + α b X_{n - 1} .$

We want this to be equal to $S_{n}$ , and $S_{n} = α X_{n} + X_{n - 1}$ . So we need

$α a + 1 = α and α b = 1.$

Hence, $α = \frac{1}{b}$ is our choice. It is then easy to check that

$E [S_{n + 1} | F_{n}] = \frac{1}{b} (a X_{n} + b X_{n - 1}) + X_{n} = (\frac{a}{b} + 1) X_{n} + X_{n - 1} = \frac{1}{b} X_{n} + X_{n - 1} = S_{n}$

and thus $S_{n}$ is a martingale, as required.
3.8 Note that

$E [X_{1} | σ (S_{n})] = E [X_{2} | σ (S_{n})] = \dots = E [X_{n} | σ (S_{n})]$

is constant, by symmetry (i.e. permuting the $X_{1}, X_{2}, \dots, X_{n}$ does not change the distribution of $S_{n}$ ). Hence, if we set $E [X_{i} | σ (S_{n})] = α$ then

$\begin{array}{r} n α = \sum_{i = 1}^{n} E [X_{i} | σ (S_{n})] = E [\sum_{i = 1}^{n} X_{i} | σ (S_{n})] = E [S_{n} | σ (S_{n})] = S_{n} . \end{array}$ Here we use the linearity of conditional expectation and the fact that $S_{n}$ is $σ (S_{n})$ measurable. Hence,

$α = E [X_{1} | σ (S_{n})] = \frac{S_{n}}{n} .$

Chapter 4

4.1 Since $Z_{n} = e^{S_{n}}$ and $S_{n} \in m F_{n}$ , also $Z_{n} \in m F_{n}$ . Since $| S_{n} | \leq n$ , we have $| e^{S_{n}} | \leq e^{n} < \infty$ , so $Z_{n}$ is bounded and hence $Z_{n} \in L^{1}$ . Hence also $M_{n} \in m F_{n}$ and $M_{n} \in L^{1}$ .

Lastly,

$\begin{aligned} E [e^{S_{n + 1}} | F_{n}] & = E [e^{X_{n + 1}} e^{S_{n}} | F_{n}] \\ = e^{S_{n}} E [e^{X_{n + 1}} | F_{n}] \\ = e^{S_{n}} E [e^{X_{n + 1}}] \\ = e^{S_{n}} (\frac{1}{2} e + \frac{1}{2} e^{- 1}) \\ = e^{S_{n}} \frac{e + \frac{1}{e}}{2} . \end{aligned}$ Here, we use the taking out what is known rule and the fact that $X_{n + 1}$ (and hence also $e^{X_{n + 1}}$ ) is independent of $F_{n}$ . We also calculate the expectation of the discrete random variable $X_{n + 1}$ , using that $P [X_{n + 1} = 1] = P [X_{n + 1} = - 1] = \frac{1}{2}$ . This gives us that

$\begin{aligned} E [M_{n + 1} | F_{n}] & = M_{n} \\ E [Z_{n + 1} | F_{n}] & \geq Z_{n} \end{aligned}$ where to deduce the last line we use that $\frac{e + \frac{1}{e}}{2} > 1$ . Thus $M_{n}$ is a martingale and $Z_{n}$ is a submartingale.
4.2 Since $X_{i} \in m F_{n}$ for all $i \leq n$ , we have $M_{n} \in m F_{n}$ . We have $| X_{i} | \leq 1$ so $| S_{n} | \leq n$ and $| M_{n} | \leq max {(q / p)^{n}, (q / p)^{- n}} = (q / p)^{- n}$ (note $p > q$ ), which implies that $S_{n} \in L^{1}$ and $M_{n} \in L^{1}$ for all $n$ .

We have

$\begin{aligned} E [S_{n + 1} | F_{n}] & = E [S_{n} + X_{n + 1} | F_{n}] \\ = S_{n} + E [X_{n + 1} | F_{n}] \\ = S_{n} + E [X_{n + 1}] \\ > S_{n} . \end{aligned}$ Here we use the taking out what is known rule, followed by the fact that $X_{n + 1}$ is independent of $F_{n}$ and the relationship between conditional expectation and independence. To deduce the final step we use that $E [X_{n + 1}] = p (1) + q (- 1) = p - q > 0$ . Therefore, we have $E [S_{n + 1} | F_{n}] \geq S_{n}$ , which means that $S_{n}$ is a submartingale.

We now look at $M_{n}$ . We have

$\begin{aligned} E [M_{n + 1} | F_{n}] & = (q / p)^{S_{n}} E [(q / p)^{X_{n + 1}} | F_{n}] \\ = (q / p)^{S_{n}} E [(q / p)^{X_{n + 1}}] \\ = (q / p)^{S_{n}} = M_{n} . \end{aligned}$ Here we use the taking out what is known rule, followed by the fact that $X_{n + 1}$ is independent of $F_{n}$ and the relationship between conditional expectation and independence. To deduce the final step we use that

$E [(q / p)^{X_{n + 1}}] = p (q / p)^{1} + q (q / p)^{- 1} = p + q = 1.$
4.3 Since $X_{i} \in F_{n}$ for all $i \leq n$ we have $S_{n} \in m F_{n}$ where $F_{n} = σ (X_{1}, \dots, X_{n})$ . Further, if we set $m = max (| a |, | b |)$ then $| S_{n} | \leq n m$ so $S_{n} \in L^{1}$ .

We have

$\begin{aligned} E [S_{n + 1} | F_{n}] & = E [S_{n} + X_{n + 1} | F_{n}] \\ = S_{n} + E [X_{n + 1} | F_{n}] \\ = S_{n} + E [X_{n + 1}] \\ = S_{n} + a p_{a} - b p_{b} \end{aligned}$ where $p_{b} = 1 - p_{a}$ . Therefore, $S_{n}$ is a martingale if and only if $a p_{a} = b p_{b}$ .
4.4 Clearly $S_{n} \in m F_{n}$ (recall that, by default, we take $(F_{n})$ to be the generated filtration of $(S_{n})$ ) hence $S_{n}^{2} \in m F_{n}$ and $S_{n}^{2} - n \in F_{n}$ . We have $| S_{n} | l e q n$ so $E [| S_{n}^{2} - n] \leq E [S_{n}^{2}] + E [n] = 2 n$ by the triangle law and monotonicity of expectation, hence $S_{n}^{2} - n \in L^{1}$ . Similarly $S_{n}^{2} \in L^{2}$ ,

We have

$\begin{aligned} E [S_{n + 1}^{2} - (n + 1) | F_{n}] & = E [(X_{n + 1} + S_{n})^{2} - (n + 1) | F_{n}] \\ = E [X_{n + 1}^{2} | F_{n}] + 2 E [S_{n} X_{n + 1} | F_{n}] + E [S_{n}^{2} | F_{n}] - (n + 1) \\ = E [X_{n + 1}^{2}] + 2 S_{n} E [X_{n + 1}] + S_{n}^{2} - (n + 1) \\ = 1 + 2 S_{n} (0) + S_{n}^{2} - (n + 1) \\ = S_{n}^{2} - n \end{aligned}$ Here, the second line follows by linearity. The third line follows by the taking out what is known rule and the independence rule, using that $X_{n + 1}$ is independent of $F_{n}$ , whilst $S_{n} \in m F_{n}$ . The fourth line follows because $X_{n + 1}^{2} = 1$ and $E [X_{n + 1}] = \frac{1}{2} (1) + \frac{1}{2} (- 1) = 0$ . Hence $M_{n} = S_{n}^{2} - n$ is a martingale.

Subtracting $n$ from both sides, we obtain $E [S_{n + 1}^{2} | F_{n}] = S_{n}^{2} + 1$ . Hence $(S_{n}^{2})$ is a submartingale.
4.5 The natural filtration is $F_{n} = σ (X_{1}, \dots, X_{n})$ . We can write

$S_{n + 1} = 𝟙 {S_{n} = 0} + 𝟙 {S_{n} \neq 0} (S_{n} + X_{n + 1}) .$

Hence, if $S_{n} \in m F_{n}$ then $S_{n + 1} \in m F_{n + 1}$ . Since $S_{1} = X_{1} \in m F_{n}$ , a trivial induction shows that $S_{n} \in m F_{n}$ for all $n$ . hence, $L_{n} \in m F_{n}$ and also $S_{n} - L_{n} \in m F_{n}$ .

We have $- n \leq S_{n} - L_{n} \leq n + 1$ (because the walk is reflected at zero and can increase by at most $1$ in each time step) so $S_{n} - L_{n}$ is bounded and hence $S_{n} - L_{n} \in L^{1}$ .

Lastly,

$\begin{aligned} E [S_{n + 1} | F_{n}] & = E [𝟙 {S_{n} = 0} + 𝟙 {S_{n} \neq 0} (S_{n} + X_{n + 1}) | F_{n}] \\ = 𝟙 {S_{n} = 0} + 𝟙 {S_{n} \neq 0} (S_{n} + E [X_{n + 1} | F_{n}]) \\ = 𝟙 {S_{n} = 0} + 𝟙 {S_{n} \neq 0} S_{n} \\ (A.2) & = 𝟙 {S_{n} = 0} + S_{n} . \end{aligned}$ Here we use linearity and the taking out what is known $(S_{n} \in m F_{n})$ rule, as well as that $X_{n + 1}$ is independent of $F_{n}$ . Hence,

$\begin{aligned} E [S_{n + 1} - L_{n + 1} | F_{n}] & = E [S_{n + 1} - \sum_{i = 0}^{n} 𝟙 {S_{i} = 0} | F_{n}] \\ = S_{n} + 𝟙 {S_{n} = 0} - \sum_{i = 0}^{n} 𝟙 {S_{i} = 0} \\ = S_{n} - \sum_{i = 0}^{n - 1} 𝟙 {S_{i} = 0} \\ = S_{n} - L_{n} \end{aligned}$ Here we use (A.2) to deduce the second line, as well as taking out what is known. Hence, $S_{n} - L_{n}$ is a martingale.
4.6 Let $B_{n}$ be number of red balls in the urn at time $n$ (i.e. after the $n^{t h}$ draw is complete). Then, the proportion of red balls in the urn just after the $n^{t h}$ step is

$M_{n} = \frac{B_{n}}{n + 3} .$

Essentially the same argument as for the two colour urn of Section 4.2 shows that $M_{n}$ is adapted to the natural filtration and also in $L^{1}$ . We have

$\begin{aligned} E [M_{n + 1} | F_{n}] & = E [M_{n + 1} 𝟙_{{(n + 1)^{t h} draw is red}} + M_{n + 1} 𝟙_{{(n + 1)^{t h} draw is not red}} | F_{n}] \\ = E [\frac{B_{n} + 1}{n + 4} 𝟙_{{(n + 1)^{t h} draw is red}} + \frac{B_{n}}{n + 4} 𝟙_{{(n + 1)^{t h} draw is not red}} | F_{n}] \\ = \frac{B_{n} + 1}{n + 4} E [𝟙_{{(n + 1)^{t h} draw is red}} | F_{n}] + \frac{B_{n}}{n + 4} E [𝟙_{{(n + 1)^{t h} draw is not red}} | F_{n}] \\ = \frac{B_{n} + 1}{n + 4} \frac{B_{n}}{n + 3} + \frac{B_{n}}{n + 4} (1 - \frac{B_{n}}{n + 3}) \\ = \frac{B_{n} (n + 4)}{(n + 3) (n + 4)} \\ = M_{n} . \end{aligned}$ Hence $(M_{n})$ is a martingale.
4.7 By the result of 3.6, $S_{n}^{2}$ is a submartingale.
- (i) $M_{n} = S_{n}^{2} + n$ is $F_{n}$ measurable (because $S_{n} \in m F_{n}$ ) and since $| S_{n} | \leq n$ we have $| M_{n} | \leq n^{2} + n < \infty$ , so $M_{n} \in L^{1}$ . Further, using the submartingale property of $S_{n}^{2}$ ,
  
  $E [S_{n + 1}^{2} + n + 1 | F_{n}] = E [S_{n + 1}^{2} | F_{n}] + n + 1 \geq S_{n}^{2} + n$
  
  so $M_{n}$ is a submartingale. But $E [M_{1}] = 2 \neq 0 = E [M_{0}]$ , so (by Lemma 3.3.6) we have that $(M_{n})$ is not a martingale.
- (ii) We have shown in Section 4.1 that $(S_{n})$ is a martingale, and in Exercise 4.4 that $M_{n} = S_{n}^{2} - n$ is a martingale. Hence $S_{n}^{2} - n + S_{n} \in F_{n}$ and $E [| S_{n}^{2} + S_{n} - n |] \leq E [| S_{n}^{2} - n |] + E [| S_{n} |] < \infty$ by the triangle law, so also $S_{n}^{2} + S_{n} - n \in L^{1}$ . Also,
  
  $\begin{aligned} E [S_{n + 1}^{2} + S_{n + 1} - (n + 1) | F_{n}] & = E [S_{n + 1}^{2} - (n + 1) | F_{n}] + E [S_{n + 1} | F_{n}] \\ = S_{n}^{2} - n + S_{n} \end{aligned}$ as required. To deduce the second line we use the martingale properties of $(S_{n})$ and $(M_{n})$ . As all martingales are submartingales, this case is also a submartingale.
  
  More generally, the sum of two martingales is always a martingale. The same is true for submartingales and also for supermartingales.
- (iii) $M_{n} = \frac{S_{n}}{n}$ is $F_{n}$ measurable (because $S_{n} \in m F_{n}$ ) and since $| S_{n} | \leq n$ we have $| S_{n} | \leq 1$ , so $M_{n} \in L^{1}$ . Since $S_{n}$ is a martingale, we have
  
  $E [\frac{S_{n + 1}}{n + 1} | F_{n}] = \frac{1}{n + 1} E [S_{n + 1} | F_{n}] = \frac{S_{n}}{n + 1} \neq \frac{S_{n}}{n} .$
  
  Hence $M_{n} = \frac{S_{n}}{n}$ is not a martingale. Also, since $S_{n}$ may be both positive or negative, we do not have $\frac{S_{n}}{n + 1} \geq \frac{S_{n}}{n}$ , so $S_{n}$ is also not a submartingale.
4.8 We showed that $S_{n}^{2} - n$ was a martingale in 4.7. We try to mimic that calculation and find out what goes wrong in the cubic case. So, we look at

$\begin{aligned} E [(S_{n + 1} - S_{n})^{3} | F_{n}] & = E [S_{n + 1}^{3} | F_{n}] - 3 S_{n} E [S_{n + 1}^{2} | F_{n}] + 3 S_{n}^{2} E [S_{n + 1} | F_{n}] - E [S_{n}^{3} | F_{n}] \\ = E [S_{n + 1}^{3} | F_{n}] - 3 S_{n} (S_{n}^{2} + 1) + 3 S_{n}^{2} S_{n} - S_{n}^{3} \\ = E [S_{n + 1}^{3} | F_{n}] - S_{n}^{3} - 3 S_{n} . \end{aligned}$ Here we use linearity, taking out what is known and that fact that $E [S_{n + 1}^{2} | F_{n}] = S_{n}^{2} + 1$ (from 4.7). However, also $S_{n + 1} - S_{n} = X_{n + 1}$ , so

$\begin{aligned} E [(S_{n + 1} - S_{n})^{3} | F_{n}] & = E [X_{n + 1}^{3} | F_{n}] \\ = E [X_{n + 1}^{3}] \\ = 0. \end{aligned}$ because $X_{n + 1}$ is independent of $F_{n}$ and $X_{n + 1}^{3} = X_{n}$ (it takes values only $1$ or $- 1$ ) so $E [X_{n + 1}^{3}] = 0$ . Putting these two calculations together, we have

$\begin{matrix} (A.3) & E [S_{n + 1}^{3} | F_{n}] = S_{n}^{3} + 3 S_{n} . \end{matrix}$

Suppose (aiming for a contradiction) that there is a deterministic function $f : N \to R$ such that $S_{n}^{3} - f (n)$ is a martingale. Then

$E [S_{n + 1}^{3} - f (n + 1) | F_{n}] = S_{n}^{3} - f (n) .$

Combining the above equation with (A.3) gives

$f (n + 1) - f (n) = 3 S_{n}$

but this is impossible because $S_{n}$ is not deterministic. Thus we reach a contradiction; no such function $f$ exists.

Chapter 5

5.1 The value of the portfolio $(1, 3)$ at time $1$ will be

$1 (1 + r) + 3 S_{1} = (1 + r) + 3 S_{1}$

where $S_{1}$ is a random variable with $P [S_{1} = s u] = p_{u}$ and $P [S_{1} = s d] = p_{d}$ .
5.2
- (a) We need a portfolio $h = (x, y)$ such that $V_{1}^{h} = Φ (S_{1}) = 1$ , so we need that
  
  $\begin{aligned} x (1 + r) + y s u & = 1 \\ x (1 + r) + y s d & = 1. \end{aligned}$ Solving this pair of linear equations gives $x = \frac{1}{1 + r}$ and $y = 0$ . So our replicating portfolio consists simply of $\frac{1}{1 + r}$ cash and no stock.
  
  Hence, the value of this contingent claim at time $0$ is $x + s y = \frac{1}{1 + r}$ .
- (b) Now we need that
  
  $\begin{array}{r} x (1 + r) + y s u = 3 \\ x (1 + r) + y s d = 1. \end{array}$ Which has solution $x = \frac{1}{1 + r} (3 - \frac{2 s u}{s u - s d})$ and $y = \frac{2}{s (u - d)}$ .
  
  Hence, the value of this contingent claim at time $0$ is $x + s y = \frac{1}{1 + r} (3 - \frac{2 s u}{s u - s d}) + \frac{2}{u - d}$ .
5.3
- (a) If we buy two units of stock, at time $1$ , for a price $K$ then our contingent claim is $Φ (S_{1}) = 2 S_{1} - 2 K$ .
- (b) In a European put option, with strike price $K \in (s d, s u)$ , we have the option to sell a single unit of stock for strike price $K$ . It is advantageous to us to do so only if $K > S_{1}$ , so
  
  $\begin{matrix} (A.4) & Φ (S_{1}) = {\begin{cases} 0 & if S_{1} = s u, \\ K - S_{1} & if S_{1} = s d . \end{cases} = max (K - S_{1}, 0) \end{matrix}$
- (c) We $S_{1} = s u$ we sell a unit of stock for strike price $K$ and otherwise do nothing. So our contingent claim is
  
  $Φ (S_{1}) = {\begin{cases} K - S_{1} & if S_{1} = s u, \\ 0 & if S_{1} = s d . \end{cases}$
- (d) Holding both the contracts in (b) and (c) at once, means that in both $S_{1} = s u$ and $S - 1 = s d$ , we end up selling a single unit of stock for a fixed price $K$ . Our contingent claim for doing so is $Φ (S_{1}) = K - S_{1}$ .
5.4
- (a) The contingent claims for the call and put options respectively are $max (S_{1} - K, 0)$ from (5.6) and $max (K - S_{1}, 0)$ from (A.4). Using the risk neutral valuation formula from Proposition 5.2.6 we have
  
  $Π_{0}^{c a l l} = \frac{1}{1 + r} E^{Q} [max (S_{1} - K, 0)], Π_{0}^{p u t} = \frac{1}{1 + r} E^{Q} [max (K - S_{1}, 0)] .$
- (b) Hence,
  
  $\begin{aligned} Π_{0}^{c a l l} - Π_{0}^{p u t} & = \frac{1}{1 + r} E^{Q} [max (S_{1} - K, 0) - max (K - S_{1}, 0)] \\ = \frac{1}{1 + r} E^{Q} [max (S_{1} - K, 0) + min (S_{1} - K, 0)] \\ = \frac{1}{1 + r} E^{Q} [S_{1} - K] \\ = \frac{1}{1 + r} (E^{Q} [S_{1}] - K) \end{aligned}$ By (5.4) (or you can use Proposition 5.2.6 again) we have $\frac{1}{1 + r} E^{Q} [S_{1}] = S_{0} = s$ , so we obtain
  
  $Π_{0}^{c a l l} - Π_{0}^{p u t} = s - \frac{K}{1 + r} .$
5.5 In the binomial model, the contingent claim of a European call option with strike price $K$ is $Φ (S_{T}) = max (S_{T} - K)$ .
5.6 A tree-like diagram of the possible changes in stock price, with the value of the contingent claim $Φ (S_{T}) = max (K - S_{1}, 0)$ written on for the final time step looks like

By (5.3), the risk free probabilities in this case are

$q_{u} = \frac{(1 + 0.25) - 0.5}{2.0 - 0.5} = 0.5, q_{d} = 1 - q_{u} = 0.5 .$

Using Proposition 5.2.6 recursively, as in Section 5.6, we can put in the value of the European put option at all nodes back to time $0$ , giving

Hence, the value of our European put option at time $0$ is $12$ .

Changing the values of $p_{u}$ and $p_{d}$ do not alter value of our put option, since $p_{u}$ and $p_{d}$ do not appear in the pricing formulae.
5.7 A tree-like diagram of the possible changes in stock price, with the value of the option at all nodes of the tree shown in (green) square boxes, and hedging portfolios $h = (x, y)$ written (in red) for each node of the tree:

The hedging portfolios are found by solving linear equations of the form $x + d S y = \tilde{Φ} (d S)$ , $x + u S y = \tilde{Φ} (u D)$ , where $\tilde{Φ}$ is the contingent claim and $S$ the initial stock price of the one period model associated to the given node. The value of the contingent claim at each node is then inferred from the hedging portfolio as $V = x + S y$ .
5.8 A tree-like diagram of the possible changes in stock price, with the value of the option at all nodes of the tree shown in (green) square boxes, and hedging portfolios $h = (x, y)$ written (in red) for each node of the tree:

The hedging portfolios are found by solving linear equations of the form $x + d S y = \tilde{Φ} (d S)$ , $x + u S y = \tilde{Φ} (u D)$ , where $\tilde{Φ}$ is the contingent claim and $S$ the initial stock price of the one period model associated to the given node. The value of the contingent claim at each node is then inferred from the hedging portfolio as $V = x + S y$ .

The hedging portfolio is the same at each node. This is because the contingent claim of our call option always exceeds zero i.e. we are certain that at time $T = 2$ we would want to exercise our call option and buy a single unit of stock for a price $K = 60$ . Since there is no interest payable on cash, this means that we can replicate our option (at all times) by holding a single unit of stock, and $- 60$ in cash.
5.9 We have that $(S_{t})$ is adapted (see Section 5.4) and since $S_{t} \in (d^{t}, u^{t})$ we have also that $S_{t} \in L^{1}$ . Hence, $S_{t}$ is a martingale under $P$ if and only if $E^{P} [S_{t + 1} | F_{t}] = S_{t}$ . That is if

$\begin{aligned} E^{P} [S_{t + 1} | F_{t}] & = E [Z_{t + 1} S_{t} | F_{t}] \\ = S_{t} E^{P} [Z | F_{t}] \\ = S_{t} E^{P} [Z] \end{aligned}$ is equal to $S_{t}$ . Hence, $S_{t}$ is a martingale under $P$ if and only if $E^{P} [Z_{t + 1}] = u p_{u} + d p_{d} = 1.$

Now consider $M_{t} = \log S_{t}$ , and assume that $0 < d < u$ and $0 < s$ , which implies $S_{t} > 0$ . Since $S_{t}$ is adapted, so is $M_{t}$ . Since $S_{t} \in S_{t} \in (d^{t}, u^{t})$ we have $M_{t} \in (t \log d, t \log u)$ so also $M_{t} \in L^{1}$ . We have

$M_{t} = \log (S_{0} \prod_{i = 1}^{t} Z_{i}) = \log (S_{0}) + \sum_{i = 1}^{t} Z_{i} .$

Hence,

$\begin{aligned} E^{P} [M_{t + 1} | F_{t}] & = \log S_{0} + \sum_{i = 1}^{t} \log (Z_{i}) + E [\log (Z_{t + 1}) | F_{t}] \\ = M_{t} + E^{P} [\log Z_{t + 1}] \\ = M_{t} + p_{u} \log u + p_{d} \log d . \end{aligned}$ Here we use taking out what is known, since $S_{0}$ and $Z_{i} \in m F_{t}$ for $i \leq t$ , and also that $Z_{t + 1}$ is independent of $F_{t}$ . Hence, $M_{t}$ is a martingale under $P$ if and only if $p_{u} \log u + p_{d} \log d = 0$ .

Chapter 6

6.1 Since $X_{n} \geq 0$ we have

$E [| X_{n} - 0 |] = E [X_{n}] = \frac{1}{2} 2^{- n} = 2^{- (n + 1)}$

which tends to $0$ as $n \to \infty$ . Hence $X_{n} \to 0$ in $L^{1}$ . Lemma 6.1.2 then implies that also $X_{n} \to 0$ in probability and in distribution. Also, $0 \leq X_{n} \leq 2^{- n}$ and $2^{- n} \to 0$ , so by the sandwich rule we have $X_{n} \to 0$ as $n \to \infty$ , and hence $P [X_{n} \to 0] = 1$ , which means that $X_{n} \to 0$ almost surely.
6.2
- (a) We have $X_{n} \overset{L^{1}}{\to} X$ and hence
  
  $| E [X_{n}] - E [X] | = | E [X_{n} - X] | \leq E [| X_{n} - X |] \to 0.$
  
  Here we use the linearity and absolute value properties of expectation. Hence, $E [X_{n}] \to E [X]$ .
- (b) Let $X_{1}$ be a random variable such that $P [X_{1} = 1] = P [X_{1} = - 1] = \frac{1}{2}$ and note that $E [X_{1}] = 0$ . Set $X_{n} = X_{1}$ for all $n \geq 2$ , which implies that $E [X_{n}] = 0 \to 0$ as $n \to \infty$ . But $E [| X_{n} - 0 |] = E [| X_{n} |] = 1$ so $X_{n}$ does not converge to $0$ in $L^{1}$ .
6.3 We have $X_{n} = X$ when $U = 2$ , and $| X_{n} - X | = \frac{1}{n}$ when $U = 0, 1$ . Hence, $| X_{n} - X | \leq \frac{1}{n}$ and by monotonicity of $E$ we have

$E [| X_{n} - X |] \leq \frac{1}{n} \to 0 as n \to \infty,$

which means that $X_{n} \overset{L^{1}}{\to} X$ .

We can write

$\begin{aligned} P [X_{n} \to X] & = P [X_{n} \to X, U = 0] + P [X_{n} \to X, U = 1] + P [X_{n} \to X, U = 2] \\ = P [1 + \frac{1}{n} \to 1, U = 0] + P [1 - \frac{1}{n} \to 1, U = 1] + P [0 \to 0, U = 2] \\ = P [U = 0] + P [U = 1] + P [U = 2] \\ = 1. \end{aligned}$ Hence, $X_{n} \overset{a . s .}{\to} X$ .

By Lemma 6.1.2, it follows that also $X_{n} \to X$ in probability and in distribution.
6.4 We have $X_{n} = X_{1} = 1 - Y$ for all $n$ , so $Y = 0 \Leftrightarrow X_{n} = 1$ and $Y = 1 \Leftrightarrow X_{n} = 0$ . Hence $| Y - X_{n} | = 1$ for all $n$ . So, for example, with $a = \frac{1}{2}$ we have $P [| X_{n} - Y | \geq a] = 1$ , which does not tend to zero as $n \to \infty$ . Hence $X_{n}$ does not converge to $Y$ in probability.

However, $X_{n}$ and $y$ have the same distribution, given by

$P [Y \leq x] = P [X_{n} \leq x] = {\begin{cases} 0 & if x < 0 \\ \frac{1}{2} & if x \in [0, 1) \\ 1 & if x \geq 1 \end{cases}$

so we do have $P [X_{n} \leq x] \to P [Y \leq x]$ as $n \to \infty$ . Hence $X_{n} \overset{d}{\to} Y$ .
6.5 Let $(X_{n})$ be the sequence of random variables from 6.1. Define $Y_{n} = X_{1} + X_{2} + \dots + X_{n}$ .
- (a) For each $ω \in Ω$ , we have $Y_{n + 1} (ω) = Y_{n} (ω) + X_{n + 1} (ω)$ . Hence, $(Y_{n} (ω)$ is an increasing sequence. Since $X_{n} (ω) \leq 2^{- n}$ for all $n$ we have
  
  $| Y_{n} (ω) | \leq 2^{- 1} + 2^{- 2} + \dots + 2^{- n} \leq \sum_{i = 1}^{\infty} 2^{- i} = 1,$
  
  meaning that the sequence $Y_{n} (ω)$ is bounded above by $1$ .
- (b) Hence, since bounded increasing sequences converge, for any $ω \in Ω$ the sequence $Y_{n} (ω)$ converges.
- (c) The value of $Y_{1}$ is either $0$ or $\frac{1}{2}$ , both with probability $\frac{1}{2}$ . The value of $Y_{2}$ is either $0, \frac{1}{4}, \frac{1}{2}, \frac{3}{4}$ , all with probability $\frac{1}{4}$ . The value of $Y_{3}$ is either $0, \frac{1}{8}, \frac{1}{4}, \frac{3}{8}, \frac{1}{2}, \frac{5}{8}, \frac{3}{4}, \frac{7}{8}$ all with probability $\frac{1}{8}$ .
- (d) We can guess from (c) that the $Y_{n}$ are becoming more and more evenly spread across $[0, 1]$ , so we expect them to approach a uniform distribution as $n \to \infty$ .
- (e) We will use induction on $n$ . Our inductive hypothesis is
  - (IH) $_{n}$ : For all $k = 0, 1, \dots, 2^{n} - 1$ , it holds that $P [Y_{n} = k 2^{- n}] = 2^{- n}$ .
  In words, this says that $Y_{n}$ is uniformly distributed on ${k 2^{- n}; k = 0, 1, \dots, 2^{n} - 1}$ .
  
  For the case $n = 1$ , we have $Y_{1} = X_{1}$ so $P [Y_{1} = 0] = P [Y_{1} = \frac{1}{2}] = \frac{1}{2}$ , hence (IH) $_{1}$ holds.
  
  Now, assume that (IH) $_{n}$ holds. We want to calculate $P [Y_{n + 1} = k 2^{- (n + 1)}$ for $k = 0, 1, \dots, 2^{n + 1} - 1$ . We consider two cases, dependent on whether $k$ is even or odd.
  - • If $k$ is even then we can write $k = 2 j$ for some $j = 0, \dots, 2^{n} - 1$ . Hence,
    
    $\begin{aligned} P [Y_{n + 1} = k 2^{- (n + 1)}] & = P [Y_{n + 1} = j 2^{- n}] \\ = P [Y_{n} = j 2^{- n} and X_{n + 1} = 0] \\ = P [Y_{n} = j 2^{- n}] P [X_{n + 1} = 0] \\ = 2^{- n} \frac{1}{2} \\ = 2^{- (n + 1)} . \end{aligned}$ Here we use that $Y_{n}$ and $X_{n + 1}$ are independent, and use (IH) $_{n}$ to calculate $P [Y_{n} = j 2^{- n}]$ .
  - • Alternatively, if $k$ is odd then we can write $k = 2 j + 1$ for some $j = 0, \dots, 2^{n} - 1$ . Hence,
    
    $\begin{aligned} P [Y_{n + 1} = k 2^{- (n + 1)}] & = P [Y_{n + 1} = j 2^{- n} + 2^{- (n + 1)}] \\ = P [Y_{n} = j 2^{- n} and X_{n + 1} = 2^{- (n + 1)}] \\ = P [Y_{n} = j 2^{- n}] P [X_{n + 1} = 2^{- (n + 1)}] \\ = 2^{- n} \frac{1}{2} \\ = 2^{- (n + 1)} . \end{aligned}$ Here, again, we use that $Y_{n}$ and $X_{n + 1}$ are independent, and use (IH) $_{n}$ to calculate $P [Y_{n} = j 2^{- n}]$ .
  In both cases we have shown that $P [Y_{n + 1} = k 2^{- (n + 1)}] = 2^{- (n + 1)}$ , hence (IH) $_{n + 1}$ holds.
- (f) We know that $Y_{n} \to Y$ almost surely, hence by Lemma 6.1.2 the convergence also holds in distribution. Hence $P [Y_{n} \leq a] \to P [Y \leq a]$ .
  
  From part (e), for any $a \in [0, 1]$ we have
  
  $(k - 1) 2^{- n} \leq a < k 2^{- n} \Rightarrow P [Y_{n} \leq a] = k 2^{- n} .$
  
  Hence $a \leq P [Y_{n} \leq a] \leq a + 2^{- n}$ , and the sandwich rule tells us that $P [Y_{n} \leq a] \to a$ . Hence $P [Y \leq a] = a$ , for all $a \in [0, 1]$ , which means that $Y$ has a uniform distribution on $[0, 1]$ .
6.7 Since $𝟙 {X \leq n + 1} \geq 1 {X \leq n}$ we have $X_{n + 1} \geq X_{n}$ , and since $X > 0$ we have $X_{n} \geq 0$ . Further, for all $ω \in Ω$ , whenever $X (w) < n$ we have $X_{n} (ω) = X (ω)$ , and since $P [X (ω) < \infty] = 1$ this means that $P [X_{n} (ω) \to X (ω)] = 1$ . Hence $X_{n} \overset{a . s .}{\to} X$ .

Therefore, the monotone convergence theorem applies and $E [X_{n}] \to E [X]$ .
6.6 Define $X_{n} = - Y_{n}$ . Then $X_{n} \geq 0$ and $X_{n + 1} \geq X_{n}$ , almost surely. Hence, $(X_{n})$ satisfies the conditions for the dominated convergence theorem and there exists a random variable $X$ such that $X_{n} \overset{a . s .}{\to} X$ and $E [X_{n}] \to E [X]$ . Define $Y = - X$ and then we have $Y_{n} \overset{a . s .}{\to} Y$ and $E [Y_{n}] \to E [Y]$ .
6.8
- (a) The equation
  
  $\begin{matrix} (A.5) & E [\sum_{i = 1}^{n} X_{i}] = \sum_{i = 1}^{n} E [X_{i}] \end{matrix}$
  
  follows by iterating the linearity property of $E$ $n$ times (or by a trivial induction). However, in $E [\sum_{i = 1}^{\infty} X_{i}] = \sum_{i = 1}^{\infty} E [X_{i}]$ both sides are defined by infinite sums, which are limits, and the linearity property does not tell us anything about limits.
- (b) We’ll take a limit as $n \to \infty$ on each side of (A.5). We’ll have to justify ourselves in each case.
  
  First, consider the right hand side. Since we have $X_{n} \geq 0$ almost surely, $E [X_{n}] \geq 0$ by monotonicity of $E$ . Therefore, $\sum_{i = 1}^{n} E [X_{i}]$ is an increasing sequence of real numbers. Hence it is convergent (either to a real number or to $+ \infty$ ) and, by definition of an infinite series, the limit is written $\sum_{i = 1}^{\infty} E [X_{i}]$ .
  
  Now, the left hand side, and here we’ll need the monotone convergence theorem. Define $Y_{n} = \sum_{i = 1}^{n} X_{i}$ . Then $Y_{n + 1} - Y_{n} = X_{n + 1} \geq 0$ almost surely, so $Y_{n + 1} \geq Y_{n}$ . Since $X_{i} \geq 0$ for all $i$ , also $Y_{n} \geq 0$ , almost surely. Hence the monotone convergence theorem applies and $E [Y_{n}] \to E [Y_{\infty}]$ where $Y_{\infty} = \sum_{i = 1}^{\infty} X_{i}$ .
  
  Putting both sides together (and using the uniqueness of limits to tell us that the limits on both sides must be equal as $n \to \infty$ ) we obtain that $E [\sum_{i = 1}^{\infty} X_{i}] = \sum_{i = 1}^{\infty} E [X_{i}]$ .
- (c) Since $P [X_{i} = 1] = P [X_{i} = - 1]$ , we have $E [X_{i}] = 0$ and hence also $\sum_{i = 1}^{\infty} E [X_{i}] = 0$ . However, the limit $\sum_{i = 1}^{\infty} X_{i}$ is not well defined, because $Y_{n} = \sum_{i = 1}^{n} X_{i}$ oscillates (it jumps up/down by $\pm 1$ on each step) and does not converge. Hence, in this case $E [\sum_{i = 1}^{\infty} X_{i}]$ is not well defined.
6.6
- (a) We have both $P [X_{n} \leq x] \to P [X \leq x]$ and $P [X_{n} \leq x] \to P [Y \leq x]$ , so by uniqueness of limits for real sequences, we have $P [X \leq x] = P [Y \leq x]$ for all $x \in R$ . Hence, $X$ and $Y$ have the same distribution (i.e. they have the same distribution functions $F_{X} (x) = F_{Y} (x)$ ).
- (b) By definition of convergence in probability, for any $a > 0$ , for any $ϵ > 0$ there exists $N \in N$ such that, for all $n \geq N$ ,
  
  $P [| X_{n} - X | > a] < ϵ and P [| X_{n} - Y | > a] < ϵ .$
  
  By the triangle inequality we have
  
  $\begin{matrix} (A.6) & P [| X - Y | > 2 a] = P [| X - X_{n} + X_{n} - Y | > 2 a] \leq P [| X - X_{n} | + | X_{n} - Y | > 2 a] . \end{matrix}$
  
  If $| X - X_{n} | + | X_{n} - Y | > 2 a$ then $| X - X_{n} | > a$ or $| X_{n} - Y | > a$ (or possibly both). Hence, continuing (A.6),
  
  $P [| X - Y | > 2 a] \leq P [| X_{n} - X | > a] + P [| X_{n} - Y | > a] \leq 2 ϵ .$
  
  Since this is true for any $ϵ > 0$ and any $a > 0$ , we have $P [X = Y] = 1$ .
6.7 Suppose (aiming for a contradiction) that there exists and random variable $X$ such that $X_{n} \to X$ in probability. By the triangle inequality we have

$| X_{n} - X_{n + 1} | \leq | X_{n} - X | + | X - X_{n + 1} |$

Hence, if $| X_{n} - X_{n + 1} | > 1$ then $| X_{n} - X | > \frac{1}{2}$ or $| X_{n + 1} - X | > \frac{1}{2}$ (or both). Therefore,

$P [| X_{n} - X_{n + 1} | > 1] \leq P [| X_{n} - X | > \frac{1}{2}] + P [| X_{n + 1} - X | > \frac{1}{2}] .$

Since $X_{n} \to X$ in probability, the right hand side of the above tends to zero as $n \to \infty$ . This implies that

$P [| X_{n} - X_{n + 1} | > 1] \to 0$

as $n \to \infty$ . But, $X_{n}$ and $X_{n + 1}$ are independent and $P [X_{n} = 1, X_{n + 1} = 0] = \frac{1}{4}$ so $P [| X_{n} - X_{n + 1} | > 1] \geq \frac{1}{4}$ for all $n$ . Therefore we have a contradiction, so there is no $X$ such that $X_{n} \to X$ in probability.

Hence, by Lemma 6.1.2, we can’t have any $X$ such that $X_{n} \to X$ almost surely or in $L^{1}$ (since it would imply convergence in probability).

Chapter 7

7.1
- (a) We have $(C \circ M)_{n} = \sum_{i = 1}^{n} 0 (S_{i} - S_{i - 1}) = 0.$
- (b) We have $(C \circ M)_{n} = \sum_{i = 1}^{n} 1 (S_{i} - S_{i - 1}) = S_{n} - S_{0} = S_{n} .$
- (c) We have
  
  $\begin{aligned} (C \circ M)_{n} & = \sum_{i = 1}^{n} S_{i - 1} (S_{i} - S_{i - 1}) \\ = \sum_{i = 1}^{n} (X_{1} + \dots + X_{i - 1}) X_{i} \\ = \frac{1}{2} {(\sum_{i = 1}^{n} X_{i})}^{2} - \frac{1}{2} \sum_{i = 1}^{n} X_{i}^{2} \\ = \frac{S_{n}^{2}}{2} - \frac{n}{2} \end{aligned}$ In the last line we use that $X_{i}^{2} = 1$ .
7.2 We have

$\begin{aligned} (X \circ M)_{n} & = \sum_{i = 1}^{n} (α C_{n - 1} + β D_{n - 1}) (M_{n} - M_{n - 1}) \\ = α \sum_{i = 1}^{n} C_{n - 1} (M_{n} - M_{n - 1}) + β \sum_{i = 1}^{n} D_{n - 1} (M_{n} - M_{n - 1}) \\ = α (C \circ M)_{n} + β (C \circ M)_{n} \end{aligned}$ as required.
7.3
- (a) We note that
  
  $| S_{n} | \leq \sum_{i = 1}^{n} | X_{i} | \leq \sum_{i = 1}^{n} 1 = n < \infty .$
  
  Hence $S_{n} \in L^{1}$ . Since $X_{i} \in F_{n}$ for all $i \leq n$ , we have also that $S_{n} \in m F_{n}$ . Lastly,
  
  $\begin{aligned} E [S_{n + 1} | F_{n}] & = E [S_{n} + X_{n + 1} | F_{n}] \\ = S_{n} + E [X_{n + 1}] \\ = S_{n} . \end{aligned}$ Therefore, $S_{n}$ is a martingale. In the above, in the second line we use that $S_{n} \in m F_{n}$ and that $X_{n + 1}$ is independent of $F_{n}$ . To deduce the last line we use that $E [X_{i}] = \frac{1}{2 n^{2}} (1) = \frac{1}{2 n^{2}} (- 1) + (1 - \frac{1}{n^{2}}) (0) = 0$ .
  
  We next check that $S_{n}$ is uniformly bounded in $L^{1}$ . We have $E [| X_{i} |] = \frac{1}{2 i^{2}} | 1 | + \frac{1}{2 i^{2}} | - 1 | + (1 - \frac{1}{i^{2}}) | 0 | = \frac{1}{i^{2}}$ . Using that $| S_{n} | \leq | X_{1} | + \dots + | X_{n} |$ , monotonicity of expectations gives that
  
  $E [| S_{n} |] \leq \sum_{i = 1}^{n} E [| X_{i} |] \leq \sum_{i = 1}^{n} \frac{1}{i^{2}} \leq \sum_{i = 1}^{\infty} \frac{1}{i^{2}} < \infty .$
  
  Hence $sup_{n} E [| S_{n} |] < \infty$ , so $(S_{n})$ is uniformly bounded in $L^{1}$ .
  
  Therefore, the martingale convergence theorem applies, which gives that there exists some real-valued random variable $S_{\infty}$ such that $S_{n} \overset{a . s .}{\to} S_{\infty}$ .
- (b) Note that $(S_{n})$ takes values in $Z$ and that, from part (b), almost surely we have that $S_{n}$ converges to a real value. It follows immediately by Lemma 7.4.1 that (almost surely) there exists $N \in N$ such that $S_{n} = S_{\infty}$ for all $n \geq N$ .
7.4 Here’s some R code to plot $1000$ time-steps of the symmetric random walk. Changing the value given to set.seed will generate a new sample.
```
> T=1000
> set.seed(1)
> x=c(0,2*rbinom(T-1,1,0.5)-1)
> y=cumsum(x)
> par(mar=rep(2,4))
> plot(y,type="l")
```
Adapting it to plot the asymmetric case and the random walk from exercise 7.3 is left for you.

In case you prefer Python, here’s some Python code that does the same job. (I prefer Python.)
```
import numpy as np
import matplotlib.pyplot as plt

p = 0.5
rr = np.random.random(1000)
step = 2*(rr < p) - 1

start = 0
positions = [start]
for x in step:
    positions.append(positions[-1] + x)

plt.plot(positions)
plt.show()
```
Extending this code as suggested is left for you. For the last part, the difference in behaviour that you should notice from the graph is that the walk from Exercise 7.3 eventually stops moving (at some random time), whereas the random walk from Q2 of Assignment 3 never keeps moving but makes smaller and smaller steps.
7.5 We will argue by contradiction. Suppose that $(M_{n})$ was uniformly bounded in $L^{1}$ . Then the martingale convergence theorem would give that $M_{n} \overset{a . s .}{\to} M_{\infty}$ for some real valued random variable $M_{\infty}$ . However, $M_{n}$ takes values in $Z$ and we have $M_{n + 1} \neq M_{n}$ for all $n$ , so Lemma 7.4.1 gives that $(M_{n})$ cannot converge almost surely. This is a contradiction.
7.6
- (a) Take the offspring distribution to be $P [G = 0] = 1$ . Then $Z_{1} = 0$ , and hence $Z_{n} = 0$ for all $n \geq 1$ , so $Z_{n}$ dies out.
- (b) Take the offspring distribution to be $P [G = 2] = 1$ . Then $Z_{n + 1} = 2 Z_{n}$ , so $Z_{n} = 2^{n}$ , and hence $Z_{n} \to \infty$ .
7.7
- (a) We have
  
  $\begin{aligned} E [M_{n + 1} | F_{n}] & = E [M_{n + 1} 𝟙 {n^{t h} draw is red} | F_{n}] + E [M_{n + 1} 𝟙 {n^{t h} draw is black} | F_{n}] \\ = E [\frac{B_{n}}{n + 3} 𝟙 {n^{t h} draw is red} | F_{n}] + E [\frac{B_{n} + 1}{n + 3} 𝟙 {n^{t h} draw is black} | F_{n}] \\ = \frac{B_{n}}{n + 3} E [𝟙 {n^{t h} draw is red} | F_{n}] + \frac{B_{n} + 1}{n + 3} E [𝟙 {n^{t h} draw is black} | F_{n}] \\ = \frac{B_{n}}{n + 3} \frac{B_{n}}{n + 2} + \frac{B_{n} + 1}{n + 3} (1 - \frac{B_{n}}{n + 2}) \\ = \frac{B_{n}^{2}}{(n + 2) (n + 3)} + \frac{B_{n} (n + 2) + (n + 2) - B_{n}^{2} - B_{n}}{(n + 2) (n + 3)} \\ = \frac{(n + 1) B_{n} + (n + 2)}{(n + 2) (n + 3)} \end{aligned}$ This is clearly not equal to $M_{n} = \frac{B_{n}}{n + 2}$ , so $M_{n}$ is not a martingale.
  
  This result might be rather surprising, but we should note that the urn process here is not ‘fair’, when considered in terms of the proportion of (say) red balls. If we started with more red balls than black balls, then over time we would expect to see the proportion of red balls reduce towards $\frac{1}{2}$ , as we see in part (b):
- (b) Your conjecture should be that, regardless of the initial state of the urn, $P [M_{n} \to \frac{1}{2} as n \to \infty] = 1$ . (It is true.)
7.8
- (a) Let $B_{n}$ be the number of red balls in the urn at time $n$ , and then $M_{n} = \frac{B_{n}}{K}$ . Let us abuse notation slightly and write $B = r$ for the event that the ball $X$ is red, and $X = b$ for the event that the ball $X$ is black. Let $X_{1}$ denote the first ball drawn on the $(n + 1)^{t h}$ draw, and $X_{2}$ denote the second ball drawn.
  
  Let $F_{n} = σ (B_{1}, \dots, B_{n})$ . It is clear that $M_{n} \in [0, 1]$ , so $M_{n} \in L^{1}$ , and that $M_{n}$ is measurable with respect to $F_{n}$ . We have
  
  $\begin{aligned} E [M_{n + 1} | F_{n}] & = E [M_{n + 1} 𝟙_{{X_{1} = r, X_{2} = r}} | F_{n}] + E [M_{n + 1} 𝟙_{{X_{1} = r, X_{2} = b}} | F_{n}] \\ + E [M_{n + 1} 𝟙_{{X_{1} = b, X_{2} = r}} | F_{n}] + E [M_{n + 1} 𝟙_{{X_{1} = b, X_{2} = b}} | F_{n}] \\ = E [\frac{B_{n}}{K} 𝟙_{{X_{1} = r, X_{2} = r}} | F_{n}] + E [\frac{B_{n} + 1}{K} 𝟙_{{X_{1} = r, X_{2} = b}} | F_{n}] \\ + E [\frac{B_{n} - 1}{K} 𝟙_{{X_{1} = b, X_{2} = r}} | F_{n}] + E [\frac{B_{n}}{K} 𝟙_{{X_{1} = b, X_{2} = b}} | F_{n}] \\ = \frac{B_{n}}{K} E [𝟙_{{X_{1} = r, X_{2} = r}} | F_{n}] + \frac{B_{n} + 1}{K} E [𝟙_{{X_{1} = r, X_{2} = b}} | F_{n}] \\ + \frac{B_{n} - 1}{K} E [𝟙_{{X_{1} = b, X_{2} = r}} | F_{n}] + \frac{B_{n}}{K} E [𝟙_{{X_{1} = b, X_{2} = b}} | F_{n}] \\ = \frac{B_{n}}{K} \frac{B_{n}}{K} \frac{B_{n}}{K} + \frac{B_{n} + 1}{K} \frac{B_{n}}{K} \frac{K - B_{n}}{K} \\ + \frac{B_{n} - 1}{K} \frac{K - B_{n}}{K} \frac{B_{n}}{K} + \frac{B_{n}}{K} \frac{K - B_{n}}{K} \frac{K - B_{n}}{K} \\ = \frac{1}{K^{3}} (B_{n}^{3} + 2 B_{n}^{2} (K - B_{n}) + B_{n} (K - B) n - B_{n} (K - B_{n}) + B_{n} (K - B_{n})^{2}) \\ = \frac{1}{K^{3}} (K^{2} B_{n}) \\ = M_{n} \end{aligned}$
- (b) Since $M_{n} \in [0, 1]$ we have $E [| M_{n} |] \leq 1$ , hence $M_{n}$ is uniformly bounded in $L^{1}$ . Hence, by the martingale convergence theorem there exists a random variable $M_{\infty}$ such that $M_{n} \overset{a . s .}{\to} M_{\infty}$ .
  
  Since $M_{n} = \frac{B_{n}}{K}$ , this means that also $B_{n} \overset{a . s .}{\to} K M_{\infty}$ . However, $B_{n}$ is integer valued so, by Lemma 7.4.1, if $B_{n}$ is to converge then it must eventually become constant. This can only occur if, eventually, the urn contains either (i) only red balls or (ii) only black balls.
  
  In case (i) we have $M_{n} = 0$ eventually, which means that $M_{\infty} = 0$ . In case (ii) we have $M_{n} = 1$ eventually, which means that $M_{\infty} = 1$ . Hence, $M_{\infty} \in {0, 1}$ almost surely.
7.9
- (a) Consider the usual Pólya urn, from Section 4.2, started with just one red ball and one black ball. After the first draw is complete we have two possibilities:
  - 1. The urn contains one red ball and two black balls.
  - 2. The urn contains two red balls and one black ball.
  In the first case we reach precisely the initial state of the urn described in 7.9. In the second case we also reach the initial state of the urn described in 7.9, but with the colours red and black swapped.
  
  If $M_{n} \overset{a . s .}{\to} 0$ , then in the first case the fraction of red balls would tend to zero, and (by symmetry of colours) in the second case the limiting fraction of black balls would tend to zero; thus the limiting fraction of black balls would always be either $1$ (the first case) or $0$ (the second case). However, we know from Proposition 7.4.3 that the limiting fraction of black balls is actually uniform on $(0, 1)$ . Hence, $P [M_{n} \to 0] = 0$ .
- (b) You should discover that having a higher proportion of initial red balls makes it more likely for $M_{\infty}$ (the limiting proportion of red balls) to be closer to $1$ . However, since $M_{\infty}$ is a random variable in this case, you may need to take several samples of the process (for each initial condition that you try) to see the effect.
  
  Follow-up challenge question: In fact, the limiting value $M_{\infty}$ has a Beta distribution, $B e (α, β)$ . You may need to look up what this means, if you haven’t seen the Beta distribution before. Consider starting the urn with $n_{1}$ red and $n_{2}$ black balls, and see if you can use your simulations to guess a formula for $α$ and $β$ in terms of $n_{1}$ and $n_{2}$ (the answer is a simple formula!).
7.10
- (a) We have that $(M_{n})$ is a non-negative martingale, hence $sup_{n} E [| M_{n} |] = sup_{n} E [M_{n}] = E [M_{0}] = 1$ . Thus $(M_{n})$ is uniformly bounded in $L^{1}$ and the (first) martingale convergence theorem applies, giving that $M_{n} \overset{a . s .}{\to} M_{\infty}$ for a real valued $M_{\infty}$ . Since $M_{n} \geq 0$ almost surely we must have $M_{\infty} \geq 0$ almost surely.
- (b) The possible values of $M_{n}$ are $W = {(q / p)^{z}; z \in Z}$ . Note that $M_{n + 1} \neq M_{n}$ for all $n \in N$ .
  
  On the event that $M_{\infty} > 0$ there exists $ϵ > 0$ such that $M_{n} \geq ϵ$ for all sufficiently large $n$ . Choose $m \in N$ such that $(q / p)^{m} \leq ϵ$ and then for all sufficiently large $n$ we have $M_{n} \in W_{m} = {(q / p)^{z}; z \geq m, z \in Z}$ . For such $n$ , since $M_{n} \neq M_{n + 1}$ we have $| M_{n} - M_{n + 1} | \geq | (q / p)^{m} - (q / p)^{m - 1} | > 0$ , which implies that $(M_{n})$ could not converge. Hence, the only possible almost sure limit is $M_{\infty} = 0$ .
  
  If $(M_{n})$ was uniformly bounded in $L^{2}$ then by the (second) martingale convergence theorem we would have $E [M_{n}] \to E [M_{\infty}]$ . However $E [M_{n}] = E [M_{0}] = 1$ and $E [M_{\infty}] = 0$ , so this is a contradiction. Hence $(M_{n})$ is not uniformly bounded in $L^{2}$ .
- (c) We have $(q / p)_{n}^{S} \overset{a . s .}{\to} 0$ . Noting that $q / p > 1$ and hence $\log (q / p) > 0$ , taking logs gives that $S_{n} \log (q / p) \overset{a . s .}{\to} - \infty$ , which gives $S_{n} \overset{a . s .}{\to} - \infty$ .
  
  For the final part, if $(S_{n})$ is a simple asymmetric random walk with upwards drift, then $(- S_{n})$ is a simple asymmetric random walk with downwards drift. Applying what we’ve just discovered thus gives $- S_{n} \overset{a . s .}{\to} - \infty$ , and multiplying by $- 1$ gives the required result.
7.11
- (a) This is a trivial induction: if $S_{n}$ is even then $S_{n + 1}$ is odd, and vice versa.
- (b) By part (a), $p_{2 n + 1} = 0$ since $0$ is an even number and $S_{2 n + 1}$ is an odd number.
  
  If $S_{n}$ is to start at zero and reach zero at time $2 n$ , then during time $1, 2, \dots, 2 n$ it must have made precisely $n$ steps upwards and precisely $n$ steps downwards. The number of different ways in which the walk can do this is $(\binom{2 n}{n})$ (we are choosing exactly $n$ steps on which to go upwards, out of $2 n$ total steps). Each one of these ways has probability $(\frac{1}{2})^{2 n}$ of occurring, so we obtain $p_{2 n} = (\binom{2 n}{n}) 2^{- 2 n}$ .
- (c) From (b) we note that
  
  $p_{2 (n + 1)} = \frac{(2 n + 2) (2 n + 1)}{(n + 1) (n + 1)} 2^{- 2} p_{2 n} = 2 (\frac{2 n + 1}{n + 1}) 2^{- 2} p_{n} = (\frac{2 n + 1}{2 n + 2}) p_{2 n} = (1 - \frac{1}{2 (n + 1)}) \frac{1}{2} p_{2 n} .$
  
  Hence, using the hint, we have
  
  $p_{2 (n + 1)} \leq \exp (- \frac{1}{2 (n + 1)}) p_{2 n} .$
  
  Iterating this formula obtains that
  
  $\begin{aligned} p_{2 n} & \leq \exp (- \frac{1}{2 n}) \exp (- \frac{1}{2 (n - 1)}) \dots \exp (- \frac{1}{2 (1)}) p_{0} \\ (A.7) & = \exp (- \frac{1}{2} \sum_{i = 1}^{n} \frac{1}{i}) . \end{aligned}$ Recall that $\sum_{i = 1}^{n} \frac{1}{i} \to \infty$ as $n \to \infty$ , and that $e^{- x} \to 0$ as $x \to \infty$ . Hence, the right hand side of the expression above tends to zero as $n \to \infty$ . Hence, since $p_{n} \geq 0$ , by the sandwich we have $p_{2 n} \to 0$ as $n \to \infty$ .
7.12
- (a) We note that
  
  $\begin{aligned} E [\frac{S_{n + 1}}{f (n + 1)} | F_{n}] & = \frac{S_{n}}{f (n + 1)} + \frac{1}{f (n + 1)} E [X_{n + 1} | F_{n}] \\ = \frac{S_{n}}{f (n + 1)} + \frac{1}{f (n + 1)} E [X_{n + 1}] \\ = \frac{S_{n}}{f (n + 1)} \end{aligned}$ Here we use that $S_{n} \in m F_{n}$ and that $X_{n + 1}$ is independent of $F_{n}$ .
  
  Since $\frac{S_{n}}{f (n)}$ is a martingale we must have
  
  $\begin{matrix} (A.8) & \frac{S_{n}}{f (n + 1)} = \frac{S_{n}}{f (n)} . \end{matrix}$
  
  Since $P [S_{n} \neq 0] > 0$ this implies that $\frac{1}{f (n + 1)} = \frac{1}{f (n)}$ , which implies that $f (n + 1) = f (n)$ . Since this holds for any $n$ , we have $f (1) = f (n)$ for all $n$ ; so $f$ is constant.
- (b) If $\frac{S_{n}}{f (n)}$ is to be a supermartingale then, in place of (A.8), we would need
  
  $\begin{matrix} (A.9) & \frac{S_{n}}{f (n + 1)} \leq \frac{S_{n}}{f (n)} . \end{matrix}$
  
  Note that we would need this equation to hold true with probability one. To handle the $\geq$ we now need to care about the sign of $S_{n}$ . The key idea is that $S_{n}$ can be both positive or negative, so the only way that (A.9) can hold is if $\frac{1}{f (n + 1)} = \frac{1}{f (n)}$ .
  
  We will use some of the results from the solution of exercise 7.11. In particular, from for odd $n \in N$ from 7.11(b) we have $P [S_{n} = 0] = 0$ , and for even $n \in N$ from (A.7) we have $P [S_{n} = 0] < 1$ . In both cases we have $P [S_{n} \neq 0] > 0$ . Since $S_{n}$ and $- S_{n}$ have the same distribution we have $P [S_{n} < 0] = P [- S_{n} < 0] = P [S_{n} > 0]$ , and hence
  
  $P [S_{n} \neq 0] = P [S_{n} < 0] + P [S_{n} > 0] = 2 P [S_{n} < 0] = 2 P [S_{n} > 0] .$
  
  Hence, for all $n \in N$ we have $P [S_{n} > 0] > 0$ and $P [S_{n} < 0] > 0$ .
  
  Now, consider $n \geq 1$ . We have that here is positive probability that $S_{n} > 0$ . Hence, by (A.9), we must have $\frac{1}{f (n + 1)} \leq \frac{1}{f (n)}$ , which means that $f (n + 1) \geq f (n)$ . But, there is also positive probability that $S_{n} < 0$ , hence by (A.9) we must have $\frac{1}{f (n + 1)} \geq \frac{1}{f (n)}$ , which means that $f (n + 1) \leq f (n)$ . Hence $f (n + 1) = f (n)$ , for all $n$ , and by a trivial induction we have $f (1) = f (n)$ for all $n$ .
7.13
- (a) Since $M_{n} = \frac{Z_{n}}{μ^{n}}$ we note that
  
  $\begin{array}{r} (A.10) & \frac{1}{μ^{2 (n + 1)}} (Z_{n + 1} - μ Z_{n})^{2} = (M_{n + 1} - M_{n})^{2} . \end{array}$ Further,
  
  $Z_{n + 1} - μ Z_{n} = \sum_{i = 1}^{Z_{n}} (X_{i}^{n + 1} - μ)$
  
  so it makes sense to define $Y_{i} = X_{i}^{n + 1} - μ$ . Then $E [Y_{i}] = 0$ ,
  
  $E [Y_{i}^{2}] = E [(X_{i}^{n + 1} - μ)^{2}] = var (X_{i}^{n + 1}) = σ^{2},$
  
  and $Y_{1}, Y_{2}, \dots, Y_{n}$ are independent. Moreover, the $Y_{i}$ are identically distributed and independent of $F_{n}$ . Hence, by taking out what is known ( $Z_{n} \in m F_{n}$ ) we have
  
  $\begin{aligned} E [(Z_{n + 1} - μ Z_{n})^{2} | F_{n}] & = E [{(\sum_{i = 1}^{Z_{n}} Y_{i})}^{2} | F_{n}] \\ = \sum_{i = 1}^{Z_{n}} E [Y_{i}^{2} | F_{n}] + \sum_{\overset{i, j = 1}{i \neq j}}^{Z_{n}} E [Y_{i} Y_{j} | F_{n}] \\ = \sum_{i = 1}^{Z_{n}} E [Y_{i}^{2}] + \sum_{\overset{i, j = 1}{i \neq j}}^{Z_{n}} E [Y_{i}] E [Y_{j}] \\ = Z_{n} E [Y_{1}]^{2} + 0 \\ = Z_{n} σ^{2} . \end{aligned}$ So, from (A.10) we obtain
  
  $E [(M_{n + 1} - M_{n})^{2} | F_{n}] = \frac{Z_{n} σ^{2}}{μ^{2 (n + 1)}} .$
- (b) By part (a) and (3.4), we have
  
  $E [M_{n + 1}^{2} | F_{n}] = M_{n}^{2} + \frac{Z_{n} σ^{2}}{μ^{2 (n + 1)}} .$
  
  Taking expectations, and using that $E [Z_{n}] = μ^{n}$ , we obtain
  
  $E [M_{n + 1}^{2}] = E [M_{n}^{2}] + \frac{σ^{2}}{μ^{n + 2}} .$
7.14
- (a) Note that $X_{n + 1} \geq X_{n} \geq 0$ for all $n$ , because taking an $inf$ over $k > n + 1$ is an infimum over a smaller set than over $k > n$ . Hence, $(X_{n})$ is monotone increasing and hence the limit $X_{\infty} (ω) = lim_{n \to \infty} X_{n} (ω)$ exists (almost surely).
- (b) Since $| M_{n} | \leq inf_{k > n} | M_{k} |$ , we have $X_{n} \leq | M_{n} |$ . By definition of $inf$ , for each $ϵ > 0$ and $n \in N$ there exists some $n^{'} \geq n$ such that $X_{n} \geq | X_{n^{'}} | - ϵ$ . Combining these two properties, we obtain
  
  $| M_{n^{'}} | - ϵ \leq X_{n} \leq | M_{n} | .$
- (c) Letting $n \to \infty$ , in the above equation, which means that also $n^{'} \to \infty$ because $n^{'} \geq n$ , we take an almost sure limit to obtain $| M_{\infty} | - ϵ \leq X_{\infty} \leq | M_{\infty} |$ . Since we have this inequality for any $ϵ > 0$ , in fact $X_{\infty} = | M_{\infty} |$ almost surely.
- (d) We have shown that $(X_{n})$ is an increasing sequence and $X_{n} \overset{a . s .}{\to} X_{\infty}$ , and since each $| M_{n} | \geq 0$ , we have also $X_{n} \geq 0$ . Hence, the monotone convergence theorem applies to $X_{n}$ and $X_{\infty}$ , and therefore $E [X_{n}] \to E [X_{\infty}]$ .
- (e) From the monotone convergence theorem, writing in terms of $M_{n}$ rather than $X_{n}$ , we obtain
  
  $\begin{matrix} (A.11) & E [inf_{k \geq n} | M_{n} |] \to E [| M_{\infty} |] . \end{matrix}$
  
  However, since $inf_{k \geq n} | M_{n} | \leq | M_{n} | \leq C = sup_{j \in N} E [| M_{j} |]$ , the monotonicity property of expectation gives us that, for all $n$ ,
  
  $\begin{matrix} (A.12) & E [inf_{k \geq n} | M_{n} |] \leq C . \end{matrix}$
  
  Combining (A.11) and (A.12), with the fact that limits preserve weak inequalities, we obtain that $E [| M_{\infty} |] \leq C$ , as required.
7.15
- (a) All the ${\tilde{X}}_{i}^{n + 1}$ have the same distribution, and are independent by independence of the $X_{i}^{n + 1}$ and $C_{i}^{n + 1}$ . Thus, ${\tilde{Z}}_{n}$ is a Galton-Watson process with off-spring distribution $\tilde{G}$ given by the common distribution of the $X_{i}^{n + 1}$ .
- (b) By definition of ${\tilde{X}}_{i}^{n + 1}$ we have $X_{i}^{n + 1} \leq {\tilde{X}}_{i}^{n + 1}$ . Hence, from (7.9), if $Z_{n} \leq {\tilde{Z}}_{n}$ then also $Z_{n + 1} \leq {\tilde{Z}}_{n + 1}$ . Since $Z_{0} = {\tilde{Z}}_{0} = 1$ , an easy induction shows that $Z_{n} \leq {\tilde{Z}}_{n}$ for all $n \in N$ .
- (c) We have
  
  $\begin{aligned} f (α) & = E [{\tilde{X}}_{i}^{n + 1}] \\ = P [G = 0] P [C = 1] + \sum_{i = 1}^{\infty} i P [G = i] \\ = α P [G = 0] + \sum_{i = 1}^{\infty} i P [G = i] . \end{aligned}$ Recall that each $X_{i}^{n + 1}$ has the same distribution as $G$ . Hence,
  
  $μ = E [X_{i}^{n + 1}] = E [G] = \sum_{i = 1}^{\infty} i P [G = i] = f (0) .$
  
  Since $μ < 1$ we therefore have $f (0) < 1$ . Moreover,
  
  $f (1) = P [G = 0] + \sum_{i = 1}^{\infty} i P [G = i] \geq \sum_{i = 0}^{\infty} P [G = i] = 1.$
  
  It is clear that $f$ is continuous (in fact, $f$ is linear). Hence, by the intermediate value theorem, there is some $α \in [0, 1]$ such that $f (α) = 1$ .
- (d) By (c), we can choose $α$ such that $f (α) = 1$ . Hence, $E [{\tilde{X}}_{i}^{n + 1}] = 1$ , so by (a) ${\tilde{Z}}_{n}$ is a Galton-Watson process with an offspring distribution that has mean $\hat{μ} = 1$ . By Lemma 7.4.6, $\tilde{Z}$ dies out, almost surely. Since, from (b), $0 \leq Z_{n} \leq {\tilde{Z}}_{n}$ , this means $Z_{n}$ must also die out, almost surely.

Chapter 8

8.1 We have $U^{- 1} < 1$ so, for all $ω \in Ω$ , $X_{n} (ω) = U^{- n} (ω) \to 0$ as $n \to \infty$ . Hence $X_{n} (ω) \to 0$ almost surely as $n \to \infty$ . Further, $| X_{n} | \leq 1$ for all $n \in N$ and $E [1] = 1 < \infty$ , so the dominated convergence theorem applies and shows that $E [X_{n}] \to E [0] = 0$ as $n \to \infty$ .
8.2 We check the two conditions of the dominated convergence theorem. To check the first condition, note that if $n \geq | X |$ then $X 𝟙_{{| X | \leq n}} = X$ . Hence, since $| X | < \infty$ , as $n \to \infty$ we have $X_{n} = X 𝟙_{{| X | \leq n}} \to X$ almost surely.

To check the second condition, set $Y = | X |$ and then $E [| Y |] = E [| X |] < \infty$ so $Y \in L^{1}$ . Also, $| X_{n} | = | X 𝟙_{{| X | \leq n}} | \leq | X | = Y$ , so $Y$ is a dominated random variable for $(X_{n})$ . Hence, the dominated convergence theorem applies and $E [X_{n}] \to E [X]$ .
8.3
- (a) We look to use the dominated convergence theorem. For any $ω \in Ω$ we have $Z (ω) < \infty$ , hence for all $n \in N$ such that $n > Z (ω)$ we have $X_{n} (ω) = 0$ . Therefore, as $n \to \infty$ , $X_{n} (ω) \to 0$ , which means that $X_{n} \to 0$ almost surely.
  
  We have $| X_{n} | \leq Z$ and $Z \in L^{1}$ , so we can use $Z$ are the dominating random variable. Hence, by the dominated convergence theorem, $E [X_{n}] \to E [0] = 0$ .
- (b) We have
  
  $E [Z] = \int_{1}^{\infty} x f (x) d x = \int_{1}^{\infty} x^{- 1} d x = {[\log x]}_{1}^{\infty} = \infty$
  
  which means $Z \notin L^{1}$ and also that
  
  $\begin{aligned} E [X_{n}] & = E [𝟙_{{Z \in [n, n + 1)}} Z] = \int_{1}^{\infty} 𝟙_{{x \in [n, n + 1)}} x f (x) d x \\ = \int_{n}^{n + 1} x^{- 1} d x = {[\log x]}_{n}^{n + 1} = \log (n + 1) - \log n = \log (\frac{n + 1}{n}) . \end{aligned}$ As $n \to \infty$ , we have $\frac{n + 1}{n} = 1 + \frac{1}{n} \to 1$ , hence (using that $\log$ is a continuous function) we have $\log (\frac{n + 1}{n}) \to \log 1 = 0$ . Hence, $E [X_{n}] \to 0$ .
- (c) Suppose that we wanted to use the DCT in (b). We still have $X_{n} \to 0$ almost surely, but any dominating random variable $Y$ would have to satisfy $Y \geq | X_{n} |$ for all $n$ , meaning that also $Y \geq Z$ , which means that $E [Y] \geq E [Z] = \infty$ ; thus there is no dominating random variable $Y \in L^{1}$ . Therefore, we can’t use the DCT here, but we have shown in (b) that the conclusion of the DCT does hold: we have that $E [X_{n}]$ does tend to zero.
  
  We obtain that the conditions of the DCT are sufficient but not necessary for its conclusion to hold.
8.4 Note that ${T = n} = {T \leq n} ∖ {T \leq n - 1} .$ If $T$ is a stopping time then both parts of the right hand side are elements of $F_{n}$ , hence so is ${T = n}$ .

Conversely, if ${T = n} \in F_{n}$ for all $n$ , then for $i \leq n$ we have ${T = i} \in F_{i} \subseteq F_{n}$ . Hence the identity ${T \leq n} = \cup_{i = 1}^{n} {T = i}$ shows that ${T \leq n} \in F_{n}$ , and therefore $T$ is a stopping time.
8.5 We can write ${R = n} = {S_{i} \neq 0 for i = 1, \dots, n - 1} \cap {S_{n} = 0} = {S_{n} = 0} \cap (⋂_{i = 1}^{n} Ω ∖ {S_{i} = 0})$ . By exercise 8.4 we have ${S_{i} = 0} \in F_{n}$ for all $i \leq n$ , hence ${R = n} \in F_{n}$ for all $n$ .
8.6 We can write

${S + T \leq n} = ⋃_{k = 0}^{n} ⋃_{j = 0}^{n - k} {S \leq j} \cap {T \leq k}$

Since $S$ and $T$ are both stopping times we have ${T \leq n}, {S \leq n} \in F_{n}$ for all $n \in N$ , which means that ${T \leq j}, {S \leq k} \in F_{n}$ for all $j, k \leq n$ . Hence ${S + T \leq n} \in F_{n}$ and therefore $S + T$ is also a stopping time.
8.7 If $A \in F_{S}$ then $A \cap {S \leq n} \in F_{n}$ . Since $T$ is a stopping time we have ${T \leq n} \in F_{n}$ , hence

$A \cap {S \leq n} \cap {T \leq n} \in F_{n} .$

As $S \leq T$ we have ${T \leq n} \subseteq {S \leq n}$ , hence $A \cap {T \leq n} \in F_{n}$ .
8.8
- (a) We have
  
  $T = inf {n > 0; B_{n} = 2} .$
  
  Note that $T \geq n$ if and only if $B_{i} = 1$ for all $i = 1, 2, \dots, n - 1$ . That is, if and only if we pick a red ball out of the urn at times $i - 1, 2, \dots, n - 1$ . Hence,
  
  $\begin{aligned} P [T \geq n] & = \frac{1}{2} \frac{2}{3} \dots \frac{n - 1}{n} \\ = \frac{1}{n} . \end{aligned}$ Therefore, since $P [T = \infty] \leq P [T \geq n]$ for all $n$ , we have $P [T = \infty] = 0$ and $P [T < \infty] = 1$ .
- (b) We note that $T$ is a stopping time, with respect to $F_{t} = σ (B_{i}; i \leq n)$ , since
  
  ${T \leq n} = {a red ball was drawn at some time i \leq n} = ⋃_{i = 1}^{n} {B_{i} = 2} .$
  
  We showed in Section 4.2 that $M_{n}$ was a martingale. Since $M_{n} \in [0, 1]$ the process $(M_{n})$ is bounded. By part (a) for any $n \in N$ we have $P [T = \infty] \leq P [T \geq n] = \frac{1}{n}$ , hence $P [T = \infty] = 0$ and $P [T < \infty] = 1$ . Hence, we have condition (b) of the optional stopping theorem, so
  
  $E [M_{T}] = E [M_{0}] = \frac{1}{2} .$
  
  By definition of $T$ we have $B_{T} = 2$ . Hence $M_{T} = \frac{2}{T + 2}$ , so we obtain $E [\frac{1}{T + 2}] = \frac{1}{4}$ .
8.9
- (a) We showed in exercise 7.8 that, almost surely, there was a point in time at which the urn contained either all red balls or all black balls. Hence, $P [T < \infty] = 1$ .
  
  Moreover, almost surely, since the urn eventually contains either all red balls or all black balls, we have either $M_{T} = 1$ (all red balls) or $M_{T} = 0$ (all black balls).
- (b) We note that $T$ is a stopping time, since
  
  ${T \leq n} = {for some i \leq n we have M_{i} \in {0, 1}} = ⋃_{i = 0}^{n} M_{i}^{- 1} ({0, 1}) .$
  
  In exercise 7.8 we showed that $M_{n}$ was a martingale. Since $M_{n} \in [0, 1]$ in fact $M_{n}$ is a bounded martingale, and we have $P [T < \infty]$ from above, so conditions (b) of the optional stopping theorem apply. Hence,
  
  $E [M_{T}] = E [M_{0}] = \frac{r}{r + b} .$
  
  Lastly, since $M_{T}$ is either $0$ or $1$ (almost surely) we note that
  
  $\begin{aligned} E [M_{T}] & = E [M_{T} 𝟙_{{M_{T} = 1}} + M_{T} 𝟙_{{M_{T}] = 0}}] \\ = E [𝟙_{{M_{T} = 1}}] + 0 \\ = P [M_{T} = 1] . \end{aligned}$ Hence, $P [M_{T} = 1] = \frac{r}{r + b}$ .
8.10
- (a) We use the filtration $F_{n} = σ (N_{i}; i \leq n)$ . We have $P_{n} \in m F_{n}$ and since $0 \leq P_{n} \leq 1$ we have also that $P_{n} \in L^{1}$ . Also,
  
  $\begin{aligned} E [P_{n + 1} | F_{n}] & = E [P_{n + 1} 𝟙 {the n^{t h} ball was red} | F_{n}] + E [P_{n + 1} 𝟙 {the n^{t h} ball was blue} | F_{n}] \\ = E [\frac{N_{n} - 1}{2 m - n - 1} 𝟙 {the n^{t h} ball was red} | F_{n}] + E [\frac{N_{n}}{2 m - n - 1} 𝟙 {the n^{t h} ball was blue} | F_{n}] \\ = \frac{N_{n} - 1}{2 m - n - 1} E [𝟙 {the n^{t h} ball was red} | F_{n}] + \frac{N_{n}}{2 m - n - 1} E [𝟙 {the n^{t h} ball was blue} | F_{n}] \\ = \frac{N_{n} - 1}{2 m - n - 1} \frac{N_{n}}{2 m - n} + \frac{N_{n}}{2 m - n - 1} \frac{2 m - n - N_{n}}{2 m - n} \\ = \frac{N_{n}}{2 m - n} \\ = P_{n} . \end{aligned}$ Here we use taking out what is known (since $N_{n} \in m F_{n}$ ), along with the definition of our urn process to calculate e.g. $E [𝟙 {the n^{t h} ball was red} | F_{n}]$ as a function of $N_{n}$ . Hence $(P_{n})$ is a martingale.
- (b) Since $0 \leq P_{n} \leq 1$ , the process $(P_{n})$ is a bounded martingale. The time $T$ is bounded above by $2 m$ , hence condition (b) for the optional stopping theorem holds and $E [P_{T}] = E [P_{0}]$ . Since $P_{0} = \frac{1}{2}$ this gives us $E [P_{T}] = \frac{1}{2}$ . Hence,
$\begin{aligned} P [(T + 1)^{s t} ball is red] & = P [N_{T + 1} = N_{T} + 1] \\ = \sum_{i = 1}^{2 m - 1} P [N_{T + 1} = N_{T} + 1 | T = i] P [T = i] \\ = \sum_{i = 1}^{2 m - 1} \frac{m - 1}{2 m - i} P [T = i] \\ = E [P_{T}] \\ = \frac{1}{2} . \end{aligned}$ as required.

Chapter 9

9.1
- (a) We have that $T$ is a stopping time and (from Section 4.1) $M_{n} = S_{n} - (p - q) n$ is a martingale. We want to apply the optional stopping theorem to $(M_{n})$ .
  
  We have $E [T] < \infty$ from Lemma 9.1.1 and for all $n$ we have
  
  $| M_{n + 1} - M_{n} | \leq | S_{n + 1} - S_{n} | + | p - q | = 1 + | p - q | .$
  
  Thus, condition (c) for the optional stopping theorem applies and $E [M_{T}] = E [M_{0}]$ . That is,
  
  $E [S_{T} - (T) (p - q)] = E [S_{0} - (0) (p - q)] = E [S_{0} - 0 (p - q)] = 0.$
  
  We thus have $E [S_{T}] = (p - q) E [T]$ as required.
- (b) We have
  
  $\begin{aligned} E [S_{T}] & = E [S_{T} 𝟙 {T = T_{a}} + S_{T} 𝟙 {T = T_{b}}] \\ = E [a 𝟙 {T = T_{a}} + b 𝟙 {T = T_{b}}] \\ = a P [T = T_{a}] + b P [T = T_{b}] \end{aligned}$ Putting equations (9.5) and (9.6) into the above, and then using part (a) gives us
  
  $E [T] = \frac{1}{p - q} (a \frac{(q / p)^{b} - 1}{(q / p)^{b} - (q / p)^{a}} + b \frac{1 - (q / p)^{a}}{(q / p)^{b} - (q / p)^{a}}) .$
9.2
- (a) We have $P [T_{a} < \infty] = P [T_{b} > \infty] = 1$ from Lemma 9.3.2. Hence also $P [T = T_{a} < \infty or T = T_{b} < \infty] = 1$ . Since these two possibilities are disjoint, in fact $P [T = T_{a}] + P [T = T_{b}] = 1$ .
  
  For the second equation, we will use that $(S_{n})$ is a martingale. We want to apply the optional stopping theorem, but none of the conditions apply directly to $(S_{n})$ . Instead, set $M_{n} = S_{n \land T}$ and note that Lemma 8.2.4 implies that $(M_{n})$ is a martingale. By definition of $T$ we have $| M_{n} | \leq max {| a |, | b |}$ , hence $(M_{n})$ is bounded. We have $P [T < \infty]$ from above, hence conditions (b) of the optional stopping theorem apply to $(M_{n})$ . We thus deduce that
  
  $E [M_{T}] = E [M_{0}] = E [S_{0}] = 0.$
  
  Hence
  
  $\begin{aligned} E [M_{T}] & = E [M_{T} 𝟙_{{T = T_{a}}} + M_{T} 𝟙_{{T = T_{b}}}] \\ = E [M_{T_{a}} 𝟙_{{T = T_{a}}}] + E [M_{T_{b}} 𝟙_{{T = T_{b}}}] \\ = E [S_{T_{a}} 𝟙_{{T = T_{a}}}] + E [S_{T_{a}} 𝟙_{{T = T_{a}}}] \\ = E [a 𝟙_{{T = T_{a}}}] + E [b 𝟙_{{T = T_{a}}}] \\ = a P [T = T_{a}] + b P [T = T_{a}] \end{aligned}$ as required. In the above, the first line partitions on the value of $T$ (using what we already deduced). The third line follows from the second because $T a \leq T$ , hence $M_{T_{a}} = S_{T_{a} \land T} = S_{T_{a}}$ , and similarly for $T_{b}$ .
- (b) Solving the two equations (this is left for you) gives that $P [T = T_{a}] = \frac{b}{b - a}$ and $P [T = T_{b}] = \frac{- a}{b - a}$ .
- (c) From Exercise 4.4 we have that $S_{n}^{2} - n$ is a martingale. Again, none of the conditions of optional stopping directly apply, but $X_{n} = S_{n \land T}^{2} - (n \land T)$ is a martingale by Theorem 8.2.4. Similarly to above, we have $| M_{n} | \leq max {a^{2} + | a |, b^{2} + | b |}$ , so $(X_{n})$ is bounded. Hence conditions (b) of the optional stopping theorem apply and
  
  $E [X_{T}] = E [X_{0}] = E [S_{0}^{2} - 0 =] = 0.$
  
  Thus $E [S_{T}^{2}] = E [T]$ , which gives
  
  $\begin{aligned} E [T] & = E [S_{T}^{2} 𝟙_{{T = T_{a}}} + S_{T}^{2} 𝟙_{{T = T_{a}}}] \\ = E [a^{2} 𝟙_{{T = T_{a}}} + b^{2} 𝟙_{{T = T_{a}}}] \\ = a^{2} P [T = T_{a}] + b^{2} P [T = T_{b}] \\ = \frac{a^{2} b}{b - 1} + \frac{b^{2} (- a)}{b - a} \\ = - a b . \end{aligned}$ Here, the second line follows because when $T = T_{a}$ we have $S_{T} = S_{T_{a}} = a$ , hence also $S_{T_{a}}^{2} = a^{2}$ , and similarly for $S_{T_{b}}^{2}$ . The fourth line uses part (b).
9.3
- (a) Note that movements up at by $+ 2$ , whilst movements down are by $- 1$ . In order for $S_{n}$ to return to zero at time $n$ , it must have made twice as many movements down as up. In particular $n$ must therefore be a multiple of $3$ , so $P [S_{3 n + 1} = 0] = P [S_{3 n + 2} = 0] = 0$ .
  
  For $S_{3 n}$ , we can count the possible ways to have $n$ movements up and $2 n$ movements down by choosing $n$ upwards movements out of $3 n$ movements, that is $(\binom{3 n}{n})$ . Taking the probabilities into account and then using (9.7) we obtain
  
  $P [S_{3 n} = 0] = (\binom{3 n}{n}) {(\frac{1}{3})}^{n} {(\frac{2}{3})}^{2 n} = \frac{(3 n)!}{n! (2 n)!} \frac{2^{2 n}}{3^{3 n}} \sim \frac{\sqrt{6 π n} (3 n / e)^{3 n}}{\sqrt{8} π n (n / e)^{n} (2 n / e)^{2 n}} \frac{2^{2 n}}{3^{3 n}} = \frac{\sqrt{3}}{2 \sqrt{π n}} .$
- (b) The argument that $E [G] < \infty$ , under the assumption $P [R < \infty] = 1$ , is exactly the same as before.
  
  We need to modify the next part, beginning with (9.9) which is replaced by the result in part (a). Note that that $S_{3 n}$ now plays the role of what was previously $S_{2 n}$ , being the possible returns to zero. Having (a) introduces an extra factor of $\frac{\sqrt{3}}{2}$ , but using the same method as before we can still deduce that $E [G] = \infty$ . We thus arrive at a contradiction in the same way.
9.4
- (a) We have $E [X_{i}] = (p) (1) + (1 - \frac{p}{2}) (- 1 - 2) = \frac{5 p}{2} - 3$ . The condition $p \in (\frac{5}{6}, 1]$ implies that $E [X_{i}] > 0$ . Writing $μ = E [X_{i}]$ , it follows from the strong law of large numbers that (almost surely) there exists $N \in N$ such that for all $n \geq N$ we have $S_{n} \geq n \frac{μ}{2}$ . Hence $S_{n} \overset{a . s .}{\to} \infty$ .
- (b) We calculate
  
  $\begin{aligned} E [T_{1}] & = E [T_{1} 𝟙_{{S_{1} = 1}}] & + E [T_{1} 𝟙_{{S_{1} = - 1}}] & + E [T_{1} 𝟙_{{S_{1} = - 2}}] \\ = E [1 𝟙_{{S_{1} = 1}}] & + E [E [T_{1} 𝟙_{{S_{1} = - 1}} | F_{1}]] & + E [E [T_{1} 𝟙_{{S_{1} = - 2}} | F_{1}]] \\ = p & + E [𝟙_{{S_{1} = - 1}} E [T_{1} | F_{1}]] & + E [𝟙_{{S_{1} = - 2}} E [T_{1} | F_{1}]] \\ = p & + E [𝟙_{{S_{1} = - 1}} E [1 + T_{1}^{'} + T_{1}^{″} | F_{1}]] & + E [𝟙_{{S_{1} = - 2}} E [1 + T_{1}^{'} + T_{1}^{″} + T_{1}^{‴} | F_{1}]] \\ = p & + E [𝟙_{{S_{1} = - 1}} E [1 + T_{1}^{'} + T_{1}^{″}]] & + E [𝟙_{{S_{1} = - 1}} E [1 + T_{1}^{'} + T_{1}^{″} + T_{1}^{‴}]] \\ = p & + E [𝟙_{{S_{1} = - 1}} (1 + 2 E [T_{1}])] & + E [𝟙_{{S_{1} = - 1}} (1 + 3 E [T_{1}])] \\ = p & + \frac{1 - p}{2} (1 + 2 E [T_{1}]) & + \frac{1 - p}{2} (1 + 3 E [T_{1}]) . \end{aligned}$ In the first line of the above we partition on the value of $S_{1}$ . The first term of the second line follows because if $S_{1} = 1$ then $T_{1} = 1$ , and the other terms use the relationship between expectation and conditional expectation. The third line follows because $S_{1} \in F_{1}$ .
  
  The fourth line uses that, on the event $S_{1} = - 1$ , $T_{1}$ is equal to $1$ (accounting for the first move $0 \mapsto - 1$ ) plus two independent copies ( $T_{1}^{'}$ and $T_{1}^{″}$ ) of $T_{1}$ (accounting for the time to move from $- 1 \mapsto 0$ and then $0 \mapsto 1$ ). Similarly, on the event $S_{1} = - 1$ , $T_{1}$ is equal to $1$ plus three independent copies of $T_{1}$ . Note that it is crucial here that $(S_{n})$ can only jump upwards by $1$ in each step of time, meaning that it cannot go above $1$ without first being equal to $1$ .
  
  The strong Markov property gives that $T_{1}^{'}, T_{1}^{″}$ and $T_{1}^{‴}$ are independent of $F_{1}$ , leading to the fifth line. The remainder of the calculation is straightforward. We thus obtain
  
  $E [T_{1}] = p + \frac{1 - p}{2} (2 + 5 E [T_{t}]) .$
  
  Solving this equation gives $E [T_{1}] = \frac{2}{5 p - 3}$ .
9.5 Recall that $S_{n}$ even when $n$ even, and odd when $n$ is odd. Hence $T_{1}$ is odd. From the fact given in the question, there exists $N \in N$ such that for all $n \geq N$ we have $P [T_{1} = 2 n - 1] \geq \frac{1}{4 \sqrt{π} n^{3 / 2}}$ . Hence

$E [T_{1}] \geq \sum_{n = N}^{\infty} (2 n - 1) P [T_{1} = 2 n - 1] \geq \frac{1}{4 \sqrt{π}} \sum_{n = N}^{\infty} \frac{2 n - 1}{n^{3 / 2}} \geq \frac{1}{2 \sqrt{π}} \sum_{n = N}^{\infty} \frac{1}{n^{1 / 2}} = \infty$

as required.
9.6
- (a) Note that $M_{n}^{(θ)} = \prod_{i = 1}^{n} (e^{θ X_{i}} / \cosh θ)$ . Since $X_{i} \in m F_{n}$ for all $i \leq n$ , $S_{n} \in m F_{n}$ for all $n$ by Proposition 2.2.6. Since $| S_{n} | \leq n$ we have $| M_{n} | \leq \frac{e^{θ n}}{(\cosh θ)^{n}} < \infty$ , hence $M_{n} \in L^{1}$ . We have also that
  
  $\begin{aligned} E [M_{n + 1} | F_{n}] & = (\prod_{i = 1}^{n} \frac{e^{θ X_{i}}}{\cosh θ}) E [\frac{e^{θ X_{n + 1}}}{\cosh θ} | F_{n}] \\ = M_{n} E [\frac{e^{θ X_{n + 1}}}{\cosh θ}] \\ = M_{n} . \end{aligned}$ Here we use the taking out what is known rule, the fact that $X_{n + 1}$ is independent of $F_{n}$ and the relationship between conditional expectation and independence. To deduce the final line we note that $E [e^{θ X_{i}} / \cosh θ] = \frac{1}{2} (e^{θ} + e^{- θ}) / \cosh θ = 1$ .
  
  Note: This is a more general case of the martingale from Exercise 4.1.
- (b) Condition (1) fails because $T_{m}$ is not bounded – to see this, note that for any $N \in N$ there is a positive probability that $(S_{n})_{n = 0}^{N}$ remains below zero, in which case $T_{m} \geq N$ . Condition (b) fails because $(M_{n})$ is unbounded. Condition (c) fails because we have $E [T_{1}] = \infty$ by Lemma 9.3.4, and $T_{m} \geq T_{1}$ so also $E [T_{m}] = \infty$ .
- (c) By Lemma 8.2.4 the process $L_{n}^{(θ)} = M_{n \land T}^{(θ)}$ is a martingale, and by definition of $T$ we have that $\frac{e^{- m θ}}{(\cosh θ)^{n}} \leq L_{n}^{(θ)} \leq \frac{e^{m θ}}{(\cosh θ)^{n}}$ . Since $\cosh θ \geq 1$ , we thus have $| L_{n}^{(θ)} | \leq e^{m θ} + e^{- m θ}$ , so the martingale $(L_{n}^{θ})$ is bounded. By Lemma 9.3.2 $P [T < \infty] = 1$ , so conditions (b) of the optional stopping theorem apply to $(L_{n}^{(θ)})$ and $T$ , hence
  
  $E [\frac{e^{θ S_{T}}}{(\cosh θ)^{T}}] = E [\frac{e^{θ S_{0}}}{(\cosh θ)^{0}}] = 1.$
  
  By Lemma 9.3.2 we have $P [T < \infty] = 1$ , hence almost surely either $T = T_{m}$ or $T = T_{- m}$ . Hence,
  
  $\begin{aligned} 1 & = E [\frac{e^{θ S_{τ}}}{(\cosh θ)^{T}} 𝟙_{{T = T_{m}}} + \frac{e^{θ S_{τ}}}{(\cosh θ)^{T}} 𝟙_{{T = T_{- m}}}] \\ = E [\frac{e^{m θ}}{(\cosh θ)^{T_{m}}} 𝟙_{{T = T_{m}}}] + E [\frac{e^{- m θ}}{(\cosh θ)^{T_{- m}}} 𝟙_{{T = T_{- m}}}] \\ = e^{m θ} E [\frac{1}{(\cosh θ)^{T_{m}}} 𝟙_{{T = T_{m}}}] + e^{- m θ} E [\frac{1}{(\cosh θ)^{T_{- m}}} 𝟙_{{T = T_{- m}}}] \\ = (e^{m θ} + e^{- m θ}) E [\frac{1}{(\cosh θ)^{T_{m}}} 𝟙_{{T = T_{m}}}] \end{aligned}$ In the above, the final line follows by symmetry about zero, which implies that the two expectations are in fact equal. Thus $E [\frac{𝟙_{{T = T_{m}}}}{(\cosh θ)^{T_{m}}}] = \frac{1}{2 \cosh (m θ)}$ , for all $m \in N$ . With this in hand we can calculate
  
  $\begin{aligned} E [\frac{1}{(\cosh θ)^{T}}] & = E [\frac{𝟙_{{T = T_{m}}}}{(\cosh θ)^{T_{m}}}] + E [\frac{𝟙_{{T = T_{- m}}}}{(\cosh θ)^{T_{- m}}}] \\ = \frac{1}{2 \cosh (m θ)} + \frac{1}{2 \cosh (- m θ)} \\ = \frac{1}{\cosh (m θ)} . \end{aligned}$ In the first line we partition on whether $T$ is equal to $T_{m}$ or $T_{- m}$ . In the second line we apply the formulae deduced above at both $m$ and $- m$ , and to deduce the final line we recall that $\cosh (x) = \cosh (- x)$ .
  
  Note: $T_{m}$ and $𝟙_{{T = T_{m}}}$ are not independent, but if they were then it would be possible to write a simpler solution. Check that you didn’t fall into this trap.
9.7 We have show in Lemma 9.5.1 that $(S_{n})$ will almost surely return to the origin in finite time. By applying the strong Markov property at each return to the origin, we obtain that $(S_{n})$ will return to the origin infinitely often.

Fix some $z \in Z^{2}$ . There is a path from the origin to $z$ , and with positive probability the walk will take that path, hence $P [S_{n} = z for some n \in N]$ is positive. By the strong Markov property, this means the probability that $(S_{n})$ visits $z$ in between any two successive visits to the origin, is also positive. By the strong Markov property, whether this occur is independent, for each successive pair of returns to the origin. Hence we have infinitely many trials to try and visit $z$ , each with the same positive success probability. This will result in infinitely many visits to $z$ , almost surely.

This holds for any $z \in Z^{2}$ , which completes the argument.
9.8
- (a) If $L = \infty$ then the set ${n; S_{n} = 0}$ much be infinite, meaning that $(S_{n})$ has returned infinitely many times to the origin. However, this has probability zero; by the strong Markov property, after each visit there is a probability $P [R = \infty] > 0$ of not returning again, independently of the past; eventually that will happen.
- (b) $L$ is not a stopping time. One way to see this is to note that ${L = 0} = {R = \infty}$ , which depends on the whole path taken by the walk, so is not an element of $F_{n}$ for any $N \in N$ .
- (c) The event ${L = 2 n}$ is the event that $S_{2 n} = 0$ , but the walk is never zero after that time. Hence
  
  $\begin{aligned} P [L = 2 n] & = E [𝟙_{{L = 2 n}}] \\ = E [𝟙_{{S_{2 n} = 0}} 𝟙_{{S_{k} \neq 0 for all k > 2 n}}] \\ = E [E [𝟙_{{S_{2 n} = 0}} 𝟙_{{S_{k} \neq 0 for all k > 2 n}} | F_{2 n}]] \\ = E [𝟙_{{S_{2 n} = 0}} E [𝟙_{{S_{k} \neq 0 for all k > 2 n}} | F_{2 n}]] \\ = E [𝟙_{{S_{2 n} = 0}} P [R = \infty]] \\ = P [R = \infty] P [S_{2 n} = 0] . \end{aligned}$ Note that we have used the Markov property to deduce the fifth line.
  
  Recall that $S_{n}$ is even if and only if $n$ is even. Taking the above equation and summing over $n$ , noting that $\sum_{n = 0}^{\infty} P [L = 2 n] = 1$ we obtain
  
  $\begin{array}{r} 1 = P [R = \infty] \sum_{n = 0}^{\infty} P [S_{2 n} = 0] = P [R = \infty] E [G] . \end{array}$ The required result follows by noting that $P [R < \infty] = 1 - P [R = \infty]$ .
- (d) During the proof of Lemma 9.3.4, whilst we were arguing by contradiction, we noted that if $p = P [R < \infty] < 1$ then $G$ would have a Geometric( $p$ ) distribution. In part (c) we’ve discovered the mean of this geometric distribution. We also came close, for the same reason, during the proofs of Lemmas 9.5.1 and 9.5.2, and in Remark 9.4.2. In all of those cases, all we needed was to note that $E [G]$ would be finite – we didn’t need to know its value.
9.9 Let $B_{r} = {z \in Z^{3}; | z | < n}$ . Note that $B_{r}$ is a finite set.

Consider some $z \in Z^{3}$ . If $(S_{n})$ visits $z$ then, the first time that it does, we may apply the strong Markov property. From then on the walk will behave independently of its past, again moving one unit of space in a random direction on each step of time. Apply part (c) of Exercise 9.9, the expected number of returns to $z$ is finite, hence there will almost surely be a last return time to $z$ .

We can do the above for any $z \in B_{N}$ . Since $B_{r}$ is finite, this means that almost surely there is a last return time to the set $B_{r}$ . That is, there exists some random variable $N \in N$ such that $S_{n} \notin B_{r}$ for all $n \geq N$ . Hence $| S_{n} | \geq r$ for all $n \geq N$ . Since this holds for any $r \in N$ , we have $| S_{n} | \overset{a . s .}{\to} \infty$ .

Stochastic Processes and Financial Mathematics (part one)

Chapter A Solutions to exercises (part one)

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Stochastic Processes and Financial Mathematics
(part one)