Martingale sequences: the concept, examples, and basic patterns (Page 2/2)

Page 2 / 2

Notation . When we write $(X_{N}, Z_{N})$ is a martingale (submartingale, supermartingale), we are asserting X _N is integrable, Z _N is a decision sequence, $X_{N} \sim Z_{N}$ , and X _N is a MG (SMG, SRMG) relative to Z _N .

Definition . If Y _N is integrable and Z _N is a decision sequence, then

Y _N is absolutely fair relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] = 0 a . s . \forall n \in N$
Y _N is favorable relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] \geq 0 a . s . \forall n \in N$
Y _N is unfavorable relative to Z _N iff
$Y_{N} \sim Z_{N} and E [Y_{n + 1} | W_{n}] \leq 0 a . s . \forall n \in N$

Notation . When we write $(Y_{N}, Z_{N})$ is absolutely fair (favorable, unfavorable), we are asserting Y _N is integrable, Z _N is a decision sequence, $Y_{N} \sim Z_{N}$ , and Y _N is absolutely fair (favorable, unfavorable) relative to Z _N . IXA2-2

Ixa2-1

If X _N is a basic sequence and Y _N is the corresponding incremental sequence, then

$(X_{N}, Z_{N})$ is a martingale iff $(Y_{N}, Z_{N})$ is absolutely fair.
$(X_{N}, Z_{N})$ is a submartingale iff $(Y_{N}, Z_{N})$ is favorable.
$(X_{N}, Z_{N})$ is a supermartingale iff $(Y_{N}, Z_{N})$ is unfavorable.

Let * be any one of the symbols $=, \geq$ , or $\leq$ . Then by linearity and (CE7)

E [X_{n + 1} | W_{n}] = E [Y_{n + 1} | W_{n}] + E [X_{n} | W_{n}] = E [Y_{n + 1} | W_{n}] + X_{n} * X_{n} a . s . iff E [Y_{n + 1} | W_{n}] * 0 a . s .

Remarks

$(X_{N}, Z_{N})$ is a SMG iff $(- X_{N}, Z_{N})$ is a SRMG
We write (S)MG to indicate the same statement can be made for a MG or a SMG with the appropriate choice of = or $\geq$
We write ( ≥ ) to indicate simultaneously two cases:
- $(\geq)$ read as = in all places (for a MG)
- $(\geq)$ read as $\geq$ in all places (for a SMG)

Some basic patterns

Ixa3-1

If $(X_{N}, Z_{N})$ is a (S)MG and $X_{N} \sim H_{N}$ , with $H_{N} \sim Z_{N}$ , then $(X_{N}, H_{N})$ is a (S)MG.

Let $K_{n} = (H_{0}, H_{1}, \dots, H_{n})$ . By (CE9) , the (S)MG definition, monotonicity, and (CE7)

E [X_{n + 1} | K_{n}] = E {E [X_{n + 1} | W_{n}] | K_{n}} (\geq) E [X_{n} | K_{n}] = X_{n} a . s .

Ixa3-2

For integrable $X_{N} \sim Z_{N}$ , the following are equivalent

$\begin{matrix} a & (X_{N}, Z_{N}) & is a (S)MG \\ b & E [X_{n + k} | W_{n}] (\geq) X_{n} a . s . & \forall & n, k \in N \\ c & E [I_{C} X_{n + 1}] (\geq) E [I_{C} X_{n}] & \forall & C \in σ (W_{n}) & \forall & n \in N \\ d & E [I_{C} X_{n + k}] (\geq) E [I_{C} X_{n}] & \forall & C \in σ (W_{n}) & \forall & n, k \in N \end{matrix}$

b $\Rightarrow$ a as a special case
a $\Rightarrow$ b By (CE9) , (a), and monotonicity
$E [X_{n + k} | W_{n}] = E {E [X_{n + k} | W_{n + k - 1}] | W_{n}} (\geq) E [X_{n + k - 1} | W_{n}] a . s .$
$k - 1$ iterations yield $E [X_{n + k} | W_{n}] (\geq) X_{n} a . s .$
d $\Rightarrow$ c as a special case
c $\Rightarrow$ a By (CE1) and (c), $E [I_{C} X_{n + 1}] = E {I_{C} E [X_{n + 1} | W_{n}]} (\geq) E [I_{C} X_{n}] a . s .$ . Since $X_{n} \sim W_{n} a . s .$ and $E [X_{n + 1} | W_{n}] \sim W_{n} a . s .$ , the result follows from the uniqueness property (E5)
b $\Rightarrow$ d By (CE1) , (b), and monotonicity $E [I_{C} X_{n + k}] = E {I_{C} E [X_{n + k} | W_{n}]} (\geq) E [I_{C} X_{n}]$

We thus have $d \Rightarrow c \Rightarrow a \Leftrightarrow b \Rightarrow d$

Ixa3-3

If $(X_{N}, Z_{N})$ is a (S)MG, then $E [X_{n + k}] (\geq) E [X_{n}] (\geq) E [X_{0}]$

Ixa3-4

$(X_{N}, Z_{N})$ is a (S)MG iff $E [X_{q} - X_{p} | W_{n}] (\geq) 0 a . s . \forall n \leq p < q \in N$

EXERCISE. Note $X_{q} - X_{p} = Y_{p + 1} + \dots + Y_{q}$

IXA3-2

Ixa3-5

If $(X_{N}, Z_{N})$ is an $L^{2}$ MG, then

$\begin{matrix} E [X_{q} - X_{p}] = 0 & \forall & p < q \in N \\ E [X_{n} (X_{q} - X_{p})] = 0 & \forall & n \leq p < q \in N \\ E [(X_{n} - X_{m}) (X_{q} - X_{p})] = 0 & \forall & m < n \leq p < q \in N \\ E [X_{p} X_{q}] = E [X_{p \land q}^{2}] & \forall & p, q \in N \\ E [{(X_{q} - X_{p})}^{2}] = E [X_{q}^{2}] - E [X_{p}^{2}] \geq 0 & \forall & p < q \in N \\ E [X_{p}^{2}] = \sum_{k = 0}^{p} E [Y_{k}^{2}] & \forall & p \in N \end{matrix}$

$E [X_{q} - X_{p}] = E {E [X_{q} - X_{p} | W_{n}]} = 0$ by (CE1b) and Thm IXA3-4
$E [X_{n} (X_{q} - X_{p})] = E {X_{n} E [X_{q} - X_{p} | W_{n}]} = 0$ by (CE1b) , (CE8) , and Thm IXA3-4
As in b, since $X_{n} - X_{m} \sim W_{n}$
Suppose $p < q$ . Then, since $X_{p} \sim W_{p}$ , $E [X_{p} X_{q}] = E {X_{p} E [X_{q} | W_{p}]} = E [X_{p}^{2}]$ by definition of MG. For $q < p$ , interchange $p, q$ in the argument above.
$E [{(X_{q} - X_{p})}^{2}] = E [X_{q}^{2}] - 2 E [X_{p} X_{q}] + E [X_{p}^{2}] = E [X_{q}^{2}] - 2 E [X_{p}^{2}] + E [X_{p}^{2}]$ by d, above
By c, $E [Y_{j} Y_{k}] = 0$ for $j \neq k$ . Hence, $E [X_{p}^{2}] = E [{(\sum_{k = 0}^{p} Y_{k})}^{2}] = \sum_{j} \sum_{k} E [Y_{j} Y_{k}] = \sum_{k = 0}^{p} E [Y_{k}^{2}]$

A variety of weighted sums of increments are useful.

Ixa3-6

Suppose $(X_{N}, Z_{N})$ is a (S)MG and Y _N is the incremental sequence. Let H ₀ be a (nonnegative) constant and let $H_{n} \sim W_{n - 1}, n \geq 1$ , be bounded (nonnegative). Set

X_{n}^{*} = \sum_{k = 0}^{n} H_{k} Y_{k} = \sum_{k = 0}^{n} Y_{k}^{*} \forall n \in N

Then $(X_{N}^{*}, Z_{N})$ is a (S)MG.

$E [Y_{n + 1}^{*} | W_{n}] = E [H_{n + 1} Y_{n + 1} | W_{n}] = H_{n + 1} E [Y_{n + 1} | W_{n}] a . s .$ by (CE8)

For MG case: $E [Y_{n + 1}^{*} | W_{n}] = 0 a . s .$ for arbitrary bounded H _n

For SMG case: $E [Y_{n + 1}^{*} | W_{n}] \geq 0 a . s .$ for $H_{n} \geq 0$ , bounded

The conclusion follows from [link]

Remark . This result extends the pattern in the introductory gambling example. [link] IXA3-3

Ixa3-7

In Theorem IXA3-6 , if $E [X_{0}] \geq 0$ and $0 \leq H_{n} \leq 1 a . s . \forall n \in N$ , then $0 \leq E [X_{n}^{*}] \leq E [X_{n}] \forall n \in N$

$E [Y_{n + 1} | W_{n}] \geq H_{n + 1} E [Y_{n + 1} | W_{n}] (\geq) 0 a . s .$ , by hypothesis, and $H_{n + 1} E [Y_{n + 1} | W_{n}] = E [Y_{n + 1}^{*} | W_{n}] a . s .$ , by (CE8) . Thus, by monotonicity and (CE1b)

E [Y_{n + 1}] (\geq) E [Y_{n + 1}^{*}] (\geq) 0 \forall n \in N and E [Y_{0}] = E [X_{0}] \geq H_{0} E [Y_{0}] = E [Y_{0}^{*}]

Hence

E [X_{n}] = \sum_{k = 0}^{n} E [Y_{k}] \geq \sum_{k = 0}^{n} E [Y_{k}^{*}] = E [X_{n}^{*}] \geq 0

Some important special cases

Ixa3-8

Suppose integrable $X_{N} \sim Z_{N}$ . If $X_{n + 1} - X_{n} (\geq) 0 a . s . \forall n \in N$ , then $(X_{N}, Z_{N})$ is a (S)MG.

Apply monotonicity and Theorem IXA3-4

Ixa3-9

Suppose X _N has independent increments.

If $E [X_{n}] = c$ , invariant with n , then X _N is a MG.
If $E [X_{n + 1} - X_{n}] (\geq) 0, \forall n \in N$ , then $(X_{N}$ is a (S)MG.

For any n , consider any $C \in σ (U_{n})$ . By independent increments, ${I_{C}, (X_{n + 1} - X_{n})}$ is independent. Hence, $E [I_{C} X_{n + 1}] - E [I_{C} X_{n}] = E [I_{C} (X_{n + 1} - X_{n})] = E [I_{C}] E [(X_{n + 1} - X_{n})] (\geq) 0$ . The desired result follows from Theorem IXA3-2(c) .

Ixa3-10

Suppose g is a convex Borel function on an interval I which contains the range of all X _n and

$E [| g (X_{n}) |] < \infty \forall n \in N$ , Let $H_{n} = g (X_{n}) \forall n \in N$ ,

If $(X_{N}, Z_{N})$ is a MG, then $(H_{N}, Z_{N})$ is a SMG.
If g is nondecreasing and $(X_{N}, Z_{N})$ is a SMG, then so is $(H_{N}, Z_{N})$

a. By Jensen's inequality and the definition of a MG
$E [g (X_{n + 1}) | W_{n}] \geq g (E [X_{n + 1} | W_{n}]) = g (X_{n}) a . s .$
b. By Jensen's inequality
$E [g (X_{n + 1}) | W_{n}] \geq g (E [X_{n + 1} | W_{n}]) a . s .$
Since $E [X_{n + 1} | W_{n}] \geq X_{n} a . s .$ and g is nondecreasing, we have
$g (E [X_{n + 1} | W_{n}]) \geq g (X_{n}) a . s .$

Some commonly utilized convex functions

$g (t) = | t |$
$g (t) = t^{2}$ g is increasing for $t \geq 0$
$g (t) = u (t) t$ $g (X_{n}) = X_{n}^{+}$ g nondecreasing for all t
$g (t) = - u (- t) t$ $g (X_{n}) = X_{n}^{-}$ g nonincreasing for all t
$g (t) = e^{a t}, a > 0$ g is increasing for all t

Ixa3-11

Consider integrable $X_{N} \sim Z_{N}$ .

If $E [X_{n + 1} | W_{n}] = a X_{n} a . s . \forall n$ and $X_{n}^{*} = \frac{1}{a^{n}} X_{n} \forall n$ , then $(X_{N}^{*}, Z_{N})$ is a MG
If $E [X_{n + 1} | W_{n}] \geq a X_{n} a . s ., a > 0, \forall n$ and $X_{n}^{*} = \frac{1}{a^{n}} X_{n} \forall n$ , then $(X_{N}^{*}, Z_{N})$ is a SMG

E [X_{n + 1}^{*} | W_{n}] = \frac{1}{a^{n + 1}} E [X_{n + 1} | W_{n}] (\geq) \frac{1}{a^{n + 1}} a X_{n} = X_{n}^{*} a . s .

The restrictionl $a > 0$ is needed in the $\geq$ case.

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Topics in applied probability. OpenStax CNX. Sep 04, 2009 Download for free at http://cnx.org/content/col10964/1.2

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Topics in applied probability' conversation and receive update notifications?

Ask

	9 Neuroanatomy 09 The Auditory System By Stephen Voron Start Quiz
©flickr: Dave	Infection Control By Cath Yu Start Quiz
	NCE Ch 11 Counseling Families, Diagnosis... By Anh Dao Start Quiz
	Capitalism Market Economy By Robert Murphy Start Test
	English Proficiency Test By Anindyo Mukhopadhyay Start Quiz
	17 Sociology 17 Government and Politics MCQ By OpenStax Start Quiz
	Art Reviewer MCQ By Edgar Delgado Start Quiz
	14 Dr Landholt Large Animal Medicine-GI quiz By Brooke Delaney Start Exam
	25 AP 25 Urinary System MCQ By OpenStax Start Quiz
	Fireside Poets Quiz By Alec Moffit Start Quiz