<< Chapter < Page | Chapter >> Page > |
In this chapter, you will learn to:
We will now study stochastic processes, experiments in which the outcomes of events depend on the previous outcomes. Such a process or experiment is called a Markov Chain or Markov process . The process was first studied by a Russian mathematician named Andrei A. Markov in the early 1900s.
A small town is served by two telephone companies, Mama Bell and Papa Bell. Due to their aggressive sales tactics, each month 40% of Mama Bell customers switch to Papa Bell, that is, the other 60% stay with Mama Bell. On the other hand, 30% of the Papa Bell customers switch to Mama Bell. The above information can be expressed in a matrix which lists the probabilities of going from one state into another state. This matrix is called a transition matrix .
The reader should observe that a transition matrix is always a square matrix because all possible states must have both rows and columns. All entries in a transition matrix are non-negative as they represent probabilities. Furthermore, since all possible outcomes are considered in the Markov process, the sum of the row entries is always 1.
Professor Symons either walks to school, or he rides his bicycle. If he walks to school one day, then the next day, he will walk or cycle with equal probability. But if he bicycles one day, then the probability that he will walk the next day is $1/4$ . Express this information in a transition matrix.
We obtain the following transition matrix by properly placing the row and column entries. Note that if, for example, Professor Symons bicycles one day, then the probability that he will walk the next day is $1/4$ , and therefore, the probability that he will bicycle the next day is $3/4$ .
In [link] , if it is assumed that the first day is Monday, write a matrix that gives probabilities of a transition from Monday to Wednesday.
Let $W$ denote that Professor Symons walks and $B$ denote that he rides his bicycle.
We use the following tree diagram to compute the probabilities.
The probability that Professor Symons walked on Wednesday given that he walked on Monday can be found from the tree diagram, as listed below.
$P\left(\text{Walked Wednesday}\mid \text{Walked Monday}\right)=P\left(\text{WWW}\right)+P\left(\text{WBW}\right)=1/4+1/8=3/8$ .
$P\left(\text{Bicycled Wednesday}\mid \text{Walked Monday}\right)=P\left(\text{WWB}\right)+P\left(\text{WBB}\right)=1/4+3/8=5/8$ .
$P\left(\text{Walked Wednesday}\mid \text{Bicycled Monday}\right)=P\left(\text{BWW}\right)+P\left(\text{BBW}\right)=1/8+3/\text{16}=3/5/\text{16}$ .
$P\left(\text{Bicycled Wednesday}\mid \text{Bicycleed Monday}\right)=P\left(\text{BWB}\right)+P\left(\text{BBB}\right)=1/8+9/\text{16}=\text{11}/\text{16}$ .
We represent the results in the following matrix.
Alternately, this result can be obtained by squaring the original transition matrix.
We list both the original transition matrix $T$ and ${T}^{2}$ as follows:
The reader should compare this result with the probabilities obtained from the tree diagram.
Consider the following case, for example,
It makes sense because to find the probability that Professor Symons will walk on Wednesday given that he bicycled on Monday, we sum the probabilities of all paths that begin with $B$ and end in $W$ . There are two such paths, and they are $\text{BWW}$ and $\text{BBW}$ .
Notification Switch
Would you like to follow the 'Applied finite mathematics' conversation and receive update notifications?