<< Chapter < Page Chapter >> Page >
This chapter covers principles of Markov Chains. After completing this chapter students should be able to: write transition matrices for Markov Chain problems; find the long term trend for a Regular Markov Chain; Solve and interpret Absorbing Markov Chains.

Chapter overview

In this chapter, you will learn to:

  1. Write transition matrices for Markov Chain problems.
  2. Find the long term trend for a Regular Markov Chain.
  3. Solve and interpret Absorbing Markov Chains.

Markov chains

We will now study stochastic processes, experiments in which the outcomes of events depend on the previous outcomes. Such a process or experiment is called a Markov Chain or Markov process . The process was first studied by a Russian mathematician named Andrei A. Markov in the early 1900s.

A small town is served by two telephone companies, Mama Bell and Papa Bell. Due to their aggressive sales tactics, each month 40% of Mama Bell customers switch to Papa Bell, that is, the other 60% stay with Mama Bell. On the other hand, 30% of the Papa Bell customers switch to Mama Bell. The above information can be expressed in a matrix which lists the probabilities of going from one state into another state. This matrix is called a transition matrix .

This matrix depict the flow of customers from mama bell to papa bell and vice versa.

The reader should observe that a transition matrix is always a square matrix because all possible states must have both rows and columns. All entries in a transition matrix are non-negative as they represent probabilities. Furthermore, since all possible outcomes are considered in the Markov process, the sum of the row entries is always 1.

Professor Symons either walks to school, or he rides his bicycle. If he walks to school one day, then the next day, he will walk or cycle with equal probability. But if he bicycles one day, then the probability that he will walk the next day is 1 / 4 size 12{1/4} {} . Express this information in a transition matrix.

We obtain the following transition matrix by properly placing the row and column entries. Note that if, for example, Professor Symons bicycles one day, then the probability that he will walk the next day is 1 / 4 size 12{1/4} {} , and therefore, the probability that he will bicycle the next day is 3 / 4 size 12{3/4} {} .

This matrix shows the probability that Professor Symons will bicycle or walk to work.

Got questions? Get instant answers now!
Got questions? Get instant answers now!

In [link] , if it is assumed that the first day is Monday, write a matrix that gives probabilities of a transition from Monday to Wednesday.

Let W size 12{W} {} denote that Professor Symons walks and B size 12{B} {} denote that he rides his bicycle.

We use the following tree diagram to compute the probabilities.

The Tree diagram depicts the probability that Professor Symons will either walk or bicycle to work on Monday, Tuesday, or Wednesday.

The probability that Professor Symons walked on Wednesday given that he walked on Monday can be found from the tree diagram, as listed below.

P Walked Wednesday Walked Monday = P WWW + P WBW = 1 / 4 + 1 / 8 = 3 / 8 size 12{P left ("Walked Wednesday" \lline " Walked Monday" right )=P left ( ital "WWW" right )+P left ( ital "WBW" right )=1/4+1/8=3/8} {} .

P Bicycled Wednesday Walked Monday = P WWB + P WBB = 1 / 4 + 3 / 8 = 5 / 8 size 12{P left ("Bicycled Wednesday" \lline " Walked Monday" right )=P left ( ital "WWB" right )+P left ( ital "WBB" right )=1/4+3/8=5/8} {} .

P Walked Wednesday Bicycled Monday = P BWW + P BBW = 1 / 8 + 3 / 16 = 3 / 5 / 16 size 12{P left ("Walked Wednesday" \lline " Bicycled Monday" right )=P left ( ital "BWW" right )+P left ( ital "BBW" right )=1/8+3/"16"=3/5/"16"} {} .

P Bicycled Wednesday Bicycleed Monday = P BWB + P BBB = 1 / 8 + 9 / 16 = 11 / 16 size 12{P left ("Bicycled Wednesday" \lline " Bicycleed Monday" right )=P left ( ital "BWB" right )+P left ( ital "BBB" right )=1/8+9/"16"="11"/"16"} {} .

We represent the results in the following matrix.

This matrix depicts the probability that Professor Symons will walk or bicycle to work on Monday or Wednesday.

Alternately, this result can be obtained by squaring the original transition matrix.

We list both the original transition matrix T size 12{T} {} and T 2 size 12{T rSup { size 8{2} } } {} as follows:

T = 1 / 2 1 / 2 1 / 4 3 / 4 size 12{T= left [ matrix { 1/2 {} # 1/2 {} ##1/"4" {} # "3"/"4"{} } right ]} {}
T 2 1 / 2 1 / 2 1 / 4 3 / 4 1 / 2 1 / 2 1 / 4 3 / 4 1 / 4 + 1 / 8 1 / 4 + 3 / 8 1 / 8 + 3 / 16 1 / 8 + 9 / 16 3 / 8 5 / 8 5 / 16 11 / 16 size 12{ matrix { T rSup { size 8{2} } {} # ={} {} # left [ matrix {1/2 {} # 1/2 {} ## 1/4 {} # 3/4{}} right ] left [ matrix {1/2 {} # 1/2 {} ## 1/4 {} # 3/4{}} right ] {} ##{} # ={} {} # left [ matrix { 1/4+1/8 {} # 1/4+3/8 {} ##1/8+3/"16" {} # 1/8+9/"16"{} } right ]{} ## {} # ={} {} # left [ matrix {3/8 {} # 5/8 {} ## 5/"16" {} # "11"/"16"{}} right ]{}} } {}

The reader should compare this result with the probabilities obtained from the tree diagram.

Consider the following case, for example,

P Walked Wednesday Bicycled Monday = P BWW + P BBW = 1 / 8 + 3 / 16 = 5 / 16 size 12{P left ("Walked Wednesday" \lline " Bicycled Monday" right )=P left ( ital "BWW" right )+P left ( ital "BBW" right )=1/8+3/"16"=5/"16"} {} .

It makes sense because to find the probability that Professor Symons will walk on Wednesday given that he bicycled on Monday, we sum the probabilities of all paths that begin with B size 12{B} {} and end in W size 12{W} {} . There are two such paths, and they are BWW size 12{ ital "BWW"} {} and BBW size 12{ ital "BBW"} {} .

Got questions? Get instant answers now!
Got questions? Get instant answers now!

Questions & Answers

where we get a research paper on Nano chemistry....?
Maira Reply
what are the products of Nano chemistry?
Maira Reply
There are lots of products of nano chemistry... Like nano coatings.....carbon fiber.. And lots of others..
learn
Even nanotechnology is pretty much all about chemistry... Its the chemistry on quantum or atomic level
learn
Google
da
no nanotechnology is also a part of physics and maths it requires angle formulas and some pressure regarding concepts
Bhagvanji
Preparation and Applications of Nanomaterial for Drug Delivery
Hafiz Reply
revolt
da
Application of nanotechnology in medicine
what is variations in raman spectra for nanomaterials
Jyoti Reply
I only see partial conversation and what's the question here!
Crow Reply
what about nanotechnology for water purification
RAW Reply
please someone correct me if I'm wrong but I think one can use nanoparticles, specially silver nanoparticles for water treatment.
Damian
yes that's correct
Professor
I think
Professor
Nasa has use it in the 60's, copper as water purification in the moon travel.
Alexandre
nanocopper obvius
Alexandre
what is the stm
Brian Reply
is there industrial application of fullrenes. What is the method to prepare fullrene on large scale.?
Rafiq
industrial application...? mmm I think on the medical side as drug carrier, but you should go deeper on your research, I may be wrong
Damian
How we are making nano material?
LITNING Reply
what is a peer
LITNING Reply
What is meant by 'nano scale'?
LITNING Reply
What is STMs full form?
LITNING
scanning tunneling microscope
Sahil
how nano science is used for hydrophobicity
Santosh
Do u think that Graphene and Fullrene fiber can be used to make Air Plane body structure the lightest and strongest. Rafiq
Rafiq
what is differents between GO and RGO?
Mahi
what is simplest way to understand the applications of nano robots used to detect the cancer affected cell of human body.? How this robot is carried to required site of body cell.? what will be the carrier material and how can be detected that correct delivery of drug is done Rafiq
Rafiq
if virus is killing to make ARTIFICIAL DNA OF GRAPHENE FOR KILLED THE VIRUS .THIS IS OUR ASSUMPTION
Anam
analytical skills graphene is prepared to kill any type viruses .
Anam
Any one who tell me about Preparation and application of Nanomaterial for drug Delivery
Hafiz
what is Nano technology ?
Bob Reply
write examples of Nano molecule?
Bob
The nanotechnology is as new science, to scale nanometric
brayan
nanotechnology is the study, desing, synthesis, manipulation and application of materials and functional systems through control of matter at nanoscale
Damian
Is there any normative that regulates the use of silver nanoparticles?
Damian Reply
what king of growth are you checking .?
Renato
What fields keep nano created devices from performing or assimulating ? Magnetic fields ? Are do they assimilate ?
Stoney Reply
why we need to study biomolecules, molecular biology in nanotechnology?
Adin Reply
?
Kyle
yes I'm doing my masters in nanotechnology, we are being studying all these domains as well..
Adin
why?
Adin
what school?
Kyle
biomolecules are e building blocks of every organics and inorganic materials.
Joe
If March sales will be up from February by 10%, 15%, and 20% at Place I, Place II, and Place III, respectively, find the expected number of hot dogs, and corn dogs to be sold
Logan Reply
8. It is known that 80% of the people wear seat belts, and 5% of the people quit smoking last year. If 4% of the people who wear seat belts quit smoking, are the events, wearing a seat belt and quitting smoking, independent?
William Reply
Mr. Shamir employs two part-time typists, Inna and Jim for his typing needs. Inna charges $10 an hour and can type 6 pages an hour, while Jim charges $12 an hour and can type 8 pages per hour. Each typist must be employed at least 8 hours per week to keep them on the payroll. If Mr. Shamir has at least 208 pages to be typed, how many hours per week should he employ each student to minimize his typing costs, and what will be the total cost?
Chine Reply
At De Anza College, 20% of the students take Finite Mathematics, 30% take Statistics and 10% take both. What percentage of the students take Finite Mathematics or Statistics?
Chalton Reply

Get the best Algebra and trigonometry course in your pocket!





Source:  OpenStax, Applied finite mathematics. OpenStax CNX. Jul 16, 2011 Download for free at http://cnx.org/content/col10613/1.5
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Applied finite mathematics' conversation and receive update notifications?

Ask