# 0.16 Markov chains

 Page 1 / 4
This chapter covers principles of Markov Chains. After completing this chapter students should be able to: write transition matrices for Markov Chain problems; find the long term trend for a Regular Markov Chain; Solve and interpret Absorbing Markov Chains.

## Chapter overview

In this chapter, you will learn to:

1. Write transition matrices for Markov Chain problems.
2. Find the long term trend for a Regular Markov Chain.
3. Solve and interpret Absorbing Markov Chains.

## Markov chains

We will now study stochastic processes, experiments in which the outcomes of events depend on the previous outcomes. Such a process or experiment is called a Markov Chain or Markov process . The process was first studied by a Russian mathematician named Andrei A. Markov in the early 1900s.

A small town is served by two telephone companies, Mama Bell and Papa Bell. Due to their aggressive sales tactics, each month 40% of Mama Bell customers switch to Papa Bell, that is, the other 60% stay with Mama Bell. On the other hand, 30% of the Papa Bell customers switch to Mama Bell. The above information can be expressed in a matrix which lists the probabilities of going from one state into another state. This matrix is called a transition matrix .

The reader should observe that a transition matrix is always a square matrix because all possible states must have both rows and columns. All entries in a transition matrix are non-negative as they represent probabilities. Furthermore, since all possible outcomes are considered in the Markov process, the sum of the row entries is always 1.

Professor Symons either walks to school, or he rides his bicycle. If he walks to school one day, then the next day, he will walk or cycle with equal probability. But if he bicycles one day, then the probability that he will walk the next day is $1/4$ . Express this information in a transition matrix.

We obtain the following transition matrix by properly placing the row and column entries. Note that if, for example, Professor Symons bicycles one day, then the probability that he will walk the next day is $1/4$ , and therefore, the probability that he will bicycle the next day is $3/4$ .

In [link] , if it is assumed that the first day is Monday, write a matrix that gives probabilities of a transition from Monday to Wednesday.

Let $W$ denote that Professor Symons walks and $B$ denote that he rides his bicycle.

We use the following tree diagram to compute the probabilities.

The probability that Professor Symons walked on Wednesday given that he walked on Monday can be found from the tree diagram, as listed below.

$P\left(\text{Walked Wednesday}\mid \text{Walked Monday}\right)=P\left(\text{WWW}\right)+P\left(\text{WBW}\right)=1/4+1/8=3/8$ .

$P\left(\text{Bicycled Wednesday}\mid \text{Walked Monday}\right)=P\left(\text{WWB}\right)+P\left(\text{WBB}\right)=1/4+3/8=5/8$ .

$P\left(\text{Walked Wednesday}\mid \text{Bicycled Monday}\right)=P\left(\text{BWW}\right)+P\left(\text{BBW}\right)=1/8+3/\text{16}=3/5/\text{16}$ .

$P\left(\text{Bicycled Wednesday}\mid \text{Bicycleed Monday}\right)=P\left(\text{BWB}\right)+P\left(\text{BBB}\right)=1/8+9/\text{16}=\text{11}/\text{16}$ .

We represent the results in the following matrix.

Alternately, this result can be obtained by squaring the original transition matrix.

We list both the original transition matrix $T$ and ${T}^{2}$ as follows:

$T=\left[\begin{array}{cc}1/2& 1/2\\ 1/\text{4}& \text{3}/\text{4}\end{array}\right]$
$\begin{array}{ccc}{T}^{2}& & \left[\begin{array}{cc}1/2& 1/2\\ 1/4& 3/4\end{array}\right]\left[\begin{array}{cc}1/2& 1/2\\ 1/4& 3/4\end{array}\right]\\ & & \left[\begin{array}{cc}1/4+1/8& 1/4+3/8\\ 1/8+3/\text{16}& 1/8+9/\text{16}\end{array}\right]\\ & & \left[\begin{array}{cc}3/8& 5/8\\ 5/\text{16}& \text{11}/\text{16}\end{array}\right]\end{array}$

The reader should compare this result with the probabilities obtained from the tree diagram.

Consider the following case, for example,

$P\left(\text{Walked Wednesday}\mid \text{Bicycled Monday}\right)=P\left(\text{BWW}\right)+P\left(\text{BBW}\right)=1/8+3/\text{16}=5/\text{16}.$

It makes sense because to find the probability that Professor Symons will walk on Wednesday given that he bicycled on Monday, we sum the probabilities of all paths that begin with $B$ and end in $W$ . There are two such paths, and they are $\text{BWW}$ and $\text{BBW}$ .

how can chip be made from sand
is this allso about nanoscale material
Almas
are nano particles real
yeah
Joseph
Hello, if I study Physics teacher in bachelor, can I study Nanotechnology in master?
no can't
Lohitha
where is the latest information on a no technology how can I find it
William
currently
William
where we get a research paper on Nano chemistry....?
nanopartical of organic/inorganic / physical chemistry , pdf / thesis / review
Ali
what are the products of Nano chemistry?
There are lots of products of nano chemistry... Like nano coatings.....carbon fiber.. And lots of others..
learn
Even nanotechnology is pretty much all about chemistry... Its the chemistry on quantum or atomic level
learn
da
no nanotechnology is also a part of physics and maths it requires angle formulas and some pressure regarding concepts
Bhagvanji
hey
Giriraj
Preparation and Applications of Nanomaterial for Drug Delivery
revolt
da
Application of nanotechnology in medicine
has a lot of application modern world
Kamaluddeen
yes
narayan
what is variations in raman spectra for nanomaterials
ya I also want to know the raman spectra
Bhagvanji
I only see partial conversation and what's the question here!
what about nanotechnology for water purification
please someone correct me if I'm wrong but I think one can use nanoparticles, specially silver nanoparticles for water treatment.
Damian
yes that's correct
Professor
I think
Professor
Nasa has use it in the 60's, copper as water purification in the moon travel.
Alexandre
nanocopper obvius
Alexandre
what is the stm
is there industrial application of fullrenes. What is the method to prepare fullrene on large scale.?
Rafiq
industrial application...? mmm I think on the medical side as drug carrier, but you should go deeper on your research, I may be wrong
Damian
How we are making nano material?
what is a peer
What is meant by 'nano scale'?
What is STMs full form?
LITNING
scanning tunneling microscope
Sahil
how nano science is used for hydrophobicity
Santosh
Do u think that Graphene and Fullrene fiber can be used to make Air Plane body structure the lightest and strongest. Rafiq
Rafiq
what is differents between GO and RGO?
Mahi
what is simplest way to understand the applications of nano robots used to detect the cancer affected cell of human body.? How this robot is carried to required site of body cell.? what will be the carrier material and how can be detected that correct delivery of drug is done Rafiq
Rafiq
if virus is killing to make ARTIFICIAL DNA OF GRAPHENE FOR KILLED THE VIRUS .THIS IS OUR ASSUMPTION
Anam
analytical skills graphene is prepared to kill any type viruses .
Anam
Any one who tell me about Preparation and application of Nanomaterial for drug Delivery
Hafiz
what is Nano technology ?
write examples of Nano molecule?
Bob
The nanotechnology is as new science, to scale nanometric
brayan
nanotechnology is the study, desing, synthesis, manipulation and application of materials and functional systems through control of matter at nanoscale
Damian
If March sales will be up from February by 10%, 15%, and 20% at Place I, Place II, and Place III, respectively, find the expected number of hot dogs, and corn dogs to be sold
8. It is known that 80% of the people wear seat belts, and 5% of the people quit smoking last year. If 4% of the people who wear seat belts quit smoking, are the events, wearing a seat belt and quitting smoking, independent?
Mr. Shamir employs two part-time typists, Inna and Jim for his typing needs. Inna charges $10 an hour and can type 6 pages an hour, while Jim charges$12 an hour and can type 8 pages per hour. Each typist must be employed at least 8 hours per week to keep them on the payroll. If Mr. Shamir has at least 208 pages to be typed, how many hours per week should he employ each student to minimize his typing costs, and what will be the total cost?
At De Anza College, 20% of the students take Finite Mathematics, 30% take Statistics and 10% take both. What percentage of the students take Finite Mathematics or Statistics?