<< Chapter < Page Chapter >> Page >
Additional examples of Matlab calculations for decision problems (see Matlab Procedures for Markov Decision Processes).

Data

There are three types.  In all types, we need the following:

A = the vector of actions ( 1 × m ) m = the number of actions P H : P H ( i ) = P ( H = u i ) ( 1 × s ) s = the number of values of H P X H : P X H ( i , j ) = P ( X = x j | H = u i ) ( s × q ) q = the number of values of X

  •  The usual type.  In addition to the above, we need       L = [ L ( a , y k ) ] ( m × n ) m = the number of actions P Y H : P Y H ( i , k ) ] = P ( Y = y k | H = u i ) ( s × n ) n = the number of values of Y
  •  The matrix R H = [ r ( a , i ) ] is given.   L and P Y H are not needed.
  •  Sometimes Y = H .  In this case R H = L , which we need, in addition to the above.

Calculated quantities

  1. R H = [ r ( a , i ) ] ( m × s )      [Risk function = expected loss, given H ]                r ( a , i ) = E [ L ( a , Y ) | H = u i ] = k L ( a , k ) P ( Y = y k | H = u i ) MATLAB:  RH = L*PYH'
  2. P X ( 1 × q )                P X ( j ) = P ( X = x j ) = i P ( H = u i ) P ( X = j | H = u i ) MATLAB:   PX = PH*PXH
  3. P H X ( q × s )                P H X ( i , ) = P ( H = u j | X = x i ) = P ( X = x i | H = u i ) P ( H = u j ) / P ( X = X i ) MATLAB: [a,b] = meshgrid(PH,PX)      PHX = PXH'.*a./b
  4. R X = [ R ( a , j ) ] ( m × q )       [Expected risk, given X ]                R ( a , j ) = E [ r ( a , H ) | X = x j ] = i r ( a , i ) P ( H = u i | X = x j ) MATLAB:  RX = RH*PHX'
  5. Select d * from R X :   d * ( j ) is the action a (row number) for minimum expected loss, given X = j . Set D = [ d * ( 1 ) , d * ( 2 ) , d * ( q ) ] .
  6. Calculate the Bayesian risk B D for d * .                B D = E [ R ( d * ( X ) , X ) ] = j R X ( D ( j ) , j ) P X ( j ) MATLAB:  RD*PX'
Actions are represented in calculations by action number (position in the matrix). In some cases, each action has a value other than its position number. Theactual values can be presented in the final display.

File dec.m

% file dec.m % Version of 12/12/95disp('Decision process with experimentation') disp('There are three types, according to the data provided.')disp('In all types, we need the row vector A of actions,') disp('the row vector PH with PH(i) = P(H = u_i),')disp('the row vector X of test random variable values, and') disp('the matrix PXH with PXH(i,j) = P(X = x_j|H = u_i).')disp('Type 1.  Loss matrix L of L(a,k)') disp('         Matrix PYH with PYH(i,k) = P(Y = y_k|H = u_i)')disp('Type 2.  Matrix RH of r(a,i) = E[L(a,Y)|H = u_i].')disp('         L and PYH are not needed for this type.') disp('Type 3.  Y = H, so that only RH = L is needed.')c   = input('Enter type number  '); A   = input('Enter vector A of actions ');PH  = input('Enter vector PH of parameter probabilities  '); PXH = input('Enter matrix PXH of conditional probabilities  ');X   = input('Enter vector X of test random variable values  '); s = length(PH);q = length(X); if c == 1 L   = input('Enter loss matrix L  ');  PYH = input('Enter matrix PYH of conditional probabilities  '); RH  = L*PYH'; elseif c == 2 RH  = input('Enter matrix RH of expected loss, given H  '); else L   = input('Enter loss matrix L  ');  RH  = L;end PX   = PH*PXH;        % (1 x s)(s x q) = (1 x q)[a,b] = meshgrid(PH,PX);PHX = PXH'.*a./b;     % (q x s) RX  = RH*PHX';        % (m x s)(s x q) = (m x q)[RD D] = min(RX);     % determines min of each col                      % and row on which min occurs S = [X; A(D); RD]'; BD = RD*PX';          % Bayesian riskh  = ['  Optimum losses and actions']; sh = ['  Test value  Action     Loss']; disp(' ')disp(h) disp(sh)disp(S) disp(' ')disp(['Bayesian risk  B(d*) = ',num2str(BD),])

General case

% file dec1.m % Data for Problem 22-11type = 1; A = [10 15];          % Artificial actions list PH = [0.3 0.2 0.5];   % PH(i) = P(H = i) PXH = [0.7 0.2 0.1;   % PXH(i,j) = P(X = j|H= i)      0.2 0.6 0.2;       0.1 0.1 0.8]; X = [-1 0  1]; L = [1  0 -2;         % L(a,k) = loss when action number is a, outcome is k    3 -1 -4];PYH = [0.5 0.3 0.2;   % PYH(i,k) = P(Y = k|H = i)       0.2 0.5 0.3;      0.1 0.3 0.6];  dec1dec Decision process with experimentationThere are three types, according to the data provided. In all types, we need the row vector A of actions,the row vector PH with PH(i) = P(H = i), the row vector X of test random variable values, andthe matrix PXH with PXH(i,j) = P(X = j|H = i). Type 1.  Loss matrix L of L(a,k)        Matrix PYH with PYH(i,k) = P(Y = k|H = i) Type 2.  Matrix RH of r(a,i) = E[L(a,Y)|H = i].         L and PYH are not needed in this case.Type 3.  Y = H, so that only RH = L is needed. Enter type number  typeEnter vector A of actions A Enter vector PH of parameter probabilities  PHEnter matrix PXH of conditional probabilities  PXH Enter vector X of test random variable values  XEnter loss matrix L  L Enter matrix PYH of conditional probabilities  PYH   Optimum losses and actions Test value  Action     Loss   -1.0000   15.0000   -0.2667        0   15.0000   -0.9913    1.0000   15.0000   -2.1106  Bayesian risk  B(d*) = -1.3

Intermediate steps in solution of Example 1, to show results of various operations

RH RH  =  0.1000   -0.4000   -1.1000      0.4000   -1.1000   -2.4000 PXPX  =  0.3000    0.2300    0.4700 aa   =  0.3000    0.2000    0.5000       0.3000    0.2000    0.5000      0.3000    0.2000    0.5000 bb   =  0.3000    0.3000    0.3000       0.2300    0.2300    0.2300      0.4700    0.4700    0.4700 PHXPHX =  0.7000    0.1333    0.1667       0.2609    0.5217    0.2174      0.0638    0.0851    0.8511 RXRX  = -0.1667   -0.4217   -0.9638      -0.2667   -0.9913   -2.1106

R H Given

% file dec2.m   % Data for type in which RH is giventype = 2; A = [1 2]; X = [-1 1 3]; PH = [0.2 0.5 0.3]; PXH = [0.5 0.4 0.1;   % PXH(i,j) = P(X = j|H = i)      0.4 0.5 0.1;       0.2 0.4 0.4]; RH = [-10   5 -12;       5 -10  -5];    % r(a,i) = expected loss when                      %   action is a, given H = i  dec2 decDecision process with experimentation ------------------- Instruction lines edited outEnter type number  type Enter vector A of actions AEnter vector PH of parameter probabilities  PH Enter matrix PXH of conditional probabilities  PXHEnter vector X of test random variable values  X Enter matrix RH of expected loss, given H  RH   Optimum losses and actions Test value  Action     Loss   -1.0000    2.0000   -5.0000   1.0000    2.0000   -6.0000    3.0000    1.0000   -7.3158  Bayesian risk  B(d*) = -5.89

Example 3

Carnival example (type in which Y = H )

A carnival is scheduled to appear on a given date.  Profits to be earned depend heavily on the weather.  If rainy, the carnivalloses $15 (thousands); if cloudy, the loss is $5 (thousands); if sunny, a profit of $10 (thousands) is expected.  If thecarnival sets up its equipment, it must give the show;  if it decides not to set up, it forfeits $1,000.  For an additional cost of $1,000, it candelay setup until the day before the show and get the latest weather report.

      Actual weather H = Y is 1 rainy, 2 cloudy, or 3 sunny.

The weather report X has values 1, 2, or 3, corresponding to predictions rainy, cloudy, or sunny respectively.

      Reliability of the forecast is expressed in terms of P ( X = j | H = i ) – see matrix P X H

      Two actions: 1 set up;  2 no set up.

      Possible losses for each action and weather condition are in matrix L .

% file dec3,m % Carnival problemtype = 3;             % Y = H  (actual weather) A = [1  2];           % 1: setup  2: no setup X = [1  2  3];        % 1; rain,  2: cloudy, 3: sunny L = [16 6 -9;         % L(a,k) = loss when action number is a, outcome is k     2 2  2];         % --with premium for postponing setupPH = 0.1*[1 3 6];     % P(H = i)PXH = 0.1*[7 2 1;     % PXH(i,j) = P(X = j|H = i)           2 6 2;          1 2 7];  dec3dec Decision process with experimentation------------------- Instruction lines edited out Enter case number  caseEnter vector A of actions A Enter vector PH of parameter probabilities  PHEnter matrix PXH of conditional probabilities  PXH Enter vector X of test random variable values  XEnter loss matrix L  L   Optimum losses and actions  Test value  Action     Loss   1.0000    2.0000    2.0000    2.0000    1.0000    1.0000   3.0000    1.0000   -6.6531  Bayesian risk  B(d*) = -2.56

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Topics in applied probability. OpenStax CNX. Sep 04, 2009 Download for free at http://cnx.org/content/col10964/1.2
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Topics in applied probability' conversation and receive update notifications?

Ask