Conditional independence, given a random vector  (Page 2/4)

(CI1) $E\left[I_M\left(X\right)I_N\left(Y\right)\right|Z]=E\left[I_M\left(X\right)\right|Z]E\left[I_N\left(Y\right)\right|Z]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{sets}M,N$

(CI2) $E\left[I_M\left(X\right)\right|Z,Y]=E\left[I_M\left(X\right)\right|Z]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{sets}M$

(CI3) $E\left[I_M\left(X\right)I_Q\left(Z\right)\right|Z,Y]=E\left[I_M\left(X\right)I_Q\left(Z\right)\right|Z]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{sets}M,Q$

(CI4) $E\left[I_M\left(X\right)I_Q\left(Z\right)\right|Y]=E\left\{E\left[I_M\left(X\right)I_Q\left(Z\right)\right|Z]\right|Y\}\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{sets}M,Q$

As an example of the kinds of argument needed to verify these equivalences, we show the equivalence of (CI1) and (CI2) .

• (CI1) implies (CI2) . Set $e_1(Y,Z)=E\left[I_M\left(X\right)\right|Z,Y]$ and $e_2(Y,Z)=E\left[I_M\left(X\right)\right|Z]$ . If we show
  $$E\left[I_N\left(Y\right)I_Q\left(Z\right)e_1(Y,Z)\right]=E[I_N\left(Y\right)I_Q\left(Z\right)e_2(Y,Z)]\text{for}\text{all}\text{Borel}N,Q$$
  then by the uniqueness property (E5b) for expectation we may assert $e_1(Y,Z)=e_2(Y,Z)\mathrm{a}.\mathrm{s}.$ Using the defining property (CE1) for conditional expectation, we have
  $$E\left\{I_N\left(Y\right)I_Q\left(Z\right)E\left[I_M\left(X\right)\right|Z,Y]\right\}=E\left[I_N\left(Y\right)I_Q\left(Z\right)I_M\left(X\right)\right]$$
  On the other hand, use of (CE1) , (CE8) , (CI1) , and (CE1) yields
  $$E\left\{I_N\left(Y\right)I_Q\left(Z\right)E\left[I_M\left(X\right)\right|Z]\right\}=E\left\{I_Q\left(Z\right)E\left[I_N\left(Y\right)E\left[I_M\left(X\right)\right|Z]\right|Z]\right\}$$
  $$=E\left\{I_Q\left(Z\right)E\left[I_M\left(X\right)\right|Z]E\left[I_N\left(Y\right)\right|Z]\right\}=E\left\{I_Q\left(Z\right)E\left[I_M\left(X\right)I_N\left(Y\right)\right|Z]\right\}$$
  $$=E\left[I_N\left(Y\right)I_Q\left(Z\right)I_M\left(X\right)\right]$$
  which establishes the desired equality.
• (CI2) implies (CI1) . Using (CE9) , (CE8) , (CI2) , and (CE8) , we have
  $$E\left[I_M\left(X\right)I_N\left(Y\right)\right|Z]=E\left\{E\left[I_M\left(X\right)I_N\left(Y\right)\right|Z,Y]\right|Z\}$$
  $$=E\left\{I_N\left(Y\right)E\left[I_M\left(X\right)\right|Z,Y]\right|Z\}=E\left\{I_N\left(Y\right)E\left[I_M\left(X\right)\right|Z]\right|Z\}$$
  $$=E\left[I_M\left(X\right)\right|Z]E\left[I_N\left(Y\right)\right|Z]$$

Use of property (CE8) shows that (CI2) and (CI3) are equivalent. Now just as (CI1) extends to (CI5) , so also (CI3) is equivalent to

(CI6) $E\left[g\right(X,Z\left)\right|Z,Y]=E[g(X,Z)\left|Z\right]\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{functions}g$

Property (CI6) provides an important interpretation of conditional independence:

$E\left[g\right(X,Z\left)\right|Z]$ is the best mean-square estimator for $g(X,Z)$ , given knowledge of Z . The condition $\{X,Y\}\mathrm{ci}|Z$ implies that additional knowledge about Y does not modify that best estimate. This interpretation is often the most useful as a modeling assumption.

Similarly, property (CI4) is equivalent to

(CI8) $E\left[g\right(X,Z\left)\right|Y]=E\{E\left[g\right(X,Z\left)\right|Z\left]\right|Y\}\mathrm{a}.\mathrm{s}.\text{for}\text{all}\text{Borel}\text{functions}g$

Property (CI7) is an alternate way of expressing (CI6) . Property (CI9) is just a convenient way of expressing the other conditions.

The additional properties in Appendix G are useful in a variety of contexts, particularly in establishing properties of Markov systems. We refer to them as needed.

The bayesian approach to statistics

In the classical approach to statistics, a fundamental problem is to obtain information about the population distribution from the distribution in a simplerandom sample. There is an inherent difficulty with this approach. Suppose it is desired to determine the population mean μ . Now μ is an unknown quantity about which there is uncertainty. However, since it is a constant, we cannotassign a probability such as $P(a<\mu \le b)$ . This has no meaning.

The Bayesian approach makes a fundamental change of viewpoint. Since the population mean is a quantity about which there is uncertainty, it is modeled as a random variable whose value is to be determined by experiment. In this view, the population distribution is conceived as randomly selected from a class of suchdistributions. One way of expressing this idea is to refer to a state of nature . The population distribution has been “selected by nature” from a class of distributions. The mean value is thus a random variable whose value is determined by this selection. Toimplement this point of view, we assume