The probabilistic forward projection can be considered as a Markov process because the ``decision'' part is removed once the actions are given. Suppose that is given and is applied. What is the probability distribution over ? This was already specified in (10.6) and is the one-stage forward projection. Now consider the two-stage probabilistic forward projection, . This can be computed by marginalization as

Computing further forward projections requires nested summations, which marginalize all of the intermediate states. For example, the three-stage forward projection is

A convenient expression of the probabilistic forward projections can be obtained by borrowing nice algebraic properties from linear algebra. For each action , let its

in which

(10.16) |

For each , the th column of must sum to one and can be interpreted as the probability distribution over that is obtained if is applied from state .

Let denote an -dimensional column vector that represents any probability distribution over . The product yields a column vector that represents the probability distribution over that is obtained after starting with and applying . The matrix multiplication performs inner products, each of which is a marginalization as shown in (10.13). The forward projection at any stage, , can now be expressed using a product of state transition matrices. Suppose that is fixed. Let , which indicates that is known (with probability one). The forward projection can be computed as

The th element of is .

(10.18) |

over the sequence of states . The two-stage forward projection yields

(10.19) |

over .

Steven M LaValle 2020-08-14