12.1.3 Algorithms for Probabilistic I-Spaces (POMDPs)

For the probabilistic case, the methods of Section 10.2 cannot be applied because ${\cal I}_{prob}$ is a continuous space. Dynamic programming methods for continuous state spaces, as covered in Section 10.6, are needed. The main difficulty is that the dimension of ${\vec{X}}$ grows linearly with the number of states in $X$. If there are $n$ states in $X$, the dimension of ${\vec{X}}$ is $n-1$. Since the methods of Section 10.6 suffer from the curse of dimensionality, the general dynamic programming techniques are limited to problems in which $X$ has only a few states.