##

10.2.1 Value Iteration

Fortunately, the value iteration method of Section 2.3.1
extends nicely to handle uncertainty in prediction. This was the main
reason why value iteration was introduced in Chapter
2. Value iteration was easier to describe in Section
2.3.1 because the complications of nature were avoided. In
the current setting, value iteration retains most of its efficiency
and can easily solve problems that involve thousands or even millions
of states.

The state space, , is assumed to be finite throughout Section
10.2.1. An extension to the case of a countably infinite
state space can be developed if cost-to-go values over the entire
space do not need to be computed incrementally.

Only backward value iteration is considered here. Forward versions
can be defined alternatively.

**Subsections**

Steven M LaValle
2020-08-14