11.1.2 Defining the History Information Space

This section defines the most basic and natural I-space. Many others will be derived from it, which is the topic of Section 11.2. Suppose that $X$, $U$, and $f$ have been defined as in Formulation 10.1, and the notion of stages has been defined as in Formulation 2.2. This yields a state sequence $x_1$, $x_2$, $\ldots$, and an action sequence $u_1$, $u_2$, $\ldots$, during the execution of a plan. However, in the current setting, the state sequence is not known. Instead, at every stage, an observation, $y_k$, is obtained. The process depicted in Figure 11.2.

Figure 11.2: In each stage, $k$, an observation, $y_k \in Y$, is received and an action $u_k \in U$ is applied. The state, $x_k$, however, is hidden from the decision maker.

In previous formulations, the action space, $U(x)$, was generally allowed to depend on $x$. Since $x$ is unknown in the current setting, it would seem strange to allow the actions to depend on $x$. This would mean that inferences could be made regarding the state by simply noticing which actions are available.^11.1Instead, it will be assumed by default that $U$ is fixed for all $x \in X$. In some special contexts, however, $U(x)$ may be allowed to vary.