The information space is just another state space

It will become important throughout this chapter and Chapter 12 to view the I-space as an ordinary state space. It only seems special because it is derived from another state space, but once this is forgotten, it exhibits many properties of an ordinary state space in planning. One nice feature is that the state in this special space is always known. Thus, by converting from an original state space to its I-space, we also convert from having imperfect state information to always knowing the state, albeit in a larger state space.

One important consequence of this interpretation is that the state transition equation can be lifted into the I-space to obtain an information transition function, ${f_{\cal I}}$. Suppose that there are no sensors, and therefore no observations. In this case, future I-states are predictable, which leads to

$${\eta}_{k+1} = {f_{\cal I}}({\eta}_k,u_k) .$$ (11.24)

The function ${f_{\cal I}}$ generates ${\eta}_{k+1}$ by concatenating $u_k$ onto ${\eta}_k$.

Now suppose that there are observations, which are generally unpredictable. In Section 10.1, the nature action $\theta_k \in \Theta(x,u)$ was used to model the unpredictability. In terms of the information transition equation, $y_{k+1}$ serves the same purpose. When the decision is made to apply $u_k$, the observation $y_{k+1}$ is not yet known (just as $\theta_k$ is unknown in Section 10.1). In a sequential game against nature with perfect state information, $x_{k+1}$ is directly observed at the next stage. For the information transition equation, $y_{k+1}$ is instead observed, and ${\eta}_{k+1}$ can be determined. Using the history I-state representation, (11.14), simply concatenate $u_k$ and $y_{k+1}$ onto the histories in ${\eta}_k$ to obtain ${\eta}_{k+1}$. The information transition equation is expressed as

$${\eta}_{k+1} = {f_{\cal I}}({\eta}_k,u_k,y_{k+1}) ,$$ (11.25)

with the understanding that $y_{k+1}$ plays the same role as $\theta_k$ in the case of perfect state information and unpredictable future states. Even though nature causes future I-states to be unpredictable, the current I-state is always known. A plan, $\pi : {\cal I}\rightarrow U$, now seems like a state-feedback plan, if the I-space is viewed as a state space. The transitions are all specified by ${f_{\cal I}}$.

The costs in this new state space can be derived from the original cost functional, but a maximization or expectation is needed over all possible states given the current information. This will be covered in Section 12.1.