11.4.2 Continuous-Time Information Spaces

Now assume that there is a continuum of stages. Most of the components of Section 11.4.1 remain the same. The spaces $X$, $Y$, $\Psi(x)$, $U$, and $\Theta(x,u)$ remain the same. The sensor mapping also remains the same. The main difference occurs in the state transition equation because the effect of nature must be expressed in terms of velocities. This was already introduced in Section 10.6. In that context, there was only uncertainty in predictability. In the current context there may be uncertainties in both predictability and in sensing the current state.

For the discrete-stage case, the history I-states were based on action and observation sequences. For the continuous-time case, the history instead becomes a function of time. As defined in Section 7.1.1, let $T$ denote a time interval, which may be bounded or unbounded. Let ${\tilde{y}_t}: [0,t] \rightarrow Y$ be called the observation history up to time $t \in T$. Similarly, let ${\tilde{u}_t}: [0,t) \rightarrow U$ and ${\tilde{x}_t}: [0,t] \rightarrow X$ be called the action history and state history, respectively, up to time $t \in T$.

Thus, the three kinds of sensor mappings in the continuous-time case are as follows:

1. A state-sensor mapping is expressed as $y(t) = h(x(t))$, in which $x(t)$ and $y(t)$ are the state and observation, respectively, at time $t \in T$.
2. A state-nature mapping is expressed as $y(t) = h(x(t),\psi(t))$, which implies that nature chooses some $\psi(t) \in \Psi(x(t))$ for each $t \in T$.
3. A history-based sensor mapping, which could depend on all of the states obtained so far. Thus, it depends on the entire function ${\tilde{x}_t}$. This could be denoted as $y(t) = h({\tilde{x}_t},\psi(t))$ if nature can also interfere with the observation.

If ${\tilde{u}_t}$ and ${\tilde{y}_t}$ are combined with the initial condition ${\eta_0}$, the history I-state at time $t$ is obtained as

$${\eta}_t = ({\eta_0},{\tilde{u}_t},{\tilde{y}_t}) .$$ (11.53)

The history I-space at time $t$ is the set of all possible ${\eta}_t$ and is denoted as ${{\cal I}_t}$. Note that ${{\cal I}_t}$ is a space of functions because each ${\eta}_t \in {{\cal I}_t}$ is a function of time. Recall that in the discrete-stage case, every ${{\cal I}_k}$ was combined into a single history I-space, ${\cal I}_{hist}$, using (11.18) or (11.19). The continuous-time analog is obtained as

$${\cal I}_{hist}= \displaystyle\strut \bigcup_{t \in T} {{\cal I}_t},$$ (11.54)

which is an irregular collection of functions because they have different domains; this irregularity also occurred in the discrete-stage case, in which ${\cal I}_{hist}$ was composed of sequences of varying lengths.

A continuous-time version of the cost functional in Formulation 11.1 can be given to evaluate the execution of a plan. Let $L$ denote a cost functional that may be applied to any state-action history $({\tilde{x}_t},{\tilde{u}_t})$ to yield

$$L({\tilde{x}_t},{\tilde{u}_t}) = \int_0^t l(x(t^\prime),u(t^\prime))dt^\prime + l_F(x(t)) ,$$ (11.55)

in which $l(x(t^\prime),u(t^\prime))$ is the instantaneous cost and $l_F(x(t))$ is a final cost.