Your History

Menu

Stochastic Discrete-Time Dynamical System

Prerequisites

Stochastic Discrete-Time System with Input | \(p\left( \mathbf{X}_{n+1} = \mathbf{x}_{j} \,\vert\, \mathbf{randomVar}_{n} = \mathbf{x}_{i}, \mathbf{U}_{n} = a \right) = T_a(i, j)\)
Output Function | \( O \)

Description

A Markov system with an update operator can be extended by system inputs. It is possible to define an output function as another probability matrix, resulting in a set of equations that can describe the behaviour of the system.

\[\begin{cases} p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000011}{j}} \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}}, \mathbf{U}_n = a \right) = \htmlClass{sdt-0000000027}{T}_a(\htmlClass{sdt-0000000018}{i}, \htmlClass{sdt-0000000011}{j}) \\ p\left( \mathbf{Y}_n = \htmlClass{sdt-0000000086}{\mathbf{y}}_k \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_n = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}} \right) = \htmlClass{sdt-0000000056}{O}( i, k ) \end{cases}\]

Symbols Used:

This symbol represents a random variable. It is a measurable function that maps a sample space of possible outcomes to a a measurable space.
\( j \)

This is a secondary symbol for an iterator, a variable that changes value to refer to a series of elements
\( i \)

This is the symbol for an iterator, a variable that changes value to refer to a sequence of elements.

\( T \)

This is the symbol for a dynamical system's update operator.
\( \mathbf{x} \)

This symbol represents a state of the dynamical system at some time point.
\( O \)

This symbol represents the output function of a dynamical system.
\( \mathbf{y} \)

This symbol represents the output of a dynamical system.
\( n \)

This symbol represents any given whole number, \( n \in \htmlClass{sdt-0000000014}{\mathbb{W}}\).

Derivation

  1. Consider a controlled Markov Chain - a stochastic discrete-time dynamical system with a finite state space and inputs.
  2. The update equation is given by:
    \[p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000011}{j}} \,\vert\, \mathbf{randomVar}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}}, \mathbf{U}_{\htmlClass{sdt-0000000117}{n}} = a \right) = \htmlClass{sdt-0000000027}{T}_a(\htmlClass{sdt-0000000018}{i}, \htmlClass{sdt-0000000011}{j})\]
  3. Note the definition of an output function:

    The symbol \( O \) represents a function that generates outputs of a dynamical system upon observing a particular state. This corresponds to a "measurement" of the system, often an appropriate analogy given the partial observability of real-life dynamical systems.

  4. Assuming the output observations are stochastic and independent of the system input, the output can be described by an output function in the form of another Markov transition matrix:
    \[ p\left( \mathbf{Y}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000086}{\mathbf{y}}_k \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}} \right) = \htmlClass{sdt-0000000056}{O}( i, k ) \]
  5. Grouping the two system equations, we get:
    \[ \begin{cases} p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000011}{j}} \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}}, \mathbf{U}_n = a \right) = \htmlClass{sdt-0000000027}{T}_a(\htmlClass{sdt-0000000018}{i}, \htmlClass{sdt-0000000011}{j}) \\ p\left( \mathbf{Y}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000086}{\mathbf{y}}_k \,\vert\, \mathbf{X}_n = \htmlClass{sdt-0000000046}{\mathbf{x}}_i \right) = \htmlClass{sdt-0000000056}{O}( i, k ) \end{cases} \]
    as required.

References

  1. Jaeger, H. (n.d.). Neural Networks (AI) (WBAI028-05) Lecture Notes BSc program in Artificial Intelligence. Retrieved May 17, 2024, from https://www.ai.rug.nl/minds/uploads/LN_NN_RUG.pdf