Description

In the case of a stochastic discrete-time system obeying the Markov property, a Markov transition matrix can be used as the update operator.

Symbols Used:

Derivation

1. Consider a stochastic system with discrete time and a finite number of \( \vert \htmlClass{sdt-0000000038}{\mathcal{X}} \vert = n \) possible states:
   \[ \htmlClass{sdt-0000000038}{\mathcal{X}} = \{ \htmlClass{sdt-0000000046}{\mathbf{x}}_1, \htmlClass{sdt-0000000046}{\mathbf{x}}_2, ..., \htmlClass{sdt-0000000046}{\mathbf{x}}_n \} \]
2. Assume the system obeys the Markov property:
   \[p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_j \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_i, ... , \mathbf{\htmlClass{sdt-0000000005}{X}}_{1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_1 \right) = p\left( \mathbf{X}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_j \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_i \right)\]
3. The transition from the current state \( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}} \) to the following state \( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} \) takes place as dictated by the system's transition matrix:
   \[\left[ \htmlClass{sdt-0000000087}{T} \right]_{\htmlClass{sdt-0000000018}{i},\htmlClass{sdt-0000000011}{j}} = p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000011}{j}} \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}} \right)\]
4. The distribution of \( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} \) can be computed (and its realization through e.g. distribution sampling).
5. Thus the transition matrix functions as an update operator with values given by:
   \[ p\left( \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}+1} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000011}{j}} \,\vert\, \mathbf{\htmlClass{sdt-0000000005}{X}}_{\htmlClass{sdt-0000000117}{n}} = \htmlClass{sdt-0000000046}{\mathbf{x}}_{\htmlClass{sdt-0000000018}{i}} \right) = \htmlClass{sdt-0000000027}{T}(\htmlClass{sdt-0000000018}{i}, \htmlClass{sdt-0000000011}{j}) \]
   as required.