Need help now? Support now: crisis lines by country.

Pain words · Mathematical Institute for Research of Pain

Markov chain

Also called Markov model

A Markov chain is a random process in which the chance of each next state depends only on the present state, not on the path that led there.

In plain words

Sort each day’s pain into a few states, such as low, moderate and high. A Markov chain is a model in which tomorrow’s chances depend only on today’s state. Two days that are both “moderate” have the same chances for tomorrow in the model, however each of them was reached.

Whether real pain behaves this way is a question for data. Often the state has to record more than today’s rating, for example how long a flare has lasted, before the assumption is reasonable.

What researchers mean

A Markov chain in discrete time on a countable state space SS is a sequence of random variables X0,X1,X2,…X_0, X_1, X_2, \dots with the Markov property: given the present state, the past carries no further information about the next step. In the time-homogeneous case the chance of moving from state ii to state jj is a fixed number pijp_{ij}, and the matrix P=(pij)P = (p_{ij}) has non-negative entries with each row summing to 1. The name honours the Russian mathematician Andrei Markov. Continuous-time chains, and Markov processes on continuous state spaces such as diffusions, carry the same idea further.

The transition chances are parameters of the model. Proportions counted from a finite diary are estimates of them, and they carry estimation error.

The maths

In discrete time, with transition chances pijp_{ij}, for every step nn and all states:

ℙ(Xn+1=j∣Xn=i,Xn−1=in−1,…,X0=i0)=pij\begin{aligned} &\mathbb{P}(X_{n+1} = j \mid X_n = i, \\ &\qquad X_{n-1} = i_{n-1}, \dots, X_0 = i_0) = p_{ij} \end{aligned}
Xn+1=jX_{n+1} = j
next state is j
Xn=iX_n = i
present state is i
Xn−1=in−1,…,X0=i0X_{n-1} = i_{n-1}, \dots, X_0 = i_0
the earlier path
pijp_{ij}
chance of i to j

The equation holds in the model by assumption; for real ratings it has to be checked, and if memory matters the state must be enlarged to carry it.

These pages explain words, reasoning and evidence. Nothing here is medical advice or a forecast about any person.

Sources

  1. Norris JR. Markov Chains. Cambridge: Cambridge University Press; 1997. doi:10.1017/CBO9780511810633

Questions about this

Inside the Institute

Pain math explains what the Institute does, and Questions about pain answers some questions people in pain ask, in the same layers. The rooms, the Lab and the full Pain Bias Codex open with an invite code.

No code yet? Ask for an invite. A person reads every request and writes back by hand with an invite code, so it may take a few days.

Need help now? Support now: crisis lines by country.

Except where a page says otherwise, the Institute’s own text and figures are shared under CC BY-NC 4.0. Pictures and quotations from others keep their own credits and licences.

Back to the invite page · Sign in

Paper