Markov chains are a mathematical model for random passing between states with pre-known transition probabilities. Depending on the number of possible states, we know dicrete and continuous - time markov chains. Markov chains with a finite number of states can be effectively considered with linear algebra tools. Let S={1,2,…,n} be the finite set of states of a markov chain. The transition probability between states s_i and s_j , denoted by p_ij, gives us the transition matrix P of size n×n . Matrix obtained like this is a (row) stochastic matrix. Special algebraic properties of stochastic matrices allow us to predict the behaviour of Markov chains after a certain number of steps. With the help of knowing Markov chains, various simpler mathematical problems and simple board games, such as Hi Ho Cherry O, Snakes and ladders, Monopoly, can be dealt with.
|
