Transition Matrices (DP IB Applications & Interpretation (AI)): Revision Note

Transition Matrices

What is a transition matrix?

• A transition matrix T shows the transition probabilities between the current state and the next state

  • The columns represent the current states

  • The rows represent the next states

• The element of T in the i^th row and j^th column gives the transition probability t_ij of :

  • the next state being the state corresponding to row i

  • given that the current state is the state corresponding to column j

• The probabilities in each column must add up to 1

• The transition matrix depends on how you assign the states to the columns

  • Each transition matrix for a Markov chain will contain the same elements

    • The rows and columns may be in different orders though

    • E.g. Sunny (S) & Cloudy (C) could be in the order S then C or C then S

What is an initial state probability matrix?

• An initial state probability matrix s₀ is a column vector which contains the probabilities of each state being chosen as the initial state

  • If you know which state was chosen as the initial state then that entry will be 1 and the others will all be zero

• You can find the state probability matrix s₁ which contains the probabilities of each state being chosen after one interval of time

  • s₁ = Ts₀

How do I find expected values after one interval of time?

• Suppose the Markov change represents a population moving between states

  • Examples include:

    • People in a town switching gyms each year

    • Children choosing a type of sandwich for their lunch each day

• Suppose the total population is fixed and equals N

• You can multiply the state probability matrix s₁ by N to find the expected number of members of the population at each state