# Two state Markov model - State Probability Distribution

Discussion in 'MATLAB' started by Adshak, Oct 2, 2008.

1. Dear All,

Let us consider that I have a Markov Model with two states:

State_ 0 and State_1

For this I have a transition rate Matrix,which is symmetric and
balanced, example:

R_ij = ( 0 1; 1 0) , where R_01 = and R_10 = 1; ,

The probability transition matrix is given by use of time interval dt
(dt = 0.001) as follows:

P_ij = (1-R_01dt R_01dt; R_10dt 1-R_10dt);

which gives me the transition probability matrix:

(0.999 0.001; 0.001 0.999)

Now if I analytically derive the equation of the total probability
density function (steady state) of the markov model this has been
found to be:

Pdf of 2state markov model = p_0 *R_01*exp(-R_01*t) + p_1*R_10*exp(-
R10*t) -----> (1)

i.e basically the sum of probability distributions of being in states
0 and states 1 respectively.

here p_0 = R_10/(R_10+R_01) = 0.5 and p_1 = R_01/(R_10+R_01) = 0.5

So substituting the values above inthe equation (1):

pdf = 0.5 * 1 * exp(-1*t) + 0.5 *1*exp(-1*t)

If I would have plotted this against the actual pdf based on simulated
results,this fits exactly so this equation holds good for the
symmetric rate matrix. This remains the same for all values of R as
long as R_01 = R_10.

My question is how would the equation change in case of R_01 not equal
to R_10.

For example if R_01 is much greater than R_10. I have tried this thing
with the above equation, but the curve observed is anything but a fit
to the actual pdf.

Any suggestions, I will be really grateful to you.

Thanks.

2. ### karlGuest

All rows and columns of the matrix have to add up to unity. Try to
find such a matrix with R_01 =/ R_10.

Ciao

Karl

karl, Oct 3, 2008

3. ### karlGuest

Oops, that was wrong as far as I see now.

karl, Oct 3, 2008