MARKOV CHAINS. Andrey Markov

Transcription

1 MARKOV CHAINS

2 MARKOV CHAINS Andrey Markov

3 Markov Chains General Description We want to describe the behavior of a system as it moves (makes transiaons) probabilisacally from state to state. Basic or Markov AssumpAon The future depends only on the present (current state) and not on the past. That is, the future depends on the state we are in, not on how we arrived at this state.

4 Example 1 - Brand loyalty or Market Share For ease, assume that all cola buyers purchase either Coke or Pepsi in any given week. That is, there is a duopoly. Assume that if a customer purchases Coke in one week there is a 90% chance that the customer will purchase Coke the next week (and a 10% chance that the customer will purchase Pepsi). Similarly, 80% of Pepsi drinkers will repeat the purchase from week to week.

5 Example 1 - Developing the Markov Matrix States State 1 - Coke was purchased State 2 - Pepsi was purchased (note: states are qualitaave) Markov (transiaon or probability) Matrix From\To Coke Pepsi Coke Pepsi

6 Graphical DescripAon 1 The States From\To Coke Coke.9.1 Pepsi.2.8 Pepsi

7 Graphical DescripAon 2 TransiAons from Coke.9.1 From\To Coke Coke.9.1 Pepsi.2.8 Pepsi

8 Graphical DescripAon 3 All transiaons From\To Coke Coke.9.1 Pepsi.2.8 Pepsi

9 Example 1 - StarAng CondiAons Percentages IdenAfy probability of (percentage of shoppers) starang in either state (We will assume a 50/50 starang market share in our example that follows.) Assume we start in one specific state (by se`ng one probability to 1 and the remaining probabiliaes to 0) Counts (numbers) IdenAfy number of shoppers starang in either state

10 Example 1 From\To Coke Pepsi Coke Pepsi StarAng ProbabiliAes = 50% (or 50 people) each QuesAons What will happen in the short run (next 3 periods)? What will happen in the long run? Do starang probabiliaes influence long run?

11 Graphical Solution After 1 Transition.9(50)=45.8(50)=40.1(50)=5 (50)Coke(55) (50)Pepsi(45).2(50)=10 From\To Coke Coke.9.1 Pepsi.2.8 Pepsi

12 Graphical Solution After 2 Transitions.9(55)=49.5.8(45)=36.1(55)=5.5 (55)Coke(58.5) (45)Pepsi(41.5).2(45)=9 From\To Coke Coke.9.1 Pepsi.2.8 Pepsi

13 Graphical Solution After 3 Transitions.9(58.5)= (41.5)=33.2.1(58.5)=5.85 (58.5)Coke(60.95) (41.5)Pepsi(39.05).2(41.5)=8.3

14 Analyzing Markov Chains Using QM for Windows Module Markov Chains Number of states 2 Number of transiaons - 3

15 Example 1 Ager 3 transiaons n- step TransiAon probabiliaes End of Period 1 Coke Pepsi Coke Pepsi End prob (given iniaal) End of Period 2 Coke Pepsi Coke Pepsi End prob (given iniaal) End of Period 3 Coke Pepsi Coke Pepsi End prob (given iniaal) step transition matrix 2 step transition matrix 3 step transition matrix

16 Example 1 - Results (3 transitions, start =.5,.5) From\To Coke Pepsi Coke Pepsi Ending probability Steady State probability Note: We end up alternaang between Coke and Pepsi 3 step transition matrix Depends on initial conditions Independent of initial conditions

17 Example 2 - Student Progression Through a University States Freshman Sophomore Junior Senior Dropout Graduate (note: again, states are qualitaave)

18 Example 2 - Student Progression Through a University - States Freshman Sophomore Junior Senior Drop out Graduate Note that eventually you must end up in Grad or Drop-out.

19 Classification of states Absorbing Those states such that once you are in you never leave. Graduate, Drop Out Recurrent Those states to which you will always both leave and return at some time. Coke, Pepsi Transient States that you will eventually never return to Freshman, Sophomore, Junior, Senior

20 Example 3 - Diseases States no disease pre-clinical (no symptoms) clinical death (note: again states are qualitative) Purpose Transition probabilities can be different for different testing or treatment protocols

21 Example 4 - Customer Bill paying States State 0: Bill is paid in full State i: Bill is in arrears for i months, i= 1,2,,11 State 12: Deadbeat

22 Example 5 HIV infecaons Based on Can Difficult- to- Reuse Syringes Reduce the Spread of HIV among InjecAon Drug Users Caulkins, et. al. Interfaces, Vol 28, No. 3, May- June 1998, pp State State 0 Syringe is uninfected State 1 Syringe is infected Notes: P(0, 1) =.14 14% of drug users are infected with HIV P(1, 0) = % of the Ame the virus dies; 33% of the Ame it is killed by bleaching

23 Example 6 - Baseball States State 0 - no outs, bases empty State 1 - no outs, runner on first State 2 - no outs, runner on second State 3 - no outs, runner on third State 4 - no outs, runners on first, second State 5 - no outs, runners on first, third State 6 - no outs, runners on second, third State 7 - no outs, runners on first, second, third. Repeat for 1 out and 2 outs for a total of 24 states Moneyball by Michael Lewis, p 134