Course Archives Statistical Quality Control & Operations Research Unit
Course:Markov Analysis & Modelling
Level: Postgraduate
Time: Currently offered
Past Exams

Syllabus: Introduction: Stochastic processes- first order stationary, second order stationary, orderly stochastic processes, time in stochastic process.
Random walk model, Birth-death process
Markov property, Strong Markov properties, Markov Process
Markov Chain: Transition probabilities discrete time Markov chain, digraph representation ---definition and basic properties, class structure, first passage time, classification of states—transient / recurrent /irreducible /aperiodic /irregular /ergodic Markov chain, finite Markov chain.
Rate of convergence to stationarity
Continuous time Markov chains: continuous time random processes, some properties of Exponential distribution, Poisson process.
Examples of Markov Chain Application: Brand selection problem, Inventory Management etc. with use of softwares.

Reference Texts:
1. Stochastic processes: By Doob J L, John Wiley, 1990
2. Probability, random variables, and stochastic processes: By Papoulis A , McGraw-Hill, 1984
3. Markov Processes Vol 1 and II: By Dynkin E B ,Springer Verlag, Berlin, 1965
4. Finite Markov Chains: By Kemeny J. G, Snell J. L, Springer Verlag, New York,1976
5. Elements of the theory of Markov Processes and their applications: By Barucha-Reid AT, McGraw Hill, London, 1960
6. Introduction to Finite Markov Processes: By Adke S R and Manjunath S M, Wiley Eastern, Calcutta, 1984

Top of the page

Past Exams
Supplementary and Back Paper

Top of the page

[Indian Statistical Institute]