Course Archives Statistical Quality Control & Operations Research Unit

Course:Markov Analysis & Modelling Level: Postgraduate Time: Currently offered

Syllabus Past Exams Syllabus: Introduction: Stochastic processes- first order stationary, second order stationary, orderly stochastic processes, time in stochastic process. Random walk model, Birth-death process Markov property, Strong Markov properties, Markov Process Markov Chain: Transition probabilities discrete time Markov chain, digraph representation ---definition and basic properties, class structure, first passage time, classification of states—transient / recurrent /irreducible /aperiodic /irregular /ergodic Markov chain, finite Markov chain. Rate of convergence to stationarity Continuous time Markov chains: continuous time random processes, some properties of Exponential distribution, Poisson process. Examples of Markov Chain Application: Brand selection problem, Inventory Management etc. with use of softwares. Reference Texts: 1. Stochastic processes: By Doob J L, John Wiley, 1990 2. Probability, random variables, and stochastic processes: By Papoulis A , McGraw-Hill, 1984 3. Markov Processes Vol 1 and II: By Dynkin E B ,Springer Verlag, Berlin, 1965 4. Finite Markov Chains: By Kemeny J. G, Snell J. L, Springer Verlag, New York,1976 5. Elements of the theory of Markov Processes and their applications: By Barucha-Reid AT, McGraw Hill, London, 1960 6. Introduction to Finite Markov Processes: By Adke S R and Manjunath S M, Wiley Eastern, Calcutta, 1984 Top of the page Past Exams

Midterm Semestral Supplementary and Back Paper Top of the page

[Indian Statistical Institute]