Simple procedures for finding mean first passage times in Markov chains
Loading...
Date
2005
DOI
Open Access Location
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Massey University
Rights
Abstract
The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations. By exploring the solution of a related set of equations, using suitable generalized inverses of the Markovian kernel I – P, where P is the transition matrix of a finite irreducible Markov chain, we are able to derive elegant new results for finding the mean first passage times. As a by-product we derive the stationary distribution of the Markov chain without the necessity of any further computational procedures. Standard techniques in the literature, using for example Kemeny and Snell’s fundamental matrix Z, require the initial derivation of the stationary distribution followed by the computation of Z, the inverse I – P + eπT where eT = (1, 1, …,1) and πT is the stationary probability vector. The procedures of this paper involve only the derivation of the inverse of a matrix of simple structure, based upon known characteristics of the Markov chain together with simple elementary vectors. No prior computations are required. Various possible families of matrices are explored leading to different related procedures.
Description
Keywords
First passage time, Markov chains
Citation
Hunter, J.J. (2005), Simple procedures for finding mean first passage times in Markov chains, Research Letters in the Information and Mathematical Sciences, 8, 209-226