Markov Chains: Theory and Applications

book “Ancient Inhabitants of Jebel Moya” published by the Cambridge Press under the joint authorship of Rao and two anthropologists. On the basis of work done at CU during the two year period, 1946-1948, Rao earned a Ph.D. degree and a few years later Sc.D. degree of CU and the rare honor of life fellowship of Kings College, Cambridge. He retired from ISI in 1980 at the mandatory age of 60 after working for 40 years during which period he developed ISI as an international center for statistical education and research. He also took an active part in establishing state statistical bureaus to collect local statistics and transmitting them to Central Statistical Organization in New Delhi. Rao played a pivitol role in launching undergraduate and postgraduate courses at ISI. He is the author of 475 research publications and several breakthrough papers contributing to statistical theory and methodology for applications to problems in all areas of human endeavor. There are a number of classical statistical terms named after him, the most popular of which are Cramer-Rao inequality, Rao-Blackwellization, Rao’s Orthogonal arrays used in quality control, Rao’s score test, Rao’s Quadratic Entropy used in ecological work, Rao’s metric and distance which are incorporated in most statistical books. He is the author of 10 books, of which two important books are, Linear Statistical Inference which is translated into German, Russian, Czec, Polish and Japanese languages,and Statistics and Truth which is translated into, French, German, Japanese, Mainland Chinese, Taiwan Chinese, Turkish and Korean languages. He directed the research work of 50 students for the Ph.D. degrees who in turn produced 500 Ph.D.’s. Rao received 38 hon. Doctorate degree from universities in 19 countries spanning 6 continents. He received the highest awards in statistics in USA,UK and India: National Medal of Science awarded by the president of USA, Indian National Medal of Science awarded by the Prime Minister of India and the Guy Medal in Gold awarded by the Royal Statistical Society, UK. Rao was a recipient of the first batch of Bhatnagar awards in 1959 for mathematical sciences and and numerous medals in India and abroad from Science Academies. He is a Fellow of Royal Society (FRS),UK, and member of National Academy of Sciences, USA, Lithuania and Europe. In his honor a research Institute named as CRRAO ADVANCED INSTITUTE OF MATHEMATICS, STATISTICS AND COMPUTER SCIENCE was established in the campus of Hyderabad University. Arni S.R. Srinivasa Rao works in pure mathematics, applied mathematics, probability, and artificial intelligence and applications in medicine. He is a Professor at the Medical College of Georgia, Augusta University, U.S.A, and the Director of the Laboratory for Theory and Mathematical Modeling housed within the Division of Infectious Diseases, Medical College of Georgia, Augusta, U.S.A. Previously, Dr. Rao conducted research and/or taught at Mathematical Institute, University of Oxford (2003, 2005-07), Indian Statistical Institute (1998-2002, 2006-2012), Indian Institute of Science (2002-04), University of Guelph (2004-06). Until 2012, Dr. Rao held a permanent faculty position at the Indian Statistical Institute. He has won the Heiwa-Nakajima Award (Japan) and Fast Track Young Scientists Fellowship in Mathematical Sciences (DST, New Delhi). Dr. Rao also proved a major theorem in stationary population models, such as, Rao's Partition Theorem in Populations, Rao-Carey Theorem in stationary populations, and developed mathematical modeling-based policies for the spread of diseases like HIV, H5N1, COVID-19, etc. He developed a new set of network models for understanding avian pathogen biology on grid graphs (these were called chicken walk models), AI Models for COVID-19 and received wide coverage in the science media. Recently, he developed concepts such as “Exact Deep Learning Machines”, and “Multilevel Contours” within a bundle of Complex Number Planes.

Preface
Arni S.R. Srinivasa Rao
1. Markov Chain Estimation, Approximation, and Aggregation for Average Reward Markov Decision Processes and Reinforcement Learning
Ronald Ortner
2. Ladder processes: symmetric functions and semigroups
Philip Feinsilver
3. Continuous-time Markov Chains and Models: Study via Forward Kolmogorov System
Alexander Zeifman
4. Analysis of Data Following Finite-State Continuous-Time Markov Chains
Wenyaw Chan
5. Computational applications of poverty measurement through Markov model for income classes
Guglielmo D’Amico
6. Estimation and calibration of continuous time Markov chains
Manuel L. Esquível
7. Additive High-Order Markov Chains
Serhii Melnyk, Galyna Prytula and Oleg Victorovich Usatenko
8. The role of the random-product technique in the theory of Markov chains on a countable state space.
Brian Fralix, Amin Khademi and Farhad Hasankhani
9. On estimation problems based on type I Longla copulas
Martial Longla
10. Long time behaviour of continuous time Markov chains
Xueping Huang
11. To be Determined
Alan Krinik