Modeling and Analysis of Stochastic Systems, Second Edition

Vidyadhar G. Kulkarni is a Norman Johnson Professor in the Department of Statistics and Operations Research at the University of North Carolina at Chapel Hill.

Introduction What in the World Is a Stochastic Process? How to Characterize a Stochastic Process What Do We Do with a Stochastic Process? Discrete-Time Markov Chains: Transient Behavior Definition and Characterization Examples DTMCs in Other Fields Marginal Distributions Occupancy Times Computation of Matrix Powers DTMCs: First Passage Times Definitions Cumulative Distribution Function of T Absorption Probabilities Expectation of T Generating Function and Higher Moments of T DTMCs: Limiting Behavior Exploring the Limiting Behavior by Examples Irreducibility and Periodicity Recurrence and Transience Determining Recurrence and Transience: Infinite DTMCs Limiting Behavior of Irreducible DTMCs Examples: Limiting Behavior of Infinite State-Space Irreducible DTMCs Limiting Behavior of Reducible DTMCs DTMCs with Costs and Rewards Reversibility Poisson Processes Exponential Distributions Poisson Process: Definitions Event Times in a Poisson Process Superposition and Splitting of Poisson Processes Non-Homogenous Poisson Process Compound Poisson Process Continuous-Time Markov Chains Definitions and Sample Path Properties Examples Transient Behavior: Marginal Distribution Transient Behavior: Occupancy Times Computation of P(t): Finite State-Space Computation of P(t): Infinite State-Space First-Passage Times Exploring the Limiting Behavior by Examples Classification of States Limiting Behavior of Irreducible CTMCs Limiting Behavior of Reducible CTMCs CTMCs with Costs and Rewards Phase-Type Distributions Reversibility Queueing Models Introduction Properties of General Queueing Systems Birth and Death Queues Open Queueing Networks Closed Queueing Networks Single Server Queues Retrial Queue Infinite Server Queue Renewal Processes Introduction Properties of N(t) The Renewal Function Renewal-Type Equation Key Renewal Theorem Recurrence Times Delayed Renewal Processes Alternating Renewal Processes Semi-Markov Processes Renewal Processes with Costs/Rewards Regenerative Processes Markov Regenerative Processes Definitions and Examples Markov Renewal Process and Markov Renewal Function Key Renewal Theorem for MRPs Extended Key Renewal Theorem Semi-Markov Processes: Further Results Markov Regenerative Processes Applications to Queues Diffusion Processes Brownian Motion Sample Path Properties of BM Kolmogorov Equations for Standard Brownian Motion First Passage Times Reflected SBM Reflected BM and Limiting Distributions BM and Martingales Cost/Reward Models Stochastic Integration Stochastic Differential Equations Applications to Finance Epilogue Appendix A: Probability of Events Appendix B: Univariate Random Variables Appendix C: Multivariate Random Variables Appendix D: Generating Functions Appendix E: Laplace-Stieltjes Transforms Appendix F: Laplace Transforms Appendix G: Modes of Convergence Appendix H: Results from Analysis Appendix I: Difference and Differential Equations Answers to Selected Problems References Index Exercises appear at the end of each chapter.