Markov Decision Processes in Practice

Richard Boucherie received M.Sc. degrees in 1988 in applied mathematics and theoretical physics from the Universiteit Leiden, and received the Ph.D. degree in econometrics in 1992 from the Vrije Universiteit, Amsterdam. Since 2000 he is with the department of Applied Mathematics of the University of Twente, where he was appointed in 2003 as full professor of Stochastic Operations Research. His research interests are in queueing theory, Petri nets and random walks with application areas including wireless and sensor networks, healthcare, road traffic, and network intrusion detection and prevention. Richard is co-founder of the University of Twente Center for Healthcare Operations Improvement and Research (CHOIR) in the area of healthcare logistics, and chair of the Postdoctorate programme in healthcare logistics. In 2014 he co-founded the spin-off company Rhythm, that carries out actual implementations of healthcare logistics solutions in healthcare organisations.  < Nico M. van Dijk  has been active in the area of Stochastic Operations Research for over 30 years. He has always been stimulated by real-life stochastics , as reflected by his Ph.D. on Controlled Markov Processes: Time-discretization (1983), by his Wiley book on Queueing Networks and Product Forms : A systems approach (1993) and by practical application papers such as on communications, call centers, railways and ICU (Intensive care unit) systems. For a decade he works in close cooperation with the Dutch Blood banks. For joint work on blood supply, the OR team that he guided, became finalist for the EURO 2009 Excellence in OR Practice Award and received the INFORMS 2011 First Prize Interactive Poster Award. He is affiliated with the Stochastic Operations Research group at the University of Twente.

One-Step Improvement  Ideas And Computational Aspects.- Value Function Approximation In Complex Queueing Systems.- Approximate Dynamic Programming By Practical Examples.- Server Optimization Of Infinite Queueing Systems.- Structures Of Optimal Policies In Mdps With Unbounded Jumps: The State Of Our Art.- Markov Decision Processes For Screening And Treatment Of Chronic Diseases.- Stratified Breast Cancer Follow-Up Using A Partially Observable MDP.- Advance Patient Appointment  Scheduling.- Optimal Ambulance Dispatching.- Blood Platelet Inventory Management.- Stochastic Dynamic Programming For Noise Load Management.- Allocation In A Vertical Rotary Car Park.- Dynamic Control Of Traffic Lights.- Smart Charging Of Electric Vehicles.- Analysis Of A Stochastic Lot Scheduling Problem With Strict Due-Dates.- Optimal Fishery Policies.- Near-Optimal Switching Strategies For A Tandem Queue.- Wireless Channel Selection With Restless Bandits.- Flexible Staffing For Call Centers With Non-Stationary Arrival Rates.- MDP For Query-Based Wireless Sensor Networks.- Optimal Portfolios And Pricing Of Financial Derivatives Under Proportional Transaction Costs.