Fast Sequential Monte Carlo Methods for Counting and Optimization

REUVEN Y. RUBINSTEIN, DSC, was Professor Emeritus in the Faculty of Industrial Engineering and Management at Technion-Israel Institute of Technology. The author of over 100 articles and six books, Dr. Rubinstein was also the inventor of the popular score-function method in simulation analysis and generic cross-entropy methods for combinatorial optimization and counting. AD RIDDER, PHD, is Associate Professor of Operations Research at Vrije Universiteit Amsterdam. His research interests include rare event simulation, applied probability problems, queuing models, and Monte Carlo methods. RADISLAV VAISMANis the author of numerous journal articles, and his research interests include rare event simulation, randomized algorithms, and on-line planning.

Preface xi

1. Introduction to Monte Carlo Methods 1

2. Cross-Entropy Method 6

2.1. Introduction 6

2.2. Estimation of Rare-Event Probabilities 7

2.3. Cross-Entrophy Method for Optimization 18

2.3.1. The Multidimensional 0/1 Knapsack Problem 21

2.3.2. Mastermind Game 23

2.3.3. Markov Decision Process and Reinforcement Learning 25

2.4. Continuous Optimization 31

2.5. Noisy Optimization 33

2.5.1. Stopping Criterion 35

3. Minimum Cross-Entropy Method 37

3.1. Introduction 37

3.2. Classic MinxEnt Method 39

3.3. Rare Events and MinxEnt 43

3.4. Indicator MinxEnt Method 47

3.4.1. Connection between CE and IME 51

3.5. IME Method for Combinatorial Optimization 52

3.5.1. Unconstrained Combinatorial Optimization 52

3.5.2. Constrained Combinatorial Optimization: The Penalty Function Approach 54

4. Splitting Method for Counting and Optimization 56

4.1. Background 56

4.2. Quick Glance at the Splitting Method 58

4.3. Splitting Algorithm with Fixed Levels 64

4.4. Adaptive Splitting Algorithm 68

4.5. Sampling Uniformly on Discrete Regions 74

4.6. Splitting Algorithm for Combinatorial Optimization 75

4.7. Enhanced Splitting Method for Counting 76

4.7.1. Counting with the Direct Estimator 76

4.7.2. Counting with the Capture–Recapture Method 77

4.8. Application of Splitting to Reliability Models 79

4.8.1. Introduction 79

4.8.2. Static Graph Reliability Problem 82

4.8.3. BMC Algorithm for Computing S(Y) 84

4.8.4. Gibbs Sampler 85

4.9. Numerical Results with the Splitting Algorithms 86

4.9.1. Counting 87

4.9.2. Combinatorial Optimization 101

4.9.3. Reliability Models 102

4.10. Appendix: Gibbs Sampler 104

5. Stochastic Enumeration Method 106

5.1. Introduction 106

5.2. OSLA Method and Its Extensions 110

5.2.1. Extension of OSLA: nSLA Method 112

5.2.2. Extension of OSLA for SAW: Multiple Trajectories 115

5.3. SE Method 120

5.3.1. SE Algorithm 120

5.4. Applications of SE 127

5.4.1. Counting the Number of Trajectories in a Network 127

5.4.2. SE for Probabilities Estimation 131

5.4.3. Counting the Number of Perfect Matchings in a Graph 132

5.4.4. Counting SAT 135

5.5. Numerical Results 136

5.5.1. Counting SAW 137

5.5.2. Counting the Number of Trajectories in a Network 137

5.5.3. Counting the Number of Perfect Matchings in a Graph 140

5.5.4. Counting SAT 143

5.5.5. Comparison of SE with Splitting and SampleSearch 146

A. Additional Topics 148

A.1. Combinatorial Problems 148

A.1.1. Counting 149

A.1.2. Combinatorial Optimization 154

A.2. Information 162

A.2.1. Shannon Entropy 162

A.2.2. Kullback–Leibler Cross-Entropy 163

A.3. Efficiency of Estimators 164

A.3.1. Complexity 165

A.3.2. Complexity of Randomized Algorithms 166

Bibliography 169

Abbreviations and Acronyms 177

List of Symbols 178

Index 181