Limits of Detection in Chemical Analysis

Edward Voigtman is emeritus professor of chemistry at the University of Massachusetts – Amherst, having retired after 29 years as a faculty member. His interests include ultrasensitive detection techniques, applications of signal/noise theory, optical calculus-based computer simulation of spectrometric systems and analytical detection limit theory and practice.

Preface xv

Acknowledgment xix

About the Companion Website xx

1 Background 1

1.1 Introduction 1

1.2 A Short List of Detection Limit References 2

1.3 An Extremely Brief History of Limits of Detection 2

1.4 An Obstruction 3

1.5 An Even Bigger Obstruction 3

1.6 What Went Wrong? 4

1.7 Chapter Highlights 5

References 5

2 Chemical Measurement Systems and their Errors 9

2.1 Introduction 9

2.2 Chemical Measurement Systems 9

2.3 The Ideal CMS 10

2.4 CMS Output Distributions 12

2.5 Response Function Possibilities 12

2.6 Nonideal CMSs 15

2.7 Systematic Error Types 15

2.7.1 What Is Fundamental Systematic Error? 16

2.7.2 Why Is an Ideal Measurement System Physically Impossible? 16

2.8 Real CMSs, Part 1 17

2.8.1 A Simple Example 18

2.9 Random Error 19

2.10 Real CMSs, Part 2 21

2.11 Measurements and PDFs 22

2.11.1 Several Examples of Compound Measurements 22

2.12 Statistics to the Rescue 23

2.13 Chapter Highlights 24

References 24

3 The Response, Net Response, and Content Domains 25

3.1 Introduction 25

3.2 What is the Blank’s Response Domain Location? 27

3.3 False Positives and False Negatives 28

3.4 Net Response Domain 29

3.5 Blank Subtraction 29

3.6 Why Bother with Net Responses? 31

3.7 Content Domain and Two Fallacies 31

3.8 Can an Absolute Standard Truly Exist? 33

3.9 Chapter Highlights 34

References 34

4 Traditional Limits of Detection 37

4.1 Introduction 37

4.2 The Decision Level 37

4.3 False Positives Again 38

4.4 Do False Negatives Really Matter? 40

4.5 False Negatives Again 40

4.6 Decision Level Determination Without a Calibration Curve 41

4.7 Net Response Domain Again 41

4.8 An Oversimplified Derivation of the Traditional Detection Limit, XDC 42

4.9 Oversimplifications Cause Problems 43

4.10 Chapter Highlights 43

References 43

5 Modern Limits of Detection 45

5.1 Introduction 45

5.2 Currie Detection Limits 46

5.3 Why were p and q Each Arbitrarily Defined as 0.05? 48

5.4 Detection Limit Determination Without Calibration Curves 49

5.5 A Nonparametric Detection Limit Bracketing Experiment 49

5.6 Is There a Parametric Improvement? 51

5.7 Critical Nexus 52

5.8 Chapter Highlights 53

References 53

6 Receiver Operating Characteristics 55

6.1 Introduction 55

6.2 ROC Basics 55

6.3 Constructing ROCs 57

6.4 ROCs for Figs 5.3 and 5.4 59

6.5 A Few Experimental ROC Results 60

6.6 Since ROCs may Work Well, Why Bother with Anything Else? 64

6.7 Chapter Highlights 65

References 65

7 Statistics of an Ideal Model CMS 67

7.1 Introduction 67

7.2 The Ideal CMS 67

7.3 Currie Decision Levels in all Three Domains 70

7.4 Currie Detection Limits in all Three Domains 71

7.5 Graphical Illustrations of eqns 7.3–7.8 72

7.6 An Example: are Negative Content Domain Values Legitimate? 74

7.7 Tabular Summary of the Equations 76

7.8 Monte Carlo Computer Simulations 77

7.9 Simulation Corroboration of the Equations in Table 7.2 78

7.10 Central Confidence Intervals for Predicted x Values 80

7.11 Chapter Highlights 81

References 81

8 If Only the True Intercept is Unknown 83

8.1 Introduction 83

8.2 Assumptions 83

8.3 Noise Effect of Estimating the True Intercept 83

8.4 A Simple Simulation in the Response and NET Response Domains 84

8.5 Response Domain Effects of Replacing the True Intercept by an Estimate 86

8.6 Response Domain Currie Decision Level and Detection Limit 88

8.7 NET Response Domain Currie Decision Level and Detection Limit 88

8.8 Content Domain Currie Decision Level and Detection Limit 89

8.9 Graphical Illustrations of the Decision Level and Detection Limit Equations 89

8.10 Tabular Summary of the Equations 90

8.11 Simulation Corroboration of the Equations in Table 8.1 91

8.12 Chapter Highlights 93

9 If Only the True Slope is Unknown 95

9.1 Introduction 95

9.2 Possible “Divide by Zero” Hazard 96

9.3 The t Test for tslope 96

9.4 Response Domain Currie Decision Level and Detection Limit 97

9.5 NET Response Domain Currie Decision Level and Detection Limit 97

9.6 Content Domain Currie Decision Level and Detection Limit 97

9.7 Graphical Illustrations of the Decision Level and Detection Limit Equations 98

9.8 Tabular Summary of the Equations 99

9.9 Simulation Corroboration of the Equations in Table 9.1 99

9.10 Chapter Highlights 101

References 101

10 If the True Intercept and True Slope are Both Unknown 103

10.1 Introduction 103

10.2 Important Definitions, Distributions, and Relationships 104

10.3 The Noncentral t Distribution Briefly Appears 105

10.4 What Purpose Would be Served by Knowing 𝛿? 106

10.5 Is There a Viable Way of Estimating 𝛿? 106

10.6 Response Domain Currie Decision Level and Detection Limit 107

10.7 NET Response Domain Currie Decision Level and Detection Limit 107

10.8 Content Domain Currie Decision Level and Detection Limit 108

10.9 Graphical Illustrations of the Decision Level and Detection Limit Equations 108

10.10 Tabular Summary of the Equations 109

10.11 Simulation Corroboration of the Equations in Table 10.3 109

10.12 Chapter Highlights 109

References 111

11 If Only the Population Standard Deviation is Unknown 113

11.1 Introduction 113

11.2 Assuming 𝜎0 is Unknown, How may it be Estimated? 114

11.3 What Happens if 𝜎0 is Estimated by s0? 114

11.4 A Useful Substitution Principle 116

11.5 Response Domain Currie Decision Level and Detection Limit 116

11.6 NET Response Domain Currie Decision Level and Detection Limit 117

11.7 Content Domain Currie Decision Level and Detection Limit 117

11.8 Major Important Differences From Chapter 7 117

11.9 Testing for False Positives and False Negatives 120

11.10 Correction of a Slightly Misleading Figure 121

11.11 An Informative Screencast 121

11.12 Central Confidence Intervals for 𝜎 and s 122

11.13 Central Confidence Intervals for YC and YD 122

11.14 Central Confidence Intervals for XC and XD 123

11.15 Tabular Summary of the Equations 123

11.16 Simulation Corroboration of the Equations in Table 11.1 123

11.17 Chapter Highlights 125

References 125

12 If Only the True Slope is Known 127

12.1 Introduction 127

12.2 Response Domain Currie Decision Level and Detection Limit 127

12.3 NET Response Domain Currie Decision Level and Detection Limit 128

12.4 Content Domain Currie Decision Level and Detection Limit 128

12.5 Graphical Illustrations of the Decision Level and Detection Limit Equations 128

12.6 Tabular Summary of the Equations 128

12.7 Simulation Corroboration of the Equations in Table 12.1 129

12.8 Chapter Highlights 129

13 If Only the True Intercept is Known 131

13.1 Introduction 131

13.2 Response Domain Currie Decision Level and Detection Limit 132

13.3 NET Response Domain Currie Decision Level and Detection Limit 132

13.4 Content Domain Currie Decision Level and Detection Limit 132

13.5 Tabular Summary of the Equations 133

13.6 Simulation Corroboration of the Equations in Table 13.1 133

13.7 Chapter Highlights 135

References 135

14 If all Three Parameters are Unknown 137

14.1 Introduction 137

14.2 Response Domain Currie Decision Level and Detection Limit 137

14.3 NET Response Domain Currie Decision Level and Detection Limit 138

14.4 Content Domain Currie Decision Level and Detection Limit 138

14.5 The Noncentral t Distribution Reappears for Good 138

14.6 An Informative Computer Simulation 139

14.7 Confidence Interval for xD, with a Major Proviso 142

14.8 Central Confidence Intervals for Predicted x Values 143

14.9 Tabular Summary of the Equations 143

14.10 Simulation Corroboration of the Equations in Table 14.1 143

14.11 An Example: DIN 32645 145

14.12 Chapter Highlights 146

References 147

15 Bootstrapped Detection Limits in a Real CMS 149

15.1 Introduction 150

15.2 Theoretical 151

15.2.1 Background 151

15.2.2 Blank Subtraction Possibilities 151

15.2.3 Currie Decision Levels and Detection Limits 152

15.3 Experimental 153

15.3.1 Experimental Apparatus 153

15.3.2 Experiment Protocol 153

15.3.3 Testing the Noise: Is It AGWN? 156

15.3.4 Bootstrapping Protocol in the Experiments 157

15.3.5 Estimation of the Experimental Noncentrality Parameter 160

15.3.6 Computer Simulation Protocol 160

15.4 Results and Discussion 161

15.4.1 Results for Four Standards 161

15.4.2 Results for 3–12 Standards 162

15.4.3 Toward Accurate Estimates of XD 163

15.4.4 How the XD Estimates Were Obtained 164

15.4.5 Ramifications 165

15.5 Conclusion 165

Acknowledgments 166

References 166

15.6 Postscript 167

15.7 Chapter Highlights 167

16 Four Relevant Considerations 169

16.1 Introduction 169

16.2 Theoretical Assumptions 170

16.3 Best Estimation of 𝛿 171

16.4 Possible Reduction in the Number of Expressions? 172

16.5 Lowering Detection Limits 174

16.6 Chapter Highlights 178

References 178

17 Neyman–Pearson Hypothesis Testing 181

17.1 Introduction 181

17.2 Simulation Model for Neyman–Pearson Hypothesis Testing 181

17.3 Hypotheses and Hypothesis Testing 183

17.3.1 Hypotheses Pertaining to False Positives 183

17.3.1.1 Hypothesis 1 183

17.3.1.2 Hypothesis 2 183

17.3.2 Hypotheses Pertaining to False Negatives 185

17.3.2.1 Hypothesis 3 185

17.3.2.2 Hypothesis 4 185

17.4 The Clayton, Hines, and Elkins Method (1987–2008) 189

17.5 No Valid Extension for Heteroscedastic Systems 191

17.6 Hypothesis Testing for the 𝛿critical Method 192

17.6.1 Hypothesis Pertaining to False Positives 192

17.6.1.1 Hypothesis 5 192

17.6.2 Hypothesis Pertaining to False Negatives 192

17.6.2.1 Hypothesis 6 192

17.7 Monte Carlo Tests of the Hypotheses 192

17.8 The Other Propagation of Error 193

17.9 Chapter Highlights 197

References 197

18 Heteroscedastic Noises 199

18.1 Introduction 199

18.2 The Two Simplest Heteroscedastic NPMs 199

18.2.1 Linear NPM 201

18.2.2 Experimental Corroboration of the Linear NPM 202

18.2.3 Hazards with Heteroscedastic NPMs 203

18.2.4 Example: A CMS with Linear NPM 204

18.3 Hazards with ad hoc Procedures 206

18.4 The HS (“Hockey Stick”) NPM 207

18.5 Closed-Form Solutions for Four Heteroscedastic NPMs 209

18.6 Shot Noise (Gaussian Approximation) NPM 210

18.7 Root Quadratic NPM 211

18.8 Example: Marlap Example 20.13, Corrected 211

18.9 Quadratic NPM 211

18.10 A Few Important Points 212

18.11 Chapter Highlights 212

References 213

19 Limits of Quantitation 215

19.1 Introduction 215

19.2 Theory 217

19.3 Computer Simulation 219

19.4 Experiment 221

19.5 Discussion and Conclusion 223

Acknowledgments 224

References 224

19.6 Postscript 225

19.7 Chapter Highlights 226

20 The Sampled Step Function 227

20.1 Introduction 227

20.2 A Noisy Step Function Temporal Response 229

20.3 Signal Processing Preliminaries 230

20.4 Processing the Sampled Step Function Response 231

20.5 The Standard t-Test for Two Sample Means When the Variance is Constant 232

20.6 Response Domain Decision Level and Detection Limit 233

20.7 Hypothesis Testing 233

20.8 Is There any Advantage to Increasing Nanalyte? 233

20.9 NET Response Domain Decision Level and Detection Limit 235

20.10 NET Response Domain SNRs 235

20.11 Content Domain Decision Level and Detection Limit 235

20.12 The RSDB–BEC Method 236

20.13 Conclusion 237

20.14 Chapter Highlights 237

References 237

21 The Sampled Rectangular Pulse 239

21.1 Introduction 239

21.2 The Sampled Rectangular Pulse Response 239

21.3 Integrating the Sampled Rectangular Pulse Response 240

21.4 Relationship Between Digital Integration and Averaging 242

21.5 What is the Signal in the Sampled Rectangular Pulse? 243

21.6 What is the Noise in the Sampled Rectangular Pulse? 243

21.7 The Noise Bandwidth 244

21.8 The SNR with Matched Filter Detection of the Rectangular Pulse 245

21.9 The Decision Level and Detection Limit 245

21.10 A Square Wave at the Detection Limit 246

21.11 Effect of Sampling Frequency 247

21.12 Effect of Area Fraction Integrated 247

21.13 An Alternative Limit of Detection Possibility 248

21.14 Pulse-to-Pulse Fluctuations 248

21.15 Conclusion 249

21.16 Chapter Highlights 250

References 250

22 The Sampled Triangular Pulse 251

22.1 Introduction 251

22.2 A Simple Triangular Pulse Shape 251

22.3 Processing the Sampled Triangular Pulse Response 253

22.4 The Decision Level and Detection Limit 254

22.5 Detection Limit for a Simulated Chromatographic Peak 254

22.6 What Should Not be Done? 256

22.7 A Bad Play, in Three Acts 256

22.8 Pulse-to-Pulse Fluctuations 258

22.9 Conclusion 258

22.10 Chapter Highlights 259

References 259

23 The Sampled Gaussian Pulse 261

23.1 Introduction 261

23.2 Processing the Sampled Gaussian Pulse Response 262

23.3 The Decision Level and Detection Limit 263

23.4 Pulse-to-Pulse Fluctuations 263

23.5 Conclusion 264

23.6 Chapter Highlights 264

References 264

24 Parting Considerations 267

24.1 Introduction 267

24.2 The Measurand Dichotomy Distraction 269

24.3 A “New Definition of LOD” Distraction 273

24.4 Potentially Important Research Prospects 274

24.4.1 Extension to Method Detection Limits 274

24.4.2 Confidence Intervals in the Content Domain 275

24.4.3 Noises Other Than AGWN 275

24.5 Summary 276

References 277

Appendix A Statistical Bare Necessities 279

Appendix B An Extremely Short Lightstone® Simulation Tutorial 299

Appendix C Blank Subtraction and the 𝜂1∕2 Factor 311

Appendix D Probability Density Functions for Detection Limits 321

Appendix E The Hubaux and Vos Method 325

Bibliography 331

Glossary of Organization and Agency Acronyms 335

Index 337