Exploratory Data Analysis Using Fisher Information (eBook)

Professor Emeritus - B. Roy Frieden: B.S., M.S., Ph.D., Professor of Optical Sciences, Univ. of Arizona 1966-2002; books "Probability, Statistical Optics and Data Testing" (Springer-Verlag, 3rd ed., 2001), "Physics from Fisher Information" (Cambridge Univ. Press, 1998), "Science from Fisher Information" (Cambridge Univ. Press, 2004); inventor of principle of Extreme physical information (EPI) Dr. Robert A. Gatenby: BSE, Princeton Univ. MD, University of Pennsylvania.

Table of Contents 
5 
Contributor Details 7
1 Introduction to Fisher Information: Its Origin, Uses, and Predictions 
14 
1.1. Mathematical Tools 16
1.1.1. Partial Derivatives 16
1.1.2. Euler-Lagrange Equations 17
1.1.2.1. Extremum Problem 17
1.1.2.2. Solution 17
1.1.2.3. Nature of Extremum 18
1.1.2.4. Building in Constraints 18
1.1.2 .5. Example: MFI Lagrangian 19
1.1.3. Dirac Delta Function 19
1.1.4. Unitary Transformation 20
1.2. A Tutorial on Fisher Information 21
1.2.1. Definition and Basic Properties 21
1.2.1.1. Comparisonwith ShannonInformation 21
1.2.1.2. System Input-Output Law 22
1.2.1.3. UnbiasedEstimators 22
1.2.1.4. Use of Schwarz Inequality 23
1.2.1.5. Cramer-Rao Inequality 24
1.2.1.6. Fisher Coordinates 24
1.2.1.7. Efficient Estimation 25
1.2.1.8. Examples of Tests for Efficiency 25
1.2.2. Alternative Forms of Fisher Information I 26
1.2.2.1. Multiple Parameters and Data 26
1.2.2.2. Shift-InvariantCases 27
1.2.2.3. One-Dimensional Applications 28
1.2.2.4. No Shift-Invariance, Discrete Data 28
1.2.2.5. Amplitude q-Forms of I 29
1.2.2.6. Tensor Form of I 29
1.2.2.7. One-Dimensional q-Form of I 30
1.2.2.8. Complex Amplitude 1/J-Form of I 30
1.2.2.9. Principal Value Integrals 31
1.2.2.10. I in Curved Space 
31 
1.2.2.11. I for Gluon Four-Position Measurement 
32 
1.2.2.12. Local vs Global Measures of Disorder 34
1.3. Introduction to Source Fisher Information J 34
1.3.1. EPI Zero Principle 34
1.3.2. Efficiency Constant K 35
1.3.3. Fisher I -Theorem 35
1.3.3.1. I as a Montonic Measure of Time 36
1.3.3.2. Re-Expressing the Second Law 36
1.4. Extreme Physical Information (EPI) 36
1.4.1. Relation to Anthropic Principle 37
1.4.2. Varieties of EPI Solution 
38 
1.4.3. Data Information is Generic 38
1.4.4. Underpinnings: A "Participatory" Universe 39
1.4.5. A "Cooperative" Universe and Its Implications 40
1.4.6. EPI as an Autopoietic Process 44
1.4.7. Drawbacks of Classical "Action Principle" 44
1.5. Getting the Source Information J for a Given Scenario 45
1.5.1. Exact, Unitary Scenarios: Type (A) Abduction 45
1.5.2. Exhaustivity Property ofEPI 45
1.5.3. Inexact, Classical Scenarios : Type (B) Deduction 46
1.5.4. Empirical Scenarios: Type (C) Induction 47
1.5.5. Summary 47
1.6. Information Game 47
1.6.1. Minimax Nature ofSolution 48
1.6.2. Saddlepoint Property 48
1.6.3. Game Aspect ofEPI Solution 49
1.6.4. Information Demon 49
1.6.5. Peirce Graphs 49
1.6.6. Game Corollary 50
1.6.7. Science Education 51
1.7. Predictions of the EPI Approach 51
2 Financial Economics from Fisher Information 
55 
2.1. Constructing Probability Density Functions on Price Fluctuation 
55 
2.1.1. Summary 55
2.1.2. Background 56
2.1.3. Variational Approaches to the Determination of PriceValuation Fluctuation 
56 
2.1.4. Trade as a Measurement in EPI Process 57
2.1.5. Intrinsic vs Actual Data Values 
58 
2.1.6. Incorporating Data Values into EPI 59
2.1.7. Information J and the "Technical" Approach to Valuation 
60 
2.1.8. Net EPI Principle 
61 
2.1.9. SWE Solutions 61
2.2. Yield Curve Statics and Dynamics 63
2.2.1. Summary 63
2.2.2. Background 63
2.2.3. PDFin the Term Structure ofInterest Rates, and Yield Curve Construction 
64 
2.2.4. PDF for a Perpetual Annuity 
68 
2.2.5. Yield Curve Dynamics 70
2.2.6. Relation to Nelson Siegel Approach and Dynamical Fokker-Planck Solution 
73 
2.2.7. A Measure of Volatility 74
2.2.8. Equilibrium Distributions 76
2.2.9. Aoki Theory 76
2.2.10. Non-equilibrium Distributions 77
2.3. Information and Investment 77
2.3.1. Summary 77
2.3.2. Background 78
2.3.3. Information I 79
2.3.4. Information J 80
2.3.5. Phase Space 80
2.3.6. Optimized Information and q-Theory 82
2.3.7. Other Optimized Strategies 83
2.3.8. Investment Parameters 84
2.3.9. Uncertainty Principle on Capital and Investment Flow 
85 
2.3.10. Market Efficiency 85
3 Growth Characteristics of Organisms 
87 
3.1. Information in Living Systems: A Survey 87
3.1.1. Summary 87
3.1.2. Introduction 88
3.1.3. Ideal Requirements of Biological Information 
89 
3.1.4. Some Alternative Information Types 91
3.1.5. Kullback-Leibler Information 91
3.1.6. Principle ofExtreme K-L Information 92
3.1.7. Bias Property 93
3.1.8. A Transition to the EPI Principle 93
3.1.9. Biological Interplay ofSystem and Reference Probabilities 
93 
3.1.10. Application ofK-L Principle to Developmental Biology 
94 
3.1.11. Shannon Information Types 94
3.1.12. Information as an Expenditure of Energy 
96 
3.1.13. Some Problems with Biological Uses of Shannon Information 
96 
3.1.14. Intracellular Information Dynamics 97
3.1.14.1. Bioinformatics and Network Analysis 97
3.1.14.2. Hub Dynamics 99
3.1.14.3. Dynamics of System Failures 100
3.1.15. Information and Cellular Fitness 100
3.1.16. Cellular Information Utilization 101
3.1.17. Potential Controversies-Is All Cellular Information Stored in the Genome and Transmitted by Proteins? 
102 
3.1.18. Multicellular Information Dynamics 104
3.1.19. Information and Disease 105
3.1.20. Conclusions 107
3.2. Applications of IT and EPI to Carcinogenesis 107
3.2.1. Summary 107
3.2.2. Introduction 108
3.2.3. Bound and Free Intracellular Information 109
3.2.4. Information Dynamics Before and After Reproduction,With and Without Mutation 110
3.2.5. Limits ofInformation Degradation in Carcinogenesis 112
3.2.5.1. Mutation Rates for Various Gene Proliferation Types 113
3.2.5 .2. Angiogenesis 113
3.2.6. The Evolving Microenvironment and Resulting Mutation Rate 
114 
3.2.7. Application of EPI to Tumor Growth 
115 
3.2.7.1. Fisher vs Shannon Informations 116
3.2.7.2. Brief Review of EPI 116
3.2.8. Fisher Variable, Measurements 117
3.2.9. Implementation of EPI 
118 
3.2.9.1. Recourse to Probability Amplitude 118
3.2.9.2. Self-Consistency Approach, General Considerations 118
3.2.10. EPI Solution: A Power Law 119
3.2.11. Determining the Power by Minimizing the Information 
119 
3.2.12. Experimental Verification 120
3.2.13. Implications of Solution 
121 
3.2.13.1. Efficiency K 121
3.2.13.2. Fibonacci Constant 122
3.2.13.3 . Uncertainty in Onset Time 123
3.2.14. Alternative Growth Model Using Monte Carlo Techniques 
124 
3.2.15. Conclusions 125
3.3. Appendix A: Derivation of EPI Power Law Solution 128
3.3.1. General Solution 128
3.3.2. Boundary-Value Conditions 130
3.3.3. Error in the Estimated Duration of the Cancer 130
4 Information and Thermal Physics 
132 
4.1. Connections Between Thermal and Information Physics 133
4.1.1. Summary 133
4.1.2. Introduction 133
4.1.3. A Scientific Theory's Structure 134
4.1.4. Fisher-Related Activity in Contemporary Physics 135
4.1.5. BriefPrimer on Fisher's Information Measure 136
4.2. Hamiltonian Systems 136
4.2.1. Summary 136
4.2.2. Classical Statistical Mechanics a la Fisher 
137 
4.2.3. Canonical Example 140
4.2.4. General Quadratic Hamiltonians 140
4.2.5. Free Particle 141
4.2.6. N Harmonic Oscillators 142
4.2.7. A Nonlinear Problem: Paramagnetic System 142
4.2.8. Conclusions 143
4.3. The Place of A Fisher Thermal Physics 144
4.3.1. Summary 144
4.3.2. Preliminaries 144
4.3.3. The Standard Macroscopic Theory 144
4.3.3.1. Macroscopic Thermodynamics 
144 
4.3 .3.2. Legendre Structure 145
4.3.4. Statistical Mechanics 146
4.3.5. Axioms ofInformation Theory 146
4.4. Modem Approaches to Statistical Mechanics 147
4.4.1. Summary 147
4.4.2. Jaynes' Reformulation 147
4.4.3. Legendre Structure in Jaynes' Formulation 148
4.4.4. Non-Shannon Information Measures 149
4.4.5. Legendre Structure Preserved by a ChangeofMeasure Is --> IT
149 
4.4.6. Still More General Measures 150
4.5. Fisher Thermodynamics 151
4.5.1. Summary 151
4.5.2. FIM Concavity and Second Law 152
4.5.3. Minimizing FIM Leads to a Schrodinger-Like Equation 152
4.5.4. FIM Legendre Transform Structure 154
4.5.5. Shannon's S vs Fisher's I 154
4.5.6. Discussion 155
4.6. The Grad Approach in Ten Steps 157
4.6.1. Summary 157
4.6.2. Steps ofthe Approach 157
4.7. Connecting Excited Solutions of the Fisher-SWE to Nonequilibrium Thermodynamics 
158 
4.7.1. Summary 158
4.7.2. Establishing the Connection 158
4.7.3. Application: Viscosity 159
4.7.3.1. Boltzmann Equation in the Relaxation Approximation 159
4.7.3.2. Generalities on Viscosity 160
4.7.4. Comparison with the Grad Treatment 164
4.7.5. The Fisher Treatment of Viscosity 165
4.7.5.1. Ground State 165
4.7.5 .2. Admixture of Excited State s 166
5 Parallel Information Phenomena of Biology and Astrophysics 
168 
5.1. Corresponding Quarter-Power Laws of Cosmology and Biology 
168 
5.1.1. Summary 168
5.1.2. Introduction 169
5.1.3. Cosmological Attributes 171
5.1.3.1. Unitless Physical Constants 171
5.1.3.2. Dirac Hypothesis for the Cosmological Constants 172
5.1.4. Objectives 173
5.1.5. Biological Attributes 173
5.1.6. Corresponding Attributes ofBiology and Cosmology 176
5.1.7. Discussion 180
5.1.8. Concluding Remarks 184
5.2. Quantum Basis for Systems Exhibiting Population Growth 
185 
5.2.1. Summary 185
5.2.2. Introduction 185
5.2.3. Hartree Approximation 186
5.2.4. Schrodinger Equation 186
5.2.5. Force-Free Medium 187
5.2.6. Case ofa Complex Potential 188
5.2.7. SWE Without Planck Constant 189
5.2.8. Growth Equation 189
5.2.9. How Could Such a System Be Realized? 190
5.2.10. Current Evidence for Nanolife 191
6 Encryption of Covert Information Through a Fisher Game 
194 
6.1. Summary 194
6.1.1. Fisher Information and Extreme Physical Information 194
6.1.2. Fisher Game 195
6.1.3. Extreme Physical Information vs Minimum Fisher Information 
196 
6.1.4. Time-Independent Schrodinger Equation 
196 
6.1.5. Real Probability Amplitudes and Lagrangians 197
6.1.6. MFI Output and the Time-Independent Schrodinger-Like Equation 
198 
6.1.7. Quantum Mechanical Methods in Pattern Recognition Problems 
198 
6.1.8. Solutions of the TISE and TISLE 
198 
6.1.9. Information Theory and Securing Covert Information 199
6.1.10. Rationale for Encrypting Covert Information in a Statistical Distribution 
201 
6.1.11. Reconstruction Problem 202
6.1.12. Ill-Conditioned Nature of Encryption Problem 
202 
6.1.13. Use ofthe Game Corollary 203
6.1.14. Objectives to be Accomplished 203
6.2. Embedding Covert Information in a Statistical Distribution 
204 
6.2.1. Operators and the Dirae Notation 204
6.2.2. Eigenstructure of the Constraint Operator 
205 
6.2.3. The Encryption and Decryption Operators 206
6.2.4. Generic Strategy for Encryption and Decryption 206
6.3. Inference of the Host Distribution Using Fisher Information 
207 
6.3.1. Correspondence Between MFI and MaxEnt 208
6.3.2. Amplitudes and Pseudo-Potentials Satisfying MFI and MaxEnt 
208 
6.3.3. Modified Game Corollary 209
6.3.4. Fisher Game vs MaxEnt 212
6.4. Implementation of Encryption and Decryption Procedures 
216 
6.4.1. Encryption Process 219
6.4.2. Decryption 223
6.4.3. Summary of the Encryption-Decryption Strategy 224
6.4.4. Security Against Malicious Attacks 225
6.5. Extension of the Encryption Strategy 227
6.6. Summary of Concepts 228
7 Applications of Fisher Information to the Management of Sustainable Environmental Systems 
230 
7.1. Summary 230
7.2. Introduction 231
7.3. Fisher Information Theory 232
7.3.1. Definition 232
7.3.2. Shift-Invariant Cases 233
7.3.3. Phase States s of Dynamic Systems 
234 
7.4. Probability Law on State Variable s 235
7.5. Evaluating the Information 237
7.6. Dynamic Order 238
7.7. Dynamic Regimes and Fisher Information 239
7.8. Evaluation of Fisher Information 240
7.9. Applications to Model Systems 242
7.9.1. Two-Species Model System 242
7.9.2. Multispecies Model: Species and Trophic Levels 246
7.9.3. Ecosystems with Pseudo-Economies: Agriculture and Industry 
248 
7.10. Applications to Real Systems 250
7.10.1. North Pacific Ocean 251
7.10.2. Global Climate 253
7.10.3. Sociopolitical Data 254
7.11. Summary 256
8 Fisher Information in Ecological Systems 
258 
8.1. Predicting Susceptibility to Population Cataclysm 258
8.1.1. Summary 258
8.1.2. Introduction 259
8.1.3. A Biological Uncertainty Principle 259
8.1.4. Ramification of the Uncertainty Principle 
260 
8.1.5. Necessary Condition for Cataclysm 261
8.1.6. Scenarios of Cataclysm from Fossil Record 262
8.1.7. Usefully Conservative Scenario 263
8.1.8. Resulting Changes in Population Occurrence Rates 263
8.1.9. Getting the Information 264
8.1.10. Square Root Decision Rule 265
8.1.11. Final Decision Rule, Conditions of Use 265
8.1.12. Ideally Breeding Rabbits 267
8.1.13. Homo Sapiens 268
8.2. Finding the Mass Occurrence Law For Living Creatures 269
8.2.1. Summary 269
8.2.2. Background 269
8.2.3. Cramer-Rao Inequality and Efficient Estimation 271
8.2.4. Efficiency Condition 272
8.2.5. Objectives 272
8.2.6. How Can the Efficiency Condition be Satisfied? 273
8.2.7. Power-Law Solution 274
8.2.8. Normalized Law 275
8.2.9. Unbiasedness Condition 275
8.2.10. Asymptotic Power b = 1+E, with E Small 
276 
8.2.11. Discussion 278
8.2.12. Experimental Evidence for a l/x Law 279
8.3. Derivation of Power Laws of Nonliving and Living Systems 
280 
8.3.1. Summary 280
8.3.1.1. General Allometric Laws 281
8.3.1.2. Biological Allometric Laws 281
8.3.1.3. On Models for Biological Allometry 282
8.3.2. PriorKnowledge Assumed 283
8.3.3. Measurement Channel for Problem 284
8.3.3.1. Measurement, System Function 284
8.3.3.2. Some Caveats to EPI Derivation 285
8.3.4. Data Information I 285
8.3.5. Source Information lea) 286
8.3.5.1. Microlevel Contributions 286
8.3.5.2. Fourier Analysis 286
8.3.6. Net EPI Problem 
288 
8.3.7. Synopsis of the Approach 
288 
8.3.8. Primary Variation ofthe System Function Leads to a Family of Power Laws 
289 
8.3.9. Variation of the Attribute Parameters Gives Powers a - an = n/4 
290 
8.3.10. Secondary Extremization Through Choice of h(x) 
291 
8.3.10.1. Special Form of Function h(x) 292
8.3.10.2. Resulting variational principle in Base Function h(x) 292
8.3.10.3. Secondary Variational Principle in Associated Function k(x ) 293
8.3.10.4 . Result k(x ) = 0, Giving Base Function hex ) Proportional to x 293
8.3.11. Final Allometric Laws 294
8.3.12. Alternative Model Su = L 295
8.3.13. Discussion 295
9 Sociohistory: An Information Theory of Social Change 
298 
9.1. Summary 298
9.1.1. Philosophical Background 299
9.1.2. Boundary Considerations 300
9.1.3. Sociohistory: Historical Aspects 301
9.1.4. Kant's Notion of the Noumenon 302
9.1.5. Extreme Phenomenal Information 302
9.1.6. Complex Systems and Chaos 303
9.1.7. fin-Yang Nature of Sociohistory 
303 
9.1.8. Dialectic Process 304
9.1.9. Hegelian Doctrine of the Dialectic 
304 
9.1.10. Principle ofImmanent Change 305
9.2. Social Cybernetics 305
9.2.1. SVS Theory 306
9.2.2. Collective Mind 308
9.2.3. Global Noumenon 308
9.2.4. Relative Noumenon 309
9.3. Developing a Formal Theory of Sociohistory 309
9.3.1. The Ontological Basis for SVS Theory 309
9.3.2. Extreme Phenomenal Information 314
9.3.3. System Informations I, J 314
9.3.4. Information Channel 315
9.3.5. Information I 316
9.3.6. Fisher I as a Measure of the Arrow of Time 
317 
9.3.7. EPI Zero Condition 317
9.3.8. I is General, J is Specific 318
9.3.9. EPI Extremum Principle 318
9.3.10. Knowledge Game 319
9.4. Sociocultural Dynamics 320
9.4.1. Cultural Driving Forces 320
9.4.2. Sensate and Ideational Aspects 321
9.5. The Paradigm of Sociohistory 322
9.5.1. The Propositional Base Through SVS 322
9.5.2. Dispersed Agents 322
9.5.3. Ideational vs Sensate Dispersed Agents 323
9.5.4. The Dynamics of Viable Holons 324
9.5.5. Emergent States and EPI 327
9.5.6. Sensate and Ideational Aspects ofthe Informations 329
9.5.7. Role of Efficiency Constant K 
330 
9.5.8. Coefficients ofInformation 331
9.5.9. Role ofK in Defining States ofSociety 332
9.5.10. Emerging Balance Between Cultural Dispositions 333
9.6. Exploring the Dynamic of Cultural Disposition 334
9.6.1. Time Evolution of the Informations 
334 
9.6.2. Consequences ofEnantiomer Imbalance 336
9.6.3. A Quantification Using Information Parameter K 336
9.7. The Sociocultural Propositions of EPI 338
9.8. An Illustrative Application of EPI to Sociocultural Dynamics 
338 
9.8.1. General Problem of Population Growth and Motion 
339 
9.8.2. Population Growth and Depletion Coefficients 340
9.8.3. EPI Solution 341
9.9. A Case Illustration: Postcolonial Iran 343
9.9.1. Sensate vs Ideational Mindsets 343
9.9.2. Cultural Instability 344
9.9.3. Quantitative Growth Effects 345
9.9.4. Manifestations in Political Power and Dominance 346
9.10. Overview 347
References 349
Index 369

"7 Applications of Fisher Information to the Management of Sustainable Environmental Systems (p. 217-218)

AUDREYL. MAYER, CHRISTOPHER W. PAWLOWSKI, BRIAN D. FATH, ANDHERIBERTOCABEZAS

All organisms alter their surroundings, and humans now have the ability to affect environments at increasingly larger temporal and spatial scales. Indeed , mechanical and engineering advances of the twentieth century greatly enhanced the scale of human activities. Among these are the use and redistribution of natural resources . Unfortunately, these activities can have unexpected and unintended consequences. Environmental systems often respond to these activities with diminished or lost capacity of natural function. Fortunately, environmental management can play an important role in ameliorating these negative effects.

The aim is to promote sustainable development, i.e., enrichment of the lives of the majority of people without seriously degrading the diversity and richness of the environment. However, the management tools themselves often fall prey to the same narrow levels of perspective that generated the negative conditions. The challenge is to develop a system-level index, one that indicates the organizati on and direction of ecological system dynamics. This index could detect when the system is changing its configuration to a new, perhaps less desirable, dynamic regime and may be incorporated into a sustainable management plan for the system. In this chapter, we demonstrate the use of Fisher information (FI) as such an environmental system index.

7.1. Summary

We derive an expression for FI based on sampling of the system trajectory as it evolves in the phase space defined by the state variables of the system. This FI index is derived as a measure ofsystem dynami c order; as defined by its speed and acceleration along periodic steady-state trajectories. We illustrate the concepts on data collected from both computer model simulations and real-world environmental systems. PI is found to provide a valuable tool to identify impending and in-progress shifts in regime, as distinguished from normal cycles, fluctuations , and noise in the systems.

7.2. Introduction

"Sustainability" is often used in a qualitative sense. However, there is at present a great need to quantitatively measure (and monitor) its many qualitative aspects in real systems. Real systems are regarded as sustainable if they can maintain their current, desirable productivity and character without creating unfavorable conditions elsewhere or in the future [1-4]. Sustainability therefore incorporates both concern for the future of the current system (temporal sustainability) and concern about the degree to which some areas and cultures of the planet are improved at the expense of other areas and cultures (spatial sustainability). That is, sustainability is to hold over both space and time."