Computers and Creativity (eBook)

Jon McCormack is an electronic media artist and researcher in computing. He holds a degree in Applied Mathematics and Computer Science from Monash University, a Graduate Diploma of Art (Film and Television) from Swinburne University and a PhD in Computer Science from Monash University, Melbourne. He is currently Associate Professor in Computer Science, an ARC Australian Research Fellow and director of the Centre for Electronic Media Art (CEMA) at Monash University. His research spans generative art and design, evolutionary systems, creativity, visualisation, interaction, machine learning, L-systems and developmental models. His artworks have been widely exhibited at leading galleries, museums and symposia, including the Museum of Modern Art (New York, USA), Tate Gallery (Liverpool, UK), ACM SIGGRAPH (USA), Prix Ars Electronica (Austria) and the Australian Centre for the Moving Image (Australia). He is the recipient of 16 awards for new media art and research including prizes at Ars Electronica (Austria), Images du Futur (Canada), New Voices, New Visions (USA), Alias/Wavefront (USA), The John Lansdown Award for Interactive Media (Europe/UK), and Nagoya Biennial (Japan). Mark d’Inverno holds an MA and MSc in Mathematics from Oxford University and a PhD from University College London in Artificial Intelligence. He is Professor of Computer Science at Goldsmiths, University of London and for four years between 2007 and 2011 was head of the Department of Computing which has championed interdisciplinary research and teaching around computers and creativity for nearly a decade. He has published over 100 articles including books, journal and conference articles and led research projects in a diverse range of fields relating to computer science including multi-agent systems, systems biology, art, music and social media. During the final editing of this book he took a research sabbatical shared between the Artificial Intelligence Research Institute and the Sony Computer Science Laboratory in Paris. He is currently the principal investigator or co-investigator  on a range of EU and UK projects spanning community building, sharing social experiences, inspiring creativity, cultural engagement and music education. He is a critically acclaimed jazz pianist and composer and over the last 25 years has led a variety of successful bands in a range of different musical genres.

Computers and Creativity 3
Foreword 5
Preface 7
Why Does Computing Matter to Creativity? 7
Summary of Contributions 9
Acknowledgements 14
Contents 15
Contributors 17
Part I: Art 23
Chapter 1: The Painting Fool: Stories from Building an Automated Painter 24
1.1 Introduction 24
1.2 The Painting Fool in Context 27
1.3 Guiding Principles 32
1.3.1 Ever-Decreasing Circles 32
1.3.2 Paradigms Lost 32
1.3.3 The Whole Is More Than a Sum of the Parts 33
1.3.4 Climbing the Meta-mountain 33
1.3.5 The Creativity Tripod 34
1.3.6 Beauty Is in the Mind of the Beholder 35
1.3.7 Good Art Changes Your Mind 36
1.4 Illustrative Projects 37
1.4.1 Non-photorealistic Rendering 38
1.4.2 Emotional Modelling 41
1.4.3 Scene Construction 45
1.4.4 Collage Generation 50
1.4.5 Paint Dances 52
1.5 Future Directions 53
1.6 Conclusions 56
References 57
Chapter 2: Creative Ecosystems 60
2.1 Creative Systems 60
2.1.1 Spaces of Possibility 62
2.2 Evolutionary Computing and Creativity 64
2.3 Ecosystems 66
2.3.1 Biological Ecosystems 67
2.3.2 Ecosystem Models in the Creative Arts 68
Design and Architecture. 68
Music and Performance. 69
Visual and Installation Art. 69
2.4 Ecosystem Design Patterns 72
2.4.1 Environments: Conditions and Resources 72
2.4.2 Self-observation and Feedback 73
2.4.3 Automation and the Creative Role of the Artist 76
2.5 Conclusions 78
References 79
Chapter 3: Construction and Intuition: Creativity in Early Computer Art 82
3.1 Introduction 82
3.2 The First Narration: On Random Polygons 85
3.2.1 Georg Nees 90
3.2.2 A. Michael Noll 92
3.2.3 Frieder Nake 92
3.3 The Second Narration: On Three Artists 95
3.3.1 Vera Molnar 97
3.3.2 Charles Csuri 98
3.3.3 Manfred Mohr 100
3.4 The Third Narration: On Two Programs 102
3.4.1 Harold Cohen: AARON 104
3.4.2 Frieder Nake: Generative Aesthetics I 107
3.5 The Fourth and Last Narration: On Creativity 111
3.6 Conclusion 112
References 114
Chapter 4: Evaluation of Creative Aesthetics 116
4.1 Introduction 117
4.2 Background: Evaluation of Artistic Artefacts 117
4.3 A Conversation on Evaluation 119
4.4 Conclusion 132
References 132
Part II: Music 133
Chapter 5: Musical Virtuosity and Creativity 134
5.1 Virtuosos as Exceptional Humans 134
5.1.1 Virtuosity in Art 134
5.1.2 The Cognitive Science Perspective on Virtuosity 136
5.1.3 Virtuosity as an Attraction Device 136
5.1.4 Virtuosos as Creators 137
5.2 The Case of Jazz 138
5.2.1 The Rules of the Game 139
5.2.2 Bebop Phrases 139
5.2.3 The Melodic/Harmonic Interplay 140
5.2.3.1 Harmonic Consistency 140
5.2.3.2 Continuity 141
The One-Step-Max Theorem 141
5.2.4 Playing Outside and Side-Slipping 141
5.2.5 Virtuosity Is to Improvisation as Running Is to Walking 143
5.2.6 Claims 144
5.3 Modelling Jazz Improvisation Generation 145
5.3.1 Non-Markovian Approaches 145
5.3.2 Markov Chain Approaches 146
5.4 A Note-Based Jazz Generator 148
5.4.1 Pitches for Representation, Beats for Generation 148
5.4.2 Handling Harmony 149
5.4.3 Chord Change Negotiation 151
5.4.4 An Example Training Set 152
5.5 Escaping Markovian Boredom 155
5.5.1 Side-Slips and Formal Transforms 155
5.5.2 The Control Issue 157
5.5.3 Reusing Intentional Scores 159
5.6 Virtuoso: A Virtuoso Enabling Interactive System 161
5.7 Discussion 162
References 163
Chapter 6: Live Algorithms: Towards Autonomous Computer Improvisers 166
6.1 Introduction 166
6.2 The Field: Creative Group Improvisation 168
6.2.1 Collective Improvisation 168
6.2.2 The Individual (Human or Machine) in Interaction 168
6.2.2.1 Autonomy 168
6.2.2.2 Novelty 169
6.2.2.3 Participation 169
6.2.2.4 Leadership 170
6.2.3 Relationship of the Four Attributes to Creativity 170
6.3 Theoretical Considerations 171
6.3.1 P, Q and f 171
6.3.2 De?nition of a Live Algorithm 171
6.3.3 Architecture 171
6.3.4 The Live Algorithm from the Outside 175
6.3.5 Arti?cial Intelligence 178
6.4 Live Algorithms in Context 180
6.4.1 Live Algorithm Behaviour 180
6.4.1.1 Shadowing 180
6.4.1.2 Mirroring 181
6.4.1.3 Coupling 181
6.4.1.4 Negotiation 182
6.4.2 Agency and Live Algorithms 183
6.4.3 Live Algorithms as Musicians 185
6.5 Prototypes 186
6.6 Further Considerations 189
6.6.1 Embodiment 189
6.6.2 Learning 190
6.6.3 Anticipated Criticisms 190
6.6.4 Cultural Embeddedness 191
6.6.5 A Final Note 191
References 192
Chapter 7: The Extended Composer 194
7.1 Introduction 194
7.1.1 Thinking Through Tools 195
7.1.2 The Computer as Meta-tool 197
7.1.3 Digital Partners in Creative Practice 197
7.2 Computational Aides for Algorithmic Inspiration 199
7.2.1 Computational Strategies and Algorithmic Aides 200
7.3 The Human-Computer Partnership: Characteristics and Categories 201
7.3.1 Feedback 202
7.3.2 Exploration 204
7.3.3 Intimacy 207
7.3.4 Interactivity 208
7.3.5 Introspection 210
7.3.6 Time 212
7.3.7 Authorship 214
The "Inhuman" Argument 214
The "Invisible Hand" Argument 215
The "Creative Vitalism" Argument 215
7.3.8 Value 216
7.4 In Summary 217
7.4.1 Future Explorations 218
7.4.2 Final Re?ections 219
References 219
Chapter 8: Between Material and Ideas: A Process-Based Spatial Model of Artistic Creativity 223
8.1 Introduction 223
8.1.1 Background 224
8.1.2 Outline 226
8.2 Tools 226
8.3 The Model 228
8.3.1 Material Space and Representation 229
8.3.2 The Conceptual Representation 233
8.3.3 Interplay Between Representations 234
8.3.4 Example Scenarios 235
8.3.5 Appreciation and Novelty 237
8.3.6 The Model in Context 238
8.3.7 Craft and Skill 240
8.3.8 New Tools and Tool Design 241
8.3.9 Social and Cultural Creativity 242
8.3.10 Abstraction Levels 242
8.4 Implications for Computational Creativity 243
8.4.1 Implementation of the Model 244
8.4.2 Conceptual Representations 245
8.4.3 Re-conceptualisation 246
8.4.4 Memory and Learning 247
8.5 Final Remarks 248
References 249
Chapter 9: Computer Programming in the Creative Arts 252
9.1 Introduction 252
9.2 Creative Processes 255
9.2.1 Creative Process of Bricolage 256
9.3 Anthropomorphism and Metaphor in Programming 257
9.4 Symbols and Space 259
9.5 Components of Creativity 263
9.6 Programming in Time 265
9.6.1 Interactive Programming 266
9.7 Conclusion 267
References 268
Part III: Theory 270
Chapter 10: Computational Aesthetic Evaluation: Past and Future 271
10.1 Introduction 271
10.1.1 What Do We Mean by Computational Aesthetic Evaluation? 272
10.1.2 Why Is Computational Aesthetic Evaluation so Dif?cult if not Impossible? 273
10.2 A Brief History of Computational Aesthetic Evaluation 273
10.2.1 Formulaic and Geometric Theories 274
10.2.2 Design Principles 276
10.2.3 Arti?cial Neural Networks and Connectionist Models 277
10.2.4 Evolutionary Systems 278
10.2.5 Interactive Evolutionary Computation 279
10.2.6 Automated Fitness Functions Based on Performance Goals 280
10.2.7 Evolutionary Fitness Measured as Error Relative to Exemplars 282
10.2.8 Automated Fitness Functions Based on Complexity Measures 283
10.2.9 Automated Fitness Functions in Evolutionary Music Systems 283
10.2.10 Multi-objective Aesthetic Fitness Functions in Evolutionary Systems 284
10.2.11 Biologically Inspired Extensions to Simple Evolutionary Computation 285
10.2.11.1 Coevolution 286
10.2.11.2 Niche Construction by Agents 287
10.2.11.3 Agent Swarm Behaviour 288
10.2.11.4 Curious Agents 289
10.2.11.5 Human Aesthetics, Meta-aesthetics, and Alternatives to Fitness Functions 290
10.2.12 Complexity Based Models of Aesthetics 290
10.3 The Future of Computational Aesthetic Evaluation 293
10.3.1 The Origins of Art and the Art Instinct 293
10.3.2 Psychological Models of Human Aesthetics 295
10.3.2.1 Arnheim-Gestalt and Aesthetics 295
10.3.2.2 Berlyne-Arousal Potential and Preferences 295
10.3.2.3 Martindale-Prototypicality and Neural Networks 297
10.3.3 Empirical Studies of Human Aesthetics 299
10.3.4 Neuroaesthetics 300
10.3.5 Computing Inspired by Neurology 300
10.3.6 The Neocortex and Hierarchical Temporal Memory 301
10.3.7 Computer Architectures for Evolvable Hardware 302
10.4 Conclusion 302
References 303
Chapter 11: Computing Aesthetics with Image Judgement Systems 310
11.1 Introduction 310
11.2 Validation Approaches for AJS 314
11.2.1 Psychological Tests 314
11.2.2 User Evaluation and Popularity Prediction 317
11.2.3 Style and Author Classi?cation 320
11.3 The Evolution of an AJS 322
11.3.1 A Heuristic AJS 322
11.3.2 Learning AJSs 324
11.3.2.1 Feature Extraction 325
Pre-processing 325
Metrics Application 326
Feature Building 327
11.3.2.2 DJT Experiments 327
11.3.2.3 Author Identi?cation Experiments 328
11.3.2.4 Image Classi?cation Based on Online Evaluation 330
11.3.2.5 Integration in an Image Generation System 330
11.4 Conclusions 332
References 333
Chapter 12: A Formal Theory of Creativity to Model the Creation of Art 338
12.1 The Basic Idea 338
12.2 Relation to Previous, Less Formal Work 339
12.3 Formal Details 341
12.3.1 Continuous Time Formulation 344
12.4 Previous Approximative Implementations of the Theory 345
12.5 Aesthetic Reward = Change of Subjective Compressibility? 347
12.6 Low-Complexity Art as End Product of a Search Process Modelled by the Formal Theory of Creativity 348
12.7 Conclusion 349
References 349
Chapter 13: Creativity Re?ned: Bypassing the Gatekeepers of Appropriateness and Value 353
13.1 Introduction 353
13.2 What Is Creativity? 355
13.3 De?ning Creativity 357
13.3.1 Methods for Discovering Novel Representations 359
13.3.2 Objective Versus Psychological Creativity 360
13.4 Objections and Replies 361
13.4.1 The Failure of Randomness 361
13.4.2 The Verstehen Objection 362
13.4.3 The Very Possibility of Creativity 363
13.5 Consequences 363
13.5.1 The Irrelevance of Value 363
13.5.2 The Irrelevance of Appropriateness 364
13.5.3 Inferring Frameworks from Patterns 364
13.5.4 Creativity Viewed as Compression 364
13.5.5 Degrees of Creativity 365
13.6 Examples: Creativity in Human Endeavour 365
Number Theory. 365
Visual Arts. 365
13.7 Examples: Creativity in Nature 366
From Physics and Chemistry to Evolutionary Biology. 366
Ecosystems. 366
Dancing Bowerbirds, Painting Elephants and Primate Typists. 367
13.8 Realising Our De?nition of Creativity in Software 368
13.8.1 The Automatic Generation of Creative Biomorphs 368
13.8.2 Testing Creative Software Against Human Concepts of Creativity 370
13.9 Discussion 371
13.10 Conclusions 372
References 373
Chapter 14: Generative and Adaptive Creativity: A Uni?ed Approach to Creativity in Nature, Humans and Machines 375
14.1 Questions About Creativity 375
14.2 Generative and Adaptive Creativity 377
14.3 Generative and Adaptive Creativity in the Arts, in Humans, Human Groups and in Silico 379
14.3.1 The Creativity of Social Systems Is More than the Sum of Individual Creative Acts 380
14.3.2 Social Systems Can Exhibit Both Generative and Adaptive Creativity 382
14.3.2.1 The Causes and Effects of Culture 382
14.3.2.2 Social Groups as Adaptive Units 383
14.3.2.3 Social Groups as Non-adaptive Generators 385
14.3.2.4 Modelling Creativity in Social Systems 386
14.3.3 Individual Humans Can Exhibit Generative and Adaptive Creativity 387
14.4 Generative and Adaptive Approaches to Arts-Based Computational Creativity 388
14.4.1 Generative Creative Systems Can Be Externally Useful 389
14.4.2 Adaptive Creative Systems Can Be Useful to Others 390
14.5 Conclusion 392
References 393
Chapter 15: Creating New Informational Primitives in Minds and Machines 396
15.1 Introduction 396
15.2 Emergence and Creativity 399
15.2.1 What Constitutes a New Primitive? 400
15.2.2 Primitives and Interpretive Frames 401
15.2.3 Novel Combinations of Closed Sets of Primitives 402
15.2.4 Limits on Computations on Existing Primitives 403
15.2.5 Creation of New Primitives 405
15.2.6 Combinatoric and Creative Emergence in Aesthetic Contexts 407
15.3 Creativity in Self-constructing Cybernetic Percept-Action Systems 408
15.3.1 A Taxonomy of Adaptive Devices 408
15.3.2 Semiotics of Adaptive Devices 410
15.3.3 Capabilities and Limitations of Adaptive Devices 411
15.3.4 Pask's "Organic Analogues to the Growth of a Concept" 413
15.3.5 Organisational Closure and Epistemic Autonomy 415
15.4 Recognising Different Types of Creativity 416
15.4.1 Emergence-Relative-to-a-Model 416
15.4.2 Tracking Emergent Functions in a Device 416
15.5 New Signal Primitives in Neural Systems 418
15.5.1 New Primitives in Signalling Networks 419
15.5.2 Brains as Networks of Adaptive Pattern-Resonances 420
15.5.3 Regenerative Loops 421
15.5.4 Multidimensional Signals 423
15.5.5 Temporal Coding and Signal Multiplexing 423
15.5.6 Emergent Annotative Tags and Their Uses 425
References 427
Part IV: Epilogue 431
Chapter 16: Computers and Creativity: The Road Ahead 432
16.1 Where to From Here? 432
Index 436