Common Lisp Recipes (eBook)

Edmund Weitz is well-known in the Common Lisp community for his open-source libraries and for being one of the organizers of the European Common Lisp Meeting. He has a Ph.D. in mathematics and has been a free-lance Common Lisp consultant for clients in the US, Europe, and Asia since 2002. He now works as a professor for math and computer science at the University of Applied Sciences in Hamburg, Germany.

Table of Contents 5
About the Author 16
About the Technical Reviewer 17
Preface 18
Who This Book Is For 18
Who This Book Is (Probably) Not For 19
How to Read This Book 19
What's In and What Not 20
The HyperSpec 21
Which Implementation 21
Source Code 22
The Index 22
Typographical Conventions 22
Acknowledgements 23
1. Symbols and Packages 25
1-1. Understanding the Role of Packages and the Symbol Nomenclature 25
1-2. Making Unique Symbols 29
1-3. Making Symbols Inaccessible 33
How Can We Fix This? 35
1-4. Avoiding Name Conflicts 36
When Name Conflicts Do Not Occur 39
1-5. Using Symbols As Stand-Ins for Arbitrary Forms 39
1-6. Searching for Symbols by Name 41
1-7. Iterating Through All Symbols of a Package 43
What To Do If You Don't Like LOOP 44
1-8. Understanding Common Lisp's Case (In)Sensitivity 45
Style Hint: Don't Use CamelCase! 49
1-9. Using Symbols As String Substitutes 50
So, What Should You Use? 51
1-10. "Overloading" of Standard Common Lisp Operators 52
2. Conses, Lists, and Trees 55
2-1. Understanding Conses 55
List Access 58
Testing Whether Something Is a Cons or a List 59
2-2. Creating Lists 61
Converting Vectors to Lists 62
2-3. Transposing a Matrix 63
2-4. Using List Interpolation 64
2-5. Adding Objects to the End of a List 66
The Tail Wagging the List 68
2-6. "Splicing" into a List 69
2-7. Detecting Shared Structure in Lists 73
Isolating the Non-Shared Part 75
2-8. Working with Trees 75
More Complicated Trees 78
Common Lisp's Standard Tree Functions 79
2-9. Working with Stacks 79
2-10. Implementing a Queue 80
2-11. Destructuring and Pattern Matching 82
3. Strings and Characters 84
3-1. Getting the ASCII Code of a Character 84
The Other Way Around 86
The Limit 86
3-2. Naming Characters 86
3-3. Using Different Character Encodings 88
3-4. Comparing Strings or Characters 90
Internationalization 93
3-5. Escaping Characters in String Literals and Variable Interpolation 94
Is It Still a Literal? 95
3-6. Controlling Case 95
What About Unicode? 97
3-7. Accessing or Modifying a Substring 98
3-8. Finding a Character or a Substring Within a String 100
3-9. Trimming Strings 100
3-10. Processing a String One Character at a Time 102
3-11. Joining Strings 104
3-12. Reading CSV Data 106
4. Numbers and Math 109
4-1. Using Arbitrarily Large Integers 109
4-2. Understanding Fixnums 111
4-3. Performing Modular Arithmetic 114
Efficiency Considerations 115
4-4. Switching Bases 116
4-5. Performing Exact Arithmetic with Rational Numbers 118
Various Ways of Converting Numbers to Integers 120
How Not to Use FLOOR and Friends 121
Converting Floating-Point Numbers to Rationals and Vice Versa 121
Mixing Rationals and Floats 123
4-6. Controlling the Default Float Format 124
4-7. Employing Arbitrary Precision Floats 126
4-8. Working with Complex Numbers 128
4-9. Parsing Numbers 131
4-10. Testing Whether Two Numbers Are Equal 133
Don't Ever Use EQ with Numbers! 134
4-11. Computing Angles Correctly 135
4-12. Calculating Exact Square Roots 137
5. Arrays and Vectors 138
5-1. Working with Multiple Dimensions 138
5-2. Understanding Vectors and Simple Arrays 140
5-3. Obtaining the Size of an Array 141
5-4. Providing Initial Contents 142
A Warning About Identical Objects 143
5-5. Treating Arrays As Vectors 144
5-6. Making the Length of Vectors Flexible 146
5-7. Adjusting Arrays 148
5-8. Using an Array As a "Window" into Another Array 150
5-9. Restricting the Element Type of an Array 152
Upgrading Element Types 154
5-10. Copying an Array 155
A Warning About Object Identity 156
6. Hash Tables, Maps, and Sets 157
6-1. Understanding the Basics of Hash Tables 157
Why Does GETHASH Return Two Values? 159
How Many Entries Does the Hash Table Have? 160
6-2. Providing Default Values For Hash Table Lookups 160
6-3. Removing Hash Table Entries 162
6-4. Iterating Through a Hash Table 163
Don't Rely on Any Order! 166
Don't Modify While You're Iterating! 167
Can't This Be More Concise, Please? 167
6-5. Understanding Hash Table Tests and Defining Your Own 168
What Is SXHASH For? 172
6-6. Controlling Hash Table Growth 172
6-7. Getting Rid of Hash Table Entries Automatically 175
6-8. Representing Maps As Association Lists 178
Combining Lookup and Manipulation 181
Why Would Anybody Prefer Alists over Hash Tables? 182
6-9. Representing Maps As Property Lists 183
When to Prefer Plists over Alists 185
The Plist of a Symbol 186
6-10. Working with Sets 186
Representing Sets As Hash Tables 189
Representing Sets As Bit Patterns 189
7. Sequences and Iteration 191
7-1. Filtering a Sequence 191
7-2. Searching a Sequence 192
7-3. Sorting and Merging Sequences 195
7-4. Mixing Different Sequence Types 197
7-5. Re-Using a Part of a Sequence 197
7-6. Repeating Some Values Cyclically 199
Alternatives 201
7-7. Counting Down 202
7-8. Iterating over "Chunks" of a List 204
7-9. Closing over Iteration Variables 206
7-10. "Extending" Short Sequences in Iterations 207
7-11. Breaking out of LOOP 208
7-12. Making Sense of the MAP… Zoo 211
The Sequence Variants 214
7-13. Defining Your Own Sequence Types 214
7-14. Iterating with iterate 216
7-15. Iterating with series 220
What the Example Does 221
8. The Lisp Reader 223
8-1. Employing the Lisp Reader for Your Own Code 223
Why READ Is Potentially Dangerous 225
What READ Doesn't Do 225
The Optional Arguments to READ 226
Go Wild! 226
8-2. Troubleshooting Literal Object Notation 226
This Also Applies to Strings! 228
8-3. Evaluating Forms at Read Time 228
What to Look Out For 230
Alternatives 230
8-4. Embedding Literal Arrays into Your Code 231
The Usual Warning 232
8-5. Understanding the Different Ways to Refer to a Function 233
8-6. Repeating Something You Already Typed 234
They Don't Only Look Identical, They Are Identical! 236
8-7. Safely Experimenting with Readtables 236
Temporarily Switching to Standard IO Syntax 238
8-8. Changing the Syntax Type of a Character 239
The Six Syntax Types 240
How to Actually Change the Syntax Type 242
Some Things Never Change 242
8-9. Creating Your Own Reader Macros 243
What Reader Macro Functions Do 244
8-10. Working with Dispatching Macro Characters 246
8-11. Preserving Whitespace 248
9. Printing 250
9-1. Using the Printing Primitives 250
Printing Objects So That They Can Be Read Back in Again 254
Shortcuts 255
9-2. Printing to and into Strings 256
9-3. Printing NIL As a List 258
9-4. Extending FORMAT Control Strings Over More Than One Line 259
9-5. Using Functions As FORMAT Controls 260
9-6. Creating Your Own FORMAT Directives 262
9-7. Recursive Processing of FORMAT Controls 264
9-8. Controlling How Your Own Objects Are Printed 266
9-9. Controlling the Pretty Printer 268
9-10. Printing Long Lists 272
9-11. Pretty-Printing Compound Objects 276
Using the Pretty Printer from FORMAT 279
9-12. Modifying the Pretty Printer 281
10. Evaluation, Compilation, Control Flow 283
10-1. Comparing Arbitrary Lisp Objects 283
Comparing State 284
Constants 287
10-2.Using Constant Variables as Keys in CASE Macros 287
10-3. Using Arbitrary Variable Names for Keyword Parameters 289
Keyword Names Don't Have to Be Keywords 290
Keyword Dames Don't Have to Be Constant 291
10-4. Creating ``Static Local Variables,'' Like in C 291
10-5.``Preponing'' the Computation of Values 293
10-6. Modifying the Behavior of Functions You Don't Have the Source Of 296
10-7. Swapping the Values of Variables (or Places) 298
10-8. Creating Your Own Update Forms for ``Places'' 301
Using DEFSETF 303
Using DEFINE-SETF-EXPANDER 304
So, Which One Do I Use? 309
Using DEFINE-MODIFY-MACRO 309
Multiple-Valued Places 311
10-9. Working with Environments 312
10-10. Commenting Out Parts of Your Code 317
Some Notes About and #|
How , #|, and Others Are Implemented
11. Concurrency 321
11-1. Managing Lisp Processes 322
Escape Hatches 325
Threads Are Expensive 326
11-2. Accessing Shared Resources Concurrently 326
Locks 330
Atomic Operations 331
More Problems 332
11-3. Using Special Variables in Concurrent Programs 335
Per-Thread Initial Bindings 337
Variables That Are Always Global 337
11-4. Communicating with Other Threads 338
Alternatives 340
11-5. Parallelizing Algorithms Without Threads and Locks 340
What the Example Does 343
Fine-Tuning 344
Ptrees 344
Alternatives 348
11-6. Determining the Number of Cores 348
12. Error Handling and Avoidance 350
12-1. Checking Types at Run Time 350
Alternatives 352
12-2. Adding Assertions to Your Code 353
Disabling Assertions in ``Production Code'' 355
12-3. Defining Your Own Conditions 355
How Conditions Are Printed 357
12-4. Signaling a Condition 358
Condition Designators 360
12-5. Handling Conditions 361
Ignoring Errors 366
12-6. Providing and Using Restarts 367
Visible Restarts 371
Predefined Restarts 372
12-7. Getting Rid of Warning Messages 373
12-8. Protecting Code from Non-Local Exits 374
``WITH-'' Macros 377
13. Objects, Classes, Types 378
13-1. Defining Types 378
Compound Type Specifiers 380
Derived Types 382
13-2. Using Classes As Types 383
13-3. Writing Methods for Built-In Classes 384
13-4. Providing Constructors for Your Classes 386
13-5. Marking Slots As ``Private'' 389
13-6. Changing the Argument Precedence Order 391
13-7. Automatically Initializing Slots on First Usage 393
13-8. Changing and Redefining Classes on the Fly 394
Objects Changing Their Class 397
Redefining Classes 398
13-9. Making Your Objects Externalizable 400
13-10. Using and Defining Non-Standard Method Combinations 402
Rolling Your Own 405
Arbitrarily Complex Method Combinations 406
13-11. Extending and Modifying CLOS 408
What the example does 410
14. I/O: Streams and Files 413
14-1. Redirecting Streams 413
Other Ways to Do It 414
Synonym Streams 415
14-2. Flushing an Output Stream 416
14-3. Determining the Size of a File 417
14-4. Reading a Whole File at Once 418
Alternatives 420
14-5. Sending Data to Two Streams in Parallel 420
Synonym Streams 422
14-6. Sending Data to ``/dev/null'' 423
14-7. Pretending a String Is a Stream 424
More Details 426
14-8. Concatenating Streams 427
14-9. Processing Text Files Line by Line 428
What Happens at the End of a Line? 429
What Happens at the End of the File? 430
14-10. Working with Binary Data 431
Reading or Writing Several Bytes at Once 432
You Might Get Bigger Chunks Than You Asked For 433
14-11. Reading ``Foreign'' Binary Data 434
Floating-Point Values 437
14-12. Using Random Access I/O 438
Different Characters May Have Different Lengths 440
14-13. Serializing Lisp Objects 441
Shared Structure 443
Is It Readable? 444
Can This Be Done Faster, Please? 445
What About JSON or Other Formats? 446
14-14. Customizing Stream Behavior 446
15. Pathnames, Files, Directories 450
15-1. Getting and Setting the Current Directory 450
Shortcuts and Deviations 452
15-2. Testing Whether a File Exists 452
What About Directories? 454
15-3. Creating a Directory 454
Implementation-Specific Alternatives 456
What Might Go Wrong 456
15-4. Finding Files Matching a Pattern 457
15-5. Splitting a Filename into its Component Parts 460
15-6. Renaming a File 462
Implementation-Specific Alternatives 464
Don't Expect ``Move'' Behavior! 464
15-7. Deleting a File 465
What Does ``Success'' Mean Anyway? 466
15-8. Deleting a Directory 467
15-9. Copying a File 468
15-10. Processing the Contents of a Directory Recursively 469
The CL-FAD Library 470
15-11. Getting the Pathname a Stream Is Associated With 471
15-12. Dealing with Symbolic Links 472
What If I Want the Symlinks? 474
15-13. Navigating a Directory Tree 474
15-14. Figuring Out (Source) File Locations Programmatically 476
15-15. Understanding Logical Pathnames 478
What Exactly Are Logical Pathnames? 482
So, Maybe Logical Pathnames Aren't Totally Useless… 482
16. Developing and Debugging 484
16-1. Embracing Lisp's Image-Based Development Style 484
The Role of the Source Code 486
16-2. Deciding Which IDE to Use 486
A Brief History of Emacs (As Seen from Lisp) 489
Alternatives (?) 490
16-3. Debugging with the Debugger 490
Entering the Debugger Intentionally 493
Without SLIME 494
Logging Backtraces 495
16-4. Tracing Functions 496
Graphical Tracing 498
16-5. Stepping Through Your Code 499
16-6. Acquiring Information About Functions, Macros, and Variables 501
Accessing the HyperSpec 503
Cross-Reference Information 504
16-7. Inspecting and Modifying (Compound) Objects 504
The SLIME Inspector 507
16-8. Browsing Your Lisp Image 507
Alternatives 508
16-9. ``Undoing'' Definitions 509
16-10. Distinguishing Your IDE's Streams 512
16-11. Utilizing the REPL's Memory 514
IDE History Features 515
16-12. Recording Your Work 516
17. Optimization 518
17-1. Understanding the Importance of the Right Algorithms 519
17-2. Deciding If and Where to Optimize 520
Instrumentation 523
Statistical Profiling 525
Some Things Cannot Be Profiled 527
Where to Go From Here 527
A Warning About Empirical Data 528
What Does TIME Do? 528
CPUs Can Deceive You 528
17-3. Asking the Compiler to Optimize 530
Implementation-Defined Optimize Qualities 532
Can I Have This Global, Please? 532
Compilers Aren't Wizards 533
17-4. Obtaining Optimization Hints from the Compiler 533
17-5. Helping the Compiler by Providing Type Information 537
Generic Operations 540
Boxing 540
How to Declare Types 541
The Scope of Type Declarations 542
Declaring the Return Type of Forms 543
Type Inference 543
Pitfalls 544
17-6. Reducing ``Consing'' 544
``Consing'' and the Heap 547
Reusing Data Structures 548
Destructive Functions 549
``Hidden'' Consing 550
``Tuning'' the Garbage Collector 551
17-7. Using the Stack Instead of the Heap 551
Multiple Values 555
17-8. Optimizing Recursive Functions 557
17-9. Helping the Compiler with Alternative Implementation Strategies 559
17-10. Avoiding Repeated Computations 562
17-11. Avoiding Function Calls 564
Alternatives That Aren't Really Alternatives 568
The NOTINLINE Declaration and Compiler Macros 568
17-12. Utilizing the Disassembler 568
17-13. Switching to Machine Code 571
Inline Assembly Code 572
17-14. Optimizing Array Access 573
17-15. Comparing Different Implementations 575
18. Libraries 577
18-1. Organizing Your Code 577
Components 578
Dependencies 579
How Does ASDF Find System Definitions? 581
Additional Information 582
What About the Names? 582
Advanced Usage of ASDF 582
18-2. Employing Open Source Libraries 583
Installing Quicklisp 584
Using Quicklisp for Your Own Code 585
18-3. Creating a Project's Skeleton Automatically 585
18-4. Avoiding Wheel-Reinvention 586
18-5. Using Libraries to Write Portable Code 588
18-6. Utilizing Regular Expressions 590
Regex Syntax 591
Scanners 592
Convenience Features 593
Modifying and Dissecting Strings 595
More Information 595
18-7. Obtaining Data via HTTP 595
Parsing HTML 597
18-8. Creating Dynamic Web Sites with Lisp 598
Generating HTML 602
Web Frameworks 603
19. Interfacing with Other Languages 604
19-1. Calling C Functions from Lisp 605
How the FFI Finds and Loads Shared Libraries 607
How the FFI Calls C Functions 609
How the FFI Converts Between C and Lisp Types 611
The ``stdcall Problem'' 612
19-2. Working with C Pointers 613
Typed Pointers 615
19-3. Accessing and Generating C Arrays 617
Giving C Access to Lisp Arrays 619
19-4. Handling C Structs and Unions 620
Passing Structs by Value 624
19-5. Converting Between Lisp and C Strings 624
19-6. Calling Lisp Functions from C 627
19-7. Generating FFI Code Automatically 628
19-8. Embedding C in Common Lisp 630
19-9. Calling C++ from Lisp 631
Automating the Process 633
ECL and Clasp 635
19-10. Using Java from Lisp 635
Alternatives 638
19-11. Reading and Writing JSON 641
19-12. Reading and Writing XML 644
19-13. Using Prolog from Common Lisp 647
20. Graphical User Interfaces 650
20-1. Using a Web Browser as the GUI for Your Lisp Program 651
What the Example Does 654
20-2. Building Applications with the ``Lisp Toolkit'' 655
What the Example Does 658
20-3. Creating Common Lisp GUIs Through Java 659
What the Example Does 661
A Better Example 661
20-4. Using CAPI to Build Graphical User Interfaces 665
What the Example Does 668
20-5. Using Lisp on Mobile Devices 670
Alternatives 673
21. Persistence 674
21-1. Serializing Your Data 675
21-2. Accessing Relational Databases 677
21-3.Keeping Your Database in RAM 681
21-4. Using a Lisp Object Database 685
22. The World Outside 689
22-1. Accessing Environment Variables 689
The Windows Registry and Application-Specific Settings 691
22-2. Accessing the Command-Line Arguments 691
22-3. Querying Your Lisp for Information About Its Environment 692
22-4. Delivering Stand-Alone Executables 694
Delivering Programs with the Commercial Lisps 697
22-5. Customizing Your Lisp 697
Saving an Image 699
22-6. Running External Programs 700
22-7. Embedding Lisp 704
Creating Shared Libraries with LispWorks 706
Embedding ABCL in a Java Program 707
22-8. Measuring Time 708
22-9. Working with Dates and Times 710
The local-time Library 713
22-10. Working with the Garbage Collector 715
Finalizers 717
Index 720