Python Cookbook

David Beazley is an independent software developer and book author living in the city of Chicago. He primarily works on programming tools, provide custom software development, and teach practical programming courses for software developers, scientists, and engineers. He is best known for his work with the Python programming language, for which he has created several open-source packages (e.g., Swig and PLY) and authored the acclaimed Python Essential Reference. He also has significant experience with systems programming in C, C++, and assembly language.

Brian K. Jones is a system administrator in the department of computer science at Princeton University.

Chapter 1 Data Structures and Algorithms
Unpacking a Sequence into Separate Variables
Unpacking Elements from Iterables of Arbitrary Length
Keeping the Last N Items
Finding the Largest or Smallest N Items
Implementing a Priority Queue
Mapping Keys to Multiple Values in a Dictionary
Keeping Dictionaries in Order
Calculating with Dictionaries
Finding Commonalities in Two Dictionaries
Removing Duplicates from a Sequence while Maintaining Order
Naming a Slice
Determining the Most Frequently Occurring Items in a Sequence
Sorting a List of Dictionaries by a Common Key
Sorting Objects Without Native Comparison Support
Grouping Records Together Based on a Field
Filtering Sequence Elements
Extracting a Subset of a Dictionary
Mapping Names to Sequence Elements
Transforming and Reducing Data at the Same Time
Combining Multiple Mappings into a Single Mapping
Chapter 2 Strings and Text
Splitting Strings on Any of Multiple Delimiters
Matching Text at the Start or End of a String
Matching Strings Using Shell Wildcard Patterns
Matching and Searching for Text Patterns
Searching and Replacing Text
Searching and Replacing Case-Insensitive Text
Specifying a Regular Expression for the Shortest Match
Writing a Regular Expression for Multiline Patterns
Normalizing Unicode Text to a Standard Representation
Working with Unicode Characters in Regular Expressions
Stripping Unwanted Characters from Strings
Sanitizing and Cleaning Up Text
Aligning Text Strings
Combining and Concatenating Strings
Interpolating Variables in Strings
Reformatting Text to a Fixed Number of Columns
Handling HTML and XML Entities in Text
Tokenizing Text
Writing a Simple Recursive Descent Parser
Performing Text Operations on Byte Strings
Chapter 3 Numbers, Dates, and Times
Rounding Numerical Values
Performing Accurate Decimal Calculations
Formatting Numbers for Output
Working with Binary, Octal, and Hexadecimal Integers
Packing and Unpacking Large Integers from Bytes
Performing Complex-Valued Math
Working with Infinity and NaNs
Calculating with Fractions
Calculating with Large Numerical Arrays
Performing Matrix and Linear Algebra Calculations
Picking Things at Random
Converting Days to Seconds, and Other Basic Time Conversions
Determining Last Friday’s Date
Finding the Date Range for the Current Month
Converting Strings into Datetimes
Manipulating Dates Involving Time Zones
Chapter 4 Iterators and Generators
Manually Consuming an Iterator
Delegating Iteration
Creating New Iteration Patterns with Generators
Implementing the Iterator Protocol
Iterating in Reverse
Defining Generator Functions with Extra State
Taking a Slice of an Iterator
Skipping the First Part of an Iterable
Iterating Over All Possible Combinations or Permutations
Iterating Over the Index-Value Pairs of a Sequence
Iterating Over Multiple Sequences Simultaneously
Iterating on Items in Separate Containers
Creating Data Processing Pipelines
Flattening a Nested Sequence
Iterating in Sorted Order Over Merged Sorted Iterables
Replacing Infinite while Loops with an Iterator
Chapter 5 Files and I/O
Reading and Writing Text Data
Printing to a File
Printing with a Different Separator or Line Ending
Reading and Writing Binary Data
Writing to a File That Doesn’t Already Exist
Performing I/O Operations on a String
Reading and Writing Compressed Datafiles
Iterating Over Fixed-Sized Records
Reading Binary Data into a Mutable Buffer
Memory Mapping Binary Files
Manipulating Pathnames
Testing for the Existence of a File
Getting a Directory Listing
Bypassing Filename Encoding
Printing Bad Filenames
Adding or Changing the Encoding of an Already Open File
Writing Bytes to a Text File
Wrapping an Existing File Descriptor As a File Object
Making Temporary Files and Directories
Communicating with Serial Ports
Serializing Python Objects
Chapter 6 Data Encoding and Processing
Reading and Writing CSV Data
Reading and Writing JSON Data
Parsing Simple XML Data
Parsing Huge XML Files Incrementally
Turning a Dictionary into XML
Parsing, Modifying, and Rewriting XML
Parsing XML Documents with Namespaces
Interacting with a Relational Database
Decoding and Encoding Hexadecimal Digits
Decoding and Encoding Base64
Reading and Writing Binary Arrays of Structures
Reading Nested and Variable-Sized Binary Structures
Summarizing Data and Performing Statistics
Chapter 7 Functions
Writing Functions That Accept Any Number of Arguments
Writing Functions That Only Accept Keyword Arguments
Attaching Informational Metadata to Function Arguments
Returning Multiple Values from a Function
Defining Functions with Default Arguments
Defining Anonymous or Inline Functions
Capturing Variables in Anonymous Functions
Making an N-Argument Callable Work As a Callable with Fewer Arguments
Replacing Single Method Classes with Functions
Carrying Extra State with Callback Functions
Inlining Callback Functions
Accessing Variables Defined Inside a Closure
Chapter 8 Classes and Objects
Changing the String Representation of Instances
Customizing String Formatting
Making Objects Support the Context-Management Protocol
Saving Memory When Creating a Large Number of Instances
Encapsulating Names in a Class
Creating Managed Attributes
Calling a Method on a Parent Class
Extending a Property in a Subclass
Creating a New Kind of Class or Instance Attribute
Using Lazily Computed Properties
Simplifying the Initialization of Data Structures
Defining an Interface or Abstract Base Class
Implementing a Data Model or Type System
Implementing Custom Containers
Delegating Attribute Access
Defining More Than One Constructor in a Class
Creating an Instance Without Invoking init
Extending Classes with Mixins
Implementing Stateful Objects or State Machines
Calling a Method on an Object Given the Name As a String
Implementing the Visitor Pattern
Implementing the Visitor Pattern Without Recursion
Managing Memory in Cyclic Data Structures
Making Classes Support Comparison Operations
Creating Cached Instances
Chapter 9 Metaprogramming
Putting a Wrapper Around a Function
Preserving Function Metadata When Writing Decorators
Unwrapping a Decorator
Defining a Decorator That Takes Arguments
Defining a Decorator with User Adjustable Attributes
Defining a Decorator That Takes an Optional Argument
Enforcing Type Checking on a Function Using a Decorator
Defining Decorators As Part of a Class
Defining Decorators As Classes
Applying Decorators to Class and Static Methods
Writing Decorators That Add Arguments to Wrapped Functions
Using Decorators to Patch Class Definitions
Using a Metaclass to Control Instance Creation
Capturing Class Attribute Definition Order
Defining a Metaclass That Takes Optional Arguments
Enforcing an Argument Signature on *args and **kwargs
Enforcing Coding Conventions in Classes
Defining Classes Programmatically
Initializing Class Members at Definition Time
Implementing Multiple Dispatch with Function Annotations
Avoiding Repetitive Property Methods
Defining Context Managers the Easy Way
Executing Code with Local Side Effects
Parsing and Analyzing Python Source
Disassembling Python Byte Code
Chapter 10 Modules and Packages
Making a Hierarchical Package of Modules
Controlling the Import of Everything
Importing Package Submodules Using Relative Names
Splitting a Module into Multiple Files
Making Separate Directories of Code Import Under a Common Namespace
Reloading Modules
Making a Directory or Zip File Runnable As a Main Script
Reading Datafiles Within a Package
Adding Directories to sys.path
Importing Modules Using a Name Given in a String
Loading Modules from a Remote Machine Using Import Hooks
Patching Modules on Import
Installing Packages Just for Yourself
Creating a New Python Environment
Distributing Packages
Chapter 11 Network and Web Programming
Interacting with HTTP Services As a Client
Creating a TCP Server
Creating a UDP Server
Generating a Range of IP Addresses from a CIDR Address
Creating a Simple REST-Based Interface
Implementing a Simple Remote Procedure Call with XML-RPC
Communicating Simply Between Interpreters
Implementing Remote Procedure Calls
Authenticating Clients Simply
Adding SSL to Network Services
Passing a Socket File Descriptor Between Processes
Understanding Event-Driven I/O
Sending and Receiving Large Arrays
Chapter 12 Concurrency
Starting and Stopping Threads
Determining If a Thread Has Started
Communicating Between Threads
Locking Critical Sections
Locking with Deadlock Avoidance
Storing Thread-Specific State
Creating a Thread Pool
Performing Simple Parallel Programming
Dealing with the GIL (and How to Stop Worrying About It)
Defining an Actor Task
Implementing Publish/Subscribe Messaging
Using Generators As an Alternative to Threads
Polling Multiple Thread Queues
Launching a Daemon Process on Unix
Chapter 13 Utility Scripting and System Administration
Accepting Script Input via Redirection, Pipes, or Input Files
Terminating a Program with an Error Message
Parsing Command-Line Options
Prompting for a Password at Runtime
Getting the Terminal Size
Executing an External Command and Getting Its Output
Copying or Moving Files and Directories
Creating and Unpacking Archives
Finding Files by Name
Reading Configuration Files
Adding Logging to Simple Scripts
Adding Logging to Libraries
Making a Stopwatch Timer
Putting Limits on Memory and CPU Usage
Launching a Web Browser
Chapter 14 Testing, Debugging, and Exceptions
Testing Output Sent to stdout
Patching Objects in Unit Tests
Testing for Exceptional Conditions in Unit Tests
Logging Test Output to a File
Skipping or Anticipating Test Failures
Handling Multiple Exceptions
Catching All Exceptions
Creating Custom Exceptions
Raising an Exception in Response to Another Exception
Reraising the Last Exception
Issuing Warning Messages
Debugging Basic Program Crashes
Profiling and Timing Your Program
Making Your Programs Run Faster
Chapter 15 C Extensions
Accessing C Code Using ctypes
Writing a Simple C Extension Module
Writing an Extension Function That Operates on Arrays
Managing Opaque Pointers in C Extension Modules
Defining and Exporting C APIs from Extension Modules
Calling Python from C
Releasing the GIL in C Extensions
Mixing Threads from C and Python
Wrapping C Code with Swig
Wrapping Existing C Code with Cython
Using Cython to Write High-Performance Array Operations
Turning a Function Pointer into a Callable
Passing NULL-Terminated Strings to C Libraries
Passing Unicode Strings to C Libraries
Converting C Strings to Python
Working with C Strings of Dubious Encoding
Passing Filenames to C Extensions
Passing Open Files to C Extensions
Reading File-Like Objects from C
Consuming an Iterable from C
Diagnosing Segmentation Faults
Appendix Further Reading
Online Resources
Books for Learning Python
Advanced Books
Index
Colophon