Professional Python

Luke Sneeringer is a Python developer whose home is in Austin (Texas, USA), where he works as a developer for Ansible.

INTRODUCTION xxv

PART I: FUNCTIONS

CHAPTER 1: DECORATORS 3

Understanding Decorators 3

Decorator Syntax 4

Order of Decorator Application 5

Where Decorators Are Used 6

Why You Should Write Decorators 6

When You Should Write Decorators 7

Additional Functionality 7

Data Sanitization or Addition 7

Function Registration 7

Writing Decorators 7

An Initial Example: A Function Registry 7

Execution-Time Wrapping Code 9

A Simple Type Check 9

Preserving the help 10

User Verification 11

Output Formatting 12

Logging 14

Variable Arguments 15

Decorator Arguments 16

How Does This Work? 17

The Call Signature Matters 18

Decorating Classes 20

Type Switching 22

A Pitfall 24

Summary 25

CHAPTER 2: CONTEXT MANAGERS 27

What Is a Context Manager? 27

Context Manager Syntax 28

The with Statement 28

The enter and exit Methods 28

Exception Handling 29

When You Should Write Context Managers 30

Resource Cleanliness 30

Avoiding Repetition 31

Propagating Exceptions 31

Suppressing Exceptions 32

A Simpler Syntax 37

Summary 38

CHAPTER 3: GENERATORS 41

Understanding What a Generator Is 41

Understanding Generator Syntax 41

The next Function 43

The StopIteration Exception 45

Python 2 46

Python 3 47

Communication with Generators 47

Iterables Versus Iterators 49

Generators in the Standard Library 50

range 50

dict.items and Family 50

zip 51

map 51

File Objects 52

When to Write Generators 53

Accessing Data in Pieces 53

Computing Data in Pieces 54

Sequences Can Be Infi nite 54

When Are Generators Singletons? 54

Generators within Generators 55

Summary 56

PART II: CLASSES

CHAPTER 4: MAGIC METHODS 59

Magic Method Syntax 59

Available Methods 60

Creation and Destruction 61

__init__ 61

__new__ 62

__del__ 62

Type Conversion 63

__str__, __unicode__, and __bytes__ 63

__bool__ 64

__int__, __fl oat__, and __complex__ 65

Comparisons 65

Binary Equality 65

Relative Comparisons 67

Operator Overloading 68

Overloading Common Methods 71

Collections 75

Other Magic Methods 77

Summary 77

CHAPTER 5: METACLASSES 79

Classes and Objects 79

Using type Directly 80

Creating a Class 81

Creating a Subclass 81

The type Chain 82

The Role of type 82

Writing Metaclasses 83

The new Method 83

new Versus init 83

A Trivial Metaclass 84

Metaclass Inheritance 84

Using Metaclasses 87

Python 3 87

Python 2 88

What About Code That Might Run on Either Version? 88

When Is Cross-Compatibility Important? 89

When to Use Metaclasses 89

Declarative Class Declaration 89

An Existing Example 89

How This Works 90

Why This Is a Good Use for Metaclasses 91

Class Verification 91

Non-Inheriting Attributes 93

The Question of Explicit Opt-In 94

Meta-Coding 95

Summary 97

CHAPTER 6: CLASS FACTORIES 99

A Review of type 99

Understanding a Class Factory Function 100

Determining When You Should Write Class Factories 102

Runtime Attributes 102

Understanding Why You Should Do This 103

Attribute Dictionaries 104

Fleshing Out the Credential Class 104

The Form Example 105

Dodging Class Attribute Consistency 106

Class Attributes Versus Instance Attributes 107

The Class Method Limitation 108

Tying This in with Class Factories 109

Answering the Singleton Question 109

Summary 111

CHAPTER 7: ABSTRACT BASE CLASSES 113

Using Abstract Base Classes 113

Declaring a Virtual Subclass 115

Why Declare Virtual Subclasses? 115

Using register as a Decorator 117

__subclasshook__ 117

Declaring a Protocol 119

Other Existing Approaches 119

Using NotImplementedError 120

Using Metaclasses 120

The Value of Abstract Base Classes 122

Abstract Properties 124

Abstract Class or Static Methods 125

Built-in Abstract Base Classes 126

Single-Method ABCs 126

Alternative-Collection ABCs 127

Using Built-In Abstract Base Classes 128

Additional ABCs 128

Summary 128

PART III: DATA

CHAPTER 8: STRINGS AND UNICODE 131

Text String Versus Byte String 131

String Data in Python 132

Python 3 Strings 132

Python 2 Strings 134

six 136

Strings with Non-ASCII Characters 136

Observing the Difference 136

Unicode Is a Superset of ASCII 137

Other Encodings 137

Encodings Are Not Cross-Compatible 138

Reading Files 139

Python 3 139

Specifying Encoding 139

Reading Bytes 140

Python 2 140

Reading Other Sources 141

Specifying Python File Encodings 141

Strict Codecs 143

Suppressing Errors 143

Registering Error Handlers 144

Summary 145

CHAPTER 9: REGULAR EXPRESSIONS 147

Why Use Regular Expressions? 147

Regular Expressions in Python 148

Raw Strings 148

Match Objects 149

Finding More Than One Match 149

Basic Regular Expressions 150

Character Classes 150

Ranges 151

Negation 151

Shortcuts 152

Beginning and End of String 153

Any Character 154

Optional Characters 154

Repetition 155

Repetition Ranges 155

Open-Ended Ranges 156

Shorthand 156

Grouping 157

The Zero Group 159

Named Groups 159

Referencing Existing Groups 160

Lookahead 161

Flags 163

Case Insensitivity 163

ASCII and Unicode 163

Dot Matching Newline 163

Multiline Mode 164

Verbose Mode 164

Debug Mode 164

Using Multiple Flags 165

Inline Flags 165

Substitution 165

Compiled Regular Expressions 166

Summary 167

PART IV: EVERYTHING ELSE

CHAPTER 10: PYTHON 2 VERSUS PYTHON 3 171

Cross-Compatibility Strategies 171

The __future__ Module 172

2to3 172

Writing Changes 173

Limitations 174

six 174

Changes in Python 3 175

Strings and Unicode 175

The print Function 176

Division 176

Absolute and Relative Imports 177

Removal of “Old-Style” Classes 178

Metaclass Syntax 179

six.with_metaclass 179

six.add_metaclass 180

Exception Syntax 180

Handling Exceptions 181

Exception Chaining 181

Dictionary Methods 182

Function Methods 183

Iterators 183

Standard Library Relocations 184

Merging “Fast” Modules 184

io 184

pickle 184

The URL Modules 185

Renames 185

Other Package Reorganizations 185

Version Detection 186

Summary 186

CHAPTER 11: UNIT TESTING 187

The Testing Continuum 187

The Copied Ecosystem 188

The Isolated Environment 188

Advantages and Disadvantages 189

Speed 189

Interactivity 189

Testing Code 190

Code Layout 190

Testing the Function 191

The assert Statement 192

Unit Testing Frameworks 192

Running Unit Tests 193

Failures 193

Errors 194

Skipped Tests 195

Loading Tests 196

Mocking 197

Mocking a Function Call 197

Asserting Mocked Calls 199

Inspecting Mocks 201

Call Count and Status 201

Multiple Calls 202

Inspecting Calls 203

Other Testing Tools 203

coverage 203

tox 204

Other Test Runners 205

Summary 205

CHAPTER 12: CLI TOOLS 207

optparse 207

A Simple Argument 207

name == ‘ main__’ 208

OptionParser 208

Options 209

Types of Options 209

Adding Options to OptionParser 209

Options with Values 210

Non-String Values 211

Specifying Option Values 212

Positional Arguments 214

Counters 214

List Values 215

Why Use optparse? 216

argparse 216

The Bare Bones 217

Arguments and Options 217

Option Flags 217

Alternate Prefi xes 218

Options with Values 219

Positional Arguments 222

Reading Files 223

Why Use argparse? 224

Summary 224

CHAPTER 13: ASYNCIO 225

The Event Loop 225

A Simple Event Loop 226

Running the Loop 226

Registering Tasks and Running the Loop 227

Delaying Calls 227

Partials 228

Running the Loop until a Task Completes 228

Running a Background Loop 229

Coroutines 230

Nested Coroutines 231

Futures and Tasks 232

Futures 232

Tasks 232

Callbacks 234

No Guarantee of Success 235

Under the Hood 235

Callbacks with Arguments 235

Task Aggregation 236

Gathering Tasks 236

Waiting on Tasks 237

Timeouts 238

Waiting on Any Task 239

Waiting on an Exception 239

Queues 240

Maximum Size 242

Servers 242

Summary 244

CHAPTER 14: STYLE 245

Principles 245

Assume Your Code Will Require Maintenance 245

Be Consistent 246

Think About Ontology, Especially with Data 246

Do Not Repeat Yourself 246

Have Your Comments Explain the Story 247

Occam’s Razor 247

Standards 248

Trivial Rules 248

Documentation Strings 248

Blank Lines 249

Imports 249

Variables 250

Comments 250

Line Length 251

Summary 251

INDEX 253