Mastering Regular Expressions

Jeffrey Friedl was raised in the countryside of Rootstown, Ohio, and had aspirations of being an astronomer until one day noticing a TRS-80 Model I sitting unused in the corner of the chem lab (bristling with a full 16k RAM, no less). He eventually began using UNIX (and regular expressions) in 1980. With degrees in computer science from Kent (B.S.) and the University of New Hampshire (M.S.), he is now an engineer with Omron Corporation, Kyoto, Japan. He lives in Nagaokakyou-city with Tubby, his long-time friend and Teddy Bear, in a tiny apartment designed for a (Japanese) family of three. Jeffrey applies his regular-expression know-how to make the world a safer place for those not bilingual in English and Japanese. He built and maintains the World Wide Web Japanese-English dictionary server, http://www.itc.omron.com/cgi-bin/j-e, and is active in a variety of language-related projects, both in print and on the Web. When faced with the daunting task of filling his copious free time, Jeffrey enjoys riding through the mountainous countryside of Japan on his Honda CB-1. At the age of 30, he finally decided to put his 6'4" height to some use, and joined the Omron company basketball team. While finalizing the manuscript for Mastering Regular Expressions., he took time out to appear in his first game, scoring five points in nine minutes of play, which he feels is pretty darn good for a geek. When visiting his family in The States, Jeffrey enjoys dancing a two-step with his mom, binking old coins with his dad, and playing schoffkopf with his brothers and sisters.

Preface. Why I Wrote This Book. Intended Audience. How to Read This Book. This Book, as a Story. This Book, as a Reference. Organization. The Introduction. The Details. Tool-Specific Information. Typographical Conventions. Exercises. Personal Comments and Acknowledgments. Shoulders to Stand On. Other Thanks In the Future. Chapter 1. Introduction to Regular Expressions Solving Real Problems Regular Expressions as a Language The Filename Analogy The Language Analogy The Regular-Expression Frame of Mind Searching Text Files: Egrep Egrep Metacharacters Start and End of the Line Character Classes Matching Any Character -- Dot Alternation Word Boundaries In a Nutshell Optional Items Other Quantifiers: Repetition Ignoring Differences in Capitalization Parentheses and Backreferences The Great Escape Expanding the Foundation Linguistic Diversification The Goal of a Regular Expression A Few More Examples Regular Expression Nomenclature Improving on the Status Quo Summary Personal Glimpses. Chapter 2. Extended Introductory Examples About the Examples A Short Introduction to Perl Matching Text with Regular Expressions Toward a More Real-World Example Side Effects of a Successful Match Intertwined Regular Expressions Intermission Modifying Text with Regular Expressions Automated Editing A Small Mail Utility That Doubled-Word Thing. Chapter 3. Overview of Regular Expression Features and Flavors A Casual Stroll Across the Regex Landscape The World According to Grep The Times They Are a'/|Changin' At a Glance POSIX Care and Handling of Regular Expressions Identifying a Regex Doing Something with the Matched Text Other Examples Care and Handling: Summary Engines and Chrome Finish Chrome and Appearances Engines and Drivers Common Metacharacters Character Shorthands Strings as Regular Expressions Class Shorthands, Dot, and Character Classes Anchoring Grouping and Retrieving Quantifiers Alternation Guide to the Advanced Chapters Tool-Specific Information. Chapter 4. The Mechanics of Expression Processing Start Your Engines! Two Kinds of Engines New Standards Regex Engine Types From the Department of Redundancy Department Match Basics About the Examples Rule 1: The Earliest Match Wins The ``Transmission'' and the Bump-Along Engine Pieces and Parts Rule 2: Some Metacharacters Are Greedy Regex-Directed vs. Text-Directed NFA Engine: Regex-Directed DFA Engine: Text-Directed The Mysteries of Life Revealed Backtracking A Really Crummy Analogy Two Important Points on Backtracking Saved States Backtracking and Greediness More About Greediness Problems of Greediness Multi-Character ``Quotes'' Laziness? Greediness Always Favors a Match. Is Alternation Greedy? Uses for Non-Greedy Alternation Greedy Alternation in Perspective Character Classes vs. Alternation NFA/, DFA/, and POSIX ``The Longest-Leftmost'' POSIX and the Longest-Leftmost Rule Speed and Efficiency DFA and NFA in Comparison Practical Regex Techniques Contributing Factors Be Specific Difficulties and Impossibilities Watching Out for Unwanted Matches Matching Delimited Text Knowing Your Data and Making Assumptions Additional Greedy Examples Summary Match Mechanics Summary Some Practical Effects of Match Mechanics. Chapter 5. Crafting a Regular Expression A Sobering Example A Simple Change -- Placing Your Best Foot Forward More Advanced -- Localizing the Greediness Reality Check A Global View of Backtracking More Work for a POSIX NFA Work Required During a Non-Match Being More Specific Alternation Can Be Expensive A Strong Lead The Impact of Parentheses Internal Optimizations First-Character Discrimination Fixed-String Check Simple Repetition Needless Small Quantifiers Length Cognizance Match Cognizance Need Cognizance String/Line Anchors Compile Caching Testing the Engine Type Basic NFA vs. DFA Testing Traditional NFA vs. POSIX NFA Testing Unrolling the Loop Method 1: Building a Regex From Past Experiences The Real ``Unrolling the Loop'' Pattern Method 2: A Top-Down View Method 3: A Quoted Internet Hostname Observations Unrolling C Comments Regex Headaches A Na/(:ive View Unrolling the C Loop The Freeflowing Regex A Helping Hand to Guide the Match A Well-Guided Regex is a Fast Regex Wrapup Think! The Many Twists and Turns of Optimizations. Chapter 6. Tool-Specific Information Questions You Should Be Asking Something as Simple as Grep... In This Chapter Awk Differences Among Awk Regex Flavors Awk Regex Functions and Operators Tcl Tcl Regex Operands Using Tcl Regular Expressions Tcl Regex Optimizations GNU Emacs Emacs Strings as Regular Expressions Emacs's Regex Flavor Emacs Match Results Benchmarking in Emacs Emacs Regex Optimizations. Chapter 7. Perl Regular Expressions The Perl Way Regular Expressions as a Language Component Perl's Greatest Strength Perl's Greatest Weakness A Chapter, a Chicken, and The Perl Way An Introductory Example: Parsing CSV Text Regular Expressions and The Perl Way Perl Unleashed Regex-Related Perlisms Expression Context Dynamic Scope and Regex Match Effects Special Variables Modified by a Match ``Doublequotish Processing'' and Variable Interpolation Perl's Regex Flavor Quantifiers -- Greedy and Lazy Grouping String Anchors Multi-Match Anchor Word Anchors Convenient Shorthands and Other Notations Character Classes Modification with /Q and Friends: True Lies The Match Operator Match-Operand Delimiters Match Modifiers Specifying the Match Target Operand Other Side Effects of the Match Operator Match Operator Return Value Outside Influences on the Match Operator The Substitution Operator The Replacement Operand The /e Modifier Context and Return Value Using /g with a Regex That Can Match Nothingness The Split Operator Basic Split Advanced Split Advanced Split's Match Operand Scalar-Context Split Split's Match Operand with Capturing Parentheses Perl Efficiency Issues ``There's More Than One Way to Do It'' Regex Compilation, the /o Modifier, and Efficiency Unsociable $ and Friends The Efficiency Penalty of the /i Modifier Substitution Efficiency Concerns Benchmarking Regex Debugging Information The Study Function Putting It All Together Stripping Leading and Trailing Whitespace Adding Commas to a Number Removing C Comments Matching an Email Address Final Comments Notes for Perl4 Appendix A. Online Information General Information Mastering Regular Expressions O'Reilly Associates OAK Archive's Virtual Software Library The GNU Archive Yahoo! Other Web Links Awk C Library Packages Egrep Emacs Flex Perl Python Tcl Appendix B. Email Regex Program