Theory of Computation

Vivek Kulkarni is currently working as Principal Architect in Persistent Systems Ltd. He has more than 18 years of experience in academia and software industry. He has served as a subject chairman for multiple subjects for the Board of Computer Engineering, University of Pune. He has also worked in organizations such as BMC Software, Symantec Corporation, and Tech-Mahindra. He is also one of the inventors for System and Method of Universal Programming Language Conversion, which has been internationally recognized and patented.

PREFACE 00 ; FOREWARD 00 ; FEATURES OF THE BOOK 00 ; 1. PRELIMINARIES 1 ; 1.1 INTRODUCTION 1 ; 1.2 BASIC CONCEPTS 1 ; 1.2.1 SYMBOL 1 ; 1.2.2 ALPHABET 1 ; 1.2.3 STRING (OR WORD) 2 ; 1.3 SETS 2 ; 1.3.1 OPERATIONS 3 ; 1.3.2 CARDINALITY 7 ; 1.3.3 COUNTABLE AND UNCOUNTABLE SETS 7 ; 1.4 RELATIONS 8 ; 1.4.1 PROPERTIES 10 ; 1.4.2 CLOSURE PROPERTIES 10 ; 1.5 GRAPH 11 ; 1.5.1 DIRECTED GRAPH (OR DIGRAPH) 12 ; 1.5.2 TREE 13 ; 1.6 LANGUAGE 13 ; 1.6.1 FORMAL LANGUAGES 14 ; 1.7 MATHEMATICAL INDUCTION 14 ; 2. FINITE STATE MACHINES 20 ; 2.1 INTRODUCTION 20 ; 2.1.1 CONCEPT OF BASIC MACHINE 21 ; 2.2 FINITE STATE MACHINE 22 ; 2.2.1 EXAMPLES 22 ; 2.2.2 TRANSITION DIAGRAM (OR TRANSITION GRAPH) 24 ; 2.2.3 TRANSITION MATRIX 24 ; 2.3 FINITE AUTOMATA 29 ; 2.3.1 TRANSITION GRAPH 30 ; 2.3.2 FUNCTIONS 31 ; 2.3.3 ACCEPTANCE OF A STRING 32 ; 2.3.4 ACCEPTANCE OF A LANGUAGE 32 ; 2.3.5 SOME EXAMPLES OF FA AS ACCEPTOR 33 ; 2.3.6 FA AS FINITE CONTROL 35 ; 2.4 DETERMINISTIC FINITE AUTOMATA 36 ; 2.5 NON-DETERMINISTIC FINITE AUTOMATA 36 ; 2.6 EQUIVALENCE OF NFA AND DFA 37 ; 2.6.1 NFA TO DFA CONVERSION (METHOD I) 38 ; 2.6.2 DFA MINIMIZATION 41 ; 2.6.3 NFA TO DFA CONVERSION (METHOD II) 43 ; 2.7 NFA WITH ?-TRANSITIONS 47 ; 2.7.1 SIGNIFICANCE OF NFA WITH ?-TRANSITIONS 49 ; 2.7.2 STATE TRANSITION TABLE FOR NFA WITH ?-TRANSITIONS 50 ; 2.7.3 ?-CLOSURE OF A STATE 50 ; 2.8 EQUIVALENCE OF NFA AND NFA WITH ?-TRANSITIONS 50 ; 2.9 EQUIVALENCE OF DFA AND NFA WITH ?-TRANSITIONS 53 ; 2.9.1 INDIRECT CONVERSION METHOD 53 ; 2.9.2 DIRECT CONVERSION METHOD 55 ; 2.10 FINITE AUTOMATA WITH OUTPUT 57 ; 2.10.1 MOORE MACHINE 57 ; 2.10.2 MEALY MACHINE 59 ; 2.10.3 FINITE STATE TRANSDUCER 63 ; 2.11 EQUIVALENCE OF MOORE AND MEALY MACHINES 63 ; 2.11.1 MOORE TO MEALY CONVERSION 64 ; 2.11.2 MEALY TO MOORE CONVERSION 66 ; 2.11.3 ADDITIONAL EXAMPLES ON MOORE AND MEALY MACHINES 68 ; 2.12 FSM EQUIVALENCE 75 ; 2.12.1 MOORE'S ALGORITHM 75 ; 2.13 DFA MINIMIZATION (ANOTHER APPROACH) 77 ; 2.14 PROPERTIES AND LIMITATIONS OF FSM 79 ; 2.15 ADDITIONAL FSM EXAMPLES 80 ; 2.16 TWO-WAY FINITE AUTOMATON 83 ; 3. REGULAR EXPRESSIONS 94 ; 3.1 INTRODUCTION 94 ; 3.2 REGULAR EXPRESSION FORMALISM 95 ; 3.3 EXAMPLES OF REGULAR EXPRESSIONS 96 ; 3.4 EQUIVALENCE OF REGULAR EXPRESSIONS AND FINITE AUTOMATA 102 ; 3.4.1 KLEENE'S THEOREM 102 ; 3.4.2 REGULAR EXPRESSION TO FA CONVERSION 103 ; 3.4.3 DFA TO REGULAR EXPRESSION CONVERSION 109 ; 3.5 REGULAR SETS AND THEIR CLOSURE PROPERTIES 120 ; 3.5.1 FORMAL DEFINITION FOR REGULAR SETS 120 ; 3.5.2 CLOSURE PROPERTIES OF REGULAR SETS 120 ; 3.6 PUMPING LEMMA FOR REGULAR LANGUAGES 121 ; 3.6.1 APPLICATIONS OF PUMPING LEMMA 123 ; 3.7 DECISION ALGORITHMS FOR REGULAR SETS 125 ; 3.8 APPLICATIONS OF REGULAR EXPRESSIONS AND FINITE AUTOMATA 126 ; 3.8.1 LEXICAL ANALYSER 127 ; 3.8.2 TEXT EDITORS 128 ; 3.8.3 'GREP' COMMAND 128 ; 3.9 ADDITIONAL EXAMPLES 128 ; 3.10 MYHILL-NERODE THEOREM 133 ; 4. TURING MACHINES 139 ; 4.1 INTRODUCTION 139 ; 4.2 ELEMENTS OF A TURING MACHINE 140 ; 4.3 TURING MACHINE FORMALISM 141 ; 4.4 INSTANTANEOUS DESCRIPTION 143 ; 4.5 TRANSITION GRAPH FOR TURING MACHINE 145 ; 4.6 SOLVED PROBLEMS 146 ; 4.7 COMPLEXITY OF A TURING MACHINE 199 ; 4.8 COMPOSITE AND ITERATIVE TURING MACHINES 200 ; 4.9 UNIVERSAL TURING MACHINE 203 ; 4.10 MULTI-TAPE TURING MACHINE 205 ; 4.11 MULTI-STACK TURING MACHINE 206 ; 4.12 MULTI-TRACK TURING MACHINE 206 ; 4.13 SOLVABLE, SEMI-SOLVABLE, AND UNSOLVABLE PROBLEMS 207 ; 4.14 HALTING PROBLEM 208 ; 4.15 RECURSIVELY ENUMERABLE AND RECURSIVE LANGUAGES 210 ; 4.16 FUNCTIONS 210 ; 4.16.1 TOTAL RECURSIVE FUNCTIONS 211 ; 4.16.2 PARTIAL RECURSIVE FUNCTIONS 211 ; 4.17 CHURCH'S TURING HYPOTHESIS 211 ; 4.18 POST'S CORRESPONDENCE PROBLEM 212 ; 4.19 ADDITIONAL TURING MACHINE EXAMPLES 213 ; 4.20 LINEAR BOUNDED AUTOMATA 226 ; 5. GRAMMARS 233 ; 5.1 INTRODUCTION 233 ; 5.2 CONSTITUENTS OF GRAMMAR 234 ; 5.3 FORMAL DEFINITION OF A GRAMMAR 234 ; 5.4 GRAMMAR NOTATIONS 235 ; 5.5 DERIVATION PROCESS 236 ; 5.5.1 LEFTMOST DERIVATION 236 ; 5.5.2 RIGHTMOST DERIVATION 237 ; 5.5.3 DERIVATION EXAMPLES 238 ; 5.6 DERIVATION TREE 239 ; 5.7 CONTEXT-FREE LANGUAGES 240 ; 5.7.1 EXAMPLES OF CFLS 240 ; 5.8 AMBIGUOUS CONTEXT-FREE GRAMMAR 252 ; 5.8.1 REMOVAL OF AMBIGUITY 253 ; 5.9 SIMPLIFICATION OF CFG 257 ; 5.9.1 REMOVAL OF USELESS SYMBOLS 257 ; 5.9.2 REMOVAL OF UNIT PRODUCTIONS 258 ; 5.9.3 ELIMINATION OF ?-PRODUCTIONS 260 ; 5.10 NORMAL FORMS 265 ; 5.10.1 CHOMSKY NORMAL FORM 265 ; 5.10.2 GREIBACH NORMAL FORM 267 ; 5.11 CHOMSKY HIERARCHY 269 ; 5.11.1 UNRESTRICTED GRAMMAR (TYPE-0 GRAMMAR) 270 ; 5.11.2 CONTEXT-SENSITIVE GRAMMAR (TYPE-1 GRAMMAR) 270 ; 5.11.3 CONTEXT-FREE GRAMMAR (TYPE-2 GRAMMAR) 271 ; 5.11.4 REGULAR GRAMMAR (TYPE-3 GRAMMAR) 271 ; 5.12 EQUIVALENCE OF RIGHT-LINEAR AND LEFT-LINEAR GRAMMARS 273 ; 5.12.1 CONVERSION OF RIGHT-LINEAR GRAMMAR TO EQUIVALENT LEFT-LINEAR GRAMMAR 273 ; 5.12.2 CONVERSION OF LEFT-LINEAR GRAMMAR TO EQUIVALENT RIGHT-LINEAR GRAMMAR 275 ; 5.13 EQUIVALENCE OF REGULAR GRAMMARS AND FINITE AUTOMATA 277 ; 5.13.1 RIGHT-LINEAR GRAMMAR AND FA 277 ; 5.13.2 LEFT-LINEAR GRAMMAR AND FA 280 ; 5.14 PUMPING LEMMA FOR CONTEXT-FREE LANGUAGES 283 ; 5.14.1 APPLICATION OF PUMPING LEMMA 286 ; 5.14.2 OGDEN'S LEMMA 288 ; 5.15 KURODA NORMAL FORM 288 ; 5.16 DYCK LANGUAGE 289 ; 5.17 DERIVATION GRAPH 289 ; 5.18 APPLICATIONS OF CFG 291 ; 5.18.1 PARSER (OR SYNTAX ANALYSER) 291 ; 5.19 BACKUS-NAUR FORM 292 ; 6. PUSHDOWN STACK-MEMORY MACHINE 300 ; 6.1 INTRODUCTION 300 ; 6.2 ELEMENTS OF A PDM 301 ; 6.2.1 PICTORIAL REPRESENTATION OF PDM ELEMENTS 301 ; 6.3 PUSHDOWN AUTOMATA 302 ; 6.4 FINITE AUTOMATA VS PDA 304 ; 6.4.1 EXAMPLES OF PDA THAT ACCEPT REGULAR LANGUAGES 304 ; 6.4.2 RELATIVE COMPUTATIONAL POWERS OF PDA AND FA 307 ; 6.5 PDA ACCEPTING CFLS 307 ; 6.5.1 INSTANTANEOUS DESCRIPTION OF PDA 311 ; 6.5.2 ACCEPTANCE OF CFL BY EMPTY STACK 312 ; 6.5.3 ACCEPTANCE OF CFL BY FINAL STATE 312 ; 6.5.4 STATE TRANSITION DIAGRAM FOR A PDA 312 ; 6.6 DPDA VS NPDA 317 ; 6.6.1 RELATIVE POWERS OF DPDA/NPDA AND NFA/DFA 320 ; 6.7 EQUIVALENCE OF CFG AND PDA 321 ; 6.7.1 NPDA CONSTRUCTION USING CHOMSKY NORMAL FORM 326 ; 6.8 CLOSURE PROPERTIES OF CFLS 329 ; 6.9 ADDITIONAL PDA EXAMPLES 332 ; 7. PARSING TECHNIQUES 339 ; 7.1 INTRODUCTION 339 ; 7.2 INTRODUCTION TO PARSING 339 ; 7.3 TOP-DOWN PARSING 340 ; 7.3.1 WHY LEFTMOST DERIVATION? 341 ; 7.3.2 WORKING OF A TOP-DOWN PARSER 341 ; 7.3.3 SOME POTENTIAL PROBLEMS IN TOP-DOWN PARSING AND THEIR SOLUTIONS 342 ; 7.3.4 RECURSIVE DESCENT PARSING 346 ; 7.4 BOTTOM-UP PARSING 350 ; 7.4.1 WHY RIGHTMOST REDUCTION? 350 ; 7.4.2 WORKING OF A BOTTOM-UP PARSER 350 ; 7.4.3 OPERATOR PRECEDENCE PARSING 351 ; 7.5 AUTOMATIC CONSTRUCTION OF BOTTOM-UP PARSERS 352 ; 7.5.1 LR(0) GRAMMAR 352 ; 7.5.2 SLR PARSER 357 ; 7.5.3 LR(1) GRAMMAR 363 ; 7.5.4 CANONICAL-LR PARSER 365 ; 7.5.5 LALR PARSER 370 ; 8. POST MACHINE 376 ; 8.1 INTRODUCTION 376 ; 8.2 ELEMENTS OF POST MACHINE 376 ; 8.3 PUSHDOWN STACK-MEMORY MACHINE VS POST MACHINE 377 ; 8.4 PICTORIAL REPRESENTATION OF POST MACHINE ELEMENTS 378 ; 8.5 FINITE STATE MACHINE VS POST MACHINE 379 ; 8.6 POST MACHINE THAT ACCEPTS CONTEXT-FREE LANGUAGES 381 ; 8.7 NON-DETERMINISTIC POST MACHINE 390 ; 8.8 POST MACHINE THAT ACCEPTS NON-CFLS 394 ; 9. UNDECIDABILITY 405 ; 9.1 INTRODUCTION 405 ; 9.2 RECURSIVE AND RECURSIVELY ENUMERABLE LANGUAGES 405 ; 9.2.1 SOME IMPORTANT RESULTS WITH RECURSIVE AND RE LANGUAGES 406 ; 9.3 GODEL NUMBERING (OR GODEL ENCODING) 408 ; 9.3.1 ENCODING OF TURING MACHINES 408 ; 9.4 NON-RECURSIVELY ENUMERABLE LANGUAGES 409 ; 9.4.1 DIAGONALIZATION LANGUAGE 410 ; 9.4.2 LD NOT RECURSIVELY ENUMERABLE 410 ; 9.5 UNIVERSAL LANGUAGE 411 ; 9.6 REDUCIBILITY AND UNDECIDABLE PROBLEMS 411 ; 9.7 RICE'S THEOREM 412 ; 9.8 POST'S CORRESPONDENCE PROBLEM 413 ; 9.9 UNDECIDABLE PROBLEMS FOR CONTEXT-FREE GRAMMARS 413 ; 9.10 GREIBACH'S THEOREM 414 ; 9.11 HILBERT'S PROBLEM 415 ; 9.12 ACKERMANN'S FUNCTION 416 ; 10. COMPLEXITY AND CLASSIFICATION OF PROBLEMS 421 ; 10.1 INTRODUCTION 421 ; 10.2 COMPLEXITY OF A PROBLEM 421 ; 10.2.1 MATHEMATICAL NOTATIONS FOR TIME COMPLEXITY MEASURE 422 ; 10.2.2 TIME AND SPACE COMPLEXITY OF A TURING MACHINE 424 ; 10.3 CLASSIFICATION OF PROBLEMS 424 ; 10.3.1 NON-DETERMINISTIC ALGORITHM 424 ; 10.3.2 SATISFIABILITY 425 ; 10.3.3 P-TYPE AND NP-TYPE PROBLEMS 426 ; 11. PRODUCTION SYSTEMS 431 ; 11.1 INTRODUCTION 431 ; 11.2 POST-MARKOV-THUE PRODUCTION SYSTEM 432 ; 11.2.1 FORMAL DEFINITION 433 ; 11.2.2 EXAMPLES 433 ; 11.3 POST CANONICAL SYSTEM 436 ; 11.4 POST NORMAL FORM 437 ; 11.5 POST-MARKOV-THUE SYSTEM AND TURING MACHINE 437 ; 11.6 POST-MARKOV-THUE SYSTEM AND FINITE STATE MACHINE 438 ; 11.7 MARKOV ALGORITHM 440 ; 11.7.1 FORMAL DEFINITION 440 ; 11.7.2 EXAMPLES 440 ; 11.8 LABELLED MARKOV ALGORITHM 447 ; 11.8.1 FORMAL DEFINITION 448 ; 11.8.2 SOME EXAMPLES 448 ; APPENDIX A: IMPLEMENTATIONS 453 ; APPENDIX B: MODEL QUESTION PAPERS 512 ; GLOSSARY 516 ; BIBLIOGRAPHY 00 ; INDEX 00