Python Developer's Handbook

Andre Lessa is a database administrator, Web developer, and programmer with Deloitte Consulting. He has bene using Python for several years, and is the author and Webmaster of two popular Brazilian Web sites.

Introduction.
I. BASIC PROGRAMMING.

1. Introduction.


Introduction to Python.



Python!? What Is It?



Why Use Python?



Readability. It Is Simple to Get Support. Fast to Learn. Fast to Code. Reusability. Portability. Object-Oriented Programming. Overall Conclusion.



Main Technical Features.



Automatic Memory Management. Exception Handling. Rich Core Library. Web Scripting Support and Data Handling. Built-In Elements. Development Flow. Clear Syntax and a Diversity of Useful Lexical Elements. Embeddable and Extendable. Objects Distribution. Databases. GUI Application. Introspection. Third-Party Projects Integration.



Python Distribution.



System Requirements.



Installing and Configuring Python.



UNIX Environment. Macintosh Environment. Windows Environment.



Python and Other Languages.



Python Versus C/C++. Python Versus Perl. Python Versus Tcl. Python Versus Smalltalk. Python Versus Java. Conclusion.



Patches and Bugs List. PSA and the Python Consortium.



Support and Help. Python Conferences and Workshops.



Summary.

2. Language Review.


Language Review. The Shell Environment.



Command Line Interpreter.



Programs.



Indentation. Lexical Analysis. Reserved Words. Identifiers.



Built-In Data Types.



Immutable Data Types. Mutable Data Types. Numbers. Strings. True and False Logical Values.



Operators.



Augmented Assignment.



Expressions.



Built-In Functions. Sequence Functions. Object Manipulation. Mathematical/Logical Functions. Code Functions. Type Conversion.



Control Statements.



if/elif/else. for. while. break/continue.



Data Structures.



Lists. Built-In Methods. Ranges. Tuples. Dictionaries (hash tables).



Functions and Procedures.



Functions. Dynamic Namespace.



Modules and Packages.



Built-In Methods. from in Contrast to import. Releasing and Reloading Modules.



Input and Output.



Displaying Information. Formatting Operations.



File Handling.



Opening a File. Supported Methods. File Object Attributes.



Summary. Code Example.

3. Python Libraries.


Python Libraries.



The Library Reference. The Standard Library of Modules.



Python Services.



sys. types. UserDict. UserList. operator. traceback. linecache. pickle. cPickle. copy_reg. shelve. copy. marshal. imp. parser. symbol. token. keyword. tokenize. pyclbr. code. codeop. pprint. repr. py_compile. compileall. dis. new. site. user. __builtin__. __main__.



The String Group.



string. re. regex. regsub. struct fpformat. StringIO. cStringIO.



Miscellaneous.



math. cmath. random. whrandom. bisect. array. ConfigParser. fileinput. calendar. cmd. shlex.



Generic Operational System.



os. os.path. dircache. stat. statcache. statvfs. cmp cmpcache. time. sched. getpass. curses. getopt. tempfile. errno. glob. fnmatch. shutil. locale. mutex.



Optional Operational System.



signal. socket. select. thread. threading. Queue. anydbm. dumbdbm. dbhash. whichdb. bsddb. zlib. gzip. rlcompleter.



Debugger. Profiler. Internet Protocol and Support.



cgi. urllib. httplib. ftplib gopherlib. poplib. imaplib. nntplib. smtplib. telnetlib. urlparse. SocketServer. BaseHTTPServer. SimpleHTTPServer. CGIHTTPServer. asyncore.



Internet Data Handling.



sgmllib. htmllib. htmlentitydefs. xmllib. formatter. rfc822. mimetools. MimeWrite. multifile. binhex. uu. binascii. base64. xdrlib. mailcap. mimetypes. quopri. mailbox. mhlib. mimify. netrc.



Restricted Execution.



rexec. Bastion.



Multimedia.



audioop. imageop. aifc. sunau. wave. chunk. colorsys. rgbimg. imghdr. sndhdr.



Cryptographic.



md5. sha. mpz. rotor.



UNIX Specific.



posix. pwd. grp. crypt. dlmodule. dbm. gdbm. termios. TERMIOS. tty. pty. fcntl. pipes. posixfile. resource. nis. syslog. popen2. commands.



SGI IRIX Specific.



al. AL. cd. fl. FL. flp. fm. gl. DEVICE. GL. imgfile. jpeg.



Sun OS Specific.



sunaudiodev. SUNAUDIODEV.



MS Windows Specific.



msvcrt. winsound.



Macintosh Specific.



findertools. macfs. macostools.



Undocumented Modules.



Frameworks. Miscellaneous Useful Utilities. Platform Specific Modules. Multimedia. Obsolete. Extension Modules. New Modules on Python 2.0.



Summary.



Python Services. The String Group. Miscellaneous. Generic Operational System. Optional Operational System. Debugger. Profiler. Internet Protocol and Support. Internet Data Handling. Restricted Execution. Multimedia. Cryptographic. OS Specific (UNIX, SGI IRIX, SUN OS, MS Windows, and Macintosh). Undocumented Modules. New Modules in Python 2.0.

4. Exception Handling.


Exception Handling. Standard Exceptions (Getting Help from Other Modules). Raising Exceptions.



Raising an Exception to Leave the Interpreter. Raising an Exception to Leave Nested Loops. Raising String Exceptions. Instancing an Exception Class. Debugging Your Code.



Catching Exceptions.



Catching Standard Errors.



try/finally. Creating User-defined Exceptions. The Standard Exception Hierarchy. Summary. Code Examples.

5. Object-Oriented Programming.


Object-Oriented Programming. An Introduction to Python OOP.



Class Instances.



Python Classes and Instances.



Attributes of a Class. The Python Class Browser. Python Instances.



Methods Handling.



Accessing Unbounded Methods. Handling Global Class Variables. Calling Methods from Other Methods.



Special Methods.



Method Attributes. Overloading Operators.



Inheritance. Polymorphism. Encapsulation. Metaclasses. Summary. Code Examples.

II. ADVANCED PROGRAMMING.

6. Extending and Embedding Python.


Extending and Embedding Python. The Python/C API. Extending.



Creating New Extensions. Importing an Extension Module. Formatting Strings. Exporting Constants. Error Checking. Reference Counting. Building Extensions in C++.



Compiling and Linking Extension Modules.



Linking Static Extensions to the Interpreter. Linking Dynamic Extensions to the Interpreter.



SWIG-The Simple Wrapper Interface Generator. Other Wrappers. Embedding.



Implementing Callback Functions. Embedding the Python Interpreter. Embedding Python in C++. Embedding Python in Other Applications.



Summary. Code Examples.



Wrapping C Functions.

7. Objects Interfacing and Distribution.


Object Interfacing and Distribution. Interfacing Objects. Introduction to COM Objects.



The COM Specification. COM Interfaces. ADO. ActiveX.



Implementing COM Objects in Python.



The win32com Package. Talking to Windows Applications. Word and Excel. Visual Basic. Delphi.



Distributing Objects with Python.



Inter-Language Unification (ILU). CORBA Binding and Implementation. Fnorb. DCOM. OMF. Hector.



Summary. Code Examples.



Parking Lot (File parkinglot.py).

8. Working with Databases.


Working with Databases. Flat Databases.



Text Data. Binary Data-The struct Module.



DBM (Database Managers) Databases.



dbm Module. gdbm Module. bsddb Module. dbhash Module. anydbm Module. dumbdbm Module. whichdb Module.



Object Serialization and Persistent Storage.



pickle Module. cPickle Module. copy_reg Module. marshal Module. shelve Module. More Sources of Information.



The ODBC Module.



mxODBC. calldll. unixODBC. Other Interesting ODBC Web Pages.



ADO (ActiveX Data Objects). Using SQL.



SQL Mini-Tutorial. PostgreSQL Databases. MySQL Modules. The GadFly SQL Module. MetaKit Database Engine.



Python DB API.



DB-API Specification v2.0.



Summary.

9. Other Advanced Topics.


Other Advanced Topics. Manipulating Images.



Python Imaging Library. Other Modules.



Working with Sounds.



winsound Module. sndhdr Module. wave Module. aifc Module. audiodev Module.



Restricted Execution Mode.



Protecting the Application Environment.



Scientific Computing.



Numerical Extensions. Other Scientific Extensions. Computer Programming for Everybody.



Regular Expressions.



Regular Expression Functions and Object Methods.



Threads.



Python Threads. Python Thread Modules. Microthreads.



Summary. Code Examples.



HTML Parsing Tool (File: parsing.py). TV Network Audiences (File: audience.py).

III. NETWORK PROGRAMMING.

10. Basic Network Background.


Networking. Networking Concepts.



Protocols. Addresses. Sockets.



HTTP.



M2Crypto, by Ng Pheng Siong's. CTC (Cut The Crap), by Constantinos Kotsokalis. Alfajor, by Andrew Cooke. Building Web Servers. Setting Up the Client Side of the HTTP Protocol.



Accessing URLs.



The urllib Module. The urlparse Module.



FTP.



Transferring Data.



SMTP/POP3/IMAP.



Handling Email Services.



Newsgroups-Telnet and Gopher. Summary.

11. Web Development.


Web Development. Configuring Web Servers for Python/CGI Scripts.



Python in Apache. AOLserver Web Server. Microsoft IIS and PWS.



Third-Party Internet Applications.



Grail Web Browser. Zope Web Application Server. Mailman-GNU Mailing List Manager. Medusa Internet Server.



Other Applications.



BSCW. LDAP. WebLog.



Site Management Tools.



WebDAV/PyDAV. Zebra. httpd_log. Linbot. Python-Friendly Internet Solution Providers (ISPs). mxCGIPython. HTMLgen. Document Template. Persistent CGI. Webchecker. LinkChecker. FastCGI.



Summary.

12. Scripting Programming.


Web Programming. An Introduction to CGI. The cgi Module.



Functions.



Creating, Installing, and Running Your Script.



Sending Information to Python Scripts. Working with Form Fields and Parsing the Information. Security. Sessions. Data Storage. Locking. Cookies. Creating Output for a Browser. Uploading/Uploaded Files. Environment Variables. Debugging and Testing Your Script.



Python Active Scripting.



Using COM Objects. ASP and Microsoft ActiveX Scripting. Python Server Pages.



Summary.

13. Data Manipulation.


Parsing and Manipulating Data. XML Processing.



Introduction to XML. Writing an XML File. Python XML Package. xmllib. The SAX API. DOM: The Document Object Model. XSL Transformations (XSLT). XBEL-XML Bookmark Exchange Language. RPC-What Is It? Simple Object Access Protocol (SOAP). PythonPoint. Pyxie.



XML-RPC.



The Python Implementation. Working with Zope.



XDR Data Exchange Format.



xdrlib.



Handling Other Markup Languages.



sgmllib. htmllib. htmlentitydefs. formatter.



MIME Parsing and Manipulation.



rfc822. mimetools. MimeWriter. multifile. mailcap. mimetypes. base64. quopri. mailbox. mimify.



Generic Conversion Functions.



netrc. mhlib. binhex. uu. binascii.



Summary.

IV. GRAPHICAL INTERFACES.

14. Python and GUIs.


Python GUI Toolkits. The Tkinter Module. Overview of Other GUI Modules.



Pythonwin/MFC. wxPython. STDWIN. PyKDE. Wpy. PyGTK. PyOpenGL. wafepython. pyFLTK. FXPy. Motif. PyAmulet. DynWin. JPI. FORMS.



Designing a Good Interface. Summary.

15. Tkinter.


Introduction to Tcl/Tk. Tkinter.



Checking the Installation. Hello Python World.



Geometry Management.



pack(). grid(). place().



Handling Tkinter Events.



Mouse Events. Keyboard Events. Event Attributes. Event Callbacks. Protocols.



Tkinter Widgets.



Widget Standard Options. Widgets Reference. General Widget Methods.



Designing Applications. PMW-Python Mega Widgets. Tkinter Resources. Summary.

V. DEVELOPING WITH PYTHON.

16. Development Environment.


Building Python Applications. Development Strategy.



Optimizing the Code. Style Guide.



Integrated Development Environments. IDLE.



Installing and Configuring IDLE. Python Shell. File Menu. Edit Menu. Writing an IDLE Extension. Python 2.0 and IDLE.



Pythonwin.



The Pythonwin Environment. Keyboard Bindings.



Summary.

17. Development Tools.


The Development Process of Python Programs. Compiling Python.



Windows. UNIX.



Editing Code.



Emacs.



Python Scripts. Generating an Executable Python Bytecode. Interpreter. Debugging the Application.



The Base Debugger Module (bdb). The Python Debugger (pdb). Disassembling Python Bytecodes.



Profiling Python.



Python Profiler. Analyzing Profiles with the pstats Module.



Distributing Python Applications.



SqueezeTool. Python2C-The Python to C Translator. Small Python. Gordon McMillan's Installer. Distutils.



Summary.

VI. PYTHON AND JAVA.

18. JPython.


Welcome to JPython.



JPython Features. CPython Versus Jpython. JPython Resource Links.



Java Integration.



Java Certification. Java Links.



Downloading and Installing JPython.



Downloading the CPython Library. Licensing. JVMs That Support JPython.



The Interpreter. The JPython Registry.



Registry Properties. Finding the Registry File.



Creating Graphical Interfaces. Embedding.



JPython in a Java Application. Java in a JPython Application.



jpythonc. Running JPython Applets. Summary.

VII. APPENDIXES.

Appendix A. Python/C API.


Python/C API.



Include Files. Objects, Types, and Reference Counts. Exceptions. Embedding Python.



The Very High Level Layer. Reference Counting. Exception Handling. Standard Exceptions.



Deprecation of String Exceptions.



Utilities.



OS Utilities. Process Control. Importing Modules.



Abstract Objects Layer.



Object Protocol. Number Protocol. Sequence Protocol. Mapping Protocol.



Concrete Objects Layer.



Fundamental Objects. Sequence Objects. Mapping/Dictionary Objects. Numeric Objects. Other Objects.



Initialization, Finalization, and Threads.



Thread State and the Global Interpreter Lock.



Memory Management.



Memory Interface. Examples.



Defining New Object Types.



Common Object Structures. Mapping Object Structures. Number Object Structures. Sequence Object Structures. Buffer Object Structures.

Appendix B. Running Python on Specific Platforms.


Python on Win32 Systems. Python on MacOS Systems. Python on UNIX Systems. Other Platforms.



Python for OS/2. Python for Windows 3.1. Python for DOS. Python for BeOS. Python for VMS. Python for Psion. Python for Windows CE. Python for Anything Else.

Appendix C. Python Copyright Notices.


Python 2.0 License Information.



HISTORY OF THE SOFTWARE.



Python's Copyright Notice (version 1.6). Python's Copyright Notice (until version 1.5.2). Copyright Notice of the profile and pstats Modules. Copyright Notice of JPython with OROMatcher. Copyright Notice of JPython without OROMatcher.

Appendix D. Migrating to Python 2.0.


Python 1.6 or Python 2.0. Which One to Choose? New Development Process. Enhancements.



Unicode Support. List Comprehension. Strings Manipulation. Augmented Assignment. Garbage Collection. Maximum Recursion.



Expected Code Breaking.

Index.