Expert Python Programming - Second Edition

Michal Jaworski, Tarek Ziade, Tarek Ziade

• Publisher: Packt Publishing
• Year: 2016
• eBook ISBN: 9781785884399
• Edition: 2
• Topic: Computer Science
• Subtopic: Programming in Python
• Index: Computer Science

Become an ace Python programmer by learning best coding practices and advance-level concepts with Python 3.5About This Book• Based on the latest stable version of Python (version 3.5)• Creating well manageable code that will run in various environments with different sets of dependencies• Packed with advanced concepts and best practices to write efficient Python codeWho This Book Is ForThe book would appeal to web developers and Python programmers who want to start using version 3.5 and write code efficiently. Basic knowledge of Python programming is expected.What You Will Learn• Conventions and best practices that are widely adopted in the python community• Package python code effectively for community and production use• Easy and lightweight ways to automate code deployment on remote systems• Improve your code's quality, reliability, and performance• Write concurrent code in python• Extend python with code written in different languagesIn DetailPython is a dynamic programming language, used in a wide range of domains by programmers who find it simple, yet powerful. Even if you find writing Python code easy, writing code that is efficient and easy to maintain and reuse is a challenge.The focus of the book is to familiarize you with common conventions, best practices, useful tools and standards used by python professionals on a daily basis when working with code.You will begin with knowing new features in Python 3.5 and quick tricks for improving productivity. Next, you will learn advanced and useful python syntax elements brought to this new version. Using advanced object-oriented concepts and mechanisms available in python, you will learn different approaches to implement metaprogramming. You will learn to choose good names, write packages, and create standalone executables easily.You will also be using some powerful tools such as buildout and vitualenv to release and deploy the code on remote servers for production use. Moving on, you will learn to effectively create Python extensions with C, C++, cython, and pyrex. The important factors while writing code such as code management tools, writing clear documentation, and test-driven development are also covered.You will now dive deeper to make your code efficient with general rules of optimization, strategies for finding bottlenecks, and selected tools for application optimization.By the end of the book, you will be an expert in writing efficient and maintainable code.Style and approach An easy-to-follow guide that covers industry followed best practices in Python programming

Expert Python Programming Second Edition

---

Table of Contents

Expert Python Programming Second Edition

Credits

About the Authors

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Current Status of Python

Where are we now and where we are going?

Why and how does Python change?

Getting up to date with changes – PEP documents

Python 3 adoption at the time of writing this book

The main differences between Python 3 and Python 2

Why should I care?

The main syntax differences and common pitfalls

Syntax changes

Changes in the standard library

Changes in datatypes and collections

The popular tools and techniques used for maintaining cross-version compatibility

Not only CPython

Why should I care?

Stackless Python

Jython

IronPython

PyPy

Modern approaches to Python development

Application-level isolation of Python environments

Why isolation?

Popular solutions

virtualenv

venv

buildout

Which one to choose?

System-level environment isolation

Virtual development environments using Vagrant

Containerization versus virtualization

Popular productivity tools

Custom Python shells – IPython, bpython, ptpython, and so on

Setting up the PYTHONSTARTUP environment variable

IPython

bpython

ptpython

Interactive debuggers

Useful resources

Summary

2. Syntax Best Practices – below the Class Level

Python's built-in types

Strings and bytes

Implementation details

String concatenation

Collections

Lists and tuples

Implementation details

List comprehensions

Other idioms

Dictionaries

Implementation details

Weaknesses and alternatives

Sets

Implementation details

Beyond basic collections – the collections module

Advanced syntax

Iterators

The yield statement

Decorators

General syntax and possible implementations

As a function

As a class

Parametrizing decorators

Introspection preserving decorators

Usage and useful examples

Argument checking

Caching

Proxy

Context provider

Context managers – the with statement

General syntax and possible implementations

As a class

As a function – the contextlib module

Other syntax elements you may not know yet

The for … else … statement

Function annotations

The general syntax

The possible uses

Summary

3. Syntax Best Practices – above the Class Level

Subclassing built-in types

Accessing methods from superclasses

Old-style classes and super in Python 2

Understanding Python's Method Resolution Order

super pitfalls

Mixing super and explicit class calls

Heterogeneous arguments

Best practices

Advanced attribute access patterns

Descriptors

Real-life example – lazily evaluated attributes

Properties

Slots

Metaprogramming

Decorators – a method of metaprogramming

Class decorators

Using the __new__() method to override instance creation process

Metaclasses

The general syntax

New Python 3 syntax for metaclasses

Metaclass usage

Metaclass pitfalls

Some tips on code generation

exec, eval, and compile

Abstract Syntax Tree

Import hooks

Projects using code generation patterns

Falcon's compiled router

Hy

Summary

4. Choosing Good Names

PEP 8 and naming best practices

Why and when to follow PEP 8?

Beyond PEP 8 – team-specific style guidelines

Naming styles

Variables

Constants

Naming and usage

Public and private variables

Functions and methods

The private controversy

Special methods

Arguments

Properties

Classes

Modules and packages

The naming guide

Using the has or is prefix for Boolean elements

Using plurals for variables that are collections

Using explicit names for dictionaries

Avoiding generic names

Avoiding existing names

Best practices for arguments

Building arguments by iterative design

Trust the arguments and your tests

Using *args and **kwargs magic arguments carefully

Class names

Module and package names

Useful tools

Pylint

pep8 and flake8

Summary

5. Writing a Package

Creating a package

The confusing state of Python packaging tools

The current landscape of Python packaging thanks to PyPA

Tool recommendations

Project configuration

setup.py

setup.cfg

MANIFEST.in

Most important metadata

Trove classifiers

Common patterns

Automated inclusion of version string from package

README file

Managing dependencies

The custom setup command

Working with packages during development

setup.py install

Uninstalling packages

setup.py develop or pip -e

Namespace packages

Why is it useful?

PEP 420 – implicit namespace packages

Namespace packages in previous Python versions

Uploading a package

PyPI – Python Package Index

Uploading to PyPI – or other package index

.pypirc

Source packages versus built packages

sdist

bdist and wheels

Standalone executables

When are standalone executables useful?

Popular tools

PyInstaller

cx_Freeze

py2exe and py2app

Security of Python code in executable packages

Making decompilation harder

Summary

6. Deploying Code

The Twelve-Factor App

Deployment automation using Fabric

Your own package index or index mirror

PyPI mirroring

Deployment using a package

Common conventions and practices

The filesystem hierarchy

Isolation

Using process supervision tools

Application code should be run in user space

Using reverse HTTP proxies

Reloading processes gracefully

Code instrumentation and monitoring

Logging errors – sentry/raven

Monitoring system and application metrics

Dealing with application logs

Basic low-level log practices

Tools for log processing

Summary

7. Python Extensions in Other Languages

Different language means – C or C++

How do extensions in C or C++ work

Why you might want to use extensions

Improving performance in critical code sections

Integrating existing code written in different languages

Integrating third-party dynamic libraries

Creating custom datatypes

Writing extensions

Pure C extensions

A closer look at Python/C API

Calling and binding conventions

Exception handling

Releasing GIL

Reference counting

Cython

Cython as a source to source compiler

Cython as a language

Challenges

Additional complexity

Debugging

Interfacing with dynamic libraries without extensions

ctypes

Loading libraries

Calling C functions using ctypes

Passing Python functions as C callbacks

CFFI

Summary

8. Managing Code

Version control systems

Centralized systems

Distributed systems

Distributed strategies

Centralized or distributed?

Use Git if you can

Git flow and GitHub flow

Continuous development processes

Continuous integration

Testing every commit

Merge testing through CI

Matrix testing

Continuous delivery

Continuous deployment

Popular tools for continuous integration

Jenkins

Buildbot

Travis CI

GitLab CI

Choosing the right tool and common pitfalls

Problem 1 – too complex build strategies

Problem 2 – too long building ...

Table of contents

• Expert Python Programming Second Edition