Mastering Python

Rick van Hattem

• Publisher: Packt Publishing
• Year: 2022
• eBook ISBN: 9781800202108
• Edition: 2
• Topic: Computer Science
• Subtopic: Open Source Programming
• Index: Computer Science

Use advanced features of Python to write high-quality, readable code and packagesKey Features• Extensively updated for Python 3.10 with new chapters on design patterns, scientific programming, machine learning, and interactive Python• Shape your scripts using key concepts like concurrency, performance optimization, asyncio, and multiprocessing• Learn how advanced Python features fit together to produce maintainable codeBook DescriptionEven if you find writing Python code easy, writing code that is efficient, maintainable, and reusable is not so straightforward. Many of Python's capabilities are underutilized even by more experienced programmers. Mastering Python, Second Edition, is an authoritative guide to understanding advanced Python programming so you can write the highest quality code. This new edition has been extensively revised and updated with exercises, four new chapters and updates up to Python 3.10. Revisit important basics, including Pythonic style and syntax and functional programming. Avoid common mistakes made by programmers of all experience levels. Make smart decisions about the best testing and debugging tools to use, optimize your code's performance across multiple machines and Python versions, and deploy often-forgotten Python features to your advantage. Get fully up to speed with asyncio and stretch the language even further by accessing C functions with simple Python calls. Finally, turn your new-and-improved code into packages and share them with the wider Python community. If you are a Python programmer wanting to improve your code quality and readability, this Python book will make you confident in writing high-quality scripts and taking on bigger challengesWhat you will learn• Write beautiful Pythonic code and avoid common Python coding mistakes• Apply the power of decorators, generators, coroutines, and metaclasses• Use different testing systems like pytest, unittest, and doctest• Track and optimize application performance for both memory and CPU usage• Debug your applications with PDB, Werkzeug, and faulthandler• Improve your performance through asyncio, multiprocessing, and distributed computing• Explore popular libraries like Dask, NumPy, SciPy, pandas, TensorFlow, and scikit-learn• Extend Python's capabilities with C/C++ libraries and system callsWho this book is forThis book will benefit more experienced Python programmers who wish to upskill, serving as a reference for best practices and some of the more intricate Python techniques. Even if you have been using Python for years, chances are that you haven't yet encountered every topic discussed in this book. A good understanding of Python programming is necessary

10

Testing and Logging – Preparing for Bugs

When programming, most developers plan a bit and immediately start writing code. After all, we all expect to write bug-free code! Unfortunately, we don’t. At some point, an incorrect assumption, a misinterpretation, or just a silly mistake is bound to happen. Debugging (covered in Chapter 11, Debugging – Solving the Bugs) will always be required at some point, but there are several methods that you can use to prevent bugs or, at the very least, make it much easier to solve them when they do occur.

To prevent bugs from occurring in the first place, test-driven development or, at the very least, functional/regression/unit tests, are very useful. The standard Python installation alone offers several options such as the doctest, unittest, and test modules. The doctest module allows you to combine tests with example documentation. The unittest module allows you to easily write regression tests. The test module is meant for internal usage only, so unless you are planning to modify the Python core, you probably won’t need this one.

The test modules we will discuss in this chapter are:

• doctest
• py.test (and why it’s more convenient than unittest)
• unittest.mock

The py.test module has roughly the same purpose as the unittest module, but it’s much more convenient to use and has many more options and plugins available.

After learning how to avoid the bugs, it’ll be time to take a look at logging so that we can inspect what is happening in our program and why. The logging module in Python is highly configurable and can be adjusted for just about any use case. If you’ve ever written Java code, you should feel right at home with the logging module, as its design is largely based on the log4j module and is very similar in both implementation and naming. The latter makes it a bit of an odd module in Python as well, as it is one of the few modules that does not follow the pep8 naming standards.

This chapter will explain the following topics:

• Combining documentation with tests using doctest
• Regression and unit tests using py.test and unittest
• Testing with fake objects using unittest.mock
• Testing multiple environments using tox
• Using the logging module effectively
• Combining logging and py.test

Using documentation as tests with doctest

The doctest module is one of the most useful modules within Python. It allows you to combine documenting your code with tests to make sure that it keeps working as it is supposed to.

By now the format should be very familiar to you; most of the code samples in this book use the doctest format, which offers the advantage that both the input and the output are shown intertwined. Especially in demonstrations, this is much more convenient than having a block of code followed by the output.

A simple doctest example

Let’s start with a quick example: a function that squares the input. The following example is a fully functional command-line application, containing not only code but also functioning tests. The first few tests cover how the function is supposed to behave when executing normally, followed by a few tests to demonstrate the expected errors:

def square(n: int) -> int: '''  Returns the input number, squared  >>> square(0)  0  >>> square(1)  1  >>> square(2)  4  >>> square(3)  9  >>> square()  Traceback (most recent call last):  ...  TypeError: square() missing 1 required positional argument: 'n'  >>> square('x')  Traceback (most recent call last):  ...  TypeError: can't multiply sequence by non-int of type 'str'  Args:  n (int): The number to square  Returns:  int: The squared result  ''' return n * n if __name__ == '__main__': import doctest doctest.testmod() 

It can be executed as any Python script, but the regular...

Table of contents

• Preface
• Getting Started – One Environment per Project
• Interactive Python Interpreters
• Pythonic Syntax and Common Pitfalls
• Pythonic Design Patterns
• Functional Programming – Readability Versus Brevity
• Decorators – Enabling Code Reuse by Decorating
• Generators and Coroutines – Infinity, One Step at a Time
• Metaclasses – Making Classes (Not Instances) Smarter
• Documentation – How to Use Sphinx and reStructuredText
• Testing and Logging – Preparing for Bugs
• Debugging – Solving the Bugs
• Performance – Tracking and Reducing Your Memory and CPU Usage
• asyncio – Multithreading without Threads
• Multiprocessing – When a Single CPU Core Is Not Enough
• Scientific Python and Plotting
• Artificial Intelligence
• Extensions in C/C++, System Calls, and C/C++ Libraries
• Packaging – Creating Your Own Libraries or Applications
• Other Books You May Enjoy
• Index