Learning Concurrency in Python

Elliot Forbes

• Publisher: Packt Publishing
• Year: 2017
• ISBN:
  9781787283169
• Edition: 1
• Topic: Computer Science
• Subtopic: Programming in Python
• Index: Computer Science

Practically and deeply understand concurrency in Python to write efficient programsAbout This Book• Build highly efficient, robust, and concurrent applications• Work through practical examples that will help you address the challenges of writing concurrent code• Improve the overall speed of execution in multiprocessor and multicore systems and keep them highly availableWho This Book Is ForThis book is for Python developers who would like to get started with concurrent programming. Readers are expected to have a working knowledge of the Python language, as this book will build on these fundamentals concepts.What You Will Learn• Explore the concept of threading and multiprocessing in Python• Understand concurrency with threads• Manage exceptions in child threads• Handle the hardest part in a concurrent system — shared resources• Build concurrent systems with Communicating Sequential Processes (CSP)• Maintain all concurrent systems and master them• Apply reactive programming to build concurrent systems• Use GPU to solve specific problemsIn DetailPython is a very high level, general purpose language that is utilized heavily in fields such as data science and research, as well as being one of the top choices for general purpose programming for programmers around the world. It features a wide number of powerful, high and low-level libraries and frameworks that complement its delightful syntax and enable Python programmers to create.This book introduces some of the most popular libraries and frameworks and goes in-depth into how you can leverage these libraries for your own high-concurrent, highly-performant Python programs. We'll cover the fundamental concepts of concurrency needed to be able to write your own concurrent and parallel software systems in Python.The book will guide you down the path to mastering Python concurrency, giving you all the necessary hardware and theoretical knowledge. We'll cover concepts such as debugging and exception handling as well as some of the most popular libraries and frameworks that allow you to create event-driven and reactive systems.By the end of the book, you'll have learned the techniques to write incredibly efficient concurrent systems that follow best practices.Style and approachThis easy-to-follow guide teaches you new practices and techniques to optimize your code, and then moves toward more advanced ways to effectively write efficient Python code. Small and simple practical examples will help you test the concepts yourself, and you will be able to easily adapt them for any application.

Reactive Programming

While event-driven programming might revolve around events, in reactive programming, we deal purely with data. In other words, every time we receive a new piece of data, we consider this to be an event. Due to this definition, you could technically call it a branch of event-driven programming. However, due to its popularity and the differences in the way it does things, I couldn't help but put reactive programming in a chapter of its own.

In this chapter, we will dive deeper into one of the most popular libraries available in Python when it comes to reactive programming, RxPY. We'll cover in depth some of the features of this library and how we can utilize this to create our own asynchronous programs.

We'll come to terms with some of the basics necessary of RxPY to get us started:

• Dealing with observers and observables
• Lambda functions and how we can use them
• The multitude of operators and how we can chain these to achieve a desired state
• The differences between both hot and cold observables
• Multicasting

We'll also take a brief look at the PyFunctional library, and how this differs from RxPY and how we can leverage that in certain scenarios. You should note that some of these examples from the official documentation have also been covered in a video course called Reactive Python for Data Science by Thomas Nield. I highly recommend this course as Thomas covers a lot of material that I've not had a chance to in this chapter. You can find this course at http://shop.oreilly.com/product/0636920064237.do.

Basic reactive programming

Reactive programming is a paradigm that is totally unlike that of your more traditional imperative style of programming. Being aware of the strengths and weaknesses of reactive programming could help you turn software disasters into potential successes.

With reactive programming, we can totally destroy the imperative style and instead focus on representing our data as a stream of events. We can subscribe to these subsequent streams and take action upon receiving these events. This helps us simplify our system's flow that could quickly become very unwieldy and unmaintainable if we were to follow a more traditional architecture and style.

Reactive libraries take away the complexity of us having to push our events to various functions within our systems, and enable us to effectively work with data as queryable, real-time streams. We can essentially fashion programs that will run infinitely against an infinite stream of events such as constant stock quotes or social media interactions.

This reactive programming paradigm has been taken up by the likes of data scientists who may have streams of statistical data or sensory data coming in, which they have to analyze and make decisions.

Maintaining purity

In a reactive paradigm, it's important that we try to make all our transactions stateless. By enforcing stateless transactions, we essentially reduce the number of potential side-effects that could impact our program's execution.

This pure style of programming is one that functional programmers tend to live by, and it's proving to be an incredibly powerful paradigm when it comes to designing highly resilient distributed systems. Overall, it's something that I would try and follow right from the start as you develop these new systems.

ReactiveX, or RX

Reactive Extensions, or RX, first hit the scenes in around 2010 and has been heavily adopted by large tech companies such as Netflix using RxJava. It has since grown into something far bigger and more prevalent within the industry.

It comes in many different flavors for each of the different programming languages currently out there. The most popular of these are as follows:

• RxJava for Java
• RxJS for JavaScript
• RxPy for Python
• RxSwift for Swift

The full list of Rx flavors can be found at https://github.com/ReactiveX.

Reactive Extensions for Python, or RxPY as it has been condensed to, is a library for composing asynchronous and event-based programs using observable collections and LINQ-style query operators in Python. I first came across a similar version of ReactiveX when I was working with the new Angular framework, and my experience with it was great. It let me turn a web socket stream into an observable, which could subsequently be watched from within my Angular application and displayed, in real-time, in the browser.

RxPY is equally useful, and it paves the way for you to write some incredibly interesting applications while handling the underlying complexity of dealing with observers and observables.

One of the best examples I could envisage this library being used is in, say, a stock trading application. You could in theory have an API that constantly checks the price of certain stocks and in turn stream it back to your RxPY-bas...