![]()
Parallel Programming with Python
![]()
Table of Contents
Parallel Programming with Python
Credits
About the Author
Acknowledgments
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and more
Why subscribe?
Free access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Contextualizing Parallel, Concurrent, and Distributed Programming
Why use parallel programming?
Exploring common forms of parallelization
Communicating in parallel programming
Understanding shared state
Understanding message passing
Identifying parallel programming problems
Deadlock
Starvation
Race conditions
Discovering Python's parallel programming tools
The Python threading module
The Python multiprocessing module
The parallel Python module
Celery – a distributed task queue
Taking care of Python GIL
Summary
2. Designing Parallel Algorithms
The divide and conquer technique
Using data decomposition
Decomposing tasks with pipeline
Processing and mapping
Identifying independent tasks
Identifying the tasks that require data exchange
Load balance
Summary
3. Identifying a Parallelizable Problem
Obtaining the highest Fibonacci value for multiple inputs
Crawling the Web
Summary
4. Using the threading and concurrent.futures Modules
Defining threads
Advantages and disadvantages of using threads
Understanding different kinds of threads
Defining the states of a thread
Choosing between threading and _thread
Using threading to obtain the Fibonacci series term with multiple inputs
Crawling the Web using the concurrent.futures module
Summary
5. Using Multiprocessing and ProcessPoolExecutor
Understanding the concept of a process
Understanding the process model
Defining the states of a process
Implementing multiprocessing communication
Using multiprocessing.Pipe
Understanding multiprocessing.Queue
Using multiprocessing to compute Fibonacci series terms with multiple inputs
Crawling the Web using ProcessPoolExecutor
Summary
6. Utilizing Parallel Python
Understanding interprocess communication
Exploring named pipes
Using named pipes with Python
Writing in a named pipe
Reading named pipes
Discovering PP
Using PP to calculate the Fibonacci series term on SMP architecture
Using PP to make a distributed Web crawler
Summary
7. Distributing Tasks with Celery
Understanding Celery
Why use Celery?
Understanding Celery's architecture
Working with tasks
Discovering message transport (broker)
Understanding workers
Understanding result backends
Setting up the environment
Setting up the client machine
Setting up the server machine
Dispatching a simple task
Using Celery to obtain a Fibonacci series term
Defining queues by task types
Using Celery to make a distributed Web crawler
Summary
8. Doing Things Asynchronously
Understanding blocking, nonblocking, and asynchronous operations
Understanding blocking operations
Understanding nonblocking operations
Understanding asynchronous operations
Understanding event loop
Polling functions
Using event loops
Using asyncio
Understanding coroutines and futures
Using coroutine and asyncio.Future
Using asyncio.Task
Using an incompatible library with asyncio
Summary
Index
![]()
Parallel Programming with Python
Copyright © 2014 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: June 2014
Production reference: 1180614
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78328-839-7
www.packtpub.com
![]()
Author
Jan Palach
Reviewers
Cyrus Dasadia
Wei Di
Michael Galloy
Ludovic Gasc
Kamran Hussain
Bruno Torres
Commissioning Editor
Rebecca Youé
Acquisition Editor
Llewellyn Rozario
Content Development Editor
Sankalp Pawar
Technical Editors
Novina Kewalramani
Humera Shaikh
Copy Editors
Roshni Banerjee
Sarang Chari
Gladson Monteiro
Project Coordinator
Lima Danti
Proofreaders
Simran Bhogal
Maria Gould
Paul Hindle
Indexers
Mehreen Deshmukh
Rekha Nair
Tejal Soni
Priya Subramani
Graphics
Disha Haria
Abhinash Sahu
Production Coordinator
Saiprasad Kadam
Cover Work
Saiprasad Kadam
![]()
Jan Palach has been a software developer for 13 years, having worked with scientific visualization and backend for private companies using C++, Java, and Python technologies. Jan has a degree in Information Systems from Estácio de Sá University, Rio de Janeiro, Brazil, and a postgraduate degree in Software Development from Paraná State Federal Technological University. Currently, he works as a senior system analyst at a private company within the telecommunication sector implementing C++ systems; howev...
