![]()
Mastering Python High Performance
![]()
Table of Contents
Mastering Python High Performance
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and more
Why subscribe?
Free access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Profiling 101
What is profiling?
Event-based profiling
Statistical profiling
The importance of profiling
What can we profile?
Execution time
Where are the bottlenecks?
Memory consumption and memory leaks
The risk of premature optimization
Running time complexity
Constant time – O(1)
Linear time – O(n)
Logarithmic time – O(log n)
Linearithmic time – O(nlog n)
Factorial time – O(n!)
Quadratic time – O(n^)
Profiling best practices
Build a regression-test suite
Mind your code
Be patient
Gather as much data as you can
Preprocess your data
Visualize your data
Summary
2. The Profilers
Getting to know our new best friends: the profilers
cProfile
A note about limitations
The API provided
The Stats class
Profiling examples
Fibonacci again
Tweet stats
line_profiler
kernprof
Some things to consider about kernprof
Profiling examples
Back to Fibonacci
Inverted index
getOffsetUpToWord
getWords
list2dict
readFileContent
saveIndex
__start__
getOffsetUpToWord
getWords
list2dict
saveIndex
Summary
3. Going Visual – GUIs to Help Understand Profiler Output
KCacheGrind – pyprof2calltree
Installation
Usage
A profiling example – TweetStats
A profiling example – Inverted Index
RunSnakeRun
Installation
Usage
Profiling examples – the lowest common multiplier
A profiling example – search using the inverted index
Summary
4. Optimize Everything
Memoization / lookup tables
Performing a lookup on a list or linked list
Simple lookup on a dictionary
Binary search
Use cases for lookup tables
Usage of default arguments
List comprehension and generators
ctypes
Loading your own custom C library
Loading a system library
String concatenation
Other tips and tricks
Summary
5. Multithreading versus Multiprocessing
Parallelism versus concurrency
Multithreading
Threads
Creating a thread with the thread module
Working with the threading module
Interthread communication with events
Multiprocessing
Multiprocessing with Python
Exit status
Process pooling
Interprocess communication
Pipes
Events
Summary
6. Generic Optimization Options
PyPy
Installing PyPy
A Just-in-time compiler
Sandboxing
Optimizing for the JIT
Think of functions
Consider using cStringIO to concatenate strings
Actions that disable the JIT
Code sample
Cython
Installing Cython
Building a Cython module
Calling C functions
Solving naming conflicts
Defining types
Defining types during function definitions
A Cython example
When to define a type
Limitations
Generator expressions
Comparison of char* literals
Tuples as function arguments
Stack frames
How to choose the right option
When to go with Cython
When to go with PyPy
Summary
7. Lightning Fast Number Crunching with Numba, Parakeet, and pandas
Numba
Installation
Using Numba
Numba's code generation
Eager compilation
Other configuration settings
No GIL
NoPython mode
Running your code on the GPU
The pandas tool
Installing pandas
Using pandas for data analysis
Parakeet
Installing Parakeet
How does Parakeet work?
Summary
8. Putting It All into Practice
The problem to solve
Getting data from the Web
Postprocessing the data
The initial code base
Analyzing the code
Scraper
Analyzer
Summary
Index
![]()
Mastering Python High Performance
Copyright © 2015 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: September 2015
Production reference: 1030915
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78398-930-0
www.packtpub.com
![]()
Author
Fernando Doglio
Reviewers
Erik Allik
Mike Driscoll
Enrique Escribano
Mosudi Isiaka
Commissioning Editor
Kunal Parikh
Acquisition Editors
Vivek Anantharaman
Richard Brookes-Bland
Content Development Editors
Akashdeep Kundu
Rashmi Suvarna
Technical Editor
Vijin Boricha
Copy Editors
Relin Hedly
Karuna Narayanan
Project Coordinator
Milton Dsouza
Proofreader
Safis Editing
Indexer
Mariammal Chettiyar
Graphics
Sheetal Aute
Production Coordinator
Arvindkumar Gupta
Cover Work
Arvindkumar Gupta
![]()
Fernando Doglio has been working as a web developer for the past 10 years.
During that time, he shifted his focus to the Web and grabbed the opportunity of working with most of the leading technologies, such as PHP, Ruby on Rails, MySQL, Python, Node.js, AngularJS, AJAX, REST APIs, and so on.
In his spare time, Fernando likes to tinker and learn new things. This is why his GitHub account keeps getting new repos every month. He's also a big open source supporter and tries to win the support of new people with the help of his website, lookingforpullrequests.com.
You can reach him on Twitter at @deleteman123.
When he is not programming, he spends time with his family.
