OpenCV with Python Blueprints

Michael Beyeler

• Publisher: Packt Publishing
• Year: 2015
• eBook ISBN: 9781785282690
• Edition: 1
• Topic: Computer Science
• Subtopic: Programming Languages
• Index: Computer Science

Design and develop advanced computer vision projects using OpenCV with Python

About This Book

• Program advanced computer vision applications in Python using different features of the OpenCV library
• Practical end-to-end project covering an important computer vision problem
• All projects in the book include a step-by-step guide to create computer vision applications

Who This Book Is For

This book is for intermediate users of OpenCV who aim to master their skills by developing advanced practical applications. Readers are expected to be familiar with OpenCV's concepts and Python libraries. Basic knowledge of Python programming is expected and assumed.

What You Will Learn

• Generate real-time visual effects using different filters and image manipulation techniques such as dodging and burning
• Recognize hand gestures in real time and perform hand-shape analysis based on the output of a Microsoft Kinect sensor
• Learn feature extraction and feature matching for tracking arbitrary objects of interest
• Reconstruct a 3D real-world scene from 2D camera motion and common camera reprojection techniques
• Track visually salient objects by searching for and focusing on important regions of an image
• Detect faces using a cascade classifier and recognize emotional expressions in human faces using multi-layer peceptrons (MLPs)
• Recognize street signs using a multi-class adaptation of support vector machines (SVMs)
• Strengthen your OpenCV2 skills and learn how to use new OpenCV3 features

In Detail

OpenCV is a native cross platform C++ Library for computer vision, machine learning, and image processing. It is increasingly being adopted in Python for development. OpenCV has C++/C, Python, and Java interfaces with support for Windows, Linux, Mac, iOS, and Android. Developers using OpenCV build applications to process visual data; this can include live streaming data from a device like a camera, such as photographs or videos. OpenCV offers extensive libraries with over 500 functions

This book demonstrates how to develop a series of intermediate to advanced projects using OpenCV and Python, rather than teaching the core concepts of OpenCV in theoretical lessons. Instead, the working projects developed in this book teach the reader how to apply their theoretical knowledge to topics such as image manipulation, augmented reality, object tracking, 3D scene reconstruction, statistical learning, and object categorization.

By the end of this book, readers will be OpenCV experts whose newly gained experience allows them to develop their own advanced computer vision applications.

Style and approach

This book covers independent hands-on projects that teach important computer vision concepts like image processing and machine learning for OpenCV with multiple examples.

OpenCV with Python Blueprints

---

Table of Contents

OpenCV with Python Blueprints

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Fun with Filters

Planning the app

Creating a black-and-white pencil sketch

Implementing dodging and burning in OpenCV

Pencil sketch transformation

Generating a warming/cooling filter

Color manipulation via curve shifting

Implementing a curve filter by using lookup tables

Designing the warming/cooling effect

Cartoonizing an image

Using a bilateral filter for edge-aware smoothing

Detecting and emphasizing prominent edges

Combining colors and outlines to produce a cartoon

Putting it all together

Running the app

The GUI base class

The GUI constructor

Handling video streams

A basic GUI layout

A custom filter layout

Summary

2. Hand Gesture Recognition Using a Kinect Depth Sensor

Planning the app

Setting up the app

Accessing the Kinect 3D sensor

Running the app

The Kinect GUI

Tracking hand gestures in real time

Hand region segmentation

Finding the most prominent depth of the image center region

Applying morphological closing to smoothen the segmentation mask

Finding connected components in a segmentation mask

Hand shape analysis

Determining the contour of the segmented hand region

Finding the convex hull of a contour area

Finding the convexity defects of a convex hull

Hand gesture recognition

Distinguishing between different causes of convexity defects

Classifying hand gestures based on the number of extended fingers

Summary

3. Finding Objects via Feature Matching and Perspective Transforms

Tasks performed by the app

Planning the app

Setting up the app

Running the app

The FeatureMatching GUI

The process flow

Feature extraction

Feature detection

Detecting features in an image with SURF

Feature matching

Matching features across images with FLANN

The ratio test for outlier removal

Visualizing feature matches

Homography estimation

Warping the image

Feature tracking

Early outlier detection and rejection

Seeing the algorithm in action

Summary

4. 3D Scene Reconstruction Using Structure from Motion

Planning the app

Camera calibration

The pinhole camera model

Estimating the intrinsic camera parameters

The camera calibration GUI

Initializing the algorithm

Collecting image and object points

Finding the camera matrix

Setting up the app

The main function routine

The SceneReconstruction3D class

Estimating the camera motion from a pair of images

Point matching using rich feature descriptors

Point matching using optic flow

Finding the camera matrices

Image rectification

Reconstructing the scene

3D point cloud visualization

Summary

5. Tracking Visually Salient Objects

Planning the app

Setting up the app

The main function routine

The Saliency class

The MultiObjectTracker class

Visual saliency

Fourier analysis

Natural scene statistics

Generating a Saliency map with the spectral residual approach

Detecting proto-objects in a scene

Mean-shift tracking

Automatically tracking all players on a soccer field

Extracting bounding boxes for proto-objects

Setting up the necessary bookkeeping for mean-shift tracking

Tracking objects with the mean-shift algorithm

Putting it all together

Summary

6. Learning to Recognize Traffic Signs

Planning the app

Supervised learning

The training procedure

The testing procedure

A classifier base class

The GTSRB dataset

Parsing the dataset

Feature extraction

Common preprocessing

Grayscale features

Color spaces

Speeded Up Robust Features

Histogram of Oriented Gradients

Support Vector Machine

Using SVMs for Multi-class classification

Training the SVM

Testing the SVM

Confusion matrix

Accuracy

Precision

Recall

Putting it all together

Summary

7. Learning to Recognize Emotions on Faces

Planning the app

Face detection

Haar-based cascade classifiers

Pre-trained cascade classifiers

Using a pre-trained cascade classifier

The FaceDetector class

Detecting faces in grayscale images

Preprocessing detected faces

Facial expression recognition

Assembling a training set

Running the screen capture

The GUI constructor

The GUI layout

Processing the current frame

Adding a training sample to the training set

Dumping the complete training set to a file

Feature extraction

Preprocessing the dataset

Principal component analysis

Multi-layer perceptrons

The perceptron

Deep architectures

An MLP for facial expression recognition

Training the MLP

Testing the MLP

Running the script

Putting it all together

Summary

Index

OpenCV with Python Blueprints

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: October 2015

Production reference: 1141015

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78528-269-0

www.packtpub.com

Credits

Author

Michael Beyeler

Reviewers

Jia-Shen Boon

Florian LE BOURDAIS

Steve Goldsmith

Rahul Kavi

Scott Lobdell

Vipul Sharma

Commissioning Editor

Akram Hussain

Acquisition Editor

Divya Poojari

Content Development Editor

Zeeyan Pinheiro

Technical Editor

Namrata Patil

Copy Editor

Vikrant Phadke

Project Coordinator

Suzanne Coutinho

Proofreader

Safis Editing

Indexer

Rekha Nair

Production Coordinator

Melwyn D'sa

Cover Work

Melwyn D'sa

About the Author

Michael Beyeler is a PhD candidate in the department of computer science at the University of California, Irvine, where he is working on computational models of the brain as well as th...

Table of contents

• OpenCV with Python Blueprints