Hands-On GPU-Accelerated Computer Vision with OpenCV and CUDA

Effective techniques for processing complex image data in real time using GPUs

Bhaumik Vaidya

• Publisher: Packt Publishing
• Year: 2018
• ISBN:
  9781789343687
• Edition: 1
• Topic: Computer Science
• Subtopic: Parallel Programming
• Index: Computer Science

Discover how CUDA allows OpenCV to handle complex and rapidly growing image data processing in computer and machine vision by accessing the power of GPU

Key Features

• Explore examples to leverage the GPU processing power with OpenCV and CUDA
• Enhance the performance of algorithms on embedded hardware platforms
• Discover C++ and Python libraries for GPU acceleration

Book Description

Computer vision has been revolutionizing a wide range of industries, and OpenCV is the most widely chosen tool for computer vision with its ability to work in multiple programming languages. Nowadays, in computer vision, there is a need to process large images in real time, which is difficult to handle for OpenCV on its own. This is where CUDA comes into the picture, allowing OpenCV to leverage powerful NVDIA GPUs. This book provides a detailed overview of integrating OpenCV with CUDA for practical applications.

To start with, you'll understand GPU programming with CUDA, an essential aspect for computer vision developers who have never worked with GPUs. You'll then move on to exploring OpenCV acceleration with GPUs and CUDA by walking through some practical examples.

Once you have got to grips with the core concepts, you'll familiarize yourself with deploying OpenCV applications on NVIDIA Jetson TX1, which is popular for computer vision and deep learning applications. The last chapters of the book explain PyCUDA, a Python library that leverages the power of CUDA and GPUs for accelerations and can be used by computer vision developers who use OpenCV with Python.

By the end of this book, you'll have enhanced computer vision applications with the help of this book's hands-on approach.

What you will learn

• Understand how to access GPU device properties and capabilities from CUDA programs
• Learn how to accelerate searching and sorting algorithms
• Detect shapes such as lines and circles in images
• Explore object tracking and detection with algorithms
• Process videos using different video analysis techniques in Jetson TX1
• Access GPU device properties from the PyCUDA program
• Understand how kernel execution works

Who this book is for

This book is a go-to guide for you if you are a developer working with OpenCV and want to learn how to process more complex image data by exploiting GPU processing. A thorough understanding of computer vision concepts and programming languages such as C++ or Python is expected.

Getting Started with OpenCV with CUDA Support

So far, we have seen all the concepts related to parallel programming using CUDA and how it can leverage the GPU for acceleration. From this chapter on, we will try to use the concept of parallel programming in CUDA for computer vision applications. Though we have worked on matrices, we have not worked on actual images. Basically, working on images is similar to manipulation of two-dimensional matrices. We will not develop the entire code from scratch for computer vision applications in CUDA, but we will use the popular computer vision library that is called OpenCV. Though this book assumes that the reader has some familiarity with working with OpenCV, this chapter revises the concepts of using OpenCV in C++. This chapter describes the installation of the OpenCV library with CUDA support on Windows and Ubuntu. Then it describes how to test this installation and run a simple program. This chapter describes the use of OpenCV in working with images and videos by developing simple codes for it. This chapter also compares the performance of a program with CUDA support to one without it.

The following topics will be covered in this chapter:

• Introduction to image processing and computer vision
• Introduction to OpenCV with CUDA support
• Installation of OpenCV with CUDA support on Windows and Ubuntu
• Working with images using OpenCV
• Working with videos using OpenCV
• Arithmetic and logical operations on images
• Color-space conversions and image thresholding
• Performance comparison between CPU and GPU OpenCV programs

Technical requirements

This chapter requires a basic understanding of image processing and computer vision. It needs familiarity with the basic C or C++ programming language and all the code samples explained in previous chapters. All of the code used in this chapter can be downloaded from the following GitHub link: https://github.com/PacktPublishing/Hands-On-GPU-Accelerated-Computer-Vision-with-OpenCV-and-CUDA. The code can be executed on any operating system, though it has only been tested on Ubuntu 16.04.

Check out the following video to see the Code in Action:
http://bit.ly/2xF5cQV

Introduction to image processing and computer vision

The volume of image and video data available in the world is increasing day by day. The increasing use of mobile devices to capture images and use of the internet to post them has allowed production of enormous amounts of video and image data every day. Image processing and computer vision are used in many applications across various domains. Doctors use MRI and X-ray images for medical diagnoses. Space scientists and chemical engineers use images for space exploration and analysis of various genes at the molecular level. Images can be used to develop autonomous vehicles and video surveillance applications. They can also be used in agricultural applications and to identify faulty products during manufacturing. All these applications need to process images on a computer at a high speed. We are not going to look at how images are captured by a camera sensor and converted into digital images for computer storage. In this book, we will only cover the processing of an image on a computer, where we assume it is already stored.

Many people use the terms image processing and computer vision interchangeably. However, there is a difference between these two fields. Image processing is concerned with improving the visual quality of images by modifying pixel values, whereas computer vision is concerned with extracting important information from the images. So in image processing, both input and output are images, while in computer vision, input is an image but output is the information extracted from that image. Both have a wide variety of applications, but image processing is mainly used at the pre-processing stage of computer vision applications.

An image is stored as a multidimensional matrix. So processing an image on a computer is nothing more than manipulating this matrix. We saw how to work with matrices in CUDA in previous chapters. The code for reading, manipulating, and displaying images in CUDA might get very long, tedious, and hard to debug. So we will use a library that contains APIs for all of these functions and which can leverage the advantage of CUDA-GPU acceleration for processing images. This library is called OpenCV, which is an acronym for Open Computer Vision. In the next section, this library is explained in detail.

Introduction to OpenCV

OpenCV is a computer vision library developed with computational efficiency in mind and keeping the focus on real-time performance. It is written in C/C++ and it contains more than a hundred functions that help in computer vision applications. The main advantage of OpenCV is that it is open source and released under a Berkley software distribution (BSD) license which allows free use of OpenCV in research and commercial applications. This library has an interface in C, C++, Java, and Python languages and it can be used in all operating systems, such as Windows, Linux, macOS, and Android, without modifying a single line of code.

This library can also take advantage of multi-core processing and OpenGL and CUDA for parallel processing. As OpenCV is lightweight, it can be used on embedded platforms such as Raspberry Pi as well. This makes it ideal for deploying computer vision applications on embedded systems in real-life scenarios. We are going to explore this in the next few chapters. These features have made OpenCV a default choice for computer vision developers. It has a wide developer base and user community that helps constantly in improving the library. The downloads for OpenCV are in the millions and increasing day by day. The other popular computer vision and image processing tool is MATLAB, so you might wonder what are the advantages of using OpenCV over MATLAB. The following table shows a comparison between these two tools:

Parameter	OpenCV	MATLAB
Program speed	Higher because it is developed using C/C++	Lower than OpenCV
Resources needed	OpenCV is a lightweight library so it consumes very little memory both in terms of hard disk and RAM. A normal OpenCV program will require less than 100MB RAM.	MATLAB is very bulky. The latest MATLAB version installation can consume more than 15 GB space on the hard disk and a large chunk of RAM (more than 1 GB) when it is in use.
Portability	OpenCV can run on all operating systems that can run C language.	MATLAB can only run on Windows, Linux, and MAC.
Cost	The use of OpenCV in commercial or academic applications is completely free.	MATLAB is a licensed software so you have to pay a large amount to use it in your academic or commercial applications.
Ease of use	OpenCV is comparatively difficult to use as it has less documentation and difficult to remember syntax. It also does not have its own development environment.	MATLAB has its own integrated development environment with built-in help resources, which makes it easy for a new programmer to use.

MATLAB and OpenCV both have their pros and cons. But when we want to use computer vision in embedded applications and take advantage of parallel processing, OpenCV is the ideal choice. So, in this book, OpenCV is described for accelerating computer vision applications using GPU and CUDA. OpenCV has APIs in C, C++, Python, and Java. It is written in C/C++ so API in those languages will be the fastest. Moreover, CUDA acceleration is more supported in C/C++ API, so, in this book, we will use OpenCV with C/C++ API. In the next section, we will see how to install OpenCV on various operating systems.

Installation of OpenCV with CUDA support

Installation of OpenCV with CUDA is not as trivial as you might think. It involves many steps. In this section, all the steps for installing OpenCV in Windows and Ubuntu are explained with screens...