Mastering Computer Vision with TensorFlow 2.x

Build advanced computer vision applications using machine learning and deep learning techniques

Krishnendu Kar

• Publisher: Packt Publishing
• Year: 2020
• Print ISBN: 9781838827069
• eBook ISBN: 9781838826932
• Edition: 1
• Topic: Informatik
• Subtopic: Informatik Allgemein

Apply neural network architectures to build state-of-the-art computer vision applications using the Python programming language

Key Features

• Gain a fundamental understanding of advanced computer vision and neural network models in use today
• Cover tasks such as low-level vision, image classification, and object detection
• Develop deep learning models on cloud platforms and optimize them using TensorFlow Lite and the OpenVINO toolkit

Book Description

Computer vision allows machines to gain human-level understanding to visualize, process, and analyze images and videos. This book focuses on using TensorFlow to help you learn advanced computer vision tasks such as image acquisition, processing, and analysis. You'll start with the key principles of computer vision and deep learning to build a solid foundation, before covering neural network architectures and understanding how they work rather than using them as a black box. Next, you'll explore architectures such as VGG, ResNet, Inception, R-CNN, SSD, YOLO, and MobileNet. As you advance, you'll learn to use visual search methods using transfer learning. You'll also cover advanced computer vision concepts such as semantic segmentation, image inpainting with GAN's, object tracking, video segmentation, and action recognition. Later, the book focuses on how machine learning and deep learning concepts can be used to perform tasks such as edge detection and face recognition. You'll then discover how to develop powerful neural network models on your PC and on various cloud platforms. Finally, you'll learn to perform model optimization methods to deploy models on edge devices for real-time inference. By the end of this book, you'll have a solid understanding of computer vision and be able to confidently develop models to automate tasks.

What you will learn

• Explore methods of feature extraction and image retrieval and visualize different layers of the neural network model
• Use TensorFlow for various visual search methods for real-world scenarios
• Build neural networks or adjust parameters to optimize the performance of models
• Understand TensorFlow DeepLab to perform semantic segmentation on images and DCGAN for image inpainting
• Evaluate your model and optimize and integrate it into your application to operate at scale
• Get up to speed with techniques for performing manual and automated image annotation

Who this book is for

This book is for computer vision professionals, image processing professionals, machine learning engineers and AI developers who have some knowledge of machine learning and deep learning and want to build expert-level computer vision applications. In addition to familiarity with TensorFlow, Python knowledge will be required to get started with this book.

Section 1: Introduction to Computer Vision and Neural Networks

In this section, you will develop your understanding of the theory as well as learn hands-on techniques about the application of a convolutional neural network for image processing. You will learn key concepts such as image filtering, feature maps, edge detection, the convolutional operation, activation functions, and the use of fully connected and softmax layers in relation to image classification and object detection. The chapters provide many hands-on examples of an end-to-end computer vision pipeline using TensorFlow, Keras, and OpenCV. The most important learning that you will take from these chapters is to develop an understanding and intuition behind different convolutional operations – how images are transformed through different layers of convolutional neural networks.

By the end of this section, you will be able to do the following:

• Understand how image filters transform an image (chapter 1)
• Apply various types of image filters for edge detection (chapter 1)
• Detect simple objects using OpenCV contour detection and Histogram of Oriented Gradients (HOG) (Chapter 1)
• Find the similarity between objects using Scale-invariant feature transform (SIFT), Local Binary Patterns (LBP) pattern matching, and color matching (chapters 1 and 2)
• Face detection using the OpenCV cascade detector (chapter 3)
• Input big data into a neural network from a CSV file list and parse the data to recognize columns, which can then be fed to the neural network as x and y values (chapter 3)
• Facial keypoint and facial expression recognition (chapter 3)
• Develop an annotation file for facial keypoints (chapter 3)
• Input big data into a neural network from files using the Keras data generator method (chapter 4)
• Construct your own neural network and optimize its parameters to improve accuracy (chapter 4)
• Write code to transform an image through different layers of the convolutional neural network (chapter 4)

This section comprises the following chapters:

• Chapter 1, Computer Vision and TensorFlow Fundamentals
• Chapter 2, Content Recognition Using Local Binary Pattern
• Chapter 3, Facial Detection Using OpenCV and CNN
• Chapter 4, Deep Learning on Images

Computer Vision and TensorFlow Fundamentals

Computer vision is rapidly expanding in many different applications as traditional techniques, such as image thresholding, filtering, and edge detection, have been augmented by deep learning methods. TensorFlow is a widely used, powerful machine learning tool created by Google. It has user configurable APIs available to train and build complex neural network model in your local PC or in the cloud and optimize and deploy at scale in edge devices.

In this chapter, you will gain an understanding of advanced computer vision concepts using TensorFlow. This chapter discusses the foundational concepts of computer vision and TensorFlow to prepare you for the later, more advanced chapters of this book. We will look at how to perform image hashing and filtering. Then, we will learn about various methods of feature extraction and image retrieval. Moving on, we will learn about visual search in applications, its methods, and the challenges we might face. Then, we will look at an overview of the high-level TensorFlow software and its different components and subsystems.

The topics we will be covering in this chapter are as follows:

• Detecting edges using image hashing and filtering
• Extracting features from an image
• Object detection using Contours and the HOG detector
• An overview of TensorFlow, its ecosystem, and installation

Technical requirements

If you have not done so already, install Anaconda from https://www.anaconda.com. Anaconda is a package manager for Python. You also need to install OpenCV for all of the computer vision work you will be carrying out, using pip install opencv-python. OpenCV is a library of built-in programming functions for computer vision work.

Detecting edges using image hashing and filtering

Image hashing is a method used to find similarity between images. Hashing involves modifying an input image to a fixed size of binary vector through transformation. There are different algorithms for image hashing using different transformations:

• Perpetual hash (phash): A cosine transformation
• Difference hash (dhash): The difference between adjacent pixels

After a hash transformation, images can be compared quickly with the Hamming distance. The Python code for applying a hash transformation is shown in the following code. A hamming distance of 0 shows an identical image (duplicate), whereas a larger hamming distance shows that the images are different from each other. The following snippet imports Python packages, such as PIL, imagehash, and distance. imagehash is a Python package that supports various types of hashing algorithms. PIL is a Python imaging library, and distance is a Python package that calculates the hamming distance between two hashed images:

from PIL import Image
import imagehash
import distance
import scipy.spatial
hash1 = imagehash.phash(Image.open(…/car1.png))
hash2 = imagehash.phash(Image.open(…/car2.png))
print hamming_distance(hash1,hash2)

Image filtering is a fundamental computer vision operation that modifies the input image by applying a kernel or filter to every pixel of the input image. The following are the steps involved in image filtering, starting from light entering a camera to the final transformed image:

1. Using a Bayer filter for color pattern formation
2. Creating an image vector
3. Transforming the image
4. Linear filtering—convolution with kernels
5. Mixing Gaussian and Laplacian filters
6. Detecting edges in the image

Using a Bayer filter for color pattern formation

A Bayer filter transforms a raw image into a natural, color-processed image by applying a demosaic algorithm. The image sensor consists of photodiodes, which p...

Table of contents

• Title Page
• Copyright and Credits
• About Packt
• Contributors
• Preface
• Section 1: Introduction to Computer Vision and Neural Networks
• Computer Vision and TensorFlow Fundamentals
• Content Recognition Using Local Binary Patterns
• Facial Detection Using OpenCV and CNN
• Deep Learning on Images
• Section 2: Advanced Concepts of Computer Vision with TensorFlow
• Neural Network Architecture and Models
• Visual Search Using Transfer Learning
• Object Detection Using YOLO
• Semantic Segmentation and Neural Style Transfer
• Section 3: Advanced Implementation of Computer Vision with TensorFlow
• Action Recognition Using Multitask Deep Learning
• Object Detection Using R-CNN, SSD, and R-FCN
• Section 4: TensorFlow Implementation at the Edge and on the Cloud
• Deep Learning on Edge Devices with CPU/GPU Optimization
• Cloud Computing Platform for Computer Vision
• Other Books You May Enjoy