Hands-On Computer Vision with Julia

Build complex applications with advanced Julia packages for image processing, neural networks, and Artificial Intelligence

Dmitrijs Cudihins

• Publisher: Packt Publishing
• Year: 2018
• Edition: 1
• eBook ISBN:
  9781788999236
• Topic: Informatique
• Subtopic: Vision par ordinateur et reconnaissance de formes

Explore the various packages in Julia that support image processing and build neural networks for video processing and object tracking.About This Book• Build a full-fledged image processing application using JuliaImages • Perform basic to advanced image and video stream processing with Julia's APIs• Understand and optimize various features of OpenCV with easy examplesWho This Book Is ForHands-On Computer Vision with Julia is for Julia developers who are interested in learning how to perform image processing and want to explore the field of computer vision. Basic knowledge of Julia will help you understand the concepts more effectively.What You Will Learn• Analyze image metadata and identify critical data using JuliaImages• Apply filters and improve image quality and color schemes• Extract 2D features for image comparison using JuliaFeatures• Cluster and classify images with KNN/SVM machine learning algorithms• Recognize text in an image using the Tesseract library• Use OpenCV to recognize specific objects or faces in images and videos• Build neural network and classify images with MXNetIn DetailHands-On Computer Vision with Julia is a thorough guide for developers who want to get started with building computer vision applications using Julia. Julia is well suited to image processing because it's easy to use and lets you write easy-to-compile and efficient machine code.This book begins by introducing you to Julia's image processing libraries such as Images.jl and ImageCore.jl. You'll get to grips with analyzing and transforming images using JuliaImages; some of the techniques discussed include enhancing and adjusting images. As you make your way through the chapters, you'll learn how to classify images, cluster them, and apply neural networks to solve computer vision problems. In the concluding chapters, you will explore OpenCV applications to perform real-time computer vision analysis, for example, face detection and object tracking. You will also understand Julia's interaction with Tesseract to perform optical character recognition and build an application that brings together all the techniques we introduced previously to consolidate the concepts learned.By end of the book, you will have understood how to utilize various Julia packages and a few open source libraries such as Tesseract and OpenCV to solve computer vision problems with ease.Style and approachReaders will be taken through various packages that support image processing in Julia, and will also tap into open-source libraries such as Open CV and Tesseract to find the optimum solution to problems encountered in computer vision.

Introduction to Neural Networks

This chapter will primarily focus on understanding the difference between classic computer vision and neural networks, and also shows how to prepare images for use in deep learning models, and assist in building and training your first classifiers.

The following topics are covered in this chapter:

• Introduction to MXNet and its building blocks
• Building a digit classifier for the MNIST dataset
• Building a multiclass image classifier with CIFAR-10
• Building the cats and dogs classifier
• Reusing the models

Technical requirements

Users are required to have prior knowledge of Julia, but no previous knowledge of math or statistics is required.

Users are required to have the MLDatasets.jl, MXNet, and MXNet.jl packages installed. Please follow the guide from MXNet which is available at https://mxnet.incubator.apache.org/install/windows_setup.html#install-the-mxnet-package-for-julia .

In short, the user is required to configure MXNet and set an environmental variable before installing the MXNet.jl package.

GitHub repository with source code: https://github.com/PacktPublishing/Hands-On-Computer-Vision-with-Julia/tree/master/Chapter06

Introduction

You may have already seen that running corner detection on a simple image can return thousands of different features, and precisely mapping them with another image could be an issue when using classic computer vision techniques.

We need a tool that can express a human-like behavior to solve this problem quickly and efficiently, and that is where deep learning and neural networks step in.

Deep learning has become one of the hottest topics in the industry. Indeed, neural networks have proven to solve complex problems efficiently, such as image recognition or natural language speech. Luckily for us, we have a solution in Julia to follow the hype!

Let's jump into a quick introduction to neural networks and MXNet and quickly proceed to the programming part.

The need for neural networks

There are millions of use cases requiring that customers build their deep learning application. For a long time, this was a very complicated task involving arcane knowledge and tools which only expert scientists could master.

Remember the process of removing noise from the image by using erosion? Depending on complexity, you would be applying it again and again until the image was clean and ready. The neural network could handle this automatically just from training from examples.

Imagine that you have to improve a face-recognition example and match two and tens of other photos from different angles. Your task would be primarily focused on adjusting the threshold and verifying them, again and again, by adjusting the thresholds as you do this.

Today, a simple neural network can replace a computer vision expert in the following areas:

• Image classification: This is identifying whether there are cats or dogs in an image
• Image segmentation: This is separating an image into multiple segments
• Image clustering: This is grouping similar images together
• Image captioning: This describes the content of an image in the text
• Object detection and tracking: This identifies an object in an image and follows it when it moves

The most amazing thing is that we will cover all of these in this and the subsequent chapter!

The need for MXNet

The benefits of using MXNet over the other packages supporting neural networks such as KNet.jl, Flux.jl, or Tensorflow.jl are the following:

• Requires little experience in both Julia and deep learning
• A large number of pre-trained models and weights
• Has GPU support
• One of the fastest frameworks out there on the market
• Backed by Amazon and Microsoft
• Lastly, it is actively used in production

All of these benefits offer you a privilege so that you can develop a high-performance model in Julia and any time in the process port it to C++, Python, or R and run it in AWS, Azure, or Google.

First steps with the MNIST dataset

MNIST is the dataset that is always discussed first when making first steps in the world of neural networks and image classification. MNIST is a database of grayscale images of handwritten digits. It has a training set of 60,000 examples, and a test set of 10,000 examples.

In the following activities, we will be predicting the value written on an image by building our first neural network.

Getting the data

In order to get to the process of building the neural networks quickly, we will be using the MNIST dataset, which is available in the MLDatasets.jl package. The package provides easy and user-friendly access to some of the datasets publicly available out there on the internet. If you don't have the MLDatasets package installed, you can do so by running the Pkg.add command:

Pkg.add("MLDatasets")

The moment the MLDatasets package is loaded, the MNIST dataset can be easily downloaded and made available with the traindata and testdata functions from the MNIST module:

using Images, ImageView, MLDatasets
train_x, train_y = MNIST.traindata()
test_x, test_y = MNIST.testdata()

The first time you make a call to any of the MLDatasets modules, it will present you with terms of service (TOS) and offer to download the data to your local machine. Please be aware that depending on a dataset's type, the size can vary and take over 100 MB.

As you can see, both functions return tuples, such as train_x and train_y. train_x corresponds to images of data and train_y to the value in an image. The neural network will use the data from train_x to train and predict the value of train_y.

Next, we will create a preview of a set of the first 10 images from the training dataset. Sometimes, it helps to identify issues with data. This is shown in the following code:

preview_img = zeros(size(train_x, 1), 0)

for i = 1:10
 preview_img = hcat(preview_img, train_x[:, :, i])
end

imshow(Gray.(preview_img))

We have used hcat to join the images together and imshow to preview the result:

In order to see what the corresponding values are for each image, we printed out the first 10 values from the train_y dataset. This is shown as follows:

train_y[1:10]
Main> 5, 0, 4, 1, 9, 2, 1, 3, 1, 4

Now that we have seen the data, it is time to move on to creating our first neural network.

Preparing the data

What could be the requirements for our first neural network? It should accept our images as an input, do some magic, and return probabilities for the image corresponding to one of 10 classes—that is, if the num...

Table of contents

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Getting Started with JuliaImages
• Image Enhancement
• Image Adjustment
• Image Segmentation
• Image Representation
• Introduction to Neural Networks
• Using Pre-Trained Neural Networks
• OpenCV
• Assessments
• Other Books You May Enjoy