Artificial Vision and Language Processing for Robotics

Create end-to-end systems that can power robots with artificial vision and deep learning techniques

Álvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre

• Publisher: Packt Publishing
• Year: 2019
• Print ISBN:
  9781838552268
• Edition: 1
• eBook ISBN:
  9781838557669
• Topic: Computer Science
• Subtopic: Computer Vision & Pattern Recognition
• Index: Computer Science

Create end-to-end systems that can power robots with artificial vision and deep learning techniques

Key Features

• Study ROS, the main development framework for robotics, in detail
• Learn all about convolutional neural networks, recurrent neural networks, and robotics
• Create a chatbot to interact with the robot

Book Description

Artificial Vision and Language Processing for Robotics begins by discussing the theory behind robots. You'll compare different methods used to work with robots and explore computer vision, its algorithms, and limits. You'll then learn how to control the robot with natural language processing commands. You'll study Word2Vec and GloVe embedding techniques, non-numeric data, recurrent neural network (RNNs), and their advanced models. You'll create a simple Word2Vec model with Keras, as well as build a convolutional neural network (CNN) and improve it with data augmentation and transfer learning. You'll study the ROS and build a conversational agent to manage your robot. You'll also integrate your agent with the ROS and convert an image to text and text to speech. You'll learn to build an object recognition system using a video.

By the end of this book, you'll have the skills you need to build a functional application that can integrate with a ROS to extract useful information about your environment.

What you will learn

• Explore the ROS and build a basic robotic system
• Understand the architecture of neural networks
• Identify conversation intents with NLP techniques
• Learn and use the embedding with Word2Vec and GloVe
• Build a basic CNN and improve it using generative models
• Use deep learning to implement artificial intelligence(AI)and object recognition
• Develop a simple object recognition system using CNNs
• Integrate AI with ROS to enable your robot to recognize objects

Who this book is for

Artificial Vision and Language Processing for Robotics is for robotics engineers who want to learn how to integrate computer vision and deep learning techniques to create complete robotic systems. It will prove beneficial to you if you have working knowledge of Python and a background in deep learning. Knowledge of the ROS is a plus.

Chapter 1

Fundamentals of Robotics

Learning Objectives

By the end of this chapter, you will be able to:

• Describe important events in the history of robotics
• Explain the importance of using artificial intelligence, artificial vision and natural language processing
• Classify a robot depending on its goal or function
• Identify the parts of a robot
• Estimate a robot’s position using odometry

This chapter covers the brief history of robotics, classifies different types of robots and its hardware, and explains a way to find a robot's position using odometry.

Introduction

The robotics sector represents the present and the future of humanity. Currently, there are robots in the industrial sector, in research laboratories, in universities, and even in our homes. The discipline of robotics is continually evolving, which is one of the reasons it is worth studying. Every robot needs someone to program it. Even those based on AI and self-learning need to be given initial goals. Malfunctioning robots need technicians and constant maintenance, and AI-based systems need constant data inputs and monitoring to be effective.

In this book, you will learn and practice lots of interesting techniques, focusing on artificial computer vision, natural language processing, and working with robots and simulators. This will give you a solid basis in some cutting-edge areas of robotics.

History of Robotics

Robotics stemmed from the need to create intelligent machines to perform tasks that were difficult for humans. But it wasn’t called “robotics” at first. The term “robot” was coined by a Czech writer, Karel Čapek, in his work R.U.R. (Rossum’s Universal Robots). It is derived from the Czech word robota, which means servitude and is related to forced labor.

Čapek’s work became known worldwide, and the term “robot” did too, so much so that this term was later used by the famous teacher and writer Isaac Asimov in his work; he termed robotics as the science that studies robots and their features.

Here you can see a timeline of the important events that have shaped the history of robotics:

Figure 1.1: History of robotics

Figure 1.2: History of robotics continued

Figures 1.1 and 1.2 give a useful timeline of the beginnings and evolution of robotics.

Artificial Intelligence

AI refers to a set of algorithms developed with the objective of giving a machine the same capabilities as that of a human. It allows a robot to take its own decisions, interact with people, and recognize objects. This kind of intelligence is present not just in robots, but also in plenty of other applications and systems (even though people may be unaware of it).

There are many real-world products already using this kind of technology. Here’s a list of some of them to show you the kind of interesting applications you can build:

• Siri: This is a voice assistant created by Apple, and is included in their phones and tablets. Siri is very useful as it is connected to the internet, allowing it to look up data instantly, send messages, check the weather, and do much more.
• Netflix: Netflix is an online film and TV service. It runs on a very accurate recommendation system that is developed using AI that recommends films to users based on their viewing history. For example, if a user usually watches romantic movies, the system will recommend romantic series and movies.
• Spotify: Spotify is an online music service similar to Netflix. It uses a recommendation system to make accurate song suggestions to users. To do so, it considers songs that the user has previously heard and the kind of music added to the user’s library.
• Tesla's self-driving cars: These cars are built using AI that can detect obstacles, people, and even traffic signals to ensure the passengers have a secure ride.
• Pacman: Like almost any other video game, Pacman’s enemies are programmed using AI. They use a specific technique that constantly computes the collision distance, taking into account wall boundaries, and they try to trap Pacman. As it is a very simple game, the algorithm is not very complex, but it is a good example that highlights the importance of AI in entertainment.

Natural Language Processing

Natural Language Processing (NLP) is a specialized field in AI that involves studying the different ways of enabling communication between humans and machines. It is the only technique that can make robots understand and reproduce human language.

If a user uses an application that is supposed to be capable of communicating, the user then expects the application to have a human-like conversation. If the humanoid robot uses badly formed phrases or does not give answers related to the questions, the user’s experience wouldn’t be good and the robot wouldn’t be an attractive buy. This is why it is very important to understand and make good use of NLP in robotics.

Let’s have a look at some real-world applications that use NLP:

• Siri: Apple’s voice assistant, Siri, uses NLP to understand what the user says and gives back a meaningful response.
• Cortana: This is another voice assistant that was created by Microsoft and is included in the Windows 10 operating system. It works in a similar way to Siri.
• Bixby: Bixby is a part of Samsung that is integrated in the newest Samsung phones, and its user experience is similar to using Siri or Cortana.

  Note

  You may be asking which one of these three is the best; however, it depends on each user’s likes and dislikes.
• Phone operators: Nowadays, calls to customer services are commonly answered by answering machines. Most of these machines are phone operators that work by receiving a keyword input. Most modern operators are developed using NLP in order to have more realistic conversations with clients over the phone.
• Google Home: Google’s virtual home assistant uses NLP to respond to users’ questions and to perform given tasks.

Computer Vision

Computer vision is a commonly used technique in robotics that can use different cameras to simulate the biomechanical three-dimensional movement of the human eye. It can be defined as a set of methods used to acquire, analyze, and process images and transform them into information that can be valuable for a computer. This means that the information gathered is transformed into numerical data, so that the computer can work with it. This will be covered in the chapters ahead.

Here’s a list of some real-world examples that use computer vision:

• Autonomous cars: Autonomous cars use computer vision to obtain traffic and environment information and to decide what to do on the basis of this information. For example, the car would stop if it captures a crossing pedestrian in its camera.
• Phone camera applications: Many phone-based camera applications include effects that modify a picture taken using the camera. For example, Instagram allows the user to use filters in real time that modify the image by mapping the user’s face to the filter.
• Tennis Hawk-Eye: This is a computer-based vision system used in tennis to track the trajectory of the ball and display its most likely path on the court. It is used to check whether the ball has bounced within the court’s boundaries.

Types of Robots

When talking about AI and NLP, it is important to take a look at real-world robots, because these robots can give you a fair idea of the development and improvement of existing models. But first, let’s talk about the different kinds of robots that we can find. Generally, they can be classified as industrial-based robots and service-based robots, which we will discuss in the following sections.

Industrial Robots

Industrial robots are used in manufacturing processes and don’t usually have a human form. In general, they pretty much look like other machines. This is because they are built with the aim of executing a specific industrial task.

Service Robots

Service robots work, either partially or entirely, in an autonomous manner, and perform useful tasks for humans. These robots can also be further divided into two groups:

• Personal robots: These are commonly used in menial house-cleaning tasks, or in the entertainment industry. This is the kind of machine that people always imagine when discussing robots, and they are often imagined to have human-like features.
• Field robots: These are robots in charge of military and exploratory tasks. They are b...

Table of contents

• Preface
• Chapter 1
• Chapter 2
• Chapter 3
• Chapter 4
• Chapter 5
• Chapter 6
• Chapter 7
• Chapter 8
• Chapter 9
• Appendix