Machine Learning

Algorithms and Applications

Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier Mohammed Bashier

• Publisher: CRC Press
• Year: 2016
• ISBN:
  9781315354415
• Edition: 1
• Topic: Ciencia de la computación
• Subtopic: Ciencias computacionales general

Machine learning, one of the top emerging sciences, has an extremely broad range of applications. However, many books on the subject provide only a theoretical approach, making it difficult for a newcomer to grasp the subject material. This book provides a more practical approach by explaining the concepts of machine learning algorithms and describing the areas of application for each algorithm, using simple practical examples to demonstrate each algorithm and showing how different issues related to these algorithms are applied.

Chapter 1

Introduction to Machine Learning

1.1 Introduction

Learning is a very personalized phenomenon for us. Will Durant in his famous book, The Pleasures of Philosophy, wondered in the chapter titled “Is Man a Machine?” when he wrote such classical lines:

Here is a child; … See it raising itself for the first time, fearfully and bravely, to a vertical dignity; why should it long so to stand and walk? Why should it tremble with perpetual curiosity, with perilous and insatiable ambition, touching and tasting, watching and listening, manipulating and experimenting, observing and pondering, growing—till it weighs the earth and charts and measures the stars?… [1]

Nevertheless, learning is not limited to humans only. Even the simplest of species such as amoeba and paramecium exhibit this phenomenon. Plants also show intelligent behavior. Only nonliving things are the natural stuffs that are not involved in learning. Hence, it seems that living and learning go together. In nature-made nonliving things, there is hardly anything to learn. Can we introduce learning in human-made nonliving things that are called machines? Enabling a machine capable of learning like humans is a dream, the fulfillment of which can lead us to having deterministic machines with freedom (or illusion of freedom in a sense). During that time, we will be able to happily boast that our humanoids resemble the image and likeliness of humans in the guise of machines.

1.2 Preliminaries

Machines are by nature not intelligent. Initially, machines were designed to perform specific tasks, such as running on the railway, controlling the traffic flow, digging deep holes, traveling into the space, and shooting at moving objects. Machines do their tasks much faster with a higher level of precision compared to humans. They have made our lives easy and smooth.

The fundamental difference between humans and machines in performing their work is intelligence. The human brain receives data gathered by the five senses: vision, hearing, smell, taste, and tactility. These gathered data are sent to the human brain via the neural system for perception and taking action. In the perception process, the data is organized, recognized by comparing it to previous experiences that were stored in the memory, and interpreted. Accordingly, the brain takes the decision and directs the body parts to react against that action. At the end of the experience, it might be stored in the memory for future benefits.

A machine cannot deal with the gathered data in an intelligent way. It does not have the ability to analyze data for classification, benefit from previous experiences, and store the new experiences to the memory units; that is, machines do not learn from experience.

Although machines are expected to do mechanical jobs much faster than humans, it is not expected from a machine to: understand the play Romeo and Juliet, jump over a hole in the street, form friendships, interact with other machines through a common language, recognize dangers and the ways to avoid them, decide about a disease from its symptoms and laboratory tests, recognize the face of the criminal, and so on. The challenge is to make dumb machines learn to cope correctly with such situations. Because machines have been originally created to help humans in their daily lives, it is necessary for the machines to think, understand to solve problems, and take suitable decisions akin to humans. In other words, we need smart machines. In fact, the term smart machine is symbolic to machine learning success stories and its future targets. We will discuss the issue of smart machines in Section 1.4.

The question of whether a machine can think was first asked by the British mathematician Alan Turing in 1955, which was the start of the artificial intelligence history. He was the one who proposed a test to measure the performance of a machine in terms of intelligence. Section 1.4 also discusses the progress that has been achieved in determining whether our machines can pass the Turing test.

Computers are machines that follow programming instructions to accomplish the required tasks and help us in solving problems. Our brain is similar to a CPU that solves problems for us. Suppose that we want to find the smallest number in a list of unordered numbers. We can perform this job easily. Different persons can have different methods to do the same job. In other words, different persons can use different algorithms to perform the same task. These methods or algorithms are basically a sequence of instructions that are executed to reach from one state to another in order to produce output from input.

If there are different algorithms that can perform the same task, then one is right in questioning which algorithm is better. For example, if two programs are made based on two different algorithms to find the smallest number in an unordered list, then for the same list of unordered number (or same set of input) and on the same machine, one measure of efficiency can be speed or quickness of program and another can be minimum memory usage. Thus, time and space are the usual measures to test the efficiency of an algorithm. In some situations, time and space can be interrelated, that is, the reduction in memory usage leading to fast execution of the algorithm. For example, an efficient algorithm enabling a program to handle full input data in cache memory will also consequently allow faster execution of program.

1.2.1 Machine Learning: Where Several Disciplines Meet

Machine learning is a branch of artificial intelligence that aims at enabling machines to perform their jobs skillfully by using intelligent software. The statistical learning methods constitute the backbone of intelligent software that is used to develop machine intelligence. Because machine learning algorithms require data to learn, the discipline must have connection with the discipline of database. Similarly, there are familiar terms such as Knowledge Discovery from Data (KDD), data mining, and pattern recognition. One wonders how to view the big picture in which such connection is illustrated.

SAS Institute Inc., North Carolina, is a developer of the famous analytical software Statistical Analysis System (SAS). In order to show the connection of the discipline of machine learning with different related disciplines, we will use the illustration from SAS. This illustration was actually used in a data mining course that was offered by SAS in 1998 (see Figure 1.1).

Figure 1.1 Different disciplines of knowledge and the discipline of machine learning. (From Guthrie, Looking backwards, looking forwards: SAS, data mining and machine learning, 2014, http://blogs.sas.com/content/subconsciousmusings/2014/08/22/looking-backwards-looking-forwards-sas-data-mining-and-machine-learning/2014. With permission.)

In a 2006 article entitled “The Discipline of Machine Learning,” Professor Tom Mitchell [3, p.1] defined the discipline of machine learning in these words:

Machine Learning is a natural outgrowth of the intersection of Computer Science and Statistics. We might say the defining question of Computer Science is ‘How can we build machines that solve problems, and which problems are inherently tractable/intractable?’ The question that largely defines Statistics is ‘What can be inferred from data plus a set of modeling assumptions, with what reliability?’ The defining question for Machine Learning builds on both, but it is a distinct question. Whereas Computer Science has focused primarily on how to manually program computers, Machine Learning focuses on the question of how to get computers to program themselves (from experience plus some initial structure). Whereas Statistics has focused primarily on what conclusions can be inferred from data, Machine Learning incorporates additional questions about what computational architectures and algorithms can be used to most effectively capture, store, index, retrieve and merge these data, how multiple learning subtasks can be orchestrated in a larger system, and questions of computational tractability [emphasis added].

There are some tasks that humans perform effortlessly or with some efforts, but we are unable to explain how we perform them. For example, we can recognize the speech of our friends without much difficulty. If we are asked how we recognize the voices, the answer is very difficult for us to explain. Because of the lack of understanding of such phenomenon (speech recognition in this case), we cannot craft algorithms for such scenarios. Machine learning algorithms are helpful in bridging this gap of understanding.

The idea is very simple. We are not targeting to understand the underlying processes that help us learn. We write computer programs that will make machines learn and enable them to perform tasks, such as prediction. The goal of learning is to construct a model that takes the input and produces the desired result. Sometimes, we can understand the model, whereas, at other times, it can also be like a black box for us, the working of which cannot be intuitively explained. The model can be considered as an approximation of the process we want machines to mimic. In such a situation, it is possible that we obtain errors for some input, but most of the time, the model provides correct answers. Hence, another measure of performance (besides performance of metrics of speed and memory usage) of a machine learning algorithm will be the accuracy of r...