A Concise Introduction to Machine Learning

A.C. Faul

• Editore: Chapman and Hall/CRC
• Anno: 2019
• ISBN:
  9781351204736
• Edizione: 1
• Argomento: Economics
• Categoria: Statistics for Business & Economics

The emphasis of the book is on the question of Why – only if why an algorithm is successful is understood, can it be properly applied, and the results trusted. Algorithms are often taught side by side without showing the similarities and differences between them. This book addresses the commonalities, and aims to give a thorough and in-depth treatment and develop intuition, while remaining concise.

This useful reference should be an essential on the bookshelves of anyone employing machine learning techniques.

The author's webpage for the book can be accessed here.

Chapter 1

Introduction

When thinking about machine learning, it seems prudent to start thinking about how we learn. King Frederick II (26 December 1194 - 13 December 1250) was Holy Roman Emperor and King of Sicily in the 13th century. King Frederick was a passionate patron of the sciences and arts. He spoke six languages which were Latin, Sicilian, German, French, Greek and Arabic. He desired to determine the “god given” language. The Italian Franciscan friar Salimbene de Adam writes in his Cronica [25] that Frederick bade “foster-mothers and nurses to suckle and bathe and wash the children, but in no ways to prattle or speak with them; for he would have learned whether they would speak the Hebrew language (which had been the first), or Greek, or Latin, or Arabic, or perchance the tongue of their parents of whom they had been born. But he laboured in vain, for the children could not live without clappings of the hands, and gestures, and gladness of countenance, and blandishments.” In other words the physical needs of the children were satisfied, but they were raised without any human interaction. It is doubtful whether this is a true account, since this is the only account and Salimbene was a political opponent of Frederick II. Nevertheless, nobody doubts that sensory stimulation and experiences are essential for learning in any respect.

Blakemore and Cooper [4] experimented with kittens. The kittens were brought up in a dark room and only brought out at certain times and then placed in an environment with either only horizontal or only vertical lines. Kittens brought up in the horizontal environment showed no reaction to vertical lines. There was no brain activity. Indeed, when the inclination of a horizontal line was changed gradually towards a vertical line the brain activity became less and less.

The experiment showed that only what the brain is presented with by the environment is learned. This is an efficient preparation for the future. This is also true for human vision. Australian Aborigines have the sharpest vision ever measured, about four times better than the vision of those of white ethnicity. This means that they can see objects sharply at six meters distance which the average white person can see clearly at only 1.5 meters, a quarter of the distance. Often the eyesight deteriorates with old age. Ophthalmologist Professor Fred Hollows [22] corrected the vision of an elderly Aboriginal man back to the average white person’s vision with glasses. The reaction was “Thank you for trying, but this is hopeless. I used to be able to see much better.” The Australian outback is a wide open landscape and good vision in the distance is vital for survival.

We can conclude that there is a need for experiences. However, how does a machine “experience”? We can view our senses as taking measurements and our brain interprets these and draws conclusions. A machine can take various measurements and then perform calculations, but can it emulate the power of a human brain?

Perhaps to answer this question, we need to take a step back and not look at how we learn, but how we teach. Traditional teaching is from the front of the classroom. The pupils are given a set of instructions and are expected to reproduce these. This is very similar to procedural programming where the flow of instructions is encoded. Object oriented programming was a further development where instructions depend on the nature of the data.

However, our brain retains information much better when it has the positive experience of discovery. So often there is teacher-led learning. Galileo illustrates this beautifully in his book “Discourses and Mathematical Demonstrations Relating to Two New Sciences” [15] (Figure 1.2). The two sciences are the science of motion and the science of materials and construction. The ideas are developed as a dialogue between three characters, Salviati, Sagredo and Simplicio. The latter is portrayed as a simpleton, the pupil to be instructed. In the science of motion, the starting point is the observation that even though objects have different masses, they reach the ground at the same time. However, this is very difficult to quantify, since it is over all too fast. Galileo therefore developed the inclined plane experiment which he describes as such:

Figure 1.1: Vertical environment. Reprinted by permission from Macmillan Publishers Ltd: Nature [4], copyright (1970).

Figure 1.2: Discorsi e Dimostrazioni Matematiche Intorno a Due Nuove Scienze Image in the public domain.

“A piece of wooden moulding or scantling, about 12 cubits long, half a cubit wide, and three finger-breadths thick, was taken; on its edge was cut a channel a little more than one finger in breadth; having made this groove very straight, smooth, and polished, and having lined it with parchment, also as smooth and polished as possible, we rolled along it a hard, smooth, and very round bronze ball. Having placed this board in a sloping position, by raising one end some one or two cubits above the other, we rolled the ball, as I was just saying, along the channel, noting, in a manner presently to be described, the time required to make the descent. We repeated this experiment more than once in order to measure the time with an accuracy such that the deviation between two observations never exceeded one-tenth of a pulse-beat. Having performed this operation and having assured ourselves of its reliability, we now rolled the ball only one-quarter the length of the channel; and having measured the time of its descent, we found it precisely one-half of the former. Next we tried other distances, compared the time for the whole length with that for the half, or with that for two-thirds, or three-fourths, or indeed for any fraction; in such experiments, repeated a full hundred times, we always found that the spaces traversed were to each other as the squares of the times, and this was true for all inclinations of the plane, i.e., of the channel, along which we rolled the ball. We also observed that the times of descent, for various inclinations of the plane, bore to one another precisely that ratio which, as we shall see later, the Author had predicted and demonstrated for them.

For the measurement of time, we employed a large vessel of water placed in an elevated position; to the bottom of this vessel was soldered a pipe of small diameter giving a thin jet of water which we collected in a small glass during the time of each descent, whether for the whole length of the channel or for part of its length; the water thus collected was weighed, after each descent, on a very accurate balance; the differences and ratios of these weights gave us the differences and ratios of the times, and this with such accuracy that although the operation was repeated many, many times, there was no appreciable discrepancy in the results.”

In the 19th century an apparatus for this experiment was built and can now be seen in the Museo Galileo in Florence, Italy. This experiment is repeated by school children all over the world again and again, stacking their books to create an inclined plane. Only stop watches have replaced the water clock. This is supervised learning and a regression problem. In regression we try to find a relationship between two or more parameters. In this case it is distance, d, and time, t.

Note that the relationship between the two parameters, distance and time, is not linear. In fact,

d1d2=t12t22.

We can rephrase this so that distance and time are primary parameters and the square of the time is a secondary parameter. Then we have found a linear relationship between a primary parameter, the distance, and a secondary parameter, the square of time. We will encounter this again when discussing the kernel trick. Another way of viewing this is as an instance of deep learning since another layer of abstraction is added by the square. Deep learning is trying to uncover hidden relationships.

However, how did Galileo arrive at this experiment? Remember that the starting point was the observation that objects of different masses reach the ground at the same time. He realized that air resistance is a factor, becoming dominant for extremely light objects with a lot of air resistance such as feathers. Thus he needed to make his experiment as independent from air resistance as possible and arrived at a bronze ball. More importantly, however, he needed to slow the experiment down in order to make accurate measurements. Galileo noticed that a ball rolling down a ramp which is smoothly connected to an upward ramp will reach essentially the same level it started from. It will roll backwards and forwards until it comes to a rest because of friction and air resistance. If both ramps have the same inclination, this is not that surprising, bu...