Ascend AI Processor Architecture and Programming

Principles and Applications of CANN

Xiaoyao Liang

• Publisher: Elsevier
• Year: 2020
• Print ISBN: 9780128234884
• eBook ISBN: 9780128234891
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Ascend AI Processor Architecture and Programming: Principles and Applications of CANN offers in-depth AI applications using Huawei's Ascend chip, presenting and analyzing the unique performance and attributes of this processor. The title introduces the fundamental theory of AI, the software and hardware architecture of the Ascend AI processor, related tools and programming technology, and typical application cases. It demonstrates internal software and hardware design principles, system tools and programming techniques for the processor, laying out the elements of AI programming technology needed by researchers developing AI applications.Chapters cover the theoretical fundamentals of AI and deep learning, the state of the industry, including the current state of Neural Network Processors, deep learning frameworks, and a deep learning compilation framework, the hardware architecture of the Ascend AI processor, programming methods and practices for developing the processor, and finally, detailed case studies on data and algorithms for AI.- Presents the performance and attributes of the Huawei Ascend AI processor- Describes the software and hardware architecture of the Ascend processor- Lays out the elements of AI theory, processor architecture, and AI applications- Provides detailed case studies on data and algorithms for AI- Offers insights into processor architecture and programming to spark new AI applications

Chapter 1: Theoretical basis

Abstract

This chapter gives a brief introduction to artificial intelligence (AI). It starts by reviewing the history of AI. Then it introduces deep learning, which is the recent trend in AI, from its history, applications to future challenges. The last part of the chapter reviews the theory of neural network, which is the basis of deep learning. It introduces the neuron model, perception, multilayer perception, convolutional neural network (CNN), as well as application examples.

Keywords

Artificial intelligence; AI; Deep learning; Neural network; Perception; Convolutional neural network; CNN

1.1: Brief history of artificial intelligence

When the development of a skill reaches the peak, it can reflect highly anthropomorphic intelligence. From Ancient China’s Master Yan’s ability to sing and dance, to Ancient Arabic Jazari’s automatic puppets (as shown in Fig. 1.1), the relentless pursuit of intelligence never ends. Human beings hope to give wisdom and thought to machines, which can be used to liberate productive forces, facilitate people’s lives, and promote social development. From ancient myths to science fiction, and now to modern science and technology, they all demonstrate the human desire for intelligence. The birth of artificial intelligence (AI) was a slow and long process, whereas the improvement of AI is keeping pace with the development of human knowledge and even transcending human wisdom in some aspects. In the early days, the development of Formal Reasoning provided a research direction for the mechanization of human intelligence.

Fig. 1.1 Jazari’s automatic elephant clock. Picture from: https://commons.wikimedia.org/wiki/File:Al-jazari_elephant_clock.png.

In the mid-17th century, Gottfried Wilhelm Leibniz, Rene Descartes, and Thomas Hobbes (Fig. 1.2) devoted themselves to the systematic study of rational thinking. These studies led to the emergence of the formal symbol system that became the beacon of AI research. By the 20th century, the contributions of Bertrand Arthur William Russell, Alfred North Whitehead, and Kurt Godel in the field of mathematical logic provided a theoretical basis for the mechanization of mathematical reasoning. The creation of the Turing Machine provided supporting evidence of machine thought from the view of semiotics. In the engineering domain, from Charles Babbage’s original idea of “Instrumental Analysis” to the large ENIAC decoding machine serving in World War II, we have witnessed the realization of the theories of Alan Turing and John von Neumann (see Fig. 1.3), which has accelerated the development of AI.

Fig. 1.2 Leibniz, Descartes, Hobbes (from left to right). Leibniz photo source: https://commons.wikimedia.org/wiki/File:Gottfried_Wilhelm_Leibniz.webp; Descartes photo source: https://en.wikipedia.org/wiki/Ren%C3%A9_Descartes; Hobbes photo source: https://commons.wikimedia.org/wiki/File:Thomas_Hobbes_(portrait).webp.

Fig. 1.3 Turing and Von Neumann (from left to right). Turing photo source: https://upload.wikimedia.org/wikipedia/commons/thumb/a/a1/Alan_Turing_Aged_16.webp/220px-Alan_Turing_Aged_16.webp; Von Neumann photo source: https://commons.wikimedia.org/wiki/File:JohnvonNeumann-LosAlamos.webp.

1.1.1: Birth of AI

In the mid-20th century, scientists from different fields made a series of preparations for the birth of AI, including Shannon’s information theory, Turing’s computational theory, and the development of Neurology.

In 1950, Turing published the paper Computing Machinery and Intelligence. In this paper, the famous Turing test was proposed: If a machine can answer any questions to it, using the same words that an ordinary person would, then we may call that machine intelligent. The proposal of the Turing test is of great significance to the development of AI in more recent times. In 1951, the 24-year-old Marvin Minsky, as well as Dean Edmonds, built the Stochastic Neural Analog Reinforcement Calculator. Minsky continued to work in the field of AI, playing a huge role in promoting the development of AI, which contributed to his winning of Turing Award. In 1955, a program called Logic Theorist was introduced and subsequently refined. The ingenious method proved 38 of 52 theorems in the Principles of Mathematics. With this work, the authors Allen Newell and Herbert Simon opened a new methodology for intelligent machines.

A year later, 10 participants at the Dartmouth Conference, including McCarthy, Shannon, and Nathan Rochester, argued that “any aspect of learning or intelligence should be accurately described so that people can build a machine to simulate it.” In that moment, AI entered the world with its mission clearly defined, opening up a brand new world of science.

1.1.2: Set sail

After the Dartmouth Conference, AI developments proceeded akin to a volcanic eruption. The conference started a wave of activity that swept across the globe. Through these developments, significant progress was achieved. Computers were shown to succeed at more advanced tasks, such as solving algebraic problems, geometric proofs, and certain problems in the field of language processing. These advances made researchers enthusiastic and confident about the improvement of AI, and attracted a large amount of funds to the research field.

In 1958, Herbert Gelernter implemented a geometric theorem-roving machine based on a search algorithm. Newell and Simon extended the application of search-based reasoning through a “General Problem Solver” program. At the same time, search-based reasoning was applied to decision-making, such as STRIPS, a Stanford University robotic system. In the field of natural language, Ross Quillian developed the first Semantic Web. Subsequently, Joseph Weizenbaum created the first dialogue robot, ELIZA. ELIZA was sufficiently lifelike that it could be mistaken for a human being when communicating with a person. The advent of ELIZA marked a great milestone in AI. In June 1963, MIT received funding from the United States Agency for Advanced Research Projects (ARPA) to commercialize the MAC (The Project on Mathematics and Computation) project. Minsky and McCarthy were major participants in the project. The MAC project played an important role in the history of AI and also in the development of computer science, giving birth to the famous MIT Computer Science and Artificial Intelligence Laboratory.

In this period, people came to expect the rapid acceleration of AI developments. As Minsky predicted in 1970: “in three to eight years we will have a machine with average human intelligence.” However, the development of AI still required the lengthy process of continuous improvement and maturity, which meant that progress would be slowly moving forward.

1.1.3: Encounter bottlenecks

In the early 1970s, the development of AI gradually slowed down. At that time, the best AI programs could only solve problems in constrained environments, which was difficult to meet the needs of real-world applications. This result arose because AI research had met a bottleneck that was difficult to break through. In terms of computing power, the memory size and processor speed of computers at that time were insufficient for actual AI requirements, which need high-performance computing resources. An obvious example was that in natural language research, only a small vocabulary containing less than 20 words could be processed. Computational complexity was another concern. Richard Karp proved in 1972 that the time complexity of many problems is proportional to the power of input size, which implies that AI is almost impossible in problems that might lead to an exponential explosion. In the fields of natural language and machine vision, a large amount of external cognitive information is needed as the basis for recognition. Researchers found that the construction of AI databases is very difficult even if the target is to reach the level of a child’s cognition. For computers, the ability to deal with mathematical problems such as theorem proving and geometry is much stronger than the ability to deal with tasks that seem extremely simple to humans, such as object recognition. This unexpected challenge made researchers almost want to quit their studies in these areas.

As a result of these factors, government agencies gradually lost patience with the prospects of AI and began to shift funding to other projects. At the same time, AI gradually faded out of people’s vision.

1.1.4: Moving on again

After several years at a low ebb, the emergence of “expert systems,” along with the resurgence of ...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• About the Author
• Preface
• Chapter 1: Theoretical basis
• Chapter 2: Industry background
• Chapter 3: Hardware architecture
• Chapter 4: Software architecture
• Chapter 5: Programming methods
• Chapter 6: Case studies
• Index