Distributed and Cloud Computing

From Parallel Processing to the Internet of Things

Kai Hwang, Jack Dongarra, Geoffrey C. Fox

• Publisher: Morgan Kaufmann
• Year: 2013
• ISBN:
  9780128002049
• Topic: Computer Science
• Subtopic: Systems Architecture
• Index: Computer Science

Distributed and Cloud Computing: From Parallel Processing to the Internet of Things offers complete coverage of modern distributed computing technology including clusters, the grid, service-oriented architecture, massively parallel processors, peer-to-peer networking, and cloud computing. It is the first modern, up-to-date distributed systems textbook; it explains how to create high-performance, scalable, reliable systems, exposing the design principles, architecture, and innovative applications of parallel, distributed, and cloud computing systems.

Topics covered by this book include: facilitating management, debugging, migration, and disaster recovery through virtualization; clustered systems for research or ecommerce applications; designing systems as web services; and social networking systems using peer-to-peer computing. The principles of cloud computing are discussed using examples from open-source and commercial applications, along with case studies from the leading distributed computing vendors such as Amazon, Microsoft, and Google. Each chapter includes exercises and further reading, with lecture slides and more available online.

This book will be ideal for students taking a distributed systems or distributed computing class, as well as for professional system designers and engineers looking for a reference to the latest distributed technologies including cloud, P2P and grid computing.

• Complete coverage of modern distributed computing technology including clusters, the grid, service-oriented architecture, massively parallel processors, peer-to-peer networking, and cloud computing
• Includes case studies from the leading distributed computing vendors: Amazon, Microsoft, Google, and more
• Explains how to use virtualization to facilitate management, debugging, migration, and disaster recovery
• Designed for undergraduate or graduate students taking a distributed systems course—each chapter includes exercises and further reading, with lecture slides and more available online

Part 1

Systems Modeling, Clustering, and Virtualization

Outline

Introduction

Chapter 1 Distributed System Models and Enabling Technologies

Chapter 2 Computer Clusters for Scalable Parallel Computing

Chapter 3 Virtual Machines and Virtualization of Clusters and Data Centers

Introduction

The first three chapters cover systems models and review two enabling technologies. We model distributed systems and cloud platforms in Chapter 1. The clustering technology is presented in Chapter 2, and virtualization technology in Chapter 3. These two technologies enable distributed and cloud computing. The system models include computer clusters, computing grid, P2P networks, and cloud computing platform. System clustering is supported by hardware, software, and middleware advances. Virtualization creates virtual machines, virtualized clusters, automation of datacenters, and building of elastic cloud platforms.

Chapter 1: Distributed System Models and Enabling Technologies

This introductory chapter assesses the evolutional changes in computing and IT trends in the past 30 years. We study both high-performance computing (HPC) for scientific computing and high-throughput computing (HTC) systems for business computing. We examine clusters/MPP, grids, P2P networks, and Internet clouds. These systems are distinguished by their platform architectures, OS platforms, processing algorithms, communication protocols, security demands, and service models applied. The study emphasizes the scalability, performance, availability, security, energy-efficiency, workload outsourcing, data center protection, and so on.

This chapter is authored by Kai Hwang with partial contributions by Geoffrey Fox (Section 1.4.1) and Albert Zomaya (Section 1.5.4), and assisted by Nikzad Babaii Rizvandi, Young-Choon Lee, Xiaosong Lou, and Lizhong Chen. The final manuscript was edited by Jack Dongarra.

Chapter 2: Computer Clusters for Scalable Parallel Computing

Clustering enables the construction of scalable parallel and distributed systems for both HPC and HTC applications. Today’s cluster nodes are built with either physical servers or virtual machines. In this chapter, we study clustered computing systems and massively parallel processors. We focus on design principles and assess the hardware, software, and middleware support needed. We study scalability, availability, programmability, single system image, job management, and fault tolerance. We study the clustering MPP architectures in three top supercomputers reported in recent years, namely China’s Tiahe-1A, the Cray XT5 Jaguar, and IBM’s RoadRunner.

This chapter is coauthored by Kai Hwang and Jack Dongarra with partial contributions by Rajkumar Buyya and Ninghui Sun. Special technical assistances are from Zhiwei Xu, Zhou Zhao, Xiaosong Lou, and Lizhong Chen.

Chapter 3: Virtual Machines and Virtualization of Clusters and Data Centers

Virtualization technology was primarily designed for the sharing of expensive hardware resources by multiplexing virtual machines (VM) on the same set of hardware hosts. The surge of interest in installing virtual machines has widened the scope of system applications and upgraded computer performance and efficiency in recent years. We study VMs, live migration of VMs, virtual cluster construction, resource provisioning, virtual configuration adaptation, and the design of virtualized data centers for cloud computing applications. We emphasize the roles of using virtual clusters and virtualized resource management for building dynamic grids and cloud platforms.

This chapter is coauthored by Zhibin Yu and Kai Hwang with technical assistance from Hai Jin, Xiaofei Liao, Chongyuan Qin, Lizhong Chen, and Zhou Zhao.

Chapter 1

Distributed System Models and Enabling Technologies

Chapter Outline

Summary

1.1 Scalable Computing over the Internet

1.1.1 The Age of Internet Computing

1.1.2 Scalable Computing Trends and New Paradigms

1.1.3 The Internet of Things and Cyber-Physical Systems

1.2 Technologies for Network-based Systems

1.2.1 Multicore CPUs and Multithreading Technologies

1.2.2 GPU Computing to Exascale and Beyond

1.2.3 Memory, Storage, and Wide-Area Networking

1.2.4 Virtual Machines and Virtualization Middleware

1.2.5 Data Center Virtualization for Cloud Computing

1.3 System Models for Distributed and Cloud Computing

1.3.1 Clusters of Cooperative Computers

1.3.2 Grid Computing Infrastructures

1.3.3 Peer-to-Peer Network Families

1.3.4 Cloud Computing over the Internet

1.4 Software Environments for Distributed Systems and Clouds

1.4.1 Service-Oriented Architecture (SOA)

1.4.2 Trends toward Distributed Operating Systems

1.4.3 Parallel and Distributed Programming Models

1.5 Performance, Security, and Energy Efficiency

1.5.1 Performance Metrics and Scalability Analysis

1.5.2 Fault Tolerance and System Availability

1.5.3 Network Threats and Data Integrity

1.5.4 Energy Efficiency in Distributed Computing

1.6 Bibliographic Notes and Homework Problems

Acknowledgments

References

Homework Problems

This chapter presents the evolutionary changes that have occurred in parallel, distributed, and cloud computing over the past 30 years, driven by applications with variable workloads and large data sets. We study both high-performance and high-throughput computing systems in parallel computers appearing as computer clusters, service-oriented architecture, computational grids, peer-to-peer networks, Internet clouds, and the Internet of Things. These systems are distinguished by their hardware architectures, OS platforms, processing algorithms, communication protocols, and service models applied. We also introduce essential issues on the scalability, performance, availability, security, and energy efficiency in distributed systems.

scalable computing, distributed systems, virtualization, parallel programming, security, peer-to-peer networks

Summary

This chapter presents the evolutionary changes that have occurred in parallel, distributed, and cloud computing over the past 30 years, driven by applications with variable workloads and large data sets. We study both high-performance and high-throughput computing systems in parallel computers appearing as computer clusters, service-oriented architecture, computational grids, peer-to-peer networks, Internet clouds, and the Internet of Things. These systems are distinguished by their hardware architectures, OS platforms, processing algorithms, communication protocols, and service models applied. We also introduce essential issues on the scalability, performance, availability, security, and energy efficiency in distributed systems.

1.1 Scalable Computing Over the Internet

Over the past 60 years, computing technology has undergone a series of platform and environment changes. In this section, we assess evolutionary changes in machine architecture, operating system platform, network connectivity, and application workload. Instead of using a centralized computer to solve computational problems, a parallel and distributed computing system uses multiple computers to solve large-scale problems over the Internet. Thus, distributed computing becomes data-intensive and network-centric. This section identifies the applications of modern computer systems that practice parallel and distributed computing. These large-scale Internet applications have significantly enhanced the quality of life and information services in society today.

1.1.1 The Age of Internet Computing

Billions of people use the Internet every day. As a result, supercomputer sites and large data centers must provide high-performance computing services to huge numbers of Internet users concurrently. Because of this high demand, the Linpack Benchmark for high-performance computing (HPC) applications is no longer optimal for measuring system performance. The emergence of computing clouds instead demands high-throughput computing (HTC) systems built with parallel and distributed computing technologies [5,6,19,25]. We have to upgrade data centers using fast servers, storage systems, and high-bandwidth networks. The purpose is to advance network-based computing and web services with the emerging new technologies.

1.1.1.1 The Platform Evolution

Computer technology has gone through five generations of development, with each generation lasting from 10 to 20 years. Successive generations are overlapped in about 10 years. For instance, from 1950 to 1970, a handful of mainframes, including the IBM 360 and CDC 6400, were built to satisfy the demands of large businesses and government organizations. From 1960 to 1980, lower-cost minicomputers such as the DEC PDP 11 and VAX Series became popular among small businesses and on college campuses.

From 1970 to 1990, we saw widespread use of personal computers built with VLSI microprocessors. From 1980 to 2000, massive numbers of portable computers and pervasive devices appeared in both wired and wireless applications. Since 1990, the use of both HPC and HTC systems hidden in clusters, grids, or Internet clouds has proliferated. These systems are employed by...