Multi-Agent Coordination

A Reinforcement Learning Approach

Arup Kumar Sadhu, Amit Konar

• Publisher: Wiley-IEEE Press
• Year: 2020
• Print ISBN: 9781119699033
• eBook ISBN: 9781119699026
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

Discover the latest developments in multi-robot coordination techniques with this insightful and original resource

Multi-Agent Coordination: A Reinforcement Learning Approach delivers a comprehensive, insightful, and unique treatment of the development of multi-robot coordination algorithms with minimal computational burden and reduced storage requirements when compared to traditional algorithms. The accomplished academics, engineers, and authors provide readers with both a high-level introduction to, and overview of, multi-robot coordination, and in-depth analyses of learning-based planning algorithms.

You'll learn about how to accelerate the exploration of the team-goal and alternative approaches to speeding up the convergence of TMAQL by identifying the preferred joint action for the team. The authors also propose novel approaches to consensus Q-learning that address the equilibrium selection problem and a new way of evaluating the threshold value for uniting empires without imposing any significant computation overhead. Finally, the book concludes with an examination of the likely direction of future research in this rapidly developing field.

Readers will discover cutting-edge techniques for multi-agent coordination, including:

• An introduction to multi-agent coordination by reinforcement learning and evolutionary algorithms, including topics like the Nash equilibrium and correlated equilibrium
• Improving convergence speed of multi-agent Q-learning for cooperative task planning
• Consensus Q-learning for multi-agent cooperative planning
• The efficient computing of correlated equilibrium for cooperative q-learning based multi-agent planning
• A modified imperialist competitive algorithm for multi-agent stick-carrying applications

Perfect for academics, engineers, and professionals who regularly work with multi-agent learning algorithms, Multi-Agent Coordination: A Reinforcement Learning Approach also belongs on the bookshelves of anyone with an advanced interest in machine learning and artificial intelligence as it applies to the field of cooperative or competitive robotics.

1
Introduction: Multi‐agent Coordination by Reinforcement Learning and Evolutionary Algorithms

This chapter provides an introduction to the multi‐agent coordination by reinforcement learning (RL) and evolutionary algorithms (EAs). A robot (agent) is an intelligent programmable device capable of performing complex tasks and decision‐making like the human beings. Mobility is part and parcel of modern robots. Mobile robots employ sensing‐action cycles to sense the world around them with an aim to plan their journey to the desired destination. Coordination is an important issue in modern robotics. In recent times, researchers are taking keen interest to synthesize multi‐agent‐coordination in complex real‐world problems, including transportation of a box/stick, formation control for defense applications, and soccer playing by multiple robots by utilizing the principles of RL, theory of games (GT), dynamic programming (DP), and/or evolutionary optimization (EO) algorithms. This chapter provides a thorough survey of the existing literature of RL with a brief overview of EO to examine the role of the algorithms in the context of multi‐agent coordination. The study includes the classification of multi‐agent coordination based on different criterion, such as the level of cooperation, knowledge sharing, communication, and the like. The chapter also includes multi‐robot coordination employing EO, and specially RL for cooperative, competitive, and their composition for application to static and dynamic games. The later part of the chapter deals with an overview of the metrics used to compare the performance of the algorithms in coordination. Two fundamental metrics of performance analysis are defined, where the first one is required to study the learning performance, while the other to measure the performance of the planning algorithm. Conclusions are listed at the end of the chapter with possible explorations for the future real‐time applications.

1.1 Introduction

A robot is an intelligent and programm...

Table of contents

• Cover
• Table of Contents
• Title Page
• Copyright Page
• Preface
• Acknowledgments
• About the Authors
• 1 Introduction
• 2 Improve Convergence Speed of Multi‐Agent Q‐Learning for Cooperative Task Planning
• 3 Consensus Q‐Learning for Multi‐agent Cooperative Planning
• 4 An Efficient Computing of Correlated Equilibrium for Cooperative Q‐Learning‐Based Multi‐Robot Planning
• 5 A Modified Imperialist Competitive Algorithm for Multi‐Robot Stick‐Carrying Application
• 6 Conclusions and Future Directions
• Index
• End User License Agreement