Visual Perception and Control of Underwater Robots

Junzhi Yu, Xingyu Chen, Shihan Kong

• Publisher: CRC Press
• Year: 2021
• Print ISBN: 9780367695781
• eBook ISBN: 9781000346619
• Edition: 1
• Topic: Computer Science
• Subtopic: Computer Vision & Pattern Recognition
• Index: Computer Science

Visual Perception and Control of Underwater Robots covers theories and applications from aquatic visual perception and underwater robotics. Within the framework of visual perception for underwater operations, image restoration, binocular measurement, and object detection are addressed. More specifically, the book includes adversarial critic learning for visual restoration, NSGA-II-based calibration for binocular measurement, prior knowledge refinement for object detection, analysis of temporal detection performance, as well as the effect of the aquatic data domain on object detection.

With the aid of visual perception technologies, two up-to-date underwater robot systems are demonstrated. The first system focuses on underwater robotic operation for the task of object collection in the sea. The second is an untethered biomimetic robotic fish with a camera stabilizer, its control methods based on visual tracking.

The authors provide a self-contained and comprehensive guide to understand underwater visual perception and control. Bridging the gap between theory and practice in underwater vision, the book features implementable algorithms, numerical examples, and tests, where codes are publicly available. Additionally, the mainstream technologies covered in the book include deep learning, adversarial learning, evolutionary computation, robust control, and underwater bionics. Researchers, senior undergraduate and graduate students, and engineers dealing with underwater visual perception and control will benefit from this work.

CHAPTER 1

Introduction

1.1 Research Background

The ocean contains abundant natural resources, and also provides a broad space for human development. Constantly deepening the understanding of the ocean and constantly tapping the potential of the ocean is the only way for a country and society to develop. In the era of ocean exploration, as an important tool for humans to go to the ocean, underwater robots emerged and developed gradually. For example, Yuan et al. developed an underwater glide robot that can perform deep-sea exploration tasks [1]; Cai et al. built a hybrid-driven underwater robot that can achieve high maneuverability in a marine environment [2]; Gong et al. designed underwater software whose manipulator has been successfully applied to the target capture task of marine animals [3]. Relying on advanced mechanical structures, control strategies, and perception methods, underwater robots have gradually replaced humans in their motion capabilities, can perform some complex underwater operations, and have played an important role in ocean-oriented resource development and ecological protection.

For instance, “marine ranch” is an emerging economic model that predicts the use of the marine environment to cultivate and manage fishery resources. At present, observation and collection of undersea products primarily rely on artificial underwater operations, which seriously damage the health of the workers (see Figure 1.1). In addition, seafood can also be collected by trawl fishing, but this manner causes a long period of damage to the soil of the seabed, violating the concept of ecological environment to some extent. For observation and collection tasks for marine life, robots need two capabilities. On the one hand, robots should be equipped with a visual perception system, capable of optically identifying and positioning biological targets on the seabed. On the other hand, robots should be equipped with a soft mechanical arm, which can carry out non-destructive grasping of biological targets with small Young’s modulus. Replacing traditional artificial underwater activities with single robotic operations will effectively alleviate the problems of manpower shortage and operational risk. Replacing traditional trawl fishing with clustered robotic operations can effectively reduce the damage to marine ecology. It can be seen that the underwater robot requires a high level of both motion performance and perception ability.

FIGURE 1.1 Marine ranch located at Zhangzidao, Dalian, China.

In recent years, under the guidance of computer, internet, and multimedia technology, artificial intelligence technology has developed rapidly, and it has been widely used in many real-world scenarios such as intelligent video surveillance, smart home, and smart retail. In particular, in the field of computer vision, artificial intelligence technology has effectively solved the problems of image classification, object detection, and object segmentation [4]. Among them, object detection technology can provide the robot with the information of “what” and “where” the object is. In this sense, object detection task can be divided into two sub-tasks, namely classification and localization. Among them, classification task is responsible for judging the possibility of the appearance of the object of interest, and localization task usually outputs a rectangular bounding box to represent the position of the object in the image or video. With the advent of deep convolutional neural networks (CNN) [5], the performance of object detection has been significantly improved. At the same time, the establishment of datasets (e.g., PASCAL VOC [6], MS COCO [7], ImageNet [8]) also significantly promote the development of the target detection field. The two-stage detector represented by Faster RCNN [9] divides object detection into two stages of region proposal and detection. This mode achieves a high detection accuracy. The single-stage detectors represented by YOLO [10] and SSD [11] abandon the region proposal stage and use end-to-end convolutional neural networks to directly classify and locate objects based on prior knowledge. In contrast, this detection mode has a faster inference speed.

In autonomous robot systems, scene perception servers as the basis for robot decision-making and control, and object detection is the basis of scene perception. As illustrated in Figure 1.2, in the robot task for marine pastures, target detection is the basic sensing task. On the basis of target detection, upper-level perception tasks such as multi-target tracking, key point extraction, and 3D measurement can be carried out. Eventually, the perception information will guide the robot’s decision-making and control, so as to achieve autonomous operation. Although object detection methods have been extensively and deeply researched, there are few works to study target detection in robot tasks. On the one hand, unlike images, the robot’s visual information has strong temporal consistency, and traditional object detection methods are difficult to capture this temporal information. Therefore, how to use the temporal information to improve the perception level is the key problem of object detection in robot tasks. On the other hand, u...

Table of contents

• Cover
• Half-Title
• Title
• Copyright
• Contents
• CHAPTER 1 INTRODUCTION
• CHAPTER 2 Adaptive Real-Time Underwater Visual Restoration with Adversarial Critical Learning
• CHAPTER 3 A NSGA-II-Based Calibration for Underwater Binocular Vision Measurement
• CHAPTER 4 Joint Anchor-Feature Refinement for Real-Time Accurate Object Detection in Images and Videos
• CHAPTER 5 Rethinking Temporal Object Detection from Robotic Perspectives
• CHAPTER 6 Reveal of Domain Effect: How Visual Restoration Contributes to Object Detection in Aquatic Scenes
• CHAPTER 7 IWSCR: An Intelligent Water Surface Cleaner Robot for Collecting Floating Garbage
• CHAPTER 8 Underwater Target Tracking Control of an Untethered Robotic Fish with a Camera Stabilizer
• CHAPTER 9 Summary and Outlook