A Practical Introduction to Human-in-the-Loop Cyber-Physical Systems

David Nunes, Jorge Sa Silva, Fernando Boavida

• Publisher: Wiley-IEEE Press
• Year: 2017
• Print ISBN: 9781119377771
• eBook ISBN: 9781119377788
• Edition: 1
• Topic: Tecnologia e ingegneria
• Subtopic: Comunicazioni mobili e wireless

The first book focusing on one of the hottest new topics in Internet of Things systems research and development

Studies estimate that by 2020 we will have a vast Internet of Things (IoT) network comprising 26 billion connected devices, including everything from light bulbs to refrigerators, coffee makers to cars. From the beginning, the concept of cyber-physical systems (CPS), or the sensing and control of physical phenomena through networks of devices that work together to achieve common goals, has been implicit in the IoT enterprise. This book focuses on the increasingly hot topic of Human-in-the-loop Cyber-Physical Systems (HiTLCPS)—CPSs that incorporate human responses in IoT equation.

Why have we not yet integrated the human component into CPSs? What are the major challenges to achieving HiTLCPS? How can we take advantage of ubiquitous sensing platforms, such as smartphones and personal devices to achieve that goal? While mature HiTLCPS designs have yet to be achieved, or a general consensus reached on underlying HiTLCPS requirements, principles, and theory, researchers and developers worldwide are on the cusp of realizing them. With contributions from researchers at the cutting edge of HiTLCPS R&D, this book addresses many of these questions from the theoretical and practical points of view.

• An essential primer on a rapidly emerging Internet-of-Things concept, focusing on human-centric applications
• Discusses new topics which, until now, have only been available in research papers scattered throughout the world literature
• Addressed fundamental concepts in depth while providing practical insights into the development of complete HiTLCPS systems
• Includes a companion website containing full source-code for all of the applications described

This book is an indispensable resource for researchers and app developers eager to explore HiTL concepts and include them into their designs. It is also an excellent primer for advanced undergraduates and graduate students studying IoT, CPS, and HiTLCPS.

Chapter 1
Introduction

Humans are a remarkable species. For most of our history, we have used our intellectual ability to create and develop many different tools and processes to assist us and ease our lives. Since the days our ancestors discovered how to control fire, around 300,000 years ago, we have achieved an exponential technological progress. From the invention of wheeled vehicles, around 6,000 years ago, to the transistor, invented just 70 years ago, many were the technological advances that have drastically changed the way we experience and perceive our reality.

The last few decades have seen an unprecedented surge of technological advancement, particularly in the area of computer science, resulting in some of the most revolutionary human inventions yet: we have developed personal desktop and portable computers, as well as a global network that interconnects all kinds of computerized devices, aptly called the Internet. Despite the fact that they have been in existence for an extremely short time, these technologies have transformed, and will continue to transform, the way our world and society work, at a very fundamental level and at an incredibly fast pace.

1.1 The Rise of Cyber-Physical Systems

Interestingly, once the Internet was in place, we quickly achieved the power to extend it to our traditional tools and appliances, which then became “interconnected”. One of the first “tools” ever connected to the Internet was the Carnegie Mellon University Computer Science Department's Coke Machine, in the early 1980s [19], which was able to report its stock and label it as “cold” or not, depending on how much time it had been inside the machine. An idea began to spread: a vision of an interconnected world where information on most everyday objects was accessible.

Since then, scientists and engineers have developed this idea into a concept that is known as the “Internet of Things” (IoT). The idea started small, considering scenarios where radio-frequency identification allowed the “tagging” and managing of objects by computers. Each object would carry a radio-frequency identification (RFID) tag, a small, traceable chip which could be wirelessly scanned by a nearby RFID reader. The RFID tag enabled the automatic identification of the object and allowed it to be traced/managed through the Internet.

The continued advances in miniaturization allowed us to go beyond the simple tagging and identification of everyday objects. As predicted by Gordon Moore, back in 1965, the amount of computing power in integrated circuits has been doubling every 18 months for the last 50 years [20]. The remarkable work of computer industry engineers and scientists has led to many new technologies. The continuous integration of computational resources into all kinds of objects made our tools “intelligent”. Everything from light bulbs to refrigerators, microwaves, and coffee machines will soon be connected to the Internet. In fact, some studies estimate that we will have an IoT with 26 billion connected devices by 2020 [21].

We can see evidence of this trend all around us. The Internet now interconnects a large number of highly heterogeneous devices, from traditional desktop PCs to laptops, tablets, and smartphones.

For example, the area of sensing technologies and wireless sensor networks (WSNs) is becoming increasingly prominent. WSNs are composed of dozens or even hundreds of autonomous “sensor nodes”, small computerized devices that are capable of collecting physical world data and forwarding it by means of wireless communication. They can be used to monitor environmental luminosity, temperature, pressure, sound, and many other parameters, and can be spatially distributed in an ad hoc fashion. These technologies have been receiving a great deal of attention from the research community due to their potential in almost every application area. In fact, WSN deployments can now be found in many industrial, medical, and domestic environments. Recent studies in WSNs have brought great advancements in this area, namely in terms of energy efficiency and integration capabilities, with sensors being provided as services [22 23], accessible through the Internet [24]. Sensors are now indispensable devices, for they allow us to collect data from real-world phenomena, handle this data in digital form, and ultimately extend the Internet to the physical world.

In fact, the number of sensors that nowadays can be deployed on humans can turn them into walking sensor networks. Humans can use smart-shirts; carry a smartphone with several sensors and networking capabilities (e.g. global system for mobile communications (GSM), Bluetooth, long-term evolution (LTE)); and use Google glasses, iPods, smart watches, and shoes with sensors. In terms of sensing applied to individual users, Bosch Sensory Swarms and the Qualcomm Swarm Lab at UC Berkeley estimate that the number of sensors in personal devices can add up to 1000 wireless sensors per person, to be deployed over the next 10 to 15 years [25], resulting in large amounts of data being available for processing, and allowing a wide range of sensing applications to be deployed. This reality depends, of course, on the drastic reduction of sensor production costs, which are expected to come down to negligible values over time, as with most silicon-based hardware [26].

As for automated actuation, the world has seen a gradual increase in the number of installed robots per year. The 2015 World Robot Statistics study, issued by the International Federation of Robotics (IFR) [27], indicates that the total number of professional service robots sold in 2014 rose by 11.5% compared to 2013, from 21,712 to 24,207 units. IFR expects that, for the 2015–2018 period, sales of service robots for professional use will increase to about 152,375 units, while sales of robots for personal use will reach about 35 million units, with a total estimated value of about $40 billion. Global sales of industrial robots, on the other hand, will experience a yearly growth of 15% until 2018, while the number of sold units will double to around 400,000.

Interwoven with the concept of IoT is the concept of cyber-physical systems (CPSs), which consist in the sensing and control of physical phenomena through networks of devices that work together to achieve common goals. These CPSs represent a confluence of robotics, wireless sensor networks, mobile computing, and the IoT, to achieve highly monitored, easily controlled, and adaptable environments.

The IoT and CPS concepts have been pushed by two distinct communities. IoT was initially developed using a computer science perspective, mostly supported by the European Commission. The goal was to develop a network of smart objects with self-configuration capabilities on top of the current Internet. This development effort included hardware, software, standards, and interoperable communication protocols and languages that describe these intelligent devices [28]. IoT builds on several requirements, namely the development of intelligence in devices, interfaces and services; the assurance of security and privacy; systems integration; communication interoperability; and data “semantization” and management [29].

On the other hand, the concept of CPSs was initially supported by the US National Science Foundation (NSF). CPSs stem from an engineering perspective and concern the control and monitoring of physical environments and phenomena through sensing and actuation systems consisting of several distributed computing devices [30]. These systems are mostly interdisciplinary, requiring expertise and skills in mathematical abstractions (algorithms, processes) that model physical phenomena, smart devices and services, effective actuation, security and privacy, systems integration, communication, and data processing [31].

Thus, IoT tended to focus more on openness and the networking of intelligent devices, while CPSs were more concerned with applicability, modeling of physical processes, and problem solving, often through closed-looped systems. While their core philosophy and focus were initially different, their many similarities, such as intensive information processing, comprehensive intelligent services, and efficient interconnection and data exchange, have led to both terms being used interchangeably [32] without clearly identified borders [30].

CPSs combine elements from robotics, wireless sensor networks, and mobile computing, among others, to achieve specific goals. From industrial applications that monitor and actuate on several industrial processes, to social applications that aggregate data from various users in order to achieve goals, such as reducing pollution and traffic in metropolitan areas, CPSs can encompass a multitude of domains. For example, improvement of personal health can be achieved through body networks that integrate the user's vital signs and activity levels with environmental information on pollutants or noise to suggest healthier and more pleasant walking routes, restaurants, and leisure activities. CPSs can also be used in transportation, as many modern vehicles feature cruise control systems that maintain the automobile's speed or perform parking maneuvers, not to mention autonomous driving. All these systems combine sensors, actuators, and the computational capabilities of the devices to achieve the desired results. In fact, these sensors and actuators can be used not only in individual objects but also in structures and buildings in order to monitor, for example, their structural health.

While IoT and CPS technologies do exist, current systems are still designed with a specific scientific, industrial, or engineering application in mind. They are, for example, typically responsible for collecting data from sensors and analyzing it for a certain task. This objective-driven approach results in academic or industrial systems that may be effective for their targeted scenarios but are very constrained in applicability and, therefore, narrow in their usability.

Nevertheless, we know from previous experience that this happens during the beginning of any new paradigm-changing technology, as was the case with most information technologies. The most striking example is the Internet itself, which initially only connected the University of California at Los Angeles, the Augmentation Research Center at Stanford Research Institute, the University of California at Santa Barbara, and the University of Utah's Computer Science Department. This and other initial computer networks continued to grow and merge, giving birth to the Internet as we know it today.

Much like the Internet, it is very likely that existing disconnected and restricted CPS deployments, whose primary beneficiaries are privileged users who already benefit from and explore their capabilities, are just the initial steps towards a future where the vast majority of intelligent devices are interconnected in massive, non-centralized networks. In fact, some researchers argue that future CPSs will become ubiquitous and distributed, with many data streams overlaying the network, provided by large amounts of sensors. They also argue that these streams should be open for use, without centrally controlled authorization, through self-advertising and discovery by nearby users. Thus, data acquisition, processing, and visualization should be focused on users, not administrators or scientists [33].

1.2 Humans as Elements of Cyber-Physical Systems

The reduction of production costs of silicon-based hardware [34] continues to fuel this increasingly pervasive technological world, endowing people with the ability to access extremely rich and dynamic pools of data pertaining to their surrounding environment. The epitome of these ideas was first put forward by the renowned computer scientist Mark Weiser, in his famous 1991 article “The computer for the twenty-first century” [35]. Weiser maintained that, as devices became smaller in size, more powerful, and efficient, they would eventually disappear. Technology would become so intrinsic to everyday life that we would no longer perceive it as an isolated concept but as an inherent part of our existence. This idea came to be known as “ubiquitous computing”, and the concept of “calm” technology arose. This concept is a direct antitheses of the stressful use of technology, which is still prevalent. Each time we have to navigate menus, errors, bugs, or unintuitive setups, we become stressed by our computers and appliances. On the other hand, Weiser suggested that the true purpose of computers was to help us in a way similar to intuition. He propounded the view that the ideal computer would be something invisible that could truly understand human nature and interpret people's unconscious actions and desires. Instead of humans adapting to technology and learning how to use it, it would be technology that would adapt to the disposition of human beings.

Weiser also predicted that these “calm” interactions would be informative but not intrusive, not demanding the user's attention, and would make use of human intuitive clues. He was right, since we can see his vision materializing with every passing day. Computers no longer require people to sit in front of them; machines now enter the human environment embedded in all kinds of objects, integrating computing in the course of everyday human activities. In fact, current technology is quickly evolving towards these principles predicted 25 years ago: current mobile devices replaced traditional buttons with much more intuitive touchscreens, and software developers give an ever-increasing importance to usability and non-intrusiveness. The number of portable mobile devices has also grown exponentially, and the number of communication interfaces used by them has also grown. It is not hard to imagine a near future where we get up in the morning and our home also “wakes up” and automatically launches and executes many of the routines corresponding to that particular day of the week, under the control of several computing devices. In fact, as computation evolves, humans will most likely stop “using” computer devices, that is human–computer interaction will no longer require direct user attention and will become intuitive, as if it is second nature. In the words of Weiser, “The more you can do by intuition, the smarter you are; the computer should extend your unconscious.”

It is not sufficient for interconnected and intelligent tools to communicate with each other without any human involvement. Human technology is made by humans, for humans. In order to promote the creation of systems that are useful to the average person, it is necessary to consider efficient and intuitive operation. Therefore, in addition to providing basic functionality, openness, heterogeneity, and integration capabilities, it is equally important to discern how systems or tools can be used within a certain context.

These ideas have been previously explored as context awareness, or context adaptation, for mobile and wireless networking [36] and IoT [37]. Actually, increasing context awareness is a cross-cutting challenge for the design of highly optimized networking systems tha...

Table of contents

• Cover
• Title Page
• Copyright
• Dedication
• Table of Contents
• List of Figures
• List of Tables
• Foreword
• Preface
• Acknowledgments
• List of Abbreviations
• About the Companion Website
• Chapter 1: Introduction
• Part I: Evolution and Theory
• Part II: Human-in-the-Loop: Hands-On
• Part III: Future of Human-In-the-Loop Cyber-Physical Systems
• Appendix A: EmotionTasker's full code
• References
• Index
• End User License Agreement