Emerging Extended Reality Technologies for Industry 4.0

Early Experiences with Conception, Design, Implementation, Evaluation and Deployment

Jolanda G. Tromp, Dac-Nhuong Le, Chung Van Le, Jolanda G. Tromp, Dac-Nhuong Le, Chung Van Le

• Publisher: Wiley-Scrivener
• Year: 2020
• Print ISBN: 9781119654636
• eBook ISBN: 9781119654735
• Edition: 1
• Topic: Computer Science
• Subtopic: Artificial Intelligence (AI) & Semantics
• Index: Computer Science

In the fast-developing world of Industry 4.0, which combines Extended Reality (XR) technologies, such as Virtual Reality (VR) and Augmented Reality (AR), creating location aware applications to interact with smart objects and smart processes via Cloud Computing strategies enabled with Artificial Intelligence (AI) and the Internet of Things (IoT), factories and processes can be automated and machines can be enabled with self-monitoring capabilities. Smart objects are given the ability to analyze and communicate with each other and their human co-workers, delivering the opportunity for much smoother processes, and freeing up workers for other tasks. Industry 4.0 enabled smart objects can be monitored, designed, tested and controlled via their digital twins, and these processes and controls are visualized in VR/AR. The Industry 4.0 technologies provide powerful, largely unexplored application areas that will revolutionize the way we work, collaborate and live our lives. It is important to understand the opportunities and impact of the new technologies and the effects from a production, safety and societal point of view.

PART I
EXTENDED REALITY EDUCATION

The stakeholders in the AIIoT simulation-based optimization of planning, processing and delivery of operations, are the following three human user groups: human society, human operators, and human developers of the systems. Based on the exponential growth and all-pervasiveness of the AIIoT technologies that are embedded throughout our processes and will be driving our systems, it is rapidly clearly becoming more urgent to prepare a labor force with the required digital skills at all levels of education and training in order to be able to harness and benefit from the digital AIIoT transformations.

New job categories will arise with tasks that require technical capabilities and soft skills – essential human skills to manage the errors and problem solving that machines cannot handle. Governments and companies must plan to accelerate the creation of industrial engineering jobs dedicated to 3D modeling, 3D simulations, big data analytics, ML, robotics and development and customization of integrations of AIIoT-driven simulations and robotics solutions. Chapter 1 presents the results from an international survey regarding the use of XR technologies in the classroom to deliver classes, to teach the development of XR technologies and to research XR technologies, and a summary of the lessons learned.

There is a global need for skilled engineers and operators in order to research, build, test, deploy and maintain these new AIIoT-driven products, services, machines and platforms. To achieve positive economics for investment, robots must replace humans on the work floor, rather than support them. Routine manual activities can become fully automated. Routine and non-routine human activities will change, and the share of non-routine activities will increase for the human operator. Manual work will shift towards non-routine tasks, which means that workers must acquire more advanced skills. Chapter 2 presents a use-case study of a XR e-Health, e-Learning application for teaching anatomy, showing how disruptive new technologies can be to traditional education and accelerating opportunities for learning.

• Chapter 1: Mixed Reality Use in Higher Education: Results from an International Survey
• Chapter 2: Using 3D Simulation in Medical Education

CHAPTER 1
MIXED REALITY USE IN HIGHER EDUCATION: RESULTS FROM AN INTERNATIONAL SURVEY

J. Riman¹, N. Winters², J. Zelenak³, I. Yucel⁴, J. G. Tromp^5,*

¹ SUNY Fashion Institute of Technology, New York, New York, USA

² SUNY Delhi College of Technology, Delhi, New York, USA

³ University at Albany - State University of New York, Albany, New York, USA

⁴ SUNY Polytechnic Institute, Utica, New York, USA

⁵ Duy Tan University, Da Nang, Vietnam

* Corresponding author: [email protected]

Abstract

Respondents identified some challenges in implementing mixed reality in their work with a majority reporting student reluctance, faculty reluctance, and lack of infrastructure and hardware as significant challenges. There was a significant reduction in perceived value added by the research respondents. Poor user experience, difficult to use hardware and software, and lack of educational content were among the lowest ranked challenges.

Keywords: Mixed reality, survey, community of practice

1.1 Introduction

Mixed reality (MR) is comprised of augmented reality (AR), virtual reality (VR) and arguably, 360-degree video. AR and VR are in use in numerous commercial applications from Pokemon GO to the New York Times. These tools have serious implications for higher education in areas that include virtual labs, student engagement, and student success and retention [2-5]. The State University of New York (SUNY) FACT² Mixed Reality Task Group was charged with exploring the use of Mixed Realities in the higher education setting and analyze the opportunities they offer to enhance the teaching, learning, and professional development experiences of students and faculty using paths of inquiry [1] such as: What are the opportunities for these emergent tools to be integrated into higher education outcomes? What training, tools and hardware are needed to initiate and support integration into teaching and learning? Describe the learning curve to optimize course and degree outcomes. Is there enough research and experience to frame the potential benefits of these tools in fully online, hybrid and conventional modalities?

Additionally, the task group sought to recruit collaborators from SUNY and beyond (faculty, instructional designers, content and product manufacturers) who have subject matter expertise and experience with the goal of augmenting and expanding teaching and learning opportunities that can be sustained as a community of practice (CoP). Effective strategies for creating and sustaining a CoP were researched and a special group was created to explore the tools and methods being developed to support course and degree outcomes and lay the groundwork for a CoP.

The task group met a total of 20 times throughout the 2017-2018 academic year and consisted of 18 initial members. The task group developed a survey to investigate the current uses of MR in higher education and research. The survey was circulated internationally. The results are reported here.

1.2 Organizational Framework

The following organizational framework was created to coordinate the work of the task group:

• Action research with volunteer faculty, instructional designers and students.
• Teaching: What can/should we do now, in the near or distant future in the education space?
• Learner: How will learners of all ages use these tools in and out of the learning space?
• New Skills: How will these new skills change certifi...

Table of contents

• Cover
• Table of Contents
• List of Figures
• List of Tables
• Foreword
• Introduction
• Preface
• Acknowledgments
• Acronyms
• PART I: EXTENDED REALITY EDUCATION
• PART II: INTERNET OF THINGS
• PART III: MOBILE TECHNOLOGY
• PART IV: TOWARDS DIGITAL TWINS AND ROBOTICS
• PART V: BIG DATA ANALYTICS
• PART VI: TOWARDS COGNITIVE COMPUTING
• Also of Interest
• End User License Agreement