Augmented Reality with Unity AR Foundation

A practical guide to cross-platform AR development with Unity 2020 and later versions

Jonathan Linowes

• Publisher: Packt Publishing
• Year: 2021
• ISBN:
  9781838982966
• Edition: 1
• Topic: Ciencia de la computación
• Subtopic: Programación de juegos

Explore the world of augmented reality development with the latest features of Unity and step-by-step tutorial-style examples with easy-to-understand explanations

Key Features

• Build functional and interactive augmented reality applications using the Unity 3D game engine
• Learn to use Unity's XR and AR components, including AR Foundation and other standard Unity features
• Implement common AR application user experiences needed to build engaging applications

Book Description

Augmented reality applications allow people to interact meaningfully with the real world through digitally enhanced content.

The book starts by helping you set up for AR development, installing the Unity 3D game engine, required packages, and other tools to develop for Android (ARCore) and/or iOS (ARKit) mobile devices. Then we jump right into the building and running AR scenes, learning about AR Foundation components, other Unity features, C# coding, troubleshooting, and testing. We create a framework for building AR applications that manages user interaction modes, user interface panels, and AR onboarding graphics that you will save as a template for reuse in other projects in this book. Using this framework, you will build multiple projects, starting with a virtual photo gallery that lets you place your favorite framed photos on your real-world walls, and interactively edit these virtual objects. Other projects include an educational image tracking app for exploring the solar system, and a fun selfie app to put masks and accessories on your face. The book provides practical advice and best practices that will have you up and running quickly.

By the end of this AR book, you will be able to build your own AR applications, engaging your users in new and innovative ways.

What you will learn

• Discover Unity engine features for building AR applications and games
• Get up to speed with Unity AR Foundation components and the Unity API
• Build a variety of AR projects using best practices and important AR user experiences
• Understand the core concepts of augmented reality technology and development for real-world projects
• Set up your system for AR development and learn to improve your development workflow
• Create an AR user framework with interaction modes and UI, saved as a template for new projects

Who this book is for

This augmented reality book is for game developers interested in adding AR capabilities to their games and apps. The book assumes beginner-level knowledge of Unity development and C# programming, familiarity with 3D graphics, and experience in using existing AR applications. Beginner-level experience in developing mobile applications will be helpful to get the most out of this AR Unity book.

Section 1 – Getting Started with Augmented Reality

This section provides a basic introduction to developing AR applications and games with Unity, including AR technology concepts and how to use the Unity Editor. We cover the Unity XR plugin architecture, the AR Foundation toolkit, and other productivity tools. By the end of this section, you will be prepared to begin creating your own AR applications with Unity.

This section comprises the following chapters:

• Chapter 1, Setting Up for AR Development
• Chapter 2, Your First AR Scene
• Chapter 3, Improving the Developer Workflow

Chapter 1: Setting Up for AR Development

Augmented reality (AR) is widely recognized as the next-generation computing platform where digital content is seamlessly merged into real-world experiences. This book will help get you started with developing your own AR applications using the Unity 3D game engine and the AR Foundation toolkit provided by Unity.

In this chapter, you will take your first steps by setting up your computer for AR development using the Unity 3D game engine. We will begin by briefly defining augmented reality, thus setting the context for this industry and some of the basics of AR technology. We will then install the Unity software, the AR Foundation toolkit, and make sure your system has been set up to develop for Android and/or iOS mobile devices. Finally, we'll build and run a test scene to verify things are working as they should.

We will cover the following topics:

• Defining augmented reality
• Getting started with Unity, including installation and using Unity
• Preparing your project for AR development, including XR plugins, AR Foundation, Input System, and the Universal Render Pipeline
• Setting up for mobile development (Android ARCore and iOS ARKit)
  Note for Experienced Readers
  If you are already familiar with Unity, already have it installed on your system, and are set up to build for your iOS or Android mobile device, you may be able to skim through details related to those topics that are interspersed in this chapter.

Technical requirements

First, you need a PC or Mac that's capable of running Unity. The minimum requirements are not difficult; almost any PC or Mac today will be sufficient (see https://docs.unity3d.com/Manual/system-requirements.html).

If you are developing for iOS, you will need a Mac running OSX with the current version of XCode installed, and an Apple developer account. If you are developing for Android, you can use either a Windows PC or Mac. We will discuss this further throughout this chapter.

It is not practical to develop for AR without a device capable of running your application. For this chapter (and this book as a whole), you will need either an iOS device that supports Apple ARKit (search the web as Apple does not appear to publish a list; for example, https://ioshacker.com/iphone/arkit-compatibility-list-iphone-ipad-ipod-touch) or an Android device that supports ARCore (https://developers.google.com/ar/discover/supported-devices).

Because this chapter is largely about installing tools and packages according to your requirements, please work through the topics in this chapter for additional technical requirements and to learn how to install them. The GitHub repository for this book can be found at https://github.com/PacktPublishing/Augmented-Reality-with-Unity-AR-Foundation.

Defining Augmented Reality

According to the Merriam-Webster dictionary, the word augment means "to make greater, more numerous, larger, or more intense," while reality is defined as "the quality or state of being real." Considering this, we realize that "augmented reality" is all about using digital content to improve our real world to add better information, understanding, and value to our experiences.

Augmented reality is most commonly associated with visual augmentation, where computer-generated graphics are combined with actual real-world visuals. When using a handheld mobile phone or tablet, for instance, AR combines graphics with the on-screen video (I call this video see-through AR). Using wearable AR glasses, graphics are directly added to your visual field (optical see-through AR).

But AR is not simply a computer graphic overlay. In his acclaimed 1997 research report, A Survey of Augmented Reality (http://www.cs.unc.edu/~azuma/ARpresence.pdf), Ronald Azuma proposed that AR must meet the following characteristics:

• Combines the real and virtual: The virtual objects are perceived as real-world objects that are sharing the physical space around you.

1. Interactive in real time: AR is experienced in real time, not pre-recorded. For example, cinematic special effects that combine real action with computer graphics do not count as AR.
   • Registered in 3D: The graphics must be registered to real-world 3D locations. For example, a heads-up display (HUD) where information is simply overlayed in the visual field is not AR.

To register a virtual object in 3D, the AR device must have the ability to track its location in 3D space and map the surrounding environment to place objects in the scene. There are multiple technologies and techniques for positional and orientation tracking (together referred to as pose tracking), as well as environmental feature detection, including the following:

• Geolocation: GPS provides low-resolution tracking of your location on the Earth (GPS accuracy is measured in feet or meters). This is usually good enough for wayfinding in a city and identifying nearby businesses, for example, but not for more specific positioning.
• Image Tracking: Images from the device's camera can be used to match the predefined or real-time 2D images, such as QR code markers, game cards, or product packaging, to display AR graphics that track an image's pose (3D position and orientation) relative to the camera space.
• Motion Tracking: Using the device's camera and other sensors (including inertial measurement by IMU motion sensors), you can compute your position and orientation in 3D, and detect visually distinct features in the environment. Academically, you may see this referred to as Simultaneous Localization and Mapping (SLAM).
• Environme...