Section 1 – Getting Started with Augmented Reality
This section provides a basic introduction to developing AR applications and games with Unity, including AR technology concepts and how to use the Unity Editor. We cover the Unity XR plugin architecture, the AR Foundation toolkit, and other productivity tools. By the end of this section, you will be prepared to begin creating your own AR applications with Unity.
This section comprises the following chapters:
- Chapter 1, Setting Up for AR Development
- Chapter 2, Your First AR Scene
- Chapter 3, Improving the Developer Workflow
Chapter 1: Setting Up for AR Development
Augmented reality (AR) is widely recognized as the next-generation computing platform where digital content is seamlessly merged into real-world experiences. This book will help get you started with developing your own AR applications using the Unity 3D game engine and the AR Foundation toolkit provided by Unity.
In this chapter, you will take your first steps by setting up your computer for AR development using the Unity 3D game engine. We will begin by briefly defining augmented reality, thus setting the context for this industry and some of the basics of AR technology. We will then install the Unity software, the AR Foundation toolkit, and make sure your system has been set up to develop for Android and/or iOS mobile devices. Finally, we'll build and run a test scene to verify things are working as they should.
We will cover the following topics:
- Defining augmented reality
- Getting started with Unity, including installation and using Unity
- Preparing your project for AR development, including XR plugins, AR Foundation, Input System, and the Universal Render Pipeline
- Setting up for mobile development (Android ARCore and iOS ARKit)
Note for Experienced Readers
If you are already familiar with Unity, already have it installed on your system, and are set up to build for your iOS or Android mobile device, you may be able to skim through details related to those topics that are interspersed in this chapter.
Technical requirements
First, you need a PC or Mac that's capable of running Unity. The minimum requirements are not difficult; almost any PC or Mac today will be sufficient (see https://docs.unity3d.com/Manual/system-requirements.html).
If you are developing for iOS, you will need a Mac running OSX with the current version of XCode installed, and an Apple developer account. If you are developing for Android, you can use either a Windows PC or Mac. We will discuss this further throughout this chapter.
It is not practical to develop for AR without a device capable of running your application. For this chapter (and this book as a whole), you will need either an iOS device that supports Apple ARKit (search the web as Apple does not appear to publish a list; for example, https://ioshacker.com/iphone/arkit-compatibility-list-iphone-ipad-ipod-touch) or an Android device that supports ARCore (https://developers.google.com/ar/discover/supported-devices).
Because this chapter is largely about installing tools and packages according to your requirements, please work through the topics in this chapter for additional technical requirements and to learn how to install them. The GitHub repository for this book can be found at https://github.com/PacktPublishing/Augmented-Reality-with-Unity-AR-Foundation.
Defining Augmented Reality
According to the Merriam-Webster dictionary, the word augment means "to make greater, more numerous, larger, or more intense," while reality is defined as "the quality or state of being real." Considering this, we realize that "augmented reality" is all about using digital content to improve our real world to add better information, understanding, and value to our experiences.
Augmented reality is most commonly associated with visual augmentation, where computer-generated graphics are combined with actual real-world visuals. When using a handheld mobile phone or tablet, for instance, AR combines graphics with the on-screen video (I call this video see-through AR). Using wearable AR glasses, graphics are directly added to your visual field (optical see-through AR).
But AR is not simply a computer graphic overlay. In his acclaimed 1997 research report, A Survey of Augmented Reality (http://www.cs.unc.edu/~azuma/ARpresence.pdf), Ronald Azuma proposed that AR must meet the following characteristics:
- Combines the real and virtual: The virtual objects are perceived as real-world objects that are sharing the physical space around you.
- Interactive in real time: AR is experienced in real time, not pre-recorded. For example, cinematic special effects that combine real action with computer graphics do not count as AR.
- Registered in 3D: The graphics must be registered to real-world 3D locations. For example, a heads-up display (HUD) where information is simply overlayed in the visual field is not AR.
To register a virtual object in 3D, the AR device must have the ability to track its location in 3D space and map the surrounding environment to place objects in the scene. There are multiple technologies and techniques for positional and orientation tracking (together referred to as pose tracking), as well as environmental feature detection, including the following:
- Geolocation: GPS provides low-resolution tracking of your location on the Earth (GPS accuracy is measured in feet or meters). This is usually good enough for wayfinding in a city and identifying nearby businesses, for example, but not for more specific positioning.
- Image Tracking: Images from the device's camera can be used to match the predefined or real-time 2D images, such as QR code markers, game cards, or product packaging, to display AR graphics that track an image's pose (3D position and orientation) relative to the camera space.
- Motion Tracking: Using the device's camera and other sensors (including inertial measurement by IMU motion sensors), you can compute your position and orientation in 3D, and detect visually distinct features in the environment. Academically, you may see this referred to as Simultaneous Localization and Mapping (SLAM).
- Environme...
