Learn ARCore - Fundamentals of Google ARCore
eBook - ePub

Learn ARCore - Fundamentals of Google ARCore

Micheal Lanham

Buch teilen
  1. 274 Seiten
  2. English
  3. ePUB (handyfreundlich)
  4. Über iOS und Android verfügbar
eBook - ePub

Learn ARCore - Fundamentals of Google ARCore

Micheal Lanham

Angaben zum Buch
Buchvorschau
Inhaltsverzeichnis
Quellenangaben

Über dieses Buch

Create next-generation Augmented Reality and Mixed Reality apps with the latest version of Google ARCore

Key Features

  • Harness the power of the Google's new augmented reality (AR) platform ARCore to build cutting-edge Augmented reality apps
  • Learn core concepts of Environmental Understanding, Immersive Computing, and Motion Tracking with ARCore
  • Extend your application by combining ARCore with OpenGL, Machine Learning and more.

Book Description

Are you a mobile developer or web developer who wants to create immersive and cool Augmented Reality apps with the latest Google ARCore platform? If so, this book will help you jump right into developing with ARCore and will help you create a step by step AR app easily.

This book will teach you how to implement the core features of ARCore starting from the fundamentals of 3D rendering to more advanced concepts such as lighting, shaders, Machine Learning, and others.

We'll begin with the basics of building a project on three platforms: web, Android, and Unity. Next, we'll go through the ARCore concepts of motion tracking, environmental understanding, and light estimation. For each core concept, you'll work on a practical project to use and extend the ARCore feature, from learning the basics of 3D rendering and lighting to exploring more advanced concepts.

You'll write custom shaders to light virtual objects in AR, then build a neural network to recognize the environment and explore even grander applications by using ARCore in mixed reality. At the end of the book, you'll see how to implement motion tracking and environment learning, create animations and sounds, generate virtual characters, and simulate them on your screen.

What you will learn

  • Build and deploy your Augmented Reality app to the Android, Web, and Unity platforms
  • Implement ARCore to identify and visualize objects as point clouds, planes, surfaces, and/or meshes
  • Explore advanced concepts of environmental understanding using Google ARCore and OpenGL ES with Java
  • Create light levels from ARCore and create a C# script to watch and propagate lighting changes in a scene
  • Develop graphics shaders that react to changes in lighting and map the environment to place objects in Unity/C#
  • Integrate motion tracking with the Web ARCore API and Google Street View to create a combined AR/VR experience

Who this book is for

This book is for web and mobile developers who have broad programming knowledge on Java or JavaScript or C# and want to develop Augmented Reality applications with Google ArCore. To follow this book no prior experience with AR development, 3D, or 3D math experience is needed.

Häufig gestellte Fragen

Wie kann ich mein Abo kündigen?
Gehe einfach zum Kontobereich in den Einstellungen und klicke auf „Abo kündigen“ – ganz einfach. Nachdem du gekündigt hast, bleibt deine Mitgliedschaft für den verbleibenden Abozeitraum, den du bereits bezahlt hast, aktiv. Mehr Informationen hier.
(Wie) Kann ich Bücher herunterladen?
Derzeit stehen all unsere auf Mobilgeräte reagierenden ePub-Bücher zum Download über die App zur Verfügung. Die meisten unserer PDFs stehen ebenfalls zum Download bereit; wir arbeiten daran, auch die übrigen PDFs zum Download anzubieten, bei denen dies aktuell noch nicht möglich ist. Weitere Informationen hier.
Welcher Unterschied besteht bei den Preisen zwischen den Aboplänen?
Mit beiden Aboplänen erhältst du vollen Zugang zur Bibliothek und allen Funktionen von Perlego. Die einzigen Unterschiede bestehen im Preis und dem Abozeitraum: Mit dem Jahresabo sparst du auf 12 Monate gerechnet im Vergleich zum Monatsabo rund 30 %.
Was ist Perlego?
Wir sind ein Online-Abodienst für Lehrbücher, bei dem du für weniger als den Preis eines einzelnen Buches pro Monat Zugang zu einer ganzen Online-Bibliothek erhältst. Mit über 1 Million Büchern zu über 1.000 verschiedenen Themen haben wir bestimmt alles, was du brauchst! Weitere Informationen hier.
Unterstützt Perlego Text-zu-Sprache?
Achte auf das Symbol zum Vorlesen in deinem nächsten Buch, um zu sehen, ob du es dir auch anhören kannst. Bei diesem Tool wird dir Text laut vorgelesen, wobei der Text beim Vorlesen auch grafisch hervorgehoben wird. Du kannst das Vorlesen jederzeit anhalten, beschleunigen und verlangsamen. Weitere Informationen hier.
Ist Learn ARCore - Fundamentals of Google ARCore als Online-PDF/ePub verfügbar?
Ja, du hast Zugang zu Learn ARCore - Fundamentals of Google ARCore von Micheal Lanham im PDF- und/oder ePub-Format sowie zu anderen beliebten Büchern aus Informatique & Développement Web. Aus unserem Katalog stehen dir über 1 Million Bücher zur Verfügung.

Information

Jahr
2018
ISBN
9781788833639

Recognizing the Environment

Throughout this book, we have looked at the numerous ways of how our device, with the help of ARCore, can track the user, understand the user's world, and render an alternate reality. ARCore uses the device's sensors and camera as inputs to constantly update what it perceives as the user's real world. However, what if we wanted to do more for the user; perhaps identify a certain object, sign, or landmark? That would require a much more advanced set of tools. Even just 5 years ago, this would seem like an incredibly daunting task. With the advent of OpenAI, thanks to Mr. Musk, many other companies have started to open source and make their tools available. This has led to phenomenal explosive growth in these technologies, colloquially referred to as Machine Learning (ML), and broadened their accessibility to everyone. Fortunately, for those interested in developing AR apps, this is a good thing. We want all the help we can get when it comes to recognizing and understanding the user's environment.
For this chapter, we will introduce ML and explore how we can use it to create better AR apps for our users. In this chapter, we will cover the following topics:
  • Introduction to ML
  • Deep reinforcement learning
  • Programming a neural network
  • Training a neural network
  • TensorFlow
Machine Learning is a very advanced subject that can take years of study in order to master. However, for our purposes, we will learn some basic techniques, which the reader can extend on later, either through more learning or implementing their own solution.
If you already have an in-depth understanding of neural networks, convolutional neural networks, and TensorFlow, feel free to breeze over this chapter.

Introduction to ML

Machine Learning is a term widely used to refer to artificial intelligence and related computer predictive analytical models. The name Machine Learning, while perhaps overly generalized, fits better than the term AI. However, Machine Learning is itself such a broad term that it perhaps needs some further explanation and clarification. A machine obviously refers to a computer, or other device and learning tends to denote an algorithm or model that will evolve or learn over time. However, this is often not the case in many Machine Learning models. Therefore, for our purposes, we will use the broader term of Machine Learning to refer to any tool or algorithm that can be trained to recognize the environment or parts of the environment in AR, thus allowing us, the developers, to better augment our user's world.
Data science and Machine Learning go hand in hand. Data science is all about making sense of data, extracting patterns, and making predictions. In essence, when you start writing Machine Learning models in order to recognize objects or the environment, you are really just analyzing data, which means you can also, very loosely, call yourself a data scientist.
Machine Learning is a big area and is only getting bigger every day, so let's break down the specific problems we would like ML to help us with:
  • Target detection: Targets have been used in AR for some time. It has been the primary tracking and reference point for many AR apps previous to ARCore.
  • Image recognition: This spawns into a whole set of sub-applications, all of which we will deal with in detail later.
  • Object detection: Being able to detect an object in 3D from point cloud data is no easy feat, but it has been done and is getting better.
  • Face detection: Detecting a person's face in an image has been around for years and has been used to great effect in many apps.
  • Person detection: Detecting people or motion has great possibilities. Think Kinect comes to AR.
  • Hand/Gesture detection: Not to be confused with touch gestures. This is where we detect a user's hand motions or gestures in front of a device's camera.
  • Pose detection on object: Related to object detection, but now we also detect the position and orientation of the object.
  • Light source detection: Being able to place realistic lights in a scene to make virtual object rendering more realistic. We already looked at the importance of lighting in Chapter 7, Light Estimation.
  • Environment detection: Recognizing the environment a user has moved into has great application in mapping buildings or other locations where GPS is unavailable, which applies to most internal spaces.
Each of those problems may require different tools and techniques to solve those issues. In ML, it's not always about using the tool but the final answer and what works. Think about this as you build any ML you need for your app. Try a variety of ML tools and techniques; differences in size and performance of ML models can be critical, and it's something you need to consider.
A Machine Learning algorithm walks into a restaurant.
The waiter asks, "What will you have?
The algorithm says, "What's everyone else having?"
- Unknown
In the following table is a summary of the current major ML providers and the types of AR problems they can be used to solve:
Toolset Pros/Cons
Machine Learning task
Targets/Image Object/Pose Face Person Hand Light Environment
Vuforia Mature and easy to use. Requires internet connectivity. Yes Yes/Paid
XZIMG Face and image/target tracking supported for Unity and other platforms. Yes Yes
ARToolkit Mature OpenSource platform for image tacking and feature detection. Yes
EasyAR Pro license gets object and feature tracking. Yes Yes/Paid
Google Face Detection API Low level Android API. Yes
OpenCV A mature low-level API for Android, commercial version ported to Unity. Still requires low level knowledge. Yes Yes Yes Yes Yes Coming Coming
Google TensorFlow Still in its infancy but quickly becoming the platform standard for CNN. Low level and advanced ML knowledge required. Yes Yes Yes Yes Yes coming coming
Google ARCore Currently, identifies planes, feature points, and light. Yes Yes
We only included the main players who have built an AR platform for a mobile ARCore-supported device. Web technologies were omitted from this due to their limitations, although many of the mentioned technologies require internet connectivity and support web platforms as well. If you quickly review the table, you can also clearly see two main contenders that have the potential to dominate the entire space; that's because these are both low-level technologies that often back larger platforms such as Vuforia. Both of these platforms now support mobile pretrained networks for fast recognition on mobile devices. This may not seem like a big deal yet, but after we get into training our own models, you will see why.

Linear regression explained

Let's discuss the basic premise behind what Machine Learning is and what it attempts to accomplish. Take a look at the following chart that shows some fictional sales data for your next app:

Chart of fictional sales data
Now, just looking at the chart, you can see that as the x values increase (perhaps days on sale), it appears that our sales also increase: y value (sales). By just eyeing the chart, we ourselves can make predictions by following the trend of the points. Try it; how many sales are for an x value (bottom axis) of 25? Give it a guess, and write it down. With your guess secured, we will use a technique called linear regression to find a good answer.
Linear regression has been around for years and is considered as the base for many statistical data analysis methods. It is the basis for many other Machine Learning algorithms used in data science and predictive analysis today. This technique works by finding a solution (a line, curve, or whatever) that...

Inhaltsverzeichnis