Augmented Reality with Kinect

Rui Wang

• Publisher: Packt Publishing
• Year: 2013
• ISBN:
  9781849694384
• Edition: 1
• Topic: Computer Science
• Subtopic: Information Technology
• Index: Computer Science

In Detail

Microsoft Kinect changes the notion of user interface design. It differs from most other user input controllers as it enables users to interact with the program without touching the mouse or a trackpad. It utilizes motion sensing technology and all it needs is a real-time cameras, tracked skeletons, and gestures.

Augmented Reality with Kinect will help you get into the world of Microsoft Kinect programming with the C/C++ language. The book will cover the installation, image streaming, skeleton and face tracking, multi-touch cursors and gesture emulation. Finally, you will end up with a complete Kinect-based game.

Augmented Reality with Kinect will help you get into the world of Kinect programming, with a few interesting recipes and a relatively complete example. The book will introduce the following topics: the installation and initialization of Kinect applications; capturing color and depth images; obtaining skeleton and face tracking data; emulating multi-touch cursors and gestures; and developing a complete game using Kinect features.

The book is divided in such a way so as to ensure that each topic is given the right amount of focus. Beginners will start from the first chapter and build up to developing their own applications.

Approach

This book is a mini tutorial with plenty of code examples and strategies to give you many options when building your own applications.

Who this book is for

This book is meant for readers who are familiar with C/C++ programming and want to write simple programs with Kinect. The standard template library can also be used as it is simple enough to understand.

Augmented Reality with Kinect

---

Table of Contents

Augmented Reality with Kinect

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Getting Started with Kinect

Installation of Kinect

Setting up your Kinect software on PCs

Why did I do that?

The idea of the AR-based Fruit Ninja game

Summary

2. Creating Your First Program

Preparing the development environment

Building the Visual Studio project

Starting the device

Initializing and using Kinect in C++

Understanding the code

Additional information

Summary

3. Rendering the Player

Choosing image stream types

Obtaining color and depth images

Drawing color and depth as textures

Understanding the code

An incorrect way to combine depth and color

A traditional way for background subtraction

Understanding the code

Aligning color with depth

Generating a color image from depth

Understanding the code

Additional information

Using a green screen with Kinect

Making a magic photographer

Understanding the code

Additional information

Summary

4. Skeletal Motion and Face Tracking

Understanding skeletal mapping

Obtaining joint positions

Drawing the skeleton

Understanding the code

Drawing the linetrails following the hands

Drawing the path for specified joints

Understanding the code

Face tracking in Kinect

Detecting a face from the camera

Detecting and drawing the face rectangle

Understanding the code

Constructing the face model

Drawing the parametric face model

Understanding the code

Summary

5. Designing a Touchable User Interface

Multitouch systems

Locating the cursors

Drawing cursors from two hands

Understanding the code

Additional information

Common touching gestures

Recognizing holding and swiping gestures

Drawing cursors using two hands

Understanding the code

Additional information

Sending cursors to external applications

Emulating Windows mouse with cursors

Understanding the code

Summary

6. Implementing the Scene and Gameplay

Integrating the current code

Integrating existing elements in a scene

Understanding the code

Cutting the fruits

Adding and handling fruit objects

Understanding the code

Playing the game

Adding simple game logic

Understanding the code

Additional information

Summary

A. Where to Go from Here

libfreenect – the pioneer of Kinect middleware

OpenNI – a complete and powerful Kinect middleware

Free and open source resources

Commercial products using Kinect

Index

Augmented Reality with Kinect

Copyright © 2013 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: July 2013

Production Reference: 1040713

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-84969-438-4

www.packtpub.com

Cover Image by Suresh Mogre (<[email protected]>)

Credits

Author

Rui Wang

Reviewers

Ricard Borràs Navarra

Vangos Pterneas

Acquisition Editor

Kartikey Pandey

Commissioning Editors

Llewellyn Rozario

Priyanka Shah

Technical Editors

Sumedh Patil

Aniruddha Vanage

Copy Editors

Insiya Morbiwala

Alfida Paiva

Laxmi Subramanian

Project Coordinator

Amigya Khurana

Proofreader

Maria Gould

Indexer

Rekha Nair

Graphics

Abhinash Sahu

Production Coordinator

Conidon Miranda

Cover Work

Conidon Miranda

About the Author

Rui Wang is a Software Engineer at Beijing Crystal Digital Technology Co., Ltd. (Crystal CG), in charge of the new media interactive application design and development. He wrote a Chinese book called OpenSceneGraph Design and Implementation in 2009. He also wrote the book OpenSceneGraph 3.0 Beginner's Guide in 2010 and OpenSceneGraph 3.0 Cookbook in 2012, both of which are published by Packt Publishing and co-authored by Xuelei Qian. In his spare time he also writes novels and is a guitar lover.

I must express my deep gratitude to the entire Packt Publishing team for their great work in producing a series of high-quality Mini books, including this one, which introduces the cutting-edge Kinect-based development. Many thanks to Zhao Yang and my other colleagues at Crystal CG, for their talented ideas on some of the recipes in this book. And last but not the least, I'll extend my heartfelt gratitude to my family for their love and support.

About the Reviewers

Ricard Borràs Navarra has always been working around computer vision and machine learning. He started creating machines that apply pattern recognition to quality assortment in the cork production industry. Later, he applied these techniques for human tracking in complex scenarios, creating audience measurement, and people-counter solutions for retail stores.

With the eruption of Kinect, he started working on and deploying augmented reality interactive applications based on this great device. These applications were targeted at the marketing and retail sectors.

All these projects were developed by him as an Inspecta (www.inspecta.es) employee. Also, Ricard has developed several freelance projects based on augmented reality.

Vangos Pterneas is a professional Software Engineer, passionate about natural user interfaces. He has been a Kinect enthusiast ever since the release of the very first unofficial SDKs and has already published a couple of commercial Kinect applications.

He has worked for Microsoft Innovation Center as a .NET developer and consultant and he's now running his own company...