Real world

trigger/tracker

Digital world

overlay

Augmented Reality

Device

Triggers / Trackers

Marker

 

Classifier

 

SLAM

 

Geolocation

Marker

Specific image as point of reference for detection and tracking to attach your overlay

Classifier

Using image recognition to determine general categories

TensorFlow

Open source library for general machine learning

 

Inception-V3 - deep convolutional neural network (CNN)

 

ImageNet - Currently 14,197,122 images

SLAM

Simultaneous Localization and Mapping

 

Map the environment while tracking your device location inside of it

 

NO marker needed - just attach your digital overlay to a wall, floor, table, etc

 

Use changes in Inertial Measurement Unit (IMU) data to keep your overlay in place

Geolocation

Launch overlay based on real world physical location

Devices

Standard Camera Handheld Devices (iPhone, Galaxy, Pixel)

 

3D Camera Handheld Devices (Tango, Structure Sensor)

 

Head Mounted Displays (Meta, Hololens, Magic Leap, ODG, Daqri, Epson, Contact lens)

 

Inputs (Camera, Leap Motion, OpenBCI, game controllers, UltraHaptics)

Digital Overlay

Code

 

Powerful Desktop/Web Based Editors

 

Simple Mobile Creation

Code the AR Digital Overlay

Graphics APIs: OpenGL, WebGL, Vulkan, DirectX (Microsoft), Metal (Apple)

 

Tool APIs - UnityScript, Blippbuilder Script, Blender Python API 

 

3D file formats: MD2, Draco, COLLADA, OBJ, FBX, GLTF

Edit the AR Digital Overlay

Unity

 

Blippbuilder

 

ZapWorks

 

Aurasma

 

Wikitude Studio

AR vs VR vs MR?

Important thing is that we are building and delivering 3D content that exists in a 3D environment

 

Virtual Reality is a closed simulation

 

Augmented Reality places virtual objects in the physical world

 

Mixed Reality brings physical objects into the virtual world (and includes AR)

 

Content created should work in both environments!

Challenges to Solve to Bring AR Mainstream

Not enough content

 

Real world object recognition technology needs to get better

 

Real world object interaction (SLAM) needs to get better on current devices

 

Ubiquity of 3D cameras and head mounted displays

 

Real time content delivery

 

Social acceptance