My Experience with (Project) Tango

A practical introduction to consumer SLAM

This presentation available here:

 

https://goo.gl/XuYGur

About Me

• Background in Data Management & Enterprise Architecture
• Working on my third startup.
• Photo nerd
• Passionate about VR/AR*
• Have been playing around with volumetric capture for about 5-6 years
• Only recently gotten serious about it (2nd half of 2016)

About Constructive Realities  (link)

• Explorer - Kit with hardware and capture software
• WebVR based content editing
• Post-processing & Publishing
• More

Volumetric data capture and management solutions for home & business.  We mix hardware & software to make reality capture, processing and authorization easier and more affordable.

CV Terminology

Structured Light (Laser Scanning & Kinect)

ToF - Time of Flight  (Kinect V2 & Tango)

SfM (Structure from Motion)

(more) CV Terminology

SLAM - Simultaneous Localization and Mapping (wikipedia)

Loop Closure - (wikipedia)

Pose Estimate

"...updating a map of an unknown environment while simultaneously keeping track of an agent's location within it."

The system's best guess about the position and orientation of the device at a point in time.

The identification and integration new information path representing already-mapped area(s)*

Odometry  (wikipedia)

..."the use of data from motion sensors to estimate change in position over time."

Kinect V1

CV Toybox (Stuff i've Played With)

Structure.io                     (link)

Fab SCAN                                              (link)

Google Tango                  (link)

Artec EVA                          (link)

But Let's Focus on Tango

(disclaimer: mapping; not entertainment)

TLDR; Google Consumer SLAM

 

As Google as consumed the outside mapping work with Google Earth and Google Maps, it will cross the threshold and do the same for indoor spaces.

 

Tango is Google first step to this end.   

 

Takeaway is that Tango is not just a CV API, It also contains high level semantics to describe spaces (IA)

see: Area Description Manager

 

 

Inspirational Video (Link)

Lenovo Phab 2 Pro

• Snapdragon Proccessor
• Infineon ToF Sensor (PMD design) 
• 64GB Storage
• Worst RGB Camera on a current generation phone

Takeaway:  If you don't HAVE TO HAVE ONE NOW, likely much better to wait for the Asus Zenphone AR (link)

(mostly) Consistent  Tango API Implementation

• C++
• Java
• C# (Unity)

 

API Provides update callbacks for ket capture data like: 

• onFrameAvailable (2D)
• onPointCloudAvailable (3D)
• onPoseAvailable (6-DoF)

 

API

Tango (mapping) App Landscape - March 2017

Maybe a few other random ones...

*

RTAB MAP

powerful, adaptable, Extensible

• Open source Project
  • https://introlab.github.io/rtabmap/
  • Permissive BSD license if openCV w/o nonfree
• Prebuilt binaries forAndroid, Windows, Linux & MacOS
• Capable of saving all captured data (RBG/D/6DoF).
• Runs online and offline
• Many many options in offline mode
• Multi session!
• Mailing List, Wiki, Tutorials & More.

RTAP-MAP is *just* algorithm at the core of this project ; there is a lot of tooling built up around to facilitate capture, post & reprocessing.

 

It's a great tool

RTAB MAP - Overview

More About the algorithm

Links:

• Online Global Loop Closure Detection for Large-Scale Multi-Session Graph-Based SLAM
• Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation
• More about the loop closure approach

DayTime Exterior

Captured with prototype stabilizer system in mid day light (slightly cloudy) Capture time approximately 3 mins.

 

Indoor Room 

Haphazardly created. but colors worked out nicely

Examples 1

Bedroom

Captured with prototype stabilizer system in low illumination.  Capture time approximately 4 mins.

 

Shop

Single model created from 5 mapping sessions. Capture time ~10 minutes.

Examples 2

Bedroom

Handheld capture in ample sunlight Capture time approximately 2 mins.

 

Examples 3

Live Demo?