Building AR apps

in React Native

@VladimirNovick

GDE & Independent Consultant

Web / Mobile / VR / AR / IoT

vnovick.com

Augmented Reality

Why AR is the future

It's reality 2.0 version

1901 First recorded AR reference 
(L. Frank Baum - "The Master Key")
1952 Sensorama
Morton Heilig
1968 Sword of Damocles

Ivan Sutherland

1982 AR for weather broadcasters
Dan Reitan
1992 Virtual Fixtures
Louis Rosenberg

1996 First AR markers
​CyberCode

How it all started

1998 – NASA develops AR dashboard for X38
1999 Eye tap - wearable AR glasses
Steve Mann
​
2008 AR used by BMW for commercial purposes

2010 – Kinnect

2012 – Google Glass​
1996 First AR markers
​CyberCode

90's - 2000's

2013 – AR is used in car manufacturing
2016 AR/VR investment reaches 1.1 billion 
​
2016 Microsoft Hololens/Meta 2

2016 – Pokemon Go

2017 – Apple anounced ARKit, Google - ARCore
2018 Magic Leap (light field technology)

Recent years

How we can use AR today

AR for Retail

  • Preview products before buying
  • Get deals based on physical markers
  • Get additional info about products
  • Share product images with friends and family before purchasing

AR for Games

  • Interact with digital world by leveraging physical environments
  • Much more addictive
  • Social sharing is easier and more encouraged
  • Explore physical world with digital overlay

AR for Manufacturing

  • Get valuable information without distraction
  • Instructional videos can be more immersive
  • Various set of tools can be used digitally

Location based AR

  • Get information based on intersection between location and image recognition
  • Use markers for displaying additional location info and data
  • Help user with more precise navigation and orientation
  • Ads and suggestions have physical context

How we can get started

AR libraries we can use

  • react-native-arkit / react-native-arcore
  • Unity3d, Unreal
  • ViroReact

How to get started

  • get your Viro api key from viromedia.com 
  • npm install -g react-viro-cli
  • react-viro init MyAwesomeARApp

It's Demo time

https://github.com/vnovick/armonster-demo

Basic anatomy of an app

wraps every AR scene

setting up lights

positioning UI

a helper to select AR surfaces

Coordinate system

Right handed coordinate system

  • Positive Z is behind the viewer
  • Axis directions are changed when applied rotation prop
  • To determine positive rotation use "thumb" rule

Lights and Materials Demo

Ambient light

"Sun-like light"

Spotlight

Materials

  • diffuseTexture
  • specularTexture
  • normalTexture

Textures

3D Models

  • .Obj files
  • .vrx files  fbx models transformed with  ./bin/ViroFBX script
  • GLTF files

3d model primitive

loading resources

Skeletal animations

Rendering shadows

Detecting AR planes

Portals

Portal scene wrapper

Portal wrapper

Portal 3d model

360 image

Particles

ARKit 2 Demo

https://github.com/vnovick/armonster-arkit2-businesscard

Where to get 3d stuff

When to develop AR apps with Viro?

When it's an app and not a full blown AR game

Start building AR apps today

Thanks

@VladimirNovick

vnovick.com