The state of Augmented Reality and React Native in 2023

Software Architect & Consultant

Web / Mobile / XR / IoT / Web3 / AI

Augmented Reality

1901 First recorded AR reference 
(L. Frank Baum - "The Master Key")
1952 Sensorama
Morton Heilig
1968 Sword of Damocles 

Ivan Sutherland

1982 AR for weather broadcasters
Dan Reitan
1992 Virtual Fixtures
Louis Rosenberg
1996 First AR markers
​CyberCode
1998 – NASA develops AR dashboard for X38
1999 Eye tap - wearable AR glasses
Steve Mann
2008 AR used by BMW for commercial purposes

2010 – Kinnect

2012 – Google Glass​

90's - 2000's

2010-2020

2013 – AR is used in car manufacturing
2016 AR/VR investment reaches 1.1 billion 
2016 Microsoft Hololens/Meta 2

2016 – Pokemon Go

2017 – Apple anounced ARKit, Google - ARCore
2018 Magic Leap (light field technology)

ARKit / ARCore

AR with React Native

What we can do with it?

Anatomy of the scene

AR Scene

props passed and are accessible within scene component

Top level component

AR Scene

Lighting

Plane selector

AR Component

So is AR just a gimmick?

Let's see what it is capable of!

Features for mobile

https://unity.com/features/mobile

React Native and Unity

JS

Native

Unity as a library

Demo

GameObject in Unity

Method name to call

Payload

In Unity

Native iOS Plugin

Unity Native Plugins (iOS)

NativeCallProxy.h

In Unity Project under Assets/Plugins/iOS create two files

Unity Native Plugins (iOS)

NativeCallProxy.mm

Sending message to Native in Unity

ButtonBehavior.cs

public static extern void sendMessageToMobileApp(string message);
NativeAPI.sendMessageToMobileApp("The button has been tapped!");

Assign script to gameObject in Unity and  click handler to button in Unity

Receiving message from RN in Unity

We need to parse JSON we receive from RN

RNInterop.cs
RNInterop.cs

RN Integration steps (iOS)

Build Unity Project for iOS into folder within your RN project

unity/builds/ios

Open Xcode workspace of your React Native project

ios/rnunitynative.xcworkspace

right click and add Files to workspace. Select Unity Project

Add UnityFramework to Embedded Frameworks,

Remove UnityFramework from Build Phases Link Binary with Libraries

Drag and drop Embed Frameworks before Compile sources

Add Privacy - Camera Usage Description key  to Info.plist

1. Change Data membership to be UnityFramework

2.Change build products path for UnityFramework to be your RN ios build folder

Do it every time after you build Unity Project

3.Change Native Plugins Target membership to UnityFramework & Public

4.Build UnityFramework separately

Integrate with React Native

Create

ReactNativeUnityViewManager.mm
Expose module to JS
Export view property to be callback​
Expose postMessage method to JS
Send events to JS
Make sure we run on main thread
UnityFramework API to call method by name
ReactNativeUnityView.mm
Framework wrapper to handle loading and options

Getting back to JS side

Let's get to the cool news.

There is a library for that! 😉

No need to write Native code

Still need to do all the setup and build steps

Now you know how it actually works 🎉

Helpful links

Demo code

Learn unity

Unity Asset store

Unity AR Samples

Thank You

The state of Augmented Reality and React Native in 2023

By Vladimir Novick

The state of Augmented Reality and React Native in 2023

  • 45