Augmented Reality

with Ionic 2

Marcell Kiss

martzellk

marcellkiss@budacode.com

Developer

Organizer

Founder

About AR

"a technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view."

"Extending the view of the real world with any information"

Useful or Entertaining

AR Industries

Apps?

Industries?

PokemonGO...

Navigation

Education

Healthcare

Military

Sports

Games

Promotion

Interior Design

Types of AR

Screens

- Glasses

- External screen (mobile, tv, etc.)

Positioning
- GPS
- Beacons
- Marker based
- Depth perception

Image recognition
- 2D / 3D
- One time / Continuous

Sensors
- Camera
- Gyroscope
- GPS
- Bluetooth
- Special cameras
(Hololens, Project Tango)
- Gesture sensors
- Etc.

AR Example - QR code

AR Example - PokemonGO

AR Example - Marker

AR Example - Project Tango

About Ionic

Since 2013

Millions of developers

Millions of apps

The Project

"Pokemon GO" like promotion for app bars

How to AR?

SDKs

ARToolkit

Vuforia

Wikitude

Cross-platform compatibility is key

And the winner is...

Wikitude

So we can AR!

About Wikitude

Augmented Reality SDK with extensions for Cordova, Titanium, Xamarin, Unity

+ Wikitude Cloud Recognition

+ Wikitude Studio

Get Started with Wikitude

Example on Ionic Marketplace

ionic plugin add https://github.com/Wikitude/wikitude-cordova-plugin.git

Obtain an SDK (here) and set it in your app

WikitudePlugin._sdkKey = '...';

Project Structure

Components, Services, Pipes, etc.

AR World

Ionic App

Different
HTML & CSS & JS

Structure: Ionic App

Ionic App

Use WikitudePlugin (Docs)

// Set the SDK
WikitudePlugin._sdkKey

// Check, whether the device is supported
WikitudePlugin.isDeviceSupported(successCallback, errorCallback, options)

// Load an AR World
WikitudePlugin.loadArchitectWorld(successCallback, errorCallback, arWorldHtmlPath, type, options)

// Communicate: INPUT
WikitudePlugin.callJavaScript(codeString)

// Communicate: OUTPUT
WikitudePlugin.setOnUrlinvokeCallback(callback)

Structure: AR World

AR World

AR world in src/assets

wikitude-world
|- assets
|  |- model.wt3
|  |- indi.png
|
|- css
|  |- default.css
|
|- js
|  |- wikitude-world.js
|
|- index.html

Structure: AR World #2

AR World

Index.html

<html>
  <head>
    ...
    <script src="https://www.wikitude.com/libs/architect.js"></script>
    <script type="text/javascript" src="../ade.js"></script>
    <link rel="stylesheet" href="css/default.css">
  </head>
  <body onLoad="javascript:AR.logger.activateDebugMode();">
    <a href="javascript: World.takeAction('screenshot')">Snapshot</a>
    <a href="javascript: World.takeAction('exit')">Exit</a>
    <div id="loadingMessage" class="info">Loading ...</div>
    <script src="js/wikitude-world.js"></script>
  </body>
</html>

Structure: AR World #3

Wikitude-world.js

var World = {
  startRendering: function startRenderingFn(obj) {

    var indicatorImage = new AR.ImageResource('assets/indi.png');

    var indicatorDrawable = new AR.ImageDrawable(indicatorImage, 0.1, {
      verticalAnchor: AR.CONST.VERTICAL_ANCHOR.TOP
    });

    var myModel = new AR.Model(obj.modelUrl, {
      onClick: () => this.takeAction('collect')
    });

    var location = new AR.RelativeLocation(null, 5, 0, 2);

    var obj = new AR.GeoObject(location, {
      drawables: {
        cam: [myModel],
        indicator: [indicatorDrawable]
      }
    });
  }
  ...
}

Communication #1

Ionic App

AR World

WikitudePlugin.callJavaScript(`World.startRendering(${JSON.stringify(obj)});`);

Ionic App says:

Communication #2

Ionic App

AR World

WikitudePlugin.setOnUrlInvokeCallback((url) => {
  if (url.indexOf('action=close') > -1) WikitudePlugin.close();
});

... and the Ionic App listens:

  takeAction: function takeActionFn(action) {
    document.location = 'architectsdk://button?action=' + action;
  }

AR World says:

The Project: IN / OUT

Ionic App

AR World

close / collect / snapshot

modelPath (*.wt3) / text

What did we learn?

Developing an AR app is not that difficult

Wikitude helps a lot

Wikitude handles AR as a different "app"

Dataflow needs pre-planning

Extra: Debugging just on device

Thanks

for your attention!

Q & A

@martzellk