If you can Dream It,
you can Do It
with the
Google Assistant

DevFest Florida

19 Jan 2019

(mostly)

Allen Firstenberg

Software developer / Consultant

 

Google Developer Expert for Assistant, IoT, Wearables, Identity

 

http://spiders.com/

http://prisoner.com/

 

http://prisoner.com/ubi/

Tell me about you

  • Mobile or web?
  • Alexa or Assistant?
  • Mac vs Windows?
  • JavaScript?
    Java?
    Something else?
  • Magic Kingdom? Epcot? Hollywood Studios? Animal Kingdom?

What are we talking about?

What is the Google Assistant?

A conversation

between you and Google that helps you get things done

in your world.

What are Actions on Google?

Actions on Google is the API

that lets you engage in conversations

with other services

through the Google Assistant.

How do we build an Action?

  • Design
  • Develop
  • Deploy

What can Actions do?

  • Query databases based on what the user is asking.
  • Collect responses from the user (such as a form) and save them.
  • Get some information about the user, with their permission.
  • Fun things: Quizzes, personality tests, RPGs, quotes.

 

Can you express it as a conversation?

Conversation Design

You're doing it wrong

  • Figure out what you want your code to do.
  • Decide how you're going to present those results.
  • "Build a screen" to get inputs.

Personality?

Beware of picking up hitchhiking personalities.

Building Intents

Finally, the code

// Import the Dialogflow service function
const {dialogflow} = require('actions-on-google');

const action = dialogflow();
const functions = require('firebase-functions');
exports.webhook = functions.https.onRequest(action);

action.intent('favorite color', conv => {
  let color = conv.parameters.color;
  conv.ask( `My favorite color is ${color}, too. What would you like to do now?` );
});

Adding an API

const {dialogflow} = require('actions-on-google');

// Add a request-promise module
const rp = require('request-promise-native');

const action = dialogflow();
const functions = require('firebase-functions');
exports.webhook = functions.https.onRequest(action);

action.intent('favorite color', conv => {
  let color = conv.parameters.color;

  // Options to make the request
  let options = {
    uri: 'http://www.colourlovers.com/api/colors',
    qs: {
      keywords: color,
      format: "json"
    },
    json: true
  };

  // Make the request
  return rp(options)
    .then( data => {
      // Handle the result
      conv.ask( `<speak>Did you know that ${data[0].title} has a hex value of`+
                ` <say-as interpret-as="characters">${data[0].hex}</say-as>.`+
                ` What would you like to do now?</speak>` );
    });

});

Authenticating APIs

Google Sign In for Assistant

const {dialogflow} = require('actions-on-google');

const action = dialogflow();
const functions = require('firebase-functions');
exports.webhook = functions.https.onRequest(action);

function getCredentials( id ){
  // Get the user's OAuth credentials from firebase
  // Return a Promise
}

function saveUserColor( credentials, name ){
  // Use the Google Drive API to create a file using the user's credentials
  // Return a Promise
}

action.intent('favorite color', conv => {
  let color = conv.parameters.color;
  const payload = conv.user.profile.payload;
  const id = payload.sub;  // OAuth Connect "sub" claim

  return getCredentials( id )
    .then( credentials => saveUserColor( credentials, color ) )
    .then( () => {
      conv.ask( `Ok, I've created a file named ${color}. What now?` );
    });
});

Authenticating APIs

Credentials with Google Sign In for Assistant

Keep in mind...

  • Remember the user's context
  • Vary your responses
  • Intents are what the user says,
    not how you respond.
  • Internationalization has new twists

Multivocal makes it easier

  • Reduce boilerplate.
  • Use sane defaults
    • Session storage
    • Suffixes and prompting
  • Make it easy to override those defaults.
  • Code generates results/values.
  • Responses through configuration
    • Templates
    • Internationalization

https://multivocal.info/

Using Multivocal

const Multivocal = require('multivocal');
exports.webhook = Multivocal.processFirebaseWebhook;

const config = {
  Local: {
    "en": {
      Response: {
        "Intent.favorite color": [
          "My favorite colour is {{Parameter/color}}, too.",
          "I agree, {{Parameter/color}} is rather nice."
        ]
      },
      Suffix: {
        "Default": [
          "What would you like to do now?"
        ]
      }
    },
    "en-US": {
      Response: {
        "Intent.favorite color": [
          "My favorite color is {{Parameter/color}}, too.",
          "I agree, {{Parameter/color}} is very nice."
        ]
      }
    }
  }
};
new Multivocal.Config.Simple(config);

Summary

  • Most of what you need to do - you already know how to do.

 

  • Create a personality first and write scripts for that personality.
  • Remember to write your scripts how people actually talk. Test this.
  • Intents are what the user says and does, not how you respond.
  • In JavaScript, use Promises correctly.
  • Don't mix your logic and your response.

https://spiders.com/

https://prisoner.com/ubi/

https://actions.google.com/design

https://developers.google.com/actions

Stack Overflow: actions-on-google

https://multivocal.info/

If you can Dream It, you can Do It with the Google Assistant

By Allen “Prisoner” Firstenberg

Made with Slides.com

If you can Dream It, you can Do It with the Google Assistant

  • 1,057

More from Allen “Prisoner” Firstenberg