Take Action!

Building For the Google Assistant

GDG Bronx

22 Jun 2018

Audio Credit: CBS Studios / Paramount

Allen Firstenberg

Software developer / Consultant

 

Google Developer Expert for Assistant, IoT, Wearables, Identity

 

http://spiders.com/

http://prisoner.com/

 

http://prisoner.com/ubi/

What are Actions on Google?

Actions on Google is the API

that lets a user engage in conversations

with other services

through the Google Assistant.

Dialogflow

How do we build an Action?

  • Design
  • Develop
  • Deploy

Image Credit: CBS Studios / Paramount

What details?

  • Intents
  • Fulfillment
  • SSML
  • Context and storage
  • Error recovery
  • Multi-modal

Image Credit: CBS Studios / Paramount

Intents

Fulfillment

Fulfillment

Hosting

  • Dialogflow inline editor
  • Any web server, as long as   
    • Public URL
    • Valid HTTPS

 

  • Some good tools include
    • Firebase Cloud Functions
    • AWS Lambda
    • ngrok (testing)

Image Credit: CBS Studios / Paramount

Fulfillment

Coding

  • node.js
    • actions-on-google
    • dialogflow-fulfillment
    • multivocal
  • Everyone
    • Anything that can read and write JSON

Image Credit: CBS Studios / Paramount

Speech Synthesis Markup

<speak>
  So next you're going to tell them how to use
  Speech Synthesis Markup Language or
  <say-as interpret-as='characters'>SSML</say-as>
  to shape the persona by doing things such as
  <prosody rate='slow'>
    inserting <break time='0.5s'/>
    dramatic <break time='0.5s'/>
    pauses. <break time='1s' />
  </prosody>
  Or making noises
  <audio src='https://actions.google.com/sounds/v1/cartoon/cartoon_boing.ogg'/>
  <prosody rate='fast'>Or talking very quickly.</prosody>
  <prosody pitch='high'><voice gender='male' variant='2'>Or even changing</voice></prosody>
  <prosody pitch='low'><voice gender='female' variant='2'>my voice</voice></prosody>
</speak>

Image Credit: CBS Studios / Paramount and giphy

Context

Session Storage

  • This conversation only
  • "user data" string
  • context parameters

 

  • Values used for processing
  • Location

User Storage

  • Long-term
  • "userStorage" string
  • database

 

  • Track number of visits
  • Unique ID
  • Name
  • "Usual" location

Identity and Storage

Error recovery

Error recovery

No match

Error recovery

No input

Error recovery

Additional tips

  • Use Context to keep track of where you are in the conversation and for contextual help.

 

  • Use session storage to keep track of how many times you've had to recover.

 

  • Use user storage to keep track of how you have helped this user in the past.

Image Credit: CBS Studios / Paramount

Multimodal

Rich Media

(use them - but don't assume)

  • Display cards
  • Lists and Options
  • Suggestion chips
  • Links
  • Audio playback

Surface Capabilities

  • Available for session
  • Available for user
  • Change surface

Image Credit: CBS Studios / Paramount

What's Next?

Codelabs!

  • https://codelabs.developers.google.com/
  • Search for "Assistant"

 

Documentation and Help

  • https://developes.google.com/actions/     
  • https://docs.digalogflow.com/
  • https://multivocal.info/
  • Stack Overflow tags:
    • ​actions-on-google
    • dialogflow

Me

https://prisoner.com/ubi/

Image Credit: CBS Studios / Paramount

Made with Slides.com