Take Action!
Building For the Google Assistant
#BAFYC
VoiceCamp - GDG NYC
22 Jun 2018
Audio Credit: CBS Studios / Paramount
Allen Firstenberg
Software developer / Consultant
Google Developer Expert for Assistant, IoT, Wearables, Identity
http://spiders.com/
http://prisoner.com/
http://prisoner.com/ubi/
What are Actions on Google?
Actions on Google is the API
that lets a user engage in conversations
with other services
through the Google Assistant.
Dialogflow
How do we build an Action?
- Design
- Develop
- Deploy
Image Credit: CBS Studios / Paramount
What details?
- Intents
- Fulfillment
- SSML
- Context and storage
- Error recovery
- Multi-modal
Image Credit: CBS Studios / Paramount
Intents
Fulfillment
Fulfillment
Hosting
- Dialogflow inline editor
- Any web server, as long as
- Public URL
- Valid HTTPS
- Some good tools include
- Firebase Cloud Functions
- AWS Lambda
- ngrok (testing)
Image Credit: CBS Studios / Paramount
Fulfillment
Coding
- node.js
- actions-on-google
- dialogflow-fulfillment
- multivocal
- Everyone
- Anything that can read and write JSON
Image Credit: CBS Studios / Paramount
Speech Synthesis Markup
<speak>
So next you're going to tell them how to use
Speech Synthesis Markup Language or
<say-as interpret-as='characters'>SSML</say-as>
to shape the persona by doing things such as
<prosody rate='slow'>
inserting <break time='0.5s'/>
dramatic <break time='0.5s'/>
pauses. <break time='1s' />
</prosody>
Or making noises
<audio src='https://actions.google.com/sounds/v1/cartoon/cartoon_boing.ogg'/>
<prosody rate='fast'>Or talking very quickly.</prosody>
<prosody pitch='high'><voice gender='male' variant='2'>Or even changing</voice></prosody>
<prosody pitch='low'><voice gender='female' variant='2'>my voice</voice></prosody>
</speak>
Image Credit: CBS Studios / Paramount and giphy
Context
Session Storage
- This conversation only
- "data" object
- context parameters
- Values used for processing
- Location
User Storage
- Long-term
- "user storage"
- database
- Track number of visits
- Unique ID
- Name
- "Usual" location
Identity and Storage
Error recovery
Error recovery
No match
Error recovery
No input
Error recovery
Additional tips
- Use Context to keep track of where you are in the conversation and for contextual help.
- Use session storage to keep track of how many times you've had to recover.
- Use user storage to keep track of how you have helped this user in the past.
Image Credit: CBS Studios / Paramount
Multimodal
Rich Media
(use them - but don't assume)
- Display cards
- Lists and Options
- Suggestion chips
- Links
- Audio playback
Surface Capabilities
- Available for session
- Available for user
- Change surface
Image Credit: CBS Studios / Paramount
What's Next?
Documentation and Help
- https://developes.google.com/actions/
- https://docs.digalogflow.com/
- https://multivocal.info/
-
Stack Overflow tags:
- actions-on-google
- dialogflow
Me
https://prisoner.com/ubi/
Image Credit: CBS Studios / Paramount
Codelabs!
- https://codelabs.developers.google.com/
- Search for "Assistant"
Text
#AoGDevs
@ActionsOnGoogle
Take Action! Building for the Google Assistant. GDG NYC VoiceCamp 2018
By Allen “Prisoner” Firstenberg
