![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4200721/google-dots.gif)
Take Action!
Building For the Google Assistant
GDG Bronx
22 Jun 2018
Audio Credit: CBS Studios / Paramount
Allen Firstenberg
Software developer / Consultant
Google Developer Expert for Assistant, IoT, Wearables, Identity
http://spiders.com/
http://prisoner.com/
http://prisoner.com/ubi/
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5051859/allen-glass-home-profile.jpg)
What are Actions on Google?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4191449/action-flow.png)
Actions on Google is the API
that lets a user engage in conversations
with other services
through the Google Assistant.
Dialogflow
How do we build an Action?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4248656/Android_storage_vault.jpg)
- Design
- Develop
- Deploy
Image Credit: CBS Studios / Paramount
What details?
- Intents
- Fulfillment
- SSML
- Context and storage
- Error recovery
- Multi-modal
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5048333/mccoy-spock.jpg)
Image Credit: CBS Studios / Paramount
Intents
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5048346/Screenshot_2018-06-20_at_21.18.34.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5048378/kisspng-curve-arrow-clip-art-curved-arrows-5aa958f5c72ce9.4175687715210477978158.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5049687/Screenshot_2018-06-21_at_08.31.56.png)
Fulfillment
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5049687/Screenshot_2018-06-21_at_08.31.56.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5048405/Screenshot_2018-06-20_at_21.51.31.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5048378/kisspng-curve-arrow-clip-art-curved-arrows-5aa958f5c72ce9.4175687715210477978158.png)
Fulfillment
Hosting
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4341673/star-trek-cloud.jpg)
- Dialogflow inline editor
- Any web server, as long as
- Public URL
- Valid HTTPS
- Some good tools include
- Firebase Cloud Functions
- AWS Lambda
- ngrok (testing)
Image Credit: CBS Studios / Paramount
Fulfillment
Coding
- node.js
- actions-on-google
- dialogflow-fulfillment
- multivocal
- Everyone
- Anything that can read and write JSON
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4245216/Uhura-ear-piece-03.jpg)
Image Credit: CBS Studios / Paramount
Speech Synthesis Markup
<speak>
So next you're going to tell them how to use
Speech Synthesis Markup Language or
<say-as interpret-as='characters'>SSML</say-as>
to shape the persona by doing things such as
<prosody rate='slow'>
inserting <break time='0.5s'/>
dramatic <break time='0.5s'/>
pauses. <break time='1s' />
</prosody>
Or making noises
<audio src='https://actions.google.com/sounds/v1/cartoon/cartoon_boing.ogg'/>
<prosody rate='fast'>Or talking very quickly.</prosody>
<prosody pitch='high'><voice gender='male' variant='2'>Or even changing</voice></prosody>
<prosody pitch='low'><voice gender='female' variant='2'>my voice</voice></prosody>
</speak>
Image Credit: CBS Studios / Paramount and giphy
Context
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5051279/Screenshot_2018-06-21_at_16.41.58.png)
Session Storage
- This conversation only
- "user data" string
- context parameters
- Values used for processing
- Location
User Storage
- Long-term
- "userStorage" string
- database
- Track number of visits
- Unique ID
- Name
- "Usual" location
Identity and Storage
Error recovery
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5049750/Screenshot_2018-06-21_at_09.02.25.png)
Error recovery
No match
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5051378/Screenshot_2018-06-21_at_17.10.31.png)
Error recovery
No input
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/5051397/Screenshot_2018-06-21_at_17.12.59.png)
Error recovery
Additional tips
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4248647/quark3.jpg)
- Use Context to keep track of where you are in the conversation and for contextual help.
- Use session storage to keep track of how many times you've had to recover.
- Use user storage to keep track of how you have helped this user in the past.
Image Credit: CBS Studios / Paramount
Multimodal
Rich Media
(use them - but don't assume)
- Display cards
- Lists and Options
- Suggestion chips
- Links
- Audio playback
Surface Capabilities
- Available for session
- Available for user
- Change surface
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4249254/Virtual_device01.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4245223/MainViewerRearProjection.jpg)
Image Credit: CBS Studios / Paramount
What's Next?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/607776/images/4341637/make-it-so.jpg)
Codelabs!
- https://codelabs.developers.google.com/
- Search for "Assistant"
Documentation and Help
- https://developes.google.com/actions/
- https://docs.digalogflow.com/
- https://multivocal.info/
-
Stack Overflow tags:
- actions-on-google
- dialogflow
Me
https://prisoner.com/ubi/
Image Credit: CBS Studios / Paramount
Take Action! Building for the Google Assistant
By Allen “Prisoner” Firstenberg
Take Action! Building for the Google Assistant
- 1,793