Building Chatbots with DialogFlow

By: Sumit Raj

What will be covered?

Day 1:

  • Introducing Chatbots in DialogFlow
  • Building Blocks of Chatbots
    • Agents
    • Introduction to Intents
    • Introduction to Entities
    • Fulfillmentt

About Myself:

  • Work as Senior Solutions Architect at GeoSpark
  • Author of "Building Chatbots with Python". Published in multiple languages
  • Mentored over 1000 students so far
  • Avid writer on Quora
  • I love coding, learning new things and music.

Why we need Chatbots?

  • The Business Perspective
    • Accessibility
    • Efficiency
    • Availability
    • Scalability
    • Cost
    • Insights

 

Why we need Chatbots?

  • The Developer’s Perspective​
    • Feature Releases and Bug Fixes
    • Market Demand
    • Learning Curve

 

What Kind of Problems Can I Solve Using Chatbots?

  • Can the Problem be Solved by Simple Question and Answer or Back-and-Forth Communication?
  • Does It Have Highly Repetitive Issues That Require Either Analyzing or Fetching of Data?
  • Can Your Bot’s Task be Automated and Fixed?

Let's get into the game

What is DialogFlow?

  • Formerly known as Api.ai. Now it is known as Dialogflow.
  • Gives users new methods to interact with their product by building engaging voice and text-based conversational interfaces.
  • Dialogflow is powered by AI.
  • It helps you connect with users on your website, mobile app, Google Assistant, Amazon Alexa, Facebook Messenger, and other popular platforms and devices.

More about DialogFlow

1. User talks to the input device.
2. User query goes into the Dialogflow engine.
3. Dialogflow tries to recognize the intent.
4. Based on the intent, a fulfillment is done and data is returned from database.
5. Response is returned to the intent.
6. Response is converted into actionable data.
7. User’s request for information is given back to the output device.

Good things about Dialogflow

  • Supports integration of more than 20 platforms like Slack, Skype, Facebook, Twitter, Telegram, Line, Twilio
  • Supports more than 20 languages
  • Easy to learn
  • No coding skills required

Who uses Dialogflow?

Prerequisites and tools

  • DialogFlow Account
  • Python 3.6.5 
  • Flask
  • AWS account or Ngrok
  • Jupyter Notebook

Setting up DialogFlow

  1. Create an account at https://dialogflow.com, and log in to the account.
  2. Create an agent.

Hands-on time

Agents

  • A Dialogflow agent is a virtual agent that helps your users do conversations with the bot you build.

  • Agents also serve as a top-level container for settings and data:

    • Agent settings for language options, machine learning settings, and other settings that control the behavior of your agent.

                                                   Intent
When a user interacts with a chatbot, what is his intention to use chatbot/what is he asking for.

For example, when a user says, “Book a movie ticket” to a chatbot, we as humans can understand that user wants to book a movie ticket.

 

Entities

  • Intents have metadata about the intent called “Entities”.
  • In the above example, Booking a ticket could be an intent and entity is “movie” which could have been something else as well like flight, concert etc.
  • You can have general entities labelled for use throughout the intents. Entities could represent like a quantity, count or volume.
  • Intents can have multiple entities as well.
    • For example: Order me a shoe of size 8.
    • There could be two entities here as defined below

Category: Shoe
Size: 8

 

 

                                  Confidence Score

  • Every time you try to find what intent an utterance may belong to; your model will come up with a confidence score.
  • This score tells you how confident your machine learning model is about recognizing the intent of the user.

Utterances

  • Utterances are nothing but different forms of the same question/intent your user may show
  • Remember we discussed about Switching off the light intent ?
  • It is suggested to have an optimum 10 utterances per intent and minimum 5

 

Training the bot

  • Training means to build a model which will learn from the existing intents/entities & utterances on how to classify the new utterances and provide a confidence score along with it.
  • When we train the system using utterances – this is called supervised learning.

Things we'll be learning while doing under Intents

  • Create and manage intents
  • Training Phrases or Utterances
  • Actions and parameters
  • Responses
  • Default intents
  • Rich messages

Things we'll be learning while doing under entities

  • Entities
  • Create and manage entities
  • System entities
  • Developer entities
  • Session entities
  • Regex entities
  • Fuzzy matching

System Entities

  • Dialogflow provides many system entities to extract common types from end-user expressions.
  • For example, the @sys.number type can be used to extract values like 1,2,3 or one, two, three.
@sys.unit-currency
5 dollars {"amount":5,"currency":"USD"}
25 pounds {"amount":25,"currency":"GBP"}


 

 

Developer Entities

  • Also known as Custom Entities

  • You can create custom entities for matching data specific to your agent. For example, you could define a vegetable entity type that can match the types of vegetables available for purchase with a grocery store agent.

Session Entities

  • A session represents a conversation between a Dialogflow agent and an end-user.
  • You can create special entities, called session entities during a session.
  • Session entities can extend or replace custom entity types and only exist during the session that they were created for.
  • All session data, including session entities, is stored by Dialogflow for 20 minutes.

For example, if your agent has a @fruit entity type that includes "pear" and "grape", that entity type could be updated to include "apple" or "orange", depending on the information your agent collects from the end-user. The updated entity type would have the "apple" or "orange" entity entry for the rest of the session.

 

Regex Entities

  • Some entities need to match patterns rather than specific terms.
  • For example, national identification numbers, IDs, license plates, and so on.
  • With regexp entities, you can provide regular expressions for matching.
  • Each regexp entity corresponds to a single pattern, but you can provide multiple regular expressions if they all represent variations of a single pattern.
  • During agent training, all regular expressions of a single entity are combined with the alternation operator (|) to form one compound regular expression.

 

For example, if you provide the following regular expressions for a phone number:

  • ^[2-9]\d{2}-\d{3}-\d{4}$

Fuzzy Matching

By default, entity matching requires an exact match for one of the entity entries. This works well for single-word entity entry values and synonyms but may present a problem for multi-word values and synonyms. For example, consider a ball entity that should be matched for the following end-user expression parts:

  • "ball"
  • "red ball"
  • "ball red"
  • "small ball"
  • "ball small"

For a match to occur, you normally need to define an entity entry value and synonyms for each of these permutations. However, with fuzzy matching enabled, the ordering of the words in a value or synonym does not matter. The following will trigger a match for all of the examples above:

  • "ball"
  • "red ball"
  • "small ball"
  • "small red ball"
  • "small ball red"
  • "red small ball"
  • "red ball small"
  • "ball small red"
  • "ball red small"

Build a bot of your choice

  • Minimum 5 intents
  • All possible entities
    • System Entities 
    • Developer Entities
  • Use rich responses
  • Use contexts
    • Input Context
    • Output Context
  •  

Fulfillment

  • Fulfillment
  • How fulfillment works?
  • Configure fulfillment
  • Webhook for slot filling
  • Using external API for fulfillment

Fulfillment Flow & its working

Ref: dialogflow.com

Webhook for Slot Filling

  • When an intent is matched at runtime, the Dialogflow agent continues collecting information from the end-user until the end-user has provided data for each of the required parameters. This process is called slot filling.

  • By default, Dialogflow does not send a fulfillment webhook request until it has collected all required data from the end-user.

  • If webhook for slot filling is enabled, Dialogflow sends a fulfillment webhook request for each conversational turn during slot filling.

Using external API for fulfillment

  • Build an API using a language of your own choice
  • Define your request and response logic
  • Return response as per dialogflow
  • Deploy on web with SSL on

Let's make it public

Deploying your chatbot on web

  • Building a webhook endpoint
  • Setting up the webhook endpoint for dynamic data retrieval
  • Configuring Intents with Fulfillment
  • Using Ngrok to crank up an API and use in DialogFlow

Integrating with Third-party Applications

  • Setting up Facebook App
  • Integration with Facebook messenger

Quick Introduction to NLP for chatbots

  • Introduction to NLP
  • What is spaCy?
  • POS Tagging
  • Stemming and Lemmatization
  • Named-Entity Recognition
  • Stop-words
  • Dependency Parsing
  • Noun Chunks
  • Finding Similarity

Case-Studies

Made with Slides.com