Chatbot Workshop

Vincent Chiang @

江品陞 @

Slide @

Source Code @

zaoldyeck @

Goals

NLU

Multi Skills

3rd API

We will use


Scenario

Line Cloud

Webhook

NLU

Intent, Slots

Third-party APIs

Intent A

Intent B

Intent C

Intent D

Intent E

Fetch data by intent

Web Server

Line Cloud

Send Message to User

Third-party APIs

Intent A

Intent B

Intent C

Intent D

Intent E

Response Data

Web Server

Generate Message

Your first echo chatbot

$ mkdir project_name # 建立專案目錄

$ cd project_name # 移動工作目錄至至專案目錄

$ npm init # npm 初始化

$ npm install nodemon bottender md5 axios @kkbox/kkbox-js-sdk --save # 安裝套件

$ vi package.json # 編輯 package.json, 內容在下面連結

$ vi config.js # 新增&編輯 config.js, 內容在下面連結

$ mkdir src # 建立資料夾 src

update package.json

add config.js

Update package.json

{
  "name": "integration-chatbot",
  "version": "1.0.0",
  "private": true,
  "scripts": {
    "dev": "nodemon src/index.js",
    "start": "node src/index.js"
  },
  "dependencies": {
    "@kkbox/kkbox-js-sdk": "^1.3.3",
    "axios": "^0.18.0",
    "bottender": "^0.15.7",
    "md5": "^2.2.1",
    "nodemon": "^1.18.4"
  }
}

Fill config.js(See following slides)

module.exports = {
    line: {
        channelSecret: 'your_line_channel_secret',
        accessToken: 'your_line_channel_access_token'
    },
    messenger: {
        accessToken: '',
        appSecret: '',
        verifyToken: '',
    },
    telegram: {
        accessToken: '',
    },
    olami: {
        appKey: 'your_olami_app_key',
        appSectet: 'your_olami_app_sectet'
    },
    kkbox: {
        id: 'your_kkbox_app_id',
        secret: 'your_kkbox_app_secret'
    }
}

Start coding

$ vi src/index.js # 新增 index.js 主程式, 內容在下面連結

$ vi src/handler.js # 新增 handler.js, 內容在下面連結

Project Directory
├── src
|   ├── handler.js
|   └── index.js
├── config.js
├── package.json
└── package-lock.json

add index.js

index.js

const express = require('express')
const bodyParser = require('body-parser')
const {LineBot} = require('bottender')
const {registerRoutes} = require('bottender/express')

const {lineHandler} = require('./handler')
const config = require('../config')

const server = new express()

server.use(
    bodyParser.json({
        verify: (req, res, buf) => {
            req.rawBody = buf.toString()
        }
    })
)

const bots = {
    line: new LineBot(config.line).onEvent(lineHandler)
}

registerRoutes(server, bots.line, {path: '/line'})

server.listen(process.env.PORT || 5000, () => {
    console.log('server is listening on 5000 port...')
})
const {LineHandler} = require('bottender')

exports.lineHandler = new LineHandler()
    .onText(async context => {
            const text = context.event.text
            const reply = text
            await context.replyText(reply)
        }
    )

handler.js

Running server

$ npm run dev # 執行程式, 等同於指令 nodemon src/index.js, 之後維持在 background 運作
# 開新的 Terminal
$ ngrok http 5000 # 利用 ngrok 建立 public domain 至 localhost 5000 port 的 poxy, 之後維持在 background 運作

Webhook

ngrok server

proxy

ngrok client

Node.js HTTP server

listen port 5000

localhost

Line Cloud

ngrok server

proxy

ngrok client

localhost

Generate Message

Send Message to User

Line Cloud

Setting webhook

Try it

What is NLU?

台南今天天氣如何?

台積電的股價

播放周杰倫的告白氣球

 

Grammar:<locate>今天天氣如何?

Intent:weather

Slot:台南

 

Grammar:<stock_name>的股價

Intent:stock

Slot:台積電

 

Grammar:播放<artist_name>的<track_name>

Intent:play_music

Slot:artist_name => 周杰倫,

track_name => 告白氣球

Line Cloud

Webhook

Intent: weather

Slots: {locate: 台南, date: today}

Third-party APIs

Weather API

Intent B

Intent C

Intent D

Intent E

Fetch Weather Data

台南今天天氣如何?

NLU

Web Server

Line Cloud

Third-party APIs

Weather API

Intent B

Intent C

Intent D

Intent E

Response Data

台南今天天氣晴,室溫 28 度

Web Server

Generate Message

Send Message to User

Add NLU feature

  • Rule-based
  • Default IDS module skill
  • Can add custom skill

Add NLU feature

# 開新的 Terminal
$ mkdir src/nlp

$ vi src/nlp/Olami.js

$ vi src/handler.js

Project Directory
├── src
|   ├── nlp
|   |    └── Olami.js
|   ├── handler.js
|   └── index.js
├── config.js
├── package.json
└── package-lock.json

add Olami.js

update handler.js

Olami.js(API doc)

const config = require('../../config')
const axios = require('axios')
const md5 = require('md5')

class Olami {
    constructor(appKey = config.olami.appKey, appSecret = config.olami.appSectet, inputType = 1) {
        this.URL = 'https://tw.olami.ai/cloudservice/api'
        this.appKey = appKey
        this.appSecret = appSecret
        this.inputType = inputType
    }

    nli(text, cusid = null) {
        const timestamp = Date.now()
        return axios.post(this.URL, {}, {
            params: {
                appkey: this.appKey,
                api: 'nli',
                timestamp: timestamp,
                sign: md5(`${this.appSecret}api=nliappkey=${this.appKey}timestamp=${timestamp}${this.appSecret}`),
                cusid: cusid,
                rq: JSON.stringify({'data_type': 'stt', 'data': {'input_type': this.inputType, 'text': text}})
            }
        }).then(response => {
            const nli = response.data.data.nli[0];
            return this._intentDetection(nli)
        })
    }

    _intentDetection(nli) {
        const type = nli.type
        const desc = nli.desc_obj
        const data = nli.data_obj

        function handleSelectionType(desc) {
            const descType = desc.type

            switch (descType) {
                case 'news':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.title).join('\n')
                case 'poem':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.poem_name).join('\n')
                case 'cooking':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.name).join('\n')
                default:
                    return '對不起,你說的我還不懂,能換個說法嗎?'
            }
        }

        switch (type) {
            case 'kkbox':
                return (data.length > 0) ? data[0].url : desc.result
            case 'ds':
                return desc.result + '\n請用 /help 指令看看我能怎麼幫助您'
            case 'joke':
                return data[0].content
            case 'news':
                return data[0].detail
            case 'selection':
                return handleSelectionType(desc)
            case 'baike':
                return data[0].description
            case 'cooking':
                return data[0].content
            default:
                return desc.result
        }
    }
}

module.exports = new Olami()

IDS modules(Doc)

// NLU API 回傳格式

{
    "data": {
        "nli": [
            {  
                "desc_obj" : {  
                    "result" : "",
                    "status" : int
                },
                "data_obj" : [  
                    {  
                        "xxx" : ""
                    }
                ],
                "type" : "xxx"
            }
        ]
    },
    "status": "ok"
}
// 今天星期幾?

{
    "data": {
        "nli": [
            {
                "desc_obj": {
                    "result": "今天是2017年9月13號星期三。",
                    "status": 0
                },
                "type": "date"
            }
        ]
    },
    "status": "ok"
}
// 講個笑話

{
    "data": {
        "nli": [
            {
                "desc_obj": {
                    "result": "好的,那你聽好嘍。",
                    "name": "",
                    "type": "joke",
                    "status": 0
                 },
                "data_obj": [
                    {
                        "content": "海邊,爸爸指著快要落下的太陽神氣的喊道:“下去,下去!”不一會兒,太陽果然下去了,孩童也看呆了,一面鼓掌,一面説:“爸爸,再來一次,再來一次!” "
                    }
                ],
                "type": "joke"
             }
        ]
    },
    "status": "ok"
}

Selection type(Doc)

// 今日新聞

{
    "data": {
        "nli": [  
           {  
              "desc_obj": {  
                  "result":"主人,查看詳情,請說第幾條,更多新聞,請說下一頁。",
                  "type":"news",
                  "status":0
              },
              "data_obj": [  
                  {  
                      "image_url":"https://static.ettoday.net/images/2922/b2922335.jpg",
                      "ref_url":"http://feedproxy.google.com/~r/ettoday/star/~3/8MAnn0ZHPv8/1051340.htm",
                      "time":"2017-11-13",
                      "detail":"愷樂近日推出第二波主打《三人行》,特邀老闆羅志祥、好友鼓鼓跨刀演唱,這首歌最初構想來自羅志祥,他親自打電話向好友鼓鼓邀歌,不只譜曲還要寫詞,而且還特別要求要三人對唱。而兩人也分享追求女生的方法,沒想到",
                      "title":"愷樂尬鼓鼓、羅志祥合唱! 「2天寫曲」互飆Rap帥炸"
                 },
                 {  
                     "image_url":"https://static.ettoday.net/images/2922/b2922289.jpg",
                     "ref_url":"http://feedproxy.google.com/~r/ettoday/star/~3/Gv3ZxlCBeAA/1051326.htm",
                     "time":"2017-11-13",
                     "detail":"小甜甜張可昀及王宇婕隨著《閨蜜的奇幻旅程》赴泰國錄影,製作單位幫她完成三年多來的心願。這次閨蜜的旅程中邀請甜甜的乾姊加入,製作單位騙她將出發去某一個比較遠的廟宇參拜,沒想到一下車眼前出現的是乾姊與龍波",
                     "title":"小甜甜懼怕跳水 王宇婕一把抱住:相信我會接住妳"
                 }...
              ],
              "type":"selection"
           }
        ]
    },
    "status": "ok"
}

Update handler.js

const {LineHandler} = require('bottender')
const olami = require('./nlp/Olami')

exports.lineHandler = new LineHandler()
    .onText(async context => {
            const text = context.event.text
            const userId = context._session.user.id
            const reply = await olami.nli(text, userId)
            await context.replyText(reply)
        }
    )

Well done!

Add music skill

Line Cloud

Webhook

Third-party APIs

Weather API

KKBOX Open API

Intent C

Intent D

Intent E

Fetch Music Data

NLU

Web Server

播放古典音樂類型的歌

Intent: music_play_playlist

Slots: {keyword: 古典音樂}

Line Cloud

Third-party APIs

Weather API

KKBOX Open API

Intent C

Intent D

Intent E

Response Data

Web Server

Generate Message

Send Message to User

KKBOX Open API(link)

What data it provide?(see KUBE

KUBE App download link

iOS

KKBOX Open API

Postman collection link

Get access token(doc)

Search API(doc)

Search API response

{
    "playlists": {
        "data": [
            {
                "id": "OtY2I4ebPHGasNyABp",
                "title": "動漫歌曲嚴選集",
                "description": "嚴選「蠟筆小新」、「火影忍者」、「刀劍神域 」、「死神BLEACH」、「進擊的巨人」等多部著名動畫歌曲。★世界首次,歡樂滿載的蠟筆小新特展,累積台北、高雄兩站的超人氣,台灣巡迴將進入最終站台中跟大家過寒假囉!馬上購票 http://avextw.kktix.cc/events/shinchan-taichung",
                "url": "https://event.kkbox.com/content/playlist/OtY2I4ebPHGasNyABp",
                "images": [
                    {
                        "height": 300,
                        "width": 300,
                        "url": "https://i.kfs.io/playlist/global/34601v1/cropresize/300x300.jpg"
                    },
                    {
                        "height": 600,
                        "width": 600,
                        "url": "https://i.kfs.io/playlist/global/34601v1/cropresize/600x600.jpg"
                    },
                    {
                        "height": 1000,
                        "width": 1000,
                        "url": "https://i.kfs.io/playlist/global/34601v1/cropresize/1000x1000.jpg"
                    }
                ],
                "updated_at": "2016-12-16T03:40:47+00:00",
                "owner": {
                    "id": "Ooerjv5-p-TJsFGLg5",
                    "url": "https://www.kkbox.com/tw/profile/Ooerjv5-p-TJsFGLg5",
                    "name": "KKBOX 日語小編",
                    "description": "",
                    "images": [
                        {
                            "height": 75,
                            "width": 75,
                            "url": "https://i.kfs.io/muser/global/94563302v1/cropresize/75x75.jpg"
                        },
                        {
                            "height": 180,
                            "width": 180,
                            "url": "https://i.kfs.io/muser/global/94563302v1/cropresize/180x180.jpg"
                        },
                        {
                            "height": 300,
                            "width": 300,
                            "url": "https://i.kfs.io/muser/global/94563302v1/cropresize/300x300.jpg"
                        }
                    ]
                }
            }...
        ],
        "paging": {
            "offset": 0,
            "limit": 50,
            "previous": null,
            "next": "https://api.kkbox.com/v1.1/search?limit=50&q=%E5%8B%95%E6%BC%AB%E6%AD%8C%E6%9B%B2&territory=TW&type=playlist&offset=50"
        },
        "summary": {
            "total": 75
        }
    },
    "paging": {
        "offset": 0,
        "limit": 50,
        "previous": null,
        "next": "https://api.kkbox.com/v1.1/search?limit=50&q=%E5%8B%95%E6%BC%AB%E6%AD%8C%E6%9B%B2&territory=TW&type=playlist&offset=50"
    },
    "summary": {
        "total": 75
    }
}

KKBOX HTML Widgets(link)

https://widget.kkbox.com/v1/?id={id}&type={type}

# type - {song, album, playlist}
  1. 播放<artist_name>的歌<{@=music_play_artist}>
  2. 播放<album_name>專輯的歌<{@=music_play_album}>
  3. 播放<track_name><{@=music_play_track}>
  4. 播放<keyword>類型的歌<{@=music_play_playlist}>

artist_name, album_name, track_name, keyword

Intent

  1. music_play_artist
  2. music_play_album
  3. music_play_track
  4. music_play_playlist

Download music_kkbox.osl

& 導入模組

Figure out NLI API response

{
   "nli":[
      {
         "desc_obj":{
            "status":0
         },
         "semantic":[
            {
               "app":"music_kkbox",
               "input":"播放動漫歌曲類型的歌",
               "slots":[
                  {
                     "name":"keyword",
                     "value":"動漫歌曲"
                  }
               ],
               "modifier":[
                  "music_play_playlist"
               ],
               "customer":"59e031f7e4b0a8057efdce99"
            }
         ],
         "type":"music_kkbox"
      }
   ]
}

is intent

is slot

$ mkdir src/api

$ vi src/api/KKBOX.js

$ vi src/nlp/Olami.js

Project Directory
├── src
|   ├── api
|   |    └── KKBOX.js
|   ├── nlp
|   |    └── Olami.js
|   ├── handler.js
|   └── index.js
├── config.js
├── package.json
└── package-lock.json

add KKBOX.js

update Olami.js

Integrate third-party API

KKBOX.js

const config = require('../../config')
const {Auth, Api} = require('@kkbox/kkbox-js-sdk')

module.exports = class KKBOX {
    static init(clientId = config.kkbox.id, clientSecret = config.kkbox.secret) {
        return (async () => {
            const auth = new Auth(clientId, clientSecret)
            const accessToken = await auth.clientCredentialsFlow.fetchAccessToken().then(response => {
                return response.data.access_token
            })
            return new Api(accessToken)
        })()
    }
}

Update Olami.js

const config = require('../../config')
const axios = require('axios')
const md5 = require('md5')
const KKBOX = require('../api/KKBOX')

class Olami {
    constructor(appKey = config.olami.appKey, appSecret = config.olami.appSectet, inputType = 1) {
        this.URL = 'https://tw.olami.ai/cloudservice/api'
        this.appKey = appKey
        this.appSecret = appSecret
        this.inputType = inputType
    }

    nli(text, cusid = null) {
        const timestamp = Date.now()
        return axios.post(this.URL, {}, {
            params: {
                appkey: this.appKey,
                api: 'nli',
                timestamp: timestamp,
                sign: md5(`${this.appSecret}api=nliappkey=${this.appKey}timestamp=${timestamp}${this.appSecret}`),
                cusid: cusid,
                rq: JSON.stringify({'data_type': 'stt', 'data': {'input_type': this.inputType, 'text': text}})
            }
        }).then(response => {
            const nli = response.data.data.nli[0];
            return this._intentDetection(nli)
        })
    }

    _intentDetection(nli) {
        const type = nli.type
        const desc = nli.desc_obj
        const data = nli.data_obj
        const semantic = nli.semantic

        function handleSelectionType(desc) {
            const descType = desc.type

            switch (descType) {
                case 'news':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.title).join('\n')
                case 'poem':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.poem_name).join('\n')
                case 'cooking':
                    return desc.result + '\n\n' + data.map((el, index) => index + 1 + '. ' + el.name).join('\n')
                default:
                    return '對不起,你說的我還不懂,能換個說法嗎?'
            }
        }

        async function handleMusicKKBOXType(semantic) {

            function getKeyWord(semantic, dataType) {

                function getSlotValueByName(slotName) {
                    return semantic.slots.filter(slot => slot.name === slotName)[0].value
                }

                switch (dataType) {
                    case 'artist':
                        return getSlotValueByName('artist_name')
                    case 'album':
                        return getSlotValueByName('album_name')
                    case 'track':
                        return getSlotValueByName('track_name')
                    case 'playlist':
                        return getSlotValueByName('keyword')
                }
            }

            const dataType = semantic.modifier[0].split('_')[2]
            const keyWord = getKeyWord(semantic, dataType)

            const api = await KKBOX.init()
            const data = await api
                .searchFetcher
                .setSearchCriteria(keyWord, dataType)
                .fetchSearchResult()
                .then(response => {
                    return response.data[dataType + 's'].data
                })
            if (dataType === 'artist') {
                return data[0].url
            } else {
                const id = data[0].id
                return `https://widget.kkbox.com/v1/?id=${id}&type=${dataType === 'track' ? 'song' : dataType}`
            }
        }

        switch (type) {
            case 'kkbox':
                return (data.length > 0) ? data[0].url : desc.result
            case 'baike':
                return data[0].description
            case 'news':
                return data[0].detail
            case 'joke':
                return data[0].content
            case 'cooking':
                return data[0].content
            case 'selection':
                return handleSelectionType(desc)
            case 'ds':
                return desc.result + '\n請用 /help 指令看看我能怎麼幫助您'
            case 'music_kkbox':
                return handleMusicKKBOXType(semantic[0])
            default:
                return desc.result
        }
    }
}

module.exports = new Olami()

Try it

Deployment

Deploy to Heroku(link)

add Procfile

web: node src/index.js --log-file -
$ vi Procfile

$ vi .gitignore

Project Directory
├── src
|   ├── api
|   |    └── KKBOX.js
|   ├── message
|   |    ├── KKBOXMessage.js
|   |    ├── Message.js
|   |    └── TextMessage.js
|   ├── nlp
|   |    └── Olami.js
|   ├── handler.js
|   └── index.js
├── .gitignore
├── config.js
├── package.json
├── package-lock.json
└── Procfile

.gitignore

# Taken from https://github.com/github/gitignore/blob/master/Node.gitignore

# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*

# Runtime data
pids
*.pid
*.seed
*.pid.lock

# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov

# Coverage directory used by tools like istanbul
coverage

# nyc test coverage
.nyc_output

# jest-junit
junit.xml

# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt

# Bower dependency directory (https://bower.io/)
bower_components

# node-waf configuration
.lock-wscript

# Compiled binary addons (https://nodejs.org/api/addons.html)
build/Release

# Dependency directories
node_modules/
jspm_packages/

# Typescript v1 declaration files
typings/

# Optional npm cache directory
.npm

# Optional eslint cache
.eslintcache

# Optional REPL history
.node_repl_history

# Output of 'npm pack'
*.tgz

# Yarn Integrity file
.yarn-integrity

# dotenv environment variables file
.env
### macOS template
# General
.DS_Store
.AppleDouble
.LSOverride

# Icon must end with two \r
Icon

# Thumbnails
._*

# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent

# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
### JetBrains template
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

# User-specific stuff:
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/dictionaries

# Sensitive or high-churn files:
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.xml
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml

# Gradle:
.idea/**/gradle.xml
.idea/**/libraries

# CMake
cmake-build-debug/
cmake-build-release/

# Mongo Explorer plugin:
.idea/**/mongoSettings.xml

## File-based project format:
*.iws

## Plugin-specific files:

# IntelliJ
out/

# mpeltonen/sbt-idea plugin
.idea_modules/

# JIRA plugin
atlassian-ide-plugin.xml

# Cursive Clojure plugin
.idea/replstate.xml

# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties
.idea/
package-lock.json

Deploy to Heroku

$ git init # 初始化版本控制
$ git checkout -b production # 創建並切換至 production 分支
$ git add . # 將目錄下所有檔案加入版本控制

# 如果你第一次使用 Git 需要用以下兩行指令設定 yourname 及 email
$ git config --global user.name "yourname"
$ git config --global user.email "yourname@example.com"

$ git commit -m "Deploying to Heroku" # 建立快照

$ heroku login # Heroku 認證登入

$ heroku git:remote -a {your_heroku_app_name} # 加入 Heroku remote repository

$ git push heroku production:master # 將本地的 production 分支內容 push 至 Heroku master 分支

Set webhook for production

All works are done!

Free communication

Chatbot Workshop 3 hours version

By zaoldyeck

Chatbot Workshop 3 hours version

  • 1,974