Practical Natural Language Processing
Building yourself Chatbot with LINE
Du Mingzhe (mingzhe@nus.edu.sg)

Milestones
Convolutional Neural Networks
Recurrent Neural Networks
Sequence-to-sequence Models
Transformers
More infoamation can be found here: https://arxiv.org/pdf/2105.04387.pdf

Convolutional Neural Networks

Recurrent Neural Networks

Sequence-to-sequence Models
Attention

What is the colour
of the building?





Transformers

ฉัน เป็น นักเรียน

ฉัน เป็น นักเรียน

ฉัน เป็น นักเรียน

Now We’re Encoding!

Then decoding...

How to process outputs?

ฉัน เป็น นักเรียน

Hands-on
Line Official Account Manager



Account Name
Email Address
Industry
Country / Region
Company Name

Click it

Go to LINE Developers

Then you have the token

shorturl.at/ACLNS
Install Packages and Import Libraries
# Install LINE Python SDK and pyngrok
!pip install line-bot-sdk
!pip install pyngrok
# Import libraries
import json
import requests
from pyngrok import ngrok
from flask import Flask, request, abort
from linebot import LineBotApi, WebhookHandler
from linebot.exceptions import InvalidSignatureError
from linebot.models import MessageEvent, TextMessage, TextSendMessageFirst Glance at Chatbot Model
# First glance
from transformers import pipeline, Conversation
from transformers import BlenderbotTokenizer, BlenderbotForConditionalGeneration
model = BlenderbotForConditionalGeneration.from_pretrained("facebook/blenderbot-400M-distill")
tokenizer = BlenderbotTokenizer.from_pretrained("facebook/blenderbot-400M-distill")
inputs = tokenizer(["My friends are cool but they eat too many carbs."], return_tensors="pt")
next_reply_ids = model.generate(**inputs)
print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0])
# Bot: That's unfortunate. Are they trying to lose weight or are they just trying to be healthier?Chatbot Pipeline
# Build a pipeline
conversational_pipeline = pipeline("conversational", model="facebook/blenderbot-400M-distill")
conversation = Conversation()
# Get response function
def get_response(conversation, sentence):
conversation.add_user_input(sentence)
output = conversational_pipeline([conversation])
return outputLine Configuration
# Line Channel ID (DON'T share it with others)
channel_id = "[YOUR_LINE_CHANNEL_ID]"
# Line Channel Secret (DON'T share it with others)
channel_secret = "[YOUR_LINE_CHANNEL_SECRET]"
# Issue a short-term token
def issueToken(client_id, client_secret):
endpoint = "https://api.line.me/v2/oauth/accessToken"
header = {'Content-Type': 'application/x-www-form-urlencoded'}
body = {'grant_type': 'client_credentials',
'client_id': client_id,
'client_secret': client_secret}
response = requests.post(url=endpoint, data=body, headers=header)
print(response)
obj = json.loads(response.text)
return obj
# Your Line Token (DON'T share it with others)
tokenDict = issueToken(channel_id, channel_secret)
# Assume tokenDict should look like that
# {
# 'access_token': '[YOUR_LINE_CHANNEL_TOKEN]',
# 'expires_in': 2592000,
# 'token_type': 'Bearer'
# }Running Flask app on localhost
app = Flask(__name__)
handler = WebhookHandler(channel_secret)
line_bot_api = LineBotApi(tokenDict["access_token"])
@app.route("/callback", methods=['POST'])
def callback():
signature = request.headers['X-Line-Signature']
body = request.get_data(as_text=True)
print("# Webhook event:\n", body)
print('-'*100)
# Handle webhook body
try:
handler.handle(body, signature)
except InvalidSignatureError:
print("Invalid signature. Please check your channel access token/channel secret.")
abort(400)
return "OK"
@handler.add(MessageEvent, message=TextMessage)
def handle_message(event):
print(f"Message Return: {event}")
line_bot_api.reply_message(event.reply_token, TextSendMessage(text=event.message.text.strip()))Expose Localhost to Public
# Ngrok configuration
!ngrok authtoken 2EkXz20Sxld1yr6MDyEw3wGe8FG_7E5Nh1H6fH1AHCHVdVCAw
!cat /root/.ngrok2/ngrok.yml
# Setting up http tunnel
http_tunnel = ngrok.connect(5000)
print(http_tunnel)Set Line Webhook Endpoint
def setWebhook(endpoint, CHANNEL_ACCESS_TOKEN):
endpointFixed = f"https://{endpoint.split('//')[-1]}/callback"
url = "https://api.line.me/v2/bot/channel/webhook/endpoint"
header = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + CHANNEL_ACCESS_TOKEN}
body = json.dumps({'endpoint': endpointFixed})
response = requests.put(url=url, data=body, headers=header)
print(response)
setWebhook(http_tunnel.public_url, channelAccessToken)app.run()

What to do next?
Context Awareness
How to harness historical context?
Response Diversity
variety of responses
Personality-based Response
Be aware of the roles / Response based on the roles
Knowledge-Grounded System
knowledgeable / Positive Feedback / Know what don't know
Interactive Training
Learning with chatting / Reinforcement Learning
Thank you
Nasith Laosen (nasith.l@pkru.ac.th)
Du Mingzhe (mingzhe@nus.edu.sg)
Natural Language Processing in one hour
By elfsong
Natural Language Processing in one hour
- 45
