Clozer
隊名 烏龜會飛
組員：王政祺、劉至軒、蕭梓宏
一個小故事...
我
悲慘的我
欲哭有淚的我
?
或許也有人跟我一樣？
-
不想背著厚厚的解答
-
詳解翻好久，一題一題對都找不到
-
解答真的是對的嗎，它到底在說甚麼？
- ......
學霸之路
到底該怎麼做呢？
LSTM
RNN
- 只有單向記憶
- 記憶力不足
- 正確性太低
到底該怎麼做呢？
?
?
?
?
?
?
到底該怎麼做呢？
?
?
?
?
?
?
註：雖然也有雙向LSTM，但是Vanishing還是問題
Attention is All You Need!
Vaswani et. al, 2017
ELMo Model
GPT Model
ELMo & GPT Models
ELMo Model
GPT Model
- 用非常大型的模型、非常多的資料來實現多重用途的NLP模型
- Context-Based Word Encodings
ELMo & GPT Models
GPT
ELMo
Context-Based
Bi-Directional
Deeply Context-Based
Say Hello to BERT!
Devlin et. al, 2018
BERT
Bidirectional （雙向）
BERT
Bidirectional （雙向）
Encoder （編碼）
BERT
Bidirectional （雙向）
Encoder （編碼）
Representations（表示）
BERT
Bidirectional （雙向）
Encoder （編碼）
Representations（表示）
Transformers（表示）
完美解決問題！
為什麼要用BERT ?
因為去年BERT學長得冠軍
...
克漏字的最佳模型
雙向性
準確性
易用性
例子
例子
例子
!
What we had done ?
I
取得題目
110學測 英文試題
以近幾年的學測指考英文以及模擬題本＆參考書作為資料來源
II
辨識題目
利用 Google Tesseract 對圖片進行文字辨識
1. Tom is really a naughty boy. He likes to and play jokes on his younger sister when their parents are not around.
(A) alert (B) spare (C) tease (D) oppose
preprocessing
文字處理，提取需要的題目內容
III
利用Bert模型完成克漏字
對句子、選項進行預處理
pretrained 過的 Bert 模型
得到最佳的克漏字結果
碰到的小問題
Tom is really a naughty boy. He likes _____ to and play jokes on his younger sister when their parents are not around.
Tom is really a _____ naughty boy. He likes to and play jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to and play _____ jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to and play _____ jokes on his younger sister when their parents are not around.
Solution
暴力嘗試每種可能 找最佳的匹配結果
Tom is really a naughty boy. He likes _____ to and play jokes on his younger sister when their parents are not around.
Tom is really a _____ naughty boy. He likes to and play jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to and play _____ jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to and play _____ jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to and play jokes on his younger _____ sister when their parents are not around.
Tom _____ is really a naughty boy. He likes to and play jokes on his younger sister when their parents are not around.
Tom is really a naughty boy. He likes to tease and play jokes on his younger sister when their parents are not around.
IV
使用者介面
在本地server架了一個 discord 聊天機器人
作為 Clozer 與使用者互動的窗口
The quick brown fox _____ over the lazy dog.
The quick brown fox jumps over the lazy dog.
Future Prospects
Roberta - 一種基於 Bert 的 pretrained model
嘗試解決篇章結構、閱讀測驗等題目
開發一個完整的輔助學習app
