RBS meets LLMs
Leaner Technologies, Inc.
黒曜
(@kokuyouwind)
~ Type inference using LLM ~
Talk: Japanese
Slides: English (+ Japanese)
$ whoami
黒曜 / @kokuyouwind
Work at: Leaner Technologies, Inc.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/2441388/i_yumemi.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10454983/2006ver_Leaner_logo3_white.png)
Ruby Sponsor
Day 1: Sponsor Talk (Done)
Day 2: Leaner Drinkup
The idea is covered by Matz's Keynote,
but this is the first time you've heard of it!
MatzのKeynoteで出た話と発想が被っていますが、
初めて聞いたことにしてください!
Large Language Model (LLM)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10454989/ChatGPT_logo.svg.webp)
Machine learning models trained on large amounts of text data.
OpenAI
ChatGPT 4
Anthropic
Claude 3
Meta
Llama 3
大量のテキストデータを使ってトレーニングされた機械学習モデル
大規模言語モデル
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/11301762/claude-ai-icon.webp)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/11301780/Llama-2-Model-Details.png)
Example: ChatGPT
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10455067/スクリーンショット_2023-05-08_14.07.25.png)
Can we do something fun with this?
🤔
なにか面白いことに使えないかな?
Type Inference
user = User.new
# `user` is an User type variable
name = user.name
# `name` is a String type variable(?)
Humans sometimes infer type from word meanings.
人間は単語の意味から型を推測することがある
With ChatGPT,
can we infer the type
from the meaning of a word?
ChatGPTを使えば、単語の意味から
型を推測できるのでは?
Let's see some cases!
Case 1: User Builder
user = UserBuilder.new.name('kokuyouwind').build
# We infer types as follows:
# * UserBuilder#name returns an UserBuilder
# * UserBuilder#build returns an User
Case 1: User Builder
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10455143/スクリーンショット_2023-05-08_15.10.23.png)
Case 1: User Builder
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10455145/スクリーンショット_2023-05-08_15.11.29.png)
(There is extra output, but)
Perfect!!!
👏
Case 2: Company Repository
company = CompanyRepository.new.find(1).name
# We infer types as follows:
# * CompanyRepository#find returns a Company
# * Company#name returns a String
# Note.
# This expression has the same syntactic form with:
# user = UserBuilder.new.name('kokuyouwind').build
Case 2: Company Repository
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10455166/スクリーンショット_2023-05-08_15.22.16.png)
(There is extra output, but)
Perfect, again!!!
👏
Problems
-
RBS sometimes broken (たまにRBSの構文がおかしい)
-
Extra Outputs (不要な出力がある)
-
Only RBS output needed (RBSだけ出力させたい)
-
-
Single RBS (RBSがひとまとめになっている)
-
Separate User's RBS to user.rbs, and so on
(UserのRBSはuser.rbsに、などクラスごとに分割したい)
-
Improve: FewShot
Demonstrate type inference for UserBuilder,
then ask about the CompanyRepository case.
UserBuilderの模範解答を入力してから、CompanyRepositoryについて推論させる
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457932/ai_write.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457934/test100.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457936/test_print_mondaiyoushi.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457934/test100.png)
Demo Input
Main Question
Output
模範解答
メインの問い合わせ
出力
Improve: FewShot
Demo Input
Main Question
模範解答を入力
メインの問い合わせ
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457912/スクリーンショット_2023-05-09_10.50.03.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457913/スクリーンショット_2023-05-09_10.50.09.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457914/スクリーンショット_2023-05-09_10.50.14.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/482970/images/10457922/スクリーンショット_2023-05-09_10.50.20.png)
Really Perfect!!!
👏
Please talk to me at parties, etc.
ぜひドリンクアップなどで話しかけてください!
I've tried to infer types for metaprogramming,
refine existing RBS, etc.,
but I don't have time to talks. So,
メタプログラミングコードに対する型の推測や既存RBSの詳細化なども試しましたが
今回は話す時間が足りないので、
Future Works
CommandLine Tool (コマンドラインツール化)
Fine tuning for type inference (型推測用のファインチューニング)
-
Agent-style autonomous drive (エージェントスタイルの自律駆動)
Refer to the necessary files (必要なファイルの参照)
Correction with type check results (型チェックを元にした修正)
-
Run with LLMs locally (ローカルで動くLLMを使った動作)
Alpaca.cpp, ChatRWKV, or else
RBS meets LLMs - Type inference using LLM
By 黒曜
RBS meets LLMs - Type inference using LLM
LT of RubyKaigi 2023
- 1,272