PLAYING MINECRAFT
USING AI
Tiffany Souterre & Wassim Chegham
DISCLAIMER
The code shared in this project is for educational purposes only.
The code is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement. In no event shall We (the authors) or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the code or the use or other dealings in the code.
DEMO MAY NOT WORK!
THIS IS JUST A PROOF OF CONCEPT. DON'T USE IT IN PRODUCTION!
Wassim Chegham
github.com/manekinekko
Tiffany Souterre
github.com/amagash
@TiffanySouterre
Sr Developer Advocate Engineer (Azure)
Developer Relations
IA/ML
* We do not work nor represent GitHub!
amagash.github.io
GDE Web Technologies
WTM ambassador
Minecraft
A game that allows users to generate infinite & open worlds.
NPC: Non-Player Characters.
OpenAI is an ai research laboratory.
DALL-E, GPT-3, GPT-2, OpenAI Gym...
GPT-3
Codex
Trained on Code
Codex example w/ GitHub Copilot.
Can We Use AI To Control NPCs?
"jump"
Codex
Context + "jump"
bot.setControlState('jump', true)
Show me the code!
https://github.com/manekinekko/minecraft-openai
Diagram – Mineflayer
Minecraft NPC
Minecraft Local Server
Local Agent (mineflayer)
User Commands
Code – Mineflayer
import mineflayer from "mineflayer";
const bot = mineflayer.createBot({
username: "OpenAI",
host: "localhost",
port: 56794,
});
bot.on("chat", async (username, input) => {
if (username === bot.username) return;
console.log(input);
});Diagram – OpenAI
Minecraft NPC
Minecraft Local Server
Local Agent (mineflayer)
OpenAI server
User Commands
// (previous context)
// Go forward
bot.setControlState('forward', true);
Code – Codex
import mineflayer from "mineflayer";
import { context, callOpenAI, updateContext, eval } from "./core.js";
const bot = mineflayer.createBot({
username: "OpenAI",
host: "localhost",
port: 56794,
});
bot.on("chat", async (username, input) => {
if (username === bot.username) return;
const previousContext = context();
const response = await callOpenAI(input, previousContext);
eval(code);
updateContext(input, code);
});Simple Context
// Go forward
bot.setControlState('forward', true);
// Go back
bot.setControlState('back', true);
// jump
bot.setControlState('jump', true);
// Hi how are you ?
bot.chat("I'm fine, thanks!");
// What's your name ?
bot.chat("My name is " + bot.username);const STOP_WORD = "// ";
const EOL = "\n";
const openAIkey = process.env.CODEX_API_KEY;
const body = {
prompt: context + STOP_WORD + input + EOL,
max_tokens: 300,
temperature: 0,
stop: STOP_WORD,
n: 1,
};
const response = await fetch("https://api.openai.com/v1/engines/davinci-codex/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${openAIkey}`,
},
body: JSON.stringify(body),
});
if (!response.ok) {
error("api response failed with statis %s", response.statusText);
return;
}
return await response.json();Simple Context
Advanced Context
// Get bot's current position
bot.entity.position;
// Stop any movement
bot.clearControlStates();
// Come to me
findPlayer(bot, 2, target);
// Follow me
goToPlayerInterval = setInterval(() => followPlayer(bot, 2, target), 1000);
// Stop following me
clearInterval(goToPlayerInterval);
// Get dirt
mineBlock(bot, "dirt", mcData);
// Get oak log
mineBlock(bot, "oak_log");