PLAYING MINECRAFT

USING AI

Tiffany Souterre & Wassim Chegham

DISCLAIMER

The code shared in this project is for educational purposes only.

 

The code is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement. In no event shall We (the authors) or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the code or the use or other dealings in the code.

DEMO MAY NOT WORK!

THIS IS JUST A PROOF OF CONCEPT. DON'T USE IT IN PRODUCTION!

Wassim Chegham

github.com/manekinekko

Tiffany Souterre

github.com/amagash

@TiffanySouterre

Sr Developer Advocate Engineer (Azure)

Developer Relations

IA/ML

* We do not work nor represent GitHub!

amagash.github.io

GDE Web Technologies

WTM ambassador

Minecraft

A game that allows users to generate infinite & open worlds.

NPC: Non-Player Characters.

OpenAI is an ai research laboratory.

DALL-E, GPT-3, GPT-2, OpenAI Gym...

GPT-3

Codex

Trained on Code

Codex example w/ GitHub Copilot.

Can We Use AI To Control NPCs?

"jump"

Codex

Context + "jump"

bot.setControlState('jump', true)

Show me the code!

https://github.com/manekinekko/minecraft-openai

Diagram – Mineflayer

Minecraft NPC

Minecraft Local Server

Local Agent (mineflayer)

User Commands

Code – Mineflayer

import mineflayer from "mineflayer";

const bot = mineflayer.createBot({
  username: "OpenAI",
  host: "localhost",
  port: 56794,
});
  
bot.on("chat", async (username, input) => {
  if (username === bot.username) return;
  
  console.log(input);

});

Diagram – OpenAI

Minecraft NPC

Minecraft Local Server

Local Agent (mineflayer)

OpenAI server

User Commands

// (previous context)
// Go forward
bot.setControlState('forward', true);

Code – Codex

import mineflayer from "mineflayer";
import { context, callOpenAI, updateContext, eval } from "./core.js";

const bot = mineflayer.createBot({
  username: "OpenAI",
  host: "localhost",
  port: 56794,
});
  
bot.on("chat", async (username, input) => {
  if (username === bot.username) return;
  
  const previousContext = context();
  const response = await callOpenAI(input, previousContext);
  eval(code);
  updateContext(input, code);

});

Simple Context

  // Go forward
  bot.setControlState('forward', true);

  // Go back
  bot.setControlState('back', true);

  // jump
  bot.setControlState('jump', true);

  // Hi how are you ?
  bot.chat("I'm fine, thanks!");

  // What's your name ?
  bot.chat("My name is " + bot.username);
const STOP_WORD = "// ";
const EOL = "\n";
const openAIkey = process.env.CODEX_API_KEY;

const body = {
  prompt: context + STOP_WORD + input + EOL,
  max_tokens: 300,
  temperature: 0,
  stop: STOP_WORD,
  n: 1,
};

const response = await fetch("https://api.openai.com/v1/engines/davinci-codex/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${openAIkey}`,
  },
  body: JSON.stringify(body),
});

if (!response.ok) {
  error("api response failed with statis %s", response.statusText);
  return;
}

return await response.json();

Simple Context

Advanced Context

// Get bot's current position
bot.entity.position;

// Stop any movement
bot.clearControlStates();

// Come to me
findPlayer(bot, 2, target);

// Follow me 
goToPlayerInterval = setInterval(() => followPlayer(bot, 2, target), 1000);

// Stop following me
clearInterval(goToPlayerInterval);

// Get dirt
mineBlock(bot, "dirt", mcData);

// Get oak log
mineBlock(bot, "oak_log");

Advanced Context

Your turn to make it better!

Thank You

Playing Minecraft with OpenAI & GPT-3

By Wassim Chegham

Playing Minecraft with OpenAI & GPT-3

There are players who play Minecraft for the pleasure of playing, others to develop their creativity. But there is another way to play Minecraft, and that is by using an artificial intelligence generated by GPT-3. In this presentation, we'll talk a bit about AI and ML, GPT-3, and Codex, but most of all, we'll have fun generating code to control a bot in Minecraft, all in a good mood! Are you tempted?

  • 5,893