I built a robot to cheat at Pokémon!!

Sophie Déziel

Rails dev @ Acquisio

Twitch Streamer


Guiz de Pessemier

Co-founder of Outerminds

Youtuber and Twitch Streamer



Automation to the rescue!!

T. eam

R. ocket

A. utomated

S. hiny

H. unter

Two challenges

  • Controlling the game
  • Have input to make decisions

Control the game options

  • Open an modify a Joy Con
  • Linear actuators
  • Servo-motors


  • Cheap and easy to find
  • You can position them precisely
  • Easy to control
  • Slow

Image input options

  • HDMI capture card
  • Color sensor
  • Webcam


  • Easy to position close to the screen
  • Lots of Open-Source Software to control them
  • Plug and Play (most of the time)

We can go forward

We need to make it:

  • Reliable
  • Precise
  • Easy to code
  • Configurable

Make it reliable

  • Work for several hours unsupervised
  • Perfect timing
  • Must not move

Precise with "Computer vision"

  • Easy to setup
  • Grab a screenshot on demand
  • Isolate a single pixel
  • Check hue inclusion in a range
  • False positives vs false negatives

Easy to code and test

  • Use Arduino + Firmata
  • Script can be in any general purpose programming language
  • ImageMagick to pick a single pixel's hue
loop do
  press(:a, @alolan[:delay_1])
  press(:a, @alolan[:delay_2])
  press(:a, @alolan[:delay_3])
  press(:a, @alolan[:delay_4])
  press(:a, @alolan[:delay_5])
  press(:a, @alolan[:delay_6])
  press(:a, @alolan[:delay_7])
  press(:a, @alolan[:delay_8])

  fire('shiny') if shiny?

  light(:not_shiny, true)
  sleep @alolan[:delay_9]

  press(:a, @alolan[:delay_10])
  press(:a, @alolan[:delay_11])
  press(:a, @alolan[:delay_12])
  light(:not_shiny, false)



  • Control remotely
  • Edit timing, connections and color ranges
  • Visualise the webcam

Putting it all together

The working prototype!


Unplanned features

  • Controls improvements
  • Easy updates
  • Communication features (SMS and Tweets)

And now?

Want to build it?

So cool!


You can modify it:

  • Add Lights and lasers
  • Remove the Arduino and use the RP GPIO
  • Improve the interface
  • Adapt the codebase for another game

Thank you so much


OS Hardware

  • Raspberry Pi
  • Arduino


  • Raspbian
  • Motion
  • Firmata
  • Ruby
  • Rails
  • RMagick and Image Magick
  • Nginx
  • Git

Copy of I built a robot to cheat at Pokémon!!

By Sophie Déziel

Copy of I built a robot to cheat at Pokémon!!

!!Con version

  • 1,190