Mind+Drones

Tony Catapano

  • šŸ‘Øā€šŸ’» Software engineer
  • āš™ļøĀ Tinkerer (IOT, Bluetooth, Android, NFC)
  • šŸššŸ’Ø FPV Drone pilot
  • Ā tonnozĀ on the web

tonnoz

I like to fly drones

I build and fly racing drones for fun

in First Person View : with goggles on my face and real-time analog video feed

there was a tornado in Amsterdam 2 weeks ago...

...but I missed it

tonnoz

AR Drone 2.0

  • only ~100 bucks nowadays
  • packed with sensors
  • perfect for tinkering

tonnoz

AR Drone 2.0

tonnoz

tonnoz

It runs Busybox Linux

running services and ports

  FTP            : 5551
  AUTH           : 5552
  VIDEO_RECORDER : 5553
  NAVDATA        : 5554
  VIDEO          : 5555
  AT             : 5556*
  RAW_CAPTURE    : 5557
  PRINTF         : 5558
  CONTROL        : 5559

tonnoz

Commands are sent over UDP...

  • The protocol used to send/receive data is UDPĀ  which is very unreliable (does not guarantee the packet delivery). You basically send UDP packets to the Drone and pray that the drone will receive them
  • Clients must repeatedly send their instructions and include an incrementing sequence number with each command. E.g. this is the UDP command to takeoff:
AT*REF=1,512\r

tonnoz

and works via an open SSID / Telnet šŸ˜…

it is very, very easy to hack:

Ā just Telnet (no password) to the drone and make it crash in few terminal commands

(please do not try it during the demo šŸ™ )Ā 

$ telnet 192.168.1.1
$ reboot

tonnoz

ar-droneĀ npm package and The NodeCopter initiative to the rescue

tonnoz

Matador

var arDrone = require('ar-drone');
var client = arDrone.createClient();

client.takeoff();

client
  .after(5000, function() {
    this.clockwise(0.5);
  })
  .after(3000, function() {
    this.animate('flipLeft', 15);
  })
  .after(1000, function() {
    this.stop();
    this.land();
  });
$ npm install ar-drone

Fly a drone in 15 lines of code

tonnoz

from drones to the mind..

is developing implantable brainā€“machine interfaces (BMIs).

tonnoz

Neuralink (July 2019)

Neuralink is developing an high end brainā€“machine interface.Ā It is basically an ultra-fast bandwidth brain reader. The device will require a surgical operation to implant up to 96 micro threads able to carry up to 3,072 electrodes. These filaments are extremely thin (less than half a hair thick) to avoid damaging blood vessels in the brain

The external module will use Bluetooth to communicate to consumer-enabled devices (e.g. mobile phones)

tonnoz

Concerns from the future

Elon Musk reveled from a recent chat with Jack Ma (co-founder of Alibaba) at World AI Conference 2019Ā that the scope of Neuralink is to evolve the human specie to transhumanĀ in order to not not succumb to an event referred as Technological SingularityĀ , estimated to happen within the next 20 years.

tonnoz

An optimistic view..or not?

At the moment surgical brain operations are forbidden on humans that aren't in terminal conditions...

Inception for ads?

Thoughts data-leaks?

tonnoz

Meanwhile at Facebook (2017)

from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan

tonnoz

ALS Ā 

8 wpm (invasive implant, state of the art 2017)

from: Toward Clinically-Viable Brain-Machine Interfaces - prof. Krishna Shenoy - Stanford

tonnoz

Why

How much data is your brain processing right now?

from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan

tonnoz

Why

from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan

tonnoz

Why

from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan

Ā«Speech is basically a lossy compression algorithm..Ā»

Regina Dugan - VP of Engineering at Facebook

tonnoz

Does free will really exist?

from: Free Will & Neurology: brain activity to conscious decision

Ā 

A recent study has shown that when about to take a decision, a signal start to rise beforeĀ the conscious will to do it (~200ms before)

tonnoz

Facebook bets on non-invasive devices

from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan

tonnoz

P300: the oddball paradigm (non invasive)

  • Technique used to type based on randomly flashing letters in a matrix.
  • Based on rare positive/common negative flashes. Measure peak interest.
  • Action potentials fire 300ms after stimulation.
  • Available for 15 years at least, improving a lot in speed lately

tonnoz

E.E.G.

These devices are called E.E.G. (E.lectroE.ncephaloG.ram) and are the same kind that are used in hospitals for specific brain exams. They are used to measure specific frequencies that travel through different areas of the brain. They are useful to reveal patologies related to common electrical brain patterns

Ā 

tonnoz

How do they work?

By tracking changes or peaks in these frequencies and the location where they occurred, it is possible to decode them into data and therefore use it as input for all sorts of devices

10-20 system (where to position electrodes)

tonnoz

From chemical elements to Volts to [ {}, {}, ... {} ]

  • ~90 billion neurons firing every milliseconds (comparable to the nr. of stars in our galaxy)

  • Chemical reaction of Potassium (K) and Sodium (Na) produces a difference in potential (an "action potential") that can be measured

  • These analog signals can be measured and converted into digital data (e.g. array of objects)

  • Depending on the hardware, the data you get may be less or more "dirty"

  • By measuring these frequencies and mapping them to specific areas of the brain, it is possible via algorithms and AI, Ā to decode high level concepts and complex patterns like emotions or will,Ā or even predict behaviors

picture: Armagan Amcalaā€‹ā€‹r @ Codemotion 2018

tonnoz

HOW :

Brain waves

picture: Electroencephalography on Wikipedia

tonnoz

Star Wars: Force Trainer 2

for 40 euros you can get a cheap EEG device like this one

tonnoz

Hacking Star Wars toys

[
  { eSense: { attention: 75, meditation: 47 },
  { eegPower:
     { delta: 27898,
       theta: 8401,
       lowAlpha: 842,
       highAlpha: 1632,
       lowBeta: 532,
       highBeta: 2897,
       lowGamma: 557,
       highGamma: 1740 },
    poorSignalLevel: 0 },
  { blinkStrength: 68 }
]
  • 1 reading per second šŸ˜ž
  • 1 async extra reading when blinking šŸ˜‰

tonnoz

I missed the tornado so..

Demo time!

but before starting...

tonnoz

Remember: always add keyboard commands override when working with Drones & EEGs šŸ™ˆ

tonnoz

Trigger a Bamboo build

with your Mind?

Why not!

Demo!

tonnoz

Questions?

tonnoz

Codebase:

Slides:

Write me:

tonnoz

https://slides.com/tonnoz/mind-drone

https://github.com/tonnoz/mind-drone

Conclusions

The future may look intimidating, with brain implants and consciousness replacements like in Total-Recall. But it is not. It is up to us to put trust in our specie, build a safe future and why not, have fun flying robots. Until robots will not be able to fly us instead...

Prototype of drone taxi by Audi / Airbus

from Amsterdam Drone week 2018

Be curious and fly high! šŸš€

tonnoz

Mind+Drones?

By tonnoz

Made with Slides.com

Mind+Drones?

Flying a drone with nodejs via an EEG device? Let's talk about it.

  • 507

More from tonnoz