Mind+Drones
Tony Catapano
- šØāš» Software engineer
- āļøĀ Tinkerer (IOT, Bluetooth, Android, NFC)
- ššØ FPV Drone pilot
- Ā tonnozĀ on the web
I like to fly drones
I build and fly racing drones for fun
in First Person View : with goggles on my face and real-time analog video feed
there was a tornado in Amsterdam 2 weeks ago...
...but I missed it
AR Drone 2.0
- only ~100 bucks nowadays
- packed with sensors
- perfect for tinkering
AR Drone 2.0
It runs Busybox Linux
running services and ports
FTP : 5551
AUTH : 5552
VIDEO_RECORDER : 5553
NAVDATA : 5554
VIDEO : 5555
AT : 5556*
RAW_CAPTURE : 5557
PRINTF : 5558
CONTROL : 5559
Commands are sent over UDP...
- The protocol used to send/receive data is UDPĀ which is very unreliable (does not guarantee the packet delivery). You basically send UDP packets to the Drone and pray that the drone will receive them
- Clients must repeatedly send their instructions and include an incrementing sequence number with each command. E.g. this is the UDP command to takeoff:
AT*REF=1,512\r
and works via an open SSID / Telnet š
it is very, very easy to hack:
Ā just Telnet (no password) to the drone and make it crash in few terminal commands
(please do not try it during the demo š )Ā
$ telnet 192.168.1.1
$ reboot
ar-droneĀ npm package and The NodeCopter initiative to the rescue
Matador
var arDrone = require('ar-drone');
var client = arDrone.createClient();
client.takeoff();
client
.after(5000, function() {
this.clockwise(0.5);
})
.after(3000, function() {
this.animate('flipLeft', 15);
})
.after(1000, function() {
this.stop();
this.land();
});
$ npm install ar-drone
Fly a drone in 15 lines of code
from drones to the mind..
is developing implantable braināmachine interfaces (BMIs).
Neuralink (July 2019)
Neuralink is developing an high end braināmachine interface.Ā It is basically an ultra-fast bandwidth brain reader. The device will require a surgical operation to implant up to 96 micro threads able to carry up to 3,072 electrodes. These filaments are extremely thin (less than half a hair thick) to avoid damaging blood vessels in the brain
The external module will use Bluetooth to communicate to consumer-enabled devices (e.g. mobile phones)
Concerns from the future
Elon Musk reveled from a recent chat with Jack Ma (co-founder of Alibaba) at World AI Conference 2019Ā that the scope of Neuralink is to evolve the human specie to transhumanĀ in order to not not succumb to an event referred as Technological SingularityĀ , estimated to happen within the next 20 years.
An optimistic view..or not?
At the moment surgical brain operations are forbidden on humans that aren't in terminal conditions...
Inception for ads?
Thoughts data-leaks?
Meanwhile at Facebook (2017)
from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan
ALS Ā
8 wpm (invasive implant, state of the art 2017)
from: Toward Clinically-Viable Brain-Machine Interfaces - prof. Krishna Shenoy - Stanford
Why
How much data is your brain processing right now?
from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan
Why
from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan
Why
from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan
Ā«Speech is basically a lossy compression algorithm..Ā»
Regina Dugan - VP of Engineering at Facebook
Does free will really exist?
A recent study has shown that when about to take a decision, a signal start to rise beforeĀ the conscious will to do it (~200ms before)
Facebook bets on non-invasive devices
from: F8 - Building 8 (Mind Reading Technology) - Regina Dugan
P300: the oddball paradigm (non invasive)
- Technique used to type based on randomly flashing letters in a matrix.
- Based on rare positive/common negative flashes. Measure peak interest.
- Action potentials fire 300ms after stimulation.
- Available for 15 years at least, improving a lot in speed lately
E.E.G.
These devices are called E.E.G. (E.lectroE.ncephaloG.ram) and are the same kind that are used in hospitals for specific brain exams. They are used to measure specific frequencies that travel through different areas of the brain. They are useful to reveal patologies related to common electrical brain patterns
Ā
How do they work?
By tracking changes or peaks in these frequencies and the location where they occurred, it is possible to decode them into data and therefore use it as input for all sorts of devices
10-20 system (where to position electrodes)
From chemical elements to Volts to [ {}, {}, ... {} ]
-
~90 billion neurons firing every milliseconds (comparable to the nr. of stars in our galaxy)
-
Chemical reaction of Potassium (K) and Sodium (Na) produces a difference in potential (an "action potential") that can be measured
-
These analog signals can be measured and converted into digital data (e.g. array of objects)
-
Depending on the hardware, the data you get may be less or more "dirty"
-
By measuring these frequencies and mapping them to specific areas of the brain, it is possible via algorithms and AI, Ā to decode high level concepts and complex patterns like emotions or will,Ā or even predict behaviors
picture: Armagan Amcalaāār @ Codemotion 2018
HOW :
Brain waves
picture: Electroencephalography on Wikipedia
Star Wars: Force Trainer 2
for 40 euros you can get a cheap EEG device like this one
Hacking Star Wars toys
[
{ eSense: { attention: 75, meditation: 47 },
{ eegPower:
{ delta: 27898,
theta: 8401,
lowAlpha: 842,
highAlpha: 1632,
lowBeta: 532,
highBeta: 2897,
lowGamma: 557,
highGamma: 1740 },
poorSignalLevel: 0 },
{ blinkStrength: 68 }
]
- 1 reading per second š
- 1 async extra reading when blinking š
I missed the tornado so..
Demo time!
but before starting...
Remember: always add keyboard commands override when working with Drones & EEGs š
Trigger a Bamboo build
with your Mind?
Why not!
Demo!
Questions?
Codebase:
Slides:
Write me:
Conclusions
The future may look intimidating, with brain implants and consciousness replacements like in Total-Recall. But it is not. It is up to us to put trust in our specie, build a safe future and why not, have fun flying robots. Until robots will not be able to fly us instead...
Prototype of drone taxi by Audi / Airbus
from Amsterdam Drone week 2018
Be curious and fly high! š
Mind+Drones?
By tonnoz
Mind+Drones?
Flying a drone with nodejs via an EEG device? Let's talk about it.
- 652