Bringing ADB into the Firefox Process
Brandon Kase
8/1/13
outline
- Motivation
- Demo
- What I did + Technologies
- Architecture
motivation
adb
android debug bridge
+---------------+ +------------+ | +-----------> | | Android Phone| |Laptop | | <-----------+ | +---------------+ +------------+
adb
Firefoxos debug bridge
+---------------+ +------------+ | +-----------> | |FirefoxOS Phone| |Laptop | | <-----------+ | +---------------+ +------------+
adb
Super high-level overview
+-----------------+ +------------------+ | | adb is running | | | | | | | $ adb devices +--------------->|full_unagi device| | |IPC with sockets| | | | | | +--------+--------+ +--------------^---+ | | | |IPC with sockets | adb not running | | +-------------------------+ | | | Fork a subprocess | | +----->| (2 seconds of sleeping) +-------+ | | +-------------------------+
using ADB and its daemon to implement an adb Api?
- Mixture of adb in a subshell and raw sockets
- Sometimes wait for initialization and subprocess forking
- Have to kill the forked daemon when you're done
It's not ideal
Who uses adb as an api?
firefoxos simulator
outline
-
Motivation
- Demo
- What I did + Technologies
- Architecture
< Demo >
outline
-
Motivation
-
Demo - What I did + Technologies
- Architecture
so what did I do?
So what did I do?
- Compiled ADB as a shared library
-
Loaded ADB into the Simulator add-on using js-ctypes
- Started porting ADB to JavaScript
- Changed some ADB behavior
Getting the source
- ADB is open source (Apache-licensed)
- Get it:
$ git clone https://android.googlesource.com/platform/system/core
- Or (Debian's android-tools-adb)
- http://packages.debian.org/sid/android-tools-adb
Compiling adb as a shared libary
- Ported the Makefile from Debian's android-tools-adb
- Linux
- Easy
- Mac
- Easy
- Windows
- Nightmare
Compiling adb as a shared library
on windows
- Ported to Visual Studio's cl.exe compiler
- cl.exe doesn't support
- C11
- C99
- Only C89
- ADB is not written against C89
- Compile most things as C++
- C++ is not a superset of C
- Had to go through thousands of compile errors
Running adb in the simulator
- So it compiled
- Did it work?
- Linux
- Mostly
- Mac
- Mostly
- Windows
- Of course not
how do you load native code from javascript
js-ctypes
js-ctypes
is super cool
js-ctypes
Foreign-function interface to C from JavaScript
let { Cu } = require('chrome');
Cu.import("resources://gre/modules/ctypes.jsm");
let lib = ctypes.open("some_lib.so");
let some_function = lib.declare("some_function",
default_abi,
ctypes.int,
ctypes.char.ptr);
let x = some_function("just some string");
js-ctypes
Small wrapper I made for declaring functions
let lib = ctypes.open("some_lib.so");
I.declare({ name: "some_function",
returns: ctypes.int,
args: [ ctypes.char.ptr ]
}, lib);
... /* later in the code */
let some_function = I.use("some_function");
some_function("just some string");
Avoids globals and has easy-to-remember syntaxJS-ctypes
- Now you know how to call C code from JS
- How do you call JS code from C?
js-ctypes
It's complicated (call a function with a C function pointer)
/* C code */
DLL_EXPORT void long_task(void (*callback)(char *)) {
char * the_answer = something_long();
callback(the_answer);
}
/* JS code */
let CallbackType = ctypes.FunctionType(ctypes.default_abi,
ctypes.int,
[ ctypes.char.ptr ]);
let long_task = I.declare({ name: "long_task",
returns: ctypes.void_t,
args: [ CallbackType.ptr ]
}, lib);
let callbackFn = function(str) { console.log("in JS", str.readString()); };
long_task( CallbackType.ptr(callbackFn) );
It's worse in the real codejs-ctypes
Fix?
/* C code */
void long_task() {
struct a_message {
char * str;
};
struct a_message m = { something_long() }; void * res;
MSG(&res, "achannel", m);
}
/* JS code */
let jsMsgFn = function(channel, args) {
switch(channel.readString()) {
case "achannel":
let [ x ] = JsMessage.unpack(args, ctypes.char.ptr);
return JsMessage.pack(11, Number);
}
};
Use MSG macro to avoid modifying much native codeback to running adb
why didn't adb
"just work"
Why didn't adb "just work'
-
ADB calls
exit(1)
for minor errors - ADB doesn't cleanup before quitting
- ADB calls
exit(1)
to quit in every code path
-
exit(1)
crashes Firefox
Why didn't adb just work
- Needed more control over threads
- Moved threads to JavaScript Chrome Workers
- Chrome Worker: Web Worker with access to js-ctypes
- JavaScript callbacks can only be called from the same thread that they were created on
- Chrome Workers don't have access to Chrome (ironically)
- Chrome Workers can't spawn Chrome Workers (have to send a message back to UI thread)
- Made EventedChromeWorker object to help with this
Why didn't adb just work
- Windows
- If you kill a Chrome Worker while it is executing native code
- Linux:
- The runtime finishes executing the native code until it returns from all functions
- Mac:
- The runtime finishes executing the native code until it returns from all functions
- Windows:
- The runtime crashes Firefox
- Synchronous unloading of add-on
it works!
-
+ 38,104 additions
-
- 804 deletions
Before:
After:
features
- Supports all commands needed by the FirefoxOS Simulator
- adb push
- adb devices
- raw device tracking
- adb forward (port forwarding)
- adb shell
- First "layer" and-a-half of ADB ported to JavaScript
- Device tracking ported to JavaScript
TODO
- Port all "layers" to JavaScript
- Make sure all memory leaks are gone
- For now, there are only leaks if you disable or uninstall the Simulator add-on and then enable or reinstall it without restarting Firefox in between
- Make Linux and Mac exit code less hacky (more stable)
- As a side-effect of being so crashy, Windows exit handling is actually pretty robust now
outline
-
Motivation
-
Demo What I did + Technologies- Architecture
What are the "layers"
how does adb work? (before and after)
adb internals
Before
+------------+ +----------------+ | PHONE | | COMPUTER | | | Platform-specific | +------------+ | | +--------+ | drivers | | adb daemon | | | | adbd |+--------------------->| process | | | | |<---------------------++-^+---------+ | | +--------+ | USB | || | +------------+ | || SOCKETS | | || | | || | | || | | +-+v---------+ | | | adb client | | | | process | | | | | | | | | | | | | | | +------------+ | | | +----------------+
ADB INTERNALS
After
+------------+ +----------------+ | PHONE | | COMPUTER | | | Platform-specific | +------------+ | | +--------+ | drivers | | | | | | adbd |+--------------------->| Simulator | | | | |<---------------------+| Add-on | | | +--------+ | USB | | in | | +------------+ | | Firefox | | | | process | | | | | | | | | | | +------------+ | | | +----------------+
adb internals, deeper
- Architecture broken up into "layers"
- ADB commandline handling
- ADB client services
- Transport layer
- fdevents
- Platform-specific driver code
ADB commandline handling
- Mostly string parsing and other initialization code
- Completely gone now
ADB client services
- A service returns a file descriptor
/* C code */
int fd = _adb_connect("shell:ls");
/* JS code */
let output = readFully(fd);
console.log(output);- Pretty cool abstraction
- In some places unnecessary (should callback to JS directly)
transport layer
- Forwards packets from device over USB to sockets
- Part of backend for the ADB client services
- Due to limitations of Mac and Linux needs threads and synchronous reads and writes over USB
- Used to tracks devices
- Now pure JavaScript
fdevents
- Event loop system based off of file descriptors
- Runs on main server thread
- Backed by
- Linux: epoll
- Mac: select
- Windows: select
- Fun Fact:
- There is a 1000+ line Windows shim for fdevents and exposing an interface to read/write from an
int fd
just like on Unix
platform specific driver code
- Platform-specific
- Long
- Complex
- Would be hard to port to JavaScript
Adb internals, deeper, deeper
Workers Per phone UI Server Device Loop Input Output JS:Simulator starts ADB + | +--------->JS:Set up code + | | (really C to JS to UI to new worker to C) v C:server_thread+-->C:Start device + loop | (look for devices) v C:fdevent_loop + | on new device JS:device tracker<-------++------------------------------> C:Write to device | +--------------------------------------------->C:Read
from device
THanks
questions?
Bringing ADB to the Firefox Process
By bkase
Bringing ADB to the Firefox Process
- 2,034