Bringing ADB into the Firefox Process
Brandon Kase
8/1/13
outline
- Motivation
- Demo
- What I did + Technologies
- Architecture
motivation
adb
android debug bridge
+---------------+ +------------+
| +-----------> |
| Android Phone| |Laptop |
| <-----------+ |
+---------------+ +------------+
adb
Firefoxos debug bridge
+---------------+ +------------+
| +-----------> |
|FirefoxOS Phone| |Laptop |
| <-----------+ |
+---------------+ +------------+adb
Super high-level overview
+-----------------+ +------------------+
| | adb is running | |
| | | |
| $ adb devices +--------------->|full_unagi device|
| |IPC with sockets| |
| | | |
+--------+--------+ +--------------^---+
| |
| |IPC with sockets
| adb not running |
| +-------------------------+ |
| | Fork a subprocess | |
+----->| (2 seconds of sleeping) +-------+
| |
+-------------------------+
using ADB and its daemon to implement an adb Api?
- Mixture of adb in a subshell and raw sockets
- Sometimes wait for initialization and subprocess forking
- Have to kill the forked daemon when you're done
It's not ideal
Who uses adb as an api?
firefoxos simulator
outline
-
Motivation
- Demo
- What I did + Technologies
- Architecture
< Demo >
outline
-
Motivation
-
Demo - What I did + Technologies
- Architecture
so what did I do?
So what did I do?
- Compiled ADB as a shared library
-
Loaded ADB into the Simulator add-on using js-ctypes
- Started porting ADB to JavaScript
- Changed some ADB behavior
Getting the source
- ADB is open source (Apache-licensed)
- Get it:
$ git clone https://android.googlesource.com/platform/system/core - Or (Debian's android-tools-adb)
- http://packages.debian.org/sid/android-tools-adb
Compiling adb as a shared libary
- Ported the Makefile from Debian's android-tools-adb
- Linux
- Easy
- Mac
- Easy
- Windows
- Nightmare
Compiling adb as a shared library
on windows
- Ported to Visual Studio's cl.exe compiler
- cl.exe doesn't support
- C11
- C99
- Only C89
- ADB is not written against C89
- Compile most things as C++
- C++ is not a superset of C
- Had to go through thousands of compile errors
Running adb in the simulator
- So it compiled
- Did it work?
- Linux
- Mostly
- Mac
- Mostly
- Windows
- Of course not
how do you load native code from javascript
js-ctypes
js-ctypes
is super cool
js-ctypes
Foreign-function interface to C from JavaScript
let { Cu } = require('chrome');
Cu.import("resources://gre/modules/ctypes.jsm");
let lib = ctypes.open("some_lib.so");
let some_function = lib.declare("some_function",
default_abi,
ctypes.int,
ctypes.char.ptr);
let x = some_function("just some string");
js-ctypes
Small wrapper I made for declaring functions
let lib = ctypes.open("some_lib.so");
I.declare({ name: "some_function",
returns: ctypes.int,
args: [ ctypes.char.ptr ]
}, lib);
... /* later in the code */
let some_function = I.use("some_function");
some_function("just some string");
JS-ctypes
- Now you know how to call C code from JS
- How do you call JS code from C?
js-ctypes
It's complicated (call a function with a C function pointer)
/* C code */
DLL_EXPORT void long_task(void (*callback)(char *)) {
char * the_answer = something_long();
callback(the_answer);
}
/* JS code */
let CallbackType = ctypes.FunctionType(ctypes.default_abi,
ctypes.int,
[ ctypes.char.ptr ]);
let long_task = I.declare({ name: "long_task",
returns: ctypes.void_t,
args: [ CallbackType.ptr ]
}, lib);
let callbackFn = function(str) { console.log("in JS", str.readString()); };
long_task( CallbackType.ptr(callbackFn) );
js-ctypes
Fix?
/* C code */
void long_task() {
struct a_message {
char * str;
};
struct a_message m = { something_long() }; void * res;
MSG(&res, "achannel", m);
}
/* JS code */
let jsMsgFn = function(channel, args) {
switch(channel.readString()) {
case "achannel":
let [ x ] = JsMessage.unpack(args, ctypes.char.ptr);
return JsMessage.pack(11, Number);
}
};
back to running adb
why didn't adb
"just work"
Why didn't adb "just work'
-
ADB calls
exit(1)for minor errors - ADB doesn't cleanup before quitting
- ADB calls
exit(1)to quit in every code path
-
exit(1)crashes Firefox
Why didn't adb just work
- Needed more control over threads
- Moved threads to JavaScript Chrome Workers
- Chrome Worker: Web Worker with access to js-ctypes
- JavaScript callbacks can only be called from the same thread that they were created on
- Chrome Workers don't have access to Chrome (ironically)
- Chrome Workers can't spawn Chrome Workers (have to send a message back to UI thread)
- Made EventedChromeWorker object to help with this
Why didn't adb just work
- Windows
- If you kill a Chrome Worker while it is executing native code
- Linux:
- The runtime finishes executing the native code until it returns from all functions
- Mac:
- The runtime finishes executing the native code until it returns from all functions
- Windows:
- The runtime crashes Firefox
- Synchronous unloading of add-on
it works!
-
+ 38,104 additions
-
- 804 deletions
Before:
After:
features
- Supports all commands needed by the FirefoxOS Simulator
- adb push
- adb devices
- raw device tracking
- adb forward (port forwarding)
- adb shell
- First "layer" and-a-half of ADB ported to JavaScript
- Device tracking ported to JavaScript
TODO
- Port all "layers" to JavaScript
- Make sure all memory leaks are gone
- For now, there are only leaks if you disable or uninstall the Simulator add-on and then enable or reinstall it without restarting Firefox in between
- Make Linux and Mac exit code less hacky (more stable)
- As a side-effect of being so crashy, Windows exit handling is actually pretty robust now
outline
-
Motivation
-
Demo What I did + Technologies- Architecture
What are the "layers"
how does adb work? (before and after)
adb internals
Before
+------------+ +----------------+
| PHONE | | COMPUTER |
| | Platform-specific | +------------+ |
| +--------+ | drivers | | adb daemon | |
| | adbd |+--------------------->| process | |
| | |<---------------------++-^+---------+ |
| +--------+ | USB | || |
+------------+ | || SOCKETS |
| || |
| || |
| || |
| +-+v---------+ |
| | adb client | |
| | process | |
| | | |
| | | |
| | | |
| +------------+ |
| |
+----------------+ADB INTERNALS
After
+------------+ +----------------+
| PHONE | | COMPUTER |
| | Platform-specific | +------------+ |
| +--------+ | drivers | | | |
| | adbd |+--------------------->| Simulator | |
| | |<---------------------+| Add-on | |
| +--------+ | USB | | in | |
+------------+ | | Firefox | |
| | process | |
| | | |
| | | |
| +------------+ |
| |
+----------------+
adb internals, deeper
- Architecture broken up into "layers"
- ADB commandline handling
- ADB client services
- Transport layer
- fdevents
- Platform-specific driver code
ADB commandline handling
- Mostly string parsing and other initialization code
- Completely gone now
ADB client services
- A service returns a file descriptor
/* C code */
int fd = _adb_connect("shell:ls");
/* JS code */
let output = readFully(fd);
console.log(output);- Pretty cool abstraction
- In some places unnecessary (should callback to JS directly)
transport layer
- Forwards packets from device over USB to sockets
- Part of backend for the ADB client services
- Due to limitations of Mac and Linux needs threads and synchronous reads and writes over USB
- Used to tracks devices
- Now pure JavaScript
fdevents
- Event loop system based off of file descriptors
- Runs on main server thread
- Backed by
- Linux: epoll
- Mac: select
- Windows: select
- Fun Fact:
- There is a 1000+ line Windows shim for fdevents and exposing an interface to read/write from an
int fdjust like on Unix
platform specific driver code
- Platform-specific
- Long
- Complex
- Would be hard to port to JavaScript
Adb internals, deeper, deeper
Workers Per phone
UI Server Device Loop Input Output
JS:Simulator starts
ADB +
|
+--------->JS:Set up code
+
|
| (really C to JS to UI to new worker to C)
v
C:server_thread+-->C:Start device
+ loop
| (look for devices)
v
C:fdevent_loop
+
| on new device
JS:device tracker<-------++------------------------------> C:Write to device
|
+--------------------------------------------->C:Read
from device