Feedback in V8
Feedback in V8
Michael Stanton
- Google V8: Compiler Team/Manager
- V8: ICs and Feedback Vectors
- Cats, climbing and old typewriters :p
Feedback
The transmission of evaluative or corrective information about an action, event, or process to the original or controlling source; also : the information so transmitted
source: Merriam-Webster
Google proprietary
What people think we do
Google proprietary
What people think we do
What we actually do
Google proprietary
How does v8 achieve Performance?
- Compilation pipeline with learning
- HiddenClass-based object layout
- Inline Caches to maintain and observe layout
Google proprietary
Compilation pipeline with learning
Source
Code
Ignition
Byte
Code
Run for a while
Gather feedback
IC Slot | Type | Value |
---|---|---|
1 | LOAD | MONO |
2 | CALL | UNINIT |
... | ... | ... |
Feedback Vector
Google proprietary
Compilation pipeline with learning
Source
Code
Ignition
Turbofan
Byte
Code
Optimized
Code
Slot | Type | Value |
---|---|---|
1 | LOAD | MONO |
2 | CALL | UNINIT |
... | ... | ... |
Feedback Vector
Google proprietary
Compilation pipeline with learning
Source
Code
Ignition
Turbofan
Byte
Code
Optimized
Code
Deoptimization
Slot | Type | Value |
---|---|---|
1 | LOAD | MONO |
2 | CALL | UNINIT |
... | ... | ... |
Google proprietary
How does v8 achieve Performance?
- Compilation pipeline with learning
- Hidden-Class-based object layout
- Inline Caches to maintain and observe layout
Google proprietary
- Objects have a "hidden class" (called a Map in V8)
- The Map is the first pointer in every object
- The Map describes the layout in memory
- Adding/removing properties changes the Map
Hidden-Class-based Object layout
Google proprietary
Map M
x: 1
let o = { x: 1 };
o.y = 2;
Evolution of o.map
Hidden-Class-based Object layout
Google proprietary
Map M
x: 1
Map N
x: 1
y: 2
let o = { x: 1 };
o.y = 2;
Evolution of o.map
Hidden-Class-based Object layout
Google proprietary
Map M
x: 1
Map N
x: 1
y: 2
let o = { x: 1 };
o.y = 2;
Evolution of o.map
Umm...Okay...
Google proprietary
How does v8 achieve Performance?
- Compilation pipeline with learning
- Hidden-Class-based object layout
- Inline Caches to maintain and observe layout
Google proprietary
Inline Caches to maintain and observe layout
- An Inline Cache (IC) is a listening site placed in your code.
- We have them at LOAD, STORE and CALL locations.
- It caches the Map of objects that pass by in the Feedback Vector for the function.
IC Slot | IC Type | Value |
---|---|---|
1 | LOAD | MONO |
2 | CALL | UNINIT |
... | ... | ... |
Google proprietary
Every function has a Feedback Vector. It's just an array that holds state for each IC.
function processLogFile(fileName) {
this.collectEntries = true;
this.lastLogFileName_ = fileName;
var line;
while (line = readline()) {
this.processLogLine(line);
}
print();
print("Load: " + this.LoadIC);
print("Store: " + this.StoreIC);
...
}
IC Slot | IC Type | State |
---|---|---|
... | ... | ... |
10 | LOAD | MONO(M) |
11 | LOAD | UNINITIALIZED |
Inline Caches to maintain and observe layout
Google proprietary
An Inline Cache is also a state machine.
Inline Caches to maintain and observe layout
Uninitialized
Premonomorphic
Monomorphic
Polymorphic
Megamorphic
Slow
Google proprietary
mrale.ph/blog
function load(a) { return a.key; }
// pseudo-code for the LOAD IC:
if (vector[slot].state == MONO) {
if (a.map == vector[slot].map) {
return valueAtOffset(a, vector[slot].offset);
} else {
// Call into the Runtime for instructions.
// Update IC state.
}
} else {
..
}
Slot | Type | State | Map | Offset |
---|---|---|---|---|
0 | LOAD_IC | MONO | m | 0x8 |
Feedback Vector:
Google proprietary
Some facts
- We can say an IC is monomorphic when it only saw objects of one Map.
- It never makes sense to talk about an object as monomorphic.
- The term only applies to load of a particular property at a particular site.
Google proprietary
How v8 achieves Performance
- Compilation pipeline with learning
- Hidden-Class-based object layout
- ICs to maintain and observe layout
Google proprietary
Feedback Workflow
Let's look at a simple property load.
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Google proprietary
Parsing
Here is the AST from parsing:
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key
Google proprietary
Parsing
Here is the AST from parsing:
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key
Here is the feedback vector specification:
Slot | Type |
---|---|
0 | LOAD_IC |
Google proprietary
Compiling bytecode
Here is the bytecode:
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Here is the feedback vector specification:
// Run with --print-bytecode
"load" -- Parameter count 2
StackCheck
Nop
LdaNamedProperty a0, [0], [3] // "key"
Return
Slot | Type |
---|---|
0 | LOAD_IC |
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Here is the feedback vector:
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | UNINIT |
Executing...
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
We execute the LoadIC for "a.key"...
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | UNINIT |
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Return the answer & remember the Map.
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Now with a different object...
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
Is the map the same?
o1.map == o.map?
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
We remain monomorphic.
Yes
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
Google proprietary
Profiling
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
The Runtime Profiler asks:
- Is the function "hot?"
- Is there enough feedback?
// --trace-opt output
[marking <JSFunction load> for
optimized recompilation,
reason: small function,
ICs with typeinfo: 1/1 (100%),
generic ICs: 0/1 (0%)]
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
(Run with --nouse-osr to ensure we optimize load(), and not the whole script)
// load(a) - Turbofanned
push ebp // Build frame
mov ebp,esp //
push esi //
push edi //
mov eax,[ebp+0x8] // eax = a
test al,0x1 // is a an object?
jz DEOPT_0 // If not, deoptimize
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt.
mov eax,[eax+0xb] // return a.key!
mov esp,ebp // Tear down frame
pop ebp //
ret 0x8 // return for realz
...
DEOPT_0: call 0x52d06000 // Goodbye...
DEOPT_1: call 0x52d0600a // Also goodbye
// load(a) - Turbofanned
push ebp // Build frame
mov ebp,esp //
push esi //
push edi //
mov eax,[ebp+0x8] // eax = a
test al,0x1 // is a an object?
jz DEOPT_0 // If not, deoptimize
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt.
mov eax,[eax+0xb] // return a.key!
mov esp,ebp // Tear down frame
pop ebp //
ret 0x8 // return for realz
...
DEOPT_0: call 0x52d06000 // Goodbye...
DEOPT_1: call 0x52d0600a // Also goodbye
Boilerplate
// load(a) - Turbofanned
push ebp // Build frame
mov ebp,esp //
push esi //
push edi //
mov eax,[ebp+0x8] // eax = a
test al,0x1 // is a an object?
jz DEOPT_0 // If not, deoptimize
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt.
mov eax,[eax+0xb] // return a.key!
mov esp,ebp // Tear down frame
pop ebp //
ret 0x8 // return for realz
...
DEOPT_0: call 0x52d06000 // Goodbye...
DEOPT_1: call 0x52d0600a // Also goodbye
Boilerplate
Object check
// load(a) - Turbofanned
push ebp // Build frame
mov ebp,esp //
push esi //
push edi //
mov eax,[ebp+0x8] // eax = a
test al,0x1 // is a an object?
jz DEOPT_0 // If not, deoptimize
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt.
mov eax,[eax+0xb] // return a.key!
mov esp,ebp // Tear down frame
pop ebp //
ret 0x8 // return for realz
...
DEOPT_0: call 0x52d06000 // Goodbye...
DEOPT_1: call 0x52d0600a // Also goodbye
Boilerplate
Object check
Map check
// load(a) - Turbofanned
push ebp // Build frame
mov ebp,esp //
push esi //
push edi //
mov eax,[ebp+0x8] // eax = a
test al,0x1 // is a an object?
jz DEOPT_0 // If not, deoptimize
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt.
mov eax,[eax+0xb] // return a.key!
mov esp,ebp // Tear down frame
pop ebp //
ret 0x8 // return for realz
...
DEOPT_0: call 0x52d06000 // Goodbye...
DEOPT_1: call 0x52d0600a // Also goodbye
Boilerplate
Object check
Map check
The actual load!
Monomorphism: The Golden Idol
- It's tempting to see every problem as "insufficient monomorphism."
- This becomes counter-productive
- Advice to duplicate code :(
- Language expressiveness suffers
Google proprietary
Monomorphism: The Golden Idol
- It's a good thing, but there are many good things.
- Spend more time improving baseline performance
- A sports car isn't appropriate for everything
"Bike Social" by Marc van Woudenberg (license)
Google proprietary
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
The more references there are to "a" downwind of the feedback, the greater the benefit of knowing the object's class.
Downwind includes inlined functions.
Google proprietary
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
- Polymorphic: use it...but what if some of the maps are stale and never used recently?
// TurboFan emits something like this:
if (a.map == 0x43501231) { // Maps embedded in code.
return loadAtOffset(a, 0xc); // No Feedback Vector.
} else if (a.map == 0x99503211) {
return loadAtOffset(a, 0x10);
} else {
DEOPTIMIZE(); // Oh noes!
}
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
- Polymorphic: probably use it
- What about uninitialized IC sites?
Google proprietary
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
- Polymorphic: probably use it
- What about uninitialized IC sites? deoptimize
- Be careful about "hoisting" map checks out of loops!
Google proprietary
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
- Polymorphic: probably use it
- What about uninitialized IC sites? deoptimize
- Be careful about "hoisting" map checks out of loops!
You lose touch with the Feedback Vector, and have nowhere to put the new map after deoptimization.
Eventually V8 stops reoptimizing.
Google proprietary
Things to consider when optimizing
- How to use the feedback?
- Monomorphic is a no-brainer: use it
- Polymorphic: probably use it
- What about uninitialized IC sites? deoptimize
- Be careful about "hoisting" map checks out of loops!
-
How much to inline?
- Increases compilation time
- Which calls to inline, and to what depth?
Google proprietary
Optimized execution
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
// load(a) - Turbofanned
...
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt
mov eax,[eax+0xb] // return a.key
...
Feedback Vector not being used
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
load({ key: "usb", name: "francis" });
Google proprietary
Deopt path is taken
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
load({ key: "usb", name: "francis" });
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | MONO(o.map) |
// load(a) - Turbofanned
...
mov ecx,[eax-0x1] // ecx = a.map
mov edx,0x37e0b4ad // edx = o.map
cmp ecx, edx
jnz DEOPT_1 // if not same, deopt
mov eax,[eax+0xb] // return a.key
...
Google proprietary
Interpreting
function load(a) {
return a.key;
}
var o = { key: "usb" };
var o1 = { key: "port" };
for (var i = 0; i < 100000; i++) {
load(o);
load(o1);
}
load({ key: "usb", name: "francis" });
Slot | Type | Value |
---|---|---|
0 | LOAD_IC | POLY(o.map, new_map) |
Feedback is updated
Google proprietary
Conclusion
We are watching...
Conclusion
We are watching...
with love!
IC Slot | IC Type | Value |
---|---|---|
1 | LOAD | MONO |
2 | CALL | UNINIT |
... | ... | ... |
Questions?
Google proprietary
Shameless plug
Vector
By ripsawridge
Vector
How the V8 JavaScript engine uses feedback to achieve performance.
- 4,762