Feedback in V8

Feedback in V8

Michael Stanton

• Google V8: Compiler Team/Manager
• V8: ICs and Feedback Vectors
• Cats, climbing and old typewriters :p

ripsawridge@

Feedback

The transmission of evaluative or corrective information about an action, event, or process to the original or controlling source; also :  the information so transmitted

source: Merriam-Webster

Google proprietary

What people think we do

Google proprietary

What people think we do

What we actually do

Google proprietary

How does v8 achieve Performance?

• Compilation pipeline with learning
• HiddenClass-based object layout
• Inline Caches to maintain and observe layout

Google proprietary

Compilation pipeline with learning

Run for a while

Gather feedback

Feedback Vector

Google proprietary

Compilation pipeline with learning

Feedback Vector

Google proprietary

Compilation pipeline with learning

Deoptimization

Google proprietary

How does v8 achieve Performance?

• Compilation pipeline with learning
• Hidden-Class-based object layout
• Inline Caches to maintain and observe layout

Google proprietary

• Objects have a "hidden class" (called a Map in V8)
• The Map is the first pointer in every object
• The Map describes the layout in memory
• Adding/removing properties changes the Map

Hidden-Class-based Object layout

Google proprietary

Map M

x: 1

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Hidden-Class-based Object layout

Google proprietary

Map M

x: 1

Map N

x: 1

y: 2

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Hidden-Class-based Object layout

Google proprietary

Map M

x: 1

Map N

x: 1

y: 2

let o = { x: 1 };
o.y = 2;

Evolution of o.map

Umm...Okay...

Google proprietary

How does v8 achieve Performance?

• Compilation pipeline with learning
• Hidden-Class-based object layout
• Inline Caches to maintain and observe layout

Google proprietary

Inline Caches to maintain and observe layout

• An Inline Cache (IC) is a listening site placed in your code.
• We have them at LOAD, STORE and CALL locations.
• It caches the Map of objects that pass by in the Feedback Vector for the function.

Google proprietary

Every function has a Feedback Vector. It's just an array that holds state for each IC.

function processLogFile(fileName) {
  this.collectEntries = true;
  this.lastLogFileName_ = fileName;
  var line;
  while (line = readline()) {
    this.processLogLine(line);
  }
  print();
  print("Load: " + this.LoadIC);
  print("Store: " + this.StoreIC);
  ...
}

Inline Caches to maintain and observe layout

Google proprietary

An Inline Cache is also a state machine.

Inline Caches to maintain and observe layout

Uninitialized

Premonomorphic

Monomorphic

Polymorphic

Megamorphic

Slow

Google proprietary

"Whats up with Monomorphism?"

mrale.ph/blog

function load(a) { return a.key; }

// pseudo-code for the LOAD IC:
if (vector[slot].state == MONO) {
  if (a.map == vector[slot].map) {
    return valueAtOffset(a, vector[slot].offset);
  } else {
    // Call into the Runtime for instructions.
    // Update IC state.
  }
} else {
  ..
}

Feedback Vector:

Google proprietary

Some facts

• We can say an IC is monomorphic when it only saw objects of one Map.
• It never makes sense to talk about an object as monomorphic.
• The term only applies to load of a particular property at a particular site.

Google proprietary

How v8 achieves Performance

• Compilation pipeline with learning
• Hidden-Class-based object layout
• ICs to maintain and observe layout

Google proprietary

Feedback Workflow

Let's look at a simple property load.

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Google proprietary

Parsing

Here is the AST from parsing:

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key

Google proprietary

Parsing

Here is the AST from parsing:

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

// Run with --print-ast
FUNC
. NAME "load"
. PARAMS
. . VAR "a"
. RETURN
. . PROPERTY Slot(0) at 29
. . . VAR PROXY parameter[0] "a"
. . . NAME key

Here is the feedback vector specification:

Google proprietary

Compiling bytecode

Here is the bytecode:

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Here is the feedback vector specification:

// Run with --print-bytecode
"load" -- Parameter count 2

   StackCheck
   Nop
   LdaNamedProperty a0, [0], [3]  // "key"
   Return

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Here is the feedback vector:

Executing...

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

We execute the LoadIC for "a.key"...

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Return the answer & remember the Map.

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Now with a different object...

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

Is the map the same?

o1.map == o.map?

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

We remain monomorphic.

Yes

Google proprietary

Profiling

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

The Runtime Profiler asks:

• Is the function "hot?"
• Is there enough feedback?

// --trace-opt output
[marking <JSFunction load> for 
    optimized recompilation, 
  reason: small function, 
  ICs with typeinfo: 1/1 (100%), 
  generic ICs: 0/1 (0%)]

(Run with --nouse-osr to ensure we optimize load(), and not the whole script)

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

Boilerplate

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

Boilerplate

Object check

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

Boilerplate

Object check

Map check

// load(a) - Turbofanned

push ebp             // Build frame             
mov  ebp,esp         //
push esi             //
push edi             //

mov  eax,[ebp+0x8]   // eax = a
test al,0x1          // is a an object?
jz   DEOPT_0         // If not, deoptimize

mov  ecx,[eax-0x1]   // ecx = a.map
mov  edx,0x37e0b4ad  // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1         // if not same, deopt.
mov  eax,[eax+0xb]   // return a.key!

mov  esp,ebp         // Tear down frame
pop  ebp             //
ret  0x8             // return for realz

...

DEOPT_0: call 0x52d06000  // Goodbye...
DEOPT_1: call 0x52d0600a  // Also goodbye

Boilerplate

Object check

Map check

The actual load!

Monomorphism: The Golden Idol

• It's tempting to see every problem as "insufficient monomorphism."
• This becomes counter-productive
  • Advice to duplicate code :(
  • Language expressiveness suffers

Google proprietary

Monomorphism: The Golden Idol

• It's a good thing, but there are many good things.
• Spend more time improving baseline performance
• A sports car isn't appropriate for everything

"Bike Social" by Marc van Woudenberg (license)

Google proprietary

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it

The more references there are to "a" downwind of the feedback, the greater the benefit of knowing the object's class.

Downwind includes inlined functions.

Google proprietary

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: use it...but what if some of the maps are stale and never used recently?

// TurboFan emits something like this:

if (a.map == 0x43501231) {         // Maps embedded in code.
  return loadAtOffset(a, 0xc);     // No Feedback Vector.
} else if (a.map == 0x99503211) {
  return loadAtOffset(a, 0x10);
} else {
  DEOPTIMIZE();                    // Oh noes!
}

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites?

Google proprietary

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!

Google proprietary

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!

You lose touch with the Feedback Vector, and have nowhere to put the new map after deoptimization.

Eventually V8 stops reoptimizing.

Google proprietary

Things to consider when optimizing

• How to use the feedback?
  • Monomorphic is a no-brainer: use it
  • Polymorphic: probably use it
  • What about uninitialized IC sites? deoptimize
  • Be careful about "hoisting" map checks out of loops!
• How much to inline?
  • Increases compilation time
  • Which calls to inline, and to what depth?

Google proprietary

Optimized execution

// load(a) - Turbofanned
...
mov  ecx,[eax-0x1]  // ecx = a.map
mov  edx,0x37e0b4ad // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1        // if not same, deopt
mov  eax,[eax+0xb]  // return a.key
...

Feedback Vector not being used

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

Google proprietary

Deopt path is taken

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

// load(a) - Turbofanned
...
mov  ecx,[eax-0x1]  // ecx = a.map
mov  edx,0x37e0b4ad // edx = o.map
cmp  ecx, edx
jnz  DEOPT_1        // if not same, deopt
mov  eax,[eax+0xb]  // return a.key
...

Google proprietary

Interpreting

function load(a) {
  return a.key;
}

var o = { key: "usb" };
var o1 = { key: "port" };

for (var i = 0; i < 100000; i++) {
  load(o);
  load(o1);
}

load({ key: "usb", name: "francis" });

Feedback is updated

Google proprietary

Conclusion

We are watching...

Conclusion

We are watching...

with love!

Questions?

Google proprietary

Shameless plug

Bmeurer@'s talk on es.next performance