Bytecode manipulations

AR, November 2017

Who am I?

• Siberian
• Web developer with 10+ years of experience
• Python enthusiast (6+ years)
• Lead Full Stack programmer at

Aleksey Rembish

Who am I?

• Siberian
• Web developer with 10+ years of experience
• Python enthusiast (6+ years)
• Lead Full Stack programmer at

Aleksey Rembish

And so we code...

• a LPG operator?

And so we code...

• a LPG operator?

less-pipe-greater operator

And so we code...

• a LPG operator

less-pipe-greater operator

needs grammar patches

And so we code...

• a LPG operator
• increments and decrements?

And so we code...

• a LPG operator
• increments and decrements?

x = 1
y = x++ + ++x
print(y)

And so we code...

• a LPG operator
• increments and decrements?

In [1]: def increment(value):
   ...:     return ++value
   ...: 

In [2]: from dis import dis

In [3]: dis(increment)
  2           0 LOAD_FAST                0 (value)
              3 UNARY_POSITIVE      
              4 UNARY_POSITIVE      
              5 RETURN_VALUE        

And so we code...

• a LPG operator
• increments and decrements

And so we code...

• a LPG operator
• increments and decrements
• GOTO and labels!!!

WORK

Code formatting

def method():
    label: print("foo")
    goto label

Code formatting

def method():
    label: print("foo")
    goto label

def method():
    label .here
    print("foo")
    goto .here

Code formatting

def method():
    label .here
    goto .here

  2           0 LOAD_GLOBAL              0 (label)
              3 LOAD_ATTR                1 (here)
              6 POP_TOP

  3           7 LOAD_GLOBAL              2 (goto)
             10 LOAD_ATTR                1 (here)
             13 POP_TOP

             14 LOAD_CONST               0 (None)
             17 RETURN_VALUE

Function bytecode

# Python 2.x
def function():
    pass

code = function.func_code

# Python 3.x
def function():
    pass

code = function.__code__

Pure Bytecode

code.co_code
== 't\x00\x00j ... \x00S'   # Py2
== b't\x00\x00j ... \x00S'  # Py3

from opcode import opname, opmap

print(code.co_code[0])  # = 'LOAD_GLOBAL'
print(opmap['LOAD_GLOBAL'])  # = 116
opmap['LOAD_GLOBAL'] == code.co_code[0]  # = True 

Opcode arguments

from opcode import HAVE_ARGUMENT  # == 90

opcode = code.co_code[i]

if opcode >= HAVE_ARGUMENT:
    lo_byte = code.co_code[i + 1]
    hi_byte = code.co_code[i + 2]
    position = (hi_byte << 8) ^ lo_byte

Cherchez la femme

opcode = code.co_code[i]
current = opname[opcode]
command = code.co_names[previous_arg_position]

if current == 'LOAD_ATTR' \
        and previous == 'LOAD_GLOBAL':

    if command == 'label':
        # store label position
    elif command == 'goto':
        # same for goto

code.co_names
# == ('label', 'here', 'goto')

Nope, nope, NOP!

Label:

• does nothing
• stores position
• old code = 3 + 3 + 1 instructions
• new code = 7 NOPs

Goto:

• jumps to label
• old code = 3 + 3 + 1 instructions
• new code = JUMP_ABSOLUTE (3 bytes) + 4 NOPs

and JUMP!

Code rebuild

Function rebuild

Implementation

github.com/rembish/goto

slides.com/rembish/bytecode

Nice to read

• entrian.com/goto
• github.com/snoack/python-goto
• code.activestate.com/recipes/576944-the-goto-decorator
• github.com/cdjc/goto

We are hiring!

Python 3.5, async, Tornado, Flask, Docker, Kubernetes, Prometheus, GitLab CI, ...

https://kariera.seznam.cz/kariera/clanek/1248604750-python-programator-ka-sklik-cz-1248604750

Bytecode manipulations