Interactive C++ Compilation (REPL) Done in a Tiny and Embeddable Way

by Viktor Kirilov

my, myself and I

  • game programmer by passion
    • used to work in the games / VFX industries
  • the past 3 years - only open source
    • main focus: productivity, fast iteration
      • fast compile times
      • runtime compilation
    • doctest - the fastest C++ testing framework
    • RCRL - a C++ REPL << today's topic !!!
  • </about  timeout="30_mins_to_save_the_world">

IMHO

  • package management - not discussed here
  • iteration speeds
    • compile times
    • interactive compilation (REPL) << today's topic

Where is C++ universally behind others?

The sad fate of C++ devs

REPL - Read Eval Print Loop

  • interpreted languages have it (JavaScript, Python, etc.)
  • consoles/shells - cmd.exe, bash
  • can iteratively append/execute code (definitions, side effects, etc.)
  • replacing entire functions: using shared libraries OR hot-patching:
    • possible for decades - but not widely used
    • usually quite intrusive (interfaces, constraints, complicated setup)
    • in game engines: Unreal, others...
    • hot-patching (with very little setup): Live++, Recode
    • Visual Studio "Edit & Continue" - 0 setup, but limited
    • http://bit.ly/runtime-compilation-alternatives
  • interactive: REPL-like (append code and mix global/function scopes)
    • cling - by researchers at CERN - built on top of LLVM
    • RCRL - presented today

Runtime compilation for C++: HOW

  • much faster iteration times
    • no need to restart the program - can preserve state
  • less need for a scripting language
    • no need for a virtual machine
    • no binding layer
    • code in one language
  • can hack something quickly
    • introspection, queries
      • debuggers aren't infinitely powerful
    • fine-tuning values
  • interactive (REPL-like): very useful for exploration and teaching

Runtime compilation for C++: WHY

The demo host application API

#ifdef HOST_APPLICATION // export if building the application or import otherwise
#define HOST_API SYMBOL_EXPORT // __declspec(dllexport)
#else
#define HOST_API SYMBOL_IMPORT // __declspec(dllimport)
#endif

class HOST_API Object { // entire class is exported
    float m_x = 0, m_y = 0, m_r = 0.3f, m_g = 0.3f, m_b = 0.3f;
    float m_rot = 0, m_rot_speed = 1.f;

    friend HOST_API Object& addObject(float x, float y);
    Object() = default;

public:
    void translate(float x, float y);
    void colorize(float r, float g, float b);
    void set_speed(float speed);

    void draw();
};

HOST_API std::vector<Object>& getObjects();
HOST_API Object& addObject(float x, float y);

Demo 1: RCRL
Read-Compile-Run-Loop

3 section types - global, vars, once

// global
int foo() { return 42; }

// vars
int a = foo();
auto& b = a;

// once
a++;

// global
#include <iostream>
void print() { std::cout << a << b << std::endl; }

// once
print(); // ======> will result in "4343" being printed

How it works

  • submit code
  • reconstruct a .cpp file from sections in the proper order
    • include all global and vars sections (+ from the past)
    • use once sections only from the current submission
  • compile the .cpp file as a shared library (plugin)
    • link against the host application
    • do not proceed if compilation fails
  • copy the resulting plugin with a new name
  • load the copy (without unloading old ones)
    • initializes globals top-to-bottom
      • executes once statements as part of that step
      • initializes persistent variables from vars sections

How it works

a series of shared objects for each submission

  • quite small
  • fast to build/link
  • cleanup:
    • calls the destructors of variables in vars sections in reverse order
    • unloads the plugins in reverse order
      • calls the destructors of variables in global sections
      • deletes the plugins from the filesystem
  • variables can be easily put in global and once sections too
    • in global - re-initialized on every recompilation + side effects
    • in once - available only within the section
  • compilation is done in a background process so it isn't blocking

How it works

What the .cpp file looks like - global, once

// global
#include <iostream>
// once
std::cout << "hello!";
#include "path/to/rcrl_for_plugin.h"

#include <iostream>

RCRL_ONCE_BEGIN
std::cout << "hello!";
RCRL_ONCE_END
#include "path/to/rcrl_for_plugin.h"

#include <iostream>

int rcrl_anon_12 = []() {
std::cout << "hello!";
return 0; }();

submitted code

.cpp file

expanded macros

What the .cpp file looks like - vars, once

// vars
int a = 5;
// once
a++;
#include "path/to/rcrl_for_plugin.h"

int& a = *[]() { // reference to an int called "a"
    auto& address = rcrl_get_persistence("a");
    if(address == nullptr) {
        address = (void*)new int(5);
        rcrl_add_deleter(address, [](void* ptr)
            { delete static_cast<int*>(ptr); });
    }
    return static_cast<int*>(address);
}(); // immediately call the lambda

int rcrl_anon_12 = []() {
a++;
return 0; }();

submitted code

expanded macros

What the .cpp file looks like - auto in vars

// vars
auto a = 5;
#include "path/to/rcrl_for_plugin.h"

auto type_of_a = []() -> auto { // use with decltype() to deduce the type
    auto temp = (5);
    return temp;
};
decltype(type_of_a())& a = *[]() {
    auto& address = rcrl_get_persistence("a");
    if(address == nullptr) {
        address = (void*)new decltype(type_of_a())(5);
        rcrl_add_deleter(address, [](void* ptr) 
            { delete static_cast<decltype(type_of_a())*>(ptr); });
    }
    return static_cast<decltype(type_of_a())*>(address);
}();

submitted code

expanded macros

Linking to the host application

  • for interacting with the host application through the exported API
  • the .cpp file always includes a header called "rcrl_for_plugin.h"
    • with helper macros
    • with symbols exported from the host app for persistence
RCRL_SYMBOL_IMPORT void*& rcrl_get_persistence(const char* var_name);
RCRL_SYMBOL_IMPORT void   rcrl_add_deleter(void* address, void (*deleter)(void*));
set_target_properties(my_executable PROPERTIES ENABLE_EXPORTS ON)
  • set the ENABLE_EXPORTS CMake property to true on executables
  • no symbols are exported by default on Windows
    • unlike Unix (with no "-fvisibility=hidden")
    • WINDOWS_EXPORT_ALL_SYMBOLS target property (CMake)

Explicit symbol exports

#if defined _WIN32 || defined __CYGWIN__
#define SYMBOL_EXPORT __declspec(dllexport)
#define SYMBOL_IMPORT __declspec(dllimport)
#else
#define SYMBOL_EXPORT __attribute__((visibility("default")))
#define SYMBOL_IMPORT
#endif

#ifdef DLL_EXPORTS // if this is defined - the API is exported
#define MY_API SYMBOL_EXPORT
#else
#define MY_API SYMBOL_IMPORT
#endif

MY_API void bar(); // annotated free function

class MY_API MyClass {
    // everything here is exported
};

Explicit symbol exports

The health benefits:

  • improves link times
  • circumvents the GOT (Global Offset Table) for calls ==> profit
  • load times also improved (extreme templates case: 45 times)
  • reduces the size of your DSO by 5-20%
  • more info at: https://gcc.gnu.org/wiki/Visibility

Cons:

too much work if only for the RCRL technique - try to export all

Parser - vars section

  • parses the type, name and initializer for variable definitions
  • supports complex constructs​
    • templates, decltype, complex initializers
    • references (allocates a pointer to T instead of the type T)
    • auto (lambda + "decltype()" to deduce the type)
    •  
  • only variable definitions are allowed
  • tiny - less than 400 lines of code
    • saw Parsing C++ @CppCon 2018 by the CLion folks yesterday...
      • extremely illuminating and depressing at the same time :D
    • only 400 lines, but ugly, hacky and probably broken...
    • ... but only 400 lines! :D
std::map<decltype(i_return_int()), std::vector<std::string>> m = {{5, {}}, {6, {}}};

Interesting restrictions

  • "decltype()" of names from vars sections => reference to the type
  • don’t rely on the address of functions - change after recompilation
    • loading the new plugin in a new address space...
  • don't use the static keyword (either in function or global scope)
    • loading the new plugin in a new address space...
  • const refs: no lifetime extension for temporaries (get compile errors)
  • avoid the use of the preprocessor...
  • global variables will be re-initialized on each recompilation without destroying the old versions - and side effects will accumulate
    • that is what the vars sections are for
  • these are a minor price to pay for the benefits IMHO
  • read the blog post for the demo project release for the rest

RCRL API

// rcrl.h

enum Mode {
    GLOBAL,
    VARS,
    ONCE
};

std::string cleanup_plugins(bool redirect_stdout = false);

bool submit_code(std::string code, Mode default_mode = ONCE, ...);

std::string get_new_compiler_output();

bool is_compiling();

bool try_get_exit_status_from_compile(int& exitcode);

std::string copy_and_load_new_plugin(bool redirect_stdout = false);

Sample loop

while(true) { // main loop of program
    if(submit() && is_compiling() == false) { // submit code !!!
        editor.lock();              // no editing while compiling
        submit_code(editor.code()); // submit for compilation
        compiler_output.clear();    // clear old output
    }

    compiler_output += get_new_compiler_output(); // anything new?

    int status;
    if(try_get_exit_status_from_compile(status)) { // check compilation
        if(status == 0) { // on success
            history += editor.code();   // append to history
            editor.code().clear();      // clear code
            copy_and_load_new_plugin(); // load/execute!
        }
        editor.unlock(); // unlock editor regardless of success
    }
}

The repository

  • RCRL core - requires C++11 (for auto variables - C++14)
    • 5 source files - in /src/rcrl/
      • rcrl.h (66 loc) - the main API - consists of 6 functions
      • rcrl.cpp (274 loc) - the engine - hack here! :D
      • rcrl_for_plugin.h (55 loc) - for the plugin (macros, exports)
      • rcrl_parser.h (30 loc)
      • rcrl_parser.cpp (368 loc) - parses sections and variables
    • dependency on /src/third_party/tiny-process-library - for processes
  • the rest of the demo - GUI, exported API, third party libraries

How to integrate properly

  • the RCRL core (rcrl.cpp) is expected to be modified for custom needs
    • not trying to be a one-size-fits-all solution
    • the demo uses CMake to work on all platforms with any compiler
      • no need for CMake - can call the build system directly
      • ninja is exceptionally fast compared to make/msbuild
      • can even call the compiler directly
    • requires knowledge of build systems, compilers, linking, etc.
  • try to avoid linking against big static libraries
  • use a precompiled header for the common includes for the REPL
  • disabling optimizations will shorten the build time
  • the editor may be separate from the application

    • code can be sent through sockets or some other way

    • easier support for auto-completion and syntax highlighting

Room for improvement of the engine

  • global and vars sections can be merged - need a better parser
    • could we even get rid of once sections??? (LibClang ?)
  • auto complete, syntax highlighting - probably with LibClang
  • crash handling when loading the plugins after compilation
  • compiler error messages
    • mapping between lines of the submitted code and in the .cpp
  • debugging
    • ability to set breakpoints and step through code?

Demo 2 - info

  • makes use of https://github.com/iboB/dynamix
    • "A new take on polymorphism in C++"
      • compose and modify polymorphic objects at run time
    • complete separation between interfaces and implementation
      • most functionality is in modules (plugins: .dll-s)
  • reflection - tool built on top of LibClang
    • all classes have json serialization automatically generated

Demo 2 - REPL info

  • lightning fast
    • uses a precompiled header for most of the engine interface
    • using ninja as the build system
    • links just to a few static libraries
  • almost anything is accessible through exported module interfaces
    • most dependencies are built as a dynamic library (DLL/SO)
    • whatever cannot be in a plugin is exported from the executable

Demo 2 - precompiled header for RCRL

// standard includes
#include <cmath>
#include <csignal>
#include <cstdlib>
#include <cstring>
#include <cstddef>
#include <cctype>
#include <typeinfo>
#include <functional>
#include <exception>
#include <stdexcept>
#include <iterator>
#include <limits>
#include <numeric>
#include <string>
#include <utility>
#include <memory>
#include <tuple>
#include <new>
#include <random>
#include <chrono>
#include <type_traits>
#include <vector>
#include <array>
#include <list>
#include <map>
// ...continues...
#include <set>
#include <unordered_map>
#include <unordered_set>
#include <optional>
#include <variant>
// third party
#include <dynamix/dynamix.hpp>
#include <ppk_assert.h>
#include <sajson/include/sajson.h>
#include <GL/glew.h>
#include <yama/yama.hpp>
// project specific
#include "utils/suppress_warnings.h"
#include "utils/visibility.h"
#include "utils/preprocessor.h"
#include "utils/doctest/doctest_proxy.h"
#include "utils/types.h"
#include "utils/JsonData.h"
#include "utils/singleton.h"
#include "utils/transform.h"
#include "core/messages/message_macros.h"
#include "core/tags.h"
#include "core/Object.h"
#include "core/messages/messages_common.h"
#include "core/registry/registry.h"

Demo 2 - link dependencies

The camera object can be accessed through the REPL by getting it through the executable interface and then interacting with it through the exported interfaces which it implements

Demo 2: "GAME"

Demo 2 - notes

  • https://github.com/iboB/dynamix
  • all plugins (editor, camera, mesh) are hot-swappable at run time
    • can even change the memory layout of classes... at run time!!!
      • synergy: reflection + interface/implementation separation = ❤
    • just bragging - not showcased today :D

Key takeaways

  • having a REPL is extremely powerful
    • for exploring, hacking, teaching
  • APIs should be dynamically exported for interaction with the host app
  • this technique can handle almost any C++ code/construct
  • demo project:
    • not a one-size-fits-all solution - hack it to suit your workflow
    • works on any platform/compiler
    • super tiny and easy to integrate - just a few source files (<800 LOC)
    • full of comments in the C++ and CMake code
  • better parser needed
    • hopefully can get rid of the manual section changes in comments
    • current parser for vars sections is a horrible joke
  • auto complete and syntax highlighting are CRUCIAL

Random thoughts

  • this technique can be used for other compiled languages too
    • I might work on something like this for Nim
  • applications can have an optional module which enables live hacking
    • a module implementing this technique (the RCRL engine)
    • a C++ compiler (the same version used for the application)
    • application API headers with dll exported symbols
    • application export lib so it can be linked to (Windows only)

Q&A

2018 - Interactive C++ Compilation (REPL): The Lean Way

By Viktor Kirilov

2018 - Interactive C++ Compilation (REPL): The Lean Way

How to Make a Tiny, Embeddable and Powerful Interactive C++ Compiler (REPL)

  • 3,387