Coroutines in C/C++

by Sergey Lyubka, Cesanta

@CesantaHQ

Typical parser

// Producer function
static int consume(void) {
  return getchar();
}

// Consumer function
static void tokenize(void) {
  int c;
  while ((c = consume()) != EOF) {
    if (isdigit(c)) {
      do {
        add_to_token(c);
        c = consume();
      } while (isdigit(c));
      got_token(TOKEN_NUMBER);
    }
    if (!isspace(c)) {
      add_to_token(c);
      got_token(TOKEN_OPERATION);
    }
  }
}

• Tokenizer calls input reader
• Input reader returns one character at a time
• Parser can call tokenizer - then, tokenizer should return one token at a time

Complicating structure

• What if input source is not a simple stdin
• But e.g. a stream from a compressor library
• Or from a network socket?

// Before
static int consume(void) {
  return getchar();
}

// After
static int consume(void) {
  size_t i, n;
  char buf[10];
  while ((n = fread(buf, 1, sizeof(buf), stdin)) > 0) {
    for (i = 0; i < n; i++) {
      MAGIC_RETURN buf[i];  // <--- Simple return won't work
    }
  }
  MAGIC_RETURN EOF;  // <--- Simple return won't work
}

Coroutine: function with state

// To save state, all local variables are declared static
static int consume(void) {
  static size_t i, n;
  static char buf[10];
  static int state;
  switch (state) {
    case 0:
    while ((n = fread(buf, 1, sizeof(buf), stdin)) > 0) {
      for (i = 0; i < n; i++) {
        state = 1;
        return buf[i];
        case 1:;
      }
    }
  }
  return EOF;
}

Few helpful macros

// Coroutine macros
#define CR_BEGIN static int state = 0; switch (state) { case 0:
#define YIELD(x) do { state = __LINE__; return x; case __LINE__:; } while (0)
#define CR_END }

static int consume(void) {
  static size_t i, n;
  static char buf[10];
  CR_BEGIN;
  while ((n = fread(buf, 1, sizeof(buf), stdin)) > 0) {
    for (i = 0; i < n; i++) {
      YIELD(buf[i]);
    }
  }
  YIELD(EOF);
  CR_END;
}

No code modification?

• The "switch" solution requires to:
  • Declare variables that should persist state as static
  • Wrap function into CR_BEGIN / CR_END macros
• Is it possible to keep function code intact?
• Problem:
  • C/C++ uses CPU stack for the function calls
  • The hack above makes state saving possible by declaring variables static
  • If variables are not static, coroutine should execute in it's own stack
  • There is no standard C function that provides new context creation
  • Is it possible to create coroutines from standard C function in standard C?

No code modification?

• The answer is: yes!
• To be continued in the next talk!
• Reference:
  • Simon Tatham on Duff's device
  • Slides: https://slides.com/sergeylyubka/coroutines

Thank You!

contact me at

sergey.lyubka@cesanta.com