What is a DSL?

I ain't never heard of no DSL

• HTML
• XML
• CSS
• SQL
• Regular Expressions
• Dockerfile
• Bash scripts?
• Make files
• Excel spreadsheet formulas
• PostScript​

Internal DSLs

Written in a host language

• Design Pattern

• Fluent Interfaces (LINQ)

• Language Extensions (sweet.js)

• Language Reductions (JSON)

• Unit test assertions

Why do I need one?

• Can't express in existing language

• Needs to be written by newbs

• You're a special snowflake

• You got a hankerin'

YOU HAD ME AT HTML

Sample Code

test "https://api.github.com/users/davesters" {

  should be object
  size should equal 30

  fields {
    "login" should be string
    id should equal 439674
    name should equal "David Corona"
    site_admin should not equal true
  }
}

1. Lexical Analysis

A fancy name for the thing that converts your code into things that mean something

1. Lexical Analysis

Convert a stream of characters into "tokens"

count should equal 938

Define the Language

test
fields
size
each
object
array
string
number
boolean
true
false
should
not
be
equal

Keywords

Z => S
S => test t { T }
T => fields F { T }
T => each F { T }
T => A should BCE
T => e
A => size | i | t | e
B => not | e
C => be | equal
E => object | array | string        | number | boolean | true | false | t
F => i | t | e

Productions

i = identifier ,   t = literal ,   e = null

The Lexer code

function lexer(input) {
    loop over characters {
        currentChar = input[pointer];
        column = getStateTableColumn(currentChar);
    
        if (column < 0) throw new LexerException('invalid input');
    
        state = getState(currentState, column);
    
        switch (state) {
            case 5: // Found a number
                addToken('num', currentToken, backtrack: true);
            case 7: // Found a string
                addToken('str', currentToken);
            ...
            default:
                currentToken += currentChar;
        }
        pointer++;
    }
    return tokens;
}

2. Syntactic Analysis

• Top-down Parsers
  • LL Parser
  • Recursive-Descent Parser (simplest?)
• Bottom-up Parsers
  • LR parser
  • LALR parser (most common)
• Other fancy ones

Recursive-descent Parser

A set of recursive functions where each function closely models a production of the grammar

S => test t { T }

function S(tokens) {
    let token = tokens.next();
    if (!token.isKeyword('test')) throw Exception('Expected keyword `test`');

    token = tokens.next();
    if (!token.isString()) throw Exception('Expected string');

    token = tokens.next();
    if (!token.isOpeningBrace()) throw Exception('Expected token `{`');

    let node = {
        type: 'test',
        left = { type: 'str', literal: token.literal },
        right = { type: 'block', children: [] }
    };

    loop {
        node.right.children.push( T(tokens) );

        if (token.isClosingBrace( tokens.next() )) break;
        tokens.backup();
    }

    return node;
}

T => A should BCE

function T(tokens) {
    // Code to handle other T productions...

    let node = {
        type: 'should',
        right: { children: [] }
    };

    let token = tokens.next();
    let a = A(token);
    if (a) {
        node.left = a;
        token = tokens.next();
    }

    if (!token.isKeyword('should')) throw Exception('Expected `should`');

    let b = B(token);
    if (b) node.right.children.push(b);

    node.right.children.push( C(tokens) );
    node.right.children.push( E(tokens) );

    return node;
}

C => be | equal

function C(tokens) {
    let token = tokens.next();

    if (token.isKeyword('be')) {
        return {
            type: 'kwd',
            literal: 'be'
        };
    }

    if (token.isKeyword('equal')) {
        return {
            type: 'kwd',
            literal: 'equal'
        };
    }

    throw Exception('Expected keyword `be` or `equal`');
}

Parse Tree

Test

Now we have a complete parse tree or kind of an "AST"

Should

Should

Should

name

be

string

bob

3. Interpreter Code

The completely ugly non-sexy part of the code

• Recursively iterate over parse tree
• May have set of functions similar to parser
• Do something based on the nodes
• We can assume all is good with the tree
• Only ~200 lines of code for mine

DEMO

Now I can make one!

Thanks

Sample code:

https://github.com/davesters/js-dsl-example

Slides:

https://slides.com/davesters/lets-make-a-dsl

Contact:

Twitter - @davesters

Github - davesters

Website - http://www.lovesmesomecode.com