A gentle introduction to the JavaScript AST

or

How I learned to stop worrying and build my own tools and transforms

://about#me

• Björn Brauer
• Hamburg
• Freelance (currently having fun at Costa Digital)
• JavaScript magician
• @ZauberNerd
• <3 programming, table foosball, close-up magic, beer

Quick overview

Abstract(*) Syntax Tree

• Tree-like data structure
• represents the syntactic structure of source code
• generated by a parser (+ lexer)
• used for compilation, transpilation, etc
• (*) doesn't contain stylistic elements

/*spaces*/            hh.js(('2016-06-15')); // comments

How is it generated?

v a r   a n s w e r   =   4 2 ;

[
    { type: "Keyword", value: "var" },
    { type: "Identifier", value: "answer" },
    { type: "Punctuator", value: "=" },
    { type: "Numeric", value: 42 },
    { type: "Punctuator", value: ";" },
]

{
  "type": "Program",
  "body": [{
    "type": "VariableDeclaration",
    "kind": "var",
    "declarations": [{
      "type": "VariableDeclarator",
      "id": {
        "type": "Identifier",
        "name": "answer"
      },
      "init": {
        "type": "NumericLiteral",
        "value": 42
      }
    }]
  }]
}

Tokens

AST

Lexer / Scanner

Parser

How is it generated?

• Grammar definitions
• Parser generators

How is it generated?

Grammar definition

%lex

DecimalDigit [0-9]
DecimalDigits [0-9]+
NonZeroDigit [1-9]
HexDigit [0-9a-fA-F]
UnicodeIdentifierStart [\xaa\xb5\xba\xc0-...]
UnicodeIdentifierPart [\xaa\xb5\xba\xc0-...]
IdentifierStart {UnicodeIdentifierStart}|[$_a-zA-Z]|("\\"[u]{HexDigit}{4})
IdentifierPart {IdentifierStart}|{UnicodeIdentifierPart}|[0-9]
Identifier {IdentifierStart}{IdentifierPart}*
SignedInteger [+-]?[0-9]+
LineContinuation \\(\r\n|\r|\n)
NonEscapeCharacter [^\'\"\\bfnrtv0-9xu]
CharacterEscapeSequence {SingleEscapeCharacter}|{NonEscapeCharacter}
EscapeSequence {CharacterEscapeSequence}|{OctalEscapeSequence}|{HexEscapeSequence}|{UnicodeEscapeSequence}
DoubleStringCharacter ([^\"\\\n\r]+)|(\\{EscapeSequence})|{LineContinuation}
SingleStringCharacter ([^\'\\\n\r]+)|(\\{EscapeSequence})|{LineContinuation}
StringLiteral (\"{DoubleStringCharacter}*\")|(\'{SingleStringCharacter}*\')

How is it generated?

Grammar definition

%start Program /* Define Start Production */
%% /* Define Grammar Productions */

Statement
    : Block
    | VariableStatement
    | ExpressionStatement
    | IfStatement
    | IterationStatement
    | ReturnStatement
    ;

Block
    : "{" StatementList "}"
        { $$ = new BlockStatementNode($2, createSourceLocation(null, @1, @3)); }
    ;

StatementList
    : StatementList Statement
        { $$ = $1.concat($2); }
    |
        { $$ = []; }
    ;

VariableStatement
    : "VAR" VariableDeclarationList
        { $$ = new VariableDeclarationNode($2, "var", createSourceLocation(null, @1, @2)); }
    ;

Build your own transforms

• babel plugins
  • ​specify in .babelrc
  • are run during transpilation
  • changes code for the compiler/bundler
• jscodeshift
  • "codemods" are run on existing codebase
  • change code in your own codebase

export default function ({types: t}) {
  const canReplace = ({ specifiers }) => {
    return specifiers.length > 0 && specifiers.every((specifier) => {
      return t.isImportSpecifier(specifier)
        && specifier.imported.name !== 'default';
    });
  };

  const replace = (specifiers) => {
    return specifiers.map(({local, imported}) => {
      return t.importDeclaration(
        [t.importDefaultSpecifier(t.identifier(local.name))],
        t.stringLiteral(`react-router/lib/${imported.name}`)
      );
    });
  };

  return {
    visitor: {
      ImportDeclaration(path) {
        if (path.node.source.value === 'react-router') {
          if (canReplace(path.node)) {
            path.replaceWithMultiple(replace(path.node.specifiers));
          }
        }
      }
    }
  };
}

babel plugin transform-react-router-optimize

See in AST explorer