Preface

This is a "non-live" fork of conference proposal slides at

slides.com/willklein/toolmaking-with-asts

This non-live version adds brief speaker notes to the bottom of each slide, for easier consumption. These notes are highly abbreviated; the full-scale talk is more verbose.

I hope that these notes provide the relevant context for each slide, so they may offer value outside of a live presentation.

Toolmaking
With
ASTs

Will Klein

Toolmaking

First, let's talk about Toolmaking.

My Dad

This is my Dad.

Toolmaker

He is a Toolmaker.

Tools

He builds tools, by hand, that are used to automate manufacturing.

Source: https://en.wikipedia.org/wiki/Die_(manufacturing)

Manufacturing Die

Tools like this die, which can be used to repeatedly create a part.

Source: http://studentshop.pratt.duke.edu/lab+space

Milling Machines

He uses machines like these. We had some in our garage growing up.

I have a lot of respect for what he does. I realized at a young age though, I did not want to be a Toolmaker. Despite the high degree of craftsmanship and technical challenge, it can be physically demanding, and you're working with machines that can also be potentially dangerous to operate.

Developer

I grew up and became a developer.

Software

I work on web apps, usually with JavaScript.

Tools

There are tools I use every day as a developer.

function addTen(value) {
  var sum = 0;

  return value + 10
}

if (someCondition) {
  if (someOtherCondition) {
    added = addTen(someValue);
  }
}

This Code...

Occasionally I come across code like this. See any problems?

function addTen(value) {
  return value + 10;
}

if (someCondition &&
    someOtherCondition) {
  added = addTen(someValue);
}

should really be

It can be simplified, perhaps to this.

Code Review

How do we protect against this? Tooling, yes, but often code review.

Reviewer: comprehend code
Reviewer: leave comments
Submitter: review comments
Submitter: fix code (maybe)
Submitter: push changes

Manual Process

Code review is a manual process though. It has a wide feedback loop.

Tools

We want a quicker feedback loop. Sounds like we need better tools.

Linting

Specifically, we need linting. Here, we use ESLint.

function addTen(value) {
  var sum = 0;

  return value + 10
}

no-unused-vars

semi

ESLint Rules

There are built-in rules that catch many problems. These apply here.

if (someCondition) {
  if (someOtherCondition) {
    added = addTen(someValue);
  }
}

???

ESLint Rules

There's no rule to detect nesting a lone if within an if, at least not yet.

Toolmaking

Sounds like we need to do some Toolmaking.

Toolmaking
With
ASTs

...with ASTs.

Abstract

Syntax
Trees

ASTs are abstract syntax trees. Let's see what they look like.

Wizardry

When a non-programmer sees code, they may think: "Wizardry."

Wizardry?

When a programmer sees an AST, they may also think: "Wizardry."

IfStatement

if (someCondition) {}

It's actually quite simple though. Here's a basic IfStatement as a tree.

if (someCondition) {}

IfStatement

{
  "type": "Program",
  "body": [
    {
      "type": "IfStatement",
      "test": {
        "type": "Identifier",
        "name": "someCondition"
      },
      "consequent": {
        "type": "BlockStatement",
        "body": []
      },
      "alternate": null
    }
  ]
}

Here is that tree in JSON form, with a few more details.

Nested IfStatement

if (someCondition) {
  if (someOtherCondition) {
    added = addTen(someValue);
  }
}

Building on that, here's a nested IfStatement.

Nested IfStatement

And its tree. The first If's BlockStatement has some children now.

Nested IfStatement

{
  "type": "IfStatement",
    "test": {
      "type": "Identifier",
      "name": "someCondition"
    },
    "consequent": {
      "type": "BlockStatement",
      "body": [
        {
          "type": "IfStatement",
          "test": {
            "type": "Identifier",
            "name": "someOtherCondition"
          },
          "consequent": {
            "type": "BlockStatement",
            "body": [ ... ] } } ] } }

In JSON form, that shows up in the body collection.

Builds an AST of your code
Traverses the tree
For each node, run rules

ESLint: Under the Hood

Before we think about a rule, let's consider how ESLint works.

Builds an AST of your code
Traverses the tree
For each node, run rules
Rules are built-in, or plugins

ESLint: Under the Hood

ESLint also runs pluggable rules, alongside its built-in rules.

ESLint: Rule boilerplate

module.exports = function(context) {
  return {






  };
};

An ESLint rule is a function that returns an object map.

ESLint: Rule boilerplate

module.exports = function(context) {
  return {
    "IfStatement": function(node) {




    }
  };
};

This object map's keys match against node types, like IfStatement.

ESLint: Rule boilerplate

module.exports = function(context) {
  return {
    "IfStatement": function(node) {
      if (/* test some things */) {
        context.report(node,
            "Unexpected bad code.");
      }
    }
  };
};

The rule should check some things, and report on bad code.

Nested IfStatement

Let's consider how you can check for nested IfStatements.

ESLint: context

module.exports = function(context) {
  return {
    "IfStatement": function(node) {
      var ancestors = context.getAncestors(),
          parent = ancestors.pop(),
          grandparent = ancestors.pop();

      if (/* test some things */) {
        context.report(node,
            "Unexpected bad code.");
      }
    }
  };
};

As it turns out, accessing a node's ancestors is simple.

Directly nested CASE

if (someCondition)
  if (someOtherCondition) {}

Let's start with the simplest form of a nested IfStatement.

directly nested TEST

if (someCondition)
  if (someOtherCondition) {}

var ancestors = context.getAncestors(),
    parent = ancestors.pop(),
    grandparent = ancestors.pop();

if (parent.type === "IfStatement") {
  context.report(node, "Unexpected bad code.");
}

To test this, we just check if an IfStatement parent is an IfStatement.

block Nested CASE

if (someCondition) {
  if (someOtherCondition) {}
}

How about a block nested IfStatement?

block Nested Test

var ancestors = context.getAncestors(),
    parent = ancestors.pop(),
    grandparent = ancestors.pop();

if (parent.type === "IfStatement" ||
    (parent.type === "BlockStatement" &&
     grandparent.type === "IfStatement")) {
  context.report(node, "Unexpected bad code.");
}

if (someCondition) {
  if (someOtherCondition) {}
}

In that case, we do the same check with the grandparent.

Siblings test

var ancestors = context.getAncestors(),
    parent = ancestors.pop(),
    grandparent = ancestors.pop();

if (parent.type === "IfStatement" ||
    (parent.type === "BlockStatement" &&
     grandparent.type === "IfStatement" &&
     parent.body.length === 1)) {
  context.report(node, "Unexpected bad code.");
}

if (someCondition) {
  doSomething();
  if (someOtherCondition) {}
}

There is the case of siblings, which we don't want to report.

consequent test

var ancestors = context.getAncestors(),
    parent = ancestors.pop(),
    grandparent = ancestors.pop();

if (parent.type === "IfStatement" ||
    (parent.type === "BlockStatement" &&
     grandparent.type === "IfStatement" &&
     parent.body.length === 1 &&
     parent === grandparent.consequent)) {
  context.report(node, "Unexpected bad code.");
}

if (someCondition) {
  doSomething();
} else {
  if (someOtherCondition) {}
}

There is also the nesting under an else. no-lonely-if handles this.

no-nested-if

module.exports = function(context) {
  return {
    "IfStatement": function(node) {
      var ancestors = context.getAncestors(),
          parent = ancestors.pop(),
          grandparent = ancestors.pop();
        
      if (parent.type === "IfStatement" ||
          (parent.type === "BlockStatement" &&
           parent.body.length === 1 &&
           grandparent.type === "IfStatement" &&
           parent === grandparent.consequent)) {
        context.report(node, "Unexpected bad code.");
      }
    }
  };
};

Here is our full rule. Not so bad, eh?

github.com/willklein/toolmaking-with-asts

Code

This repo has a richer code example and a full rule plugin with tests.

Funny thing...

Toolmaker

I grew up and became a Toolmaker.

Source: https://en.wikipedia.org/wiki/Numerical_control

CNC Machines

There are also these machines, robots really, that you can program.

Source: https://github.com/grbl/grbl/wiki/G-Code-Examples

G-Code

( RUN IN VISE ON PARALLELS )
(Z OFFSET: TOP OF MATERIAL WITH )
( 0.375" MATERIAL ABOVE VISE JAWS )
(X0,Y0,Z0= Center, Center, Top)
(STOCK ORIGIN = X0. Y0. Z.01)
(MATERIAL TYPE= ALUMINUM inch - 6061)
(MATERIAL SIZE= X1.75 Y1.75 Z.5)
(TOOL= 1/4 2-FLUTE HSS END MILL)

In something like G-Code. My Dad was learning this while I took CS.

Funny thing...

Developer

My Dad grew older and became a Developer.

Fin

Will Klein

If you learned something or have questions, tweet me @willslab.

Toolmaking With ASTs

Toolmaking

My Dad

Toolmaker

Tools

Manufacturing Die

Milling Machines

Developer

Software

Tools

This Code...

should really be

Code Review

Manual Process

Tools

Linting

ESLint Rules

ESLint Rules

Toolmaking

Toolmaking With ASTs

Abstract

Syntax Trees

Wizardry

Wizardry?

IfStatement

IfStatement

Nested IfStatement

Nested IfStatement

Nested IfStatement

ESLint: Under the Hood

ESLint: Under the Hood

ESLint: Rule boilerplate

ESLint: Rule boilerplate

ESLint: Rule boilerplate

Nested IfStatement

ESLint: context

Directly nested CASE

directly nested TEST

block Nested CASE

block Nested Test

Siblings test

consequent test

no-nested-if

Code

Funny thing...

Toolmaker

CNC Machines

G-Code

Funny thing...

Developer

Fin

Toolmaking
With
ASTs

Toolmaking
With
ASTs

Syntax
Trees