2018 @phenomnominal

I'm Craig.

I do JavaScript at

in New Zealand

2018 @phenomnominal

Let's start with a story...

2018 @phenomnominal

JSConf attendee!

Good Guy

You're a wizard!

2018 @phenomnominal

You went to Hogwarts!!

(School of Witchcraft and Wizardry)

Place where stuff happens

2018 @phenomnominal

You can do magic!!

This is a patronus

2018 @phenomnominal

You know Dumbledore!!!

Good Guy

2018 @phenomnominal

And Harry Potter is your Best Friend!!!!

Main Good Guy

2018 @phenomnominal

But there's a problem...

Maybe
Good?
Maybe
 Bad?

2018 @phenomnominal

Voldemort is back (again!)

Literally the

Worst Guy

2018 @phenomnominal

And he has put a curse on Harry!

Siriously

Evil.

2018 @phenomnominal

2018 @phenomnominal

If we don't save him in the next 25 minutes, Voldemort is going to kill him!

2018 @phenomnominal

This is bad.

Our friend is gone.

Time is running out.

2018 @phenomnominal

But there is one bit of good news...

2018 @phenomnominal

Harry speaks Parseltongue!

He can talk to snakes!

2018 @phenomnominal

Which, as every magical being knows, is a

Turing-complete programming language!

(this is 100% canon, don't look it up)

2018 @phenomnominal

Weakly Typed

Functional

Interpreted

Garbage collected

Significant whitespace

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue 🐍🐍🐍

Variable Declarations

Parseltongue:

JavaScript:

Only one type of variable

Always start with

Assignment Operator is

<~
sss

2018 @phenomnominal

Function Expressions

Parameters are defined with an array

Return uses the assignment operator

Significant whitespace

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue:

JavaScript:

Call Expression

Passing arguments to a function also uses the Assignment operator

Arguments are declared with an array

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue:

JavaScript:

Control Flow

Use an empty array when you have no arguments

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue:

JavaScript:

Loops

Range operator is

~>

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue:

JavaScript:

Spells

Parseltongue 🐍🐍🐍

2018 @phenomnominal

Parseltongue:

JavaScript:

is magic and becomes

sssCast
alert

2018 @phenomnominal

Harry's spell ⭐️⭐️⭐️

2018 @phenomnominal

Internet browsers don't understand Parseltongue.

2018 @phenomnominal

We need to translate Harry's Parseltongue into JavaScript!

2018 @phenomnominal

Parseltongue

JavaScript

???

2018 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2018 @phenomnominal

Let's go a line at a time:

RegEx:

(the darkest of the dark arts)

2018 @phenomnominal

That gives us:

RegEx:

2018 @phenomnominal

Converting variables

2018 @phenomnominal

RegEx:

Converting loops

Hmmm...

2018 @phenomnominal

RegEx:

Converting functions

Ruh roh!

2018 @phenomnominal

RegEx:

2018 @phenomnominal

What are we really trying to do?

Transform one language into another

2018 @phenomnominal

Transpiling

(aka transfiguration)

2018 @phenomnominal

"Transpiling is a specific term for taking source code written in one language and transforming into another language that has a similar level of abstraction"

Transpiling

2018 @phenomnominal

Where do we start?

String literal

Function call

Identifier

Expression

2018 @phenomnominal

ESTree

2018 @phenomnominal

Esprima

2018 @phenomnominal

 Abstract

  disassociated from any specific instance

  the way in which linguistic elements are put together

 Syntax

 TREE

  data structure made up of vertices and edges without any cycles

2018 @phenomnominal

Wat.

2018 @phenomnominal

Wat.

2018 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

Code:

2018 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

AST:

2018 @phenomnominal

Which means...

2018 @phenomnominal

If we can get from Parseltongue to an AST...

Then we can go from that AST to JavaScript!

2018 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2018 @phenomnominal

Parseltongue

JavaScript

AST

2018 @phenomnominal

Parseltongue

JavaScript

AST

???

???

2018 @phenomnominal

2018 @phenomnominal

Remember Esprima?

2018 @phenomnominal

Remember Esprima?

Esprima takes code written in a specific language (JavaScript) and turns it into an AST!

2018 @phenomnominal

We need to make our own Esprima for Parseltongue!

How do we do that?

2018 @phenomnominal

Parseltongue

JavaScript

AST

???

???

Tokens

Lexing

2018 @phenomnominal

Lexing/Tokenising

"Lexing is the process of breaking down source code into words that are relevant to the language, which are called tokens"

2018 @phenomnominal

Lexing/Tokenising

Tokens in JavaScript

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

 

RegularExpression

Template

(we're getting a bit intense now - here's an owl)

2018 @phenomnominal

Lexing/Tokenising

Tokens in Parseltongue

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

 

Indent

LineTerminator

Space

(we're getting a bit intense now - here's an owl)

2018 @phenomnominal

Identifier characters

Space

Punctuator characters

Space

Quote character

Lexing/Tokenising

2018 @phenomnominal

Punctuator

Punctuator

String characters

Quote character

2018 @phenomnominal

2018 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

???

Tokens

Lexing

2018 @phenomnominal

Parsing

"Parsing is the process of taking the lexical tokens and applying the grammar of the language to them"

2018 @phenomnominal

Parsing

Variable Declaration

Identifier

Space

Punctuator - 

Space

String literal

<~

2018 @phenomnominal

Identifier

Space

Punctuator -

Identifier

Punctuator - 

Space

Identifier

Punctuator - 

Parsing

Function Declaration

Line Terminator

Indent

[
,
]

2018 @phenomnominal

Parsing

Function Declaration

2018 @phenomnominal

Actual LOL

2018 @phenomnominal

2018 @phenomnominal

Now we just need to go from the AST to JavaScript...

2018 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2018 @phenomnominal

Code Generating

This is really hard to do by ourselves.

We can again stand on the shoulders of half-giants!

2018 @phenomnominal

ESCodeGen

2018 @phenomnominal

2018 @phenomnominal

2018 @phenomnominal

JavaScript!

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2018 @phenomnominal

We're almost ready to save Harry!

2018 @phenomnominal

But OH NO!

Voldemort can also speak Parseltongue!

2018 @phenomnominal

Voldemort will try to stop Harry from escaping!

 

He might even use the...

2018 @phenomnominal

Unforgivable Functions

2018 @phenomnominal

We need to come up with a way to stop Voldemort from using those functions!

2018 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

???

2018 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspecting

2018 @phenomnominal

Inspecting

Transformation

Mutation

Validation

2018 @phenomnominal

ESQuery

2018 @phenomnominal

Linting

Detect bad stuff

2018 @phenomnominal

Linting

Fixing bad stuff

Tickling curse

2018 @phenomnominal

2018 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspection

2018 @phenomnominal

2018 @phenomnominal

We did it!

Harry is safe!

2018 @phenomnominal

All is well

(again)

2018 @phenomnominal

So...

2018 @phenomnominal

When do you actually use an AST?

2018 @phenomnominal

Ever had to change every file in your codebase?

2018 @phenomnominal

Ever had to generate boilerplate?

2018 @phenomnominal

Ever had to extract API documentation?

2018 @phenomnominal

You can use an AST for that!

 

(and probably should)

2018 @phenomnominal

ASTs help us with...

Performance

Maintenance

Consistency

Analysis

Robustness

Automation

2018 @phenomnominal

They're everywhere

2018 @phenomnominal

They're powerful

2018 @phenomnominal

They're useful

2018 @phenomnominal

Once you know where to find them

2018 @phenomnominal

Thanks!

Me: Craig

Twitter: @phenomnominal

Code: github.com/phenomnomnominal/parseltongue

Slides: slides.com/craigspence/fantastic-asts-and-where-to-find-them-jsconf

Resources:

2018 @phenomnominal

Fantastic ASTs and Where To Find Them

By Craig Spence

Fantastic ASTs and Where To Find Them

JSConfAu

  • 3,822