2020 @phenomnominal

I'm Craig.

I do JavaScript.

2020 @phenomnominal

Today, I'm going to tell you a story...

2020 @phenomnominal

Infoshare attendee!

Good Guy

You're a wizard!

2020 @phenomnominal

You went to Hogwarts!!

(School of Witchcraft and Wizardry)

Place where stuff happens

2020 @phenomnominal

You can do magic!!

This is a patronus

2020 @phenomnominal

You know Dumbledore!!!

Good Guy

2020 @phenomnominal

And Harry Potter is your Best Friend!!!!

Main Good Guy

2020 @phenomnominal

But there's a problem...

Maybe
Good?
Maybe
 Bad?

2020 @phenomnominal

Voldemort is back (again!)

Literally the

Worst Guy

2020 @phenomnominal

And he has put a curse on Harry!

Siriously

Evil.

2020 @phenomnominal

2020 @phenomnominal

If we don't save him in the next 25 minutes, Voldemort is going to kill him!

2020 @phenomnominal

This is bad.

Our friend is gone.

Time is running out.

2020 @phenomnominal

But there is one bit of good news...

2020 @phenomnominal

Harry speaks Parseltongue!

He can talk to snakes!

2020 @phenomnominal

Which, as every magical being knows, is a

Turing-complete programming language!

(this is 100% canon, don't look it up)

2020 @phenomnominal

Weakly Typed

Functional

Interpreted

Garbage collected

Significant whitespace

Parseltongue 🐍🐍🐍

2020 @phenomnominal

Parseltongue 🐍🐍🐍

Variable Declarations

Parseltongue:

JavaScript:

Only one type of variable

Always start with

Assignment Operator is

<~
sss

2020 @phenomnominal

Function Expressions

Parameters are defined with an array

Return uses the assignment operator

Significant whitespace

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2020 @phenomnominal

Call Expression

Passing arguments to a function also uses the Assignment operator

Arguments are declared with an array

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2020 @phenomnominal

Control Flow

Use an empty array when you have no arguments

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2020 @phenomnominal

Loops

Range operator is

~>

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2020 @phenomnominal

Spells

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

is magic and becomes

sssCast
alert

2020 @phenomnominal

2020 @phenomnominal

Harry's spell ⭐️⭐️⭐️

2020 @phenomnominal

Internet browsers don't understand Parseltongue.

2020 @phenomnominal

We need to translate Harry's Parseltongue into JavaScript!

2020 @phenomnominal

Parseltongue

JavaScript

???

2020 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2020 @phenomnominal

Let's go a line at a time:

RegEx:

(the darkest of the dark arts)

2020 @phenomnominal

That gives us:

RegEx:

2020 @phenomnominal

Converting variables

RegEx:

2020 @phenomnominal

Converting loops

Hmmm...

RegEx:

2020 @phenomnominal

Converting functions

Ruh roh!

RegEx:

2020 @phenomnominal

2020 @phenomnominal

What are we really trying to do?

Transform one language into another

2020 @phenomnominal

Transpiling

(aka transfiguration)

2020 @phenomnominal

"Transpiling is a specific term for taking source code written in one language and transforming into another language that has a similar level of abstraction"

Transpiling

2020 @phenomnominal

Where do we start?

String literal

Function call

Identifier

Expression

2020 @phenomnominal

ESTree

2020 @phenomnominal

Esprima

2020 @phenomnominal

 Abstract

  disassociated from any specific instance

  the way in which linguistic elements are put together

 Syntax

 TREE

  data structure made up of vertices and edges without any cycles

2020 @phenomnominal

Wat.

2020 @phenomnominal

Wat.

2020 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

Code:

2020 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

AST:

2020 @phenomnominal

Which means...

2020 @phenomnominal

If we can get from Parseltongue to an AST...

Then we can go from that AST to JavaScript!

2020 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2020 @phenomnominal

Parseltongue

JavaScript

AST

2020 @phenomnominal

Parseltongue

JavaScript

AST

???

???

2020 @phenomnominal

Remember Esprima?

2020 @phenomnominal

Remember Esprima?

Esprima takes code written in a specific language (JavaScript) and turns it into an AST!

2020 @phenomnominal

We need to make our own Esprima for Parseltongue!

2020 @phenomnominal

How do we do that?

2020 @phenomnominal

Parseltongue

JavaScript

AST

???

???

Tokens

Lexing

2020 @phenomnominal

Lexing/Tokenising

"Lexing is the process of breaking down source code into words that are relevant to the language, which are called tokens"

2020 @phenomnominal

Lexing/Tokenising

Tokens in JavaScript

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

 

RegularExpression

Template

(we're getting a bit intense now - here's an owl)

2020 @phenomnominal

Lexing/Tokenising

Tokens in Parseltongue

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

 

Indent

LineTerminator

Space

(we're getting a bit intense now - here's an owl)

2020 @phenomnominal

Identifier characters

Space

Punctuator characters

Space

Quote character

Lexing/Tokenising

Punctuator

Punctuator

String characters

Quote character

2020 @phenomnominal

2020 @phenomnominal

2020 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

???

Tokens

Lexing

2020 @phenomnominal

Parsing

"Parsing is the process of taking the lexical tokens and applying the grammar of the language to them"

2020 @phenomnominal

Parsing

Variable Declaration

Identifier

Space

Punctuator - 

Space

String literal

<~

2020 @phenomnominal

Identifier

Space

Punctuator -

Identifier

Punctuator - 

Space

Identifier

Punctuator - 

Parsing

Function Declaration

Line Terminator

Indent

[
,
]

2020 @phenomnominal

Parsing

Function Declaration

2020 @phenomnominal

Actual LOL

2020 @phenomnominal

2020 @phenomnominal

Now we just need to go from the AST to JavaScript...

2020 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2020 @phenomnominal

Code Generating

This is really hard to do by ourselves.

We can again stand on the shoulders of half-giants!

2020 @phenomnominal

ESCodeGen

2020 @phenomnominal

2020 @phenomnominal

JavaScript!

2020 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2020 @phenomnominal

We're almost ready to save Harry!

2020 @phenomnominal

But OH NO!

Voldemort can also speak Parseltongue!

2020 @phenomnominal

Voldemort will try to stop Harry from escaping!

 

He might even use the...

2020 @phenomnominal

Unforgivable Functions

2020 @phenomnominal

We need to come up with a way to stop Voldemort from using those functions!

2020 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

???

2020 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspecting

2020 @phenomnominal

Inspecting

Transformation

Mutation

Validation

2020 @phenomnominal

ESQuery

2020 @phenomnominal

Linting

Detect bad stuff

2020 @phenomnominal

Linting

Fixing bad stuff

Tickling curse

2020 @phenomnominal

2020 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspection

2020 @phenomnominal

2020 @phenomnominal

We did it!

Harry is safe!

2020 @phenomnominal

All is well

(again)

2020 @phenomnominal

So...

2020 @phenomnominal

When do you actually use an AST?

2020 @phenomnominal

Ever had to change every file in your codebase?

2020 @phenomnominal

Ever had to generate boilerplate?

2020 @phenomnominal

Ever had to extract API documentation?

2020 @phenomnominal

You can use an AST for that!

 

(and probably should)

2020 @phenomnominal

ASTs help us with...

Performance

Maintenance

Consistency

Analysis

Robustness

Automation

2020 @phenomnominal

They're everywhere

2020 @phenomnominal

They're powerful

2020 @phenomnominal

They're useful

2020 @phenomnominal

Once you know where to find them

2020 @phenomnominal

Thanks!

Me: Craig

Twitter: @phenomnominal

Fantastic ASTs and Where To Find Them (Infoshare)

By Craig Spence

Fantastic ASTs and Where To Find Them (Infoshare)

Infoshare 2020

  • 2,957