The Art of Crafting Codemods

About me

twitter/rajasegar_c

Front-end Developer @ Freshworks

github/rajasegar

hangaroundtheweb.com

We are hiring!

https://careers.freshworks.com

pradeep.naik@freshworks.com

The Problem(s)

1. Making changes in code

Let's start with a small change

Replace variable "hello" with "world"

let hello = "world";
function hello() {
  console.log("hello world")
}
$ sed 's/hello/world' input.js
$ sed 's/hello/world' project1/app/**/*.js project2/app/**/*.js
let world = "world";
function world() {
  console.log("world world")
}
let hello = "world";
function hello() {
  console.log("hello world")
}

Change with(in) a context

Replace only the variable name "hello" with "world"

Replace variable "hello" with "world"

The Context

  • Which one is a variable name?

  • Which one is a function name?

  • Which one is a string literal?

2. Upgrading Code-base

Types of Upgrade

  • Upgrade to new language syntax / features
  • Upgrade a lib / framework to a newer version

Upgrade to new language syntax / features

mounted() {
    var self = this
    window.addEventListener('scroll', function() {
        self.scrolled = true
    })
}
mounted() {
    window.addEventListener('scroll', () => {
        this.scrolled = true
    })
}

Arrow functions

Upgrade a lib / framework to a newer version

attached: function () {
  doSomething()
}
mounted: function () {
  this.$nextTick(function () {
    doSomething()
  })
}

Vue 1.x to 2.0

Making Changes to code

  • Not just find/replace
  • Remove code
  • Add code
  • And much more...

What is a Codemod

Code to rewrite code

  • Partial Automation
  • Human oversight
  • Occasional intervention

What codemods can do?

  • Update source code to fit a team’s coding conventions

  • Make widespread changes when an API is modified

  • Automate large-scale refactoring tasks

  • Easily update your code to take advantage of newer language features

Code Transformations

What is an AST?

A Tree representation of the abstract syntactic structure of source code written in a programming language.

An AST is basically a DOM for your code.

How Compilers create Abstract Syntax Trees?

Scanner

( Lexical Analyzer )

function helloWorld() {
    console.log('hello world');
}

Parser

( Syntax Analyzer )

function helloWorld() {
    console.log('hello world');
}
{
  "type": "Program",
  "start": 0,
  "end": 44,
  "body": [
    {
      "type": "FunctionDeclaration",
      "start": 0,
      "end": 44,
      "id": {
        "type": "Identifier",
        "start": 9,
        "end": 14,
        "name": "hello"
      },
      "expression": false,
      "generator": false,
      "params": [],
      "body": {
        "type": "BlockStatement",
        "start": 17,
        "end": 44,
        "body": [
          {
            "type": "ExpressionStatement",
            "start": 21,
            "end": 42,
            "expression": {
              "type": "CallExpression",
              "start": 21,
              "end": 41,
              "callee": {
                "type": "MemberExpression",
                "start": 21,
                "end": 32,
                "object": {
                  "type": "Identifier",
                  "start": 21,
                  "end": 28,
                  "name": "console"
                },
                "property": {
                  "type": "Identifier",
                  "start": 29,
                  "end": 32,
                  "name": "log"
                },
                "computed": false
              },
              "arguments": [
                {
                  "type": "Literal",
                  "start": 33,
                  "end": 40,
                  "value": "hello",
                  "raw": "'hello'"
                }
              ]
            }
          }
        ]
      }
    }
  ],
  "sourceType": "module"
}

https://astexplorer.net

AST

Where is it used?

  • Syntax Highlighting
  • Code Completion
  • Static Analysis (aka ESLint, etc.,)
  • Code Coverage
  • Minification
  • JIT Compilation
  • Source Maps
  • Compiling to JS languages
  • Code Refactoring & Migrations
  • And much more ...

Codemods

jscodeshift

A JavaScript codemod toolkit.

RUNNER

WRAPPER

jscodeshift

RUNNER

WRAPPER

jscodeshift

( CLI )

( recast )

RUNNER

jscodeshift

( CLI )

recast

esprima

ast-types

AST => DOM

jscodeshift  =>   jQuery

recast

https://github.com/benjamn/recast

const recast = require('recast');
const code = fs.readFileSync('code.js','utf-8');
const ast = recast.parse(code);
const faster = transform(ast);
const output = recast.print(faster).code;

/**
 * This replaces every occurrence of variable "foo".
 */
module.exports = function(fileInfo, api) {
  return api.jscodeshift(fileInfo.source)
    .findVariableDeclarators('foo')
    .renameTo('bar')
    .toSource();
}

jscodeshift

https://github.com/facebook/jscodeshift

Let's create a Codemod!

Reference Videos

  1. https://www.youtube.com/watch?v=d0pOgY8__JM
  2. https://www.youtube.com/watch?v=1X9p-RUUkak
  3. https://www.youtube.com/watch?v=8r_sXUDoPYo
  4. https://www.youtube.com/watch?v=mkg3NWcloOw&list=LLVxvrINFNKL9kCbakNFjstg&index=9&t=0s
  5. https://frontendmasters.com/courses/linting-asts/introducing-codemods-and-ast/
  6. https://www.youtube.com/watch?v=C06MohLG_3s

Blog Posts

  1. https://medium.com/@cpojer/effective-javascript-codemods-5a6686bb46fb
  2. https://benmccormick.org/2018/06/18/codemod-survival/
  3. https://www.sitepoint.com/getting-started-with-codemods/
  4. https://www.toptal.com/javascript/write-code-to-rewrite-your-code
  5. http://hangaroundtheweb.com/2019/03/codemods-the-new-age-saviors-for-js-developers/

Codemod tooling

  1. https://github.com/facebook/jscodeshift
  2. https://github.com/benjamn/recast
  3. https://github.com/benjamn/ast-types
  4. https://rajasegar.github.io/ast-builder/
  5. https://astexplorer.net
  6. https://github.com/rajasegar/awesome-codemods

Vue codemods

https://github.com/SergioCrisostomo/vue-codemods

Take aways

#1

The idea is to not break things and to build transformation tools that we can be confident in.

#2

Tool-assisted code modification is set to profoundly transform the way people evolve and maintain very large scale codebases.

Questions?

Thank you

Made with Slides.com