The Art of Crafting Codemods
About me
twitter/rajasegar_c
Front-end Developer @ Freshworks
github/rajasegar
hangaroundtheweb.com
We are hiring!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838944/pasted-from-clipboard.png)
pradeep.naik@freshworks.com
The Problem(s)
1. Making changes in code
Let's start with a small change
Replace variable "hello" with "world"
let hello = "world";
function hello() {
console.log("hello world")
}
$ sed 's/hello/world' input.js
$ sed 's/hello/world' project1/app/**/*.js project2/app/**/*.js
let world = "world";
function world() {
console.log("world world")
}
let hello = "world";
function hello() {
console.log("hello world")
}
Change with(in) a context
Replace only the variable name "hello" with "world"
Replace variable "hello" with "world"
The Context
-
Which one is a variable name?
-
Which one is a function name?
-
Which one is a string literal?
2. Upgrading Code-base
Types of Upgrade
- Upgrade to new language syntax / features
- Upgrade a lib / framework to a newer version
Upgrade to new language syntax / features
mounted() {
var self = this
window.addEventListener('scroll', function() {
self.scrolled = true
})
}
mounted() {
window.addEventListener('scroll', () => {
this.scrolled = true
})
}
Arrow functions
Upgrade a lib / framework to a newer version
attached: function () {
doSomething()
}
mounted: function () {
this.$nextTick(function () {
doSomething()
})
}
Vue 1.x to 2.0
Making Changes to code
- Not just find/replace
- Remove code
- Add code
- And much more...
What is a Codemod
Code to rewrite code
- Partial Automation
- Human oversight
- Occasional intervention
What codemods can do?
-
Update source code to fit a team’s coding conventions
-
Make widespread changes when an API is modified
-
Automate large-scale refactoring tasks
-
Easily update your code to take advantage of newer language features
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398639/code-ast-code.png)
Code Transformations
What is an AST?
A Tree representation of the abstract syntactic structure of source code written in a programming language.
![](https://media2.giphy.com/media/vh9isNb4S2Spa/giphy.gif)
An AST is basically a DOM for your code.
![](https://media0.giphy.com/media/3owzWl78kny9s2GOvC/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398406/compiler-stage.png)
How Compilers create Abstract Syntax Trees?
Scanner
( Lexical Analyzer )
function helloWorld() {
console.log('hello world');
}
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398437/tokens.png)
Parser
( Syntax Analyzer )
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398585/ast.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398437/tokens.png)
function helloWorld() {
console.log('hello world');
}
{
"type": "Program",
"start": 0,
"end": 44,
"body": [
{
"type": "FunctionDeclaration",
"start": 0,
"end": 44,
"id": {
"type": "Identifier",
"start": 9,
"end": 14,
"name": "hello"
},
"expression": false,
"generator": false,
"params": [],
"body": {
"type": "BlockStatement",
"start": 17,
"end": 44,
"body": [
{
"type": "ExpressionStatement",
"start": 21,
"end": 42,
"expression": {
"type": "CallExpression",
"start": 21,
"end": 41,
"callee": {
"type": "MemberExpression",
"start": 21,
"end": 32,
"object": {
"type": "Identifier",
"start": 21,
"end": 28,
"name": "console"
},
"property": {
"type": "Identifier",
"start": 29,
"end": 32,
"name": "log"
},
"computed": false
},
"arguments": [
{
"type": "Literal",
"start": 33,
"end": 40,
"value": "hello",
"raw": "'hello'"
}
]
}
}
]
}
}
],
"sourceType": "module"
}
AST
Where is it used?
- Syntax Highlighting
- Code Completion
- Static Analysis (aka ESLint, etc.,)
- Code Coverage
- Minification
- JIT Compilation
- Source Maps
- Compiling to JS languages
- Code Refactoring & Migrations
- And much more ...
![](https://media1.giphy.com/media/ftdluHcQBSgdpeo30i/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405382/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405383/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405385/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405387/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405394/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405397/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405399/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405405/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405412/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405421/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405428/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405432/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405435/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405436/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6405438/pasted-from-clipboard.png)
Codemods
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6398668/parse-transform-print.png)
jscodeshift
A JavaScript codemod toolkit.
RUNNER
WRAPPER
jscodeshift
RUNNER
WRAPPER
jscodeshift
( CLI )
( recast )
RUNNER
jscodeshift
( CLI )
recast
esprima
ast-types
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838836/codemod-high-level.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838839/codemod-stage1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838843/codemod-stage2.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838844/codemod-stage3.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838845/codemod-stage4.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838842/codemod-stage-final.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838836/codemod-high-level.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/330317/images/6838842/codemod-stage-final.png)
AST => DOM
jscodeshift => jQuery
![](https://media2.giphy.com/media/l4pTosVr0iHCJ11hm/giphy.gif)
recast
const recast = require('recast');
const code = fs.readFileSync('code.js','utf-8');
const ast = recast.parse(code);
const faster = transform(ast);
const output = recast.print(faster).code;
/**
* This replaces every occurrence of variable "foo".
*/
module.exports = function(fileInfo, api) {
return api.jscodeshift(fileInfo.source)
.findVariableDeclarators('foo')
.renameTo('bar')
.toSource();
}
jscodeshift
Let's create a Codemod!
![](https://media0.giphy.com/media/pGac4wa4HHDMVZhFIa/giphy.gif)
Reference Videos
- https://www.youtube.com/watch?v=d0pOgY8__JM
- https://www.youtube.com/watch?v=1X9p-RUUkak
- https://www.youtube.com/watch?v=8r_sXUDoPYo
- https://www.youtube.com/watch?v=mkg3NWcloOw&list=LLVxvrINFNKL9kCbakNFjstg&index=9&t=0s
- https://frontendmasters.com/courses/linting-asts/introducing-codemods-and-ast/
- https://www.youtube.com/watch?v=C06MohLG_3s
Blog Posts
- https://medium.com/@cpojer/effective-javascript-codemods-5a6686bb46fb
- https://benmccormick.org/2018/06/18/codemod-survival/
- https://www.sitepoint.com/getting-started-with-codemods/
- https://www.toptal.com/javascript/write-code-to-rewrite-your-code
- http://hangaroundtheweb.com/2019/03/codemods-the-new-age-saviors-for-js-developers/
Codemod tooling
- https://github.com/facebook/jscodeshift
- https://github.com/benjamn/recast
- https://github.com/benjamn/ast-types
- https://rajasegar.github.io/ast-builder/
- https://astexplorer.net
- https://github.com/rajasegar/awesome-codemods
Vue codemods
Take aways
#1
The idea is to not break things and to build transformation tools that we can be confident in.
#2
Tool-assisted code modification is set to profoundly transform the way people evolve and maintain very large scale codebases.
Questions?
![](https://media0.giphy.com/media/l0Iy8hSJalxmgTOF2/giphy.gif)
Thank you
![](https://media3.giphy.com/media/l2R0eYcNq9rJUsVAA/giphy.gif)
The Art of Crafting Codemods
By Rajasegar Chandiran
The Art of Crafting Codemods
A presentation about Codemods in Vue Hyderabad meetup #2
- 2,261