Using Grunt to Auto-Generate Gruntfiles

Joel Kemp, @mrjoelkemp

Bēhance/Adobe

The Key Takeaways

• Grunt and why we need it
• What "static analysis" really is
• Static analysis techniques for building smarter tools

The Complicated Front-end

So many tasks...

Build steps for front-end apps

• Compile Preprocessors
  • HTML: Slim, Jade
  • CSS: SASS, LESS, Stylus
  • JS: Coffeescript, Typescript, Clojurescript
• Browserify or Requirejs bundling
• Managing vendor dependencies
• Precompile Mustache/Handlebars templates
• JSHint and JSCS the codebase
• Run unit tests and/or integration tests
• And more...

So we use tools like:

Grunt, Gulp, or Broccoli

Let's talk about Grunt

• Task Runner
  • Task: a plugin and configuration options

A Sample Gruntfile

module.exports = function(grunt) {
  grunt.initConfig({
    // 1. Compile all scss files 
    sass: {
      compile: {
        files: { 
          expand: true, 
          src: ['**/*.scss'], 
          dest: 'css/' }
      }
    },
    watch: {
      // 2. Watch all scss files and run the sass task
      sass: {
        files: ['**/*.scss'],
        tasks: ['sass']
      }
    }
  });
  // 3. Load plugins
  grunt.loadNpmTasks('grunt-contrib-sass');
  grunt.loadNpmTasks('grunt-contrib-watch');
};

About these Gruntfiles

• They get large – fast
  • Yet another thing to maintain 
• Boilerplate aside from custom plugins
  • All we really wanted to do was build our app
• The ideal: "set it and forget it" 

How can we stop writing Gruntfiles?

• Based on generators
• Assumes every app is the same
• Provides the kitchen sink

• Adapts to preprocessors and JS modules
• Generates and maintains your Gruntfile
• Too ambitious, but great for small things

Auto-compiling Preprocessors

SASS and CoffeeScript

Custom Watch Hook

• Grunt has a watcher (via grunt-contrib-watch)
  • Listen for new files being added
  • Get the file extension
• If it's .scss
  • npm install grunt-contrib-sass
  • Generate a compile task
  • Generate a watch task
  • There's your gruntfile, flush to disk
• If it's .coffee
  • npm install grunt-contrib-coffee
  • Generate compile and watch tasks
  • Hotswap the Gruntfile

Hook into the watcher

// Custom callback for the watch event
grunt.event.on('watch', function(action, filepath) {
  var ext = path.extname(filepath);
  // Tell YA about added file extensions
  if (action === 'added') {
    process.send('EXTADDED:' + ext);
  }
});

More at: http://bit.ly/hijack-grunt

Gruntfile so far

grunt.initConfig({
  // 1. Compile all scss files (simplified)
  "sass": {
    "compile": {
      "files": [{ "src": ["**/*.{scss, sass}"] }]
    }
  },
  "coffee": {
    "compile": {
      "files": [{ "src": ["**/*.coffee"] }]
    }
  },
  "watch": {    
    // 2. Watch all scss files and run the sass task on change
    "sass": {
      "files": [
        "**/*.scss",
      ], 
      "tasks": ["sass"]
    },
    // CoffeeScript watch task is very similar...
  }
});

• Throw Compass into the mix... 
• Task configuration for SASS becomes dynamic
• The plugin choice changes:
  • If compass is installed, use grunt-contrib-compass
  • Otherwise, use grunt-contrib-sass
• The compile task changes:
  • Separation of sass/ from stylesheets/
• Just create SASS (LESS/Stylus) files and let Grunt do the rest

Auto-bundling JS apps

Browserify and Requirejs

Never again...

"requirejs": {
  "bundle": {
    "options": {
      // 1. What's the entry point?
      "include": "app.js", 
      // 2. What should we call the generated bundle?
      "out": "output.js"
    }
  }
}

Use the watcher again

• When you add a JavaScript file: 
• Determine if it's in AMD or CommonJS
  • If AMD, use grunt-contrib-requirejs
  • If CommonJS, use grunt-browserify
• Set up the bundle task(s)
  • Need to determine the entry point ("root") of the app
  • Auto-name the output bundles
• Set up the watch task to auto-bundle on change
• Hotswap the configuration

// Custom callback for the watch event
grunt.event.on('watch', function(action, filepath) {
  var ext = path.extname(filepath);

  if (action === 'added') {
    process.send('EXTADDED:' + ext);
  }
});

Detect CommonJS or AMD

Static analysis with Esprima

Rules for AMD and CommonJS

var isAMD = hasDefine || hasAMDTopLevelRequire;

Is it CommonJS or AMD?

define([
  './a'
], function(a) {
  'use strict';

});

Identifying the "root"

• A "root" is a module that has no dependents
  • i.e., No other module requires it
  • Represents a separate app/bundle
• The algorithm:
  • Maintain a list of all required modules
  • For each JS module:
    • Get its dependencies (via a detective)
    • Mark each dependency as "used"
    • The JS modules that aren't in the list are roots

Our Gruntfile Now

grunt.initConfig({
  // 1. Bundle task
  "requirejs": {
    // 2. Auto-generated target name
    "t0": {
      "options": {
        // 3. The determined root
        "include": "js/amd/a2.js", 
        // 4. Auto-generated output bundle name
        "out": "a2-r-bundle.js"
      }
    }
  },
  "watch": {
    // 4. Watch all "relevant" JS files and rebundle on change
    "requirejs": {
      // Avoid watching 3rd party libraries 
      // and the bundles themselves (infinite loop)
      "files": ["**/*.js", allTheExceptions],
      "tasks": ["requirejs"]
    }
  }
});

What about the other tasks?

• Precompile templates 
  • Detect .mustache/.handlebars files 
  • Use grunt-contrib-handlebars 
• JSHint and JSCS
  • Detect .jshintrc or .jscsrc files
  • Use grunt-contrib-jshint or grunt-jscs
• Run tests
  • Traverse AST looking for 'describe' or 'it' CallExpression
• Manage vendor dependencies
  • Detect CallExpression with identifier $ for jQuery
  • npm install jquery
  • Not trivial. What about shims?

What's the next step?

Where do we take our tools?

Some options/questions

• YA
  • Beta/Experimental
  • Is an almighty tool the answer?
• Can we build these techniques into Grunt or its plugins?
  • grunt-contrib-requirejs
• Or into Browserify and RequireJS?
• Gulp could use these abilities too!

Thanks :)

Joel Kemp, @mrjoelkemp

Bēhance/Adobe