Adding Webpack to Rails

@fraserxu

Adding Webpack to Marketplace

Why?

  • React-rails gem
  • Global variables(React)
  • Non standard ES6(module) code
  • Sprockets manage assets
  • Hacky to test(mock, code transform)
<%= content_tag :div,
  :class => "home-search__search",
  :data => {
    :view => autosuggest_viewloader_attribute,
    :react_components_path => javascript_path('market/pages/homepage/components/autosuggest'),
} do %>

The problem need to solve

  • Don't break the existing code
  • Don't change the way people work
  • Progressive enhancement
  • Developer experience(happiness)
  • Testability, maintainability, reusability
  • Deployment process(CI, staging, production, caching, CDN...)

The structure

.
├── README.md
├── app                                       // app entry where we use to build the app
│   └── appEntry.js
├── components                                // react components
│   └── autosuggest
├── tests                                     // tests for components
│   ├── autosuggest_list.test.js
│   └── autosuggest_list_item.test.js
└── webpack                                   // webpack configs
    ├── webpack.client.base.config.js         // shared config
    ├── webpack.client.rails.build.config.js  // build webpack config with manifest
    └── webpack.client.rails.dev.config.js    // dev webpack config with live-compile

Assets management

  • compile assets into public/webpack
  • also compile a public/webpack/manifest.json like sprockets' manifest
  • use "/webpack/app.js" in development(will use webpack dev server in the future)
  • use "https://abc123.cloudfront.net/webpack/app-bundle-def456.js" in production

webpack.rb

module Webpack
  def self.manifest
    @manifest ||= JSON.parse(File.read(Rails.root.join("public/webpack/manifest.json").to_s))
  end

  def self.asset_path(path)
    if Rails.env.development?
      "/webpack/#{path}"
    else
      "#{Rails.application.config.action_controller.asset_host}/webpack/#{manifest.fetch(path)}"
    end
  end

  def self.javascript_path(file)
    asset_path("#{file}.js")
  end
end

{
  "app.js": "app-10f44fcd836cfb718328.js",
  "vendor.js": "vendor-da69203f114ff6e7e25f.js"
}

manifest.json

Writing ES6 and React Code

  1. Write standard React component
  2. + *Componentify(expose module)
  3. + Inject React and ReatDOM in a separate file
  4. + LoadOnce script to page(vendor + app)
  5. Render(mount) to page
// import 3rd party modules, could be any modules from npm
import React from 'react'
import ReactDOM from 'react-dom'

// internal modules, usually your actually code
import AutosuggestComponent from './_autosuggest_list_component'

// here we create a AutoSuggest Object with a `renderOn` method,
// the reason behind this is that we don't want to make React and ReactDOM as global object provided by Rails
const AutoSuggest = {
   // The render on method inject the react props and DOMNode as params
  renderOn (props, reactMountNode) {
    // here we create a reactComponent
    let reactComponent = React.createElement(AutosuggestComponent, props)
    // and render it with ReactDOM
    ReactDOM.render(reactComponent, reactMountNode)
  }
}

// export the Component and we will use it in the app Entry
export default AutoSuggest
/**
 * This the entry for all the React Component
 * All the component are under EnvatoReactComponent namespace
 * e.g, EnvatoReactComponent.AutoSuggest
 */
export AutoSuggest from '../components/autosuggest'
config.output = {
  filename: '[name]-bundle.js',
  path: '../javascripts/dist',
  library: 'EnvatoReactComponent',
  libraryTarget: 'umd'
}
class Views.AutosuggestReact
  constructor: (@$el) ->
    vendorBundlePath = @$el.data().vendorBundlePath
    reactComponentsPath = @$el.data().reactComponentsPath

    if reactComponentsPath
      Market.Helpers.loadOnce([vendorBundlePath, reactComponentsPath]).done =>
        @_initialize()

  _initialize: =>
    @$searchInput        = @$el.find('.js-term')
    @analyticsParamInput = @$el.find('.js-autosuggest-analytics-param')[0]
    @reactMountNode      = @$el.find('.js-react-mount-node')[0]
    @serviceEndpoint     = @reactMountNode.dataset.autosuggestServiceEndpoint
    @initialSuggestions  = @reactMountNode.dataset.initialSuggestions
    @_renderReact()

    # GA Ecommerce tracking
    new Market.GoogleAnalytics.Autosuggestion.init()
  _renderReact: =>
    initialSuggestions = JSON.parse(@initialSuggestions)
    props =
      $searchInput: @$searchInput
      analyticsParamInput: @analyticsParamInput
      initialSuggestions: initialSuggestions
      socket:
        channel: 'autocomplete:standard'
        endpoint: @serviceEndpoint
        throttleAmount: 50
    if EnvatoReactComponent && EnvatoReactComponent.AutoSuggest
      EnvatoReactComponent.AutoSuggest.renderOn(props, @reactMountNode)

Testing

Development process

TODO: Hot-reloading with webpack hot middleware



$ foreman start

Staging server

NPM_CACHE_PATH="$CACHE_DIR/vendor/npm"
CACHE_NAME="nm_cache.tar"
export NDOE_ENV=production

if [ -z "$CACHE_DIR" ]; then
  echo "Expected \$CACHE_DIR to be set"
  exit 1
fi

# Make sure cache directory exist
if [ ! -d "$NPM_CACHE_PATH" ]; then
  mkdir -p "$NPM_CACHE_PATH"
fi

# Make sure cache file exist
# Install node packages with cache support
if [ ! -e "$NPM_CACHE_PATH"/"$CACHE_NAME" ]; then
  time (npm prune && npm install && tar -cf "$NPM_CACHE_PATH"/"$CACHE_NAME" node_modules)
else
  time (tar -xf "$NPM_CACHE_PATH"/"$CACHE_NAME" && npm prune && npm install && tar -cf "$NPM_CACHE_PATH"/"$CACHE_NAME" node_modules)
fi

# Compile es6 code into es5
time npm run build

./script/samon/deploy_staging.sh

CI - Buildkite

./script/buildkite_javascript.sh

NPM Caching strategy

  • No bundle check
  • npm cache is still slow

The Problem

  • Check cache by fetching tar file(with unique filename)
  • If not exist, `npm install`
  • If exist, fetch and extract tarball

The npm way

architecture=$(gem environment platform | cut -d ':' -f2)
hash=$(shasum -a 1 package.json | cut -d' ' -f1)
node_version=$(node -v | cut -d ' ' -f2)
filename="${hash}-${node_version}-${architecture}.tar"

Because the way npm works we can ensure the cached node_modules always works given the same `package.json` on same OS and nodejs version, even for native modules that need compile

TODO: Private node cache server?

NPM Caching strategy

#!/bin/bash

set -e -x

NODE_MODULES="node_modules"
s3_prefix="s3://marketplace-ops/buildkite/cache/node_modules/"

architecture=$(gem environment platform | cut -d ':' -f2)
hash=$(shasum -a 1 package.json | cut -d' ' -f1)
node_version=$(node -v | cut -d ' ' -f2)
filename="${hash}-${node_version}-${architecture}.tar"

get_node_modules_cache () {
  echo "Fetching node_modules from S3 npm cache"
  aws s3 cp --region us-east-1 "${s3_prefix}${filename}" . && tar xf $filename $NODE_MODULES
}

npm_install () {
  echo "Installing node_modules"
  npm prune && npm install
}

set_node_modules_cache () {
  echo "Updating S3 npm cache"
  tar -cf $filename $NODE_MODULES && aws s3 mv $filename $s3_prefix
}

get_node_modules_cache || (npm_install && set_node_modules_cache)

The result

What's next?

Market Frontend catch up - JavaScript 2016 plan

 

Thanks!

Question and Feedbacks?

Made with Slides.com