Composition

@integralist    @charlierevett

What?

Config driven page construction

Where?

  • http://www.bbc.co.uk/newsbeat
  • https://admin.live.bbc.co.uk/cosmos
  • https://github.com/BBC-News/composition
  • https://github.com/BBC-News/composition-example

How?

  • JRuby (JVM)
  • Sinatra

  • Multi-threaded component aggregation (Typhoeus)

  • Template interpolation via Mustache

  • Implemented as a reusable Gem

  • Static assets use BBC Spindle in production

Configuration Examples

Service level config:

config.ru

require "composition"

cache  = nil
config = {
  :base_url_env              => "",
  :debug                     => "true",
  :indices                   => "home_page",
  :local_static_asset_path   => "./public",
  :page_config_base_path     => "./config/page/base.yaml",
  :page_config_defaults_path => "./config/page/default.yaml",
  :page_config_path          => "./config/page",
  :site_root                 => "news",
  :static_asset_path         => "/",
  :timeout                   => "4"
}

run Composition.create(config, cache)

Configuration Examples

Page level config:

home_page.yaml

components:
  - id: "title"             # passed with broker URL when requesting components
    page_location: "header" # correlates to value defined inside mustache template
    mandatory: true         # defaults to false
    opts:
      foo: "bar"            # static query string options added to base_url
    dynamic_options: true   # add runtime query string params as part of service URL

Configuration Examples

Page level config:

base.yaml

base_url: "https://some_s3_location.com/bucket/object/path"

components:
  - id: "head"
    page_location: "orb_meta"
    mandatory: true
  - id: "bodylast"
    page_location: "orb_footer"
    mandatory: true
  - id: "bodyfirst"
    page_location: "orb_header"
    mandatory: true

Routing

  • /site-root
  • /site-root/index
  • /site-root/index/child
  • /site-root/page-type/asset-id/slug

Implementation

  • Extracts page type (otherwise assumes site root)
  • Locates matching yaml config file
  • Locates a base yaml config (e.g. page furniture; ORB)
  • Checks a default yaml config file (for missing values)

Implementation

  • Collates components ([]ComponentMeta)
  • Requests components (in parallel: []Component)
  • http://base_url/component/component.id?component.opts

Assemble page

  • Retrieves model and template files (matched by page type)
  • Interpolates the data into the template
  • Return rendered content to the user (setting appropriate HTTP headers)

Issues?

  • Typhoeus bugs
  • JVM
  • Asset Pipeline

Changes?

  • Fix the Typhoeus bugs
  • Extract "component request" mechanism?
  • Rewrite completely to avoid JVM cold starts?

Extraction

  • Contenders: Go vs Node
  • High Concurrency vs True Parallelism
  • Go routines/Channels vs Event Loop
  • Go routines aren't real threads
    • Context switches within thread pool
    • i.e. highly concurrent
  • Node is single threaded
    • Event model
    • i.e. highly concurrent

Rewrite

  • Go vs Node
  • Go is super fast (near C level speed)
  • Go has really great tooling as standard
  • Go is easy'ish to learn
  • Node is more familiar

  • It is just JS after all

  • Node has questionable stability/reliability at peak load

  • We should have a tech spike to justify our decision

Discussion

deck

By Mark McDonnell

deck

  • 1,679