Ember Outside the Box

By Michael Lange

@DingoEatingFuzz

What do you do when the framework doesn't have a solution to your problem? 

What’s a framework and what’s a library?

Frameworks call your code

You call library code

Frameworks

are boxes

 

Libraries

are bags

Frameworks are rigid

And they can feel restricting.

There's an inversion

There's an inversion

of ownership.

But frameworks are so good!

All software has unique problems

So what do we do?

You're bumping into walls, now what?

Four Quick Suggestions

  1. Desperately scroll through Ember Observer
  2. Abandon Ember and try to figure out Webpack again
  3. Scold ourselves for not listening to all those Google evangelists who tell us to only use vanilla js
  4. Leave tech, move to Wyoming, and become a rancher

What we did at Lytics

The Problem

  1. A complex UI for building boolean algebra formulas
  2. An overly complicated API for expressing these formulas
  3. A series of UX optimizations to make boolean algebra more approachable

Ember didn't have the answers.

Controllers + Views

Not so great for nested views decoupled from the router 

ED Dependent Relations

A custom answer to ED not dirtying "parent" models

But the concerns still mingled.

Image from http://www.52kitchenadventures.com/2011/09/12/melted-crayon-art-tutorial/

// Converts the composite segment into a copy of the first child segment
collapseSegment: function() {
  const collapsing = this.get('model');
  let onlyChild = collapsing.get('children.firstObject');

  // TODO dependency: This is a bug in ED, who knows if it's been fixed by now
  if (!collapsing.get('children.length')) {
    onlyChild = null;
  }

  // To maintain the root segment's ID, it must take on the operator and
  // children/fields of the only child
  if (this.get('isRoot')) {
    if (!onlyChild) {
      // Revert to an undefined type if there are no children
      collapsing.set('operator', null);
    } else if (!onlyChild.get('isNamed')) {
      const type = onlyChild.get('type');

      // Replace the collapsed segment with its only child by converting to a rule, or
      // copying its children (or just clearing them if the segment has no type yet)
      if (type === 'rule') {
        collapsing.setProperties(onlyChild.getProperties([
          'type',
          'operator',
          'fieldName',
          'fieldKey',
          'isInverted'
        ]));
        collapsing.get('values').setObjects(onlyChild.get('values').toArray());
        collapsing.get('children').clear();
      } else if (type === 'composite') {
        collapsing.get('children').setObjects(onlyChild.get('children').toArray());
      } else {
        collapsing.get('children').clear();
        collapsing.set('type', 'rule');
      }
    }
  } else {
    const siblings = this.get('parentSegment.children');

    // Replace the collapsed segment with its only child
    siblings.replace(siblings.indexOf(collapsing), 1, [ onlyChild ]);
  }
},
{{!-- This is the top-level container that is used to expand the root segment,
which only allows the segment tree to be expanded to the max depth --}}
{{#segments/node-container
  selectedField=selectedField
  model=model
  availableSegments=availableSegments
  showAddButtons=(lt model.maxDepth maxAllowedDepth)
  alwaysExpand=true
  class="top-level"}}
  {{#if model.isComposite}}
    {{segments/node-composite
      model=model
      availableSegments=availableSegments
      totalSize=totalSize
      class="item segment"}}
  {{else if model.isRule}}
    {{segments/node-rule
      model=model
      totalSize=totalSize
      class="item segment"}}
  {{/if}}
{{/segments/node-container}}
didLoad: function() {
  // Consider the following scenario:
  //
  // 1. A segment is created
  // 2. The app is reloaded
  // 3. A batch segment size request includes the id of the new segment
  // 4. The segment's size is returned as `null` since it hasn't been cached
  //
  // What needs to happen is for the segment's size to be reloaded using the
  // non-caching API. However, at this point only the segment size serializer
  // knows that the given segment doesn't have a cached size. So the size
  // could be reloaded 1) in the serializer or 2) here in the model. Note that
  // no matter how it is reloaded, the segment size adapter must use the
  // correct API based on the fact that the record has already loaded.
  //
  // I present to you the lesser of two evils.
  if (!isPresent(this.get('value'))) {
    this.reload();
  }
}.observes('value')

Quick Fixes?

Desperately scroll through Ember Observer?

Desperately scroll through Ember Observer?

Domain specific problems won't be solved by add-ons alone.

Abandon Ember and try to figure out Webpack again?

Abandon Ember and try to figure out Webpack again?

>70,000 LOC (not counting tests), and who is to say this problem wouldn't follow us?

Scold ourselves for not listening to all those Google evangelists who tell us to only use vanilla js?

Scold ourselves for not listening to all those Google evangelists who tell us to only use vanilla js?

This wasn't about time to first paint and it couldn't be fixed by service workers.

Leave tech, move to Wyoming, and become a rancher?

Leave tech, move to Wyoming, and become a rancher?

So tempting, but it defies the three virtues of a great programmer. Notably #3, hubris.

So the API was rewritten...

(i.e., refactoring time)

Now an AST API

Abstract Syntax Tree

Treat the AST like an embedded record

But don't use ED for this embedded record!

export default BaseModel.extend(Eventable, {
  name                : DS.attr('string', { defaultValue: '' }),
  description         : DS.attr('string', { defaultValue: '' }),
  isPublic            : DS.attr('boolean', { defaultValue: false }),
  slug                : DS.attr('string', { defaultValue: '' }),
  modifiedAt          : DS.attr('date'),
  kind                : DS.attr('string', { defaultValue: 'segment' }),
  tags                : DS.attr(),
  isServerSideInvalid : DS.attr('boolean', { defaultValue: false }),
  schema              : DS.belongsTo('schema', { async: false }),
  cachedSize          : DS.belongsTo('segment_size', { async: true }),
  
  // A SegmentDefinition object
  definition          : DS.attr(),

  // The segments that rely on this segment
  dependents   : DS.hasMany('segment', { async: false, inverse: 'dependencies' }),
  // The segments that this segment relies on
  dependencies : DS.hasMany('segment', { async: false, inverse: 'dependents' }),

  works : DS.hasMany('work', { async: true }),
  trend : DS.belongsTo('segment-trend', { async: true }),

  // Since it is possible to receive `null` for the cached size from the API,
  // a fallback is needed.
  size: function() {
    if (this.get('isServerSideInvalid')) { return null; }

    const cachedSize = this.get('cachedSize');
    if (cachedSize.get('value') != null || cachedSize.get('isLoading')) {
      return cachedSize;
    }

    return sizeForDefinition(this.store, this.get('definition'));
  }.property('cachedSize.value', 'isServerSideInvalid', 'definition'),
  // ...
}
normalize(type, hash, prop) {
  // The 'table' field is the schema FK
  hash.schema_id = hash.table;

  // The segment id is the segment_trend FK
  if (hash.id) {
    hash.trend_id = hash.id;
    hash.cached_size_id = hash.id;
  }

  // Keep track of all other segments this segment utilizes
  hash.dependency_ids = dependenciesForAST(hash.ast);

  // Convert the ast into a modelish
  hash.definition = SegmentDefinition.create({
    ast: hash.ast,
    store: this.store
  });

  return this._super(type, hash, prop);
},
export default function sizeForDefinition(store, definition, schema = 'user') {
  if (schema instanceof Schema) { schema = schema.get('name'); }

  return store.findRecord('segment_definition_size', JSON.stringify({
    table: schema,
    ast: definition instanceof SegmentDefinition ? definition.get('ast') : definition
  }));
}
normalizeSingleResponse(store, type, payload, recordId) {
  const sizeArray = payload.data;
  const size = Array.isArray(sizeArray) ? sizeArray[0] : null;
  const record = JSON.parse(recordId);
  const data = {
    id: recordId,
    schema_id: record.table,
    ast: record.ast,
    value: size
  };

  return this._super(store, type, data, recordId);
},
utils/segments/size-for-definition.js
serializers/segment-definition-size.js
findRecord(store, type, id, snapshot) {
  const params = { segments: `[${id}]` };
  return this.ajax(this.buildURL(type.modelName, id, snapshot), 'GET', { data: params });
},
adapters/segment-definition-size.js

Segment

SegmentDefinition

SegmentDefinitionSize

ED model

custom

custom

JSONSerializer

RESTAdapter

Make Segment Definitions "immutable"

replace imperative property changes with object diffing

invertSegment() {
  const store = this.get('store');
  const siblings = this.get('parentSegment.children');
  let inverter, original;

  if (this.get('model.isInverter')) {
    inverter = this.get('model');
    original = inverter.get('children.firstObject');

    // Replace the inverter segment with its only child
    siblings.replace(siblings.indexOf(inverter), 1, [ original ]);
  } else {
    original = this.get('model');
    inverter = store.createRecord('segment', {
      operator: 'not',
      schema: original.get('schema')
    });

    // Add the original segment as the first child of the new inverter segment
    inverter.get('children').addObject(original);

    // Replace the original segment with the new inverter segment
    siblings.replace(siblings.indexOf(original), 1, [ inverter ]);
  }
},
invertSegment() {
  const model = this.get('model');

  if (model.get('isInverted')) {
    model.transform(model.get('children.firstObject').clone());
  } else {
    model.transform(S('not', model.clone()));
  }
}

Before

After

Where have I heard this before...

Virtual DOM?

Eliminating state management

Disaster Mitigation

 

🔥

declarative programming

Imperative code describes what a program does

Declarative code describes what a program is

model.transform(S('or',
  S('and',
    S('>', 'field_name', 50),
    S('include', 'segment_name')
  ),
  S('>=', 'field_two', 99)
));

Segment Builder component focuses on UI

Declarative Segment Definition transforms make model code a detail rather than a concern

  addChild(child) {
    const targetModel = this.get('targetModel');
    const operator = targetModel.get('operator');

    if (this.get('depth') >= this.get('maxDepth')) {
      targetModel.transform(S(operator,
        ...targetModel.get('children').mapBy('ast'),
        child
      ));
    } else {
      targetModel.transform(S(operator,
        ...targetModel.get('children').mapBy('ast'),
        S(operator === 'and' ? 'or' : 'and', child)
      ));
    }
  },
  1. Fitting a custom model implementation into Ember Data

  2. Replacing common imperative get/set code with declarative transformations

  3. Making Handlebars more powerful

Outside the Box

Handlebars is intentionally low-power

The segment builder component of choice is a function of the segment definition

Strategy Pattern

In the strategy pattern, behaviors are defined as separate interfaces and specific classes that implement these interfaces. This allows better decoupling between the behavior and the class that uses the behavior.

{{#each childrenForBuilder as |child|}}
  {{#if child.builder}}
    {{component (concat "segments/builders/builder-" (or child.builder "empty"))
      definition=child.definition
      depth=nextDepth
      maxDepth=maxDepth
      blacklist=blacklist
      schema=schema}}
  {{/if}}
  <div class="segment-toggle">
    <div class="bar"></div>
    {{and-or-toggle
      value=viewOperator
      disabled=(lte targetModel.children.length 1)
      action="toggleOperator"
      toggleClass="and-or-toggle action-toggle segment-operator"}}
  </div>
{{/each}}

builder-composite.hbs

each child gets rendered using the appropriate component

{{#if model.hasSiblings}}
  <div class="segment-rule-controls">
    <a class="segment-remove action-remove" {{action "removeSegment"}}>
      {{lio-icon "trash"}}
    </a>
  </div>
{{/if}}
<div class="segment-definition">
  <span class="label label-default label-invert">Existing</span>
  <span class="segment-name">{{targetModel.reference.name}}</span>
  <div class="segment-controls">
    <div class="segment-invert">
      <small>
        <button class="btn btn-xs {{if isExcluded "btn-secondary"}} action-toggle" {{action "toggleExcluded"}}>
          {{if isExcluded "excluded" "included"}}
        </button>
      </small>
    </div>
    {{segment-size segment=model}}
  </div>
</div>

builder-reference.hbs

simple component knows nothing about how it got here

import { typeFromOperator } from '../../models/segment-definition';

/**
 * Segment Builder Stategies
 *
 * A strategy takes the form of [ componentName, predicate ]
 */
const strategies = {
  rule: [
    [ 'rule', () => true ],
  ],
  composite: [
    [ 'rule', ast => ast.op === 'not' && typeFromOperator(ast.args[0].op) === 'rule' ],
    [ 'composite', ast => ast.op === 'not' && typeFromOperator(ast.args[0].op) === 'composite' ],
    [ 'reference', ast => ast.op === 'not' && typeFromOperator(ast.args[0].op) === 'reference' ],
    [ 'composite', () => true ],
  ],
  reference: [
    [ 'reference', () => true ],
  ],
  all: [
    [ 'all', () => true ],
  ],
};

export default function componentForSegmentDefinition(definition) {
  const strategiesForType = strategies[typeFromOperator(definition.get('operator'))];
  return strategiesForType.find(strategy => strategy[1](definition.get('ast')))[0];
}

builder-strategies.js

an extensible set of predicates used to determine a builder

Don't map UI one-to-one with your API

API surface area is finite, but the design space is unbounded

Results

Before

After

225 PR comments

76 commits

116 files changed

3,292 insertions

2,522 deletions

¯\_(ツ)_/¯

The Aftermath

New Segment

Builder

  1. New Content Affinity Tab
  2. New Campaigns Tab
  3. New condition builder UI concept

Content Segment

Builder

  1. Very different UI
  2. Very different requirements
  3. No builder strategies
  4. Still using segment transforms

Ember 1.11 → 2.9 

No problems

(in this code)

Takeaways

Takeaways

  1. When given the chance to write domain-specific code, align designs with the business
  2. Maintenance opportunities are always opportunities for hammock time1
  3. Frameworks are amazing, but they will never have all the answers
  4. Frameworks are all created with philosophies that are bigger than their implementations

1: Rich Hickey on Hammock Time: https://www.youtube.com/watch?v=f84n5oFoZBc

Thank you!

Ember Outside the Box

By Michael Lange

Ember Outside the Box

What do you do when the framework doesn't have a solution to your problem?

  • 742