Why all the fuss
about Fibers?

About me

  1. Meteor developer for over 7 years.
  2. Software Architect at Vazco.
  3. Author of uniforms and a few other packages.

Everything is at radekmie.dev.

(Slides for this presentation too.)

Tens of issues, discussions, forum posts, and now pull requests... Why everyone went crazy about "Fibers"? What even is that!? I work with Meteor, a Node.js framework, and use Promises everywhere. Why the hell should I care about some C/C++ library from 2011?

~ Someone, maybe

fuss

/fʌs/

(noun) a display of unnecessary or excessive excitement, activity, or interest.

 

Fibers timeline

Fibers in Meteor

The preamble in v0.3.2 Meteor's
mongo-livedata package (source).

/**
 * Provide a synchronous Collection API using fibers, backed by
 * MongoDB.  This is only for use on the server, and mostly identical
 * to the client API.
 *
 * NOTE: the public API methods must be run within a fiber. If you call
 * these outside of a fiber they will explode!
 */

Meteor magic

// Create the collection.
import { Mongo } from 'meteor/mongo';

const Posts = new Mongo.Collection('posts');









Collection#insert returns the inserted ID immediately both on the client and the server.






// Create a new post.
const result = Posts.insert({
  title: 'Clickbait here!',
  text: 'Have you ever...',
});














// What is in the `result`?
console.log(result);

MongoDB driver

// Create the collection.
import { MongoClient } from 'mongodb';
const client = new MongoClient('mongodb://...');
const Posts = client.db().collection('posts');









Collection#insert returns a promise of the response (it includes the inserted ID).






// Create a new post.
const result = await Posts.insertOne({
  title: 'Clickbait here!',
  text: 'Have you ever...',
});














// What is in the `result`?
console.log(result);

Where's the await?









// EXTREMELY simplified version.
MongoConnection.prototype._insert =
  function (collection_name, document, callback) {
    this.rawCollection(collection_name)
      .insertOne(document)
      .then(({ insertedId }) => { callback(null, insertedId); })
      .catch(error => { callback(error, null); });
  };

Meteor#wrapAsync magically got rid of the promise...? Let's look into it!

// Exact implementation.
_.each(["insert", "update", "remove", "dropCollection", "dropDatabase"], function (method) {
  MongoConnection.prototype[method] = function (/* arguments */) {
    var self = this;
    return Meteor.wrapAsync(self["_" + method]).apply(self, arguments);
  };
});









Down the rabbit hole...

// EXTREMELY simplified version.
if (Meteor.isServer)
  var Future = Npm.require('fibers/future');

Meteor.wrapAsync = function (fn, context) {
  return function (/* ...args, callback */) {
    if (!callback) {
      if (Meteor.isClient) {
        callback = logErr; // Logs the error (if any).
      } else {
        var future = new Future();
        callback = future.resolver();
      }
    }

    callback = Meteor.bindEnvironment(callback);
    var result = fn.apply(context || this, [...args, callback]);
    return future ? future.wait() : result;
  };
};

FIBERS!

*snip*

(Barely 46 A4 pages.)

What can you do today?

  1. Try out the new *Async MongoDB methods (#12028).
    1. If you're not ready for that yet, at least ensure you'll be able to add await to all database operations (i.e., make the surrounding functions async).
    2. It's not always going to be possible, e.g., in some Atmosphere packages. If so, check #11505 whether it's already in progress; if not, do file an issue about that!
  2. Remove all of the Fiber and Future occurrences from your code. In most cases, it'll be as easy as one await.
    1. That includes Promise.await, Promise#await, and other Meteor-specific functions, as they rely on Fibers.

?

await without async

declare module 'meteor/promise' {
  export class Promise extends globalThis.Promise {
    static async any, This, Args extends any[]>(
      fn: Fn,
      allowReuseOfCurrentFiber?: boolean,
    ): (this: This, ...args: Args) => Promise>;
    static asyncApply any, This, Args extends any[]>(
      fn: Fn,
      context: This,
      args: Args,
      allowReuseOfCurrentFiber?: boolean,
    ): Promise>;

    static await(value: PromiseLike): T;
    static awaitAll(values: Iterable>): T[];

    await(): T;
  }
}

All of these are Meteor-specific. The less you use them, the easier it will be to migrate off the Fibers.

What's next?

  1. A lot of pull requests regarding async APIs
    (meteor/meteor#12028, meteor/meteor#12101, meteor/meteor#12103, meteor/meteor#12105, meteor/meteor#12156).
  2. Top-level await support (meteor/meteor#12095).
  3. Do something with client-side integrations
    (meteor/blaze#364, meteor/react-packages#360).
  4. Make Fibers truly optional.
  5. Update to Node.js 16.
  6. Update to Node.js 18.
  7. Celebrate 2077 Christmas Eve.

Questions?

Bonus!

As we're already using the latest MongoDB driver version, we should make the most of it. I plan to work on adding other collection methods that are not there yet, i.e., distinct, estimatedDocumentCount, findOneAndUpdate, insertMany...

I aim to make the Collections API on-par with the Node.js driver. Most of the work will remain on the Minimongo side, though.

Also, how about change streams...? I've conducted some heavy tests with them, and will post the results soon. However, I'd start with exposing them for the end-users first, rather than replacing oplog.