What is synchronization & concurrency in JavaScript - Use a TypeScript task queue library to introduce

2021.08.01@COSCUP in Taiwan

Grimmer

https://github.com/grimmer0125

video (Mandarin)

Outline

Introduce what is async and concurrency in JavaScript
Introduce why do we need a task queue library
Introduce to more code usage
Introduce the code implementation
Q & A

d4c-queue task queue lib features

Wrap an async/promise-returning/sync function as a queue-ready async function for reusing
Sub queues system & extra config
TypeScript / JavaScript / Node.js / Browser
Two usages
- D4C instance: synchronization & concurrency mode.
- @synchronized & @concurrent decorators on classes

Key: concurrent !!

const d4c = new D4C([{ limit: 100 }]);
const fetchGitHub = d4c.wrap(async()=>{
  return fetch('https://api.github.com/')
})

class TestController {
  @synchronized
  async fetchGitHub(url: string) {
    return fetch('https://api.github.com/')
  }
}

What is async in JavaScript

const fun3 = async () => {
  // or trigger UI/Network action
  console.log("fun3") 
}

const fun1 = async () => {
  console.log("fun1_start")
  await fun3(); 
  console.log('fun1_end');
};

function testAsync() {
  fun1();
}

testAsync();
// fun1_start: execution 1 of event loop
// fun3
// ..
// .. may include other microtask, 
// .. e.g. network msgs
// ..
// fun1_end: not execution 1 of event loop

macrotasks: setTimeout, setInterval, UI event, XMLHttpRequest ...
microtasks (=PromiseJobs in ECMAScript): promise (e.g. fetch.then) / async, queueMicrotask ...

May trigger some UI/Network actions

What is concurrency in JavaScript

const fun3 = async () => {
  console.log("fun3")
}

const fun4 = async () => {
  console.log("fun4")
}

const fun1 = async () => {
  console.log("fun1_start")
  await fun3();
  console.log('fun1_end');
};

const fun2 = async () => {
  console.log("fun2_start")
  await fun4();
  console.log('fun2_end');
};

function testConcurrency() {
  fun1();
  fun2();
}
testConcurrency();
// fun1_start
// fun3
// fun2_start
// fun4
// fun1_end
// fun2_end

fun2 is started as fun1 is not finished. They are not really executed simultaneously in main thread (a.k.a. parallelly). Concurrency: Multiple tasks are started but not finished within a time span.

Concurrency is good

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(fetch('https://api.github.com/'));
}
const results = await Promise.all(promiseList);

Network thread may wait for more than one response at the same time. Performant built-in network ability.

But some issues you may encounter and a task queue lib can solve

Concurrency
- rate limit
- causality
- race condition
Convenience

Rate limit:

just config d4c-queue limit parameter

const d4c = new D4C([{ limit: 10 }]);
const fetchGitHub = d4c.wrap(async()=>{
  return fetch('https://api.github.com/')
})

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(fetchGitHub);
}
const results = await Promise.all(promiseList);

@QConcurrency([
  { limit: 10 },
])
class TestController {
  @concurrent
  async fetchGitHub(url: string) {}
}
const test = new TestController()

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(test.fetchGitHub);
}
const results = await Promise.all(promiseList);

way1: d4c instance

way2: decorator

alternatives:

p-queue (fixed time interval)
p-throttle w/ its strict mode (time interval between arbitrary two)

Causality

E.g. initConnect & sendMessage while using a client

sendMessage(msg: string) {
  if (this.connectingStatus === 'Connected') {
    // send message
  } else if (this.connectingStatus === 'Connecting') {
    // Um...how to wait for connecting successfully?
  } else (this.connectingStatus === 'Disconnected') {
    // try to re-connect
  }
}

How?

class ServiceAdapter {
  async send_message(msg: string) {
    if (this.connectingStatus === 'Connected') {
      /** send message */
      await send_message_without_wait_connect(msg);
    } else if (this.connectingStatus === 'Connecting') {
      /** send message */
      await send_message_wait_connect(msg);
    } else {
      //..
    }
  }

  @synchronized
  async initConnect() {
    // ...
  }

  @synchronized
  async send_message_wait_connect(msg: string) {
    // ...
  }

  async send_message_without_wait_connect(msg: string) {
    // ...
  }
}

@synchronized is equal to Concurrency limit =1

Using d4c-queue, you can

Real world case:

code snippet is from embedded-pydicom-react-viewer which runs python in Browser. I'll give a talk in 2021 pycontw talk

const d4c = new D4C();

export const initPyodide = d4c.wrap(async () => {
  /** init Pyodide*/
});

/** without d4c-queue, it will throw exception while being called
 * before 'initPyodide' is finished */
export const parseByPython = d4c.wrap(async (buffer: ArrayBuffer) => {
  /** execute python code in browser */
});

Race condition

sometimes two tasks would access the same resources that are not atomic. Use Database transaction or this d4c-queue lib to lock

const fun1 = async () => {
  console.log("fun1_start")
  await dbOperation();
  console.log('fun1_end');
};

const fun2 = async () => {
  console.log("fun2_start")
  await dbOperation();
  console.log('fun2_end');
};

Real world case:

Express routing vs Apollo/NestJS GraphQL resolver

/** Express case */
app.post('/testing', async (req, res) => {
  // Do something here
});

/** Apollo server case */
const resolvers = {
  Mutation: {
    orderBook: async (_, { email, book }, { dataSources }) => {},
  },
  Query: {
    books: async () => books,
  },
};

two API callbacks will not be run concurrently

two API callbacks will be run concurrently

const d4c = new D4C();

/** Apollo server case */
const resolvers = {
  Mutation: {
    orderBook: d4c.wrap(async (_, { email, book }, { dataSources }) => {},
  }),
  Query: {
    books: d4c.wrap(async () => books),
  },
};

use d4.wrap

NestJS GraphQL synchronized resolver example with this d4c-queue

import { Query } from '@nestjs/graphql';
import { synchronized } from 'd4c-queue';

function delay() {
  return new Promise<string>(function (resolve, reject) {
    setTimeout(function () {
      resolve('world');
    }, 10 * 1000);
  });
}

export class TestsResolver {
  @Query((returns) => string)
  /** without @synchronized, two resolver may print 1/2 1/2 2/2 2/2
   *  with @synchronized, it prints: 1/2 2/2 2/2 2/2
   */
  @synchronized
  async hello() {
    console.log('hello graphql resolver part: 1/2');
    const resp = await delay();
    console.log('hello graphql resolver part: 2/2');
    return resp;
  }
}

Convenience

async wrap_function() {
  await async_fun1()
  await async_fun2()
}

async current_function() {
  wrap_function()
}

current_function() {
  const d4c = new D4C();
  d4c.apply(async_fun1);
  d4c.apply(async_fun2);
}

current_function() {
  (async ()=>{
    await async_fun1();
    await async_fun2();
  })();
}

Target:

start async_func2 after async_fun1 is finished
do not want to wait for these two to finish, as they will block the current event loop

OR (use d4c-queue)

async current_function() {
  // if we do not want to wait these two and 
  // continue the following codes, how?
  await async_fun1()
  await async_fun2()
  
  // following code
  // ..
}

OR (will force next microtask)

async current_function() {
  queueMicrotask(async () => {
    await async_fun1();
    await async_fun2();
  });
}

More introduction to d4c-queue

pass sync function into a queue and treat it as async

const resp = await d4c.wrap(async()=>{
  return fetch('https://github.com/')
})();

Passing arguments and using await to get return values

wrap or Apply

const result = await d4c.apply((x:string) => x+1, { args: [1] });

const resp = await d4c.wrap(()=>{
  for (let i=0;i<1000000000;i++){    
  }
})();

executed in execution n

executed in execution n+m

How to use on arrow function property

class TestController {
  @autobind  // way1 
  @synchronized // should be the second line
  arrowFunctionProperty1(msg: string) {
  }

  arrowFunctionProperty2 = async () => {
  };
}
// way2
const d4c = new D4C(); 
const res = await d4c.apply(testController.arrowFunctionProperty2);

Sub queue system

D4C queues (decorator) injected into your class:
- instance method queues (per instance):
    - default queue
    - tag1 queue
    - tag2 queue
- static method queues
    - default queue
    - tag1 queue
    - tag2 queue

D4C instance queues (per D4C object):
- default queue
- tag1 queue
- tag2 queue

class ServiceAdapter {
  @synchronized({ tag: 'world', inheritPreErr: true, noBlockCurr: true })
  static async staticMethod(text: string) {
    return text;
  }
}

Static method example

{ inheritPreErr: true } task1 encounters fail task2 auto becomes fail

setConcurrency

const d4c = new D4C();
d4c.setConcurrency([{ limit: 10 }]);

{ noBlockCurr: true } - task1 (pass sync) -> force another execution of event loop, makre sure it becomes async - task2 (pass async)

Points of code implementation

generic FIFO queue class
overloads method decorators:
- parentheses vs no parentheses
- static vs instance methods
class decorator (QConcurrency):
- isStatic (static vs instance methods)
task queue implementation
- D4C instance vs your classes
- core algorithm

FIFO O(1) queue class

export class Queue<T> {
  private head: Node<T> | null = null;
  private tail: Node<T> | null = null;
  public length = 0;

  public push(data: T) {
    const node = new Node(data);
    if (this.tail) {
      this.tail.next = node;
      node.prev = this.tail;
      this.tail = node;
    } else {
      this.head = node;
      this.tail = node;
    }
    this.length += 1;
    return
  }

  public shift() {
    if (this.length > 0) {
      this.length -= 1;
      const node = this.head;
      if (node.next) {
        this.head = node.next;
        this.head.prev = null;
      } else {
        this.head = null;
        this.tail = null;
      }
      return node.data;
    }
    return undefined;
  }
}

method decorators

parentheses vs no parentheses

export function synchronized(
  target: any,
  propertyKey: string,
  descriptor: PropertyDescriptor
): void
export function synchronized(option?: {
  tag?: string | symbol
  inheritPreErr?: boolean
  noBlockCurr?: boolean
}): MethodDecoratorType
export function synchronized(
  targetOrOption?: any,
  propertyKey?: string,
  descriptor?: PropertyDescriptor
): void | MethodDecoratorType {
  return _methodDecorator(targetOrOption, propertyKey, descriptor, false)
}

function _methodDecorator(
  targetOrOption: any,
  propertyKey: string,
  descriptor: PropertyDescriptor,
  isConcurrent: boolean
): void | MethodDecoratorType {
  if (checkIfDecoratorOptionObject(targetOrOption)) {

  } else {

  }
}

overload 1

overload 2

no parentheses case

parentheses case containing option

static vs instance methods

function injectQueue(
  constructorOrPrototype,
  tag: string | symbol,
  isConcurrent: boolean
) {
  if (constructorOrPrototype.prototype) {
    // constructor, means static method
    if (!constructorOrPrototype[queueSymbol]) {
      constructorOrPrototype[queueSymbol] = new Map<
        string | symbol,
        TaskQueue
      >()
    }
  } else {
    // prototype, means instance method
    if (constructorOrPrototype[queueSymbol] !== null) {
      constructorOrPrototype[queueSymbol] = null
    }
  }

  // ..
}

inject queue, also using a symbol to avoid conflict

method decorators

class decorator

export function QConcurrency(
  queuesParam: Array<{
    limit: number
    tag?: string | symbol
    isStatic?: boolean
  }>
): ClassDecorator {
  // .. 
  
  /** target is constructor */
  return (target) => {
    queuesParam.forEach((queueParam) => {
      if (isStatic) {        
        // ** check if static method decorated applied
        
        /** inject concurrency info for each tag in static method case */
        if (target[concurrentSymbol]?.[usedTag]) {
          target[concurrentSymbol][usedTag] = limit
        }
      } else {
        //** check if instance method decorated applied
        
        /** inject concurrency info for each tag in instance method case */
        if (target.prototype[concurrentSymbol]?.[usedTag]) {
          target.prototype[concurrentSymbol][usedTag] = limit
        }
      }
    })
  }
}

task queue implementation

return async function (...args: any[]): Promise<any> {
  // ..
  
  /** Assign queues */
  let taskQueue: TaskQueue
  let currTaskQueues: TaskQueuesType
  if (d4cObj) {
    /** D4C instance case */
    currTaskQueues = d4cObj.queues
  } else if (this && (this[queueSymbol] || this[queueSymbol] === null)) {
    if (this[queueSymbol] === null) {
      /** Decorator instance method first time case, using injected queues 
       * in user defined objects*/
      this[queueSymbol] = new Map<string | symbol, TaskQueue>()
    }

    currTaskQueues = this[queueSymbol]
    decoratorConcurrencyLimit = this[concurrentSymbol]?.[tag]
  } else {
    throw new Error(ErrMsg.MissingThisDueBindIssue)
  }

  /** Get sub-queue */
  taskQueue = currTaskQueues.get(tag)
 
  // .. 
}

D4C instance vs your classes

case1: queue is closure variable

case2: instance method decorator

case3: static method decorator

already injects queue before

task queue implementation

core algorithm

    /** Detect if the queue is running or not, use promise to wait it if it is running */
    let result
    let err: Error
    let task: Task
    if (taskQueue.runningTask === taskQueue.concurrency) {
      const promise = new Promise(function (resolve) {
        task = {
          unlock: resolve,
          preError: null,
          inheritPreErr: option?.inheritPreErr,
        }
      })
      taskQueue.queue.push(task)
      await promise
      taskQueue.runningTask += 1
    } else if (option?.noBlockCurr) {
      taskQueue.runningTask += 1
      await Promise.resolve()
    } else {
      taskQueue.runningTask += 1
    }

    /** Run the task */
    if (task?.preError) {
      err = new PreviousTaskError(task.preError.message ?? task.preError)
    } else {
      try {
        /** this will be constructor function for static method case */
        const value = func.apply(this, args)

        /** Detect if it is a async/promise function or not */
        if (value && typeof value.then === 'function') {
          result = await value
        } else {
          result = value
        }
      } catch (error) {
        err = error
      }
    }
    taskQueue.runningTask -= 1

    /** After the task is executed, check the following tasks */
    if (taskQueue.queue.length > 0) {
      const nextTask: Task = taskQueue.queue.shift()
      /** Pass error to next task */
      if (err && nextTask.inheritPreErr) {
        nextTask.preError = err
      }
      nextTask.unlock()
    }

    if (err) {
      throw err
    }

    return result
  } as (...args: Parameters<typeof func>) => Promise<UnwrapPromise<typeof func>>

Thank you!

Q & A time

What is synchronization & concurrency in JavaScript - Use a TypeScript task queue library to introduce

Outline

d4c-queue task queue lib features

Key: concurrent !!

What is async in JavaScript

What is concurrency in JavaScript

Concurrency is good

Network thread may wait for more than one response at the same time. Performant built-in network ability.

But some issues you may encounter and a task queue lib can solve

Concurrency

rate limit

causality

race condition

Convenience

Rate limit:

Causality

E.g. initConnect & sendMessage while using a client

Real world case:

Race condition

sometimes two tasks would access the same resources that are not atomic. Use Database transaction or this d4c-queue lib to lock

Real world case:

Convenience

More introduction to d4c-queue

Points of code implementation

FIFO O(1) queue class

method decorators

method decorators

class decorator

task queue implementation

D4C instance vs your classes

task queue implementation

core algorithm

Thank you!