What is synchronization & concurrency in JavaScript - Use a TypeScript task queue library to introduce

2021.08.01@COSCUP in Taiwan

Grimmer

https://github.com/grimmer0125

video (Mandarin)

Outline

Introduce what is async and concurrency in JavaScript
Introduce why do we need a task queue library
Introduce to more code usage
Introduce the code implementation
Q & A

d4c-queue task queue lib features

Wrap an async/promise-returning/sync function as a queue-ready async function for reusing
Sub queues system & extra config
TypeScript / JavaScript / Node.js / Browser
Two usages
- D4C instance: synchronization & concurrency mode.
- @synchronized & @concurrent decorators on classes

Key: concurrent !!

const d4c = new D4C([{ limit: 100 }]);
const fetchGitHub = d4c.wrap(async()=>{
  return fetch('https://api.github.com/')
})

class TestController {
  @synchronized
  async fetchGitHub(url: string) {
    return fetch('https://api.github.com/')
  }
}

What is async in JavaScript

const fun3 = async () => {
  // or trigger UI/Network action
  console.log("fun3") 
}

const fun1 = async () => {
  console.log("fun1_start")
  await fun3(); 
  console.log('fun1_end');
};

function testAsync() {
  fun1();
}

testAsync();
// fun1_start: execution 1 of event loop
// fun3
// ..
// .. may include other microtask, 
// .. e.g. network msgs
// ..
// fun1_end: not execution 1 of event loop

macrotasks: setTimeout, setInterval, UI event, XMLHttpRequest ...
microtasks (=PromiseJobs in ECMAScript): promise (e.g. fetch.then) / async, queueMicrotask ...

May trigger some UI/Network actions

What is concurrency in JavaScript

const fun3 = async () => {
  console.log("fun3")
}

const fun4 = async () => {
  console.log("fun4")
}

const fun1 = async () => {
  console.log("fun1_start")
  await fun3();
  console.log('fun1_end');
};

const fun2 = async () => {
  console.log("fun2_start")
  await fun4();
  console.log('fun2_end');
};

function testConcurrency() {
  fun1();
  fun2();
}
testConcurrency();
// fun1_start
// fun3
// fun2_start
// fun4
// fun1_end
// fun2_end

fun2 is started as fun1 is not finished. They are not really executed simultaneously in main thread (a.k.a. parallelly). Concurrency: Multiple tasks are started but not finished within a time span.

Concurrency is good

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(fetch('https://api.github.com/'));
}
const results = await Promise.all(promiseList);

Network thread may wait for more than one response at the same time. Performant built-in network ability.

But some issues you may encounter and a task queue lib can solve

Concurrency
- rate limit
- causality
- race condition
Convenience

Rate limit:

just config d4c-queue limit parameter

const d4c = new D4C([{ limit: 10 }]);
const fetchGitHub = d4c.wrap(async()=>{
  return fetch('https://api.github.com/')
})

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(fetchGitHub);
}
const results = await Promise.all(promiseList);

@QConcurrency([
  { limit: 10 },
])
class TestController {
  @concurrent
  async fetchGitHub(url: string) {}
}
const test = new TestController()

const promiseList = [];
for (let i=0; i<100; i++) {
  promiseList.push(test.fetchGitHub);
}
const results = await Promise.all(promiseList);

way1: d4c instance

way2: decorator

alternatives:

p-queue (fixed time interval)
p-throttle w/ its strict mode (time interval between arbitrary two)

Causality

E.g. initConnect & sendMessage while using a client

sendMessage(msg: string) {
  if (this.connectingStatus === 'Connected') {
    // send message
  } else if (this.connectingStatus === 'Connecting') {
    // Um...how to wait for connecting successfully?
  } else (this.connectingStatus === 'Disconnected') {
    // try to re-connect
  }
}

How?

class ServiceAdapter {
  async send_message(msg: string) {
    if (this.connectingStatus === 'Connected') {
      /** send message */
      await send_message_without_wait_connect(msg);
    } else if (this.connectingStatus === 'Connecting') {
      /** send message */
      await send_message_wait_connect(msg);
    } else {
      //..
    }
  }

  @synchronized
  async initConnect() {
    // ...
  }

  @synchronized
  async send_message_wait_connect(msg: string) {
    // ...
  }

  async send_message_without_wait_connect(msg: string) {
    // ...
  }
}

@synchronized is equal to Concurrency limit =1

Using d4c-queue, you can

Real world case:

code snippet is from embedded-pydicom-react-viewer which runs python in Browser. I'll give a talk in 2021 pycontw talk

const d4c = new D4C();

export const initPyodide = d4c.wrap(async () => {
  /** init Pyodide*/
});

/** without d4c-queue, it will throw exception while being called
 * before 'initPyodide' is finished */
export const parseByPython = d4c.wrap(async (buffer: ArrayBuffer) => {
  /** execute python code in browser */
});

Race condition

sometimes two tasks would access the same resources that are not atomic. Use Database transaction or this d4c-queue lib to lock

const fun1 = async () => {
  console.log("fun1_start")
  await dbOperation();
  console.log('fun1_end');
};

const fun2 = async () => {
  console.log("fun2_start")
  await dbOperation();
  console.log('fun2_end');
};

Real world case:

Express routing vs Apollo/NestJS GraphQL resolver

/** Express case */
app.post('/testing', async (req, res) => {
  // Do something here
});

/** Apollo server case */
const resolvers = {
  Mutation: {
    orderBook: async (_, { email, book }, { dataSources }) => {},
  },
  Query: {
    books: async () => books,
  },
};

two API callbacks will not be run concurrently

two API callbacks will be run concurrently

const d4c = new D4C();

/** Apollo server case */
const resolvers = {
  Mutation: {
    orderBook: d4c.wrap(async (_, { email, book }, { dataSources }) => {},
  }),
  Query: {
    books: d4c.wrap(async () => books),
  },
};

use d4.wrap

NestJS GraphQL synchronized resolver example with this d4c-queue

import { Query } from '@nestjs/graphql';
import { synchronized } from 'd4c-queue';

function delay() {
  return new Promise<string>(function (resolve, reject) {
    setTimeout(function () {
      resolve('world');
    }, 10 * 1000);
  });
}

export class TestsResolver {
  @Query((returns) => string)
  /** without @synchronized, two resolver may print 1/2 1/2 2/2 2/2
   *  with @synchronized, it prints: 1/2 2/2 2/2 2/2
   */
  @synchronized
  async hello() {
    console.log('hello graphql resolver part: 1/2');
    const resp = await delay();
    console.log('hello graphql resolver part: 2/2');
    return resp;
  }
}

Convenience

async wrap_function() {
  await async_fun1()
  await async_fun2()
}

async current_function() {
  wrap_function()
}

current_function() {
  const d4c = new D4C();
  d4c.apply(async_fun1);
  d4c.apply(async_fun2);
}

current_function() {
  (async ()=>{
    await async_fun1();
    await async_fun2();
  })();
}

Target:

start async_func2 after async_fun1 is finished
do not want to wait for these two to finish, as they will block the current event loop

OR (use d4c-queue)

async current_function() {
  // if we do not want to wait these two and 
  // continue the following codes, how?
  await async_fun1()
  await async_fun2()
  
  // following code
  // ..
}

OR (will force next microtask)

async current_function() {
  queueMicrotask(async () => {
    await async_fun1();
    await async_fun2();
  });
}

More introduction to d4c-queue

pass sync function into a queue and treat it as async

const resp = await d4c.wrap(async()=>{
  return fetch('https://github.com/')
})();

Passing arguments and using await to get return values

wrap or Apply

const result = await d4c.apply((x:string) => x+1, { args: [1] });

const resp = await d4c.wrap(()=>{
  for (let i=0;i<1000000000;i++){    
  }
})();

executed in execution n

executed in execution n+m

How to use on arrow function property

class TestController {
  @autobind  // way1 
  @synchronized // should be the second line
  arrowFunctionProperty1(msg: string) {
  }

  arrowFunctionProperty2 = async () => {
  };
}
// way2
const d4c = new D4C(); 
const res = await d4c.apply(testController.arrowFunctionProperty2);

Sub queue system

D4C queues (decorator) injected into your class:
- instance method queues (per instance):
    - default queue
    - tag1 queue
    - tag2 queue
- static method queues
    - default queue
    - tag1 queue
    - tag2 queue

D4C instance queues (per D4C object):
- default queue
- tag1 queue
- tag2 queue

class ServiceAdapter {
  @synchronized({ tag: 'world', inheritPreErr: true, noBlockCurr: true })
  static async staticMethod(text: string) {
    return text;
  }
}

Static method example

{ inheritPreErr: true } task1 encounters fail task2 auto becomes fail

setConcurrency

const d4c = new D4C();
d4c.setConcurrency([{ limit: 10 }]);

{ noBlockCurr: true } - task1 (pass sync) -> force another execution of event loop, makre sure it becomes async - task2 (pass async)

Points of code implementation

generic FIFO queue class
overloads method decorators:
- parentheses vs no parentheses
- static vs instance methods
class decorator (QConcurrency):
- isStatic (static vs instance methods)
task queue implementation
- D4C instance vs your classes
- core algorithm

FIFO O(1) queue class

export class Queue<T> {
  private head: Node<T> | null = null;
  private tail: Node<T> | null = null;
  public length = 0;

  public push(data: T) {
    const node = new Node(data);
    if (this.tail) {
      this.tail.next = node;
      node.prev = this.tail;
      this.tail = node;
    } else {
      this.head = node;
      this.tail = node;
    }
    this.length += 1;
    return
  }

  public shift() {
    if (this.length > 0) {
      this.length -= 1;
      const node = this.head;
      if (node.next) {
        this.head = node.next;
        this.head.prev = null;
      } else {
        this.head = null;
        this.tail = null;
      }
      return node.data;
    }
    return undefined;
  }
}

method decorators

parentheses vs no parentheses

export function synchronized(
  target: any,
  propertyKey: string,
  descriptor: PropertyDescriptor
): void
export function synchronized(option?: {
  tag?: string | symbol
  inheritPreErr?: boolean
  noBlockCurr?: boolean
}): MethodDecoratorType
export function synchronized(
  targetOrOption?: any,
  propertyKey?: string,
  descriptor?: PropertyDescriptor
): void | MethodDecoratorType {
  return _methodDecorator(targetOrOption, propertyKey, descriptor, false)
}

function _methodDecorator(
  targetOrOption: any,
  propertyKey: string,
  descriptor: PropertyDescriptor,
  isConcurrent: boolean
): void | MethodDecoratorType {
  if (checkIfDecoratorOptionObject(targetOrOption)) {

  } else {

  }
}

overload 1

overload 2

no parentheses case

parentheses case containing option

static vs instance methods

function injectQueue(
  constructorOrPrototype,
  tag: string | symbol,
  isConcurrent: boolean
) {
  if (constructorOrPrototype.prototype) {
    // constructor, means static method
    if (!constructorOrPrototype[queueSymbol]) {
      constructorOrPrototype[queueSymbol] = new Map<
        string | symbol,
        TaskQueue
      >()
    }
  } else {
    // prototype, means instance method
    if (constructorOrPrototype[queueSymbol] !== null) {
      constructorOrPrototype[queueSymbol] = null
    }
  }

  // ..
}

inject queue, also using a symbol to avoid conflict

method decorators

class decorator

export function QConcurrency(
  queuesParam: Array<{
    limit: number
    tag?: string | symbol
    isStatic?: boolean
  }>
): ClassDecorator {
  // .. 
  
  /** target is constructor */
  return (target) => {
    queuesParam.forEach((queueParam) => {
      if (isStatic) {        
        // ** check if static method decorated applied
        
        /** inject concurrency info for each tag in static method case */
        if (target[concurrentSymbol]?.[usedTag]) {
          target[concurrentSymbol][usedTag] = limit
        }
      } else {
        //** check if instance method decorated applied
        
        /** inject concurrency info for each tag in instance method case */
        if (target.prototype[concurrentSymbol]?.[usedTag]) {
          target.prototype[concurrentSymbol][usedTag] = limit
        }
      }
    })
  }
}

task queue implementation

return async function (...args: any[]): Promise<any> {
  // ..
  
  /** Assign queues */
  let taskQueue: TaskQueue
  let currTaskQueues: TaskQueuesType
  if (d4cObj) {
    /** D4C instance case */
    currTaskQueues = d4cObj.queues
  } else if (this && (this[queueSymbol] || this[queueSymbol] === null)) {
    if (this[queueSymbol] === null) {
      /** Decorator instance method first time case, using injected queues 
       * in user defined objects*/
      this[queueSymbol] = new Map<string | symbol, TaskQueue>()
    }

    currTaskQueues = this[queueSymbol]
    decoratorConcurrencyLimit = this[concurrentSymbol]?.[tag]
  } else {
    throw new Error(ErrMsg.MissingThisDueBindIssue)
  }

  /** Get sub-queue */
  taskQueue = currTaskQueues.get(tag)
 
  // .. 
}

D4C instance vs your classes

case1: queue is closure variable

case2: instance method decorator

case3: static method decorator

already injects queue before

task queue implementation

core algorithm

    /** Detect if the queue is running or not, use promise to wait it if it is running */
    let result
    let err: Error
    let task: Task
    if (taskQueue.runningTask === taskQueue.concurrency) {
      const promise = new Promise(function (resolve) {
        task = {
          unlock: resolve,
          preError: null,
          inheritPreErr: option?.inheritPreErr,
        }
      })
      taskQueue.queue.push(task)
      await promise
      taskQueue.runningTask += 1
    } else if (option?.noBlockCurr) {
      taskQueue.runningTask += 1
      await Promise.resolve()
    } else {
      taskQueue.runningTask += 1
    }

    /** Run the task */
    if (task?.preError) {
      err = new PreviousTaskError(task.preError.message ?? task.preError)
    } else {
      try {
        /** this will be constructor function for static method case */
        const value = func.apply(this, args)

        /** Detect if it is a async/promise function or not */
        if (value && typeof value.then === 'function') {
          result = await value
        } else {
          result = value
        }
      } catch (error) {
        err = error
      }
    }
    taskQueue.runningTask -= 1

    /** After the task is executed, check the following tasks */
    if (taskQueue.queue.length > 0) {
      const nextTask: Task = taskQueue.queue.shift()
      /** Pass error to next task */
      if (err && nextTask.inheritPreErr) {
        nextTask.preError = err
      }
      nextTask.unlock()
    }

    if (err) {
      throw err
    }

    return result
  } as (...args: Parameters<typeof func>) => Promise<UnwrapPromise<typeof func>>

Thank you!

Q & A time

COSCUP 2021 - Introduce TypeScript Task Queue - D4C

By Grimmer

COSCUP 2021 - Introduce TypeScript Task Queue - D4C

1,919

Grimmer

github.com/grimmer0125

What is synchronization & concurrency in JavaScript - Use a TypeScript task queue library to introduce

Outline

d4c-queue task queue lib features

Key: concurrent !!

What is async in JavaScript

What is concurrency in JavaScript

Concurrency is good

Network thread may wait for more than one response at the same time. Performant built-in network ability.

But some issues you may encounter and a task queue lib can solve

Concurrency

rate limit

causality

race condition

Convenience

Rate limit:

Causality

E.g. initConnect & sendMessage while using a client

Real world case:

Race condition

sometimes two tasks would access the same resources that are not atomic. Use Database transaction or this d4c-queue lib to lock

Real world case:

Convenience

More introduction to d4c-queue

Points of code implementation

FIFO O(1) queue class

method decorators

method decorators

class decorator

task queue implementation

D4C instance vs your classes

task queue implementation

core algorithm

Thank you!

COSCUP 2021 - Introduce TypeScript Task Queue - D4C

More from Grimmer