What is synchronization & concurrency in JavaScript - Use a TypeScript task queue library to introduce
Outline
- Introduce what is async and concurrency in JavaScript
- Introduce why do we need a task queue library
- Introduce to more code usage
- Introduce the code implementation
- Q & A
d4c-queue task queue lib features
-
Wrap an async/promise-returning/sync function as a queue-ready async function for reusing
-
Sub queues system & extra config
-
TypeScript / JavaScript / Node.js / Browser
-
Two usages
-
D4C instance: synchronization & concurrency mode.
-
@synchronized & @concurrent decorators on classes
-
Key: concurrent !!
const d4c = new D4C([{ limit: 100 }]);
const fetchGitHub = d4c.wrap(async()=>{
return fetch('https://api.github.com/')
})
class TestController {
@synchronized
async fetchGitHub(url: string) {
return fetch('https://api.github.com/')
}
}
What is async in JavaScript
const fun3 = async () => {
// or trigger UI/Network action
console.log("fun3")
}
const fun1 = async () => {
console.log("fun1_start")
await fun3();
console.log('fun1_end');
};
function testAsync() {
fun1();
}
testAsync();
// fun1_start: execution 1 of event loop
// fun3
// ..
// .. may include other microtask,
// .. e.g. network msgs
// ..
// fun1_end: not execution 1 of event loop
-
macrotasks: setTimeout, setInterval, UI event, XMLHttpRequest ...
-
microtasks (=PromiseJobs in ECMAScript): promise (e.g. fetch.then) / async, queueMicrotask ...
T0
T1
May trigger some UI/Network actions
What is concurrency in JavaScript
const fun3 = async () => {
console.log("fun3")
}
const fun4 = async () => {
console.log("fun4")
}
const fun1 = async () => {
console.log("fun1_start")
await fun3();
console.log('fun1_end');
};
const fun2 = async () => {
console.log("fun2_start")
await fun4();
console.log('fun2_end');
};
function testConcurrency() {
fun1();
fun2();
}
testConcurrency();
// fun1_start
// fun3
// fun2_start
// fun4
// fun1_end
// fun2_end
fun2 is started as fun1 is not finished. They are not really executed simultaneously in main thread (a.k.a. parallelly). Concurrency: Multiple tasks are started but not finished within a time span.
T0
T1
T2
Concurrency is good
const promiseList = [];
for (let i=0; i<100; i++) {
promiseList.push(fetch('https://api.github.com/'));
}
const results = await Promise.all(promiseList);
-
Network thread may wait for more than one response at the same time. Performant built-in network ability.
But some issues you may encounter and a task queue lib can solve
-
Concurrency
-
rate limit
-
causality
-
race condition
-
-
Convenience
Rate limit:
just config d4c-queue limit parameter
const d4c = new D4C([{ limit: 10 }]);
const fetchGitHub = d4c.wrap(async()=>{
return fetch('https://api.github.com/')
})
const promiseList = [];
for (let i=0; i<100; i++) {
promiseList.push(fetchGitHub);
}
const results = await Promise.all(promiseList);
@QConcurrency([
{ limit: 10 },
])
class TestController {
@concurrent
async fetchGitHub(url: string) {}
}
const test = new TestController()
const promiseList = [];
for (let i=0; i<100; i++) {
promiseList.push(test.fetchGitHub);
}
const results = await Promise.all(promiseList);
way1: d4c instance
way2: decorator
alternatives:
- p-queue (fixed time interval)
- p-throttle w/ its strict mode (time interval between arbitrary two)
Causality
E.g. initConnect & sendMessage while using a client
sendMessage(msg: string) {
if (this.connectingStatus === 'Connected') {
// send message
} else if (this.connectingStatus === 'Connecting') {
// Um...how to wait for connecting successfully?
} else (this.connectingStatus === 'Disconnected') {
// try to re-connect
}
}
How?
class ServiceAdapter {
async send_message(msg: string) {
if (this.connectingStatus === 'Connected') {
/** send message */
await send_message_without_wait_connect(msg);
} else if (this.connectingStatus === 'Connecting') {
/** send message */
await send_message_wait_connect(msg);
} else {
//..
}
}
@synchronized
async initConnect() {
// ...
}
@synchronized
async send_message_wait_connect(msg: string) {
// ...
}
async send_message_without_wait_connect(msg: string) {
// ...
}
}
@synchronized is equal to Concurrency limit =1
Using d4c-queue, you can
Real world case:
code snippet is from embedded-pydicom-react-viewer which runs python in Browser. I'll give a talk in 2021 pycontw talk
const d4c = new D4C();
export const initPyodide = d4c.wrap(async () => {
/** init Pyodide*/
});
/** without d4c-queue, it will throw exception while being called
* before 'initPyodide' is finished */
export const parseByPython = d4c.wrap(async (buffer: ArrayBuffer) => {
/** execute python code in browser */
});
Race condition
sometimes two tasks would access the same resources that are not atomic. Use Database transaction or this d4c-queue lib to lock
const fun1 = async () => {
console.log("fun1_start")
await dbOperation();
console.log('fun1_end');
};
const fun2 = async () => {
console.log("fun2_start")
await dbOperation();
console.log('fun2_end');
};
/** Express case */
app.post('/testing', async (req, res) => {
// Do something here
});
/** Apollo server case */
const resolvers = {
Mutation: {
orderBook: async (_, { email, book }, { dataSources }) => {},
},
Query: {
books: async () => books,
},
};
two API callbacks will not be run concurrently
two API callbacks will be run concurrently
const d4c = new D4C();
/** Apollo server case */
const resolvers = {
Mutation: {
orderBook: d4c.wrap(async (_, { email, book }, { dataSources }) => {},
}),
Query: {
books: d4c.wrap(async () => books),
},
};
use d4.wrap
NestJS GraphQL synchronized resolver example with this d4c-queue
import { Query } from '@nestjs/graphql';
import { synchronized } from 'd4c-queue';
function delay() {
return new Promise<string>(function (resolve, reject) {
setTimeout(function () {
resolve('world');
}, 10 * 1000);
});
}
export class TestsResolver {
@Query((returns) => string)
/** without @synchronized, two resolver may print 1/2 1/2 2/2 2/2
* with @synchronized, it prints: 1/2 2/2 2/2 2/2
*/
@synchronized
async hello() {
console.log('hello graphql resolver part: 1/2');
const resp = await delay();
console.log('hello graphql resolver part: 2/2');
return resp;
}
}
Convenience
async wrap_function() {
await async_fun1()
await async_fun2()
}
async current_function() {
wrap_function()
}
current_function() {
const d4c = new D4C();
d4c.apply(async_fun1);
d4c.apply(async_fun2);
}
current_function() {
(async ()=>{
await async_fun1();
await async_fun2();
})();
}
OR
Target:
- start async_func2 after async_fun1 is finished
- do not want to wait for these two to finish, as they will block the current event loop
OR (use d4c-queue)
async current_function() {
// if we do not want to wait these two and
// continue the following codes, how?
await async_fun1()
await async_fun2()
// following code
// ..
}
OR (will force next microtask)
async current_function() {
queueMicrotask(async () => {
await async_fun1();
await async_fun2();
});
}
More introduction to d4c-queue
- pass sync function into a queue and treat it as async
const resp = await d4c.wrap(async()=>{
return fetch('https://github.com/')
})();
- Passing arguments and using await to get return values
- wrap or Apply
const result = await d4c.apply((x:string) => x+1, { args: [1] });
const resp = await d4c.wrap(()=>{
for (let i=0;i<1000000000;i++){
}
})();
executed in execution n
executed in execution n+m
- How to use on arrow function property
class TestController {
@autobind // way1
@synchronized // should be the second line
arrowFunctionProperty1(msg: string) {
}
arrowFunctionProperty2 = async () => {
};
}
// way2
const d4c = new D4C();
const res = await d4c.apply(testController.arrowFunctionProperty2);
-
Sub queue system
D4C queues (decorator) injected into your class:
- instance method queues (per instance):
- default queue
- tag1 queue
- tag2 queue
- static method queues
- default queue
- tag1 queue
- tag2 queue
D4C instance queues (per D4C object):
- default queue
- tag1 queue
- tag2 queue
class ServiceAdapter {
@synchronized({ tag: 'world', inheritPreErr: true, noBlockCurr: true })
static async staticMethod(text: string) {
return text;
}
}
- Static method example
- { inheritPreErr: true } task1 encounters fail task2 auto becomes fail
- setConcurrency
const d4c = new D4C();
d4c.setConcurrency([{ limit: 10 }]);
- { noBlockCurr: true } - task1 (pass sync) -> force another execution of event loop, makre sure it becomes async - task2 (pass async)
Points of code implementation
- generic FIFO queue class
- overloads method decorators:
- parentheses vs no parentheses
- static vs instance methods
- class decorator (QConcurrency):
- isStatic (static vs instance methods)
- task queue implementation
- D4C instance vs your classes
- core algorithm
FIFO O(1) queue class
export class Queue<T> {
private head: Node<T> | null = null;
private tail: Node<T> | null = null;
public length = 0;
public push(data: T) {
const node = new Node(data);
if (this.tail) {
this.tail.next = node;
node.prev = this.tail;
this.tail = node;
} else {
this.head = node;
this.tail = node;
}
this.length += 1;
return
}
public shift() {
if (this.length > 0) {
this.length -= 1;
const node = this.head;
if (node.next) {
this.head = node.next;
this.head.prev = null;
} else {
this.head = null;
this.tail = null;
}
return node.data;
}
return undefined;
}
}
method decorators
parentheses vs no parentheses
export function synchronized(
target: any,
propertyKey: string,
descriptor: PropertyDescriptor
): void
export function synchronized(option?: {
tag?: string | symbol
inheritPreErr?: boolean
noBlockCurr?: boolean
}): MethodDecoratorType
export function synchronized(
targetOrOption?: any,
propertyKey?: string,
descriptor?: PropertyDescriptor
): void | MethodDecoratorType {
return _methodDecorator(targetOrOption, propertyKey, descriptor, false)
}
function _methodDecorator(
targetOrOption: any,
propertyKey: string,
descriptor: PropertyDescriptor,
isConcurrent: boolean
): void | MethodDecoratorType {
if (checkIfDecoratorOptionObject(targetOrOption)) {
} else {
}
}
overload 1
overload 2
no parentheses case
parentheses case containing option
static vs instance methods
function injectQueue(
constructorOrPrototype,
tag: string | symbol,
isConcurrent: boolean
) {
if (constructorOrPrototype.prototype) {
// constructor, means static method
if (!constructorOrPrototype[queueSymbol]) {
constructorOrPrototype[queueSymbol] = new Map<
string | symbol,
TaskQueue
>()
}
} else {
// prototype, means instance method
if (constructorOrPrototype[queueSymbol] !== null) {
constructorOrPrototype[queueSymbol] = null
}
}
// ..
}
inject queue, also using a symbol to avoid conflict
method decorators
class decorator
export function QConcurrency(
queuesParam: Array<{
limit: number
tag?: string | symbol
isStatic?: boolean
}>
): ClassDecorator {
// ..
/** target is constructor */
return (target) => {
queuesParam.forEach((queueParam) => {
if (isStatic) {
// ** check if static method decorated applied
/** inject concurrency info for each tag in static method case */
if (target[concurrentSymbol]?.[usedTag]) {
target[concurrentSymbol][usedTag] = limit
}
} else {
//** check if instance method decorated applied
/** inject concurrency info for each tag in instance method case */
if (target.prototype[concurrentSymbol]?.[usedTag]) {
target.prototype[concurrentSymbol][usedTag] = limit
}
}
})
}
}
task queue implementation
return async function (...args: any[]): Promise<any> {
// ..
/** Assign queues */
let taskQueue: TaskQueue
let currTaskQueues: TaskQueuesType
if (d4cObj) {
/** D4C instance case */
currTaskQueues = d4cObj.queues
} else if (this && (this[queueSymbol] || this[queueSymbol] === null)) {
if (this[queueSymbol] === null) {
/** Decorator instance method first time case, using injected queues
* in user defined objects*/
this[queueSymbol] = new Map<string | symbol, TaskQueue>()
}
currTaskQueues = this[queueSymbol]
decoratorConcurrencyLimit = this[concurrentSymbol]?.[tag]
} else {
throw new Error(ErrMsg.MissingThisDueBindIssue)
}
/** Get sub-queue */
taskQueue = currTaskQueues.get(tag)
// ..
}
D4C instance vs your classes
case1: queue is closure variable
case2: instance method decorator
case3: static method decorator
already injects queue before
task queue implementation
core algorithm
/** Detect if the queue is running or not, use promise to wait it if it is running */
let result
let err: Error
let task: Task
if (taskQueue.runningTask === taskQueue.concurrency) {
const promise = new Promise(function (resolve) {
task = {
unlock: resolve,
preError: null,
inheritPreErr: option?.inheritPreErr,
}
})
taskQueue.queue.push(task)
await promise
taskQueue.runningTask += 1
} else if (option?.noBlockCurr) {
taskQueue.runningTask += 1
await Promise.resolve()
} else {
taskQueue.runningTask += 1
}
/** Run the task */
if (task?.preError) {
err = new PreviousTaskError(task.preError.message ?? task.preError)
} else {
try {
/** this will be constructor function for static method case */
const value = func.apply(this, args)
/** Detect if it is a async/promise function or not */
if (value && typeof value.then === 'function') {
result = await value
} else {
result = value
}
} catch (error) {
err = error
}
}
taskQueue.runningTask -= 1
/** After the task is executed, check the following tasks */
if (taskQueue.queue.length > 0) {
const nextTask: Task = taskQueue.queue.shift()
/** Pass error to next task */
if (err && nextTask.inheritPreErr) {
nextTask.preError = err
}
nextTask.unlock()
}
if (err) {
throw err
}
return result
} as (...args: Parameters<typeof func>) => Promise<UnwrapPromise<typeof func>>
Thank you!
Q & A time
COSCUP 2021 - Introduce TypeScript Task Queue - D4C
By Grimmer
COSCUP 2021 - Introduce TypeScript Task Queue - D4C
- 1,869