The trickiest

of the project
Artem Arkhipov
Lead Web Developer at Techmagic
@ar_arkhipov
artem.arkhipov@techmagic.co

.cloud
.cloud

What is
?




.cloud

What is
?



.cloud

IS NOT about drawing only








.cloud

main technology stack:






And few third party services...
So what about tricky element??
Transform this:

To this:

The way of thinking
1. Do it on client side and send directly to S3
- Not reliable
- Browser support
- Performance is not stable
2. Use jGraph server ( Java )
- Support of some weird and old things
- We have a lot of custom stylings etc.
- Objects content is generated by angular
3. We should find serverless solution (FaaS)
- We need scalability to cope with load
- We are experienced with AWS Lambda
- Serverless concept is so cool
The way of thinking
4. Send diagram HTML and render it to image with PhantomJS or similar tools
- Not reliable
- Potentially huge requests
5. Load whole Diagram Editor in PhantomJS or Selenium and capture screenshot
- Seems to be OK
- ACTUALLY NO







But finally I met him...

Headless Chrome
- Image is a perfect copy
- High performance, it takes about 2-3 sec for 1 image
- Reliable


Next steps
Write main handler functions:
Run headless chrome inside AWS Lambda
- Launch chrome as child process
- Open virtual tabs with diagrams
- Wait till diagrams are loaded
- Capture screenshots
- Save screenshots to AWS S3
- Close tabs
- Trigger image-resizing Lambda via SNS
Create lighter Diagram Editor version
Make "delayed lambda invocation" mechanism
const CHROME_URL = 'http://127.0.0.1:9222';
const CHROME_FLAGS = [
'--headless',
'--disable-gpu',
'--remote-debugging-port=9222',
'--window-size=1200,800',
'--no-sandbox',
'--user-data-dir=/tmp/user-data',
'--hide-scrollbars',
'--enable-logging',
'--log-level=0',
'--v=99',
'--single-process',
'--data-path=/tmp/data-path',
'--ignore-certificate-errors',
'--homedir=/tmp',
'--disk-cache-dir=/tmp/cache-dir'
];Some examples
const spawn = require('child_process').spawn;
let chrome;
function spawnChrome() {
console.log('Spawning Chrome...');
chrome = spawn(__dirname+'/headless-chrome/headless_shell',
config.CHROME_FLAGS, {
cwd: '/tmp',
shell: true,
detached: true,
stdio: 'ignore'
});
chrome.unref();
chrome.on('close', (code) => {
console.log('Chrome closed with code: ', code);
chrome = null;
spawnChrome();
});
}Run chrome
Open tab & connect
const CDP = require('chrome-remote-interface');
function openTab() {
return CDP.New().then((tab) => { //create tab
return CDP({tab}).then((client) => { //create client
return Promise.all([
client.Page.enable(),
client.Runtime.enable()
]).then(() => {
return Promise.resolve({client}); //ready
})
})
})
}Navigate to page
const Promise = require('bluebird');
function navigate({url}) {
return new Promise((resolve, reject) => {
// setup any events or conditions
client.Page.loadEventFired(() => resolve())
//run navigation to page
client.Page.navigate({url});
}).timeout(config.LOAD_TIMEOUT);
}Capture and close
function capture({client}) {
return client.Page.captureScreenshot({
fromSurface: true,
format: 'png'
}).then((screenshot) => {
client.close();
const image = screenshot.data;
return Promise.resolve({image});
})
}function closeTabs() {
console.log('Cleaning opened tabs...');
return CDP.List().then((tabs) => {
return Promise.all(tabs.map(({id}) => CDP.Close({id})))
})
}High level overview


Delayed Invocation

Conclusion
Scalable and reliable
Execution time about 1 - 2.5s
160 000 images cost less than ~10$ (AWS free tier is not included!)
Artem Arkhipov
Lead Web Developer at Techmagic
@ar_arkhipov
ar.arkhipov@gmail.com
Thank You!
artem.arkhipov@techmagic.co
The trickiest element of the project
By Artem Arkhipov
The trickiest element of the project
Scalable and reliable serverless solution. Running headless chrome on AWS Lambda to regullary create screenshots.
- 589