The trickiest
![](https://media.slid.es/uploads/748315/images/4253573/elllemtn.png)
of the project
Artem Arkhipov
Lead Web Developer at Techmagic
@ar_arkhipov
artem.arkhipov@techmagic.co
![](https://media.slid.es/uploads/748315/images/4253793/noq9.png)
.cloud
.cloud
![](https://media.slid.es/uploads/748315/images/4253793/noq9.png)
What is
?
![](https://media.slid.es/uploads/748315/images/4254181/man-board.jpg)
![](https://media.slid.es/uploads/748315/images/4254186/manualgraph1.jpg)
![](https://media.slid.es/uploads/748315/images/4254195/manualgraph2.jpg)
![](https://media.slid.es/uploads/748315/images/4254229/docus.png)
.cloud
![](https://media.slid.es/uploads/748315/images/4253793/noq9.png)
What is
?
![](https://media.slid.es/uploads/748315/images/4254256/______________2017-10-23___15.56.44.png)
![](https://media.slid.es/uploads/748315/images/4254266/______________2017-10-23___15.57.08.png)
![](https://media.slid.es/uploads/748315/images/4254352/activity.png)
.cloud
![](https://media.slid.es/uploads/748315/images/4253793/noq9.png)
IS NOT about drawing only
![](https://media.slid.es/uploads/748315/images/4254314/______________2017-10-23___17.11.25.png)
![](https://media.slid.es/uploads/748315/images/4254312/______________2017-10-23___17.10.37.png)
![](https://media.slid.es/uploads/748315/images/4254311/______________2017-10-23___17.10.17.png)
![](https://media.slid.es/uploads/748315/images/4254317/______________2017-10-23___17.19.44.png)
![](https://media.slid.es/uploads/748315/images/4254315/______________2017-10-23___17.13.23.png)
![](https://media.slid.es/uploads/748315/images/4254313/______________2017-10-23___17.11.10.png)
![](https://media.slid.es/uploads/748315/images/4254343/______________2017-10-23___17.28.32.png)
![](https://upload.wikimedia.org/wikipedia/en/3/34/SFDC_logo.png)
.cloud
![](https://media.slid.es/uploads/748315/images/4253793/noq9.png)
main technology stack:
![](http://www.unixstickers.com/image/data/stickers/mongo/mongo.sh.png)
![](https://openfin.co/wp-content/uploads/2015/05/nodejs_logo.png)
![](http://www.newgeninfotech.com/images/courses/amazon-aws.png)
![](https://media.slid.es/uploads/748315/images/4064607/ezgif-4-5e05ca3976.gif)
![](http://sdtimes.com/wp-content/uploads/2015/12/1215.sdt-angularjs2a.png)
![](https://jgraph.github.io/mxgraph/docs/images/mxgraph_logo.gif)
And few third party services...
So what about tricky element??
Transform this:
![](https://media.slid.es/uploads/748315/images/4254560/______________2017-10-23___18.22.01.png)
To this:
![](https://media.slid.es/uploads/748315/images/4254575/______________2017-10-23___18.21.43.png)
The way of thinking
1. Do it on client side and send directly to S3
- Not reliable
- Browser support
- Performance is not stable
2. Use jGraph server ( Java )
- Support of some weird and old things
- We have a lot of custom stylings etc.
- Objects content is generated by angular
3. We should find serverless solution (FaaS)
- We need scalability to cope with load
- We are experienced with AWS Lambda
- Serverless concept is so cool
The way of thinking
4. Send diagram HTML and render it to image with PhantomJS or similar tools
- Not reliable
- Potentially huge requests
5. Load whole Diagram Editor in PhantomJS or Selenium and capture screenshot
- Seems to be OK
- ACTUALLY NO
![](https://media.slid.es/uploads/748315/images/4254993/______________2017-10-23___20.08.55.png)
![](https://media.slid.es/uploads/748315/images/4254992/______________2017-10-23___20.08.17.png)
![](https://media.slid.es/uploads/748315/images/4254991/______________2017-10-23___20.07.27.png)
![](https://media.slid.es/uploads/748315/images/4254990/______________2017-10-23___20.07.01.png)
![](https://media.slid.es/uploads/748315/images/4255027/______________2017-10-23___20.16.35.png)
![](https://media.slid.es/uploads/748315/images/4255028/______________2017-10-23___20.16.59.png)
![](http://i0.kym-cdn.com/photos/images/original/000/099/333/722.png)
But finally I met him...
![](https://media.slid.es/uploads/748315/images/4256948/headless_chrome.png)
Headless Chrome
- Image is a perfect copy
- High performance, it takes about 2-3 sec for 1 image
- Reliable
![](https://media.slid.es/uploads/748315/images/4257019/______________2017-10-24___11.26.58.png)
![](https://pbs.twimg.com/media/BvYztFCIMAAull8.png)
Next steps
Write main handler functions:
Run headless chrome inside AWS Lambda
- Launch chrome as child process
- Open virtual tabs with diagrams
- Wait till diagrams are loaded
- Capture screenshots
- Save screenshots to AWS S3
- Close tabs
- Trigger image-resizing Lambda via SNS
Create lighter Diagram Editor version
Make "delayed lambda invocation" mechanism
const CHROME_URL = 'http://127.0.0.1:9222';
const CHROME_FLAGS = [
'--headless',
'--disable-gpu',
'--remote-debugging-port=9222',
'--window-size=1200,800',
'--no-sandbox',
'--user-data-dir=/tmp/user-data',
'--hide-scrollbars',
'--enable-logging',
'--log-level=0',
'--v=99',
'--single-process',
'--data-path=/tmp/data-path',
'--ignore-certificate-errors',
'--homedir=/tmp',
'--disk-cache-dir=/tmp/cache-dir'
];
Some examples
const spawn = require('child_process').spawn;
let chrome;
function spawnChrome() {
console.log('Spawning Chrome...');
chrome = spawn(__dirname+'/headless-chrome/headless_shell',
config.CHROME_FLAGS, {
cwd: '/tmp',
shell: true,
detached: true,
stdio: 'ignore'
});
chrome.unref();
chrome.on('close', (code) => {
console.log('Chrome closed with code: ', code);
chrome = null;
spawnChrome();
});
}
Run chrome
Open tab & connect
const CDP = require('chrome-remote-interface');
function openTab() {
return CDP.New().then((tab) => { //create tab
return CDP({tab}).then((client) => { //create client
return Promise.all([
client.Page.enable(),
client.Runtime.enable()
]).then(() => {
return Promise.resolve({client}); //ready
})
})
})
}
Navigate to page
const Promise = require('bluebird');
function navigate({url}) {
return new Promise((resolve, reject) => {
// setup any events or conditions
client.Page.loadEventFired(() => resolve())
//run navigation to page
client.Page.navigate({url});
}).timeout(config.LOAD_TIMEOUT);
}
Capture and close
function capture({client}) {
return client.Page.captureScreenshot({
fromSurface: true,
format: 'png'
}).then((screenshot) => {
client.close();
const image = screenshot.data;
return Promise.resolve({image});
})
}
function closeTabs() {
console.log('Cleaning opened tabs...');
return CDP.List().then((tabs) => {
return Promise.all(tabs.map(({id}) => CDP.Close({id})))
})
}
High level overview
![](https://media.slid.es/uploads/748315/images/4258441/______________2017-10-24___18.17.02.png)
![](https://media.slid.es/uploads/748315/images/4258506/______________2017-10-24___18.26.35.png)
Delayed Invocation
![](https://media.slid.es/uploads/748315/images/4264538/______________2017-10-26___00.38.19.png)
Conclusion
Scalable and reliable
Execution time about 1 - 2.5s
160 000 images cost less than ~10$ (AWS free tier is not included!)
Artem Arkhipov
Lead Web Developer at Techmagic
@ar_arkhipov
ar.arkhipov@gmail.com
Thank You!
artem.arkhipov@techmagic.co
The trickiest element of the project
By Artem Arkhipov
The trickiest element of the project
Scalable and reliable serverless solution. Running headless chrome on AWS Lambda to regullary create screenshots.
- 487