Preoccupy the user's tab
Artsiom Mezin
@iketari
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285506/kissclipart-logo-twitter-facebook-linkedin-clipart-computer-ic-55da96f19a559ec1.png)
About me
A Senior front-end dev at Behavox.
Have been doing web for over 10 years.
Always curious.
![](https://media0.giphy.com/media/VpiVc57C6U9UI/giphy.gif)
What is it about?
It is about a way of implementing a remote access
![](https://media3.giphy.com/media/Mui0OSVNTpj7W/giphy.gif)
Why do we need it?
-
To support
-
To collaborate
-
To report bugs and issues
-
To teach and to learn (theoretically)
-
… could be anything else what you can do toghter
Nothing new. It exists.
![](https://microsoft.github.io/AzureTipsAndTricks/files/azuretip9.gif)
Yes, but...
-
Access is granted not just to an app but the entire system.
-
Users have to install a client part of the solution.
-
An operator has to install a host program on his side.
![](https://media3.giphy.com/media/3otWpCV7P13ZL9doU8/giphy.gif)
The Web can help
-
Browsers sandbox is isolated.
-
Web apps do not need to be installed.
-
The host also could be implemented as a web app.
![](https://media0.giphy.com/media/oOOX6ZrRPiNFe/giphy.gif)
Technological questions
-
How to share the user’s screen?
-
How to perform an event on the user's side and how to transfer it to there?
-
How to install the client part so that it would work within any web application?
![](https://media3.giphy.com/media/mNU4UAvgU6teo/giphy.gif)
How to share the user’s screen?
-
HTML2Canvas -> canvas.captureStream(frameRate)
-
Do not share a screen, share the DOM
-
navigator.getDisplayMedia() + WebRTC
HTML to Canvas
console.time('HTMLToCanvas');
html2canvas(document.querySelector('body')).then(canvas => {
let img = canvas.toDataURL("image/png");
document.body.append('<img id="result" src="'+img+'"/>');
console.timeEnd('HTMLToCanvas');
});
// HTMLToCanvas: 26687.60693359375ms / 0.0374 FPS
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285012/pasted-from-clipboard.png)
TOGETHERJS
![](https://togetherjs.com/images/togetherjs-logo@2x.png)
![](https://togetherjs.com/images/togetherjs-how-03-collaborate.png)
Title Text
MediaDevices.getDisplayMedia()
let promise = navigator.mediaDevices.getDisplayMedia(constraints);
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285521/pasted-from-clipboard.png)
Title Text
MediaDevices.getDisplayMedia()
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285509/pasted-from-clipboard.png)
Title Text
WebRTC
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285557/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285571/pasted-from-clipboard.png)
1. No plugins
2. P2P
3. Audio, Video and Data all encrypted in transit
Title Text
WebRTC. Signalling.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285557/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285581/pasted-from-clipboard.png)
Title Text
Signalling. AWS Lambda
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285594/pasted-from-clipboard.png)
![](https://d2908q01vomqb2.cloudfront.net/1b6453892473a467d07372d45eb05abc2031647a/2018/12/18/websockets-arch.png)
Title Text
Lambda. OnConnect
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285594/pasted-from-clipboard.png)
// Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: MIT-0
var AWS = require("aws-sdk");
AWS.config.update({ region: process.env.AWS_REGION });
var DDB = new AWS.DynamoDB({ apiVersion: "2012-10-08" });
exports.handler = function (event, context, callback) {
var putParams = {
TableName: process.env.TABLE_NAME,
Item: {
connectionId: { S: event.requestContext.connectionId }
}
};
DDB.putItem(putParams, function (err) {
callback(null, {
statusCode: err ? 500 : 200,
body: err ? "Failed to connect: " + JSON.stringify(err) : "Connected."
});
});
};
Title Text
Lambda. OnMessage
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285594/pasted-from-clipboard.png)
exports.handler = async (event, context) => {
const apigwManagementApi = new AWS.ApiGatewayManagementApi({
apiVersion: '2018-11-29',
endpoint: event.requestContext.domainName + '/' + event.requestContext.stage
});
const { data } = JSON.parse(event.body);
switch (data.action) {
case 'register':
await registerController(data, event, ddb, apigwManagementApi);
break;
case 'communicate':
await communicateController(data, event, ddb, apigwManagementApi);
break;
default:
break;
}
}
Title Text
All together
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285594/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6285557/pasted-from-clipboard.png)
Technological questions
-
How to share the user’s screen? -
How to perform an event on the user's side and how to transfer it to there?
-
How to install the client part so that it would work within any web application?
![](https://media1.giphy.com/media/CTX0ivSQbI78A/giphy.gif)
How to perform a desirable event on the user's side and how to transfer it there?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/6287080/pasted-from-clipboard.png)
![](https://cdn-images-1.medium.com/max/1600/1*HP8l7LMMt7Sh5UoO1T-yLQ.png)
Preoccupy
What happens when a user clicks on some element on the page?
-
The element is being located.
-
A JavaScript event is being fired.
-
A standard action is happening.
PreoccupyJS is trying to reproduce browser's behaviour
-
Find an element
-
Fire an event
-
Make changes in DOM if necessary
let el = document.elementFromPoint(x, y);
// https://developer.mozilla.org/en-US/docs/Web/API/DocumentOrShadowRoot/elementFromPoint
el.dispatchEvent(new MouseEvent('click', opts));
if (document.activeElement !== null) {
this.fireEvent('blur', document.activeElement as HTMLElement);
}
if (el.focus) {
el.focus();
}
PreoccupyJS contains a number of replaceable Actions each one for a dedicated browser event
export class ClickToAction implements PreoccupyAction {
static type: string = ActionsName.CLICK_TO;
static eventName: string = 'click';
constructor(
public payload: {
x: number;
y: number;
}
) {
this.type = ClickToAction.type;
}
performEvent(dom: DomController, stack: PreoccupyAction[]) {
dom.clickTo(this.payload);
}
static handleEvent(host: Host, event: Event): PreoccupyAction {
const payload = host.getRelativeCoordinate(event as MouseEvent);
return new ClickToAction(payload);
}
}
PreoccupyJS handles events on the host side, serialize and passes them to a transport
import { Host, RxjsTransport } from 'preoccupyjs';
// fill free to style and modify this element as you wish, but keep it in the DOM!
const hostEl = document.createElement('div');
hostEl.tabIndex = 0; // make it focusable
const transport = new RxjsTransport(options as {
subject: new Subject<object>; //e.g. WebSocketSubject
});
const host = new Host(transport, hostEl);
host.start(); // run the communication whenever your app is ready!
Transport is responsible for delivering a serialized action to the client's side, where PreoccupyJS will perform it
import { Host, RxjsTransport, DomController } from 'preoccupyjs';
//Set up the transport
const transport = new RxjsTransport(options as {
subject: new Subject() // e.g. WebSocketSubject
});
// Set up and run the client
// It's possible to specify the controlled scope of the page by passing any DOM element
const client = new Client(transport, new DomController(document.body));
// run the communication whenever your app is ready!
client.start();
Technological questions
-
How to share the user’s screen? -
How to perform an event on the user's side and how to transfer it to there? -
How to install the client part so that it would work within any web application?
![](https://media2.giphy.com/media/7IqyTHR6jJ5h6/giphy.gif)
How to install the client part so that it would work within any web application?
![](https://media0.giphy.com/media/l41lZmsXhSLVKFE8U/giphy.gif)
Bookmarklets
(in theory)
javascript:(function(){
//Statements returning a non-undefined type, e.g. assignments
})();
<a href="{bookmarkletText}"> Bookmark me </a>
import bookmarklet from './assets/bookmarklet.bundle';
// via JSX
return (
<a href={bookmarklet}>
Click me, I am a bookmarklet.
</a>
);
// via vanilla JS
let link = document.createElement('a');
link.href = bookmarklet;
link.innerHTML = 'Click me, I am a bookmaklet';
return link;
Bookmarklets
(in practice)
Demo
![](https://media0.giphy.com/media/9i9fRaNhHfu92/giphy.gif)
Thank you! Any questions?
Please tell me what to improve or what to dig into next time!
Twitter: @iketari
![](https://s3.amazonaws.com/media-p.slid.es/uploads/15313/images/5207383/Thank-you-for-coming-GIF.gif)
Title Text
PreoccupyJS
By iketari
PreoccupyJS
- 192