Demystifying

UI Code-Generation

or how to convert a UI design into Code!

Wassim Chegham //

<div class="xly_d1nz" role="group" aria-label="dialog">
  <div class="xly_inli">
    <svg width="321.00" height="557.00">
      <path stroke-width="0.5" stroke="#979797ff" fill="rgba(250,250,250,1.00)" d="M320.5 556.5,S 80.356 695.249, 0.5 556.5,S -79.789 139.5, 0.5 0.5,S 240.642 -138.623, 320.5 0.5,z"/>
    </svg>
  </div>
  <div class="xly_yp48" role="group" aria-label="ok + cancel">
    <span class="xly_x8wz">OK</span>
    <span class="xly_plfx">CANCEL</span>
  </div>
  <div class="xly_i8k7" role="group" aria-label="header">
    <div class="xly_4p0d">
      <svg width="320.00" height="199.00">
        <path fill="rgba(0,150,136,1.00)" d="M0 199,S -79.19 49.556, 0 0,S 318.117 -0.012, 320 0,L 320 199,z"/>
      </svg>
    </div>
    <div class="xly_xgfy">
      <svg width="320.00" height="32.00">
        <path fill="rgba(0,137,123,1.00)" d="M0 32,S -78.999 7.95, 0 0,S 239.748 -8.025, 320 0,S 402.581 24.258, 320 32,z"/>
      </svg>
    </div>
    <span class="xly_7h86">13</span>
    <span class="xly_dmx6">Friday</span>
    <span class="xly_2i2a">MAR</span>
    <span class="xly_mqsi">2014</span>
  </div>
  ...

@manekinekko

What if?

@manekinekko

design.sketch

design.html

@manekinekko

Microsoft

Sr. Developer Advocate (JavaScript)

WassimChegham

@manekinekko

@wassimchegham

https://wassim.dev

Open Source Creator

More at wassim.dev

Open Source Contributor

Angular GDE (core team alumni.)

Bazel Web Team contributor

GDE for the Google Assistant

Angular Universal (original core team alumni.)

GDE for GCP (alumni.)

Auth0 Ambassador

Member of the Node.js org. (OpenJS Foundation)

NestJS contributor

Compodoc core team

Compiler Design 101

@manekinekko

A Compiler is a tool that converts a program written in one language into another language or binary.

@manekinekko

2 types of compilers

@manekinekko

@manekinekko

Lexical Analysis

( e.g. TypeScript compiler )

@manekinekko

Scanner

LetKeyword

Identifier

EqualsToken

NumericLiteral

PlusToken

SemicolonToken

Lexical Analysis

NumericLiteral

( e.g. TypeScript compiler )

@manekinekko

Concrete Syntax Tree (parse tree)

( e.g. TypeScript compiler )

@manekinekko

How does the scanner validate tokens?

@manekinekko

Regular Grammar

@manekinekko

A formal set of Mathematical rules defined by
Noam Chomsky, that describes formal languages, known as regular languages.

G = (N, T, P, S)

Non terminal symbols

Terminal symbols

Production rules

Initial state

Regular Grammar (Type 3)

@manekinekko

S → A B;
A → let | ε
B → D = F
D → x
F → G + G
G → 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

let x = 1 + 2;
let x = 6 + 5;
let x = 2 + 2;
x = 1 + 2;

var x = 1 + 5;
let x = x + 5;
let 9 = 8 + 9;
let x = 9 + let;
let x = 2;

Invalid

N = \{A,B,D,F,G\}

T = \{let, ;, x, =, +, 0...9\}

S = \{A,B\}

P =

L(G) =

Syntax Analysis (parsing)

@manekinekko

Context-free Grammar (Type 2)

@manekinekko

A CFG is a set of recursive rules used to generate patterns of strings.

( e.g. ECMAScript Grammar )

@manekinekko

Is this a valid JavaScript construction?

[,[,,,,[],,,,],]

Context-free Grammar (Type 2)

( e.g. ECMAScript Grammar )

//...
ArrayLiteral:
	[ Elision   ]
	[ ElementList ]
	[ ElementList, Elision  ]

ElementList:
	Elision   AssignmentExpression
	Elision   SpreadElement
	ElementList, Elision   AssignmentExpression
	ElementList, Elision   SpreadElement

Elision:
	,
	Elision,
//...

( e.g. ECMAScript Grammar )

[]
[1]
[[]]
[1, 2]
[1 + 2]
[1, ...[2]]
[1,,,,,,,2]
[,[,,,,[],,,,],]

@manekinekko

opt

opt

opt

opt

opt

opt

Context-free Grammar (Type 2)

Error handling (example)

SyntaxError: Unexpected token '}'
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.Module._load (internal/modules/cjs/loader.js:727:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11

[1, 2, 3};
        ^

@manekinekko

[1, 2, 3];

[1, 2, 3};

Lexical + Syntax Analysis

@manekinekko

Semantic Analysis

"Semantic analysis is a process in compiler construction, usually after parsing, to gather necessary semantic information from the source code. It usually includes type checking, or makes sure a variable is declared before use."

Wikipedia:

@manekinekko

Abstract Syntax Tree (AST)

@manekinekko

A simplified version of parse tree

( e.g. TypeScript )

@manekinekko

Intermediate Code Optimisation

Constant folding

Type Checking

Value range propagation
Sparse conditional constant propagation
Dead code elimination
Global value numbering
Partial redundancy elimination
Strength reduction
Register allocation

Inlining Expansion

Static Single Assignment (SSA)

@manekinekko

Intermediate Code Optimisation

e.g. constant folding

@manekinekko

Intermediate Code Optimisation

Variable renaming (name mangling)

@manekinekko

Code Generation

e.g. transpiling to ES5

@manekinekko

Code Optimisation

@manekinekko

UI Code Generation

Real world example...

@manekinekko

Our goal...

@manekinekko

design.sketch

design.html

<div class="xly_d1nz" role="group" aria-label="dialog">
  <div class="xly_inli">
    <svg width="321.00" height="557.00">
      <path stroke-width="0.5" stroke="#979797ff" fill="rgba(250,250,250,1.00)" d="M320.5 556.5,S 80.356 695.249, 0.5 556.5,S -79.789 139.5, 0.5 0.5,S 240.642 -138.623, 320.5 0.5,z"/>
    </svg>
    ...

design.sketch

Compiler

design.html

Our goal...

@manekinekko

Reverse engineering *.sketch files

$ unzip design.sketch

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00  bplist00........
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00  ................
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69  ...g?..g@X$versi
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63  onX$objectsY$arc
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af  hiverT$top......
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00  ................
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00  ......"...M...N.
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00  ..e...j...n.....
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00  ................
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00  ................
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00  ................
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00  ................
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00  ................
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00  ................
<trimmed>
6c 38 00 29 6c 45 00 29 6c 4e 00 29 6c 5f 00 29  l8.)lE.)lN.)l_.)
6c 6e 00 29 6c 7b 00 29 6c 8a 00 29 6c 93 00 29  ln.)l{.)l..)l..)
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00  l..)l..)l.......
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00  ........gC......
00 00 00 00 00 00 00 29 6c ae                    .......)l.

.
├── Data
├── Images
│   ├── 3.png
│   └── 6.png
├── QuickLook
│   ├── Preview.png
│   └── Thumbnail.png
├── fonts
└── version

@manekinekko

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae

Decoding bplist files

HEADER
  magic number ("bplist")
  file format version (currently "0?")
OBJECT TABLE
  variable-sized objects
  Object Formats (marker byte followed by additional info in some cases)
  null    0000 0000
  bool    0000 1000                             // false
  bool    0000 1001                             // true
  fill    0000 1111                             // fill byte
  int     0001 nnnn	...                     // # of bytes is 2^nnnn, big-endian bytes
  real    0010 nnnn	...                     // # of bytes is 2^nnnn, big-endian bytes
  date    0011 0011	...                     // 8 byte float follows, big-endian bytes
  data    0100 nnnn	[int]	...             // nnnn is number of bytes unless 1111 then int count follows, followed by bytes
  string  0101 nnnn	[int]	...             // ASCII string, nnnn is # of chars, else 1111 then int count, then bytes
  string  0110 nnnn	[int]	...             // Unicode string, nnnn is # of chars, else 1111 then int count, then big-endian 2-byte uint16_t
          0111 xxxx                             // unused
  uid     1000 nnnn	...                     // nnnn+1 is # of bytes
          1001 xxxx                             // unused
  array   1010 nnnn	[int]	objref*	        // nnnn is count, unless '1111', then int count follows
          1011 xxxx                             // unused
  ser     1100 nnnn	[int]	objref*         // nnnn is count, unless '1111', then int count follows
  dict    1101 nnnn	[int]	keyref* objref*	// nnnn is count, unless '1111', then int count follows
          1110 xxxx                             // unused
          1111 xxxx                             // unused
OFFSET TABLE
  list of ints, byte size of which is given in trailer
  -- these are the byte offsets into the file
  -- number of these is in the trailer
TRAILER
  byte size of offset ints in offset table
  byte size of object refs in arrays and dicts
  number of offsets in offset table (also is number of objects)
  element # in offset table which is top level object
  offset table offset

DATA

@manekinekko

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae

{
  "frame": {
    "_class": "rect",
    "constrainProportions": false,
    "height": 224,
    "width": 344,
    "x": 0,
    "y": 0
  },
  "style": {
    "_class": "style",
    "endMarkerType": 0,
    "miterLimit": 10,
    "startMarkerType": 0,
    "windingRule": 1,
    "blur": {
      "_class": "blur",
      "isEnabled": false,
      "center": "{0.5, 0.5}",
      "motionAngle": 0,
      "radius": 10,
      "saturation": 1,
      "type": 0
    },
    ...

Parsing bplist files

@manekinekko

{
  ...
  "NSColor": {
    "_archive": "YnBsaXN0MDDUAQIDBAUGHyBYJHZlcnNpb25YJG9iamVjdHNZJGFyY2hpdmVyVCR0b3ASAAGGoKUHCBEVHFUkbnVsbNQJCgsMDQ4PEFVOU1JHQlxOU0NvbG9yU3BhY2VfEBJOU0N1c3RvbUNvbG9yU3BhY2VWJGNsYXNzRjAgMCAwABABgAKABNISDBMUVE5TSUQQAYAD0hYXGBlaJGNsYXNzbmFtZVgkY2xhc3Nlc1xOU0NvbG9yU3BhY2WiGhtcTlNDb2xvclNwYWNlWE5TT2JqZWN00hYXHR5XTlNDb2xvcqIdG18QD05TS2V5ZWRBcmNoaXZlctEhIlRyb290gAEACAARABoAIwAtADIANwA9AEMATABSAF8AdAB7AIIAhACGAIgAjQCSAJQAlgCbAKYArwC8AL8AzADVANoA4gDlAPcA+gD\/AAAAAAAAAgEAAAAAAAAAIwAAAAAAAAAAAAAAAAAAAQE="
  },
}

Parsing bplist files!!

@manekinekko

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae

{
  ...
  "NSColor": {
    "_archive": "YnBsaXN0MDDUAQIDBAUGHyBYJHZlcnNpb25YJG9iamVjdHNZJGFyY2hpdmVyVCR0b3ASAAGGoKUHCBEVHFUkbnVsbNQJCgsMDQ4PEFVOU1JHQlxOU0NvbG9yU3BhY2VfEBJOU0N1c3RvbUNvbG9yU3BhY2VWJGNsYXNzRjAgMCAwABABgAKABNISDBMUVE5TSUQQAYAD0hYXGBlaJGNsYXNzbmFtZVgkY2xhc3Nlc1xOU0NvbG9yU3BhY2WiGhtcTlNDb2xvclNwYWNlWE5TT2JqZWN00hYXHR5XTlNDb2xvcqIdG18QD05TS2V5ZWRBcmNoaXZlctEhIlRyb290gAEACAARABoAIwAtADIANwA9AEMATABSAF8AdAB7AIIAhACGAIgAjQCSAJQAlgCbAKYArwC8AL8AzADVANoA4gDlAPcA+gD\/AAAAAAAAAgEAAAAAAAAAIwAAAAAAAAAAAAAAAAAAAQE="
  },
}

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae

Parsing bplist files!!

@manekinekko

bplist

Parsing bplist files

@manekinekko

New Sketch file Format (43+)

@manekinekko

New Sketch file Format (43+)

$ unzip design.sketch

{
  "frame": {
    "_class": "rect",
    "constrainProportions": false,
    "height": 224,
    "width": 344,
    "x": 0,
    "y": 0
  },
  "style": {
    "_class": "style",
    "endMarkerType": 0,
    "miterLimit": 10,
    "startMarkerType": 0,
    "windingRule": 1,
    "blur": {
      "_class": "blur",
      "isEnabled": false,
      "center": "{0.5, 0.5}",
      "motionAngle": 0,
      "radius": 10,
      "saturation": 1,
      "type": 0
    },
    ...

.
├── document.json
├── meta.json
├── pages
│   └── <UUID>.json
├── previews
│   └── preview.png
├── text-previews
│   ├── text-previews-metadata.json
│   └── text-previews.pdf
└── user.json

@manekinekko

New Sketch file Format (43+)

@manekinekko

CSS compilation pipeline

* CSS Parser & optimiser only!

@manekinekko

CSS parser (DFS)

Leftmost Depth-Fist-Search

Advantages:

* Lower memory usage

* High performance

* Easy to implement

@manekinekko

CSS Tree

@manekinekko

CSS Optimiser

@manekinekko

Generating code from AST

@manekinekko

Can we do more?

@manekinekko

WassimChegham

@manekinekko

@wassimchegham

Demystifying UI Code Generation

By Wassim Chegham

Demystifying UI Code Generation

Wassim Chegham PRO

Senior Developer Advocate @Microsoft ★ Angular contributor ★ Bazel contributor ★ GDE @Google ★ creator of @itsjustangular / hueaction.dev / hexa.run / ngx.tools / xlayers.dev / hexa.run / autocap.cc / angular.run / nitr.ooo / async-await.xyz

Loading comments...

Demystifying

UI Code-Generation

What if?

Open Source Creator

Open Source Contributor

Compiler Design 101

A Compiler is a tool that converts a program written in one language into another language or binary.

2 types of compilers

Lexical Analysis

Lexical Analysis

Concrete Syntax Tree (parse tree)

How does the scanner validate tokens?

Regular Grammar

Regular Grammar (Type 3)

Syntax Analysis (parsing)

Context-free Grammar (Type 2)

Context-free Grammar (Type 2)

Context-free Grammar (Type 2)

Error handling (example)

Lexical + Syntax Analysis

Semantic Analysis

Abstract Syntax Tree (AST)

Intermediate Code Optimisation

Intermediate Code Optimisation

Intermediate Code Optimisation

Code Generation

Code Optimisation

UI Code Generation

Real world example...

Our goal...

Our goal...

Reverse engineering *.sketch files

Decoding bplist files

Parsing bplist files

Parsing bplist files!!

Parsing bplist files!!

Parsing bplist files

New Sketch file Format (43+)

New Sketch file Format (43+)

New Sketch file Format (43+)

CSS compilation pipeline

CSS parser (DFS)

CSS Tree

CSS Optimiser

Generating code from AST

Can we do more?

Demystifying UI Code Generation

More from Wassim Chegham