Demystifying

UI Code-Generation

or how to convert a UI design into Code!

Wassim Chegham //

<div class="xly_d1nz" role="group" aria-label="dialog">
  <div class="xly_inli">
    <svg width="321.00" height="557.00">
      <path stroke-width="0.5" stroke="#979797ff" fill="rgba(250,250,250,1.00)" d="M320.5 556.5,S 80.356 695.249, 0.5 556.5,S -79.789 139.5, 0.5 0.5,S 240.642 -138.623, 320.5 0.5,z"/>
    </svg>
  </div>
  <div class="xly_yp48" role="group" aria-label="ok + cancel">
    <span class="xly_x8wz">OK</span>
    <span class="xly_plfx">CANCEL</span>
  </div>
  <div class="xly_i8k7" role="group" aria-label="header">
    <div class="xly_4p0d">
      <svg width="320.00" height="199.00">
        <path fill="rgba(0,150,136,1.00)" d="M0 199,S -79.19 49.556, 0 0,S 318.117 -0.012, 320 0,L 320 199,z"/>
      </svg>
    </div>
    <div class="xly_xgfy">
      <svg width="320.00" height="32.00">
        <path fill="rgba(0,137,123,1.00)" d="M0 32,S -78.999 7.95, 0 0,S 239.748 -8.025, 320 0,S 402.581 24.258, 320 32,z"/>
      </svg>
    </div>
    <span class="xly_7h86">13</span>
    <span class="xly_dmx6">Friday</span>
    <span class="xly_2i2a">MAR</span>
    <span class="xly_mqsi">2014</span>
  </div>
  ...

What if?

design.sketch

design.html

Microsoft

Sr. Developer Advocate (JavaScript)

WassimChegham

https://wassim.dev

Open Source Creator

More at wassim.dev

Open Source Contributor

Angular GDE (core team alumni.)

Bazel Web Team contributor

GDE for the Google Assistant

Angular Universal (original core team alumni.)

GDE for GCP (alumni.)

Auth0 Ambassador

Member of the Node.js org. (OpenJS Foundation)

NestJS contributor

Compodoc core team

Compiler Design 101

A Compiler is a tool that converts a program written in one language into another language or binary.

2 types of compilers

Lexical Analysis

( e.g. TypeScript compiler )

Scanner

LetKeyword

Identifier

EqualsToken

NumericLiteral

PlusToken

SemicolonToken

Lexical Analysis

NumericLiteral

( e.g. TypeScript compiler )

Concrete Syntax Tree (parse tree)

( e.g. TypeScript compiler )

How does the scanner validate tokens?

Regular Grammar

A formal set of Mathematical rules defined by
Noam Chomsky, that describes formal languages, known as regular languages.

G = (N, T, P, S)

Non terminal symbols

Terminal symbols

Production rules

Initial state

Regular Grammar (Type 3)

S → A B;
A → let | ε
B → D = F
D → x
F → G + G
G → 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
let x = 1 + 2;
let x = 6 + 5;
let x = 2 + 2;
x = 1 + 2;
var x = 1 + 5;
let x = x + 5;
let 9 = 8 + 9;
let x = 9 + let;
let x = 2;

Invalid

N = \{A,B,D,F,G\}
T = \{let, ;, x, =, +, 0...9\}
S = \{A,B\}
P =
L(G) =

Syntax Analysis (parsing)

Context-free Grammar (Type 2)

A CFG is a set of recursive rules used to generate patterns of strings.

( e.g. ECMAScript Grammar )

Is this a valid JavaScript construction?

[,[,,,,[],,,,],]

Context-free Grammar (Type 2)

( e.g. ECMAScript Grammar )

//...
ArrayLiteral:
	[ Elision   ]
	[ ElementList ]
	[ ElementList, Elision  ]

ElementList:
	Elision   AssignmentExpression
	Elision   SpreadElement
	ElementList, Elision   AssignmentExpression
	ElementList, Elision   SpreadElement

Elision:
	,
	Elision,
//...

( e.g. ECMAScript Grammar )

[]
[1]
[[]]
[1, 2]
[1 + 2]
[1, ...[2]]
[1,,,,,,,2]
[,[,,,,[],,,,],]
opt
opt
opt
opt
opt
opt

Context-free Grammar (Type 2)

Error handling (example)

SyntaxError: Unexpected token '}'
    at Module._compile (internal/modules/cjs/loader.js:895:18)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:995:10)
    at Module.load (internal/modules/cjs/loader.js:815:32)
    at Function.Module._load (internal/modules/cjs/loader.js:727:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1047:10)
    at internal/main/run_main_module.js:17:11
[1, 2, 3};
        ^
[1, 2, 3];
[1, 2, 3};

Lexical + Syntax Analysis

Semantic Analysis

"Semantic analysis is a process in compiler construction, usually after parsing, to gather necessary semantic information from the source code. It usually includes type checking, or makes sure a variable is declared before use."

Wikipedia:

Abstract Syntax Tree (AST)

A simplified version of parse tree

( e.g. TypeScript )

Intermediate Code Optimisation

Intermediate Code Optimisation

Intermediate Code Optimisation

Variable renaming (name mangling)

Code Generation

e.g. transpiling to ES5

Code Optimisation

UI Code Generation

Real world example...

Our goal...

design.sketch

design.html

<div class="xly_d1nz" role="group" aria-label="dialog">
  <div class="xly_inli">
    <svg width="321.00" height="557.00">
      <path stroke-width="0.5" stroke="#979797ff" fill="rgba(250,250,250,1.00)" d="M320.5 556.5,S 80.356 695.249, 0.5 556.5,S -79.789 139.5, 0.5 0.5,S 240.642 -138.623, 320.5 0.5,z"/>
    </svg>
    ...

design.sketch

Compiler

design.html

Our goal...

Reverse engineering *.sketch files

$ unzip design.sketch
62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00  bplist00........
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00  ................
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69  ...g?..g@X$versi
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63  onX$objectsY$arc
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af  hiverT$top......
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00  ................
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00  ......"...M...N.
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00  ..e...j...n.....
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00  ................
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00  ................
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00  ................
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00  ................
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00  ................
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00  ................
<trimmed>
6c 38 00 29 6c 45 00 29 6c 4e 00 29 6c 5f 00 29  l8.)lE.)lN.)l_.)
6c 6e 00 29 6c 7b 00 29 6c 8a 00 29 6c 93 00 29  ln.)l{.)l..)l..)
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00  l..)l..)l.......
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00  ........gC......
00 00 00 00 00 00 00 29 6c ae                    .......)l.
.
├── Data
├── Images
│   ├── 3.png
│   └── 6.png
├── QuickLook
│   ├── Preview.png
│   └── Thumbnail.png
├── fonts
└── version
62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae                  

Decoding bplist files

HEADER
  magic number ("bplist")
  file format version (currently "0?")
OBJECT TABLE
  variable-sized objects
  Object Formats (marker byte followed by additional info in some cases)
  null    0000 0000
  bool    0000 1000                             // false
  bool    0000 1001                             // true
  fill    0000 1111                             // fill byte
  int     0001 nnnn	...                     // # of bytes is 2^nnnn, big-endian bytes
  real    0010 nnnn	...                     // # of bytes is 2^nnnn, big-endian bytes
  date    0011 0011	...                     // 8 byte float follows, big-endian bytes
  data    0100 nnnn	[int]	...             // nnnn is number of bytes unless 1111 then int count follows, followed by bytes
  string  0101 nnnn	[int]	...             // ASCII string, nnnn is # of chars, else 1111 then int count, then bytes
  string  0110 nnnn	[int]	...             // Unicode string, nnnn is # of chars, else 1111 then int count, then big-endian 2-byte uint16_t
          0111 xxxx                             // unused
  uid     1000 nnnn	...                     // nnnn+1 is # of bytes
          1001 xxxx                             // unused
  array   1010 nnnn	[int]	objref*	        // nnnn is count, unless '1111', then int count follows
          1011 xxxx                             // unused
  ser     1100 nnnn	[int]	objref*         // nnnn is count, unless '1111', then int count follows
  dict    1101 nnnn	[int]	keyref* objref*	// nnnn is count, unless '1111', then int count follows
          1110 xxxx                             // unused
          1111 xxxx                             // unused
OFFSET TABLE
  list of ints, byte size of which is given in trailer
  -- these are the byte offsets into the file
  -- number of these is in the trailer
TRAILER
  byte size of offset ints in offset table
  byte size of object refs in arrays and dicts
  number of offsets in offset table (also is number of objects)
  element # in offset table which is top level object
  offset table offset

DATA

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae                  
{
  "frame": {
    "_class": "rect",
    "constrainProportions": false,
    "height": 224,
    "width": 344,
    "x": 0,
    "y": 0
  },
  "style": {
    "_class": "style",
    "endMarkerType": 0,
    "miterLimit": 10,
    "startMarkerType": 0,
    "windingRule": 1,
    "blur": {
      "_class": "blur",
      "isEnabled": false,
      "center": "{0.5, 0.5}",
      "motionAngle": 0,
      "radius": 10,
      "saturation": 1,
      "type": 0
    },
    ...

Parsing bplist files

{
  ...
  "NSColor": {
    "_archive": "YnBsaXN0MDDUAQIDBAUGHyBYJHZlcnNpb25YJG9iamVjdHNZJGFyY2hpdmVyVCR0b3ASAAGGoKUHCBEVHFUkbnVsbNQJCgsMDQ4PEFVOU1JHQlxOU0NvbG9yU3BhY2VfEBJOU0N1c3RvbUNvbG9yU3BhY2VWJGNsYXNzRjAgMCAwABABgAKABNISDBMUVE5TSUQQAYAD0hYXGBlaJGNsYXNzbmFtZVgkY2xhc3Nlc1xOU0NvbG9yU3BhY2WiGhtcTlNDb2xvclNwYWNlWE5TT2JqZWN00hYXHR5XTlNDb2xvcqIdG18QD05TS2V5ZWRBcmNoaXZlctEhIlRyb290gAEACAARABoAIwAtADIANwA9AEMATABSAF8AdAB7AIIAhACGAIgAjQCSAJQAlgCbAKYArwC8AL8AzADVANoA4gDlAPcA+gD\/AAAAAAAAAgEAAAAAAAAAIwAAAAAAAAAAAAAAAAAAAQE="
  },
}

Parsing bplist files!!

62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae                  
{
  ...
  "NSColor": {
    "_archive": "YnBsaXN0MDDUAQIDBAUGHyBYJHZlcnNpb25YJG9iamVjdHNZJGFyY2hpdmVyVCR0b3ASAAGGoKUHCBEVHFUkbnVsbNQJCgsMDQ4PEFVOU1JHQlxOU0NvbG9yU3BhY2VfEBJOU0N1c3RvbUNvbG9yU3BhY2VWJGNsYXNzRjAgMCAwABABgAKABNISDBMUVE5TSUQQAYAD0hYXGBlaJGNsYXNzbmFtZVgkY2xhc3Nlc1xOU0NvbG9yU3BhY2WiGhtcTlNDb2xvclNwYWNlWE5TT2JqZWN00hYXHR5XTlNDb2xvcqIdG18QD05TS2V5ZWRBcmNoaXZlctEhIlRyb290gAEACAARABoAIwAtADIANwA9AEMATABSAF8AdAB7AIIAhACGAIgAjQCSAJQAlgCbAKYArwC8AL8AzADVANoA4gDlAPcA+gD\/AAAAAAAAAgEAAAAAAAAAIwAAAAAAAAAAAAAAAAAAAQE="
  },
}
62 70 6c 69 73 74 30 30 d4 00 00 00 01 00 00 00
02 00 00 00 03 00 00 00 04 00 00 00 05 00 00 00
06 00 03 67 3f 00 03 67 40 58 24 76 65 72 73 69
6f 6e 58 24 6f 62 6a 65 63 74 73 59 24 61 72 63
68 69 76 65 72 54 24 74 6f 70 12 00 01 86 a0 af
11 bf db 00 00 00 07 00 00 00 08 00 00 00 19 00
00 00 1e 00 00 00 22 00 00 00 4d 00 00 00 4e 00
00 00 65 00 00 00 6a 00 00 00 6e 00 00 00 81 00
00 00 88 00 00 00 89 00 00 00 8a 00 00 00 92 00
00 00 9d 00 00 00 9e 00 00 00 9f 00 00 00 a0 00
00 00 a1 00 00 00 a6 00 00 00 b7 00 00 00 b8 00
00 00 b9 00 00 00 be 00 00 00 c4 00 00 00 cb 00
00 00 d6 00 00 00 d7 00 00 00 d8 00 00 00 d9 00
00 00 da 00 00 00 db 00 00 00 e0 00 00 00 e7 00
00 00 f2 00 00 00 f3 00 00 00 f4 00 00 00 f5 00
00 00 f6 00 00 00 f7 00 00 00 fb 00 00 01 00 00
00 01 01 00 00 01 02 00 00 01 07 00 00 01 08 00
<trimmed>
6c 9e 00 29 6c a7 00 29 6c ac 00 00 00 00 00 00
04 04 00 00 00 00 00 03 67 43 00 00 00 00 00 00
00 00 00 00 00 00 00 29 6c ae                  

Parsing bplist files!!

bplist

Parsing bplist files

New Sketch file Format (43+)

New Sketch file Format (43+)

$ unzip design.sketch
{
  "frame": {
    "_class": "rect",
    "constrainProportions": false,
    "height": 224,
    "width": 344,
    "x": 0,
    "y": 0
  },
  "style": {
    "_class": "style",
    "endMarkerType": 0,
    "miterLimit": 10,
    "startMarkerType": 0,
    "windingRule": 1,
    "blur": {
      "_class": "blur",
      "isEnabled": false,
      "center": "{0.5, 0.5}",
      "motionAngle": 0,
      "radius": 10,
      "saturation": 1,
      "type": 0
    },
    ...
.
├── document.json
├── meta.json
├── pages
│   └── <UUID>.json
├── previews
│   └── preview.png
├── text-previews
│   ├── text-previews-metadata.json
│   └── text-previews.pdf
└── user.json

New Sketch file Format (43+)

CSS compilation pipeline

*

* CSS Parser & optimiser only!

CSS parser (DFS)

Leftmost Depth-Fist-Search
 

Advantages:

* Lower memory usage

* High performance

* Easy to implement

CSS Tree

CSS Optimiser

Generating code from AST

Can we do more?

WassimChegham

Demystifying UI Code Generation

By Wassim Chegham

Demystifying UI Code Generation

  • 367
Loading comments...

More from Wassim Chegham