Dark magic of code transformation
Valeriy Kuzmin, Moonfare, 2021
CTA
Automate writing boring code!
Disclaimer
- ever worked with RegExes? This is (a bit) more complex
- ugly code ahead! (in repo)
- no basics explanations (maybe)
- simplified vs metacode (yeah, I'm lazy)
- this is not the only way to solve it
Problem: DTO type sharing
Client
Server
Some logic
Some logic
Object
field1
field2
Object
field1
field2
send json over HTTP
Problem: DTO type sharing
Client
Server
Some logic
Some logic
Actually the same definition
Object
field1
field2
send json over HTTP
Some magic
Problem: DTO type sharing
#[derive(Deserialize, Serialize)]
#[serde(tag = "tag")]
pub enum InventoryAction {
Unknown,
Split { from: Uuid, count: i32 },
Merge { from: Uuid, to: Uuid },
Move { item: Uuid, index: i32 },
}
Rust
Rust: nice properties
let act = InventoryAction::Split {
from: Uuid::new(),
count: 2
}
match act {
InventoryAction::Split { from, count } => { /*do stuff*/ }
_ => {}
}
Instantiation
Pattern matching
Desired outcome (hey, redux!)
export type InventoryActionUnknown = { tag: "Unknown" };
export type InventoryActionSplit = {
tag: "Split",
from: string,
count: number,
};
// more variants...
export type InventoryAction = InventoryActionUnknown | InventoryActionSplit;
export class InventoryActionBuilder {
static InventoryActionUnknown = (): InventoryActionUnknown => ({
tag: "Unknown"
});
static InventoryActionSplit = ({
from,
count
}): InventoryActionSplit => ({
tag: "Split",
from,
count
});
// more variants...
}
Existing partial solution
export type InventoryAction =
| { tag: 'Unknown' }
| { tag: 'Split'; from: Uuid; count: number }
| { tag: 'Merge'; from: Uuid; to: number }
| { tag: 'Move'; item: Uuid; index: number };
typescript-definitions crate
#[derive(Deserialize, Serialize,
TypeScriptify, TypescriptDefinition)]
#[serde(tag = "tag")]
pub enum InventoryAction {
Unknown,
Split { from: Uuid, count: i32 },
Merge { from: Uuid, to: Uuid },
Move { item: Uuid, index: i32 },
}
Rust
Generated TS
Task 1: Advanced unions
type InventoryActionUnknown = { tag: 'Unknown' };
type InventoryActionSplit = { tag: 'Split'; from: Uuid; count: number };
export type InventoryAction =
| InventoryAcitonUnknown
| InventoryActionSplit
export type InventoryAction =
| { tag: 'Unknown' }
| { tag: 'Split'; from: Uuid; count: number };
Task 1: Advanced unions - plan
- Detect export named declarations of type aliases
- Extract tag names out of union types + make new type names out of it + remember the full 'extracted' type
- Generate extra exported type aliases
- Generate a new union type that references newly generated aliases and replace the old one
Task 1.0: understand the AST
Task 1.1: detect & do something
// simplified
return j(file.source)
.find(j.ExportNamedDeclaration)
.replaceWith((ex: ASTPath<ExportNamedDeclaration>) => {
if (thatIsNotMyGoal) {
return ex.value;
}
return newStuff;
})
.toSource();
Task 1.2: extract stuff
Task 1.2: extract stuff
// metacode
const nameToAliases = {
InventoryAction: [
["InventoryActionUnknown", someSavedNode1],
["InventoryActionSplit", someSavedNode2],
]
}
Task 1.2: extract stuff
'pairs' further
// simplified
const insertedDeclarations =
pairs.map([fullMemberName, extractedType] =>
j.exportNamedDeclaration(
j.typeAlias(
j.identifier(fullMemberName),
null,
convertToFlow(extractedType)
)
)
);
Task 1.3: generate new exports
weird quirk of jscodeshift, you can only generate... flow!
// simplified
const aliases = pairs.map(([fullName]) => j.typeParameter(fullName));
return [
...insertedDeclarations,
j.exportNamedDeclaration(
j.typeAlias(
j.identifier(unionName),
null,
j.unionTypeAnnotation(aliases)
)
)
];
Task 1.4: replace the union
Done! Oh wait, there's more...
Task 2: Builder classes
// from previous
export type InventoryAction = InventoryActionUnknown | InventoryActionSplit;
// add this
export class InventoryActionBuilder {
static InventoryActionUnknown = (): InventoryActionUnknown => ({
tag: "Unknown"
});
static InventoryActionSplit = ({
from,
count
}): InventoryActionSplit => ({
tag: "Split",
from,
count
});
// more variants...
}
Task 2: plan
- Detect union type aliases and the types they reference
- Extract properties from the target types
- Generate the builders
- GenerateĀ a class with parent union name
- For every member, generate a static 'builder' method
- For every method, generate a signature and returned literal
Task 2.0: understand the AST
Task 2.1 - detect unions & their references
Our target unions:
Task 2.1 - detect unions & their references
What they refer to:
Task 2.1 - detect unions & their references
// metacode
const node = mySource.find(j.ExportNamedDeclaration)
const { declaration } = node;
if (declaration.type === "TSTypeAliasDeclaration") {
const { typeAnnotation } = declaration;
if (typeAnnotation.type === "TSTypeLiteral") {
savedTypes[declaration.id.name] = typeAnnotation;
}
else if (typeAnnotation.type === "TSTypeUnion") {
savedUnions[declaration.id.name] = typeAnnotation
.types.map(t => t.name);
}
else {
// not interesting
}
}
Task 2.2 - understand the fields of the generated constructors
// metacode
for (const typeRefName of Object.values(savedUnions[currentUnion])) {
const memberProps = savedTypes[typeRefName]
.members.filter(m => m.type === "TSPropertySignature");
// generate a builderMethods[currentUnion][typeRefName]
// based on props
}
Task 2.3.1 - generate a class
// simplified
return j.exportNamedDeclaration(
j.classDeclaration.from({
id: j.identifier(builderClassName),
body: j.classBody(builderMethods)
})
)
nice .from constructors,
use them in complex cases!
Task 2.3.2 - generate builder methods
// simplified
const currentUnionName, currentUnionMemberName, memberProps;
const builderMethod = j.classProperty.from({
key: j.identifier(subType.typeName.name),
static: true,
accessibility: "public",
declare: false,
value: makeBuilderMethod(
currentUnionMemberName,
memberProps,
j,
currentUnionName
),
});
normal property syntax vs ES6 class field syntax
Task 2.3.3 - generate the signature and literal
class BuilderClass {
static InventoryActionSplit = (
{
from,
count
} : {
from: Uuid,
count: number,
}
) : InventoryActionSplit => (
{
tag: "Split",
from,
count
}
);
}
Reminder of the goal
actual params
their type
built entity type
actual body
Task 2.3.3 - generate the signature and literal
// simplified
const currentMember = memberProps[currentUnionMemberName];
// 'InventoryActionSplit' => 'Split'
const tag = currentUnionMemberName.replace(unionName, "");
return j.arrowFunctionExpression.from({
expression: true,
params: makeParams(j, currentMember),
returnType: j.tsTypeAnnotation(
j.tsTypeReference(j.identifier(currentUnionMemberName), null)
),
body: j.objectExpression([
j.property("init", j.identifier("tag"), j.stringLiteral(tag)),
...makeProperties(j, currentMember),
]),
});
property 'kind'
Everything together
Task 2.3.3 - generate the signature and literal
// simplified makeParams
return j.objectPattern.from({
properties: memberProps.map((name) => {
return j.property.from({
key: j.identifier(name),
shorthand: true,
value: j.identifier(name),
kind: "init",
});
}),
typeAnnotation: j.tsTypeAnnotation(
j.tsTypeLiteral(memberProps)
)
}),
actual params
actual params
their type
(simplified code)
Method signature
Task 2.3.3 - generate the signature and literal
// simplified makeProperties
const names = getPropNamesWithoutTag(memberProps);
if (names.length === 0) { // tag-only case
return [];
}
return names.map((name) => {
return j.property.from({
key: j.identifier(name),
shorthand: true,
value: j.identifier(name),
kind: 'init',
});
});
make shorthand if possible
Method body properties (returned literal)
Done! For real, this time
Results
- Robots write the code for me!
- ~12 enums in ~85KB code
- Almost have Rust features in TS
- No chance to desync
- Transform speed is so-so
- Writing it broke my brain a bit
+
-
Lessons learned
- it's kind of hard... but once it works, it's amazing!
- there aren't so many resources or examples
- do runtime typechecks!!!
- TDD approach is possible
- š„² tests debugging - especially crashes - sucksĀ
- autoformat to stay sane
CTA
Automate writing boring code!
The end!
Questions?
jscodeshift-2-at-moonfare
By Valeriy Kuzmin
jscodeshift-2-at-moonfare
- 257