Optimizing of computational graph

• Two-level IRs
• ONNX

Computational Graph: High-level IR

Typically, there're two level IRs for DL framework/compiler. Computational graph is the high-level one.

A computational graph is a (high-level) dataflow graph of machine learning.

ONNX

A universal representation of computational graph.

A translation validator for optimizing of computational graphs

• Two ONNXs as input
• Equality Saturation

Equality Saturation

Core concepts in e-graph:

• E-node: a term f(t1, t2, ...)
• E-class: an equivalent class of E-nodes (terms)

Exploration by rewriting rules

Examples

Equality Saturation for Translation Validation

1. Construct e-graph from computational graph.
2. Merge two e-graphs.
3. Exploration by rewriting rules.
4. Validate if the two e-node of original graphs belong to a same e-class.

Difficulties

• Involves many details...
• How to succinctly implement rewriting? hard code now...

Difficulties

Difficulties

Difficulties

Difficulties