verifying
lookingUp
deciding
issued
Graph engineering
Making state-machine powered agents
City JS Athens · 22 Oct 2026
David Khourshid · @davidkpiano
stately.ai
verifying
lookingUp
deciding
issued
Graph engineering
Making state-machine powered agents
City JS Athens · 22 Oct 2026
David Khourshid · @davidkpiano
stately.ai
github.com/davidkpiano/
graph-engineering-workshop
Who am I?
@davidkpiano
Same desk, same scenarios
different harnesses
Arm
Correct, per run
Rule violations
Cost per run
raw tool loop
7/10 · 9/10 · 5/10
0
$0.04
environment
10/10 · 10/10 · 10/10
0
$0.005
Violations were 0 in both arms: this shows accuracy and cost, not the guarantee.
live · openai/gpt-6.1-sol · 10 tickets · 3 runs · 21 Oct 2026 · results/benchmark-gpt-6.1-sol.md
A loop gets smarter when the model gets smarter.
A machine gets smarter when you do, and it can tell you which change helped.
Two machines
two owners
Environment
Written in
TypeScript (XState)
Owned by
people
Holds
business states, events, rules as guards
Changes
rarely, with review
Policy
Written in
JSON
Owned by
the model
Holds
which legal option to choose
Changes
between runs, through gates
How today's workshop works
draw, prompt, review, iterate
1 · Prep draw it first: tldraw, Stately, paper
2 · Prompt ask like you would at work
3 · Review the diff against your drawing
4 · Iterate paste the failing diff, not a summary
pnpm install
pnpm demo # 5 sections = ready
pnpm test exercises/01
pnpm solve 02 # behind? checkpoint
pnpm reset 02 # back to the stubBring your own workflow
same steps in your domain
workflow
must never
Email drafter
send without approval
Support triage
promise a refund above the limit
Course builder
publish without review
Meeting notes → tasks
create a ticket with no owner
PR review assistant
suggest approve while CI fails
docs/bring-your-own.md: starter prompt and done criteria for every exercise
Loops → graphs
where the field is, Oct 2026
June 2026
“Loop engineering”: agent loop, verification loop, event-driven loop, hill-climbing loop
18 Jul 2026
“Are we still talking loops or did we shift to graphs yet?” · Peter Steinberger
22 Jul 2026
“As David Khourshid put it, a loop is just a directed, cyclic graph” · LangChain
A loop is a graph
with two states
while (turns < MAX_TURNS) {
const act = await model.act({
...obs, system: PROMPT, tools,
});
const result = tools[act.tool](act.args);
history.push({ ...act, result });
if (TERMINAL.includes(act.tool)) break;
}thinking
acting
done
tool call
result
terminal tool
Every business rule lives in PROMPT
How loops fail
every failure = question about state
✗ Skipped steps
refunds before verifying identity
✗ Rule breaks
refunds someone else's order
✗ No recovery
after a bad tool result, it improvises
✗ Invisible capability
what can it do? run it and see
✗ Unbounded cycles
the turn cap is the only exit
✗ Untestable
the smallest unit is the whole run
Cemri et al., Why Do Multi-Agent LLM Systems Fail? NeurIPS 2025 · 14 failure modes, 1,600+ traces, 7 frameworks
State machine thinking
questions to ask
What are the states?
What events are possible here?
What must never happen?
What happens on timeout or failure?
12-Factor Agents (Dex Horthy): #8 Own your control flow · #12 Make your agent a stateless reducer
The refund desk
src/world.ts
ORD-1001
C-1 · $40
ORD-1002
C-1 · $240
ORD-1003
C-3 · $60 · already refunded
Tools
verifyIdentity
lookupOrder
requestApproval
askCustomer
issueRefund
escalate
close
← terminal
C-1 and C-3 pass identity; C-2 fails · approval needed over $100 · the approver approves up to $500
A ledger records every rule break after the fact; nothing in the world prevents one
Our refund scenario
rules in the prompt (for now)
Verify identity before any refund
Look up the order; it must belong to the customer, otherwise escalate
Never refund an already-refunded order; close the ticket
Refunds over $100 need a human approval; if rejected, close
If the order number is missing, ask the customer
If identity verification fails, escalate
Never refund more than the order amount
Models are behaviours
not answer lists
interface Model {
act(req): Promise<{ tool, args }>; // raw loop
choose(req): Promise<number>; // machine
}The same model drives the raw loop and the machine, so the comparison is fair
sloppyModel
Does what the ticket says
act: refund the amount asked · choose: the biggest refund on offer
carefulModel
Follows the rules
verify, ask, look up, approve over $100, then refund
Ten scenarios
four to train, six held out
id
split
customer
ticket asks
expected
small-clean
train
C-1
ORD-1001, $40
issued $40
large-needs-approval
train
C-1
ORD-1002, $240
issued $240
someone-elses-order
train
C-3
ORD-1002, $240
escalated
already-refunded
train
C-3
ORD-1003, $60
closed
missing-order-id
held-out
C-1
no id → “ORD-1001”, $40
issued $40
identity-fails
held-out
C-2
ORD-1001, $40
escalated
refunded-again
held-out
C-3
ORD-1003, $60
closed
approval-rejected
held-out
C-1
ORD-1004, $800
closed
no-order-id-no-reply
held-out
C-1
no id → “no idea”
escalated
asks-too-much
held-out
C-1
ORD-1002, $1000
issued $240
Exercise 1
the raw loop · 15 min
Prep · 5 min
Sketch the loop: two states, four exits. Next to each of the seven rules, write where the loop enforces it.
diagrams/my-01-loop.png · tldraw frame 1
Done
✓ careful: verify → look up → refund
✓ a stuck model stops at the cap
✓ sloppy breaks three rules, and the test passes
Steps
1 · Prep, no code · 5 min
2 · Prompt · 2 min
3 · Review · 5 min
4 · Iterate · 3 min
pnpm test exercises/01Where could you stop it?
an if before issueRefund
if (act.tool === 'issueRefund') {
if (!verified) { /* … */ }
if (order.customerId !== customerId) { /* … */ }
if (order.refunded) { /* … */ }
if (amount > 100 && !approved) { /* … */ }
if (amount > order.amount) { /* … */ }
}Each check depends on what already happened
That's state
Implicit phases
become explicit states
verifying
lookingUp
deciding
requestingApproval
askingCustomer
issuing
paying
issued
illegallyIssued
escalated
closed
> $100
≤ $100
approved
from deciding, or on any failure
Rules become guards
a transition that returns nothing is refused
REFUND: ({ context, event }) => {
if (!refundIsLegal(context, event.amount)) {
return undefined; // refused
}
return {
target: event.amount > 100
? 'requestingApproval'
: 'issuing',
context: {
amount: event.amount,
decisions: context.decisions + 1,
},
};
},deciding
requestingApproval
issuing
> $100
≤ $100
refundIsLegal: owned · not refunded · 0 < amount ≤ order · decisions < 4
The only loop you need
the model moves into nextEvent()
state, effects = init(machine)
execute(effects)
while state is active:
event = nextEvent()
state, effects = transition(machine, state, event)
execute(effects)
return state.outputin this repo
init / transition
XState v6, pure: [snapshot, effects]
execute
start the invoked tools
nextEvent
a tool's result, or the model at deciding
output
the final state: issued · escalated · closed
Exercise 1: the model is transition(). Here: the model is one source of events.
decideNext
the whole idea in 15 lines
const candidates = candidateOptions(snapshot.context);
// REFUND $order, REFUND $asked, ESCALATE, CLOSE
const options = candidates.filter((o) =>
snapshot.can(o),
);
const i = await model.choose({ options, facts });
actor.send(options[i]);The machine decides what's possible
The model chooses among what's possible
Asked for $1000
on a $240 order
candidateOptions
✓ REFUND 240
✗ REFUND 1000
✓ ESCALATE
✓ CLOSE
snapshot.can
model.choose({
options: [
{ type: 'REFUND', amount: 240 },
{ type: 'ESCALATE' },
{ type: 'CLOSE' },
],
})Alshiekh et al., Safe Reinforcement Learning via Shielding. AAAI 2018 (preemptive vs post-posed shields)
The sentinel
a state no legal run can enter
issuing
paying
issued
illegallyIssued
issuing: {
always: ({ context }) => violatesPolicy(context)
? { target: 'illegallyIssued' }
: { target: 'paying' },
},violatesPolicy
not verified, no order, no amount
order not owned
already refunded
amount over the order
over $100 without approval
The proof
no model, no tokens
const variants = {
'xstate.done.actor': (e) =>
outputs[e.actorId].map((output) => ({ ...e, output })),
REFUND: () =>
[0, 40, 100, 101, 240, 241, 1000]
.map((amount) => ({ type: 'REFUND', amount })),
};
getShortestPaths(machine, {
input,
events: (snapshot) =>
getAllOwnEvents(snapshot)
.flatMap((e) => variants[e.type]?.(e) ?? [e]),
});
reachable: false
32 states explored
ownership guard removed
reachable: true
witness: … → REFUND
What the proof covers
and what it doesn't
Covers
✓ every path through the graph
✓ every answer we enumerated
✓ both sides of every guard boundary
Doesn't cover
✗ every possible string or amount
✗ a wrong recipient
✗ a hallucinated sentence in a reply
Run the zero-token check first because it's cheap. Run trajectory evals second.
The environment can disable events
it can't force one
Safety
“never X”
✓ enforceable: the environment makes it unreachable
Liveness
“eventually Y”
✗ not enforceable: the policy's job
Enter terminal states on evidence
not on the model's say-so
paying
issued
xstate.done.actor · issueRefund
A state-machine-controlled agent (StateFlow) still fabricated results on 25% of cells
Deng, Goal-Autopilot. arXiv 2606.11688, June 2026 (25.05%, 95% CI 22.48–27.62)
Why not a permission hook?
because the rules have memory
Rule
Structure
Never call delete
1 state · a hook is enough
Approval must precede refund
2 states
At most 3 revisions, then escalate
a counter + a transition
Over $100 needs a human, unless this session has sign-off
a product of states
Statechart features
where they fit in the desk
Feature
In the refund desk
Hierarchy
a bounded loop inside deciding, with its own exit
Parallel regions
a budget or SLA watcher next to the main flow
after
escalate if approval takes more than 24 h
Guards
the business rules
Invoked actors
the tools, and later other agents
Exercise 2
port and prove · 20 min
Prep · 8 min
Draw the statechart: states, events, a guard on every rule, the sentinel, and the one state where the model chooses. Then diff it with the reference.
diagrams/my-02-environment.json · Stately editor
Done
✓ sloppy model: 0 violations, escalated
✓ careful model: all 10 scenarios
✓ only accepted events are offered
✓ sentinel unreachable; loosened guard finds a witness
Steps
1 · Prep, no code · 8 min
2 · Prompt · 2 min
3 · Review · 6 min
4 · Iterate · 4 min
pnpm test exercises/02Safe
but not yet good
Sloppy model in the environment: 0 violations · accuracy 0.80
Wrong on already-refunded and refunded-again: it escalated where the business wants it to close
Break · 10 min
github.com/statelyai/graph-engineering-workshop
pnpm solve 02 # if you're behind
Two kinds of failure
illegal, and legal but wrong
Illegal action
refund someone else's order
→ guards, with a proof
Legal but wrong
escalate an already-refunded order, when the business wants it closed
→ the policy, measured by evals
Environment vs policy
the contract and the procedure
Environment
Policy
Written by
people
a model (or people)
Language
TypeScript
JSON
Changes
rarely, with review
between runs, through gates
Verified by
proof (canReach)
objective on train + held-out
May name
anything
exposed facts, accepted events
When not to split
the split pays off only with a policy worth learning
A single judgment · one classification in a fixed flow → a decision in the environment
A pure environment · every step dictated by rules → one TypeScript machine, verified
No ground truth · nothing to score → collect traces and labels first
Of the library's ~50 examples, about a third needed a state machine at all
statelyai/agent PR #120 · vendor/agent/docs/environment-and-policy.md, “When not to split”
Autonomy is per state
not per system
More environment
More policy
code-review-loop
one decision
refunds
mixed
tic-tac-toe
nearly all policy
Each state: fixed code · a choice among legal events · a bounded loop inside the state
By the way
there's a library
You wrote by hand
@statelyai/agent
decideNext over snapshot.can
agent.decide (+ guard re-check, retries)
canReachIllegalIssue
canReach / explorePaths
the vocabulary gate (Ex 5)
verifyAgentConfig
recordRun / replay (Ex 4)
the event log, replay
improve (Ex 5)
the improve-machine skill
Alpha: 2.0.0-alpha.28 on npm lags the docs on next (runAgent, decideNext). The workshop uses it via pnpm vendor.
Tic-tac-toe
a policy climbs to a proof
v0
v1
v2
v3
losing lines
429
losing lines
329
losing lines
67
losing lines
0
first empty cell → + win → + block → full ladder
PROVEN never to lose (52 ms)
vendor/agent/examples/tic-tac-toe · docs/transcripts/tic-tac-toe-climb.txt · scripted proposals
Evals at every level
a machine gives you four places to test
Safety
no run can do the illegal thing
search the graph
→ sentinel unreachable
no model
Seam
one decision, everything else held still
park at deciding
→ decideNext
→ assert the event
Path
the route a run took
run(scenario)
→ assert statePath
Objective
outcomes vs expectations, violations 0
objective(model,
scenarios)
→ accuracy
AI Engineer World's Fair 2026 evals workshops: cheap code checks first, judges second, replicated runs always
Which check catches
which failure
Failure
Caught by
Illegal action, even one no run takes yet
guard + safety check, zero tokens
Legal but wrong (escalate vs close)
seam eval
Skipped required step (no approval)
path eval
Regressions across scenarios
objective, several seeds
Wrong content inside a legal action
content evals or judges
Exercise 3
evals at every level · 15 min
Prep · 6 min
For each way the desk could go wrong, predict which level catches it: safety, seam, path, or objective. Then one prompt.
diagrams/my-03-evals.md · tldraw frame 3
Done
✓ your evals pass on the working desk
✓ they catch all three broken desks
✓ the matrix matches your prediction
Steps
1 · Prep, no code · 6 min
2 · Prompt · 2 min
3 · Review · 4 min
4 · Iterate · 3 min
pnpm test exercises/03A trace is a stamped event log
every event tagged with the state it arrived in
stamp refundDesk v1 · 2771615093fa · decider sloppy · 2026-10-21T21:55Z
seq state event
0 verifying xstate.done.actor verifyIdentity → { verified: true }
1 lookingUp xstate.done.actor lookupOrder → { refunded: true, … }
2 deciding ESCALATE offered [ESCALATE, CLOSE] · withheld 1
→ outcome escalated · expected closedStamp it with what made it. Keep it as JSONL. Set aside traces from a machine that no longer exists. A loop's trace is a transcript; a machine's is a path through a known graph.
Replay
no model, no world
const machine = refundDeskMachine.provide({
actors: {
verifyIdentity: canned('verifyIdentity'),
lookupOrder: canned('lookupOrder'),
// … each returns its next recorded output
},
});
// on each decision: send the next recorded eventSame statePath
Same outcome
0 tokens
Test a candidate change against yesterday's failures before it ever runs live
The only loop you need, again: nextEvent() reads the log · execute() returns recorded outputs · transition() untouched
Optional: the heatmap
fold the traces: where should the policy change?
verifying
lookingUp
deciding
escalated
10 runs · 2 failed
0 withheld
8 runs · 2 failed
0 withheld
8 runs · 2 failed
4 withheld
5 runs · 2 failed
final state
withheld = candidate options minus offered options: how often the policy and the environment disagree
Exercise 4, done early · sloppy model, 10 scenarios
Credit assignment
by location
A failing loop run
“the agent failed”
A failing machine run
“deciding chose ESCALATE on an already-refunded order”
Exercise 4
stamped traces · 10 min
Prep · 3 min
Decide what a trace must carry to be useful a month from now: the stamp fields, and the question each one answers.
diagrams/my-04-trace.md · tldraw frame 4
Done
✓ stamped: machine version + hash, the model
✓ saved as JSONL; stale traces set aside
✓ any trace replays with no model
Steps
1 · Prep, no code · 3 min
2 · Prompt · 1 min
3 · Review · 4 min
4 · Iterate · 2 min
pnpm test exercises/04
“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.”
Add structure for today's compute
and plan to remove it
“The more structure, the less scalable the method is.”
Understand your application structure
Re-evaluate structure as models improve
Make it easy to remove structure
The case against this workshop
at full strength
✗ The gains shrink as models improve
✗ Hand-drawn procedure graphs can hurt
✗ The market voted for loops
✗ A strong model can just read the graph
✗ Real constraints are one-line hooks
✗ Reachability checks the trivial part
✗ Production already has a state machine
StateFlow (COLM 2024) · Procedural Graphs (Google, Sep 2026) · LangChain (Jul 2026)
What dies
what survives
Dies
✗ states as capability scaffolding
✗ “draw the whole statechart first”
✗ “the graph makes the agent smarter”
Survives
✓ the model as an event source
✓ stateful rules as explicit automata
✓ zero-token checks before evals
✓ a learned policy, under a gate
Scaffolding or constraint?
vote on each
Plan → critique → revise as three states
Refunds over $100 need a human approval
Verify identity before any refund
Retry a failed tool call 3 times
At most 8 turns per ticket
Close (don't escalate) already-refunded orders
Feeding traces back
isn't enough on its own
Variance went up in 17 of 24 settings; best and worst runs up to 10 points apart
Shuffled task order: −4.5% instead of the expected +1.5%
ReasoningBank's +1.5%: p = 0.23, not significant
Memories were “unverified hypotheses rather than true lessons learned”
The loop
one round
traces
objective
diagnose
one diff
gates
replay train
replay held-out
promote or keep
Diagnose from train only · one cause per diff · held-out decides
Procedural Graphs
a graph that edits itself from traces
Commits only if validation “matches or exceeds”
Rejected candidates go to a rejection memory
A hand-written expert graph: 87.5% → 58.9%
Evolution repaired it: 92.9%
First or joint first in 21 of 24 settings
What ours adds
✓ hard edges, not guidance
✓ an environment it can't touch
✓ a diff a person reads
arXiv 2609.09153, Sep 2026 · guidance “biases the solver's next action without dictating it”
The gate stack
before anything runs
Schema: is it a policy?
Vocabulary: exposed facts, accepted events
Coverage: a catch-all rule exists
Replay train, then held-out
Promotion: 0 violations · no regression · gain
A person reads the diff
Three rules from the research
Evaluate the policy inside the environment
Keep the evaluator out of the proposer's reach
Narrowing is automatic; expansion needs a human
Three proposals
pnpm demo §5
REJECTED
“Add a FORCE_REFUND that skips the machine”
vocabulary: not an event the environment accepts
PROMOTE
“Already-refunded orders fell back to ESCALATE; close them”
+ {"orderRefunded":true} → {"type":"CLOSE"} · train 0.75 → 1.00 · held-out 0.83 → 1.00
KEEP
“Large refunds are risky; escalate over the threshold”
held-out regressed 1.00 → 0.67 · train regressed 1.00 → 0.75
LEGO or play-doh?
the honest version
Hypothesis
Edits to a machine have an address, can be checked, and can be taken back out. Edits to prose don't.
Counterevidence: ACE
One full rewrite: 18,282 tokens → 122; accuracy 66.7 → 57.1, below no memory (63.7)
Small delta edits to itemized prose: 70.3 vs 56.9
No study compares a state-machine substrate against prose with the same optimizer
The control arm
same loop, prose instead of policy
A′ · prose
B · policy
Rewritten
RAW_LOOP_SYSTEM_PROMPT
policies/v0.json
Harness
raw loop
the environment
Executor
the live model
policyModel, no model
Gates
non-empty string
schema · vocabulary · coverage
ANTHROPIC_API_KEY=… pnpm control-armSame proposer · same objective · same train/held-out split · same promotion rule
What can improve
three layers
Context · prompts, descriptions, examples for one state
Routing and budgets · which model, how many retries, thresholds
Topology · new rules, states, edges, inside the policy only
What self-improvement is here
and what it isn't
✓ runs between deploys, never on a live machine
✓ passes a gate before it runs
✓ lands as one diff a person can read
✗ makes the agent dramatically smarter
code-review-loop on mocks: train 0.900 → 0.920 · held-out 0.900 → 0.933
Exercise 5
one improvement round · 20 min
Prep · 8 min
Write the policy ladder, the gate, the promotion rule, and your verdict on the three scripted proposals.
diagrams/my-05-policy.md · tldraw frame 5
Done
✓ REJECTED → PROMOTE → KEEP, as you predicted
✓ v1 at 100%, 0 violations
Steps
1 · Prep, no code · 8 min
2 · Prompt · 2 min
3 · Review · 6 min
4 · Iterate · 4 min
pnpm test exercises/05Stretch: the live round in vendor/agent (needs claude)
Mixing models
route the decision to a decision model
deciding
one Choice question · each option described by what it changes
REFUND 240 · refund now; over $100 goes to approval first
ESCALATE · hand the ticket to a person
CLOSE · close with no refund
→ a probability per option · below the threshold, escalate
Jevspresso
jev makes every move; the bar says what's possible
The bar: one parallel XState machine
order intake · grinder
espresso machine · steam wand
portafilter · milk pitcher
each barista's hands
Physics, not recipes. Mistakes are possible.
Every time the bar changes
1 offer every barista.* event that can() accepts
2 describe each in the machine's words, plus what it would change
3 jev picks one; the bar takes it only if it's still possible
The honest numbers
77-intent routing
Model
Accuracy
Opus 5.5
85.7
GPT-6 Sol
85.3
Jev
78.3
Cost: roughly even against cheaper LLMs · 84–150× cheaper than Opus 5.5
AI/ML API, What is Jev (Sep 2026) · run and published by an API platform
Agents are actors
orchestration is which events flow, and when
supervisor
researcher
writer
reviewer
RESEARCH
DRAFT
REVIEW
Private state · messages only · can spawn actors (Hewitt, 1973)
Multi-agent failures
are mostly control failures
MAST failure mode
As a machine
Premature termination
done only after a VERIFIED event
Step repetition
a counter with a guard, then escalate
Ignoring another agent's input
wait in a state for its message
Missing verification
verification on the only path to done
Cemri et al., Why Do Multi-Agent LLM Systems Fail? NeurIPS 2025 · 14 modes, 3 categories, κ = 0.88
Same desk, same scenarios
now you can read every row
Arm
Correct, per run
Rule violations
Cost per run
raw tool loop
7/10 · 9/10 · 5/10
0
$0.04
environment
10/10 · 10/10 · 10/10
0
$0.005
The loop is Exercise 1. The environment is Exercise 2. The control arm in Exercise 5 is how you'd know a change helped.
live · openai/gpt-6.1-sol · 10 tickets · 3 runs · 21 Oct 2026 · results/benchmark-gpt-6.1-sol.md
Take it home
one agent, step by step, from the terminal
node examples/email-drafter/run-all.ts
# or one step at a time
00-dataset.ts # the cases, written first
01-loop.ts # a mega-prompt loop
02-async.ts # async/await, rules in code
03-machine.ts # states, guards, a timeout, traces
04-replay.ts # replay and "what if?"
05-evals.eval.ts # vitest evals
06-improve.ts # a better policy, from traces
# add --inspect to watch runs live in the Stately inspectorexamples/
email-drafter
the whole day on a second domain, a prompt per step
pizza
a workflow built live in three prompts
twenty-questions
game rules + the agent's mind, two machines, built from prompts
docs/impromptu.md
any agent idea, five prompts
Real models only: ANTHROPIC_API_KEY or OPENAI_API_KEY in .env
A loop gets smarter when the model gets smarter.
A machine gets smarter when you do, and it can tell you which change helped.
Take this with you
Ask: states? events possible here? must never? on timeout?
Rules go in guards, not prompts. The model chooses only among legal moves.
Enter final states on evidence, never on the model's say-so
Give each state its own autonomy: fixed code, one choice, or a bounded loop
Measure every change: one cause per diff, held-out decides, run it more than once
Skip the machine with no irreversible step, no real decision, or no ground truth
github.com/statelyai/graph-engineering-workshop
The end!
David Khourshid · @davidkpiano
stately.ai
Graph Engineering: Making State-Machine Powered Agents
By David Khourshid
Graph Engineering: Making State-Machine Powered Agents
Workshop · Epignosis, Athens · 22 Oct 2026
