Making state management
Craft Conf 2024
David Khourshid · @davidkpiano
stately.ai
intelligent
stately.ai
stately.ai
stately.ai
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10790686/CleanShot_2023-09-28_at_13.42.18_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10244483/68747470733a2f2f6269746275636b65742e6f72672f736861756e65772f7061632d6d616e2f7261772f3437313438303032333361392f73686f74732f6c6561726e2e706e67.png)
import { createMachine } from "xstate";
export const ghostMachine = createMachine(
{
id: "Ghost 👻",
initial: "Wandering maze",
states: {
"Wandering maze": {
on: {
"Lose Pac-Man": {
target: "Chase Pac-Man",
},
"Pac-Man eats power pill": {
target: "Run away from Pac-Man",
},
},
},
"Chase Pac-Man": {
on: {
"Pac-Man eats power pill": {
target: "Run away from Pac-Man",
},
},
},
"Run away from Pac-Man": {
on: {
"power pill wears off": {
target: "Wandering maze",
},
"eaten by Pac-Man": {
target: "Return to base",
},
},
},
"Return to base": {
on: {
"reach base": {
target: "Wandering maze",
},
},
},
},
}
);
import { createActor } from 'xstate';
import { ghostMachine } from './ghostMachine';
const actor = createActor(ghostMachine);
actor.subscribe(state => {
console.log(state);
});
actor.start();
actor.send({ type: 'Lose Pac-Man' });
// { value: 'Chase Pac-Man', ... }
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836848/jotai-header-dark.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836835/1_fKV3_Y4usDYZKPsNp1yCvA.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836836/mobx-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836836/mobx-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10437194/logo-title-light.png)
- Make API call to auth provider
- Initiate OAuth flow
- Ensure token is valid
- Persist token as cookie
- Redirect to logged in view
Given a user is logged out,
When the user logs in with correct credentials
Then the user should be logged in
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10482116/CleanShot_2023-05-18_at_09.12.55_2x.png)
cart
shipping
contact
payment
confirmation
CHECKOUT
NEXT
NEXT
ORDER
PAYPAL
BACK
BACK
CANCEL
Identify
logical flaws
cart
shipping
contact
payment
confirmation
CHECKOUT
NEXT
NEXT
ORDER
PAYPAL
As a user, when I'm in the cart and I click the checkout button, I should be on the shipping page.
cart
shipping
contact
payment
confirmation
CHECKOUT
NEXT
NEXT
ORDER
PAYPAL
As a user, when I'm in the cart and I checkout via PayPal, I should be taken directly to the payment screen.
cart
shipping
contact
payment
confirmation
CHECKOUT
NEXT
NEXT
ORDER
PAYPAL
Shortest path
confirmation state
cart
shipping
contact
payment
confirmation
CHECKOUT
NEXT
NEXT
ORDER
PAYPAL
Shortest path
confirmation state where
shipping address is provided
import { getShortestPaths } from '@xstate/graph';
import { someMachine } from './someMachine';
// Finds all the shortest paths from
// initial state to other states
const shortestPaths = getShortestPaths(someMachine);
GRAPH
npm i @xstate/graph
But times are changing.
Users want
intelligent apps
LLMs are
not enough
Non-deterministic
Not easily explainable
Confidently wrong
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371842/eat-rocks-google-ai-overviews.jpg.webp)
Input
Output
Generative / creative
Input
Output
Goal
Start
Generative / creative
Task / goal-oriented
What is an agent?
Perform tasks → accomplish goal
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371921/CleanShot_2024-05-29_at_23.26.51_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371923/Blog-header--1-.webp)
Code demo
npm i @statelyai/agent
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444486/image6.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444464/CleanShot_2023-05-03_at_10.40.47_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444468/skiing.gif)
tl;dr: RTFM to learn faster
Creating intelligent agents
→ State machines
Determinism, explainability
→ Reinforcement learning
Exploration, exploitation
→ Large language models
Interpretation, creativity
Reinforcement learning (RL)
🤖 An intelligent agent learns
🔮 ... through trial and error
💥 ... how to take actions inside an environment
🌟 ... to maximize a future reward
State machines that can learn
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
Environment
![](https://media3.giphy.com/media/cyMqOH8rjgDHG/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241657/CleanShot_2023-02-21_at_22.46.48_2x.png)
Normal mode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241658/CleanShot_2023-02-21_at_22.47.00_2x.png)
Scatter mode
Agent
![](https://media4.giphy.com/media/gYWeVOiMmbg3kzCTq5/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10440543/shapes.png)
![](https://media4.giphy.com/media/gYWeVOiMmbg3kzCTq5/giphy.gif)
![](https://media4.giphy.com/media/hkqefnFjn2MWVl6xvq/giphy.gif)
![](https://media3.giphy.com/media/go3pCPP4899Jd3xb4p/giphy.gif)
Agent
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10242408/pac-man-path-screen-4.gif)
Policy
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10440552/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241672/4566657653_694d28a38a_b.jpeg)
Reward
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10440552/pasted-from-clipboard.png)
+10
-3
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241672/4566657653_694d28a38a_b.jpeg)
🟡 + 🍒 = 👍
🟡 + 👻 = 👎
Credit assignment
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793328/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793329/pasted-from-clipboard.png)
Exploration
vs Exploitation
no reward
Do nothing
Owner has treat
Tells dog "down"
ENVIRONMENT
Nothing changes
ENVIRONMENT
Exploration
reward++
Go down
Get treat
Owner has treat
Tells dog "down"
ENVIRONMENT
Owner praises dog
ENVIRONMENT
Exploration
Go down
Owner has treat
Tells dog "down"
ENVIRONMENT
Owner praises dog
ENVIRONMENT
Exploitation
💭
Treat?
Run away
Owner has treat
Tells dog "down"
ENVIRONMENT
Undesired outcome
ENVIRONMENT
🤬
Exploration
"Down"
Reward 🦴
Reinforcement learning
with human feedback (RLHF)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371965/CleanShot_2024-05-29_at_23.52.16_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371965/CleanShot_2024-05-29_at_23.52.16_2x.png)
LLM Context:
- Observed state transitions due to events;
causal relationship - Past interactions with the agent;
human and assistant messages - Previous planned sequences of events;
potential courses of action - Value of past actions in plain language;
reward values if applicable
-
Observations
-
History
-
Plans
-
Feedback
Goal prompt +
State machines for
intelligent logic
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444504/CleanShot_2023-05-03_at_11.11.45_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11372329/CleanShot_2024-05-30_at_08.50.40_2x.png)
Learnings
LLMs are unpredictable
Event-based logic makes our apps
more declarative
State machines & reinforcement learning
make LLM agents more intelligent/predictable
Future ideas
🔭 State machine synthesis
🤖 Adaptive UIs with RL
🌌 Deep learning for state space reduction
intelligent
Make your state management
Declarative UIs
Declarative state management
Separate app logic from view logic.
LLMs
Reinforcement learning
Use as minimally as possible.
Köszönöm Craft Conf!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9674899/logomark-black-nobg.png)
Craft Conf 2024
David Khourshid · @davidkpiano
stately.ai
Making state management intelligent - Craft Conf
By David Khourshid
Making state management intelligent - Craft Conf
This presentation introduces stately.ai and explores the limitations of LLMs. It discusses the concept of intelligent agents, state machines, reinforcement learning, and the use of large language models. It also covers credit assignment, exploration, and reinforcement learning with human feedback.
- 232