Making state management
React Summit 2024
David Khourshid · @davidkpiano
stately.ai
intelligent ✨
Why David likes
React Summit 2024
David Khourshid · @davidkpiano
stately.ai
state machines ✨
Making state management
so much
stately.ai
stately.ai
stately.ai
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11407511/Trimas-Packaging_AF-Tethered-Caps.jpeg)
function Todos(props) {
const [todos, setTodos] = useState(props.todos)
const [selectedTodo, setSelectedTodo] = useState(null)
// ...
}
function Todo(props) {
const [title, setTitle] = useState(props.todo.title)
const [content, setContent] = useState(props.todo.content)
const [completed, setCompleted] = useState(props.todo.completed)
useEffect(() => {
props.onUpdate({ title, content, completed })
}, [title, content, completed]);
// ...
}
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836848/jotai-header-dark.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836835/1_fKV3_Y4usDYZKPsNp1yCvA.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836836/mobx-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9836836/mobx-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10437194/logo-title-light.png)
import { createStore } from '@xstate/store';
const store = createStore(
{
todos: [],
selectedTodo: null
},
{
addTodo: {
todos: (ctx, { todo }) => ctx.todos.concat(todo)
},
updateTodo: {
todos: (ctx, { todo }) =>
ctx.todos.map((t) => (t.id === todo.id ? todo : t))
},
selectTodo: {
selectedTodo: (ctx, { todoId }) => todoId
},
toggleTodo: {
todos: (ctx, { todo }) =>
ctx.todos.map((t) =>
t.id === todo.id ? { ...t, completed: !t.completed } : t
)
}
}
);
STORE
npm i @xstate/store
import { useSelector } from '@xstate/store/react';
import { store } from './store';
function App() {
const todos = useSelector(store, (s) => s.context.todos);
return (
// ...
<button onClick={() => {
store.send({ type: 'addTodo', todo: {/* ... */} })
}}>
New todo
</button>
);
}
Let's talk about AI.
// /api/completion/route.ts
import { StreamingTextResponse, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
// Allow streaming responses up to 30 seconds
export const maxDuration = 30;
export async function POST(req: Request) {
const { prompt }: { prompt: string } = await req.json();
const result = await streamText({
model: openai('gpt-4-turbo'),
prompt,
});
return new StreamingTextResponse(result.toAIStream());
}
import { useCompletion } from '@ai-sdk/react';
function TodoDescription() {
const {
completion,
input,
handleInputChange,
handleSubmit
} = useCompletion({
api: '/api/completion',
});
return (
<form onSubmit={handleSubmit}>
<input
name="prompt"
value={input}
onChange={handleInputChange}
id="input"
/>
<textarea value={completion} />
<button type="submit">Submit</button>
</form>
);
}
Let's talk about really old AI.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10244483/68747470733a2f2f6269746275636b65742e6f72672f736861756e65772f7061632d6d616e2f7261772f3437313438303032333361392f73686f74732f6c6561726e2e706e67.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10790686/CleanShot_2023-09-28_at_13.42.18_2x.png)
import { createMachine } from "xstate";
export const ghostMachine = createMachine(
{
id: "Ghost 👻",
initial: "Wandering maze",
states: {
"Wandering maze": {
on: {
"Lose Pac-Man": {
target: "Chase Pac-Man",
},
"Pac-Man eats power pill": {
target: "Run away from Pac-Man",
},
},
},
"Chase Pac-Man": {
on: {
"Pac-Man eats power pill": {
target: "Run away from Pac-Man",
},
},
},
"Run away from Pac-Man": {
on: {
"power pill wears off": {
target: "Wandering maze",
},
"eaten by Pac-Man": {
target: "Return to base",
},
},
},
"Return to base": {
on: {
"reach base": {
target: "Wandering maze",
},
},
},
},
}
);
import { createActor } from 'xstate';
import { ghostMachine } from './ghostMachine';
const actor = createActor(ghostMachine);
actor.subscribe(state => {
console.log(state);
});
actor.start();
actor.send({ type: 'Lose Pac-Man' });
// { value: 'Chase Pac-Man', ... }
Users want
intelligent apps
Users want
intelligent apps
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11409459/66102bdd31d54c5c3d708350_TangoPin.webp)
import { z } from 'zod';
import { generateText, tool } from 'ai';
const result = await generateText({
model: openai('gpt-4-turbo'),
tools: {
weather: tool({
description: 'Get the weather in a location',
parameters: z.object({
location: z.string()
.describe('The location to get the weather for'),
}),
execute: async ({ location }) => {
const result = await getWeather(location);
return result;
},
}),
},
prompt: 'Is it going to rain in Amsterdam today?',
});
LLMs are
not enough
Non-deterministic
Not easily explainable
Confidently wrong
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371842/eat-rocks-google-ai-overviews.jpg.webp)
I had an idea.
What if we used LLMs
to navigate a state machine
and decide which event(s) to cause
to achieve a goal?
npm i @statelyai/agent@beta
(it's completely open-source)
What is an agent?
☑️ Performs tasks → accomplish goal
🔭 Observes → learn environment
⚠️ Receives feedback → improve over time
import { createAgent } from '@statelyai/agent';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';
import { todosMachine } from './todosMachine';
const agent = createAgent({
model: openai('gpt-4-turbo'),
name: 'todos',
events: {
'todo.add': z.object({ … }).describe('Adds a new todo'),
// …
}
});
// ...
const plan = await agent.decide({
goal,
state,
machine: todosMachine,
});
plan?.nextEvent;
// {
// type: 'todo.add',
// ...
// }
AGENT
npm i @statelyai/agent
Demo
Input
Output
Generative / creative
Input
Output
Goal
Start
Generative / creative
Agentic / goal-oriented
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11371921/CleanShot_2024-05-29_at_23.26.51_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11409192/Word_guesser.png)
Demo
Reinforcement learning (RL)
🤖 An intelligent agent learns
🔮 → through trial and error
💥 → how to take actions inside an environment
🌟 → to maximize a future reward
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444464/CleanShot_2023-05-03_at_10.40.47_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444468/skiing.gif)
RTFM to learn faster
tl;dr
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793022/CleanShot_2023-09-29_at_08.54.39_2x.png)
Environment
![](https://media3.giphy.com/media/cyMqOH8rjgDHG/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241657/CleanShot_2023-02-21_at_22.46.48_2x.png)
Normal mode
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241658/CleanShot_2023-02-21_at_22.47.00_2x.png)
Scatter mode
![](https://media4.giphy.com/media/gYWeVOiMmbg3kzCTq5/giphy.gif)
![](https://media4.giphy.com/media/hkqefnFjn2MWVl6xvq/giphy.gif)
![](https://media3.giphy.com/media/go3pCPP4899Jd3xb4p/giphy.gif)
Agent
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10242408/pac-man-path-screen-4.gif)
Policy
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10440552/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241672/4566657653_694d28a38a_b.jpeg)
Reward
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10440552/pasted-from-clipboard.png)
+1
-100
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10241672/4566657653_694d28a38a_b.jpeg)
🟡 + 🍒 =
🟡 + 👻 =
Credit assignment
+1
-100
Reinforcement learning
with human feedback (RLHF)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11409319/CleanShot_2024-06-14_at_11.10.51_2x.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11409320/CleanShot_2024-06-14_at_11.11.09_2x.png)
data
- Observed state transitions due to events;
causal relationship
- Past interactions with the agent;
human and assistant messages
- Previous planned sequences of events;
potential courses of action to reach a goal
- Value of past actions in plain language;
reward values if applicable
-
Observations
-
Messages
-
Plans
-
Feedback
@statelyai/agent
Creating intelligent agents
→ State machines
Determinism, explainability
→ Reinforcement learning
Exploration, exploitation
→ Large language models
Interpretation, creativity
Okay, let's do this.
import { getShortestPaths } from '@xstate/graph';
import { someMachine } from './someMachine';
// Finds all the shortest paths from
// initial state to other states
const shortestPaths = getShortestPaths(someMachine, {
events: [
{ type: 'setName', name: '' },
{ type: 'setName', name: 'reallyReallyLongNameProbablyDutch' },
{ type: 'setAge', age: -1 },
{ type: 'setAge', age: 30 },
// ...
]
});
GRAPH
npm i @xstate/graph
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10793329/pasted-from-clipboard.png)
Edge weight
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/11372329/CleanShot_2024-05-30_at_08.50.40_2x.png)
Espresso machine ☕️
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/10444504/CleanShot_2023-05-03_at_11.11.45_2x.png)
Demo
Learnings
LLMs are unpredictable
State machines are predictable
Declarative logic + LLMs + RL =
intelligent state management
🎲
🤖
✨
no matter which state management
library you use.
State machines are a great way to think about app logic
Dankjewel React Summit!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/174419/images/9674899/logomark-black-nobg.png)
React Summit 2024
David Khourshid · @davidkpiano
stately.ai
Making state management intelligent
By David Khourshid
Making state management intelligent
Creating intelligent state management with state machines and AI. Exploring the concept of using LLMs to navigate state machines for achieving goals. Discussing the role of agents in performing tasks, learning, and improving over time through reinforcement learning.
- 255