Making state management

React Summit 2024

David Khourshid · @davidkpiano
stately.ai

intelligent ✨

Why David likes

React Summit 2024

David Khourshid · @davidkpiano
stately.ai

state machines ✨

Making state management

so much

stately.ai

stately.ai

stately.ai

function Todos(props) {
  const [todos, setTodos] = useState(props.todos)
  const [selectedTodo, setSelectedTodo] = useState(null)
  
  // ...
}


function Todo(props) {
  const [title, setTitle] = useState(props.todo.title)
  const [content, setContent] = useState(props.todo.content)
  const [completed, setCompleted] = useState(props.todo.completed)
  
  useEffect(() => {
    props.onUpdate({ title, content, completed })
  }, [title, content, completed]);
  
  // ...
}
import { createStore } from '@xstate/store';

const store = createStore(
  {
    todos: [],
    selectedTodo: null
  },
  {
    addTodo: {
      todos: (ctx, { todo }) => ctx.todos.concat(todo)
    },
    updateTodo: {
      todos: (ctx, { todo }) =>
        ctx.todos.map((t) => (t.id === todo.id ? todo : t))
    },
    selectTodo: {
      selectedTodo: (ctx, { todoId }) => todoId
    },
    toggleTodo: {
      todos: (ctx, { todo }) =>
        ctx.todos.map((t) =>
          t.id === todo.id ? { ...t, completed: !t.completed } : t
        )
    }
  }
);

STORE

npm i @xstate/store
import { useSelector } from '@xstate/store/react';
import { store } from './store';

function App() {
  const todos = useSelector(store, (s) => s.context.todos);

  return (
    // ...
    <button onClick={() => {
      store.send({ type: 'addTodo', todo: {/* ... */} })
    }}>
      New todo
    </button>
  );
}

Let's talk about AI.

// /api/completion/route.ts
import { StreamingTextResponse, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { prompt }: { prompt: string } = await req.json();

  const result = await streamText({
    model: openai('gpt-4-turbo'),
    prompt,
  });

  return new StreamingTextResponse(result.toAIStream());
}












import { useCompletion } from '@ai-sdk/react';

function TodoDescription() {
  const {
    completion,
    input,
    handleInputChange,
    handleSubmit
  } = useCompletion({
    api: '/api/completion',
  });

  return (
    <form onSubmit={handleSubmit}>
      <input
        name="prompt"
        value={input}
        onChange={handleInputChange}
        id="input"
      />
      <textarea value={completion} />
      <button type="submit">Submit</button>
    </form>
  );
}





Let's talk about really old AI.

import { createMachine } from "xstate";

export const ghostMachine = createMachine(
  {
    id: "Ghost 👻",
    initial: "Wandering maze",
    states: {
      "Wandering maze": {
        on: {
          "Lose Pac-Man": {
            target: "Chase Pac-Man",
          },
          "Pac-Man eats power pill": {
            target: "Run away from Pac-Man",
          },
        },
      },
      "Chase Pac-Man": {
        on: {
          "Pac-Man eats power pill": {
            target: "Run away from Pac-Man",
          },
        },
      },
      "Run away from Pac-Man": {
        on: {
          "power pill wears off": {
            target: "Wandering maze",
          },
          "eaten by Pac-Man": {
            target: "Return to base",
          },
        },
      },
      "Return to base": {
        on: {
          "reach base": {
            target: "Wandering maze",
          },
        },
      },
    },
  }
);
import { createActor } from 'xstate';
import { ghostMachine } from './ghostMachine';

const actor = createActor(ghostMachine);

actor.subscribe(state => {
  console.log(state);
});

actor.start();

actor.send({ type: 'Lose Pac-Man' });
// { value: 'Chase Pac-Man', ... }

Users want
intelligent apps

Users want
intelligent apps

import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: openai('gpt-4-turbo'),
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string()
          .describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => {
        const result = await getWeather(location);
        return result;
      },
    }),
  },
  prompt: 'Is it going to rain in Amsterdam today?',
});











LLMs are
not enough

Non-deterministic

Not easily explainable

Confidently wrong

I had an idea.

What if we used LLMs

to navigate a state machine

and decide which event(s) to cause

to achieve a goal?

npm i @statelyai/agent@beta

(it's completely open-source)

What is an agent?

☑️ Performs tasks → accomplish goal

🔭 Observes → learn environment 

⚠️ Receives feedback → improve over time

import { createAgent } from '@statelyai/agent';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';
import { todosMachine } from './todosMachine';

const agent = createAgent({
  model: openai('gpt-4-turbo'),
  name: 'todos',
  events: {
    'todo.add': z.object({ … }).describe('Adds a new todo'),
    // …
  }
});

// ...

const plan = await agent.decide({
  goal,
  state,
  machine: todosMachine,
});

plan?.nextEvent;
// {
//   type: 'todo.add',
//   ...
// }











AGENT

npm i @statelyai/agent
Demo

Input

Output

Generative / creative

Input

Output

Goal

Start

Generative / creative

Agentic / goal-oriented

Demo

Reinforcement learning (RL)

🤖 An intelligent agent learns

🔮 → through trial and error

💥  how to take actions inside an environment

🌟  to maximize a future reward

RTFM to learn faster

tl;dr

Environment

Normal mode

Scatter mode

Agent

Policy

Reward

+1

-100

🟡 + 🍒 =

🟡 + 👻 = 

Credit assignment

+1

-100

Reinforcement learning

with human feedback (RLHF)

data

  • Observed state transitions due to events;
    causal relationship
     
  • Past interactions with the agent;
    human and assistant messages
     
  • Previous planned sequences of events;
    potential courses of action to reach a goal
     
  • Value of past actions in plain language;
    reward values if applicable
  • Observations

     
  • Messages

     
  • Plans

     
  • Feedback
     
@statelyai/agent

Creating intelligent agents

→ State machines

Determinism, explainability

→ Reinforcement learning

Exploration, exploitation

→ Large language models

Interpretation, creativity

Okay, let's do this.

import { getShortestPaths } from '@xstate/graph';
import { someMachine } from './someMachine';

// Finds all the shortest paths from
// initial state to other states
const shortestPaths = getShortestPaths(someMachine, {
  events: [
    { type: 'setName', name: '' },
    { type: 'setName', name: 'reallyReallyLongNameProbablyDutch' },
    { type: 'setAge', age: -1 },
    { type: 'setAge', age: 30 },
    // ...
  ]
});

GRAPH

npm i @xstate/graph

Edge weight

Espresso machine ☕️

Demo

Learnings

LLMs are unpredictable

State machines are predictable

Declarative logic + LLMs + RL =

intelligent state management

🎲

🤖

no matter which state management
library you use.

State machines are a great way to think about app logic

Dankjewel React Summit!

React Summit 2024

David Khourshid · @davidkpiano
stately.ai

Making state management intelligent

By David Khourshid

Making state management intelligent

Creating intelligent state management with state machines and AI. Exploring the concept of using LLMs to navigate state machines for achieving goals. Discussing the role of agents in performing tasks, learning, and improving over time through reinforcement learning.

  • 255