Making state management

Craft Conf 2024

David Khourshid · @davidkpiano
stately.ai

intelligent

stately.ai

stately.ai

stately.ai

import { createMachine } from "xstate";

export const ghostMachine = createMachine(
  {
    id: "Ghost 👻",
    initial: "Wandering maze",
    states: {
      "Wandering maze": {
        on: {
          "Lose Pac-Man": {
            target: "Chase Pac-Man",
          },
          "Pac-Man eats power pill": {
            target: "Run away from Pac-Man",
          },
        },
      },
      "Chase Pac-Man": {
        on: {
          "Pac-Man eats power pill": {
            target: "Run away from Pac-Man",
          },
        },
      },
      "Run away from Pac-Man": {
        on: {
          "power pill wears off": {
            target: "Wandering maze",
          },
          "eaten by Pac-Man": {
            target: "Return to base",
          },
        },
      },
      "Return to base": {
        on: {
          "reach base": {
            target: "Wandering maze",
          },
        },
      },
    },
  }
);
import { createActor } from 'xstate';
import { ghostMachine } from './ghostMachine';

const actor = createActor(ghostMachine);

actor.subscribe(state => {
  console.log(state);
});

actor.start();

actor.send({ type: 'Lose Pac-Man' });
// { value: 'Chase Pac-Man', ... }
  • Make API call to auth provider
  • Initiate OAuth flow
  • Ensure token is valid
  • Persist token as cookie
  • Redirect to logged in view

Given a user is logged out,

When the user logs in with correct credentials

Then the user should be logged in

cart

shipping

contact

payment

confirmation

CHECKOUT

NEXT

NEXT

ORDER

PAYPAL

BACK

BACK

CANCEL

Identify
logical flaws

cart

shipping

contact

payment

confirmation

CHECKOUT

NEXT

NEXT

ORDER

PAYPAL

As a user, when I'm in the cart and I click the checkout button, I should be on the shipping page.

cart

shipping

contact

payment

confirmation

CHECKOUT

NEXT

NEXT

ORDER

PAYPAL

As a user, when I'm in the cart and I checkout via PayPal, I should be taken directly to the payment screen.

cart

shipping

contact

payment

confirmation

CHECKOUT

NEXT

NEXT

ORDER

PAYPAL

Shortest path

confirmation state

cart

shipping

contact

payment

confirmation

CHECKOUT

NEXT

NEXT

ORDER

PAYPAL

Shortest path

confirmation state where
shipping address is provided

import { getShortestPaths } from '@xstate/graph';
import { someMachine } from './someMachine';

// Finds all the shortest paths from
// initial state to other states
const shortestPaths = getShortestPaths(someMachine);

GRAPH

npm i @xstate/graph

But times are changing.

Users want
intelligent apps

LLMs are
not enough

Non-deterministic

Not easily explainable

Confidently wrong

Input

Output

Generative / creative

Input

Output

Goal

Start

Generative / creative

Task / goal-oriented

What is an agent?

Perform tasks → accomplish goal

Code demo
npm i @statelyai/agent

tl;dr: RTFM to learn faster

Creating intelligent agents

→ State machines

Determinism, explainability

→ Reinforcement learning

Exploration, exploitation

→ Large language models

Interpretation, creativity

Reinforcement learning (RL)

🤖 An intelligent agent learns

🔮 ... through trial and error

💥 ... how to take actions inside an environment

🌟 ... to maximize a future reward

State machines that can learn

Environment

Normal mode

Scatter mode

Agent

Agent

Policy

Reward

+10

-3

🟡 + 🍒 = 👍

🟡 + 👻 = 👎

Credit assignment

Exploration

vs Exploitation

no reward

Do nothing

Owner has treat
Tells dog "down"

ENVIRONMENT

Nothing changes

ENVIRONMENT

Exploration

reward++

Go down

Get treat

Owner has treat
Tells dog "down"

ENVIRONMENT

Owner praises dog

ENVIRONMENT

Exploration

Go down

Owner has treat
Tells dog "down"

ENVIRONMENT

Owner praises dog

ENVIRONMENT

Exploitation

💭

Treat?

Run away

Owner has treat
Tells dog "down"

ENVIRONMENT

Undesired outcome

ENVIRONMENT

🤬

Exploration

"Down"

Reward 🦴

Reinforcement learning
with human feedback (RLHF)

LLM Context:

  • Observed state transitions due to events;
    causal relationship
  • Past interactions with the agent;
    human and assistant messages
  • Previous planned sequences of events;
    potential courses of action
  • Value of past actions in plain language;
    reward values if applicable
  • Observations
     
  • History
     
  • Plans
     
  • Feedback
     

Goal prompt +

State machines for
intelligent logic

Learnings

LLMs are unpredictable

Event-based logic makes our apps
more declarative

State machines & reinforcement learning
make LLM agents more intelligent/predictable

Future ideas

🔭   State machine synthesis

🤖   Adaptive UIs with RL

🌌   Deep learning for state space reduction

intelligent

Make your state management

Declarative UIs

Declarative state management

Separate app logic from view logic.

LLMs

Reinforcement learning

Use as minimally as possible.

Köszönöm Craft Conf!

Craft Conf 2024

David Khourshid · @davidkpiano
stately.ai

Making state management intelligent - Craft Conf

By David Khourshid

Making state management intelligent - Craft Conf

This presentation introduces stately.ai and explores the limitations of LLMs. It discusses the concept of intelligent agents, state machines, reinforcement learning, and the use of large language models. It also covers credit assignment, exploration, and reinforcement learning with human feedback.

  • 152