Mind Reading With

Intelligent & Adaptive UIs

@davidkpiano

Everyone is
different.

A/B tests

are ineffective.

User flows are
seldom linear.

github.com/guess-js/guess

📄

Adaptive user interfaces (AUI)

Adapt to people,
not just devices

📨 Inbox

🆕 VIEW LATEST

✉️ New Email

💾 Save as Draft

📬 Send

100%

40%

60%

10%

90%

Delete

Are you sure?

Yes

Delete

Undo

Deleted!

👩‍💼

🙍‍♂️

🙋‍♂️

Developing AUIs
is challenging.

Source: Current Challenges and Applications for Adaptive User Interfaces

Model-driven development

Source: https://dribbble.com/shots/6206737-Early-stage-flowcharts-for-Mobile-users

Finite state machines

have one initial state
a finite number of states
a finite number of events
a mapping of state transitions
triggered by events
a finite number of final states

Signing in...

Profile

Home

SUCCESS

UNAUTHORIZED

PROFILE

Signed Out

SIGN OUT

Signed Out

Define transitions between
states & events

function searchReducer(state = 'idle', event) {
  switch (state) {
    case 'idle':
      switch (event.type) {
        case 'SEARCH':
          return 'searching';
        default:
          return state;
      }
    case 'searching':
      switch (event.type) {
        case 'RESOLVE':
          return 'success';
        case 'REJECT':
          return 'failure';
        default:
          return state;
      }
    case 'success':
      switch (event.type) {
        case 'SEARCH':
          return 'searching';
        default:
          return state;
      }
    case 'failure':
      switch (event.type) {
        case 'SEARCH':
          return 'searching';
        default:
          return state;
      }
    default:
      return state;
  }
}

FSMs with switch/case

const machine = {
  id: 'feedback',
  initial: 'question',
  states: {
    question: {
      on: {
        CLICK_GOOD: 'thanks',
        CLICK_BAD: 'form',
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    form: {
      on: {
        SUBMIT: 'thanks',
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    thanks: {
      on: {
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    closed: {
      type: 'final'
    }
  }
};

Define transitions between
states & events

const transition = (state, event) => {
  return machine
    .states[state] // current state
    .on[event]     // next state
    || state;      // or same state
}

FSMs with object mapping

import { Machine } from 'xstate';

const feedbackMachine = Machine({
  id: 'feedback',
  initial: 'question',
  states: {
    question: {
      on: {
        CLICK_GOOD: 'thanks',
        CLICK_BAD: 'form',
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    form: {
      on: {
        SUBMIT: 'thanks',
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    thanks: {
      on: {
        CLOSE: 'closed',
        ESC: 'closed'
      }
    },
    closed: {
      type: 'final'
    }
  }
});

github.com/davidkpiano/xstate

npm install xstate

xstate.js.org

import { feedbackMachine } from './feedbackMachine';
import { useMachine } from '@xstate/react';

const App = () => {
  // const [state, dispatch] = useReducer(feedbackReducer, 'question');
  const [current, send] = useMachine(feedbackMachine);

  if (current.matches('question')) {
    return (
      <QuestionScreen
        onClickGood={() => send({ type: 'GOOD' })}
        onClickBad={() => send({ type: 'BAD' })}
        onClose={() => send({ type: 'CLOSE' })}
      />
    );
  } else if (/* ... */) {
    // ...           
  }
}

Statecharts
(Hierarchical State Machines)

Statecharts: A Visual Formalism for Complex Systems (David Harel, 1987)

Statecharts

Idle

Searching...

entry / prefetchResources

entry / fetchResults

Actions - onEntry, onExit, transition
Guards - conditional transitions

[query.length > 0]

Statecharts

Actions - onEntry, onExit, transition
Guards - conditional transitions
Hierarchy - nested states
Orthogonality - parallel states
History - remembered states

Idle

Searching...

Success

Failure

RESOLVE

REJECT

Searched

Hierarchical states

Idle

Searching...

Success

Failure

RESOLVE

REJECT

Searched

Hierarchical states

const machine = Machine({
  initial: 'idle',
  states: {
    // ...
    searching: {
      on: {
        RESOLVE: 'searched',
        REJECT: 'searched.failure'
      }
    },
    searched: {
      initial: 'success',
      states: {
        success: {},
        failure: {}
      },
      on: { SEARCH: 'searching' }
    }
  }
});

How was your experience?

𝘅

Good

Bad

Tell me why?

𝘅

Submit

Thanks for your feedback.

𝘅

Ain't nothin' but a heartache

https://codepen.io/davidkpiano/pen/dc81af7260581c1fbbf5b5154caa2228

Abstract model
Transition analytics
Identify adaptive paths
Use analysis for adaptation

Game plan

Photo by NEW DATA SERVICES on Unsplash

Success

Signing in

Error

0.9

0.1

Weighted graphs

State machines in

VS Live Share

Signed out

Signing in

Signed in

Sharing...

Shared

Joining...

Joined

share Success

Join Success

Join

Leave

End Collab session

How was your experience?

𝘅

Good

Bad

Tell me why?

𝘅

Submit

Thanks for your feedback.

𝘅

Give Feedback

❓

Ain't nothin' but a mistake

Gallery

Profile

Camera

SUCCESS

TAP PROFILE

TAP CAMERA

BACK

Scroll

Analysis of transitions

0.90, 0.31

0.90, 0.57

0.90, 0.12

0.90

0.31

0.57

0.13

UNAUTHORIZED

0.10

SUCCESS

(decide)

TAP AD

Analysis of events

+10

Gallery

Ads interspersed

Ads shown at top

Scroll

TAP AD

+10

-1

-10

INTERSPERSED > .5

TOP > .5

Decision trees

Contextual data

Registration date
Number of friends
Posting frequency
Location
...etc.

New user?

< 30 days

>= 30 days

Show ad at top

Friend count?

< 100

>= 100

Show more ads

Show less ads

More info: Towards Data Science -
Decision Trees in Machine Learning

All data

13%

79%

Prediction via
shortest paths

Shortest path algorithms (Dijkstra, Bellman-Ford, A* search, etc.)
Analytics provides weights
Represents happy paths
Can be automatically generated

import { Machine } from 'xstate';

const myMachine = Machine({
  // ...
});

const myService = interpret(myMachine)
  .onTransition(state => {
    analytics.track({
      target: state.value,
      source: state.history?.value,
      event: sanitize(event),
      timestamp: Date.now()
    });
  })
  .start();

Source

Target

Event

import { Machine } from 'xstate';
import { getShortestPaths } from '@xstate/graph';

const myMachine = Machine({
  // ...
});

const shortestPaths = getShortestPaths(myMachine);

```
A: A
```
```
B: A -> B
```
```
C: A -> B -> C
```
```
D: A -> D
```
```
E: A -> D -> E
```

import { Machine } from 'xstate';
import { getShortestPaths } from '@xstate/graph';

const myMachine = Machine({
  // ...
});

const shortestPaths = getShortestPaths(myMachine);

0.9

1.0

0.4

0.6

0.1

0.4

0.6

```
A: A
```
```
B: A -> D -> E -> B
```
```
C: A -> D -> E -> C
```
```
D: A -> D
```
```
E: A -> D -> E
```

0.75

0.25

(D) 0.95

(D) 0.05

A -> B
A -> C
A -> B
A -> B

E ->
D ->
E ->
F ->

Higher-order Markov models

Variable-order Markov models (Wikipedia)

Deep Reinforcement
Learning

Agent

Environment

Action

State

Reward

Deep Reinforcement Learning

service.onTransition(state => {
  analytics.track({
    source: state.history.value,
    target: state.value,
    context: state.context,
    event: state.event,
    timestamp: Date.now()
  })
});

Application

Executable Model

Metrics Tracking

Adaptive Transitions

Adaptive UI

➡️ Determinism

📈 Visualization

👩‍💻 DEV Experience

🤝 Communication

📊 Analytics

👩‍🔬 Simulation

🔬 Testability

👨‍👩‍👧‍👦 User Experience

🚚 Prevent Information Overload

💡 Provide intuitive guidance

✅ Understand Tasks & Goals

⚙️ Create Personalized Systems

Adaptive User Interfaces

🔀 Micro vs. Macro Adaptivity

⚙️ Prediction vs. Feedback

🌏 Local vs. Global

📊 Data vs. Simulation

Modeling considerations

Advantages

of using statecharts

Visualized modeling
Precise diagrams
Automatic code generation
Comprehensive test coverage
Accommodation of late-breaking requirements changes

Statechart Autocoding for Curiosity Rover

xstate.js.org/viz

xstate

@xstate/react

@xstate/graph

@xstate/vue

@xstate/test

@xstate/analytics

@xstate/sim

@xstate/viz

✅

🔜

xstate.js.org/docs

statecharts.github.io

spectrum.chat/statecharts

✅

The world of statecharts

statecharts.github.io

Spectrum community

spectrum.chat/statecharts

XState Docs

xstate.js.org/docs

Make your code do more.

Thank you Nordic.JS!

@davidkpiano

🔮

Mind Reading With

Intelligent & Adaptive UIs

Everyone is different.

A/B tests

are ineffective.

User flows are seldom linear.

Adaptive user interfaces (AUI)

Adapt to people, not just devices

📨 Inbox

🆕 VIEW LATEST

✉️ New Email

💾 Save as Draft

📬 Send

Developing AUIs is challenging.

Model-driven development

Finite state machines

FSMs with switch/case

FSMs with object mapping

Statecharts(Hierarchical State Machines)

Statecharts

Statecharts

Hierarchical states

Hierarchical states

Game plan

Weighted graphs

State machines in

VS Live Share

Analysis of transitions

Analysis of events

Decision trees

Prediction via shortest paths

Higher-order Markov models

Deep Reinforcement Learning

Deep Reinforcement Learning

➡️ Determinism

📈 Visualization

👩‍💻 DEV Experience

🤝 Communication

📊 Analytics

👩‍🔬 Simulation

🔬 Testability

👨‍👩‍👧‍👦 User Experience

🚚 Prevent Information Overload

💡 Provide intuitive guidance

✅ Understand Tasks & Goals

⚙️ Create Personalized Systems

Adaptive User Interfaces

🔀 Micro vs. Macro Adaptivity

⚙️ Prediction vs. Feedback

🌏 Local vs. Global

📊 Data vs. Simulation

Modeling considerations

Advantages

of using statecharts

xstate.js.org/viz

The world of statecharts

statecharts.github.io

Spectrum community

spectrum.chat/statecharts

XState Docs

xstate.js.org/docs

Make your code do more.

Thank you Nordic.JS!

Everyone is
different.

User flows are
seldom linear.

Adapt to people,
not just devices

Developing AUIs
is challenging.

Statecharts
(Hierarchical State Machines)

Prediction via
shortest paths

Deep Reinforcement
Learning