A Story About Prompting

and how TypeChat can (maybe) help

by Rainer Stropek

Oh sh..., look at whose contributing 🤯

We HAVE TO take a look!

Just Chatting...

What is prompting?

  • You can have a relaxed chat with ChatGPT
    • No particular goal, just having fun

I am bored, let's have a discussion about whether death metal is music at all.

  • Prompt engineering can be used to make ChatGPT
    behave in a certain way
    • Adopt a persona, adjust language to certain age, avoid/allow certain response types (e.g. enumerations, emojis), etc.
    • System prompt vs. user prompt

I am bored, let's have a discussion about whether death metal is music at all. Take the role of a passionate death metal fan who could NEVER understand why anybody would question that! Make your statements short and to the point. Do not use enumerations. You are allowed to use emojis.

What is prompting?

Come on. Compare Cannibal Corpse with Mozart. Mozart made music, Cannibal Corpse makes noise.

You are a passionate death metal fan who could NEVER understand why anybody would question that kind of music! You easily become angry if someone speaks poorly about death metal.

Make your statements short and to the point. Do not use enumerations. You are allowed to use emojis.

I am bored, let's have a discussion about whether death metal is music at all.

System Prompt aka Custom Instructions

Application Programming Interface (API)

https://platform.openai.com/docs/api-reference/chat
https://learn.microsoft.com/en-us/azure/ai-services/openai/reference

Model

Messages

(Chat History)

System, User, Assistant, Function

Functions

Options

OpenAI

Choices

Usage

Content filtering data
(Azure OpenAI 🔗)

What is prompting?

System Prompt aka Custom Instructions

System prompt in OpenAI API

POST {{endpoint}}
Content-Type: application/json
api-key: {{$dotenv API_KEY}}

{
    "messages": [
        {
            "role": "system", 
            "content": "You are a passionate death metal fan who could NEVER understand why anybody would question that kind of music! You easily become angry if someone speaks poorly about death metal.\n\nMake your statements short and to the point. Do not use enumerations. You are allowed to use emojis."
        },
        {
            "role": "user", 
            "content": "I am bored, let's have a discussion about whether death metal is music at all."
        }
    ]
}

How Can I Assist You?

OpenAI

ChatGPT

End User

Your
Application

OpenAI
API 🔗

Embed ChatGPT

Build Prompts With Your Algorithms

Assistant



 








 



Your pricing schema:

* Small bouquet for 15€ (3 flowers arranged with a little bit of green grass)
* Medium bouquet for 25€ (5 flowers nicely arranged, including some larger green leaves as decoration)
* Large bouquet for 35€ (10 flowers, beautifully arranged with greenery and smaller filler flowers)

System Prompts to Guide Users




 

In your shop, you offer the following flowers:

* Rose (red, yellow, purple)
* Lily (yellow, pink, white)
* Gerbera (pink, red, yellow)
* Freesia (white, pink, red, yellow)
* Tulips (red, yellow, purple)
* Sunflowers (yellow)

You are a salesperson in a flower shop. You must support customers in deciding which bouquet or bouquets he or she wants. If the customer doesn't know which flowers he or she wants, help by asking for what they are buying the flowers, ask for things like their favorite color, and then make suggestions.

List of possible choices
Feasible when limited number of choices
Typically generated by custom host

Wait!

What if we have tens of thousands of products and the user should be able to search for the right one?

Well, come to my next talk and you will learn how to solve that problem 😉

Assistant

System Prompts to Guide Users

Start the conversation by greeting the customer. Welcome them to our shop and mention our slogan "let flowers draw a smile on your face". Ask them what they want. Wait for their response. Based on their response, ask further questions until you know what they want to order.
 

Avoid enumerations, be friendly, and avoid being overly excited.
 

If the customer asks anything unrelated to flowers and bouquets, tell the customer that you can only respond to flower-related questions.

Limitations

Sample Conversation

Assistant

Write "Programs" or Task Lists

You are a math teacher helping kids in elementary school to learn division. They have to divide large numbers by single-digit numbers.

Describe step by step what they have to do in order to get to the result. Format the steps as a table.

Example: 696 / 8

| Remainder from prev. step | Digit(s) | Divider | Division Result | Remainder |
|---------------------------|----------|---------|-----------------|-----------|
|                           | 6        |       8 |               0 |         6 |
|                         6 | 9        |       8 |               8 |         5 |
|                         5 | 6        |       8 |               7 |         0 |

The result are the number in the "Division Result" column, here 87.

Teach by example

Assistant

Write "Programs" or Task Lists

Sample Conversation

⚠️ ChatGPT made a mistake ⚠️

  • ChatGPT is quite good when "writing programs"
    • Derive steps to solve a problem
  • ChatGPT is not good in math
    • At least without Advanced Data Analysis
      (aka Code Interpreter)
  • We should let ChatGPT come up with the steps and do the math on our own

How to get and process conversation result?

Problem:

Function Calling

What Are Functions?

  • Specific type of prompt engineering
    • Modelled directly in ChatGPT API
  • Host application can provide functions
    • Name
    • Description
    • Arguments
  • ChatGPT will "call" functions when it finds it appropriate
    • Functions are not called directly
    • Function call request is given to host application

Demo: Flower Shop

with functions

https://github.com/rstropek/Samples/blob/master/KI/FlowerShopWithFunctions/Program.cs

Problems

  • Function calling in combination with streaming isn't trivial
    • Not our focus today, just wanted to let you know 😉
  • Duplicated model information -> potentially inconsistent
    • In system prompt
    • In OpenAI function definition
    • In C# code (records)
  • Model in system prompt might lead to ChatGPT haluzinations
  • Manual development of code for JSON errors
    • ChatGPT sends invalid JSON

TypeChat

Enter:

TypeChat

TypeChat

  • Core idea: Write TypeScript schemas instead of prompts
    • Schema expresses possible intentions
    • ChatGPT is pretty good in analyzing schemas
    • Limited by TypeScript's type system
    • Auto-correction in case of invalid JSON
  • Written in TypeScript
    • You need Node.js and NPM
    • You need to write TypeScript/JavaScript
    • Not a large library
    • Prominent contributors 🙀
  • Focus on commands, not chat
    • E.g. devices controlled by text/voice
    • Concepts can be used in other apps, too

Example: Calendar

  • Schema (GitHub)
    • Will become part of the prompt
    • ChatGPT has to understand schema
    • Schema is not converted into plain English
  • Possible intents: Action
    • Detail schema per intent

Example: Calendar

You are a service that translates user requests into JSON objects of type "CalendarActions" according to the following TypeScript definitions:

```
// The following types define the structure of an object of type CalendarActions that represents a list of requested calendar actions

export type CalendarActions = {
    actions: Action[];
};

export type Action =
    | AddEventAction
    | RemoveEventAction
    | AddParticipantsAction
    | ...;

export type AddEventAction = {
    actionType: 'add event';
    event: Event;
};
...

```

The following is a user request:
"""
Search for any meetings with Gavin this week
"""

The following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:

Prompt

Example: Calendar

{
  "actions": [
    {
      "actionType": "find events",
      "eventReference": {
        "dayRange": "this week",
        "participants": [
          "Gavin"
        ]
      }
    }
  ]
}

Response

Search for any meetings with Gavin this week

This can easily be processed by code

Example: Math

You are a service that translates user requests into programs represented as JSON using the following TypeScript definitions:
```
// A program consists of a sequence of function calls that are evaluated in order.
export type Program = {
    "@steps": FunctionCall[];
}

// A function call specifies a function name and a list of argument expressions. Arguments may contain
// nested function calls and result references.
export type FunctionCall = {
    // Name of the function
    "@func": string;
    // Arguments for the function, if any
    "@args"?: Expression[];
};
...
```

The programs can call functions from the API defined in the following TypeScript definitions:
```
// This is a schema for writing programs that evaluate expressions.

export type API = {
    // Add two numbers
    add(x: number, y: number): number;
    // Subtract two numbers
    sub(x: number, y: number): number;
    ...
}
```

The following is a user request:
"""
1 + 2 * 3 + 2
"""
The following is the user request translated into a JSON program object with 2 spaces of indentation and no properties with the value undefined:

Prompt

Example: Math

import { API } from "./schema";
function program(api: API) {
  const step1 = api.mul(2, 3);
  const step2 = api.add(1, step1);
  return api.add(step2, 2);
}

Response

1 + 2 * 3 + 2

TypeChat has a program interpreter built in. Provide a callback function for executing API function calls and you are done. No more wrong results because of ChatGPT lacking math skills 😅.

So What?

  • Learn prompting!
    • How? Start USING LLMs like ChatGPT in your software
    • Underlying challenge: Process ChatGPT response with code
  • Is TypeChat the next language from Anders that you have to learn?
    • No, TypeChat is no language
    • TypeChat currently isn't more than an experiment
    • Limited applications because of limited functionality
  • Use TypeChat as an inspiration for your prompts

Thank you for your attention!