Chris Price & Fanis Vlachos
(Nightmare)
LLM
Booking data extractor
Email sender
Booking data extractor
Email Sender
LLM
+
Evaluation Strategy
LLM
Booking data extractor
Email Sender
LLM
Generate itinerary
Draft
prompts: 'Create a trip itinerary using the following details: {{trip_plans}}. The itinerary should be formatted as a JSON string.'
providers:
- openai:gpt-4
tests:
- description: "Output is a valid JSON string"
vars:
trip_plans: |
{
"name": "John Doe",
"destination": "New York",
"arrival": "2024-08-01",
"departure": "2024-08-04",
}
assert:
- type: is-json
prompts:
- "You are working for a travel agency. Based on this trip itinerary {{itinerary}}, write an email to the customer, presenting the suggested trip plan. The items on the itinerary should be formatted in a list. With each item on the list, invite the customer to use our booking services."
# ...
tests:
- description: "Booking services are suggested for each item on the list"
assert:
- type: llm-rubric
value: "Does each item on the list promote the booking services?"
- description: "All items from the itinerary are included in the email"
assert:
- type: javascript
value: file://./emailContainsAllItems.js
Chris Price & Fanis Vlachos