How to win a hackathon with
NexHacks Workshop 2026-01-14
Thank you for showing up
What are we talking about today?
- What is observability?
- What are evaluations?
- How do I use evals to optimize my agent?
- Demo
Arize Prize
$1,000 cash for Best Use of Arize
Assumption:
you're building
an agent
What is observability?
Observability = rich, structured logging
Automatic integrations
TypeScript
- BeeAI
- LangChain
- Mastra
- Vercel AI
Python
- Agno
- Autogen
- CrewAI
- DSPy
- Google ADK
- Graphite
- Haystack
- Instructor
- LlamaIndex
- LangChain
- LangGraph
- NVIDIA
- Portkey
- Pydantic AI
Our example agent
tracer_provider = register(
project_name="crewai-tracing-quickstart",
auto_instrument=True
)Phoenix Cloud
Tracing
Congratulations!
You can win $1k now
What are evaluations?
Unit testing for nondeterministic outputs
Solution:
LLM as a judge
Code evaluations
are also possible
Evaluations
Evaluation annotations
Evaluations
save time
Datasets
Experiments
Experiment results
Demo time!
Thank you!
These slides:
The code again:
I'm on BlueSky:
🦋 @seldo.com
How To Win NexHacks With Arize
By Laurie Voss
