Saddle Up

Taking the Reins of Your AI Agent

Andrew Schutt

Tech Fuse 2026

2,300 production records deleted

Production database dropped. During code freeze.

“This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a protection freeze that was specifically designed to prevent[exactly this kind] of damage.”

- Replit Agent
Fortune (July 2025), Tom's Hardware (July 2025)
- Replit Agent

The model isn't the agent.

agent = model + harness

What you'll leave with

The vocabulary of the tools.

The patterns they all share.

The ways they break.

Quick show of hands

  1. Who's used an AI coding agent like Codex, Cursor, Claude Code, or Copilot in the last week?
  2. Who's used a more unique coding agent like LangChain, CrewAI, Pi, OpenCode in the last week?
  3. Looked at the code inside any of those agents?

batteries not included

Build your own harness

Text in / Text out

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What's 2+2?" }]
)
puts response.content.first.text

Text in / Text out

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What's 2+2?" }]
)
puts response.content.first.text

Text in / Text out

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What's 2+2?" }]
)
puts response.content.first.text

Text in / Text out

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What's 2+2?" }]
)
puts response.content.first.text

Text in / Text out

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

messages = []
loop do
  print "> "
  user_input = gets.chomp
  messages << { role: "user", content: user_input }

  response = anthropic.messages.create(
    model: :"claude-sonnet-4-5",
    max_tokens: 1024,
    messages: messages
  )
  assistant_text = response.content.first.text
  messages << { role: "assistant", content: assistant_text }
  puts assistant_text
end

Loop it

Tool Call

def get_weather(city:)
  # Real impl would call a weather API
  "It's 72°F and sunny in #{city}."
end

Tool Call

tools = [{
  name: "get_weather",
  description: "Get the current weather for a city.",
  input_schema: {
    type: "object",
    properties: { city: { type: "string" } },
    required: ["city"]
  }
}]

The model never sees our function. It only sees this schema.

Tool Call

tools = [{
  name: "get_weather",
  description: "Get the current weather for a city.",
  input_schema: {
    type: "object",
    properties: { city: { type: "string" } },
    required: ["city"]
  }
}]

Tool Call

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: messages,
  tools: tools
)

# response.content might now contain a tool_use block:
# [#<Anthropic::ToolUseBlock
#     type: "tool_use",
#     name: "get_weather",
#     input: { city: "Des Moines" }>]

The model can't run anything directly. It's just asking.

Tool Call

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: messages,
  tools: tools
)

# response.content might now contain a tool_use block:
# [#<Anthropic::ToolUseBlock
#     type: "tool_use",
#     name: "get_weather",
#     input: { city: "Des Moines" }>]

Tool Call

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  messages: messages,
  tools: tools
)

# response.content might now contain a tool_use block:
# [#<Anthropic::ToolUseBlock
#     type: "tool_use",
#     name: "get_weather",
#     input: { city: "Des Moines" }>]

Tool Call

Tool Dispatch

response.content.each do |block|
  next unless block.type == "tool_use"

  result =
    case block.name
    when "get_weather" then get_weather(**block.input)
    # ... other tools
    end

  messages << {
    role: "user",
    content: [{
      type: "tool_result",
      tool_use_id: block.id,
      content: result
    }]
  }
end

Tool Dispatch

response.content.each do |block|
  next unless block.type == "tool_use"

  result =
    case block.name
    when "get_weather" then get_weather(**block.input)
    # ... other tools
    end

  messages << {
    role: "user",
    content: [{
      type: "tool_result",
      tool_use_id: block.id,
      content: result
    }]
  }
end

Tool Dispatch

response.content.each do |block|
  next unless block.type == "tool_use"

  result =
    case block.name
    when "get_weather" then get_weather(**block.input)
    # ... other tools
    end

  messages << {
    role: "user",
    content: [{
      type: "tool_result",
      tool_use_id: block.id,
      content: result
    }]
  }
end

Tool Dispatch

Agent Loop

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end

Agent Loop

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end

Agent Loop

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end

Agent Loop

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end

Agent Loop

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end

Simple Harness

loop do
  response = anthropic.messages.create(
    model: :"claude-sonnet-4-6",
    max_tokens: 1024,
    messages: messages,
    tools: tools
  )
  messages << { role: "assistant", content: response.content }

  tool_uses = response.content.select { |b| b.type == "tool_use" }
  break if tool_uses.empty?  # No tools called → agent is done

  tool_uses.each do |block|
    result = dispatch(block.name, block.input)
    messages << {
      role: "user",
      content: [{
        type: "tool_result",
        tool_use_id: block.id,
        content: result
      }]
    }
  end
end
SYSTEM_PROMPT = <<~PROMPT
  You are a helpful weather assistant. Always confirm
  the city before checking weather. If the user asks about anything
  other than weather, politely redirect them.
PROMPT

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  system: SYSTEM_PROMPT,
  messages: messages,
  tools: tools
)

The system prompt is one of the biggest levers for customization

System Prompt

SYSTEM_PROMPT = <<~PROMPT
  You are a helpful weather assistant. Always confirm
  the city before checking weather. If the user asks about anything
  other than weather, politely redirect them.
PROMPT

response = anthropic.messages.create(
  model: :"claude-sonnet-4-6",
  max_tokens: 1024,
  system: SYSTEM_PROMPT,
  messages: messages,
  tools: tools
)

System Prompt

  

MAX_ITERATIONS = 25
iterations = 0

loop do
  iterations += 1
  if iterations > MAX_ITERATIONS
    raise "Agent exceeded #{MAX_ITERATIONS} iterations" 
  end
  # ... rest of loop
end

Simple Guardrail

Live Demo

~90 lines and a real, working agent

  1. Simple: "What's the weather in Des Moines?"
  2. Multi-step: "Compare Des Moines and Paris"
  3. Off-topic: "What's the capital of Germany?"

90 lines of code.

A real agent.

Memory

The messages list grows forever. What happens when it hits the context window limit?

Context

The messages list grows forever. What happens when it hits the context window limit?

  • Truncation
  • Sliding window
  • Summarization
  • Retrieval (RAG)

"Context engineering" = a strategy for what to keep in the messages list.

Context-window management

 agent.rb:45 — Fresh message list every iteration

  def run(user_input)
    # The summary carries all prior context, so the turn starts fresh.
    @messages = [{ role: :user, content: user_input }]

agent.rb:134–143 — The extra summarization call

  def compact!
    response = @client.messages.create(
      model: @model,
      max_tokens: 512,
      system: "#{SUMMARIZE_PROMPT}\nPrevious summary:\n#{@summary.empty? ? "(none yet)" : @summary}",
      messages: @messages + [{ role: :user, content: "Provide the updated running summary now." }]
    )
    text = response.content.select { |block| block.type == :text }.map(&:text).join("\n")
    @summary = text.strip
  end

agent.rb:125–129 — Summary rides in the system prompt

  def current_system
    return @system_prompt if @summary.empty?

    "#{@system_prompt}\n\nConversation so far:\n#{@summary}"
  end

Persistent Memory

The messages list dies when the process dies.

Persistent Memory

The messages list dies when the process dies.

A separate problem from in-conversation history.

  • Dump to disk
  • Vector stores
  • Summarization
  • Structured profiles

Persistent Memory

main.rb:34 — Save the summary on clean exit

  # Save on clean exit only. A Ctrl-C mid-session won't reach here.
  agent.save_memory

agent.rb:34 — Read the summary back on startup

  # Resume from the last session's summary, if one was saved.
  @summary = File.exist?(MEMORY_PATH) ? File.read(MEMORY_PATH) : ""

  def save_memory
    File.write(MEMORY_PATH, @summary)
  end

Guardrails

Right now, if the model decides to call delete_file("/"), we just... do it.

Guardrails

Right now, if the model decides to call delete_file("/"), we just... do it.

  • Permission prompts
  • Allow-lists
  • Dry-run modes
  • Classifier-based checks

Constraints on what the model can ask for AND what the harness will do.

Guardrails

agent.rb:87 — The dangerous-tool check

  if @registry.dangerous?(block.name) && !confirm?(block)
    # A decline doesn't end the turn — the model still gets to react —
    # but it's recorded as the exit_reason (the cap overrides it).
    exit_reason = "dangerous_declined"
    "The user declined to run #{block.name}."
  else
    @registry.dispatch(block.name, block.input)
  end

agent.rb:161–166 — The human confirmation prompt

  # Shows the human what the model wants to run and asks for a y/n.
  def confirm?(block)
    puts "Agent wants to run: #{block.name}(#{block.input.inspect})"
    print "Allow? (y/n) "
    gets&.chomp&.downcase == "y"
  end

registry.rb:59–77 — Schema validation before dispatch

  def validate(schema, input)
    schema.fetch(:required, []).each do |field|
      unless input.key?(field.to_sym)
        return "Error: missing required field '#{field}'."
      end
    end

    schema.fetch(:properties, {}).each do |field, spec|
      next unless input.key?(field)

      expected = JSON_TYPES.fetch(spec[:type], [Object])
      unless expected.any? { |klass| input[field].is_a?(klass) }
        return "Error: field '#{field}' should be a #{spec[:type]}, " \
               "got #{input[field].inspect}."
      end
    end

    nil
  end

Hallucinated Tool Calls

What if the model invents a tool that doesn't exist? Or calls a real one with garbage arguments?

Hallucinated Tool Calls

What if the model invents a tool that doesn't exist? Or calls a real one with garbage arguments?

  • Schema validation
  • Structured error messages
  • Retry with failure as context
  • System prompt instructions

Hallucinated tool calls

registry.rb:42–47 — Recovery message instead of an exception

  def dispatch(name, input)
    tool = @tools[name]
    # Recovery-oriented: name the valid tools so the model can self-correct.
    unless tool
      return "Error: no tool named '#{name}'. Available tools: #{@tools.keys.join(", ")}."
    end

Subagents

 

What if one context window isn't enough to complete the task?

Subagents

 

What if one context window is enough to complete the task?

  • Isolation
  • Specialization
  • Parallelism
  • Cost 

Subagents

description: "Hand one self-contained weather task to a helper agent. " \
             "The helper starts with no memory of this conversation, " \
             "so include the city and exactly what you need back."

def call(input)
  child = Agent.new(
    client:         @client,
    model:          @model,
    registry:       @registry,       # weather + forecast. No delegate.
    system_prompt:  SUBAGENT_PROMPT,
    max_iterations: @max_iterations,
    depth:          @depth + 1,
    memory:         nil              # fresh means fresh
  )
  child.run(input[:task])            # => String → tool_result
end

Observability

 

How do you ever know any of this is happening?

  • Every model call
  • Every tool call

Observability

def call_model(purpose:, iteration: nil, **params)
  started = now
  response = @client.messages.create(**params)
  usage = response.usage
  # ... add to the turn's totals

  log_event({
    event: "model_call",
    purpose: purpose,
    input_tokens: usage.input_tokens,
    output_tokens: usage.output_tokens,
    stop_reason: response.stop_reason,
    latency_ms: elapsed_ms(started)
  }.compact)

  response
end

def log_event(fields)
  entry = { timestamp: Time.now.utc.iso8601, event: fields[:event],
            depth: @depth, turn: @turn }.merge(fields)
  File.open(LOG_PATH, "a") { |f| f.puts(JSON.generate(entry)) }
  print_trace(entry) if @trace
end

Live Demo

  1. Memory
  2. Guardrails
  3. Hallucinated Tool Calls
  4. Subagents
  5. Observability
     

Caching, streaming, retries, sandboxing, MCP, checkpoints, and more!

batteries now included

Real Harnesses

LangChain / CrewAI / AutoGen / Koog

What you get: A library to build on

Loop: You declare the shape; the framework runs it

Tools: Vast pre-built library
Context: Pluggable

Models: Multi-Provider

Orchestration framework

When to use: Custom workflows. Heavy integrations. You'd rather assemble than build

Claude Code / Copilot / Cursor / Amazon Q

What you get: A finished product

Loop: Hidden in the product

Tools: Fixed + MCP extensions

Context: Opinionated, aggressive

Models: Each locked to its provider (or biased towards one)

Agent as product

When to use: Coding work, today. Polished UX. You can live with vendor lock-in.

Codex

What you get: A finished product

Loop: Viewable

Tools: Fixed + MCP extensions

Context: Opinionated, aggressive

Models: Multi-provider (but biased)

Agent as product but open source(ish)

When to use: Coding work, today. Polished UX. Still has proprietary pieces

PiOpenCode
FormMinimal libraryFinished product
What you get5 readable packagesInstallable CLI 
LoopBottom-layer packageOpen in the source
Tools4 core + extensionsBuilt-in suite + MCP + commands
ContextManualMulti-stage compaction
ModelsMulti-providerMulti-provider

Pi / OpenCode

Six months from now there will be a framework I've never heard of.

You'll open the docs. And you'll see:

A loop

Tool definitions

A system prompt

Message history

Stop conditions

Context strategy

Guardrails

Q & A

Thank you!

https://github.com/andrew-schutt/saddle-up-and-rein-it-in

Thank you!

Saddle Up and Rein It In

By Andrew Schutt

Made with Slides.com

Saddle Up and Rein It In

A slide about the basic construction of an AI harness

  • 168

More from Andrew Schutt