Real World Agents: Case Studies from LlamaIndex

Jason Liu webinar 2025-10-08

What are we talking about?

• What is LlamaIndex?
• What are document agents?
• Case studies
• Takeaways

What is LlamaIndex?

LlamaIndex core = RAG

LlamaIndex workflows = Agents

developers.llamaindex.ai

LlamaCloud

RAG as a service

cloud.llamaindex.ai

LlamaParse

• The best document parser in the world
• Free for 10,000 pages/month

Why use LlamaIndex and LlamaCloud?

• Solve real business problems
• Avoid technical boilerplate
• Focus on business value

What can you actually build?

AI agents

+

RAG systems

=

Document agents

RAG is essential to the enterprise

• LLMs need your data to do anything useful
• Context windows are limited
• Speed and cost are important

RAG by itself

is insufficient

• Semantic search has limits
• Fails at complex, multi-part questions
• Needs planning and comparison

RAG needs Agents, Agents need RAG

• RAG needs agents to be good
• Agents need RAG to be useful
• = Document Agents

Case studies #1

• Domain: Government construction RFPs
• $7 billion market in Poland
• 100+ page technical documents

What they were doing

• Hours or days per RFP
• Keyword search for discovery
• False positives and missed opportunities

What they built

• Agent workflow mimicking human process
• 20-30 page standardized reports
• Executive summaries, risk assessments, recommendations

Outcomes

• Semantic search finds more relevant RFPs
• 10 minutes average processing time
• 3 → 20-30 tenders per employee per day

Bullseye use case

for LLMs

Key takeaway #1

Workflows already existed

Key takeaway #2

Case studies #2

Pursuit

• Domain: Public sector business intelligence
• 90,000+ government entities
• Budgets, plans, transcripts

What they were doing

• Manual tracking across thousands of sites
• Impossible to stay current
• Painstaking manual review
• Most opportunities never discovered

What they built

• 4 million pages parsed in one weekend
• Extracts from tables, figures, scanned images
• Searchable and filterable across all entities

Outcomes

• 25-30% increase in accuracy
• Previously invisible opportunities now visible
• Entire public sector now searchable

Scale was outside human capability

Key takeaway #1

Accuracy across messy real-world documents

Key takeaway #2

Outcomes were well-defined

Key takeaway #3

Case studies #3

Scaleport AI

• Domain: Travel insurance claims
• Medical reports from around the world
• Handwriting, notes, wild formatting

What they were doing

• 20-40 minutes per claim
• Manual extraction by adjusters
• Hundreds of claims per month
• No way to scale without hiring

What they built

• LlamaParse for document extraction
• Multiple automated checks
• Structured analysis for adjuster review

Outcomes

• 20-40 minutes → 10 minutes
• 50-75% improvement
• 2x processing capacity
• Augmentation, not replacement

Parsing was critical

Key takeaway #1

Full-workflow system

Key takeaway #2

Human-AI collaboration

Key takeaway #3

Case studies #4

Arcee AI

• Domain: Academic research papers
• 4 million pages of NLP research
• Dense PDFs with equations, tables, charts

What they were doing

• Traditional OCR failed
• Tables mangled, equations lost
• Manual extraction impractical at scale

What they built

• LlamaParse with parsing instructions
• Iterative refinement through prompts
• High-fidelity structured extraction

Outcomes

• Minimal data loss
• Tables, equations, context preserved
• High-quality dataset in fraction of time
• Better model performance

More than just text

Key takeaway #1

Parsing instructions

Key takeaway #2

Production scale

Key takeaway #3

Case studies #5

11x.ai

• Domain: AI SDR onboarding
• Training AI agents on product offerings
• Multimodal content in multiple formats

What they were doing

• Manual onboarding per product
• Weeks or months for enterprise customers
• Evaluated building in-house pipeline
• Massive engineering effort, quality issues

What they built

• LlamaParse for multimodal ingestion
• Audio, PDFs, Word docs, web pages
• Automated knowledge extraction
• Campaign messaging generated at scale

Outcomes

• Weeks → days onboarding time
• Prototype → production in 3 days
• Immediate adoption after launch

Buy instead of build

Key takeaway #1

TypeScript support!

Key takeaway #2

Fine-grained control

Key takeaway #3

Document agents are domain specific

• Tons of unstructured data
• LlamaIndex engineered for this
• Not generic agents

Workflows are critical

• Direct reflection of business processes
• Know your business before automating
• Hard-code processes for reliability

Parsing, parsing, parsing

• Quality is paramount
• Makes the difference in outcomes
• Why we talk about LlamaParse all the time

Thank you!

You can follow me on BlueSky: @seldo.com