Building a Production-Grade AI Agent Harness
From multi-agent frameworks to a minimal agent harness — what we tried, what broke, and what we shipped.
Valiance Health
Data foundation
Building a data foundation for value-based care.
Unified sources
Billing systems, clinical registries, EHRs, claims, and reporting tools — in one place.
Val
The AI-powered healthcare analyst layer on top of that foundation.
The journey to agentic
We saw AI agents everywhere. One pattern stood out — and we wanted it in our own environment.
The pattern that stood out
- AI agents were showing up everywhere.
- The one that stood out: the internal SQL agent.
- Instead of asking analysts for every query, users could ask questions directly.
- So we set out to replicate that idea in our own environment.
Where we started
Simple LLM usage
Chat completions on OpenAI and Groq. Mostly prompt engineering — nothing agentic yet.
Learn the business
Everything was fresh and new. I started exploring automation and really learning the healthcare business.
What users actually needed
- Users are analytics, back-office, and admin heavy.
- They want everything automated — which is kinda impossible, btw.
- They spent a lot of time just making sense of the cost.
Phase 1 — exploring frameworks
Five contenders, one set of hard requirements. Only one survived contact with production.
What we needed from a framework
Human in the loop
HITL as a first-class concept, not a bolt-on.
Checkpointing
First-class support to pause and resume.
Session continuation
Continue a session where it left off.
Built-in storage
DB for conversation and token tracking.
Agent ↔ agent
Collaboration between specialized agents.
Handoff
Pass work cleanly from one agent to the next.
Don't let one agent rot with context
Query agent
Takes the user question and decides where it goes.
Data analyst
Does the research and the heavy lifting.
Summarizer
Turns the work into a clean answer.
The first multi-agent sketch
Five contenders
Phase 2 — integration begins
LangGraph is one big decision graph driven by the agent model. Context flows through the state machine.
Building on the graph
- Integration begins — lots of experimenting with conditions and graph building.
- LangGraph is basically one big decision graph, driven by the agent model.
- Sonnet 3.5 → 4.5 on global Bedrock.
- Context passing through the state machine — powerful.
The agent workflow
Router
Reads intent, chooses the path, controls LangGraph state.
Data Scientist
Frames the analysis and decides what evidence is needed.
Data Analyst
Turns the business question into data operations.
SQL Gen + Execute
Generates SQL, runs it, returns structured results.
Result Analyzer
Checks output, explains findings, decides on another pass.
Right model for each node
Amazon Nova
Fast routing for intent classification and branch selection — keeps the graph entry point cheap.
Claude Sonnet
Where deeper reasoning matters: analysis planning, data interpretation, final explanation.
SQL Gen + Execute
Different prompt, permissions, and guardrails — built around safe database work.
Shared context
Nodes cooperate through state instead of forcing one model to carry every instruction and tool.
The router becomes the expansion point
Router
Every workflow starts here. New capabilities become branches, not a rewrite.
SQL + analysis
The first production path: answer data questions with query execution and explanation.
Support agent
Operational and product help routes to a support-focused model and tools.
Chart generator
Result sets branch into charts, tables, and executive summaries.
More workflows
Sandbox, reporting, data-quality checks attach behind the same router.
It works end to end
The pivot to a harness
The ecosystem moved. Claude Code got powerful, "harness" became the buzzword, and the agent moved into the terminal.
Finishing the implementation — and noticing the wind change
- Claude Code became more and more powerful.
- People moved toward the agent + subagent concept.
- Less agent-to-agent collaboration.
- Agent sandbox and filesystem dropped.
- Biggest drop by Claude → SKILLS.md.
Pivoting our agent implementation
- Harness becomes the buzzword.
- Agent sandbox, skills, filesystem.
- We understood why people were moving this way.
- Tech-savvy people → terminal → agent lives in the terminal → access to all files → does its job.
But… how do we do this in production?
Existing app?
How do we implement this inside an app that already exists?
Give agent access to /?
In production? Handing an agent the whole root directory is a non-starter.
A lead
LangChain dropped a framework called deepagent.
Exploring the sandbox path
- Explore more agent sandbox.
- RLM just dropped — agent iterates through one variable of 1M length (e.g. a whole novel).
- RLM too advanced for our use cases and not stable yet.
- So we double down on the sandbox.
Give the model a workspace, not a script
- Models keep getting more and more intelligent.
- Give the model a sandbox, throw in a CSV with 100,000 lines — "hey, analyze this."
- Skeptical at first… but the result was insane.
Replace the workflow with a harness
Fewer specialized nodes. One sandbox where the agent actually does the work.
The new shape
Get the data
Handles the question, runs the query, prepares the result set.
Agent does the job
Dump the result into a sandbox — code, files, pandas, charts, iteration.
Explain the answer
Checks output, summarizes findings, turns raw analysis into a usable answer.
What the sandbox bought us
An extra analyze step
Better, more thorough analysis than a single SQL pass.
Real tooling
Access to pandas, numpy, and other libraries.
Iterate over big data
The agent uses all that to iterate through large datasets.
Most importantly
More token savings.
Harness with sandbox
Inside the sandbox
Inside the sandbox
What the user gets back
The real work: context
The biggest improvement wasn't magic prompting. It was giving the agent the right context at the right time.
The ontology resolver
- The biggest improvement wasn't magic prompting — it was the right context at the right time.
- We added an ontology resolver so user language maps back to the right domain concepts.
- It starts by querying embedded ontology data in pgvector with vector similarity search.
- Next: a stronger graph layer, so concept relationships guide retrieval beyond text similarity.
What it runs on
What we got wrong (and right)
- Early on we over-focused on memory, compaction, session continuation, and HITL.
- Useful platform features — but not the real user problem.
- The actual problem was analytics: helping users check and understand their data fast.
- Usage was mostly fire-and-forget: ask, get the result, return to the system.
- Most users didn't need a long conversation — they came back later and started fresh.
What's next for us
A minimal harness framework, a smarter resolver, and our own open-source sandbox runtime.
A minimal harness framework
Minimal harness
Our own small framework for agent execution, still integrated with Amazon Bedrock — context, tools, sandbox, standards, result handling.
Skills + AGENTS.md
Standard instructions, reusable capabilities, consistent behavior across projects.
Sandbox included
A controlled workspace where files, code, and analysis tools work together.
Model flexible
Fits any model — Bedrock, Claude, Nova, even ILMU.
Extensible
New agents, tools, skills, and workflows attach without rebuilding the app.
More than vector search
Messy question
Users ask in business and healthcare terms, not database terms.
Concept mapping
Map intent to domain concepts, synonyms, measures, and entity relationships.
Database context
Traverse tables, embeddings, relationships, and metadata to find the right path.
Better answers
Give the query agent richer context before it generates analysis or SQL.
Our own sandbox runtime
Agent
Plans the work and asks the runtime to execute code.
Docker runtime
Runs arbitrary code inside an isolated execution environment.
Artifacts
Returns files, logs, charts, tables, and analysis output.
Control the runtime
Tune the environment for healthcare analysis instead of a generic sandbox.
RCE is risky
Letting agents run code needs strict isolation, limits, logging, and policy.
Safety is the product
Network controls, file boundaries, execution limits, audit trails, permission checks — not optional.
Want to learn how to build AI agents yourself?
I built sidegent.com — learn agents by actually building them in your browser, graded instantly. No setup, no API keys, no credit card.
Build it, don't just watch it
Interactive labs
Each lesson forks a real, working agent you configure and chat with — then grades your work the second you submit.
Machine Lab
Build on a real cloud machine. No setup, no installs — everything runs in your browser.
Prompts → tools → RAG
Write system prompts, add tools and web search, ground agents in your own docs, and ship one for real.
Sidegent Coder
An AI that codes alongside you — say what you want in plain English and watch it build with you.
Give the model a workspace, and get out of its way.
Questions? Let's talk harnesses, sandboxes, and healthcare analytics.