Building Resilient AI Agents - From Tool Chaos to Agent Harness

Big ups to @trq212 for always sharing publicly, writing articles on his experience building resilient AI Agents at Anthropic.

His latest article is about building agent harness. I just noticed that we also been trying to experiment and building to achieve the same thing as well. It’s just that we didn’t realise and didn’t know the real term for it.

The Tool Overload Problem

At first we discovered tooling for AI agents to use, it sounds so exciting but then we hit the same roadblock - the agent couldn’t decide which tool to use. Imagine there are 30 tools available for 1 agent, and the system prompt just becomes a giant mess of rules telling the agent which tool to use and when.

Decoupling Into Expert Agents

To fix this, I decided to change our agent orchestration pattern. Instead of dumping all the tooling to 1 agent, I decoupled them and created a bunch of experts that focus on specific use cases.

For example:

Before: 1 agent use 1st tool to get current date -> 2nd tool to generate SQL -> 3rd tool to execute the SQL.

After: 1st agent generates SQL -> passes the context to the 2nd agent to execute and get the data -> passes the data to the 3rd and final agent to generate insights.

Why This Pattern Works

I found that this pattern is much cleaner, and you can clean up the context to make it lighter - just pass the important stuff for the agent to process.

One of the key benefits is that you can mix and match LLM models for certain use cases - small and cheap models execute light and easy tasks, while big and powerful models handle complex tasks.

When This Pattern Doesn’t Fit

Although I think this pattern is not suitable for Claude Code because the use case is very dynamic and general, we kinda scoped the use case based on what we want and what our users want. We also have specific schema and datasource already fed to the agent and follow certain standards and output.

What’s Next - Subagents and DeepAgents

And now since Claude introduced this thing called subagents, I try to follow as well. This is where I found the DeepAgents framework by LangChain - props to the LangChain team. Such a powerful framework and it has the capability to spawn subagents as well. But I haven’t pushed it to its potential yet. Maybe soon.