Tool-calling methods for AI engineering

Tl;DR: We share the engineering journey behind rewriting a ReAct-based agent to a production-grade architecture: RAG-based (Retrieval-Augmented Generation) tool filtering cuts tool failure rate by 29%, parallel tool calling reduces query latency by 40%, and combining both yields 28% fewer tool calls per query with preserved answer quality (0.85 similarity), enabling 95% of queries to complete in ≤2 iterations.
Check it out on GitHub: https://github.com/khetansarvesh/SERA

---

Building reliable AI agents for financial research means coordinating dozens of tools. At Sentient Labs, we started with vanilla ReAct — and quickly hit its limits, especially due to its high latency. With 40+ tools in the action space, ReAct struggled with tool selection, wasted iterations on failures, and ran too slowly for production traffic. Over a series of experiments, we systematically solved each bottleneck by shrinking the action space with RAG-based tool filtering and by parallelizing tool calls.

Key Challenge: ReAct Doesn't Scale With Large Tool Arsenals

ReAct is a common default architecture for tool-using agents: think, pick a tool, observe the result, repeat. It works well when the agent has a handful of tools and the task is straightforward. But in our case, we were building a finance research agent with 40+ tools — market data APIs, total value locked (TVL) trackers, on-chain flow providers, derivatives feeds, social sentiment endpoints, and more.

At this scale, ReAct broke down in predictable ways. From OpenAI’s cheaper GPT-OSS-120B to Anthropic’s more powerful Opus 4.1, no model could reliably select the right tool from such a large action space. The ReAct agent frequently picked wrong tools or passed incorrect parameters, causing tool failures that triggered additional iterations. And naturally, each failed iteration eats into the latency budget without producing useful information.

As a result, we needed to systematically address three compounding problems:

• tool selection accuracy;
• execution latency;
• and the interaction between the two.

---

Experiment 1: Shrinking the Action Space with RAG

Instead of handing the LLM all 40+ tools and hoping it picks correctly, we added a retrieval step before tool execution. To put it simply, we embedded all tool descriptions and ran similarity searches against the user query, filtering down to the top 15 most relevant tools before the LLM ever saw them.

• Baseline: Vanilla ReAct with all 40+ tools exposed to GPT-OSS-120B
• Variant: RAG-filtered ReAct — same model, same loop, but only the top-15 retrieved tools are provided

Measuring Tool Efficiency

Before diving into the results, we needed a metric that captures both the volume and reliability of tool usage in a single number. Standard metrics like "number of tool calls" or "failure rate" each tell only half the story — an agent that makes few calls but fails on all of them isn't efficient, and neither is one that succeeds on every call but makes dozens of unnecessary ones.

We introduced a tool efficiency metric that penalizes both excessive tool usage and tool failures:

tool_efficiency = (1 - total_tool_calls / 10) × (1 - tool_failure_rate)²

where tool_failure_rate = total_failed_tools / total_tools_called within the ReAct loop. The quadratic penalty on failure rate reflects our observation that tool failures are disproportionately costly — each failure not only wastes a step but often sends the agent down an unproductive reasoning path for subsequent iterations.

We saw clear aggregate gains where fewer tools meant cleaner selection, fewer failures, and less wasted computation. But the averages mask an important pattern that only becomes visible when we break down results by iteration count.

The Three Zones: When RAG Helps (and When It Doesn't)

The chart below plots the tool efficiency difference (RAG ReAct minus Vanilla ReAct) for each query, sorted by the number of ReAct iterations (purple line, right axis). Both architectures pulled from the same arsenal of 40+ tools, with RAG ReAct narrowed to the top 15 most relevant per query. Bright pink regions indicate queries where RAG ReAct outperformed; light pink regions indicate where vanilla ReAct was better.
The chart below plots the tool efficiency difference (RAG ReAct minus Vanilla ReAct) for each query, sorted by the number of ReAct iterations (purple line, right axis). Bright pink regions indicate queries where RAG ReAct outperformed; light pink regions indicate where vanilla ReAct was better.

• Zone 1 — Low iterations (≤4 loops): Vanilla ReAct performs equally or better. These are simple queries where the agent finds the right tool quickly regardless of action space size. RAG filtering adds a retrieval step without meaningful benefit — the LLM was already navigating the tool space fine.
• Zone 2 — Medium iterations (5–8 loops): Mixed results. Neither architecture consistently dominates. These queries are complex enough that tool selection matters, but not so complex that vanilla ReAct completely fails. The advantage shifts back and forth depending on the specific query.
• Zone 3 — High iterations (>8 loops): RAG ReAct clearly and consistently wins. These are the queries where vanilla ReAct spirals — repeatedly choosing wrong tools or passing bad parameters, burning through its entire iteration budget. By narrowing the action space, RAG filtering prevents the agent from getting lost in irrelevant tools and helps it converge much faster.

This three-zone pattern gave us a crucial insight: the value of action space reduction scales with query complexity. For simple queries, the LLM handles large tool sets fine. For complex queries, a smaller, curated tool set is the difference between a 3-iteration success and a 10-iteration failure.

We also observed something interesting in Zone 3: even when both architectures produced the exact same reasoning (identical `THOUGHT` steps at a given iteration), RAG ReAct consistently selected the correct tool while vanilla ReAct did not. This suggests the issue is the “cognitive load of a large action space” on tool selection.

RAG filtering reduced wasted calls in sequential execution; but would the same hold when calls are already parallelized?

---

Experiment 2: Parallel Tool Calling

ReAct executes tool calls sequentially by design. The output of tool n feeds the reasoning for tool n+1. This makes sense for multi-hop queries where each step depends on the last.

Circling back to our finance research agent example mentioned before, when we analyzed our production traffic, we found that most queries are not multi-hop. A user asking "What's happening with BTC?" needs market data, sentiment, and on-chain flows fetched independently, not sequentially.

We then modified our ReAct agent to identify independent tool calls within a single reasoning step and execute them concurrently.

For non-multi-hop queries, parallel execution nearly halves latency, where we saw a significant 40% improvement in average query and median query time.

---

Experiment 3: Combining RAG Filtering + Parallel Execution

From previous experiments, we discovered RAG filtering cut wasted calls when tools ran sequentially, and parallel ReAct sped things up significantly, so naturally we added RAG filtering on top of our parallelized ReAct agent.

The iteration counts barely changed — both architectures already converge in 1–2 loops for most queries, although tool efficiency improved quite substantially. The RAG-filtered variant calls fewer tools per iteration and achieves higher efficiency, meaning it's doing less redundant work to arrive at the same quality answer.

We validated answer quality by measuring embedding similarity between the outputs of both systems: 0.85 — confirming that the RAG-filtered variant produces equivalent answers with less compute.

---

Results and Learnings

These three experiments revealed a clear pattern in our production traffic:

1. 95% of queries complete in ≤2 iterations — long ReAct loops are wasteful for our use case
2. RAG-based tool selection consistently outperforms brute-force selection in large action spaces
3. Parallel execution dramatically cuts latency for non-multi-hop queries
4. Combining both yields the best efficiency without sacrificing answer quality

The key learning is that architecture should match query complexity. Over-engineering simple queries with multi-step loops wastes latency and compute. Under-engineering complex queries with single-pass systems sacrifices answer quality.

The right system routes each query to the lightest architecture that can handle it well.