Integration of tool calling in agent frameworks

From: aidotengineer

Tool calling is more than just “plumbing” for AI agents; it is a crucial component that deserves more attention and can significantly enhance the flexibility and robustness of agentic frameworks [00:00:04]. While much focus has been placed on improving the agents themselves, there’s a growing need to build reusable, robust tools that can be integrated into various agentic frameworks [00:00:53].

The application layer, particularly where tools are built, requires more focus as it offers significant opportunities for improvement on top of advancing large language models (LLMs) [00:03:24].

Understanding Tool Calling

In a typical agent loop, a user asks a question or sends a prompt to the agent [00:01:56]. This agent utilizes LLMs and memory, and critically, tools [00:02:00].

For instance, if an agent is asked “how far does the moon orbit expressed as the number of trips between Amsterdam and San Francisco?”, it might use a series of tool calls [00:04:12]:

1. Search for the distance between the Earth and the Moon [00:04:51].
2. Search for the distance between Amsterdam and San Francisco [00:04:56].
3. Perform calculations, possibly using another tool or the LLM itself [00:04:59].

These tool calls often involve external APIs (e.g., web search, databases) or internal functions (e.g., JavaScript, Python for calculations) [00:05:35].

Defining Tools

The definition of a tool is crucial and includes:

• Tool Name: Advised to keep simple [00:06:06].
• Tool Description: Acts almost like a system prompt for the LLM, guiding its usage of the tool [00:06:09]. Longer descriptions are common for more complex, “agentic” tools [00:06:21].
• Input Parameters: Specifies what is needed to call the tool [00:06:39].
• Output Schema: Becoming increasingly important for type safety and structured outputs, especially when chaining tool calls or building complex agents [00:06:48]. This allows models to understand the data returned and chain calls effectively [00:07:06].

Evolution of Tool Calling Integration

Historically, agent development often meant embedding tool logic directly within the agent framework.

Traditional Tool Calling

In what is now considered “traditional” tool calling, the tool logic is intertwined with the agent’s application logic [00:09:00].

• A client application sends a prompt to the agent or AI application [00:09:23].
• The agent sends the prompt to the LLM [00:09:29].
• The LLM recommends a tool call [00:09:35].
• The application explicitly defines and handles the tool call logic, including parsing the LLM’s recommendation, executing the tool via a callback function, and managing queuing, retries, and errors [00:09:42].
• This creates a significant back-and-forth between the application logic and the LLM, making it less of a closed system [00:10:08]. An example in LangChain would involve explicitly filtering tool call messages and executing them [00:10:30].

Embedded Tool Calling

Most modern agent frameworks today utilize what is called “embedded” tool calling, where the system acts as a closed black box [00:11:19].

• The client app sends a question to the agent [00:11:36].
• The agent is a self-contained system, connecting with the LLM and the tools, handling all tool calling internally [00:11:42].
• The tool definitions and logic are defined within the same agent framework [00:12:15].
• The application sends a prompt and tools, and an answer is returned, with no control over the internal tool calling process [00:12:21].
• While easier to implement for beginners as it handles errors and retries, it offers no control over the tool calling process or decision-making [00:13:13].

Promoting Separation of Concerns

As a developer, a preference for separation of concerns is ideal, preventing tightly coupled systems [00:13:41]. This applies to agentic workflows as well.

Model Context Protocol (MCP)

The Model Context Protocol (MCP), introduced by Anthropic and adopted by others, is a significant step towards separating client-side and server-side logic in agentic applications [00:14:51].

• MCP defines a “host” (e.g., a desktop client) that contains a “client” [00:15:10].
• This client connects to “servers,” which are backend systems with access to tools or assets like data files [00:15:18].
• The host/client only sees the server, not the tools directly, as the tools are made available through the server [00:15:29].
• The MCP server handles the logic, defining and importing tools, while the MCP host and client understand how to call these servers [00:15:46].
• This provides a clear distinction between the “front side” (host/client) and “back side” (server with tools) of the system [00:15:41].

Standalone Tool Platforms

A more advanced approach is to use standalone tool platforms, which are gaining market attention [00:16:25].

• Instead of defining tools within a closed agent loop, tools are defined separately on remote servers [00:16:39].
• The agentic framework imports these tools via an SDK or API call [00:16:46].
• The tool creation, hosting, and execution are entirely separate from the agent [00:16:54].
• The agent’s role is to take the tool definition, use the LLM to decide which tool to call, and pass this to the tool platform for execution [00:17:02].
• These platforms often allow for chaining tools, handling authorization, and managing errors [00:17:59].
• This separation provides significant flexibility, allowing developers to build tools once and use them across different agentic frameworks like LangChain, LangGraph, CrewAI, or Autogen [00:19:00].
• Examples of tool platforms include Compos, Toolhouse, Arcade AI, and Wild Card [00:19:30].

Dynamic Tools

Rather than creating a multitude of static tools for every possible operation (e.g., a separate tool for each type of customer query), dynamic tools leverage LLMs to generate queries for existing APIs or databases [00:21:24].

• A dynamic tool can connect to a GraphQL schema or SQL database [00:22:36].
• Instead of passing fixed arguments, the LLM is instructed to generate a valid GraphQL query or SQL command based on the provided schema and available operations [00:22:50].
• LLMs, especially models like Llama, OpenAI models, and Claude, show strong capabilities in generating GraphQL when provided with a clear schema [00:23:28]. Simplifications like avoiding custom scalars or deep nesting can improve performance [00:23:42].
• This approach significantly reduces the need to define numerous individual tools and allows for direct integration of existing business logic [00:24:16].
• However, a trade-off exists: LLMs can sometimes hallucinate or generate incorrect syntax, leading to errors [00:24:36]. Despite this, there’s a strong future for dynamic tools over static ones [00:24:48].

Conclusion

Ultimately, an agent is only as good as its tools [00:02:56]. Ensuring that tools are reusable, robust, and easily integrated into different agentic frameworks is paramount for building effective agents [00:03:12]. The shift towards separation of concerns, utilizing protocols like MCP, and adopting standalone tool platforms or dynamic tool generation will drive greater flexibility and capability in AI agent development [00:24:50].