From: aidotengineer
Tool calling is considered more important than just “plumbing” for AI agents, as it underpins the capabilities of these agents [00:00:04]. While much attention has been given to improving AI agents, there has historically been less focus on building reusable and robust tools for them [00:00:58]. The application layer, particularly where tools are built, deserves more attention to catch up with advancements in Large Language Models (LLMs) [00:02:24], [00:03:59].
An agent’s effectiveness is directly tied to the quality of its tools [00:02:56]. Tools must be reusable and robust, enabling them to be integrated into various agent frameworks [00:03:13].
A typical tool definition includes:
- A simple tool name [00:06:06].
- A tool description, which acts like a system prompt for the LLM [00:06:09].
- Input parameters for the tool [00:06:40].
- An output schema, which is increasingly important for type safety and chaining complex tool calls for structured outputs [00:06:46], [00:07:18].
Static Tool Calling
Static tool calling refers to methods where tools are either tightly coupled with the agent framework or operate as a “black box” without external query generation by the LLM.
Traditional Tool Calling
In traditional tool calling, the client application interacts with an agent or AI application, which then prompts the LLM [00:09:02]. The LLM suggests tool calls, but the actual tool execution logic (parsing, filtering, calling tools, handling retries and errors) resides within the server-side application or agent [00:09:42], [00:11:01]. This involves a lot of back-and-forth between the application and the LLM [00:10:08].
For instance, in LangChain, this involves explicitly looking for tool call messages from the LLM, parsing them, and executing them via a callback function [00:10:47].
Embedded Tool Calling
Embedded tool calling, often a more recent evolution, makes the agent a closed system [00:12:24]. The client application only asks a question, and the agent, having the tools and connecting with the LLM, handles all tool calling internally as a black box [00:11:34], [00:12:21].
- Advantages: Easy to implement for beginners as it abstracts away error handling, retries, and queuing [00:13:13].
- Disadvantages: Limited control over the tool calling process, execution details, or decision-making [00:13:19].
Dynamic Tool Calling
Dynamic tool calling involves giving the LLM the ability to generate queries for existing APIs or databases directly, rather than defining a separate static tool for every specific operation [00:21:24].
Instead of needing millions of tools, or even hundreds that could confuse an agent, dynamic tools offer a solution [00:21:35]. For example, by connecting to a GraphQL schema or SQL database, the LLM can generate the necessary query (e.g., a GraphQL query) based on the user’s request [00:23:01], [00:23:57].
- Process: The model is given a tool that can connect to a GraphQL or SQL schema. It’s prompted to generate valid queries. By providing the schema, the model understands the available types and operations [00:23:08].
- LLM Capabilities: Models like Llama, OpenAI models, and Claude are adept at generating GraphQL, provided the schema isn’t overly complex (e.g., avoiding custom scalars or deep nesting) [00:23:35].
- Advantages: Reduces the need to define numerous individual tools, allows direct use of existing business logic, and offers greater flexibility [00:24:23], [00:24:33].
- Trade-offs: LLMs might hallucinate or mess up syntax, leading to inconsistent query generation [00:24:37].
Architectures Supporting Flexibility
Developers often seek separation of concerns to keep systems more manageable and flexible [00:13:41].
Model Context Protocol (MCP)
The Model Context Protocol (MCP), introduced by Anthropic, aims to separate client-side and server-side logic in agentic applications [00:14:53]. In this model:
- A “host” (e.g., a desktop client) contains the client [00:15:12].
- “Servers” act as backends that have access to tools or assets like data files [00:15:20].
- The client interacts with the server, which handles tool invocation, providing a clear distinction between front-end and back-end responsibilities [00:15:33], [00:15:41].
Standalone Tool Platforms
A standalone tool platform defines and hosts tools separately from the agent framework [00:16:25]. The agent framework then imports or connects to these tools via an SDK or API call [00:16:48]. This means tool creation and execution are decoupled [00:16:54].
These platforms typically offer:
- A place to create tools (via code or CLIs) [00:17:35].
- A place to execute or surface tools (via SDKs that connect to frameworks like LangChain or CrewAI) [00:17:46].
- Capabilities like chaining tools (e.g., retrieving a country based on IP, then querying a CRM for customers in that country) [00:17:59].
- Centralized handling of authorization, authentication, and errors for various underlying systems [00:18:25].
Using tools and function calling in AI SDKs provided by these platforms allows for significant flexibility [00:19:58]. Agents can easily switch between different frameworks (e.g., LangChain, LangGraph, CrewAI, Autogen) without rebuilding tools, as the tools are built once and imported [00:19:00]. The agent decides what tool to call, but the actual execution is handled by the remote tool platform [00:20:28]. This approach promotes separation of concerns, with tools in one place and the agent in another [00:20:39].
Examples of such platforms include Compos, Tool House, Arcade AI, and Wild Card [00:19:36]. IBM is also building a similar platform called WX Flow [00:19:51].
Ultimately, the importance of tool calling in AI cannot be overstated; tools are as critical as the agents themselves [00:24:58].