From: aidotengineer
Tool calling is considered more important than just “plumbing” for AI agents [00:00:04]. While significant effort is often spent on improving agents, less attention has historically been paid to building reusable and robust tools that can be integrated into different frameworks [00:00:53]. However, this trend is changing, with more tool platforms and libraries emerging [00:01:06]. The application layer, particularly where tools are built, deserves more attention [00:03:25].
The Importance of Tools
Tools are essential components of an agent’s loop, alongside the large language model (LLM) and memory [00:01:56]. The agent is only as good as its tools [00:02:56]. Problems often arise with tool integration, such as the LLM incorrectly calling or using a tool, or issues within the tool itself [00:02:35]. Therefore, ensuring tools are reusable and robust is crucial for flexibility, allowing them to be brought into any agentic framework [00:03:13].
Defining Tools
A typical tool definition includes:
- Tool Name: Advised to be kept simple [00:06:04].
- Tool Description: Functions almost like a system prompt for the LLM, often being quite extensive for more complex tools [00:06:09].
- Input Parameters: Necessary for the model to know what is needed to call the tool [00:06:40].
- Output Schema: Increasingly important for type safety, structured outputs, and chaining tool calls [00:06:46].
Evolution of Tool Calling Architectures
Traditional Tool Calling
In “traditional tool calling,” the client application sends a prompt to an AI application (agent). This application then sends the prompt to the LLM, which recommends tool calls. The application itself handles the logic of parsing these recommendations, executing the tools, and managing responses, retries, and errors [00:09:02]. This approach involves a significant amount of back-and-forth between the application logic and the LLM, and between the client app and the agent [00:10:08].
Embedded Tool Calling
More recent frameworks often use “embedded tool calling,” where the system acts as a closed circuit. The client app sends a question to the agent, which is a black box. The agent handles all tool calling logic internally, connecting with the LLM and the tools, and then returns the final answer [00:11:19]. While easier to implement initially as it abstracts away error handling and retries, it offers little control over the tool calling process or decision-making [00:13:13]. The tools are defined within the same agent framework [00:12:17].
Achieving Separation of Concerns
A key principle in software development, including AI agent engineering, is the separation of concerns [00:13:41]. This allows for flexibility and easier management of different system parts.
Model Context Protocol (MCP)
The Model Context Protocol (MCP), introduced by Anthropic, is a step towards separating client-side and server-side logic in agentic applications [00:14:51].
- Host/Client: This side connects to servers and doesn’t directly see the tools [00:15:12].
- Server: Acts as a backend with access to tools or assets (like data files), handling the logic and tool definitions [00:15:20].
This clear distinction between the front-end and back-end logic is beneficial for managing AI coding agents with third-party tools [00:15:41].
Standalone Tool Platforms
A standalone tool platform defines and hosts tools separately from the agentic framework [00:16:25].
- Tool Creation: Done via code or CLIs [00:17:40].
- Tool Execution/Surfacing: Handled via SDKs that connect to agentic frameworks like LangChain, Crew AI, or Autogen [00:17:49].
This setup allows the agent to decide which tool to call based on the LLM’s recommendation, then pass the execution to the remote tool platform [00:17:02]. Benefits include:
- Centralized Tool Management: Tools can be created from existing APIs or databases [00:17:26].
- Tool Chaining: Allows for sequencing multiple tool calls within the platform [00:17:59].
- Authorization and Error Handling: These concerns can be managed directly by the platform, reducing complexity within the agent [00:18:25].
- Flexibility: Building tools once on a platform allows for easy integration into different agentic frameworks, promoting framework independence [00:19:11].
Examples of such platforms include Compos Tool House, Arcade AI, and WildCard [00:19:35].
Dynamic Tools
Instead of creating numerous static tools for every specific function (e.g., separate tools for getting customer count by country, orders, etc.), dynamic tools connect to a schema (like GraphQL or SQL) and allow the LLM to generate the specific query [00:22:29].
- The LLM is provided with the tool definition and the schema (e.g., GraphQL schema) and is instructed to generate valid queries [00:22:59].
- Models like Llama, OpenAI models, and Claude are adept at generating GraphQL queries when given a schema, provided the schema isn’t overly complex (e.g., deep nesting, custom scalars) [00:23:29].
- This approach leverages existing business logic and APIs, reducing the need to define 20 different tools and duplicate logic [00:24:12].
However, a trade-off is the potential for LLM hallucination, where the model might generate incorrect queries [00:24:37]. Despite this, dynamic tools represent a promising future for AI agents [00:24:48].