Tubegraph

Emergence of standalone tool platforms and dynamic tools

5 min read

From: aidotengineer

The common perception that tool calling is merely “plumbing for AI agents” understates its significance [00:00:04]. While much attention has been given to improving AI agents, less focus has historically been placed on building robust, reusable tools for these agents [00:00:53]. However, this trend is changing, with more tool platforms and libraries emerging [00:01:06]. The application layer, where tools are constructed, is increasingly recognized as critical for enhancing AI capabilities [02:25].

The Importance of Tools in Agent Frameworks

An agent is only as effective as the tools it utilizes [02:56]. Often, the first component to fail when building an agent is its tools, whether due to a large language model (LLM) miscalling a tool, using the incorrect one, or internal tool breakdowns [02:35]. This highlights the need for tools that are reusable, robust, and transferable across different agent frameworks [03:13].

Understanding Tool Calling

Tool calling enables an agent to extend its capabilities beyond its internal knowledge [04:06]. For example, to answer a complex query like “How far does the moon orbit expressed as the number of trips between Amsterdam and San Francisco?”, an LLM might perform a series of tool calls [04:10]:

  1. Search for the distance between the Earth and the Moon [04:51].
  2. Search for the distance between Amsterdam and San Francisco airports [04:56].
  3. Perform calculations, possibly using a calculation tool or an external API like Wolfram Alpha [04:59].

The definition of a tool is crucial [05:54]. A typical tool definition includes:

  • Tool Name: Kept simple [06:06].
  • Tool Description: Functions almost like a system prompt for the LLM, guiding its usage [06:09].
  • Input Parameters: Specifies what data the tool needs to be called [06:40].
  • Output Schema: Increasingly important for structured outputs and chaining tool calls, ensuring type safety [06:46].

Evolution of Tool Calling Architectures

Traditional Tool Calling

In traditional tool calling, the application or agent explicitly defines and manages the tool calling logic [09:02].

  • A client application sends a prompt to the agent/AI application [09:23].
  • The agent sends the prompt to the LLM [09:31].
  • The LLM recommends a tool call, and the application must then filter, parse, and execute the tool call based on defined callback functions [09:42].
  • This involves significant back-and-forth between the application and the LLM, and requires handling queues, retries, and errors within the application logic [00:10:10].

Embedded (Meta) Tool Calling

More modern frameworks often use an embedded or “meta” tool calling approach, where the agent framework handles all tool calling internally [00:11:19].

  • The agent is a closed system; tools are passed in as a black box [00:11:24].
  • The client app sends a question to the agent, which connects with the LLM and its defined tools, performs calls, and returns the answer [00:11:34].
  • This simplifies initial implementation by abstracting away error handling and retries [00:13:13].
  • However, it offers less control over the tool calling process, how decisions are made, or the output format [00:13:19].

Separation of Concerns with MCP

Inspired by architectural principles like separation of concerns in web development (e.g., separating backend from frontend) [00:14:39], the Model Context Protocol (MCP) emerged [00:14:53]. Introduced by Anthropic, MCP aims to separate the client-side (host) from the server-side in agentic applications [00:14:57].

  • The host (e.g., a desktop client) has a client that connects to servers [00:15:12].
  • Servers act as backends, having access to tools or assets like data files [00:15:20].
  • The host and client interact only with the server, not directly with the tools [00:15:29].
  • This approach clearly distinguishes between the front-end and back-end logic, with the MCP server handling tool definition and logic [00:15:41].

Standalone Tool Platforms

A significant advancement in building robust and reusable tools is the rise of standalone tool platforms [00:16:25].

  • Definition: Tools are defined and hosted separately from the agentic framework [00:16:39].
  • Integration: Agents import these tools via an SDK or API call [00:16:48].
  • Execution: The agent uses the LLM to decide which tool to call, then passes the request to the remote tool platform for execution [00:17:02].
  • Benefits:
    • Separation of Concerns: Clear distinction between tool creation/hosting/execution and agent logic [00:17:18].
    • Ease of Creation: Platforms facilitate building tools from existing APIs or databases [00:17:26].
    • Tool Chaining: Platforms can manage sequential or complex tool calls [00:17:59].
    • Centralized Handling: Authorization, authentication, and error management can be handled by the platform, simplifying the agent’s responsibilities [00:18:25].
    • Flexibility: Tools built on a standalone platform can be easily integrated into different agent frameworks (e.g., LangChain, CrewAI, AutoGen), allowing for framework switching without rebuilding tools [00:19:11].

Examples of standalone tool platforms include Composable, Toolhouse, Arcate AI, WildCard, and IBM’s WX Flow [00:19:31].

Dynamic Tools

Instead of creating numerous static tools for every specific function within a complex system (e.g., a CRM) [00:21:34], dynamic tools offer a solution [00:22:29].

  • A single dynamic tool can connect to a schema (e.g., GraphQL or SQL) [00:22:38].
  • The LLM itself generates the necessary query (e.g., a GraphQL query) based on the schema and user’s request [00:22:55].
  • The model is provided with the tool, instructed to generate valid queries, and given the schema (type definitions, available operations) [00:22:59].
  • Benefits: Allows for the integration of existing APIs and databases into agent frameworks with less implementation overhead, avoiding duplication of business logic [00:24:09].
  • Considerations: There are trade-offs, such as the potential for LLM hallucination in generating queries or issues with complex schema features (e.g., custom scalars, deep nesting) [00:24:36]. Despite these, dynamic tools are seen as a promising future direction for tool building [00:24:48].

Ultimately, as AI agents continue to advance, ensuring the quality and flexibility of the tools they use is equally important [00:24:58]. The shift towards standalone platforms and dynamic tools reflects a growing understanding of this critical relationship.

Graph View

Backlinks