From: redpointai
Function calling is recognized as a crucial capability within AI systems, particularly for building advanced agents [00:21:34]. It serves as an extension point, allowing AI models to interact with external tools and enhance the quality of their responses [00:21:40]. Fireworks, a company specializing in inference for compound AI systems, has heavily invested in this area [00:23:26].
What is Function Calling?
Function calling enables an AI model to “call” or access other tools to improve the accuracy and utility of its answers [00:21:40]. This is particularly important when building agents [00:21:34]. In a broader sense, each individual small expert model can be considered a tool within this framework [00:24:38].
Complexities of Function Calling
The implementation of function calling is more intricate than simply calling a single tool [00:21:50]. Key complexities include:
- Context Management Models need to maintain and understand a long context of conversation in multi-turn chat scenarios to influence which tools are best to call [00:22:02].
- Multiple Tool Selection Often, models need to call into multiple tools, potentially hundreds, for a single query [00:22:11].
- Parallel and Sequential Execution Models must be capable of coordinating and executing calls to multiple tools both in parallel and sequentially [00:22:25].
- Orchestration and Precision The ability to orchestrate and execute a complex plan of tool calls is crucial, and the precision of tool selection is vital for delivering accurate results [00:22:41], [00:23:20].
- Tuning Process The intricate nature of function calling makes the tuning process for these models very complicated [00:23:23].
For example, to fulfill a request like “chart of a stock price of top three cloud providers,” an AI system using function calling would need to:
- Perform a search to identify the top three cloud providers [00:22:59].
- Execute three parallel function calls to retrieve each provider’s stock price [00:23:02].
- Make another call to a charting tool to generate the visual representation [00:23:04].
Integration into Compound AI Systems
Function calling is a critical ingredient for tying together the components of compound AI systems [00:24:48]. These systems are designed to go beyond single model services by combining multiple models across different modalities with various APIs to solve complex business problems [00:02:40], [00:04:16]. The ability of a model to intelligently orchestrate and call into different tools is essential for this composability [00:24:23], [00:24:31].
Fireworks has developed its own F1 model that operates as a complex logical reasoning inference system capable of parallel and sequential function calls, as well as orchestrating and executing complex plans [00:19:37], [00:22:34]. This internal development helps them understand the system abstraction and complexity needed to build developer-facing tools that allow developers to build their own “F1s” [00:28:02].
Future of Function Calling and Agentic Workflows
While still in early stages, the adoption curve for function calling and agentic workflows is beginning to take off [00:23:36], [00:27:08]. These systems are currently more focused on human-in-the-loop automation rather than fully autonomous operation [00:14:09], primarily because human oversight is still necessary for evaluation, maintenance, and operation in production [00:14:27].
Successful applications seen thus far include assistants for doctors (scribing), teachers, students (foreign language learning), coding (assistant to coding), and medical assistants for nurses [00:14:52]. B2B applications also leverage function calling for call center automation to make human agents more productive [00:15:41].
The challenge for the industry remains in defining the right user experience and abstraction for agentic workflows [00:27:14]. Despite the rapid changes in model capabilities, the underlying trend toward specialization and customization through solutions like function calling remains constant [00:48:18].