From: aidotengineer
When AI agents are deployed in production settings, a significant concern is how to integrate human oversight and approval into their operations [00:00:10]. For people to become comfortable with agent automation, it’s crucial that key aspects of agent execution allow for a degree of human approval [00:01:23].
Challenges with Human-in-the-Loop Agents
Traditional approaches to “human-in-the-loop” processing for agents face several challenges:
- High-Value or High-Risk Tasks Agents may encounter designated tools or tasks that involve high value or risk, such as transferring large sums of money or deleting accounts [00:01:48]. In these scenarios, there needs to be a mechanism for the agent to pause and allow a human to provide a final determination or decision [00:01:58].
- Long-Running Processes Many agents are designed to be long-running, involving numerous steps [00:02:12]. The longer a process runs, the higher the chance of failure [00:02:19]. Without a way to checkpoint or resume, all work done up to the point of failure could be lost [00:02:24].
- Agent Loop Persistence Most current agent frameworks that support human-in-the-loop functionality require the underlying agent loop to run continuously [00:07:23]. This means the physical machine or cloud instance executing the agent must remain active, even if it’s merely waiting for a human response from a communication channel like Slack [00:07:30]. This continuous running can be inefficient and costly.
Agent Continuations: A Solution
To address these challenges, Snaplogic has developed a new mechanism called “agent continuations” [00:00:44]. Inspired by the programming language theory concept of continuations [00:08:14], agent continuations allow for capturing the full state of complex agents [00:00:50]. This state capture can be used for both arbitrary human-in-the-loop processing and for creating reliable checkpoints to resume agent execution through snapshots [00:00:56].
The core idea is to pause agent execution, save its complete state, and then resume or continue the execution from that exact point at a later time [00:09:34].
How Agent Continuations Facilitate Human Approval
The process of integrating human approval with agent continuations involves the following steps:
- Tool Designation Developers can designate specific tools within an agent’s capabilities as needing approval [00:12:37].
- Suspension When the agent reaches a point where an approved tool needs to be called, or any other suspension condition is met, it will automatically suspend [00:15:12].
- Continuation Object Creation Upon suspension, a “continuation object” is created [00:15:19]. This object embeds the standard messages array (which maintains the history of interactions with the LLM) along with additional metadata required to resume execution [00:15:24].
- Application Layer Interaction The continuation object is then sent back to the application layer. This layer extracts core information to make it easy for the application to understand why the agent suspended and to present the necessary information to the user for approval [00:15:48].
- Human Review and Update A human reviews the situation and provides their approval or disapproval [00:13:44]. The application layer then updates the continuation object with this human decision [00:16:26].
- Resumption The updated continuation object is sent back to the agent [00:16:29]. The agent framework recognizes it as a continuation and uses the embedded logic and state to reconstruct the agent and resume its execution from the exact point of suspension [00:16:34].
Key Advantage: Decoupled Operations
A powerful aspect of agent continuations is that once the agent has suspended and a continuation object has been created, the underlying agent loops do not need to remain running [00:16:54]. They can be shut down because all necessary information has been captured to restart everything exactly where it left off [00:17:02]. This addresses the challenge of continuous agent loop persistence.
Support for Multi-Level Agents
Agents are becoming increasingly sophisticated, often involving a main orchestrator agent that delegates tasks to several sub-agents, which can, in turn, have their own sub-agents [00:06:00]. Agent continuations are designed to handle this complexity. The format for the continuation object is recursive, capable of supporting arbitrary layers of nesting, ensuring that even when a sub-agent requires human approval, the state of the entire multi-level agent can be saved and restored [00:18:20].
Multi-Level HR Agent
An example shows an HR agent that uses an “Account Agent” as a sub-agent [00:18:50]. The Account Agent has an “authorize account” tool that requires human approval [00:19:25]. If the HR agent initiates an account creation process that eventually leads to the sub-agent needing authorization, the entire process suspends, a nested continuation object is propagated back to the application layer, and once approved, the process resumes from that exact point within the sub-agent [00:20:01].
Implementation Details
The prototype implementation is built on top of the OpenAI Python API [00:24:12]. While other frameworks consider state management, this approach is novel in combining both a human approval mechanism and the ability to handle arbitrary nesting of complex agents [00:25:07].
The work originated from the Agent Creator research group at Snaplogic, which also provides a visual agent building interface and platform [00:25:48]. The continuation concept has been prototyped both at the Python layer and within Snaplogic’s higher-level platform [00:26:25].