From: aidotengineer
MicroVMs are a core technology behind modern AI sandboxing solutions like Arachis, an open-source code execution and computer use sandboxing service for AI agents [00:00:07]. They provide a secure, fast, and efficient environment for running untrusted AI-generated code [00:03:17].
Why AI Sandboxes Need MicroVMs
AI models, particularly those using tool calling for inference, require sandboxes for executing tools like search or code execution [00:00:59]. For reinforcement learning, sandboxes are crucial during the training phase to run reward functions at scale [00:01:12].
A key driver for using MicroVMs is security [00:03:20]. AI-generated code, like any code from untrusted sources, could be buggy or malicious [00:01:46]. Without proper isolation, such code could gain root access, compromise data, or affect the host system [00:01:47]. MicroVMs address this by providing a highly isolated execution environment [00:07:51].
Furthermore, MicroVMs offer:
- Speed Fast boot times are paramount for AI sandboxes [00:03:47]. Arachis, leveraging MicroVMs, can boot in less than 7 seconds, with ongoing efforts to reduce this to under a second [00:03:55].
- Backtracking and Snapshotting MicroVMs enable quick snapshotting and restoration of the sandbox’s state [00:04:08]. This allows AI agents to checkpoint progress, backtrack if multi-step workflows fail, replan, and continue without starting from scratch [00:04:48].
- Full Linux Sandbox Capability Agents benefit from a full Linux sandbox, allowing them to use commands like
psorlsofto debug applications during code generation, backtrack, and replan [00:01:21].
Understanding MicroVMs and Linux Sandboxing
To grasp MicroVMs, it’s helpful to understand the evolution of Linux sandboxing:
Linux Execution Model
A thread is the smallest unit of execution on Linux, represented by a task_struct in the kernel’s scheduler run queue [00:08:21]. A process is a logical construct of multiple threads, sharing resources like page tables [00:08:42]. The kernel provides privileged access to hardware, requiring system calls to switch to kernel or supervisor mode for privileged operations [00:08:56].
Containers
From a programmer’s perspective, containers package an application’s dependencies with its core logic, allowing arbitrary user code to run [00:10:14]. Technically, a container is a collection of namespaces (e.g., process, mount, network) [00:10:32]. A container sees an abstracted view of its own resources, with the host able to see inside but the container unable to see its host’s namespace [00:11:02]. Control groups (cgroups) are used with namespaces to manage resource allocation (e.g., CPU, memory) for containers [00:11:41].
Security Limitations of Containers: Containers run as native processes directly on top of the host kernel [00:12:20]. This means a kernel vulnerability can be exploited by a malicious or buggy process within a container to gain root access on the host, allowing data access or other malicious actions [00:12:33].
Virtualization
Virtualization offers a stronger primitive for running untrusted code [00:14:47]. Each Virtual Machine (VM) has its own guest user space and guest kernel, providing isolation unlike containers where processes run directly on the host kernel [00:14:55]. This significantly reduces the attack surface to the host kernel [00:15:10].
On Linux, virtualization relies on /dev/kvm, a device in the Linux kernel that exposes the processor’s virtualization stack [00:16:01]. A Virtual Machine Monitor (VMM), such as QEMU, CrossVM, or Firecracker, sits on top of /dev/kvm to spawn and manage VMs [00:15:50]. When a guest VM needs to access host resources (e.g., disk, network), it “exits” to the host, triggering a VM exit to the VMM process. The VMM then interacts with the host kernel and sends the response back to the guest with a VM resume [00:16:56]. Minimizing these VM exits is crucial for performance [00:17:25].
MicroVMs vs. Traditional VMs
MicroVMs differ from traditional VMs in several key ways:
- Security-First Design MicroVMs, pioneered by projects like CrossVM (the first Rust-based VMM) [00:18:38], are often written in memory-safe languages like Rust to prevent memory-related bugs that untrusted guest code could exploit to attack host devices [00:19:02]. They also implement stricter jailing architectures, isolating emulated devices (like block and network) into separate processes or with limited capabilities to reduce cross-device attack surfaces [00:19:11].
- “Micro” for Speed and Memory The “micro” in MicroVM refers to the VMM process itself [00:19:41]. Unlike old VMMs that support many architectures and devices, MicroVMs (e.g., CrossVM, Firecracker, Cloud Hypervisor) support only one or two common architectures (Intel, ARM) and major devices [00:19:54]. This drastically reduces code paths at boot and runtime, leading to blazing-fast boot times and lower memory consumption [00:20:17].
Arachis specifically uses Cloud Hypervisor as its MicroVM VMM [00:23:57]. Cloud Hypervisor was chosen for its hot-plugging support (adding/removing RAM at runtime), GPU support, snapshot support, and its open-source, non-company-specific control [00:22:40].
While G Visor is another option, offering better security than containers but less than MicroVMs, and easier GPU access, MicroVMs were chosen for Arachis due to their strong security guarantees, especially for multi-tenant untrusted code execution [00:23:09].
Architecture of MicroVM-based Sandboxes
In Arachis, a REST server manages the MicroVM-based sandboxes [00:05:40]. Each sandbox runs a VNC server for GUI access and a code server for execution [00:05:47].
File System
Sandboxes employ an overlay file system setup [00:25:00]. A shared, read-only base layer (root FS) is protected and shared among sandboxes [00:25:03]. On top of this, each sandbox gets its own read-write layer where new files are created [00:25:11]. When a sandbox is snapshotting, only this read-write layer needs to be persisted [00:25:31].
Networking
Each sandbox has isolated networking through a unique virtual networking interface called a tap device [00:27:12]. These tap devices connect to a Linux bridge on the host server [00:27:34]. Arachis handles port forwarding from the host to the sandbox’s code server or VNC server, simplifying external access [00:27:44].
Snapshotting Process
Arachis’s snapshotting allows agents to save the entire running state of a sandbox, including guest memory and the read-write file system layer [00:33:30]. This ensures that any processes, open windows, or files created are restored exactly as they were [00:33:51]. The process involves:
- Pausing the VM [00:34:44].
- Dumping the guest memory using the VMM’s snapshot API [00:34:50].
- Manually persisting the read-write overlay FS layer [00:34:57].
- Resuming the VM [00:35:08].
Future work on snapshotting includes moving to Btrfs for optimized incremental snapshots [00:39:24].
Conclusion
MicroVMs provide a robust foundation for building AI sandboxes due to their strong security guarantees, fast boot times, and efficient snapshotting capabilities. This allows AI agents to perform complex, multi-step workflows with reliable backtracking and recovery [00:05:02]. Ongoing work in Arachis aims to achieve sub-second boot times and advanced memory management for greater efficiency [00:39:14].