Tubegraph

Zero ops resilient agent powered apps

5 min read

From: aidotengineer

Full stack AI engineering in a serverless environment involves deploying “zero ops resilient agent-powered user-ready apps” 00:00:23. The goal for AI engineers is to get agentic workflows into the hands of users 00:00:28.

Core Components of AI Engineering Infrastructure

A modern AI engineering infrastructure typically consists of:

There are numerous options for each component:

  • Client App Frameworks: Remix, Astro, Vue, Svelte, Next.js, and others 00:01:09.
  • Agent Frameworks: LangChain, Vercel’s AI SDK, Flowise agents, OpenAI’s Agents SDK 00:01:21.
  • Orchestration Layers: Temporal, AWS Step Functions, Langsmith, Ingest 00:01:32.
  • Serverless Environments: AWS (Lambda API Gateway + Bedrock), Google Vertex, Azure AI Studio, Vercel 00:01:43.

With potentially millions of combinations, choosing the right stack that works together effectively can be challenging 00:02:00, and LLMs often struggle to provide accurate reference code 00:02:18.

A preferred combination for building full stack AI applications includes:

Client App: Next.js

Next.js is chosen for its built-in streaming, first-class server actions, file-based and nested routing, and deep integration with Vercel hosting 00:02:58.

Agent Framework: OpenAI’s Agents SDK

The OpenAI Agents SDK offers native tool calling, one-shot and multi-agent calls, built-in tracing and eval hooks for observability, and strong backing from OpenAI 00:03:20. It also allows for model interchangeability, preventing vendor lock-in 00:03:37.

Orchestration Layer: Ingest

Ingest is an event-driven orchestration layer that eliminates the need to manage JSON state machine documents 00:03:53. It operates on-demand, removing concerns about server warm-up times, and includes automatic retry, step-level observability, and a one-click integration with Vercel 00:04:01.

Serverless Environment: Vercel

Vercel is the preferred serverless environment due to:

  • Preview Deployments: Provides preview deployments with every push, integrating continuous deployment into the workflow 00:04:21.
  • Edge Network: Offers a strong edge network and automatic CDNs 00:04:50.
  • Integrated Databases: Supports various integrated database options like PostgreSQL, Redis, and blob storage 00:04:55.
  • Next.js Alignment: As the company behind Next.js, Vercel provides strong alignment for developing the ideal cloud platform for Next.js infrastructure 00:05:05.
  • Ecosystem: Part of a strong ecosystem of tools found in Next.js Forge 00:05:22.

This combination of tools allows for seamless integration of AI workflows directly into client applications 00:05:32.

Architecture Overview

The architecture for such an application involves:

  1. User Interaction: A user connects to the Next.js application 00:06:02.
  2. Workflow Trigger: The Next.js app checks a database for cached results; if new work is needed, it triggers a new workflow by sending an event to the Ingest service 00:06:05.
  3. Orchestration: Ingest, as the orchestration layer, manages connections to Python serverless functions where AI agents run 00:06:17.
  4. Agent Execution: The OpenAI Agents SDK (currently Python-only) executes within these Python functions, which Vercel automatically hosts 00:06:26. These functions handle inference by interacting with OpenAI 00:06:41.
  5. Results and Caching: Results are returned to the orchestration layer, which then informs the client app that the work is done, caching results in the database as needed 00:06:50.

Example Application: AI-Powered Newsletter Generator

An example application built with this stack demonstrates how AI agents can create a newsletter 00:07:08.

Key Features and Benefits

  • Serverless Scalability: Supports serverless scalability for long-running jobs, preventing agent tasks from crashing or exceeding cloud function time limits 00:07:34.
  • Cost Optimization: Allows for paying only for actual usage, avoiding costs for idle servers 00:07:48.
  • Local Developer Experience: Provides a smooth local development experience 00:07:56.
  • Type Safety: Achieves full type safety with Pydantic in Python and TypeScript across Next.js 00:08:03.

Local Setup and Execution

To run the example app locally, three terminals are needed: one for Python agents, one for Next.js, and one for the Ingest dev server 00:08:13.

The application allows users to enter comma-separated topics, and it generates a newsletter based on the intersection of these topics 00:09:11. For instance, generating a newsletter about “AI engineering in Seattle” 00:09:22. The app polls a Vercel blob storage for results 00:09:40. The Ingest server shows the task broken down into individual steps, each capable of scaling to the full length allowed for Vercel cloud functions 00:10:11.

Code Structure

The project structure includes:

  • Ingest Endpoint: An Ingest endpoint provides detailed observability of step-by-step processes directly from the app 00:11:26.
  • Newsletter API Endpoints: These endpoints are called by Ingest, which in turn invokes the Python agents 00:11:47.
  • Top-Level API Directory: Vercel automatically serves each file in this directory as an independent cloud function (e.g., Python functions) 00:11:57.
  • Ingest Folder: Located within the source code, this folder defines workflows triggered by specific events 00:12:10.

Agent Implementation

The AI agents are built using FastAPI, providing a quick way to assemble them 00:12:32. For example, one agent handles research, and another handles formatting 00:12:37. Vercel can deploy these independent Python files with a single line of code, without requiring a vercel.json config file 00:12:49.

Ingest Workflow Structure

Ingest workflows are defined to keep each step clear and isolated 00:13:20. Each step.run invocation executes in order, passing results to subsequent steps 00:14:16. Code outside step.run may be executed multiple times 00:14:19.

Example workflow steps:

  1. Call the research agent 00:13:54.
  2. Format the newsletter with another agent 00:13:56.
  3. Save the results to blob storage for the front end to access 00:13:58.

Deployment to Vercel

The entire Next.js repository can be deployed to Vercel 00:15:23. Vercel provides full build logs, showing how the api agents folder is deployed as a Python function, running on Python 3.12 00:15:58. Deployment is as simple as running the vercel CLI command from the terminal 00:16:24.

Environment Variables

To run this project, the following environment variables are needed:

NOTE

Environment variables can be pulled down from production and run locally using the Vercel CLI 00:16:44.

Conclusion

This combination of Next.js, OpenAI’s Agents SDK, Ingest, and Vercel provides:

  • Scalability expected from a serverless environment 00:15:29.
  • Resilience from a robust orchestration layer 00:15:33.
  • Agentic power from OpenAI’s Agents SDK 00:15:38.

The example code is open-source and available on GitHub for exploration and contributions 00:17:14.

Graph View

Backlinks