From: aidotengineer
At Sierra, the Agent Development Life Cycle (ADLC) is the core process used to build and continuously improve AI agents for businesses [00:00:24]. The company views “every agent as a product,” necessitating a fully featured developer and customer experience operations platform for their creation and refinement [00:09:06]. This approach mirrors the dedication applied to developing mobile apps or websites [00:09:19].
Evolution of AI Development
The journey towards the ADLC is informed by the history of AI. Early AI efforts, like the speaker’s work on Google Lens in 2016, focused on fundamental tasks such as distinguishing between images (e.g., Chihuahuas and blueberry muffins) [00:02:15]. This period, described as the “AI caves,” highlighted the non-deterministic nature of AI models, where consistent results were not guaranteed, feeling like a “slot machine” [00:03:48]. The iterative improvement over a decade, similar to the continuous software development life cycle (SDLC), transformed Google Lens into a highly capable tool for shopping, translation, and more [00:04:43].
Why a New Development Cycle for AI Agents?
While traditional software development is deterministic, fast, cheap, rigid, and governed by strict logic, large language models (LLMs) are often non-deterministic, slow, expensive, and flexible [00:11:43]. LLMs are creative and can reason through problems, but building on them is compared to building on a “foundation of jello” [00:11:36]. The ADLC was conceived to leverage the strengths of LLMs while integrating traditional software where beneficial [00:12:02].
The Agent Development Life Cycle (ADLC) in Practice
The ADLC is a systematic process designed to build and improve AI agents, emphasizing iterative refinement with customers in production environments [00:13:35].
Key Aspects of the ADLC:
- Quality Assurance (QA): Customers have access to Sierra’s Experience Manager, allowing them to review every conversation and monitor agent performance in real-time [00:12:51].
- Feedback and Issue Resolution: Users can report issues directly, which leads to the filing of an issue, creation of a test, and subsequent release of improvements [00:13:15]. Over time, agents accumulate hundreds to thousands of tests as they improve [00:13:29].
- Continuous Improvement: The goal is for an agent to continuously get better every day, even if not perfect at launch [00:11:19].
- Leveraging AI for Development: Recognizing the rapid advancements in AI, Sierra integrates AI into each stage of the ADLC to accelerate improvements [00:14:13]. Reasoning models, for example, act as a “force multiplier” for development, testing, and QA [00:15:06].
Case Study: Chubbies’ Duncan Smothers
Sierra partnered with Chubbies to create an AI agent named Duncan Smothers [00:07:37]. Duncan is designed to be capable and engaging, handling various customer inquiries on the Chubbies website [00:07:55]. Examples of Duncan’s capabilities include:
- Empathetically assisting with sizing and fit questions, offering product recommendations [00:08:11].
- Providing inventory tracking information [00:08:27].
- Managing package tracking and issuing refunds [00:08:36]. This highlights autonomous agents taking action beyond just answering questions [00:08:49].
The results for Chubbies include helping more customers more quickly and with higher satisfaction [00:08:58]. Chubbies even allocates a budget for its agents to “delight customers,” allowing for proactive solutions like door-dashing shorts from a retail location if unavailable online [00:13:42].
Scalability and Multimodality
The ADLC becomes more effective as customer scale increases, particularly for clients handling tens of millions of requests, where velocity and change management are crucial [00:14:26]. The process also adapts to external changes in the AI space, such as model upgrades, new reasoning paradigms, and multimodality [00:14:40].
Sierra has also applied the ADLC to voice agents, launching voice capabilities in October [00:15:26]. Customers like Sirius XM benefit from Sierra’s voice capabilities to answer calls immediately [00:15:44]. The approach to voice is similar to responsive web design, where the underlying platform and agent code remain the same but adapt to different channels and modalities [00:16:13].
Design Philosophy
Building with AI means working with models that are unpredictable, slow, and not always proficient at math, but also capable of creativity and reasoning [00:16:51]. This non-deterministic nature allows for empathetic design, putting oneself in the “shoes of the robot” or the “primordial soup of the jello” to build good experiences [00:17:10]. Sierra’s approach aims for robustness by providing LLMs with the same inputs and experiences that humans have [00:18:06].