From: aidotengineer
Sierra is a conversational AI platform designed for businesses [00:00:27]. Initially known for chat experiences and customer service, Sierra is expanding its offerings significantly [00:00:41]. By the end of the year, most interactions are projected to occur over the phone [00:00:53]. The platform serves customers across various functions including sales, subscription management, and product recommendations, covering all aspects of the customer experience [00:01:02].
Evolution of AI and Sierra’s Approach
Reflecting on the rapid advancements in AI, the speaker notes that previous discussions often framed “ancient history” in AI as starting in the 2020s [00:01:45]. The speaker, Zach Reno, brings a perspective from 2016, when he worked at Google on the first version of Google Lens [00:02:01]. This involved training computers to distinguish between visually similar objects, such as Chihuahuas and blueberry muffins [00:02:15].
The experience of building user experiences with AI then, highlighted the non-deterministic nature of AI models, where consistent results were not guaranteed, feeling like a “slot machine” [00:03:48]. This emphasizes a key challenge in AI development: the non-determinism of inputs or outputs [00:03:59].
Google Lens has since evolved significantly, now offering features like visual search, shopping integration, text translation, and even math homework assistance [00:04:07]. This progress is attributed to consistent, step-by-step iteration over a decade, guided by a robust software development life cycle [00:04:43].
Partnership with Chubbies
Chubbies, a brand known for its forward-thinking approach to customer experience, partnered with Sierra to deploy an AI agent named Duncan Smothers [00:07:14]. The collaboration was based on the belief that in 2025, an AI agent will be as essential for businesses as a website was in 1995 or a social profile in the 2000s [00:07:25].
Duncan Smothers is an AI agent on the Chubbies website capable of handling various customer inquiries:
- Sizing and Fit: Empathetically helps customers with sizing questions and offers product recommendations [00:08:11].
- Inventory Tracking: Informs customers about product stock and helps them choose new items [00:08:27].
- Package Tracking and Refunds: Provides multiple tracking numbers for orders and can issue refunds, demonstrating autonomous action beyond just answering questions [00:08:34].
The results for Chubbies include helping more customers more quickly and with higher satisfaction [00:08:58].
The Agent Development Life Cycle (ADLC)
Sierra views every AI agent as a product, necessitating a fully featured developer and customer experience operations platform [00:09:04]. The company employs dedicated agent engineering and product management teams that work directly with customers [00:09:32].
The Agent Development Life Cycle (ADLC) is Sierra’s process for building and improving AI agents [00:12:12]. It borrows concepts from the traditional software development life cycle but adapts them for the unique challenges of large language models (LLMs) [00:11:24].
LLM Challenges vs. Traditional Software
While traditional software is deterministic, fast, cheap, and rigid, LLMs are non-deterministic, slow, expensive to run, but flexible, creative, and capable of reasoning [00:11:42]. The ADLC aims to leverage LLM strengths while integrating traditional software where beneficial [00:12:02].
Quality Assurance in ADLC
A critical part of the ADLC is iterative refinement with customers in production [00:12:35]. Sierra provides an “Experience Manager” allowing customers to:
- Review every conversation an agent has [00:12:53].
- View high-level reports on agent performance in real-time [00:12:57].
- Provide feedback, which leads to issue filing, test creation, and eventually a new release [00:13:02]. Over time, a Sierra agent evolves from a handful of tests at launch to hundreds and thousands as it improves [00:13:26]. Additionally, agents can be empowered to go “above and beyond,” such as having a budget to delight customers [00:13:37].
Improvements in AI, such as reasoning models, act as a “force multiplier” in the ADLC, making development, testing, and QA more effective [00:15:02].
Voice AI Capabilities
Sierra has significantly invested in voice AI engineering challenges and solutions, launching voice capabilities generally available in October [00:15:26]. One large customer, Sirius XM, utilizes Sierra’s voice capabilities to answer calls immediately, every time [00:15:40].
Sierra’s approach to voice AI is akin to responsive web design: the same underlying platform and agent code can adapt to different channels and modalities [00:15:53]. This allows for customization of phrasing or layout while maintaining a unified system [00:16:27].
Philosophy of AI Agents
The fascinating aspect of building AI in business platforms is that LLMs remind us of ourselves: they are unpredictable, slow, and not great at math [00:16:46]. However, this human-like unpredictability also enables designers to build with empathy, putting themselves in the “shoes of the robot” to create better experiences [00:17:02].
Sierra focuses on building a reliable conversation system for voice agents by creating robust agents that receive the same rich inputs and experiences that humans do, going beyond mere transcribed text with delay [00:17:40].