From: redpointai
Intercom’s co-founder and chief strategy officer, Des Traynor, shared insights into the rapid integration of AI into their products following the release of ChatGPT. He highlighted how AI will significantly impact customer support and detailed Intercom’s strategic approach to product development and rollout.
Intercom’s Rapid AI Adoption
When ChatGPT was released, it created an “all hands on deck” moment at Intercom, leading to a significant shift in their AI and ML roadmap [00:01:09]. Traynor noted that customer support is in the “Kill Zone of AI” because large language models (LLMs) are inherently conversational and can perform tasks like looking up facts, reading, understanding, and summarizing information, which are core functions of a customer service representative [00:01:19]. Within months, Intercom launched its AI product, Finn, initially in March and broadly in July [00:01:58]. This quick action was driven by the understanding that if Intercom didn’t lead in AI adoption for customer support, another company would [00:02:40].
AI Integration in Intercom Products
Intercom adopted a “crawl, walk, run” approach to deploying AI features [00:00:22].
Initial “Zero Downside” Features
The first step involved integrating “zero downside” AI features into their existing inbox product using GPT 3.5 Turbo shortly after ChatGPT’s launch [00:03:32]. These features included:
- Summarizing conversations [00:03:57]
- Translating messages for multilingual support [00:04:00]
- Expanding or collapsing text [00:04:03]
This strategy was deemed “good advice for other companies” as it allowed users to opt-in to features without negative impact if they didn’t like the results [00:03:39].
Finn: The User-Facing Chatbot
The next major release was Finn, a user-facing chatbot, made possible by access to the GPT-4 beta. GPT-4 significantly improved the ability to “contain the bot” and reduce “hallucinations,” which was a major concern with earlier models like GPT 3.5 [00:05:15]. Key aspects of Finn’s development included ensuring it was:
- Trustworthy and reliable [00:05:47]
- Able to stay on topic, avoiding political opinions or competitor recommendations [00:05:51]
Inbox AI
Following Finn, Inbox AI expanded on the original inbox features, adding capabilities like matching a customer’s tone of voice or adhering to a standard Intercom tone [00:06:09].
Challenges in AI-Driven Research and Novel Planning
Guardrails and Hallucination Prevention
Guardrails and hallucination prevention are critical for enterprises [00:06:37]. Intercom’s approach involves:
- Torture Testing: Developing a comprehensive set of scenarios, questions, and contexts to test the AI’s behavior, identifying misbehaviors and desired behaviors [00:06:57].
- Weighted Internal Sense: Determining acceptable levels of accuracy and creativity, acknowledging that constraining the AI to reduce errors might also limit its ability to provide correct answers in some cases [00:07:09].
- Prompting and Context Management: Using sophisticated prompting to ensure the model prioritizes given context over its general knowledge, resolving conflicts between multiple data sources [00:08:29].
- Model Evaluation: Continuously evaluating different models (GPT 3.5, GPT-4, Anthropic’s Claude, Llama) across various scenarios based on trust, cost, reliability, stability, uptime, malleability, and speed [00:09:04].
Cost Optimization vs. Exploration
Intercom views itself as still being in “deep exploration mode” rather than primarily focused on cost optimization [00:11:02]. While they recognize that costs can be reduced by using smaller or open-source models, their priority is to build the best possible product enriched with AI [00:11:12].
One major cost challenge encountered was the infeasibility of automatically summarizing all 500 million conversations a month due to the expense of API calls, leading to a user-initiated “summarize” button instead [00:04:56].
Latency is a current forcing function driving the exploration of smaller models, as users expect instant responses [00:13:38]. The expectation is that technology will generally become cheaper and faster over time, so the focus remains on product excellence [00:14:43]. The shift to cost optimization is expected when models “plateau” in their capabilities, making it more sensible to optimize existing solutions rather than continuously exploring new ones [00:15:11].
Missing Tools and Infrastructure
Traynor identified a gap in tooling, particularly around prompt management, including versioning, A/B testing, and managing subtle changes to prompts across different models [00:16:16]. He noted that while this might not be a “billion-dollar opportunity,” it is an area that needs robust solutions [00:16:34]. There’s also a need for more robust infrastructure, such as handling data residency requirements for different regions (e.g., EU servers) [00:16:50].
Intercom had a “running start” in AI partly because they had already developed their own vector search capabilities for their previous AI product, “Resolution Bot” [00:17:41].
Developing and Utilizing AI Models in the Tech Industry - Organizational Structure
Intercom uses a centralized ML team comprising data scientists, AI/ML engineers, and domain experts [00:19:30]. This team, which started with about 9 people and grew to 17-20, builds and enables core AI capabilities (e.g., endpoints for answering questions or suggesting next steps) [00:19:59]. Around 150 “regular product engineers” then build user-facing features on top of these capabilities [00:20:18].
Traynor distinguishes between companies that are “AI as in they’re literally working on the bleeding edge of AI” and those that “use AI” to build new product categories or merely apply AI as “salt and pepper” to enhance existing features [00:21:03]. He suggests that only the latter can rely solely on product engineers with some AI familiarity. The former two categories require dedicated data scientists and experienced AI engineers [00:22:03].
AI Project Uncertainty
Unlike traditional software projects where uncertainty is often resolved in the design phase, ML projects introduce a second wave of uncertainty: whether the desired functionality is even possible [00:22:32]. This means companies need to view AI work as a “portfolio of bets,” with some having high probabilities of success (e.g., text expansion) and others being lower-probability, higher-reward endeavors (e.g., generating vector graphics from text) [00:23:39].
Product Strategy for AI
Incremental Adoption
Intercom helps customers transition from “AI curious to all in on AI” by enabling them to “dip their toe in” [00:26:57]. They offer options like:
- Applying AI only to free users or users who have been with the product for over a year [00:27:21]
- Using AI only during weekends or for specific query types [00:27:45]
This approach allows customers to see the value, often leading to a rapid desire for broader adoption once they realize the benefits (e.g., free users getting better support than paid users due to instant answers) [00:28:09].
Future of AI in Customer Support
Traynor predicts that the percentage of requests handled by AI will vary by industry. E-commerce, with its limited types of queries, could see nearly 100% AI handling [00:33:19]. More complex products like Google Docs, with thousands of query types, might reach 80-90% automation [00:33:57].
A key development for AI in customer support is the ability to take actions beyond just providing text answers, such as issuing refunds or canceling orders by integrating with other services like Stripe [00:35:04]. This involves significant software development for authentication, monitoring, and data logging [00:38:05].
He also envisions a future where AI acts as an assistant to human agents, proposing actions or solutions that the human manager can approve or deny based on broader context, similar to how a human supervises a self-service checkout [00:36:02].
Shifting Product Landscape due to AI
Advice for Startups
Startups should target areas where the “incumbent tech stack is pretty much irrelevant” [00:39:39]. This means looking for product categories where, if incumbents were to start again, they would build it “entirely differently” with little of the old UI or features surviving [00:41:41]. For example, an advertising optimization startup might be replaced by a fixed-fee service that automatically generates, runs, measures, and optimizes ads without user login [00:39:34].
Conversely, areas like email sending platforms (e.g., MailChimp, Klaviyo) are less vulnerable because their core challenge lies in the complex backend infrastructure (reputation, compliance, deliverability) rather than just email content generation, which AI could easily handle [00:40:33]. In such cases, AI features become an “add-in,” but the barrier to entry for a new startup remains high [00:41:09].
Advice for Incumbents
For larger companies, Traynor suggests an algorithm for AI integration:
- Find Asymmetric Upside: Start with simple AI applications to understand costs and latency [00:42:19].
- Workflow Breakdown: Analyze every product workflow (e.g., creating projects, assigning tasks, planning calendars) [00:42:38].
- Automate and Delete: If AI can reliably perform a workflow, make AI do it and remove the old functionality [00:43:04].
- Augment and Optimize: If AI can’t fully remove a workflow, use it to augment it or reduce it to a simple decision [00:43:32].
- Sprinkle AI: Add AI as “salt and pepper” to enhance the user experience and maintain a perception of being AI-complete [00:43:51].
- Educate Customers: Actively explain the value of AI to customers and how it will work [00:44:12].
Overhyped and Underhyped AI Trends
Overhyped
Traynor believes AI-powered “productivity tools” that write emails or sales pitches are overhyped [00:44:35]. He predicts people will learn to detect AI-generated content, and the focus will shift back to good human writing [00:44:42].
Underhyped
He sees the impact of AI on creativity as underhyped [00:44:56]. Comparing it to Instagram’s filters that made everyone feel like photographers, he points to tools like Kaiber (for video), Refusion (for sound augmentation), and Synthesia (for video generation) as examples of a new type of creativity that is not yet fully understood [00:45:21].
Incumbent AI Adoption
Impressive Incumbents
Beyond Microsoft and Google, Traynor was impressed by Adobe’s swift integration of AI-driven tools in Adobe’s Creative Cloud [00:46:08]. Figma and Miro also impressed him by finding genuinely useful AI applications that make sense within their products, avoiding “quick misapplications” to merely add “AI” to their homepage [00:46:21].
Disappointing Incumbents
He expressed “frustration” and “surprise” at how long it’s taking Apple and Amazon to adopt AI more broadly [00:46:48]. He noted the stark contrast between the primitive capabilities of Siri or Alexa (e.g., playing the next song) and the advanced conversational abilities of ChatGPT (e.g., generating a multi-minute story for a child) [00:46:53]. Traynor believes that when major platforms like Apple (via Siri) and Google (via Bard or “Okay Google”) properly integrate LLMs, it will normalize the idea of talking to software and drastically increase AI adoption, making AI enrichment a competitive battleground [00:29:26].