Tubegraph

Intercoms AI Strategy and Implementation

9 min read

From: redpointai

Intercom, a customer messaging platform, rapidly integrated AI capabilities into its product suite following the public release of ChatGPT in late 2022 [00:00:10]. The company, co-founded by Des Traynor, saw customer support as being “in the kill zone of AI” due to large language models’ inherent conversational abilities and their capacity to look up, understand, and summarize facts [00:02:17]. This realization led to an “all hands on deck” moment, shifting Intercom’s entire AI/ML roadmap to go “all in” on the new technology [00:01:48].

Rapid AI Adoption at Intercom

Intercom’s AI team, based in Dublin, Ireland, quickly recognized the potential of large language models after ChatGPT’s launch [00:00:37]. Des Traynor recalls playing with ChatGPT on his phone, initially testing its ability to answer factual questions, then being genuinely impressed when it could perform more creative tasks, like writing a song in the style of Rage Against the Machine about installing a Windows driver [00:01:00].

The company moved swiftly:

  • Before Christmas 2022: Shipped an initial AI product [00:01:58].
  • January 2023: Had a “reasonable release” [00:02:02].
  • March 2023: Launched Finn, their user-facing chatbot, initially [00:02:03].
  • July 2023: Broadly launched Finn [00:02:06].

This rapid pace was driven by the understanding that if Intercom didn’t lead in AI adoption for customer support, another company would [00:02:40].

Intercom’s AI Product Suite

Intercom’s initial strategy for adopting AI was to build “zero downside” features, where users could opt-in, and if they didn’t like the AI’s output, they could simply not use it [00:03:39].

Inbox AI Features

The first AI-powered features were integrated into the Intercom inbox, primarily using GPT-3.5 Turbo [00:03:43]. These included:

These features provided immediate value, leading customers to ask for automatic summarization [00:04:48]. However, automatically summarizing 500 million conversations per month was cost-prohibitive with initial model pricing [00:04:56].

Finn Chatbot

The next major release was Finn, a user-facing chatbot, launched after Intercom gained access to the GPT-4 beta [00:05:16]. GPT-4 significantly reduced hallucinations, allowing for a more contained and trustworthy bot [00:05:27]. Key developments for Finn included ensuring it was:

  • Trustworthy and reliable [00:05:47].
  • Capable of staying on topic (e.g., not giving political opinions or recommending competitors) [00:05:51].
  • Able to match a standard or customer’s specific tone of voice [00:06:16].

Finn has already provided over two million answers and is used by thousands of people, with answers being high-quality and capable of handling complex, multi-part questions [00:28:50].

Key Challenges and Solutions

Intercom faced several challenges in its AI implementation journey.

Managing Hallucinations and Guardrails

A core ingredient to managing hallucinations and ensuring appropriate behavior is a robust “torture test” [00:06:57]. This involves:

  • A long set of scenarios, questions, and contexts to observe the AI’s behavior [00:07:33].
  • Internal weighting to determine acceptable trade-offs between answer quality and occasional misbehavior (e.g., political opinions) [00:07:09].
  • Prioritizing given context over the LLM’s general knowledge (e.g., local laws for sunbeds) [00:08:23].
  • Using sophisticated prompting techniques to resolve conflicts between different data sources [00:08:33].

Intercom continuously evaluates models like GPT-3.5, GPT-4, Anthropic’s Claude, and open-source models like Llama against these scenarios, considering factors like trust, cost, reliability, stability, uptime, malleability (control), and speed [00:09:04].

Cost Optimization vs. Exploration

Initially, the cost of automatically summarizing 500 million conversations monthly would have been prohibitive, leading to the decision to make it a user-triggered feature [00:04:56]. However, Intercom remains in “deep exploration mode” for AI applications rather than prioritizing cost optimization [00:11:02].

  • The primary focus is on building the best possible customer support platform enriched with AI [00:14:09].
  • The belief is that technology generally gets cheaper and faster, so even if left untouched, models will improve [00:14:43].
  • Cost optimization will become more critical when models plateau in their capabilities, indicating a shift from acceleration to a more mature phase of the S-curve [00:15:11].

Latency Concerns

Speed is a significant factor in AI system performance [00:09:25]. Current AI interactions can feel slow, akin to “modem internet days” [00:12:27]. The expectation is that advancements, potentially like Apple integrating an LLM directly into phones or Google’s Gemini builds, will lead to “instant AI” [00:12:44]. For Intercom, latency is currently a more pressing forcing function than cost for exploring smaller or more localized models [00:13:38].

Regional Compliance (EU)

Operating AI solutions globally presents challenges, particularly with regional compliance like the EU. Getting Finn to work in the EU was complex due to server locations and data regulations, leading to unexpected partnerships, such as with Microsoft Azure [00:17:00].

Organizational Approach to AI

Centralized AI/ML Team

Intercom employs a centralized AI/ML team, comprising 17-20 people (initially around 9) with deep domain expertise in building, running, training, and using models [00:19:38]. This team enables product engineers (around 150) to build customer-facing features by providing API endpoints for tasks like answering questions or suggesting next steps in a conversation [00:20:20].

“There’s about like our our team today in total is about 17 maybe 20 people uh um in that and but when we started on this journey it was probably like nine or something like that to be clear uh I think um you know there’s a few threads I’d pull on like it’s not you know people often forget this but small teams can do an awful lot” [00:19:53]

This centralized model is critical for companies that are “AI first” or “AI as in they’re literally working on the bleeding edge of AI” [00:21:07], requiring data scientists and experienced AI engineers [00:22:04]. Companies merely “sprinkling” AI onto their products might get by with product engineers dabbling in Open AI specs [00:21:49].

AI Project Management: Portfolio of Bets

Developing AI-powered software differs from traditional software development due to a second wave of uncertainty: whether the AI functionality is even possible [00:23:01]. Unlike traditional projects where risks are often mitigated at the design stage, AI projects can lead to prolonged efforts without a clear “no” on feasibility [00:23:10].

“The worst part about the is any of this [__] even possible is that you don’t even know if you’ll ever know the answer to that question like you know all you know is it’s it hasn’t started working yet and you’ll never actually have a clean no it’s firmly not possible” [00:23:08]

Therefore, AI development should be viewed as a “portfolio of bets” [00:23:39]:

  • High Probability Bets: Features like expanding or rephrasing text have a 99% probability of success [00:23:42].
  • Low Probability Bets: More ambitious features, such as generating editable vector graphics, might only have a 20-40% probability [00:24:08]. The challenge is that one may never definitively know if they are impossible [00:24:16].

An example of a tricky problem is AI-powered sentence completion in customer support, which faces challenges in distinguishing personal answers, handling PII, and abstracting irrelevance from context [00:24:50].

Strategic Insights and Future Outlook

AI Adoption Curve in Customer Support

Customers are moving from “AI curious to all in on AI” [00:26:57]. Intercom facilitates this transition by offering low-risk ways to adopt AI, such as:

  • Piloting AI for free users [00:27:22].
  • Limiting AI usage to weekends or specific query types [00:27:43].

This “dip your toe” approach helps customers realize the value, often leading to them wanting to expand AI use after seeing superior support for pilot groups [00:27:52]. A key enabler for widespread adoption will be major tech companies like Apple and Google fully integrating LLMs into their consumer products (e.g., Siri, Bard), which will normalize “talking to software” [00:29:26].

Evolution of AI in Workflows

The percentage of requests handled by AI will vary significantly by vertical [00:33:17]:

  • High Automation (e.g., e-commerce): Simple, repetitive queries (e.g., “where is my order?”) mean nearly 100% automation is possible [00:33:31].
  • Mixed Automation (e.g., complex software): Products like Google Docs generate diverse questions, making 100% automation unlikely, but 80-90% is achievable [00:34:02].

AI agent capabilities and limitations are expanding beyond just text-based answers to include performing actions (e.g., issuing refunds on Stripe, canceling orders) [00:35:06]. This involves writing significant code for authentication, monitoring, and data logging [00:38:08]. The future might see AI suggesting complex actions for human approval, turning support reps into “line managers” who oversee AI operations [00:36:23].

  • Overhyped: Productivity tools focused on generating content like emails or sales pitches [00:44:33]. Traynor believes people will learn to detect AI-generated content, and filters will emerge, leading to a renewed appreciation for human writing [00:44:42].
  • Underhyped: The transformative impact of AI on creativity [00:44:56]. Similar to how Instagram filters made everyone feel like photographers, tools like Midjourney, Refusion (for sound), and Synthesia (for video) are enabling new forms of creativity that are yet to be fully understood [00:45:20].

Industry Impressions

  • Most Impressed By: Adobe (for quick AI integration), Figma, and Miro (for finding useful, sensible AI use cases) [00:46:08]. Shopify and Cot also received positive mentions [00:46:41].
  • Most Disappointed By: Apple and Amazon [00:46:48]. Current voice assistants like Siri and Alexa seem primitive compared to advanced LLMs like ChatGPT, which can generate complex, long-form stories [00:47:01]. The hope is for a “leveling out” in 2024 with more widespread consumer adoption of advanced AI [00:47:41].

For startups, the advice is to target areas where incumbent “tech stack is pretty much irrelevant” [00:41:39], meaning if they started over, they would build it entirely differently with AI at the core, rendering existing UI and features obsolete [00:41:48]. For incumbents, the recommended algorithm is:

  1. Remove what AI can remove: Delete features or workflows that AI can automate entirely [00:43:04].
  2. Optimize what remains: If AI can’t remove it, let it augment or simplify the workflow into a clear decision set [00:43:32].
  3. Sprinkle AI where possible: Add AI touches for completeness, even if not core to efficiency [00:43:51].
  4. Learn to sell the value: Effectively communicate the benefits of AI to customers [00:44:12].

This outlines strategic uses of AI in enterprises and challenges and strategies in enterprise AI deployment.

For more information, visit intercom.com or intercom.com/blog. Des Traynor can be found as @destraynor on social media. [00:48:02]

Graph View

Backlinks