Tubegraph

AI models and the product development process

6 min read

From: redpointai

Perplexity AI, a next-generation search product, has gained significant traction, raising a $500 million valuation in recent months [00:00:15]. The company’s CEO, Arvin Seros, shares insights into their product development philosophy, the role of AI models, and strategies for navigating the competitive AI landscape.

Core Product Development Philosophy

Perplexity AI’s development is guided by five key dimensions to ensure a top-tier product:

Achieving excellence across all five dimensions simultaneously is crucial for creating a multi-billion dollar company [02:09:00].

Complexity Behind the Seams

Despite the simplicity of its user interface, Perplexity AI involves significant underlying complexity [01:06:00]. When a user asks a query, the system undertakes a series of actions:

  • Query Understanding and Reformulation: The system tries to comprehend and rephrase the query [02:20:00].
  • Source Selection: Identifying which web pages are best to use for the query [02:24:00].
  • Content Extraction: Determining which parts of the selected pages are relevant for answering [02:28:00].
  • Answer Rendering: Deciding how the answer should be presented (e.g., summary, bullets) [02:33:00].
  • Citation and Error Minimization: Ensuring each sentence has supporting citations and minimizing hallucinations [02:41:00].
  • Multimedia Integration: Incorporating images or videos when text alone is insufficient [02:52:00].
  • Shareability: Making answers easily shareable via permalinks, an innovation later adopted by competitors [03:09:00].
  • Follow-up Questions: Suggesting further questions to help users articulate their curiosity, as people are often not good at asking precise questions [03:30:00]. This led to the “co-pilot” feature, which assists users in prompt engineering [04:25:00].

AI Model Pretraining and Fine-tuning Decisions and AI Model Selection

Perplexity AI’s journey reflects a strategic evolution in AI model development:

  • Starting with Off-the-Shelf Models: Initially, the company leveraged models from OpenAI [07:27:00]. The advice for product-focused companies is to begin with existing models to quickly launch a product, gain users, and secure funding and talent [07:59:00].
  • Transition to Fine-tuning and Open Source: Perplexity later fine-tuned smaller, faster models and increasingly utilized open-source models like LLaMA and Mistral [07:34:00]. This shift was driven by a desire to reduce dependency on third parties, especially competitors, and to control costs [10:30:00]. The company waited for the “open-source wave” to arrive, like LLaMA 2, to position themselves effectively [10:49:00].
  • Building Internal Capabilities: Over time, Perplexity has built its own inference engine and models fine-tuned on open-source foundations, aiming to rival models like GPT-3.5 Turbo [24:21:00]. This move is to address user concerns about being a “rapper” (a company merely wrapping another’s API) and to earn trust by demonstrating internal infrastructure capabilities [24:40:00].
  • Model Agnosticism: Despite building their own models, Perplexity aims to remain model agnostic, prioritizing giving users the best answer, regardless of the underlying model [13:44:00]. Users care about the best answer, not the technology stack [14:06:00].

Product Development and Prioritization

Organizational Structure

The company emphasizes vertical integration, ensuring designers, product engineers, and backend teams collaborate closely [06:01:00]. Regular meetings foster appreciation for all aspects of product development, aligning company values (quality, truth, velocity) with product goals [07:05:00].

Balancing User Types

A key challenge is balancing the needs of power users with those of new users [35:05:00]. While power users provide valuable feedback, focusing solely on their requests can make the product unintuitive for new users and hinder growth [35:45:00]. The goal is to find a sweet spot that caters to a broad audience [36:24:00].

Overcoming Hallucinations with RAG

Perplexity has been effective at using Retrieval Augmented Generation (RAG) to solve hallucinations in web search [27:45:00]. However, the CEO warns against universal solutions, noting that solving RAG for internal enterprise search is vastly different from web search due to unique indexing, embeddings, and ranking requirements [28:18:00]. The quality of retrieval is critical, as simply dumping information into a large language model can increase hallucinations [30:07:00].

Evolution of the Product and Go-to-Market Strategy

Perplexity AI’s success stemmed from iterating on different ideas before finding product-market fit:

  • Initial Ideas: Early concepts included a vision-based search (glasses with microphone/airpods) and search over databases (text-to-SQL for enterprise spreadsheets) [38:58:00]. The text-to-SQL idea faced challenges as most SQL is recurring, and new queries are often generated visually rather than typed [40:46:00].
  • Twitter Search: The team built a graph search over Twitter data, allowing users to find connections or specific tweets [42:46:00]. This led to a viral moment when Jack Dorsey tweeted about it, bringing tens of thousands of users to the site and causing a crash [46:51:00]. The system’s ability to summarize social media activity across platforms for a given handle “spooked people out,” leading to viral screenshots and further adoption [48:01:00].
  • Consumer-Facing Search: The summarization-based search, initially a Slack bot for internal use, was chosen for public launch because it resonated with early users who found it “better than Google” for quick, cited answers to complex questions [44:56:00]. The viral usage during the winter vacation period confirmed the product’s real demand [48:48:00].

The "User is Never Wrong" Philosophy

Larry Page’s early Google demos showed him insisting that the user was never wrong for not typing a query “the right way” [05:20:00]. This philosophy guides Perplexity AI’s development, leading to features like Co-pilot that assist users in formulating queries [04:25:00].

The Future of Search and AI

The future of search, according to Perplexity AI’s CEO, will be primarily about “answers” delivered by agents that perform tasks [22:23:00]. It will resemble natural conversation with friends, concise enough to be useful without rambling [23:31:00].

The company’s focus is on maximizing “knowledge velocity” – accelerating the transfer of knowledge [15:36:00]. Unlike static Wikipedia articles or slow Quora answers, Perplexity aims to provide fast, personalized, sourced answers, essentially automating the research process that would take humans hours [16:51:00].

Challenges in AI Product Development

Competition and Differentiation

Competing with incumbents like Google requires a different approach than replicating their technology [20:17:00]. Previous Google challengers focused too much on technology and not enough on market positioning or unique go-to-market strategies (e.g., privacy for DuckDuckGo, crypto for Brave) [21:17:00]. The emergence of powerful, accessible LLMs was a rare “failure” for Google, creating an opportunity for companies like Perplexity to leverage the technology [22:46:00].

Talent Acquisition

Recruiting skilled engineers and AI researchers is more challenging than raising capital in the current generative AI landscape, as top talent is scarce and crucial for destiny-changing impact [09:14:00].

Regulation

Regulation of AI is currently premature, as the widespread economic benefits are not yet fully realized [33:30:00]. Over-regulation could stifle innovation and centralize AI development among a few well-funded entities [54:47:00]. Transparency and broad participation are preferred to ensure safety and prevent control by a select few [55:05:00].

Personal Vision for AI’s Future

The CEO expresses a personal desire to use AI models to preserve memories of loved ones, such as recording parents extensively to create AI voices or personas for future interaction [56:03:00]. This reflects a broader belief in AI’s potential to democratize creative processes, like movie-making, by significantly reducing creation costs [56:55:00].

Graph View

Backlinks