From: redpointai
Perplexity AI, an “incredible next-gen search product,” has gained significant traction, raising $500 million in valuation recently [00:17:00]. The simplicity of its user experience belies the underlying complexity of its development, which involves intricate model building and integration strategies [01:06:00].
Evolution of Model Strategy
Perplexity AI’s approach to model development has evolved strategically. Initially, the company started by using off-the-shelf OpenAI models [00:27:00]. Over time, they transitioned to fine-tuning smaller, faster models, and have since incorporated open-source models, even releasing their own [00:37:00]. This journey is seen as a deliberate, pragmatic choice, with the CEO stating he would “100% want to… do it exactly the same way” if he were to go back [00:57:00].
The company’s strategy involves:
- Starting with readily available models: For product-focused companies, it’s advised not to “waste your time building your own models” initially [00:08:08]. The primary goal is to get a product out, ensure it has users, and sustain usage to attract funding and talent [00:08:40].
- Building internal expertise: While starting with external models, Perplexity benefits from co-founding team members being former AI researchers who understand model training [00:08:12].
- Waiting for the “next wave”: Decisions to build proprietary models or leverage open-source solutions (like Llama 2 or Mistral) were timed to coincide with major advancements in the open-source community [00:10:50].
Balancing Product Focus and Model Development
Perplexity’s strategy emphasizes product focus over raw model development, especially in the early stages. The philosophy is that a great engineer will join a company with a product that has users, rather than one solely focused on infrastructure or foundational models without a market [00:09:22].
A core belief is that the “user is never wrong” [00:05:27]. This user-centric approach drives decisions on product features and model choices. For example, the “Co-pilot” feature was introduced to help users formulate better queries, acknowledging that “we’re not very good at asking follow-ups” [00:03:44].
Combating Hallucinations and Improving Search Quality
A key challenge in AI, particularly for search, is the phenomenon of hallucinations. Perplexity uses Retrieval-Augmented Generation (RAG) to address this, backing answers with citations [00:02:42]. However, solving hallucinations isn’t a simple “plug-and-play” solution, especially for enterprise AI model management or internal search [00:28:11].
Key aspects of Perplexity’s approach to search quality and RAG include:
- Holistic Approach: It’s not just about training a large embedding model. The process involves significant work in indexing, snippet generation, text retrieval, and advanced ranking signals beyond just vector dot products [00:28:32].
- Context Management: Counter-intuitively, throwing more information at long-context models can increase the chance of hallucinations; therefore, precise retrieval is crucial [00:30:07].
- Personalization: The goal is to maximize “knowledge velocity” or “IQ velocity,” providing fast, high-bandwidth access to personalized knowledge, unlike static platforms like Wikipedia [00:15:36].
Organizational Philosophy and Vertical Integration
Perplexity operates with a vertically integrated approach, where designers, product engineers, and backend teams work closely together [00:06:01]. This ensures that design decisions can feed data back into AI models, making the product smarter [00:06:17]. The company’s core values—quality, truth, and velocity—are aligned with its product goals, ensuring everyone appreciates the importance of good design, great answers, speed, and reliability [00:07:05].
The “Rapper” Analogy and Strategic Independence
Perplexity openly embraces the initial “rapper” label (companies that “wrap” around foundation models like ChatGPT). The CEO states, “I would rather be a rapper with 100,000 users than having some model inside and like nobody even knows who I am” [00:12:42]. The strategy is to gain users and then gradually build the capabilities to serve their own models, reducing dependence on third-party providers, especially those building competing products [00:10:28].
“You earn the user trust right like a lot of users are concerned about using our products simply because they think that we cannot build our own infrastructure so we just a rapper so at some point we’re going to fiz out we’re going to run out of money we don’t have any business so that’s that’s the number one part that we um want to address hey guys listen you know we we’re not a rapper we’re slowly building our muscle to serve everything ourselves” [00:24:43]
Perplexity aims to be “model agnostic,” prioritizing the best answer for the user regardless of the underlying model [00:14:06]. However, having the option to control their destiny and drive down costs by serving their own models is crucial [00:13:49].
Future Outlook on Models and AI Development
Perplexity believes that what is currently possible with state-of-the-art closed models (like GPT-4) will eventually be achievable with open-source models at a lower cost and faster latency [00:51:59]. This will continuously open up new possibilities for advanced features like dynamic prompt engineering and generative user interfaces.
The long-term vision for search involves becoming “answers,” provided by “agents that just do tasks for you,” simulating natural conversations with friends [00:23:25]. This aligns with the broader trend of compound AI systems, enabling complex tasks and multimodal reasoning (e.g., uploading a video and asking questions about it) [00:53:10].
Regarding AI regulation, Perplexity’s CEO considers it “too premature” given that the widespread economic benefits haven’t been fully realized [00:53:30]. He argues that stifling development now could be detrimental, and that more “eyeballs” and widespread development are better for addressing safety concerns than centralizing control [00:55:03].