Features and enhancements in Notion AI

From: redpointai

Notion AI is a suite of artificial intelligence features integrated into the Notion workspace, designed to augment user productivity across various tasks. Linus Lee, who helps lead Notion’s AI efforts, discusses the development and evolution of these features, highlighting their rapid deployment and user-centric approach [00:00:07].

Origin of Notion AI

The idea for Notion AI originated from a company offsite in late 2022. Notion’s CEO, Ivan, and co-founder, Simon, took a few days to experiment with GPT-3, which had just been released [00:03:00]. This hackathon-style exploration led to the first version of Notion AI, initially known as “Notion AI Writer” [00:03:32].

Key Features

Notion AI Writer

Launched in beta in November 2022 and generally available in February 2023, Notion AI Writer is designed to assist users directly within their documents [00:03:38]. Its capabilities include:

• Summarization: Condensing pages into various formats or extracting key ideas, action items, and topics [00:03:48].
• Editing and Revision: Fixing spelling and grammar mistakes, and improving writing style and voice [00:03:56].
• Drafting: Helping users generate initial drafts or marketing copy [00:04:05].
• Iterative Refinement: Users can conversationally follow up on generated content, asking for adjustments like making it shorter or more “punchy” [00:04:30].

AI Autofill

Released in May 2023, AI Autofill integrates AI into Notion databases, which are often used for project management [00:04:39] [00:05:19]. It can automatically populate entire columns or “properties” within a database [00:04:57]. Examples include:

• Extracting key topics from meeting notes [00:05:04].
• Pulling out core needs or company roles from user interview transcripts [00:05:08].
• Translation: Initially an unanticipated use case identified from user behavior, translation is now a built-in prompt [00:18:34] [00:19:34].

Notion Q&A

Notion Q&A is a newer feature that allows AI to understand the entire workspace and answer questions, potentially spanning multiple pages [00:05:32] [00:05:35]. It addresses the common problem of finding information in increasingly complex organizational workspaces [00:06:01] [00:06:08].

• Information Retrieval: Solves the “information finding problem” by grounding AI in factual information from the user’s workspace, leveraging retrieval-augmented generation (RAG) [00:06:28] [00:06:36].
• Contextual Understanding: Supports multi-turn conversations, allowing users to ask follow-up questions where the AI retains context from previous interactions [00:35:24].
• Multilingual Capability: Can read documents in one language (e.g., Japanese) and translate the answer back to the user’s query language [00:39:21].

Notion AI Team Structure and Development Process

Notion’s AI team operates with an “expand out and then contract” cycle, balancing user problem-solving with exploration of new AI capabilities [00:07:27] [00:07:00].

• Prototyping: The team quickly prototypes ideas, often using “dogfooding” (internal testing) to determine which approaches are most promising [00:08:03] [00:10:02]. An “annoying prototype” for Q&A, for instance, forced rapid iteration and quality improvement due to constant internal exposure [00:10:19].
• Team Composition: The core AI team consists of about a dozen people, split between data/model quality and product concerns like interface and integration [00:12:05]. They also collaborate closely with designers [00:12:41].
• Organizational Structure: The AI team owns both backend and UI components for specific AI “surfaces” (like AI Writer and Q&A chatbox) [00:13:37]. Features like Autofill are more deeply integrated with existing product teams (e.g., databases team), requiring tight collaboration [00:13:40]. There’s an ongoing discussion about centralizing foundational AI technologies versus embedding AI engineers in every product team [00:14:10] [00:15:56].

User Interaction and Interface Design

Notion prioritizes understanding how users interact with AI products to inform design and education [00:16:54].

• Internal Dogfooding: Internal usage provides the richest source of information on useful ways to use AI, though Notion employees may not be fully representative of all user segments (e.g., individuals, students) [00:17:14].
• Early Testers and Ambassadors: External partners often discover unanticipated use cases, which can inspire new features or built-in prompts, like the translation feature for Autofill [00:18:00].
• Pre-built vs. Custom Prompts: Notion provides pre-baked “formulas” or templates for common tasks (e.g., summarization, improving writing, fixing grammar) [00:19:11] [00:19:26]. While these are the most popular, advanced users often start with them and then iterate using custom prompts to fit specific needs [00:19:50]. This approach helps users overcome the “blank canvas problem” [00:20:47].

Challenges in Deploying AI

Notion faced several challenges in deploying AI features:

• Evaluation and Quality Control: Ensuring correctness and quality is particularly hard for Q&A, where answers are expected to be specific. Unlike creative writing, there’s less “wiggle room” for errors [00:23:14].
  • Edge Cases: Anticipating and handling various types of questions, such as “meta” questions about Notion itself (e.g., “how can I share this page with Jack?”) or questions involving time ranges (e.g., “what is the marketing team working on this week?“) [00:24:16] [00:24:36].
  • Evaluation Tools: Notion built most of its evaluation tools in-house due to the complexity and structured nature of Notion documents (rich text, tables, images, metadata) [00:26:28]. These tools allow for faster iteration and comparison of model outputs [00:27:24]. Evaluation involves a spectrum from deterministic programmatic checks to human annotators and in-depth analysis by ML engineers to understand why models fail [00:28:38].
• Operational Concerns: Addressing customer needs around privacy and security, and ensuring the system can handle scale for language model-augmented features [00:25:33].
• Model Selection and Integration: Notion partners with Anthropic and OpenAI for their foundational models, acknowledging the difficulty of competing at the infrastructure level [00:31:08] [00:31:22].
  • Data Privacy: A key commitment is not training models on customer data [00:31:57]. This encourages systematic thinking about generating synthetic data that accurately represents Notion workspace archetypes [00:32:11].
  • Task Understanding: Notion’s role is to understand the specific tasks its models need to perform and build appropriate datasets and evaluations around them, recognizing that criteria for a “good summary” differ significantly between document types (e.g., meeting notes vs. technical documents vs. bug reports) [00:32:42] [00:33:04].
  • Prompt Engineering: Prompts are often heavily wrapped in Notion’s own processing, including context about the page or workspace [00:34:58]. While general instructions carry over between models, fine-tuning for specific model quirks is sometimes necessary [00:38:11].

Future Outlook and Philosophy

Linus Lee believes that while context length in models is “overhyped” for most useful tasks [00:42:24], alternatives to Transformers are “underhyped” and could offer more efficient sequence modeling [00:44:36] [00:44:50].

• Generality over Specificity: He suggests that investing in general approaches that can solve multiple tasks (e.g., a single model trained on diverse datasets for various functions) yields longer-term payoffs by enabling the model to better understand the product’s domain [00:47:00].
• Generative Interfaces: Notion is exploring models that can output interactive UI elements instead of just text [00:47:24]. This requires defining the “building blocks” or “component library” for AI-generated interfaces, akin to how Notion provides rich blocks for user customization [00:40:48]. Giving AI more control over UI generation is a harder ML problem but could lead to more interesting outcomes [00:47:48].