From: redpointai
Percy Liang, a leading AI researcher and co-founder of Together AI, provides insights into the evolving role of academia in the field of AI development [00:00:03]. He highlights how academic institutions, such as Stanford where he is based [00:10:13], must adapt their research strategies to remain relevant amidst the rapid advancements from large commercial labs.
Academia’s Unique Position
Academia faces a unique challenge as powerful models like OpenAI’s O1 (GPT-4o) and GPT-5 emerge, possessing significantly more resources than academic institutions [00:10:38]. Liang emphasizes that directly competing with these entities “head and head” is pointless [00:10:46]. Instead, academia should play a different, complementary role [00:10:55].
Orthogonal Research Strategies
For academic research to thrive, it must be “orthogonal” to the progress made by large model developers [00:11:02]. This means selecting research projects that are either:
- Enhanced by better models: If models like GPT-4 or GPT-5 improve, the academic research should benefit. For example, Liang’s work on generative agents, which creates virtual worlds where AI agents interact, is enhanced by more capable language models [00:11:21]. The goal is not the raw language model, but novel use cases built upon them [00:11:40].
- Irrelevant to model advancements: The core contribution of the research should not be directly superseded by new model releases [00:11:16].
This approach is similar to that taken by startups in the AI space, which benefit from, rather than compete against, foundational model advancements [00:14:33].
Emphasis on Open Science and Open Source
A crucial role for academia is contributing to the open source models in AI development [00:12:06]. Academia is inherently about “open science” and creating knowledge for the public domain [00:12:15]. Unlike commercial labs, which may keep knowledge proprietary, academia can discover or even reinvent concepts and publish them [00:12:31]. This fosters a broader community adoption, leading to new models and products [00:12:40]. Examples include understanding data quality in pre-training or how to weight data [00:12:55].
Role in Transparency, Benchmarking, and Auditing
Academia holds a unique position to conduct impartial assessments of AI systems due to its lack of commercial interests [00:13:30]. This includes:
- Benchmarking: Developing and maintaining benchmarks to assess AI capabilities [00:13:27]. An example is Sidebench, a Capture the Flag cyber security exercise, which includes challenges human competitors take over 24 hours to solve [00:03:43]. These benchmarks can reveal subtleties in model performance and system compatibility [00:04:41]. Liang also notes the evolution of evaluation, including using language models to benchmark other language models, and developing rubrics for more objective assessments [00:31:24]. The Helm (Holistic Evaluation of Language Models) framework has evolved to cover various aspects and verticals, including safety, language, medical, and finance evaluations [00:34:42].
- Transparency and Auditing: Projects that assess the transparency of different AI providers [00:13:52]. Liang advocates for regulation that emphasizes transparency and disclosure, akin to nutrition labels for food, to help policymakers and third-party auditors understand the risks and benefits of AI [00:20:11]. He believes that regulating “downstream” (end-products) is often more effective, with “upstream” (foundation model developers) providing transparency and obligations for downstream decision-makers [00:20:59].
- Interpretability: Research into understanding why AI models make certain decisions, which is crucial for regulated industries like finance and healthcare [00:35:56]. Liang notes that access to model weights and training data, which is often withheld by commercial labs, is essential for interpretability research [00:39:47].
Challenges and Future Directions
Despite the unique advantages, academia faces challenges, particularly the lack of direct access to proprietary model weights and training data, which hinders research into interpretability [00:36:28]. This makes it difficult to debug or deeply understand model behavior.
Looking ahead, Liang sees significant opportunities for academia in:
- Scientific Discovery: Utilizing AI for research advancements and fundamental scientific discovery, such as solving open math problems or creating new research that extends human knowledge [00:48:22].
- Productivity Tools: Developing AI tools that improve researcher productivity [00:59:26].
- Exploring Underexplored Applications: Moving beyond commercial needs like RAG solutions and summarization to address more “fundamental science” applications [00:59:18].
- New Architectures: While current models like Transformers are dominant, new architectures may emerge from tackling different problem types, such as video generation or more complex agentic settings [00:42:07]. For example, the Mamba state space model architecture was inspired by mathematical breakthroughs [00:41:03].
- Robotics: While a “ChatGPT moment” for robotics is still a few years away, the progress in language and vision models can provide infrastructure and inspiration for AI models in advancing robotics and autonomous driving, especially by factoring out language and vision problems from purely robotic ones [00:52:02].
- Creative Augmentation: Developing AI as a “co-pilot” for creative endeavors, such as music composition, allowing artists to realize their visions without needing extensive traditional skills [00:54:35].
- Education: Leveraging AI as extremely good teachers and coaches, capable of breaking down complicated concepts into simple terms for diverse audiences [00:56:20].
Liang suggests that while AI capabilities are still rapidly advancing, qualitative changes are occurring, such as O1’s approach to “test time compute,” which represents a different way of using these systems [00:49:50]. He remains optimistic that AI agents will contribute novel insights into ML work in the coming years, similar to how AI has expanded the capabilities of software developers [00:58:33].