From: redpointai
The “Unsupervised Learning” AI podcast featured Jacob Efron and guest host Rob Taves from Radical Ventures, joined by Peter Welinder, VP of Product and Partnerships at OpenAI [00:00:00]. The conversation explored various aspects of AI, including where value will accrue in the AI ecosystem, OpenAI’s product strategy, enterprise AI adoption challenges, and the future of AI.
Value Accrual in the AI Ecosystem
Peter Welinder believes that most value will accrue at the application layer across many different applications [00:02:21]. OpenAI’s strategy is to enable as many builders as possible to create products on top of their technology [00:02:44]. They deliberately keep prices low at the model infrastructure layer to foster development [00:02:55], noting dramatic price cuts over the years, such as a 70% reduction and a 10x cheaper GPT-3.5 Turbo model [00:03:07]. This approach broadens access and applicability of their models to more applications [00:03:32].
OpenAI acknowledges that the right tools for this technology have yet to be invented and prefers the developer ecosystem to figure out and build these tools [00:03:54]. With millions of people building on top of their technology, there is significantly more innovation in the application and tooling space than OpenAI’s internal team of around 400 people could achieve [00:04:40]. Value capture is expected to occur at the application layer through competitive modes like network effects and branding [00:05:12].
OpenAI’s Product Strategy
OpenAI aims to remain at the platform level, providing general models and tools rather than specialized end-user applications [00:06:18]. While ChatGPT serves consumers as a general application for answering generic questions, it is not designed to be the best AI teacher, doctor, or lawyer [00:06:31]. Companies are expected to build these specialized applications by integrating with other products, proprietary content, and domain expertise [00:06:52].
OpenAI prioritizes developing core language models due to their flexibility [00:10:01]. Their strategy involved a significant bet on language models and their generality, focusing most of their compute and personnel on training these models [00:08:46]. This singular focus allows them to move quickly with features like browsing and plugins [00:09:53]. The product evolution, including plugins, code interpreter, and external API connections, aims to unify different functionalities and make it easier for research teams to deploy innovations to users [00:11:10].
OpenAI sees plugins as mini-agents that perform tasks by connecting to APIs [00:11:46]. The company anticipates models becoming smarter and more robust, leading to more agent-like behaviors where the AI only checks in when uncertain about a task [00:12:14].
Enterprise AI Adoption Challenges
The biggest challenge preventing full-steam enterprise adoption of AI models is the problem of hallucinations, where models fabricate facts [00:26:17]. This is an open research problem at the cutting edge of the field [00:26:36].
Companies are addressing this by grounding models in external data, such as internal company documentation [00:26:55]. This involves embedding questions and documents, using a vector search database to find relevant information, and then having the language model summarize and answer based on those documents [00:27:11]. Models can also be instructed to state “I don’t know” if they cannot find an answer, which improves reliability [00:27:34].
OpenAI prioritizes core model improvements—such as lower prices, higher reliability, and lower latency—because these fundamentals are crucial regardless of the tooling built on top [00:29:19]. They listen to developers to identify obstacles and build tooling that is universally needed to help them get off the ground quickly [00:28:22].
Open Source vs. Proprietary Models in Enterprise
Peter Welinder holds an “unpopular opinion” that proprietary models will consistently be superior to open-source models for the foreseeable future [00:15:19]. He expects there will always be a category of AI systems that are significantly better than their open-source counterparts [00:16:15]. This is analogous to how desktop Linux, despite existing for 30 years, has not caught up to proprietary operating systems like macOS or Windows, largely due to insufficient investment in detailed refinement [00:16:30].
The substantial capital and engineering effort required for training and running models at scale make it difficult to replicate at an open-source level [00:17:02]. Companies investing heavily in these models have no incentive to open-source their most advanced creations, partly due to investment protection and safety concerns [00:17:24].
However, Welinder is “very excited” about open-source development [00:17:59]. Open-source models are valuable for pushing research forward, trying new training approaches, and exploring different applications [00:18:10]. They also make sense for specific product areas, such as smaller models deployed on devices, at the edge, or for on-premise deployments where latency and control are paramount [00:18:28].
For applications requiring the best performance and reliability, proprietary models are the most rational choice, as they will remain a couple of years ahead of open-source options for the foreseeable future [00:19:05].
While some tasks (e.g., summarizing) may not require the smartest model, products tend to evolve towards generality over time, necessitating a generally smart model [00:20:14]. Using a less capable model for a commercial product places a company at a competitive disadvantage [00:22:28]. OpenAI believes most value will accrue in the smartest models because they can tackle the most economically valuable problems [00:22:37].
OpenAI does invest in open-sourcing auxiliary models that enable broader use cases for smarter models, such as their Whisper model for accurate audio transcription. This allows for more applications that feed into large language models, even if OpenAI doesn’t expect to monetize speech recognition directly [00:24:31].
Future Applications of AI in Workplace Automation
AI is expected to enable “co-pilots” for various professions, including lawyers and medical providers [00:22:58]. Over time, AI could extend to areas like science, assisting in drug discovery or climate change solutions, leading to significant economic output and a higher standard of living [00:23:10].
AI Risks and Safety
Peter Welinder believes that risks such as misinformation, deepfakes, and bias are largely surmountable [00:30:50]. Misinformation becomes a problem at scale, but existing platforms have infrastructure to protect against it [00:31:08]. Regarding bias, OpenAI’s goal is to provide tools for product developers and users to instruct the model on desired biases within acceptable bounds [00:31:36].
The risk receiving too little attention, according to Welinder, is the progression from AGI to superintelligence—models that are “really, really smart” and capable beyond human intellect [00:32:10]. He notes a surprising lack of research on how to ensure a beneficial outcome when this occurs [00:33:33]. OpenAI has teams dedicated to this, but there isn’t a widespread “superintelligence safety department” at universities [00:32:41]. This concern is existential for humanity [00:32:54].
Welinder speculates that AGI, defined as autonomous systems capable of economically valuable work at human levels, could be achieved before 2030 [00:34:41]. He notes a significant shift in the field over the past 15-20 years, making AGI feel like it’s “on the path of kind of just happening” [00:35:53]. Early signs of superintelligence might appear around 2030 [00:36:40].
OpenAI’s strategy involves releasing models with low stakes to learn about risks like misinformation and bias, which prepares them for higher stakes scenarios with superintelligence [00:38:42]. They prioritize establishing the right organizational processes, frameworks, and safety safeguards, rather than trying to figure it out after a superintelligent model is trained [00:39:02]. OpenAI notably held back on releasing GPT-4 for almost half a year to gain clarity on potential downsides [00:39:40].
Areas for research in superintelligence safety include:
- Interpretability: Understanding what is happening inside black-box models [00:41:18].
- Alignment Definition: Clearly specifying goals and guardrails for AI behavior [00:42:16]. This requires collaboration between technical experts, social scientists, and philosophers [00:42:45].
- Technical Approaches: Such as shaping reward functions for reinforcement learning or having one model oversee the actions of another [00:42:55].
Internal Use of AI at OpenAI
Internally at OpenAI, employees use ChatGPT for a broad range of tasks, including:
- Coding: Debugging issues and analyzing stack traces [00:44:30].
- Writing Assistance: Overcoming writer’s block, improving writing quality, and drafting emails based on desired points [00:44:44].
Future Outlook
Peter Welinder continues to believe that it’s crucial to seriously consider the implications of superintelligence [00:45:36]. He feels many people still underestimate how quickly AI capabilities are advancing beyond previous expectations [00:45:50].
OpenAI’s primary focus remains on making models better in terms of capability and safety [00:47:14]. The main concern for OpenAI’s continued leadership is losing touch with its users and developers [00:47:24]. There is a tension where model improvements can sometimes replace developer-built solutions [00:47:40]. Scaling the “great customer experience” from when OpenAI had few customers to millions is a key challenge to ensure people continue to embrace and build on their technology [00:48:34].
For those interested in learning more about OpenAI’s technology and philosophy, Welinder encourages checking out their blog [00:49:32].