From: acquiredfm
The discussion around artificial intelligence often centers on the dichotomy between open source and closed ecosystems, each presenting distinct advantages and challenges for AI builders and the broader technological landscape [00:00:39]. Hugging Face, a leading platform for AI builders, stands at the center of this debate, facilitating both open and private development [00:01:06].
Understanding the Landscape
In the past, technology was primarily built by writing millions of lines of code [00:01:21]. Today, the method has shifted to training models using datasets and building AI applications [00:01:32]. This new paradigm is often referred to as “Software 2.0” [00:20:50]. Most AI builders currently utilize platforms like Hugging Face to find models, datasets, and develop applications [00:01:39].
The debate between open and closed source in AI is somewhat misleading because open source serves as the foundation for virtually all AI, including solutions from closed-source companies like OpenAI and Anthropic, which extensively use open research and open source components [00:26:55]. The distinction lies in whether proprietary layers are built on top of this open-source foundation [00:27:24].
Historical Context
Historically, the AI field was more open. From 2017 to 2019, much of the research, including foundational developments like Transformers and BERT, was publicly shared by the research community, including Google and OpenAI [00:27:36]. This openness and collaboration significantly accelerated progress [00:28:06]. For example, OpenAI’s GPT-2 and GPT-3 were built on the Transformer architecture, leading to the current state of Generative AI and its rapid evolution [00:28:16].
However, in the last two to three years, the field has become less open, largely due to commercial considerations and “misleading arguments” concerning the safety of openness versus closeness [00:28:27].
The Case for Openness
Proponents of open source AI emphasize several key benefits:
- Foundation for all AI [00:26:57]: Even closed-source companies rely on open-source frameworks and research.
- Accelerated Progress: Openness fosters collaboration, leading to faster advancements in the field [00:28:06].
- Empowerment and Innovation: Open access to models and datasets enables thousands of new AI companies to be built, redistributing power away from a few large tech companies [00:31:09]. This allows a new generation of founders and teams to play a significant role [00:31:40].
- Safer Future: A decentralized approach to AGI (Artificial General Intelligence) is considered safer than a future where only one company controls it [00:19:23]. Providing access to technology for policy makers, non-profits, and civil society contributes to a more secure future [00:19:41].
- Societal Alignment and Inclusivity: Greater participation in the building process can lead to better, more inclusive products that address a wider range of social issues [01:07:35].
Arguments Against Openness (and Rebuttals)
A primary argument against openness revolves around the “existential risk” of AI, suggesting that sharing research is dangerous as “bad actors” could misuse it [00:29:01]. However, this argument has been used in previous technology cycles (e.g., books, nuclear), and the rapid public awareness of AI might have amplified these fears [00:29:23]. The term “artificial intelligence” itself can evoke sci-fi associations and create unnecessary fears [00:30:14].
Another argument for closed systems is the idea of data moats and tight integration between models and applications, allowing proprietary systems to continuously fine-tune and improve their models with real-time user interaction data [01:01:06]. However, the exact mechanisms for creating “moats” and “economies of scale” in AI are still unclear [01:01:45]. Many current winners, like OpenAI, did not initially have unique data advantages [01:03:21].
Hugging Face’s Model
Hugging Face has become the leading AI model hosting platforms platform for AI builders, with over 5 million users sharing more than 3 million models, datasets, and applications [00:01:06]. Their approach is famously open and community-driven AI development [00:16:42].
Key aspects of Hugging Face’s model:
- Facilitating Collaboration: The platform provides features for collaboration, versioning, bug reporting, and reviews, enabling large teams to build AI together [00:05:45].
- Supporting Both Public and Private Use: While championing openness, Hugging Face allows half of its models, datasets, and apps to remain private for internal company use [00:18:08]. This caters to companies that prefer varying degrees of openness [00:18:17].
- Sustainable AI monetization and business models: Hugging Face is profitable and has raised over $500 million, spending less than half of that [00:38:42]. They avoid the “race to the bottom on compute” by providing value-added services, integrating compute seamlessly with platform features, making the experience significantly easier for companies [00:41:51]. This “locked-in compute” model, along with Enterprise Hub offerings, allows for high margins [00:43:05].
- Community-Driven Development: The platform’s evolution has been driven by user feedback, starting from porting Google’s BERT model to PyTorch and expanding to hosting models and datasets [00:14:43]. This community contribution is central to their success [00:16:54].
The Future of AI
It is predicted that all tech companies will eventually become AI companies, building and optimizing their own models for specific use cases and constraints [00:51:04]. This contrasts with the current model of relying heavily on APIs from large foundational model companies [00:50:02]. The argument for this future rests on the idea that specialized, smaller, more efficient, and cheaper models, purpose-built for specific domains or hardware, will become prevalent [00:52:10].
The number of AI builders is expected to grow significantly, potentially exceeding the current 50 million software engineers to reach 50-100 million or even more [01:05:51]. This is because the barrier to entry for contributing to AI is lower than traditional software development; one can be an AI builder by contributing expertise or data, not just by writing code [01:06:34].
The impact of AI on gaming, financial markets, and financial services is already being seen, and the technology is unlocking new capabilities that were previously impossible [00:02:36]. As investors like Nvidia continue to back companies in this space, it highlights the perceived opportunities [00:03:11].
AI presents an opportunity to “shake things up,” challenging monopolies and fostering a new era of innovation [01:05:01]. It’s considered the “opportunity of the century” to break established positions and build something new [01:05:01].