From: allin
Sergey Brin, co-founder of Google, has expressed that as a computer scientist, he has “never seen anything as exciting as all of the AI progress that’s happened the last few years” [00:02:32]. Brin, who registered Google as a website on September 15, 1997, and founded Google with Larry Page in 1998, has recently returned to helping Google’s efforts in artificial intelligence [00:00:03], [00:01:43], [00:00:20]. He currently dedicates almost every day to this work [00:02:20].
Historical Context of AI
During Brin’s time in grad school in the 1990s, AI was considered a “footnote” in the curriculum [00:02:47]. Approaches like neural networks were largely “discarded” in the 1960s and 1970s [00:03:05]. There was a prevalent belief that these methods “don’t really work” [00:02:58].
Recent Breakthroughs and Evolution
A “miraculous” shift occurred when people working on neural networks started making progress with “a little bit more compute, a little more data, a few clever algorithms” [00:03:05], [00:03:16]. This led to “amazing” advancements in the last decade, with “new amazing capability” appearing “every month” [00:03:24], [00:03:39].
Impact Across Industries
AI’s influence extends far beyond traditional areas like search, touching “so many different elements of day-to-day life” [00:04:07], [00:04:12]. A notable example is its impact on programming, where writing code from scratch “feels really hard compared to just asking the AI to do it” [00:04:28], [00:04:30]. Brin recounts an instance where he had an AI model generate code for Sudoku puzzles, and then feed them to the AI itself for scoring [00:04:53].
Model Architectures and Trends
Historically, different AI techniques were used for distinct problems, such as a chess-playing AI being “very different than image generation” [00:06:24], [00:06:34]. Even recently, Google’s graph neural network outperformed physics forecasting models, utilizing a “totally different Arch” and training [00:06:42], [00:06:53].
In a recent International Math Olympiad, Google’s AI achieved a silver medal by employing three distinct models: a formal theorem proving model, a geometry-specific AI, and a general-purpose language model [00:07:04], [00:07:11].
Despite these specialized applications, Brin notes a clear trend towards a “more unified model,” envisioning “shared architectures and and ultimately even shared models” [00:07:45], [00:07:55].
Compute Demands and Algorithmic Improvements
The development of larger, unified AI models requires substantial computing power [00:08:06], [00:08:15]. However, Brin suggests that “algorithmic improvements” in recent years might be “outpacing the increased compute” being invested [00:08:36], [00:08:44]. Despite this, there is “huge amount of demand” for compute from cloud customers, to the extent that Google sometimes has to “turn down customers” due to lack of available compute [00:09:31], [00:09:41].
Challenges and Opportunities in AI
Brin acknowledges the challenges in AI development, particularly the need for robustness [00:11:36], [00:11:40]. He noted that earlier robotics ventures by Google were “a little too early” because they lacked the “modern AI technology” like general language models with vision and multimodal capabilities [00:12:05], [00:12:32].
He emphasizes the iterative process of bringing AI to market: from a “moment of wow” with a demo to the hard work of making it “90% of the time” correct, responsive, and available in production [00:14:26], [00:14:30], [00:14:36].
Google’s Approach to Deployment
Historically, Google was “too timid” to deploy language models, despite having invented the Transformer paper [00:16:19], [00:16:22]. This timidity stemmed from concerns about mistakes, embarrassing outputs, or “dumb” responses [00:16:26], [00:16:30].
Brin advocates for a more aggressive approach, asserting that the “magical” capability of AI, such as helping kids program complicated things with complex APIs, necessitates a willingness to “have some embarrassments” and “take some risks” [00:17:10], [00:17:17]. He believes that if AI is “something magical we’re giving the world,” it should be released, even if it “periodically get[s] stuff really wrong,” allowing people to “experiment and see what new ways they find to use it” [00:17:48], [00:18:02]. He concludes that this technology should not be “kept close to the chest and hidden until it’s like perfect” [00:18:14].
Competition and Value Creation
While there is competition among AI companies like OpenAI, Anthropic, and Mistral, Brin emphasizes that the “tremendous value to humanity” from AI goes beyond a simple race [00:19:34], [00:19:50]. He likens the impact of AI to the internet’s profound effect on information access and communication, stating that the “new AI is another big capability” accessible to “pretty much everybody in the world” [00:20:20], [00:20:29].