From: redpointai
Artificial intelligence is rapidly transforming various fields, including scientific research, software development, and education. Experts from Google’s Gemini LLM efforts, Noam Shazeer and Jack Rae, discuss the current capabilities and future potential of AI in these domains, highlighting surprising developments and ongoing challenges.
AI in Research
Initially, the concerted effort to integrate test-time compute into Gemini models focused primarily on reasoning tasks like math and code [02:44:39]. A surprising finding was that “thinking” – the process of generating thoughts before a response – generalized well beyond these specific tasks to improve creative outputs, such as composing essays with varied ideas and revisions [03:07:08].
Evaluation and Novelty
The development of effective evaluation metrics (evals) for AI models is crucial, as traditional benchmarks quickly become saturated and less relevant due to models memorizing problems [05:07:07]. New evals are constantly being developed by various AI labs to ensure models are truly reasoning about difficult tasks rather than merely adding parameters for memorization [04:09:42].
A significant debate in the field concerns whether current AI models can generate truly novel ideas or merely interpolate known concepts [02:58:18]. While some argue that much scientific discovery involves associating disjoint pieces of information, accelerating “interpolation” can still significantly advance science [02:55:00].
Math as a Research Frontier
Math is considered a prime example of a field where AI could make novel contributions, as it doesn’t necessarily require external data; humans invented math largely through thought [02:57:00]. The goal is for AI to progress from solving benchmarks to generating useful math and addressing important, unsolved problems [02:56:55]. The ultimate milestone for AI in mathematics would be a model that can pose novel, interesting mathematical questions and then solve them, potentially leading to a “complete map” of useful mathematics [03:40:40]. The hardest part is often the question posing, not the solving [03:43:55].
Culture of AI Research
The culture of AI research is described as being akin to “alchemy” in the 15th century: highly experimental, where the proof is in trying things out, and hypotheses often arise after observations [03:27:07]. The rapid pace of progress and the widespread availability of compute mean that impactful ideas can spread and be acted upon globally within months [04:53:30]. The sheer number of smart, creative people and the amount of compute available today means breakthroughs can occur even with less computational power than one might expect [04:16:17].
AI in Coding
AI is already integrated into Google’s structured monorepo environment, where it assists with tasks such as bug fixes and code reviews, enhancing developer productivity [08:10:10]. The concept of “agentic coding,” where models can tackle more open-ended and difficult tasks, is a key area of excitement [08:57:07]. The structured nature of Google’s code base makes it easier for AI to orchestrate changes and iterate on libraries efficiently [09:12:00].
One major milestone for AI is when “Gemini X writes Gemini X+1,” creating reinforcement loops where the AI itself becomes a tool to build better AI [06:46:11]. This self-acceleration is particularly promising for coding, as humans are not necessarily “that good at it,” and it offers a path to building automated software engineers and researchers [06:01:46].
The ability of open-source models to keep pace with and remain competitive against frontier, closed-source models has been very impressive [04:43:00]. This rapid catch-up, as seen with models like DeepSeek V3 and Gemma 3, suggests that the gap between open and closed source models may shrink even further [04:47:43].
AI in Education
AI in education is poised to be “incredible,” offering a type of personalized learning experience that has “never existed for humanity” [00:52:52]. For example, a four-year-old child, under supervision, uses Gemini to take pictures of plants and lizards, receiving accurate and personalized information [00:52:07]. This interaction allows children to absorb vast amounts of information and discuss complex topics using technical terms, suggesting that the next generation could become “smarter people” [00:53:36].
This indicates a significant shift in traditional learning methods, moving towards AI as a highly effective, curious, and adaptable personal encyclopedia [00:52:42]. The long-term impact of AI on teachers and students could be profound, altering how information is acquired and processed, and potentially redefining the purpose of humanity as AI takes on more physical and material needs [00:53:02].