From: redpointai
Bob McGrew, former Chief Research Officer at OpenAI for six and a half years, recently discussed the future of AI, including the integration of AI into robotics, its challenges, and opportunities [00:00:00].
Current State and Future of AI Model Capabilities
There’s a significant divergence between the outside perception and the inside view of AI progress. From the outside, after the rapid release of ChatGPT and GPT-4, it might appear that progress has slowed [01:07:07]. However, scaling pre-training progress requires massive increases in compute, often 100x for a new generation (e.g., GPT-2 to GPT-3, GPT-3 to GPT-4) [01:41:00]. This necessitates building new data centers, a slow, multi-year process [02:16:00].
Test-Time Compute and O1
While pre-training continues, significant progress is being made through techniques like reinforcement learning for test-time compute [03:00:00]. O1, for example, represents a 100x compute increase over GPT-4 by leveraging reinforcement learning to create longer, more coherent “Chain of Thought” reasoning [03:11:00]. This approach doesn’t require new data centers and has significant room for algorithmic improvements [05:02:00]. In theory, these techniques could extend thinking time from seconds to hours or even days [05:31:00].
Multimodal Models: Video and Beyond
Multimodal AI, which integrates vision, audio, and other data types, is a particularly exciting area [01:14:00]. While LLMs (language models) were invented around 2018, applying Transformer techniques to other modalities like images (DALL-E) and audio (Whisper) eventually led to their integration into main models [01:24:00].
Video, however, has been the most resistant modality [01:54:00]. Sora, for instance, marks a significant demo in video generation [01:58:00]. Unlike image generation, video creation requires extended sequences of events and thoughtful user interfaces for story unfolding [01:49:00]. Video models are also very expensive to train and run [02:21:00]. Over the next few years, video model quality, especially for extended coherent generations, is expected to improve significantly, and costs will dramatically decrease, similar to the price drops seen with GPT-3 quality tokens [02:29:00]. It’s predicted that AI-generated full-length movies, driven by directors with creative vision, could be seen in about two years [02:08:00].
Prospects and Challenges in Robotics
Bob McGrew initially joined OpenAI to focus on robotics, believing it would be the domain where deep learning became widely adopted [02:40:00]. While his 2015 prediction of five years for widespread adoption was “very wrong,” he now believes it is correct for the present [02:51:00].
Foundation models are a “huge breakthrough” for robotics because they enable faster setup and better generalization [02:09:00]. The ability to use vision and translate it into action plans comes “for free” with these models [02:37:00]. The ecosystem has also developed, making it easier to interact with robots, even conversing with them to direct actions [02:47:00].
Key Challenges in Robotics
- Reliability: This is the most immediate problem [08:55:00]. If an AI agent makes a mistake while performing a task, especially one involving real-world actions (e.g., buying something, sending an email), the consequences can range from wasted time to embarrassment or financial loss [09:13:00]. Achieving higher reliability (e.g., from 90% to 99% or 99% to 99.9%) requires an order of magnitude increase in compute and is a slow, multi-year process [09:50:00].
- Learning Environment: A major differentiator for robotics is whether learning occurs in simulation or the real world [02:29:00]. While simulators offer benefits (similar to programming without production system pain), they struggle with “floppy” objects like cloth or cardboard, which are common in the real world [02:03:00]. For general applications, real-world demonstrations are currently the only effective approach [02:31:00].
- Unconstrained Environments: Mass consumer adoption of robotics, particularly in homes, faces challenges because homes are unconstrained environments, and robot arms can pose safety risks [02:15:00].
- Context for Enterprise Deployment: For AI agents to automate tasks in an enterprise setting, they need vast amounts of context (co-workers, projects, codebases, preferences) that are ambiently present in the organization’s data (Slack, documents, Figma) [02:11:00]. Connecting to this data requires building libraries of connectors or using “computer use” models [02:12:00].
Deployment and Adoption
The deployment of AI, especially in enterprises, often requires “handholding” from consulting firms to integrate with data and define guardrails [02:50:00]. While general-purpose “computer use” models (like Anthropic’s) are compelling, achieving high reliability (e.g., 99.999%) is difficult due to the many steps involved [02:13:00]. A mix of approaches is likely, with some using specific API integrations and others using computer use as a backup [02:44:00]. Salesforce-specific computer use agents are unlikely, as application providers would want their data to be public and part of every foundation model [02:51:00].
Widespread computer use follows a pattern: compelling demos appear, but it’s too painful to use initially [01:15:00]. A year later, it’s 10x better and used for limited cases; in two years, it’s surprisingly effective but not entirely reliable [01:24:00]. Adoption depends on the required level of reliability; tasks tolerating mistakes will be automated faster [01:54:00].
For robotics, widespread adoption is expected in retail and work environments within five years, particularly in warehouse settings where mobility is already solved and pick-and-place is being addressed [02:42:00].
Broader AI Integration and Societal Impact
Despite significant advancements, AI’s impact on overall productivity statistics remains less visible than expected, similar to the internet in the 1990s [03:12:00]. This is partly because jobs comprise many tasks, and even if AI automates some, a critical non-automatable task often remains [03:28:00].
The Role of AI in “Boring” Tasks
One underexplored area is applying AI to “boring” problems where human intelligence, despite being capable, would get bored [03:41:00]. For example, AI could rigorously comparison shop across all company expenditures [03:41:00]. AI is “infinitely patient” and doesn’t need to be infinitely smart for such tasks [03:53:00]. This concept explains why productivity gains from AI are first showing up in areas like consulting, where the job involves producing output [03:50:00]. AI’s ability to help “bottom half of performers” by enabling them to write code or execute tasks they conceptually understand but couldn’t implement is seen as a hopeful aspect [03:59:00].
Societal Implications and the Future of Agency
As intelligence becomes ubiquitous and free due to AI, the scarce factor of production will shift [04:16:00]. Bob McGrew speculates that this scarce factor will be “agency” – the ability to ask the right questions, identify projects to pursue, and define desired outcomes [04:49:00]. While AI can generate content based on vague prompts, the human element of defining specific choices and desired outcomes will remain critical [05:01:00]. This shift will feel continuous, as AI progress occurs on an exponential curve [05:11:00].
AGI and the Nature of Progress
Bob McGrew has a “deep critique of AGI” (Artificial General Intelligence), believing there won’t be a single “moment” of AGI because problems are fractal, leading to continuous automation of more things [04:11:00]. The future with AGI might even feel “banal,” with self-driving cars and AI armies still resembling mundane office life [04:25:00].
The fundamental challenges have been largely solved: pre-training, inferencing, and reasoning [04:59:00]. The remaining challenge is “scaling,” which is incredibly difficult and involves systems, hardware, optimization, and data problems [04:17:00]. In this sense, reaching AGI feels “predestined” by continued scaling efforts [04:47:00].
OpenAI’s Culture and Pivotal Decisions
OpenAI’s culture is marked by frequent “re-foundings” or pivots [04:53:00]. Key shifts included:
- Transitioning from a non-profit to a for-profit entity, driven by the need to raise money [04:28:00].
- The partnership with Microsoft [04:41:00].
- The decision to build their own products with the API [04:04:00].
- The move from enterprise to consumer with ChatGPT, which, though deliberate, happened somewhat by “accident” [04:09:00].
These pivots fundamentally changed the company’s purpose and the identity of its workforce every 18 months to two years [04:22:00]. The early mission of achieving AGI by writing papers evolved into building one model that everyone in the world could use, a path discovered through exploration and necessity [04:48:00].
A critical, though less famous, decision was to “double down on language modeling” as the central focus [04:52:00]. This involved a painful restructure, shutting down exploratory projects like robotics and games teams, despite successes like the Dota 2 AI [01:00:00]. This decision stemmed from the conviction that increasing scale could solve problems, and it led to refocusing on language models and generative modeling [01:00:00].
Personal Reflections on AI Research
Bob McGrew highlights the importance of “grit” in top AI researchers [03:39:00]. He cites the example of Adithya Ramesh, who worked for 18 months to two years on DALL-E to generate a “pink panda skating on ice,” a challenge to prove neural networks were creative rather than just memorizing [03:42:00]. Such foundational problems require researchers to commit years to make their vision a reality [03:52:00].
Building a research organization, according to McGrew, is like managing artists, especially “100x the artist of any engineer” [03:50:00]. It requires a “very high touch” approach to avoid “snuffing out the artistry,” as their dedication to their vision drives them through the pain of production [04:13:00]. Unlike academia’s focus on individual credit and competition, OpenAI fostered collaboration and aimed to build “one thing” rather than just publish papers [04:53:00].
Overhyped and Underhyped in AI
- Overhyped: New architectures, as many tend to “fall apart at scale” [01:08:00].
- Underhyped: O1, which, despite being highly discussed, is still not appropriately recognized for its significance [01:24:00].
Bob McGrew left OpenAI after eight years, feeling he had accomplished his initial goals, especially with the shipment of O1 preview, which completed the research program on pre-training, multimodal, and reasoning [01:41:00]. He plans to take time to explore new areas, much like he did between Palantir and OpenAI, learning and developing new theses [01:45:00]. He continues to connect with founders and researchers, exploring topics like robotics that were outside OpenAI’s focus [01:53:00].
AI progress will continue and remain exciting, changing in its manifestations but not slowing down [01:06:00].