From: redpointai
The role of a Chief Technology Officer (CTO) is undergoing significant transformation due to the rapid advancements in AI [00:01:04]. Mike Schreer, former CTO at Facebook (for 9 years) and now founder of the venture capital firm Gigascale, shared his insights on how AI is reshaping this executive position [00:00:31].
Background: Mike Schreer’s Journey
Mike Schreer’s career spans significant technological shifts, including the internet, mobile, and the rise of AI. He started working in AI at Facebook in 2013, when the Facebook AI Research (FAIR) lab was initiated following a breakthrough in convolutional neural networks [00:13:37]. His current venture, Gigascale, invests in companies utilizing technology to combat climate change [00:00:35].
The Evolving Developer Landscape
Historically, advancements in computing power have allowed for shifts to higher-level programming languages, moving from assembly to C, and then to Python, Rust, or JavaScript [00:09:50]. This evolution prioritized programmer productivity even if it meant “throwing away compute cycles” [00:09:59].
Today, a similar shift is occurring with AI systems writing a bunch of our code [00:10:03]. While these AI systems may be less power-efficient per cycle, they continue the trend of increasing abstraction for developers [00:10:07].
“AI is to coding the way like JavaScript is to C is to Assembly Language. It’s like, okay, cool, like I’m expressing my thoughts in a higher and higher level of abstraction and like AI is the ultimate it’s like write me a piece of code that sorts the following array in the following way” [00:40:20].
Developer tooling is also moving “up the stack,” with less focus on core model architectures and more on the surrounding systems [00:19:17]. This includes managing data sets, pre-training, post-training, and handling large-scale clusters where nodes may be down [00:19:37]. This transition means development is no longer just “sitting at my desk doing work” but requires large, shared computing resources, akin to modern physics research [00:20:04].
The Future CTO’s Focus
Despite these changes, Mike Schreer believes the CTO role will be “more similar than people think it is” [00:39:40]. The core responsibility remains:
- Problem Identification: Determining “what problems are we trying to solve, what’s important to go after” [00:40:50].
- Team Organization: Organizing “this group of smart humans to go after that problem” [00:40:53].
This ability to prioritize and ensure the team is focused on the “highest leverage, most important thing” is the “universal skill” for a CTO, regardless of the tools available [00:40:57].
Strategic Hardware Decisions
A critical aspect for CTOs, particularly in hyperscale companies, is deciding whether to build proprietary hardware or outsource [00:20:55]. Understanding the supply chain and owning critical parts is crucial [00:21:00].
At Meta, the company transitioned from leasing data centers and buying off-the-shelf servers to designing nearly all its data center equipment [00:21:20]. While companies like Nvidia produce excellent general-purpose chips, the high cost of GPUs for AI compute leads CTOs to consider specialized hardware for better performance or cost advantages [00:22:29]. However, this is a “delicate balance” as specializing in hardware requires accurately predicting future algorithm trends, a challenge given the rapid pace of AI progress [00:23:09].
The Challenge of Foresight
The “pace of how fast everything is changing” in AI makes long-term strategic decisions difficult for CTOs [00:22:55].
“The bummer about chips is like the best way to make gains like it’s really hard to beat Nvidia with a general purpose chip… The only real joy is to basically specialize and say like all right I know this specific algorithm I’m going to implement this in Hardware… but you got to guess the algorithm right” [00:23:08].
Similarly, making commitments for physical infrastructure like data centers years in advance requires a calculated risk. Mike notes that “underpredicting” capacity has often been more regrettable than “overpredicting” for him, as lacking capacity can hinder product and technical development [00:25:28].
Impact on Organization Size
While the core skills remain, the size of organizations required to achieve certain outcomes may change. With advanced AI tools, companies can potentially “come up faster with smaller teams,” leading to a reduction in typical team sizes over time [00:41:50].
Key Takeaways for CTOs in the AI Era
- Embrace higher abstraction: Utilize AI systems writing code and advanced developer tools to enhance productivity [00:10:03].
- Focus on system design: The complexity shifts from individual code to managing large-scale, distributed AI training and inference systems [00:20:26].
- Strategic supply chain management: Carefully evaluate whether to build or buy key hardware components, balancing specialization with the risks of rapid technological change [00:21:00].
- Prioritize problem-solving: The fundamental role of the CTO remains identifying the most impactful problems and effectively organizing human teams to solve them [00:40:50].
- Adapt to rapid change: Be prepared to make strategic decisions with imperfect information in a fast-evolving landscape, often leaning towards over-provisioning compute capacity to avoid being capacity-constrained [00:22:55].
[!info]+ Role of Open Source Meta’s strategy of open-sourcing foundational AI technologies like PyTorch and Llama models was a deliberate choice, driven by the belief that foundational technology should be commoditized and broadly accessible [00:15:56]. This approach ensures access to the “very best technology” and fosters global collaboration, accelerating overall AI progress [00:18:02]. For a CTO, this means leveraging community contributions and collaborative development [00:18:25].