From: allin
Groq, a company in which Chamath Palihapitiya invested early, has recently gained significant attention for its Language Processing Unit (LPU) technology [06:26:00]. Chamath was the seed investor in Groq, which began its journey in 2016, extracting the core concept and team from Google [27:39:00].
The “Overnight Success” Story
After eight years of development, Groq experienced a “super viral moment” that led to a massive influx of interest [07:34:00], [27:52:00]. Just two months prior, the company had virtually no customers, but within days, it attracted 3,000 unique customers, ranging from Fortune 500 companies to individual developers [28:31:00], [28:41:00], [28:48:00]. This rapid adoption highlights the “deep tech” nature of Groq’s business, which involves a long development cycle with many difficult, interconnected components that must “click together” [32:01:00], [32:20:00].
LPU Technology: Training vs. Inference
At a high level, AI problems are divided into two distinct categories: training and inference [29:09:00].
- Training is about brute force and power, requiring vast numbers of machines, high-quality networking, and enormous energy to run for months to learn from data [29:45:00].
- Inference is about speed and cost [30:11:00]. This is the part that consumers experience when asking chatbots like ChatGPT or Gemini a question and receiving a useful answer [29:30:00].
Groq’s LPU chips are specifically designed for the inference challenge, proving to be “extremely fast and extremely cheap” [30:28:00]. They are “meaningfully faster and cheaper than any Nvidia solution” for this purpose [30:46:00].
LPU vs. CPU and GPU: A Historical Evolution
The evolution of processing units illustrates the unique contribution of the LPU:
- CPU (Central Processing Unit): The “Workhorse of all computing,” designed for serial computation. A CPU is efficient at processing one instruction, acting on it, and producing one answer [46:45:00], [47:09:00].
- GPU (Graphics Processing Unit): Jensen Huang, the founder of Nvidia, realized that CPUs “failed quite brilliantly” at certain tasks, such as graphics and video games [47:00:00], [47:29:00]. GPUs excel at parallel computation, processing many things simultaneously [47:47:00]. Around ten years ago, it became clear that the mathematical processes required for AI models were similar to those used in processing game imagery, making GPUs highly effective for AI [48:01:00].
- LPU (Language Processing Unit): Groq’s insight was that the fundamental design of GPUs had not substantially changed since 1999 [48:41:00]. They decided to discard expensive memory and other components, instead creating “small little brains” (chips) and connecting many of them together with clever software for scheduling and optimization [48:58:00]. This design, making the chip “much much smaller and cheaper,” turns out to be “hyper optimized” for large language models [49:09:00], [49:17:00].
Potential and Market Impact
Groq is currently valued as a “meager unicorn” with a valuation of around 2 trillion market capitalization [31:00:00], [31:03:00]. However, there is “a lot of market cap for Groq to gain” by scaling its production [31:07:00].
The company’s technology has the potential to “enable that monetization leap forward” for AI applications [37:28:00]. Current AI applications are often seen as “toy apps” for proofs of concept or demos, not yet production-ready [15:11:00], [36:46:00]. This is because they are too slow, require too much infrastructure, and are too costly [37:22:00]. Groq aims to address these limitations by offering a significantly faster and cheaper inference solution [37:24:00].
The success of deep tech companies like Groq, SpaceX, and Tesla demonstrates that while they take a long time and significant investment, they can build “extraordinary moats” and achieve “hundreds of billions and sometimes trillions of dollars of market value” when they work out [33:05:00], [33:39:00]. This is in contrast to software businesses that can achieve product-market fit and profitability more quickly [32:09:00]. Groq represents a “good risk” in the investment landscape because it does not aim to “debate the laws of physics” but rather solve a “well-defined bounded problem” through technical innovation in chip design and compilers [38:27:00], [39:35:00].