From: aidotengineer
D Kila, CEO at Contextual AI, shares insights on Enterprise AI from his background in AI research and as an AI company CEO [00:00:16]. While the estimated added value of AI to the global economy is projected at $4.4 trillion by McKinsey, suggesting a huge opportunity [00:00:54], there is significant frustration within enterprises regarding the lack of ROI [00:01:08]. A Forbes study indicates that only one in four businesses actually derive value from AI [00:01:26].
The Context Paradox
This disparity between the perceived opportunity and actual value is described as a “paradox,” similar to Moravec’s Paradox in robotics [00:01:36]. Moravec’s Paradox highlights that tasks considered difficult for humans (like beating chess grandmasters) are often easier for computers, while seemingly simple human tasks (like vacuuming a house or self-driving a car) are much harder [00:01:47].
A similar “context paradox” exists in Enterprise AI [00:02:11]:
- Language Models Excel at Complex Tasks [00:02:18]: They can generate code or solve mathematical problems better than most humans [00:02:27].
- Struggling with Context [00:02:41]: Unlike humans who effortlessly apply expertise and intuition to contextualize information, language models struggle with this [00:02:42].
- From Convenience to Differentiated Value [00:03:34]: Current general-purpose AI assistants offer convenience and efficiency, but enterprises aim for “business transformation” and “differentiated value” [00:03:46]. Achieving this requires superior handling of an enterprise’s internal context [00:04:00].
Lessons for Unlocking ROI with AI
Based on experiences at Contextual AI, several key lessons have been learned to bridge the gap between AI’s potential and its actual value in enterprises [00:04:28]:
1. Focus on Systems, Not Just Models
Language models are powerful, but they often represent only 20% of a larger Enterprise AI system [00:04:40]. For example, a Retrieval-Augmented Generation (RAG) system, pioneered by D Kila’s team at Facebook AI Research, is the standard method for getting Generative AI to work with specific enterprise data [00:04:52]. An average language model within an excellent RAG pipeline will outperform an amazing language model with a poor RAG pipeline [00:05:23]. The system, not just the model, solves the problem [00:05:35].
2. Specialization Over AGI
An enterprise’s expertise is its fuel [00:05:47]. While Artificial General Intelligence (AGI) has many use cases, solving difficult, domain-specific problems within an enterprise is best achieved through specialization, allowing better capture of internal knowledge [00:06:11]. This approach is often counterintuitive to the broader focus on AGI [00:06:33].
3. Leverage Enterprise Data as a Moat
A company’s long-term identity is its data, as people are transient [00:06:56]. Instead of spending extensive time cleaning data, the focus should be on enabling AI to work effectively with noisy data at scale [00:07:18]. Successfully doing so creates a “moat” or differentiated value, as this unique data defines the company [00:07:32].
4. Production is Harder Than Pilots
Building a simple AI pilot, such as a RAG system with a few documents, is relatively easy and often yields positive initial feedback [00:07:56]. However, scaling to production for tens of thousands or millions of documents, thousands of users, or numerous use cases is significantly more challenging [00:08:20]. This is compounded by Enterprise AI security and compliance requirements [00:08:50]. The correct approach is to design for production from day one, not just the pilot [00:08:57].
5. Speed Over Perfection
For production rollouts of RAG agents, speed is paramount [00:09:10]. Enterprises should deploy solutions to real users relatively early, even if “barely functional,” to gather feedback and iterate quickly [00:09:22]. Delaying deployment to achieve “perfection” makes bridging the gap from pilot to production much harder [00:09:42]. Iteration is key to successful AI production deployments [00:09:51].
6. Optimize Engineer Time
Engineers should focus on delivering business value and creating differentiated solutions, not on mundane tasks like optimizing chunking strategies or basic prompting for RAG systems [00:10:10]. State-of-the-art platforms can abstract away such complexities, allowing engineers to concentrate on strategic impact [00:10:47].
7. Make AI Easy to Consume
A significant challenge is ensuring that GenAI solutions actually get used [00:11:04]. Solutions must be easy for users to consume and integrate seamlessly into existing workflows [00:11:42]. Integrating AI into established enterprise workflows directly leads to greater production usage [00:11:58].
8. Generate “Wow” Moments
For successful AI adoption and stickiness, users need to experience “wow” moments quickly [00:12:15]. Designing onboarding experiences around these moments helps users rapidly understand the value of the AI [00:12:35]. An example includes a Qualcomm engineer discovering a 7-year-old hidden document through the system, answering long-standing questions and transforming their work [00:12:45].
9. Beyond Accuracy: Focus on Observability for Inaccuracy
While accuracy is a minimum requirement, or “table stakes,” enterprises are increasingly concerned with the missing percentage points of accuracy and how to manage potential errors [00:13:24]. The focus shifts to observability, with proper audit trails, especially in regulated industries [00:13:54]. Attribution within RAG systems, showing why an answer was generated and its source, is crucial for addressing inaccuracies [00:14:10]. Post-processing to verify claims and ensure proper attributions is also vital [00:14:23].
10. Be Ambitious
Many AI projects fail not because they aim too high, but because they aim too low, delivering “gimmicks” rather than substantial ROI [00:14:54]. To achieve significant ROI and business transformation, enterprises should pursue ambitious initiatives that fundamentally change the business, rather than simple tasks like answering basic HR questions [00:15:09]. AI represents a profound societal shift, and leaders have the opportunity to drive that change ambitiously [00:15:36].
Conclusion
The “context paradox” in Enterprise AI is a persistent challenge [00:16:04]. However, by focusing on building robust systems, specializing for unique enterprise expertise, and being ambitious in scope, organizations can transform these challenges into opportunities for successful AI adoption and impactful organizational change [00:16:08].