From: redpointai
The landscape of enterprise AI adoption and usage is rapidly evolving, presenting various models for deployment and unique challenges for organizations looking to integrate Generative AI (GenAI) solutions [00:00:38].
Current Deployment Models
Currently, there are several observed models for GenAI deployments in enterprises:
- Consulting-led [00:00:51]: Large consulting firms like Accenture come in to build custom solutions [00:00:52].
- Forward-deployed engineering/product [00:00:55]: Similar to the Palantir-esque model, involving engineers building products on-site [00:00:57].
- Out-of-the-box product [00:01:02]: Selling a ready-to-use product that enterprises are expected to implement themselves [00:01:02].
In the long term (5-10 years), a middle-ground approach is expected to prevail, acknowledging that enterprises will still require some degree of support to integrate this new, complicated technology effectively into the economy and various applications [00:01:19].
Challenges in Enterprise AI Deployment
AI agents are unique in that they require extensive access to enterprise data and systems to drive automation effectively [00:01:51]. This presents several challenges:
- Privacy [00:02:24]: Agents need to access emails, chats, calls, CRM, ERP, and HR software, demanding a degree of access rarely seen in other enterprise software, making privacy a significant concern [00:02:08].
- Custom Integration [00:02:41]: Each company uses a unique “tapestry” or “mosaic” of software, necessitating custom setup to bring all context together and integrate it into the model [00:02:43].
- Guardrails [00:04:16]: Due to the high stakes of mistakes, particularly with sensitive data like salary or customer information, significant guardrails are necessary [00:04:18]. While self-serve setup for agents would be helpful, a middle ground is expected where parts are automated but not the entire process [00:03:43].
Successful Enterprise Use Cases for GenAI
Companies like Coher observe what GenAI applications currently have product-market fit within enterprises [00:04:44]:
- Vertical Applications [00:04:55]: In healthcare, GenAI is being used to simplify note-taking and form-filling for doctors by passively listening to doctor-patient interactions and pre-populating information [00:04:57].
- General Categories (Cross-Vertical) [00:05:21]:
- Customer Support [00:05:24]: The technology is ready, and there’s a strong need across various verticals like telco, healthcare, and financial services [00:05:26].
- Research Augmentation [00:05:47]: Augmenting humans with agents that can perform a month’s worth of research in a few hours [00:05:51]. This capability can make knowledge work, such as for wealth managers, significantly more effective by allowing models to read vast amounts of data and provide robust, cited research [00:06:04]. This use case is considered ready for prime time and will be integrated into every enterprise globally [00:24:08].
Trends in AI Model Training and Deployment
The Role of Reasoning Models
Reasoning capabilities are crucial for AI models, allowing the technology to allocate different amounts of energy based on problem complexity, unlike non-reasoning models that treat all inputs similarly [00:07:31]. Reasoning has led to a “complete step change” in model improvement, enabling models to accomplish tasks that were previously impossible due to their ability to reflect, understand failures, and find alternative paths [00:16:16].
Evolution of Model Capabilities
Current models lack the ability to learn from experience, akin to humans starting as novices and becoming experts over time [00:08:46]. This capability, including learning from user feedback, is a clear missing property of intelligence that needs to be developed [00:08:58].
The “scale is all you need” hypothesis is breaking, as diminishing returns from capital and compute are evident [00:09:21]. Future progress will require smarter and more creative approaches beyond simply throwing more money at compute [00:09:33].
Custom vs. General Models
While models are becoming capable of self-developing internal experts [00:10:42], custom models remain important for integrating fundamental context about specific businesses or domains that is not available on the web [00:10:55]. This includes data from manufacturing, customer transactions, or detailed personal health records [00:11:30].
General models are extraordinary, and synthetic data can significantly bridge the gap in specialized domains [00:11:56]. It’s unlikely that organizations will operate with tens or hundreds of models; instead, a handful of customized models may suffice [00:12:09].
Data Labeling and Synthetic Data
Human data remains necessary but is often too expensive to gather at scale, especially from experts [00:13:42]. For example, finding 100,000 doctors to teach a model medicine is not viable [00:13:47]. The ability to teach models general conversation skills allows for synthetic data generation, which can then be applied to specific domains using a much smaller pool of human-provided, trustworthy data [00:14:14]. In verifiable domains like code and math, checking results makes filtering synthetic data easier [00:14:49]. For companies like Coher, an overwhelming majority of data generated for new models is synthetic [00:15:08].
Human involvement is indispensable for evaluation (Eval) [00:13:15].
Cohere’s Approach to Enterprise Partnerships and AI Application Deployment
Cohere is pushing into the application layer with its platform “North,” motivated by seeing customers repeatedly building the same applications [00:18:17]. These applications, often built by internal AI teams (not product teams), could take a year and lacked user-friendliness [00:18:29].
North aims to solve this by providing a consumer-grade product experience that is intuitive, low-latency, and equipped with enterprise features [00:19:01]:
- Customization [00:19:14]: UI customization, rebranding, custom data connections, tool integration, and even plugging in other fine-tuned models like Llama [00:19:17].
- Accelerated Deployment [00:19:40]: Sets companies forward 12-18 months in their product roadmap, enabling immediate distribution of technology to employees [00:19:43].
- Strategic Attributes [00:19:55]: Not locked into one hyperscaler ecosystem, deployable anywhere [00:20:00]. Releases weights non-commercially and supports VPC deployments [00:20:07].
This vertically integrated approach, where models and applications are built together, offers more levers to deliver the desired customer experience [00:20:55]. For example, Cohere’s Command generative model is optimized for use cases needed in North, such as interacting with ERP or CRM systems [00:21:04].
Developing and Utilizing AI Models in the Tech Industry
Team Expertise
Building the best AI products often requires deep model knowledge [00:22:34]. Even if teams aren’t training models themselves, they try to approximate that impact at the model layer [00:22:47]. Companies like Cohere aim to provide this deep model intervention for their partners, ensuring needs are met [00:23:01].
Future Use Cases
Beyond current applications, mundane back-office tasks are expected to come online as infrastructure for automation (like North) is put in place [00:24:22]. This includes tasks in finance, legal, and sales [00:25:04]. Sales, for example, involves extensive research (company strategy, internal conversations, individual briefings) that can be dramatically enhanced by AI [00:25:17].
Consolidation vs. Federation of AI Solutions
Initially, enterprises may experience a “scattershot phase” where different teams purchase disparate applications [00:26:35]. However, this will lead to an “insane maintenance burden” due to numerous data source connections across different apps [00:27:03]. This will drive a strong push towards consolidation, with enterprises seeking a single platform plugged into everything to manage all automation objectives [00:27:09].
Economic and Strategic Considerations in AI Model Deployment
Strategic Decisions
- Location [00:27:52]: Starting up in Toronto, leveraging the strong Canadian AI ecosystem (e.g., Jeff Hinton, Ilia Sutskever’s origins), has provided access to top talent and made Cohere a “darling” of the Canadian tech community [00:27:55].
- International Partnerships [00:29:40]: Cohere actively partners with companies like Fujitsu in Japan and LG in Korea, investing deeply in non-English languages to ensure their technology works well in those jurisdictions and supports local economies [00:29:42]. An open-source project, “Coher.ai,” collected data from thousands of native speakers across over a hundred languages to improve models for everyone [00:30:22].
Differentiation of Foundation Model Companies
In the future, there will likely be a handful of foundation model companies, each finding their niche [00:32:04]. For example, OpenAI focuses on the consumer front, Anthropic excels at code, and Cohere is focused on enterprise and back-office applications [00:31:32]. New generations of foundation models are also expected for specific domains like biology, chemistry, and material science [00:32:13].
Hardware and Compute Trends
Test-time compute requires significant resources, making inference 3 to 10 times more expensive [00:39:06]. However, compute is expected to become cheaper and more abundant per flop [00:39:23]. The emergence of multiple options for training compute and the ability to combine different chip types for supercomputers are positive trends for AI development [00:39:32].
Future Model Milestones
A significant future milestone for models is the ability to learn from experience and user interaction [00:44:46]. This could involve storing interaction history in a database that models can query for context [00:45:47]. Such models would grow with users, learning preferences and improving over time, fostering a stronger connection with the system [00:46:07].
AI Risks and Societal Impact
Concerns exist regarding:
- Bad Actors [00:48:43]: Potential access to powerful AI capabilities at the state level [00:48:47]. Ensuring liberal democracies gain an advantage is critical [00:48:52].
- Job Displacement [00:49:04]: The need for infrastructure to facilitate moving and retraining people into new, more fulfilling careers if particular jobs are impacted [00:49:01].
However, mass unemployment is not anticipated, as humanity has “infinite demand,” and the technology is expected to augment people, allowing them to do more and deliver more [00:47:48]. Concerns about existential risks like “Terminator” scenarios are considered less urgent than near- and mid-term issues that require focus from the public and policymakers [00:49:36].