Tubegraph

Enterprise AI deployment models

7 min read

From: redpointai

Aiden, from Cohere, discusses the future of enterprise AI adoption and usage, outlining current deployment models, challenges, and Cohere’s strategic approach to integration and application development [00:00:32].

Current Enterprise AI Deployment Models

Today, several models exist for enterprise Generative AI (GenAI) deployments [00:00:46]:

  • Consulting-based: Large consulting firms, like Accenture, are brought in to build solutions [00:00:51].
  • Forward-deployed engineering/product: Similar to Palantir, where there’s a mix of forward-deployed engineering and product development [00:00:55].
  • Out-of-the-box product: Companies attempt to sell a ready-to-use product, expecting enterprises to make it work [00:01:00].

The Future of Enterprise AI Deployment

Aiden believes that a “middle ground” model will ultimately prevail in the long term for GenAI in enterprises [00:01:19]. Given the complexity of the new technology, some degree of support will be necessary for integration into the economy and various companies for diverse applications [00:01:24].

Challenges and Strategies in Enterprise AI Deployment

AI agents require extensive access to enterprise data—including emails, chats, calls, CRM, ERP, and HR software—to drive automation effectively [00:01:55]. This presents significant challenges in AI adoption and deployment:

  • Privacy: The high degree of data access needed by AI agents makes privacy a much bigger issue than in other enterprise software [00:02:24].
  • Customization and Integration: Each company utilizes a unique “tapestry” of software, necessitating custom setup to bring all necessary context together and integrate it into the model [00:02:43].
    • While completely self-serve setup with agents is desirable, a middle ground where parts of the problem are automated, but human oversight (e.g., for sensitive data like salary) remains necessary [00:03:43].

Enterprise Adoption and Use Cases for AI

Cohere observes specific GenAI use cases that have achieved product-market fit [00:04:48]:

  • Vertical Applications: In healthcare, applications like passive listening mics during doctor-patient interactions can pre-populate notes, reducing administrative time for doctors [00:04:57].
  • General Categories:
    • Customer Support: The technology and need are aligned, leading to rapid adoption across various verticals like telco, healthcare, and financial services [00:05:21].
    • Research Augmentation: Agents can perform months of research in an hour or two, significantly augmenting human capabilities [00:05:47]. For example, wealth managers can get robust research with citations to source documents, making knowledge work much more effective [00:06:04].

Looking ahead, Aiden anticipates the “mundane back office stuff” will increasingly come online, particularly in finance, legal, and sales [00:24:22]. This will be facilitated as infrastructure for automation (like Cohere’s North platform) gains access to necessary tools and context, and provides user experience for building automations [00:24:29].

Model Capabilities and the Future

The current generation of models, particularly reasoning models, has unlocked significant capabilities [00:07:07]. Reasoning allows models to spend different amounts of energy on problems of varying complexity, which is “so obvious and intuitive” [00:08:00].

Missing Capabilities

Despite advancements, certain properties of intelligence are still missing from current AI technology [00:09:07]:

  • Learning from Experience: Models currently do not learn from past interactions or feedback from users [00:08:46]. This capability would allow models to start with basic understanding and become experts over time, similar to human learning [00:08:50]. This could be actuated by continually feeding past interactions into a queryable database [00:45:47].

Diminishing Returns of Scaling

The “scale is all you need” hypothesis is breaking, as current efforts are experiencing diminishing returns from increased capital and compute [00:09:21]. Future advancements will require smarter and more creative approaches [00:09:33].

Specialized vs. General Models

Aiden’s view on specialized models has evolved [00:10:38]. While models are becoming capable of self-developing internal “experts,” building custom AI models for enterprises remains important [00:10:46].

The Role of Custom Models and Enterprise AI Integration

Custom models are crucial for closing the gap where fundamental business or domain context is missing from models trained on the web [00:11:15]. This includes data not commonly found online, such as manufacturing data, customer transactions, or detailed personal health records [00:11:29].

Cohere partners with organizations that possess such data to create custom models accessible only to them, becoming highly proficient in those specific domains [00:11:43]. However, general models are still extraordinary, and synthetic data significantly helps bridge gaps [00:11:56]. Ultimately, an organization might use a handful of models, but not every single team will have its own fine-tuned model [00:12:09].

Data Labeling and Synthetic Data

While human data labeling remains necessary, it is often too expensive to use large pools of human experts (e.g., 100,000 doctors) to teach models [00:13:44]. Instead, a smaller pool of trusted human data is used to generate a thousandfold synthetic lookalike data [00:14:26].

  • Evaluation: Humans are still the gold standard for evaluating model usefulness, especially when building models for people [00:13:03]. It’s the one area where humans cannot be taken out of the loop yet [00:13:15].
  • Synthetic Data Generation: Models’ ability to “chitchat” (general conversation) has unlocked freedom in synthetic data generation [00:14:14]. In verifiable domains like code and math, it’s easier to filter synthetic data for quality [00:14:49]. Currently, an overwhelming majority of data Cohere generates for new models is synthetic [00:15:11].

Challenges in deploying AI models effectively

For Cohere, the focus of test-time compute for reasoning is on solving problems within businesses using human tools, rather than complex academic challenges [00:15:51]. Reasoning has led to a “complete step change” in improvement; tasks that were impossible for models before reasoning now almost never fail [00:16:16]. The ability to reflect, understand failures, and find alternative paths has been a significant unlock [00:16:47].

Cohere’s Approach: Vertical Integration

Cohere is pushing into the application layer with its platform, North [00:18:17]. This move was motivated by observing customers repeatedly building the same applications, often taking a year and resulting in poor user experiences because internal AI teams are not product teams [00:18:23].

North aims to provide a “consumer-grade product experience” for enterprises, offering [00:19:01]:

  • Customization of UI, branding, data connections, and tools [00:19:14].
  • Ability to plug in other models (e.g., Llama fine-tunes) [00:19:30].
  • Acceleration of product roadmaps by 12-18 months [00:19:43].

Cohere’s vertical integration (building both models and applications) offers a competitive advantage [00:20:55]. This allows for more levers to pull to deliver the required customer experience, as models can be optimized for specific use cases (e.g., Command optimized for North’s use cases like ERP and CRM integration) [00:21:00]. This deep integration between technology and customer needs is crucial for product quality [00:21:31].

Team Expertise in Building AI Applications

To build the best AI products, deep model knowledge is crucial, even if teams are not training models themselves [00:22:34]. This understanding allows teams to approximate model-layer impact [00:22:49]. Cohere plays this role for its partners, like Oracle, by intervening at the model level to meet their needs [00:23:03].

Consolidation of Enterprise AI Platforms

Initially, enterprise AI adoption may be “scattershot,” with different teams acquiring various small applications [00:26:35]. However, this will lead to an “insane maintenance burden” due to disparate data source connections [00:27:03]. Aiden predicts a strong push towards consolidation, favoring a single platform (like North) plugged into everything to handle various automation objectives [00:27:09].

  • Differentiation Among Labs: AI labs are increasingly differentiating their strategies [00:31:19]:
  • Emergence of New Foundation Models: Beyond current general models (language, image, video, audio), new generations of foundation models are expected for specialized domains like biology, chemistry, and material science [00:32:13]. Cancer data, for example, is siloed and not on the web, but exists in sufficient quantity for dedicated AI efforts [00:32:26].

AI Risks

Aiden expresses concern about:

  • The capabilities bad actors, especially at the state level, might access [00:48:43]. He emphasizes ensuring liberal democracies gain an advantage [00:48:49].
  • The infrastructure needed to facilitate retraining and moving people to new careers if jobs are impacted [00:49:01].

He is not afraid of “X-risk” or “Terminator” scenarios, believing that there are more immediate and tangible concerns that policy makers and the public should focus on [00:49:27]. Aiden views the future as a “much better world” rather than a utopia, driven by AI augmenting human capabilities and increasing supply to meet infinite demand [00:48:05].

For more information, visit cohere.com [00:50:14].

Graph View

Backlinks