LinkedIns GenAI platform development

From: aidotengineer

LinkedIn has embarked on a significant journey to build its own Generative AI (GenAI) platform, supporting a rapidly evolving suite of AI-powered products [00:00:19]. This platform aims to provide a unified interface for a complex GenAI ecosystem, enabling developers to build applications efficiently and responsibly [00:08:27], [00:09:15].

Evolution of LinkedIn’s GenAI Products & Platform

The development of LinkedIn’s GenAI platform has been an iterative process, growing piece by piece to meet the demands of increasingly sophisticated AI applications [00:28:28].

Early 2023: Collaborative Articles

LinkedIn launched its first formal GenAI feature, “Collaborative Articles,” in early 2023 [00:01:29]. This was a straightforward prompt-in, string-out application leveraging the GPT-4 model from OpenAI to create long-form content [00:01:45].

At this stage, the team built core components like a gateway for centralized model access and Python notebooks for prompt engineering [00:02:00]. However, there were two distinct tech stacks: Java for the online phase and Python for the backend [00:02:13]. This initial setup was not considered a full platform [00:02:24].

A key limitation of this simple approach was its inability to inject rich data into the product experience [00:02:30].

Mid 2023: Co-pilot/Coach

In mid-2023, LinkedIn began developing its second generation of GenAI products, internally called “co-pilot” or “coach” [00:02:42]. An example is a feature that analyzes a user’s profile and job description to provide personalized recommendations on job fit, using a Retrieval-Augmented Generation (RAG) process [00:02:51].

This phase saw the beginning of building core platform capabilities [00:03:13]:

• Python SDK: Built on top of the LangChain framework to orchestrate LLM calls and integrate with LinkedIn’s large-scale infrastructure [00:03:16]. This allowed developers to easily assemble applications [00:03:35].
• Unified Tech Stack: The decision was made to unify the tech stack due to the cost and errors involved in transferring Python prompts to Java [00:03:38]. Python was eventually chosen as the primary language [00:14:40].
• Prompt Management: Investment in “prompt source of truth” helped developers version their prompts and provide structure to meta-prompts [00:03:51].
• Conversational Memory: An infrastructure to track LLM interactions and retrieved content, injecting them into the final product to enable conversational bots [00:04:08].

Late 2023/Early 2024: Multi-Agent Systems

More recently, LinkedIn launched its first real multi-agent system, the “LinkedIn HR Assistant” [00:04:33]. This system automates tedious tasks for recruiters, such as posting jobs, evaluating candidates, and reaching out to them [00:04:42].

The platform evolved into an agent platform with enhanced capabilities [00:05:05]:

• Distributed Agent Orchestration Layer: The Python SDK was extended to support large-scale distributed agent execution, handling complex scenarios like retry logic and traffic shifting [00:05:08].
• Skill Registry: An investment was made in a skill registry, providing tools for developers to publish their APIs as “skills” that agents can discover and invoke easily [00:05:36]. This is critical for agents to perform actions [00:05:36].
• Experiential Memory: In addition to conversational memory, the platform introduced experiential memory—a storage for extracting, analyzing, and inferring tacit knowledge from agent-user interactions [00:06:14]. This memory is organized into working, long-term, and collective layers to make agents aware of their surroundings [00:06:35].
• Operability Investment: Recognizing the autonomous and unpredictable nature of agents, LinkedIn invested in operability [00:06:50]. An in-house solution built on OpenTelemetry tracks low-level telemetry data, allowing for replaying agent calls and analytics to guide future optimization [00:07:12].

Components of the GenAI Platform

LinkedIn’s GenAI platform is structured into four main layers [00:07:42]:

1. Orchestration
2. Prompt Engineering Tools
3. Skills Invocation
4. Content and Memory Management

Beyond these core components, the broader LinkedIn GenAI ecosystem includes sister teams working on modeling (e.g., fine-tuning open source models), responsible AI (ensuring agents behave according to policy), and AI/machine learning infrastructure for hosting models [00:07:56].

Value Proposition of the Platform

The primary value of LinkedIn’s GenAI platform is to serve as a unified interface for this complex ecosystem [00:08:20].

• Simplified Access: Developers can leverage the platform to quickly access the entire GenAI ecosystem without needing to understand every individual component [00:08:32]. For instance, developers can switch between OpenAI models and internal models by changing a single parameter in their SDK code, significantly reducing infrastructure integration complexity [00:08:50].
• Enforcing Best Practices and Governance: The centralized platform provides a mechanism to enforce best practices and governance, ensuring applications are built efficiently and responsibly [00:09:12].

Why a Dedicated GenAI Platform is Critical

Building an in-house GenAI platform is deemed critical for success [00:09:52]. GenAI systems are fundamentally different from traditional AI systems [00:09:59]:

• Blurred Lines: In traditional AI, there’s a clear separation between model optimization and model serving, allowing AI engineers and product engineers to work on different tech stacks [00:10:04]. In GenAI, this line disappears; everyone can optimize overall system performance, creating new challenges for tooling and best practices [00:10:24].
• Compound AI Systems: GenAI and agent systems are “compound AI systems,” defined as tackling AI tasks using multiple interacting components, including multiple model calls, retrievers, or external tools [00:10:49]. This requires skills across both AI engineering and product engineering [00:11:10].
• Bridging Skill Gaps: The GenAI application platform serves to bridge the skill gaps between AI engineers and product engineers [00:11:17].

Building a GenAI Team

Recruiting for a GenAI platform team can be challenging due to the specialized and evolving nature of the field [00:11:39].

Ideal Candidate (The “Unicorn”)

An ideal candidate possesses a rare combination of skills [00:11:55]:

• Strong software engineering skills, particularly in infrastructure integration [00:12:01].
• Good developer product management skills for interface design [00:12:07].
• AI and data science background to understand the latest techniques [00:12:14].
• Hands-on and capable of learning rapidly [00:12:21].

Realistic Hiring Principles

Given the difficulty in finding a single “unicorn,” LinkedIn follows several principles [00:12:35]:

1. Prioritize Software Engineering: Core software engineering skills are prioritized over AI expertise [00:12:47].
2. Hire for Potential: Due to the fast-evolving nature of GenAI, experience can quickly become outdated. Hiring for potential allows individuals to adapt and grow [00:13:03].
3. Build a Diversified Team: Instead of seeking one person with all qualifications, LinkedIn hires a diverse team including full-stack software engineers, data scientists, AI engineers, data engineers, fresh graduates from top research universities, and individuals from startup backgrounds [00:13:15]. This diverse team collaborates, allowing engineers to pick up new skills and evolve into ideal candidates [00:13:50].
4. Emphasize Critical Thinking: The GenAI landscape changes rapidly; solutions can become outdated in less than six months [00:14:14]. The team constantly evaluates open source packages, engages with vendors, and proactively deprecates solutions [00:14:21].

Key Takeaways for GenAI Platform Development

Tech Stack Choice

• Python is Recommended: LinkedIn strongly recommends Python. While they started with Java and Python, Python was chosen due to its prevalence in research and open source communities, and its scalability [00:14:37].

Key Components to Build

• Prompt Source of Truth: Essential for robust version control of prompts, which are akin to traditional model parameters. This is critical for operational stability to prevent accidental production edits and negative side effects [00:15:01].
• Memory: A crucial component for injecting rich data into the agent experience [00:15:26].
• Uplifting APIs into Skills: In the agent era, converting existing APIs into easily callable skills for agents is vital, requiring surrounding tooling and infrastructure [00:15:42].

Scaling and Adoption

• Start Small, Solve Immediate Needs: Instead of attempting to build a full-fledged platform initially, begin by addressing immediate needs. LinkedIn started with a simple Python library for orchestration and gradually expanded [00:16:07].
• Focus on Infrastructure and Scalable Solutions: Leverage existing infrastructure where possible. For instance, LinkedIn successfully used its messaging infrastructure as a memory layer, which proved both cost-efficient and scalable [00:16:29]. This highlights the importance of leveraging existing infrastructure for AI integration.
• Prioritize Developer Experience: The ultimate goal of the platform is to maximize developer productivity [00:16:46]. Adoption is key for success, so align the technology with developers’ existing workflows to ease the transition and promote success [00:16:56]. This is part of leadership and organizational strategies for AI integration.

More technical details on LinkedIn’s GenAI platform development can be found on their engineering blog [00:17:11].