From: redpointai
Logan Kilpatrick, OpenAI’s first AI hire, discussed the evolving landscape of AI adoption within enterprises, highlighting the role of custom models and the strategic integration of AI platforms.
Custom Models and Fine-tuning
The ability to fine-tune models like GPT-4 was announced at DevDay, though it was rolled out to those who had already attempted fine-tuning GPT-3.5 turbo [00:04:31]. Fine-tuning GPT-3.5 can achieve GPT-4 level performance with proper prompt engineering and token savings [00:05:58]. This requires significant effort in crafting prompts, especially when dealing with more than three or four instructions [00:05:02].
When to use Custom Models
Custom models are most beneficial for companies with a large amount of specialized data in domains where the base model isn’t already proficient, such as the legal or medical sectors [00:07:40]. Building these models is currently a significant investment, costing around $2-3 million and requiring billions of tokens for training [00:08:17]. This makes it challenging for startups without substantial capital expenditure [00:08:27].
OpenAI’s custom model program involves their research teams directly assisting companies that may lack world-class machine learning expertise to train these models [00:11:15].
Future of Custom Models
Kilpatrick believes there will always be a need for custom models, even as base models continue to improve and become more “steerable” [00:09:37]. This is because custom models can be more compute-efficient by focusing on specific, deeply relevant data and excluding unnecessary information from training sets [00:10:02]. The goal is to make the custom model offering more accessible and affordable through an API in the future [00:10:22].
Enterprise AI Adoption and Integration
OpenAI serves a wide range of users and use cases [00:11:52]. Prioritization for product development focuses on providing a world-class service, which includes significant investments in reliability [00:12:41]. Shipping new capabilities often takes precedence due to engineering resource constraints [00:12:57]. OpenAI aims to evolve into a “true Enterprise platform” by addressing rough edges like detailed dashboards, monitoring, and alerts in 2024 [00:13:28]. This aligns with enterprise AI adoption challenges and solutions.
OpenAI vs. Open Source Models
Users often choose open-source models for full ownership of weights and IP, which is critical for some business use cases [00:14:47]. Open-source models like Llama offer more customization options with fine-tuning beyond standard OpenAI offerings [00:15:52].
However, Kilpatrick believes OpenAI’s models will generally remain superior due to the immense cost and engineering effort required for training very large models [00:15:03]. The ease of use of OpenAI’s API, which removes the need to worry about GPU allocation or complex setups, also provides significant developer experience value [00:17:15]. OpenAI plans to support more fine-tuning and training techniques, like RLHF, in the future [00:16:07].
Tools and Integration in Enterprise AI
When enterprises integrate AI, they commonly use:
- Observability products: These tools allow developers to monitor API usage, spend per API key, and view logs and requests [00:17:37]. While OpenAI acknowledges the need for such features, they currently often take a backseat to shipping new capabilities [00:13:08].
- LLM orchestration frameworks: Tools like LlamaIndex and LangChain are widely used for building features [00:18:43]. Haystack and Prompt Layer are also noted [00:18:50].
- Custom infrastructure: Many technically sophisticated companies prefer to rebuild much of this infrastructure themselves rather than relying on third-party dependencies, especially given the nature of venture-backed open-source companies [00:20:25].
Key Challenges in Enterprise AI Deployment
The main objections and blocks for enterprises in deploying LLMs are:
- Robustness and Reliability: Enterprises often need to use third-party tools like Guardrails AI or other LLM companies for compliance and to ensure confidence in production environments [00:45:08]. OpenAI aims to solve many of these problems upstream on their platform [00:45:43]. This is a core part of challenges and strategies in enterprise AI deployment.
- Latency: Many use cases cannot tolerate waiting 7 seconds for a response [00:46:11]. Reducing inference time is a continuous internal development focus, with a goal that latency will no longer be an objection by late 2024 [00:46:23]. Instant responses are crucial for maintaining user flow in creative tasks [00:46:50].
“LLMs are like a clone of human thought, and in many ways, it doesn’t move at the speed of thought, and I think that’s that can be so jarring in a lot of a lot of experiences.” [00:47:03]
OpenAI’s Product Strategy and Future Outlook
OpenAI’s product footprint is broad, serving many user types and use cases [00:11:50]. They prioritize shipping new capabilities and investing in reliability [00:12:57].
Evolution of Products
- Plugins: Initially framed as a product release, plugins were more of a “research preview” with an ambitious mission [00:25:10]. They faced limitations due to resource constraints and security/privacy concerns, such as taking consequential actions or needing user consent [00:26:00]. Discoverability was also a significant challenge [00:27:23].
- Assistants API and GPTs: These offerings have solved many of the problems previously encountered with plugins [00:26:39]. They allow combinations of browsing, code interpreter, and custom actions, offering a much better interface [00:27:00]. The upcoming GPT Store is expected to resolve discoverability issues [00:27:19]. Current use cases for GPTs largely revolve around sharing prompts [00:27:48].
- Multimodal AI: Kilpatrick anticipates 2024 to be the “year of multimodal” [00:19:19]. While early for vision use cases, he believes that once the models make a jump in understanding positional relationships between objects (similar to the leap from GPT-3.5 to GPT-4), many more use cases will be unlocked [00:03:55].
- An example of a successful multimodal application is TLDraw, which converts user drawings into functional apps, showcasing the orchestrated use of various OpenAI tools like Vision [00:39:48].
Deployment Models and Strategy
OpenAI’s strategy, similar to Microsoft’s Co-pilot, is to be present where customers and users already are, rather than solely relying on users visiting chat.openai.com [00:34:40]. This includes embedding AI experiences directly into existing workflows and applications, like text messaging or email [00:31:00].
Kilpatrick suggests a need for a “text-first assistant experience” that can integrate with platforms like Twilio, bringing AI assistance to surface areas where users already work, without requiring them to navigate to new websites or learn new habits [00:30:24].
Overhyped and Underhyped in AI
- Overhyped: Prompt engineering [00:48:15]. While useful, Kilpatrick believes the fundamental nature of prompt engineering—communication—will eventually be abstracted away as models become better at understanding imprecise human requests, similar to DALL-E 3’s revised prompts [00:28:57].
- Underhyped: Observability [00:48:18]. Understanding everything happening with models is crucial for developers [00:48:27].
- Unexpected Success: Function calling [00:49:40]. This feature, which enables many interesting production use cases, was an unexpected but significant development [00:49:51].
The broader AI ecosystem, including major players like Google with Gemini, is seen as beneficial for consumers, driving innovation and expanding awareness of AI’s possibilities [00:24:26]. This contributes to the broader discussion around enterprise and consumer AI trends.
Getting Started with AI for Developers and Enterprises
Kilpatrick recommends developers and enterprises conduct an audit of their daily tasks to identify processes they dislike or wish to improve [00:54:24]. For developers, using tools like ChatGPT and GitHub Copilot is considered “table stakes” to amplify capabilities [00:55:06]. He suggests integrating AI into daily workflows to make it a habit, which helps in understanding how the technology will impact life and career [00:56:03].
He emphasizes that companies like Apple and Google have an important role in educating a wider audience about AI’s potential by integrating it into familiar experiences like Siri [00:57:25].
Conclusion
OpenAI aims to be the “AWS of AI,” offering a comprehensive platform that covers every step a developer needs, from models and fine-tuning to specific training techniques [00:59:22]. The focus remains on continually building out capabilities while striving for affordability and accessibility.