From: redpointai
The field of AI infrastructure continues to present significant opportunities for startups, even as foundational model providers expand their offerings. While users desire a “one-stop shop” for Large Language Model (LLM) capabilities, there remains considerable room for specialized solutions and developer tooling [00:27:38].
Core Opportunities in AI Infrastructure
Specialized Infrastructure for Vertical Domains
There is a strong case for vertical-specific AI infrastructure companies [00:28:23]. Examples include:
- Coding AI Startups: Companies building Virtual Machines (VMs) specifically for coding AI startups to test code and quickly spin down VMs [00:28:26]. This represents an area that foundational model providers are unlikely to enter [00:28:47].
- Mobile Environments: The demand for agents to operate across diverse environments, such as iPhone screenshots or Android, suggests a need for companies specializing in “iPhone VMs” or testing frameworks for AI models in specific mobile operating systems [00:30:38]. This addresses the fragmentation across different operating system flavors like Ubuntu [00:31:17].
LLM Operations (LLM Ops)
A growing class of LLM Ops companies is emerging, offering valuable services that developers care about [00:28:51]. These services are not necessarily low-level infrastructure but focus on managing and optimizing LLM usage, including:
- Managing prompts [00:28:56]
- Managing billing [00:28:57]
- Understanding usage patterns [00:28:57]
- Supporting multi-model and multi-provider environments, like OpenRouter [00:29:07]
Computer Use VM Space
The “computer use VM space” is still very early, but models in this area are expected to improve rapidly [00:29:57]. Key challenges and opportunities include:
- Secure and Reliable Deployment: Helping enterprises securely and reliably deploy these virtual machines within their own infrastructure [00:30:05].
- Observability: Providing tools to observe and monitor what computer use models are doing on top of these VMs [00:30:12].
- Diverse Environments: Addressing the need for the computer use tool to function effectively across various environments beyond just browsers, such as iPhone screenshots or Android [00:30:38].
- New Applications: Exploring areas like cybersecurity work, where models can find vulnerabilities in sites and surfaces [00:31:27].
Challenges for Startups
Challenges of building AI infrastructure companies are evident, particularly in the rapid evolution of foundational models.
- Balancing Simplicity and Customization: Model providers aim to offer significant power out-of-the-box with simple APIs, allowing developers to gradually access more granular control through “knobs” as needed [00:22:05]. Startups in this space must consider how to provide similar flexibility.
- Productizing Evaluation and Task Generation: One of the biggest problems remaining in AI infrastructure is productizing grading and task generation in a way that works for various domains [00:12:40]. While possible to build tools like Deep Research, making these universally accessible and easy to use is difficult [00:13:08].
- Evolving Workflows: Early agentic products in 2024 relied on clearly defined workflows with a limited number of tools (less than 10) [00:07:02]. However, 2025 has seen a shift towards “chain of thought” models that can dynamically call multiple tools, even course-correcting if they go down the wrong path [00:07:32]. The next unlock is removing the constraint on the number of tools an agent can access [00:08:05].
- Model Runtime: Increasing the available runtime for models from minutes to hours or days will yield powerful results, requiring robust infrastructure [00:08:49].
Investment Perspectives and Market Dynamics
- Foundational Model Dominance: OpenAI’s strategy is to be a one-stop shop for LLM capabilities, providing tools like web search, file search, and computer use within their API [00:27:44]. This impacts the landscape for standalone AI infrastructure companies.
- Room for Low-Level APIs: Despite the expansion of foundational model providers, there will always be a large market for AI infrastructure companies that build low-level, infinitely flexible APIs [00:28:01].
- The Scaffolding Dilemma: Developers often build significant scaffolding around current models to make them work, which provides immediate market value [00:19:17]. However, future model advancements could potentially obviate some of this scaffolding [00:19:24].
- Orchestration as a Differentiator: Being adept at orchestrating agents and tools, combining data with multiple model calls, and rapidly evaluating and improving these systems will be the biggest differentiator for application builders in the long term [00:41:02].
- Underexplored Applications: Scientific research and robotics are seen as highly underexplored applications for AI models, with the potential for significant breakthroughs once the right interfaces are found [00:41:41].
Advice for Companies
Companies should begin exploring frontier and computer use models now [00:36:42].
- Identify internal manual workflows that can be automated using multi-agent architectures [00:36:49].
- Determine which workflows require a tool interface and start building that integration [00:37:05].
- Focus on orchestrating models and tools effectively, as the current models are far more capable than how most applications utilize them [00:19:52].
- Ask employees about their least favorite tasks and explore ways to automate them to increase productivity and satisfaction [00:38:15].
Future Outlook
Model progress is expected to accelerate even further [00:33:33]. The focus will be on:
- Developing smaller, faster models that are proficient at tool use for quick classifications and guardrailing [00:32:53].
- Improving models’ ability to generate clean code differences (diffs) that can be applied seamlessly [00:33:35].
- Making the process of evaluation and fine-tuning significantly easier (by about 10 times) for developers [00:35:05].