From: aidotengineer
AI coding agents, once considered science fiction, are now a reality, with some even contributing to their own development [00:00:20]. These agents are transforming the landscape of software engineering and are predicted to dominate conversations in 2025 regarding how the field is changing [00:01:00].
Evolution of AI in Software Development
The AI Dev tools space is rapidly changing [00:00:40]:
- 2023: Characterized by autocomplete models like GitHub Copilot [00:00:49].
- 2024: Chat models began to penetrate software engineering organizations [00:00:56].
- 2025: AI agents are expected to become the primary focus [00:01:00].
AI Coding Agents: A Self-Building Capability
A notable aspect of these agents is their capacity for self-improvement. One agent developed by Augment Code, an AI research company, helped build itself, writing over 90% of its approximately 20,000 lines of code with human supervision [00:01:21].
This self-building capability extends to several core features:
Third-Party Integrations
AI agents are designed to interact with various platforms essential for software engineers, such as Slack, Linear, Jira, Notion, Google Search, and codebases [00:01:45]. For example, an agent was able to add a Google search integration by analyzing the codebase and figuring out the correct file and interface [00:02:08]. In one instance, when adding a Linear integration, the agent used its previously written Google search integration to look up the Linear API documentation, as the foundation model did not have it memorized [00:02:21].
Testing
Agents can write unit tests for new integrations or existing code, provided they are given basic process management tools like running subprocesses and handling output [00:02:37].
Optimization
Beyond writing features and tests, agents can perform optimizations. An agent tasked with profiling itself added print statements, ran sub-copies, and identified a bottleneck involving synchronous loading and hashing of repository files. It then implemented a process pool and stress test to improve performance [00:03:14].
How Agents Interact with Their Environment
AI agents operate with a “Master Level agent” that plans tasks and utilizes various tools to interact with its environment [00:04:20]. These tools include:
- Google Search: For external information retrieval [00:04:13].
- Retrieval Tool: To search and navigate the local codebase [00:04:55].
- File Editing Tool: For making quick and performant edits to files [00:05:10].
- Clarify Tool: To ask users for clarification when encountering missing information, like credentials [00:05:37].
- Memory Tool: Allows the agent to continuously learn from human interactions and store useful information for future use [00:06:01].
A “super powerful scalable Enterprise-ready context engine” is crucial for agents to function effectively, providing access to diverse context sources such as Slack, Linear, Jira, Notion, and the codebase [00:07:07]. This context, combined with the reasoning capabilities of a best-in-class foundation model and a safe code execution environment, forms the foundation for building effective agents [00:07:33].
Challenges and Insights in Developing AI Coding Agents
Building these agents has revealed several key lessons:
Assumptions vs. Reality
- L5 Agents are Here: Despite impressive Twitter demos, professional software engineering environments are complex and messy [00:08:05]. Agents are not yet at the senior software engineering level, but they are still highly useful [00:08:29].
- Agents Take Over Entire Task Categories: Instead of specializing in tasks like backend or frontend development, this technology is general-purpose. Agents’ capabilities are improving across all fronts simultaneously [00:08:55].
- Anthropomorphizing Agents: Agents possess different strengths and weaknesses than humans. An agent might struggle with math but rapidly implement a frontend feature [00:09:21].
Onboarding the Agent
Just like human new hires, agents need to be onboarded to an organization’s specific tools and practices [00:11:21]. Creating a “knowledge base”—a set of information like markdown files describing tools (e.g., Graphite), tool stacks, test procedures, and style guides—allows the agent to dynamically search and understand information it doesn’t already know [00:10:28].
Impact on Product Development
When code becomes “cheap” to write due to agents, the bottleneck shifts from engineering hours to good product insights and design [00:12:08]. This allows product teams to explore more ideas and iterate faster [00:12:29].
Importance of Context
Good context is critical [00:13:42]. Agents can understand high-level natural language instructions if they have strong codebase awareness [00:12:41]. Context isn’t limited to code; it comes in many forms (e.g., Slack, pull requests) and is multiplicative—access to multiple context sources significantly increases an agent’s utility [00:14:45].
The Critical Role of Testing
Agents, like humans, make mistakes, especially with complex edge cases like parallel programming and caching [00:14:57]. The absence of sufficient tests for such scenarios means agents can introduce bugs [00:15:08].
- Better Tests Enable More Autonomy: Upgrading a foundation model by six months improved a bug-fixing benchmark by 4%, but adding the ability for the agent to run tests, receive feedback, and iterate on fixes led to a 20% gain [00:15:19]. This demonstrates that robust test harnesses are crucial for increasing agent trustworthiness and intelligence [00:15:49].
The Future of AI Engineering
The future of AI in coding suggests a significant shift in software engineering [00:15:57]:
- AI agents are rapidly improving, with their self-building capabilities accelerating this pace [00:16:11].
- Code will remain important as a specification of systems, but the human relationship with code is changing [00:16:20].
- Good test harnesses are more critical than ever, especially for less well-tested parts of codebases [00:16:25].
- The calculus of product development is shifting, with greater emphasis on product work, gathering customer feedback, and building insights as code generation becomes cheaper [00:16:37].
This technology is poised to positively transform the software industry [00:16:50].