From: aidotengineer
Brightwave, a company specializing in research agents for the financial domain, focuses on how AI can digest large volumes of content to assist professionals in tasks like due diligence, competitive deal processes, and financial analysis [00:00:19]. This work often involves thousands of pages of content and demands rapid conviction and the identification of critical risk factors [00:00:31]. While AI agents can accelerate efficiency by orders of magnitude [03:10:00], the role of human oversight and interaction remains crucial.
The Indispensable Role of Human Oversight
Despite advancements, purely automated AI models, particularly “non-reasoning models,” often perform “greedy local search,” which can lead to significant error rates when calls are chained together [04:10:00]. For example, a 5-10% error rate in extracting organizations from an article can exponentially increase the likelihood of error in subsequent steps [04:24:00]. Achieving “globally optimal outputs” by making “locally suboptimal decisions” is still an open research problem for AI systems [04:47:00].
Human oversight is considered extremely important in these systems [10:04:00]. This is because human analysts possess unique access to non-digitized information, such as direct conversations with management or insights from portfolio managers [10:20:00]. This “taste making” ability allows humans to contextualize and prioritize information in ways AI cannot [10:31:00].
Human-AI Interaction and Steering
Interacting with language models, often described as a “skill,” involves steering their outputs [06:15:00]. In multi-turn conversations, each response can be thought of as a selection of “reactions” that nudge the model’s activations towards solving the problem at hand [06:05:00]. This process is akin to a “human orchestrated Chain of Thought” [06:33:00].
Users can directly influence the AI’s behavior:
- Decomposition: Decomposing research instructions into multiple sub-themes allows for more granular and specific queries, leading to higher-fidelity and information-dense outputs [14:30:00].
- Self-Correction: Models can self-correct when asked about the factual entailment or nature of an output (e.g., “is this actually an organization?“) [09:22:00]. However, performing this as a secondary call is often more effective than embedding it in the initial prompt, as models tend to be “credulous” of their initial outputs [09:41:00].
- Nudging and Directives: Humans can actively “nudge the model with directives” or select “interesting threads” for the AI to further investigate [10:09:00].
Product Design for Seamless Collaboration
A critical design challenge for AI-driven analysis is how to “reveal the thought process of something that’s considered 10,000 pages of content to a human in a way that’s useful and legible” [03:40:00]. This involves more than just UI/UX or product architecture problems that existed previously [03:52:00].
Key design patterns observed in AI-driven analysis include:
- Mimicking Human Workflow: AI agents should mimic the human decision-making process, decomposing tasks in a way a person would naturally approach them, such as looking for public market comparables or assessing relevant document sets [08:00:00].
- Scaffolding Workflows: Products need to provide scaffolding to orchestrate workflows and shape system behavior, specifying intent and reducing the burden on the user to become a “prompting expert” [07:31:00].
- Transparency and Auditability: The AI’s analysis should be presented as a “surface” or “high dimensional data structure,” allowing users to “turn over that cube” and see the “receipts” or audit trail of how conclusions were reached [18:44:00].
- Details on Demand: Users need the ability to get “details on demand” [17:28:00]. This includes:
- Clickable citations that provide additional context about the source and the model’s reasoning [17:40:00].
- Structured interactive outputs that allow users to “pull the thread” and ask for more information [17:52:00].
- The ability to highlight any passage of text and ask for implications or further details [18:03:00].
- Presenting findings in a structured way (e.g., fundraising timelines, ongoing litigation) where users can click to investigate specific points of interest [19:07:00]. This acts like a “magnifying glass for text” [19:26:00].
The Latency Trap and User Learning
A significant challenge in user-centric AI design is the “latency trap” [11:59:00]. If the feedback loop for user interaction is long (e.g., 8-20 minutes), users will have fewer opportunities to engage with the system and refine their mental model of how their prompts elicit specific behaviors [12:49:00]. This limits their faculty with the system and product [12:56:00].
The “final form factor” of products in this class, such as Brightwave’s research agent, is still being determined, but the design problem is extremely interesting [19:47:00]. The focus is on providing a continuous, interactive surface where users can explore and refine the AI’s analysis [18:14:00].