From: aidotengineer
Anthropic is an AI safety and research company dedicated to building the world’s best and safest large language models (LLMs) . Founded by leading experts in AI, Anthropic has consistently released frontier models while prioritizing safety techniques, research, and policy .
Their most recent model, Sonnet 3.5, launched in late October of the previous year, is noted as a leading model in the code space, performing at the top of leaderboards for evaluations like sbench, an agentic coding evaluation .
Research Directions: Model Interpretability
Anthropic’s research is distributed across model capabilities, product research, and AI safety, with a key differentiator being their focus on interpretability . This involves “reverse engineering the models” to understand how and why they are thinking, and developing capabilities to steer them in the right direction for specific use cases .
Interpretability research is still in its early stages . The approach builds in stages:
- Understanding Grasping AI decision-making .
- Detection Understanding specific behaviors and labeling them .
- Steering Influencing the AI’s input .
- Explainability Unlocking business value through interpretability methods .
In the long term, interpretability is expected to provide significant improvements in AI safety, reliability, and usability . The interpretability team uses methods to understand feature activations at the model level and has published research on “towards Mon semanticity” and “scaling mon semanticity” . As technology improves, detection landscapes could lead to a better grasp of model thinking and behavior, or even discovering “sleeper agents” for safety reasons buried within model capabilities .
Examples of Interpretability and Steering
- Feature Activation: If a model is asked about NBA match scores and mentions “Steph Curry,” it might activate a feature (e.g., “feature number 304 famous NBA players”) representing a recognizable pattern of neurons that activate when famous basketball players are mentioned .
- Model Steering (Golden Gate Claude): An example of steering the model involved “amping up the activation in the Golden Gate Direction.” This resulted in Claude responding to questions like “what should I paint my bedroom” with suggestions related to the Golden Gate Bridge, such as painting it red like the bridge .
Implementing AI: Best Practices and Common Mistakes
Anthropic’s applied AI team supports customers in technical aspects of use cases, helping design architectures, evaluations, and tweak prompts to get the best out of their models . They work closely with customers facing niche challenges in specific use case domains, applying the latest research to maximize model output . This often involves defining metrics for model evaluation and deploying iterative loops into A/B test environments and eventually production .
Case Study: Intercom’s Finn AI Agent
Intercom, an AI customer service platform, partnered with Anthropic to enhance their AI agent, Finn, which was already a market leader .
- Anthropic’s applied AI team collaborated with Intercom’s data science team on a two-week sprint .
- They compared Intercom’s hardest prompts for Finn against prompts developed with Claude, observing positive initial results .
- This led to a two-month sprint focused on fine-tuning and optimizing all prompts to maximize Claude’s performance .
- Benchmarks showed Anthropic’s model outperforming the previous LLM .
- Intercom’s Finn 2, powered by Anthropic, can solve up to 86% of customer support volume, with 51% resolution out of the box .
- The model also improved the “human element” by allowing adjustments of tone and answer length, and was effective at policy awareness (e.g., refund policies), unlocking new capabilities .
Key Practices for Implementing AI Models
1. Importance of Testing and Evaluation
A common mistake is building a robust workflow before designing evaluations . Evaluations should direct towards the desired outcome and be established early .
- Data Problems: Claude can be used for data cleanup and reconciliation when designing evaluations .
- Representative Test Cases: It’s crucial to design test cases that represent the full range of probable user interactions, including “silly examples,” to ensure the model responds appropriately or reroutes questions .
- Statistical Significance: Avoid “trusting the vibes” from a few queries; ensure enough samples are used to achieve statistically significant results, preventing unexpected outliers in production .
- “Evals are your Intellectual Property”: Effective evaluations allow organizations to navigate the “latent space” of model functions and find optimal states faster than competitors .
- Telemetry: Invest in telemetry to backtest architecture in advance .
2. Identifying Metrics and Trade-offs
Organizations typically balance intelligence, cost, and latency, often optimizing for one or two at a time . This balance should be defined in advance based on the specific use case .
- Time Sensitivity: For a customer support use case, a response within 10 seconds might be critical, as customers may abandon the page if it takes longer . Conversely, a financial research analyst agent might tolerate a 10-minute response time due to the high stakes of the subsequent decision .
- User Experience (UX): Creative UX solutions, like a “thinking box” or redirecting customers to another page, can help manage latency expectations .
3. Cautions on Fine-tuning
Fine-tuning is not a “silver bullet” .
- Cost: It involves “brain surgery on the model,” which can limit its reasoning in areas outside the specific fine-tuning domain .
- Prioritize Other Approaches: Most organizations attempt fine-tuning without a clear evaluation set or success criteria . It should only be pursued if necessary to achieve specific intelligence domain goals not met by other methods .
- Justify the Cost: The wide variance of success in fine-tuning means the effort and cost must be justified .
- Don’t Let it Slow You Down: Teams should pursue language model use cases and only integrate fine-tuning if needed, rather than waiting for it .
Alternative Methods to Enhance Model Performance
Beyond basic prompt engineering, several features and architectures can significantly impact the success of a use case:
- Prompt Caching: Can lead to a 90% cost reduction and 50% speed increase without sacrificing intelligence .
- Contextual Retrieval: Improves retrieval mechanisms, feeding information more effectively to the model and reducing processing time .
- Citations: An out-of-the-box feature .
- Agentic Architectures: Architectural decisions that can drastically change a use case’s success .