Challenges with AI models including hallucinations and bias

From: redpointai

Peter Welinder, VP of Product and Partnerships at OpenAI, discusses the challenges and strategic approaches OpenAI takes concerning their AI models, including the prevalent issues of hallucinations and bias, and the broader implications of advanced AI development [00:00:09].

Current Gaps in AI Model Development

Welinder notes that the biggest gap currently preventing widespread enterprise adoption of AI models is the problem of hallucinations [00:26:17].

Hallucinations

Hallucinations occur when AI models generate factual statements that are not true or cannot be verified [00:22:22].

• Impact: Users cannot entirely trust the models, which is a major barrier to adoption [00:26:22].
• Solution Approaches:
  • OpenAI invests significant effort to make their models more robust against hallucinations [00:26:42].
  • Companies commonly address this by “grounding” models in external data [00:26:55]. This involves:
    • Embedding questions and internal company documentation [00:27:06].
    • Using vector databases for relevant document retrieval [00:27:18].
    • Instructing the language model to synthesize answers from these documents or to state “I don’t know” if the answer isn’t found [00:27:21].
• Nature of the Problem: Hallucinations remain an open research problem at the cutting edge of the field [00:26:33].

Bias

Bias in AI models is considered impossible to eliminate entirely [00:31:34].

• OpenAI’s Approach: Provide tools for product developers and users to instruct the model to have desired biases within certain bounds [00:31:38].
• Goal: Models should not possess a particular political orientation; the user should be able to choose the model’s behavior [00:31:55].

Broader Safety Concerns and the Path to Superintelligence

Welinder distinguishes between “surmountable” risks and the more profound risk posed by superintelligence [00:30:53].

Surmountable Risks

Risks such as misinformation, deepfakes, and job displacement are seen as largely surmountable [00:30:53].

• Misinformation: Primarily a distribution channel problem, leveraging existing platforms (social media, email) which already have infrastructure to protect against it [00:31:03].
• Bias: As discussed, can be managed by providing tools for user-defined alignment [00:31:38].

Superintelligence Risk

The most critical and under-addressed risk is the emergence of superintelligence—AI models that become far smarter than humans [00:32:19].

• Lack of Research: There is a surprising scarcity of research into ensuring a positive outcome from superintelligence [00:33:33], with no dedicated “superintelligence safety departments” at universities [00:32:45].
• OpenAI’s Strategy:
  • Gradual Deployment: Release models when stakes are low to learn from risks like misinformation and bias [00:38:42]. This approach aims to build the necessary organizational processes, frameworks, and safety safeguards before encountering superintelligence [00:39:17].
  • Cautionary Holds: OpenAI delayed the release of GPT-4 by nearly half a year to gain clarity on its potential downsides [00:39:40].
  • Setting an Example: As a leader, OpenAI hopes its cautious approach will encourage accountability among other developers in the field [00:40:04].
• Potential Timeline: Welinder speculates that something resembling AGI (Artificial General Intelligence) could emerge by the end of this decade (before 2030) [00:34:51]. Superintelligence might follow, characterized by abilities such as thinking faster than humans, parallel processing, or conducting more experiments than a human [00:36:10]. He places the early signs of superintelligence around 2030 [00:46:40].
• Optimism: Despite the risks, Welinder is optimistic that humanity can navigate these challenges, similar to avoiding nuclear war due to self-preservation [00:38:01].
• Upside Potential: Superintelligence holds immense promise for solving global problems like climate change, cancer, and aging, leading to greater abundance and a higher standard of living [00:40:23].

Research Areas for Superintelligence Safety

If leading a superintelligence safety department, Welinder would focus on:

1. Interpretability: Understanding what is happening inside black-box models by studying activations within deep neural networks [00:41:18]. This can move faster than traditional neuroscience [00:41:54].
2. Defining Alignment: Achieving crisper specifications for goals and guardrails for AI [00:42:16]. This requires collaboration between technical experts, social scientists, and philosophers [00:42:45].
3. Technical Approaches: Exploring methods such as shaping reward functions for reinforcement learning, or having one AI model oversee and report on the actions of another [00:42:55].

OpenAI’s Focus and Competitive Landscape

OpenAI’s primary focus is on continually improving the intelligence and safety of its core models [00:47:14].

• Prioritization: Key priorities include lower prices, higher reliability, and lower latency for their models [00:29:21]. These are considered fundamental, as external tooling is less effective without a strong core intelligence product [00:29:38].
• Developer Tooling: While the ecosystem will build most tooling, OpenAI will provide tools for common needs to help developers get started quickly [00:28:34].

A significant concern for OpenAI is the risk of losing touch with its users and developers due to rapid growth [00:47:24]. As models become more capable, they might replace functionality that developers have built, creating tension [00:47:40]. Maintaining a strong relationship and excellent customer experience, even with millions of users, is crucial to ensuring continued embrace and development on OpenAI’s platform [00:48:34].