From: mk_thisisit
It is considered wise to anticipate a scenario where strong artificial intelligence, smarter than the smartest people, could emerge before 2030, even if the probability is small. Such a possibility requires preparation [00:00:00].
Defining Ethical Boundaries
The process of defining the ethical limits for AI is a crucial and multifaceted issue [00:11:30]. This responsibility should not fall solely on a single company; rather, it should ideally be a democratically decided process [00:12:00]. Companies can contribute by making technology available to better collect diverse public opinions and involve as many people as possible in co-creating a code of ethics [00:12:20]. Given the rapid growth in popularity of tools like ChatGPT, the urgency of addressing this problem has become apparent [00:12:37]. Discussions are ongoing regarding the best methods for collecting these opinions, potentially including forms of voting, while ensuring all demographic groups are represented [00:12:50].
Technical Implementation of Standards
Once ethical standards are defined, the second, technical challenge is to ensure that AI models adhere to them [00:13:36]. This is a complex engineering problem [00:13:43]. Research suggests that as AI models become more intelligent (e.g., the leap from GPT-3.5 to GPT-4), they become better at following imposed instructions and standards [00:14:02]. Therefore, increased intelligence paradoxically makes it easier for the AI to comply with ethical guidelines [00:14:19].
Governance and Control
OpenAI’s structure emphasizes that Microsoft, despite being a significant investor and partner, does not have control over OpenAI [00:16:09]. Microsoft does not hold a single seat on OpenAI’s board [00:16:40]. This intentional separation ensures that decisions about powerful AI models, which can profoundly impact human lives, are not made by a publicly traded corporation obligated to prioritize investor interests [00:16:53]. OpenAI is structured with a leading non-profit component to uphold its mission of ensuring that the benefits of its technology extend to all people [00:17:15]. This approach aims to address ethical considerations in AI development.
Challenges in AI Reasoning
While AI models are progressing, certain limitations present challenges and ethical considerations in AI development. For instance, developing an “automatic scientist” AI is currently impossible because the models have a probability of error at each step of their reasoning [00:09:51]. Even if this error rate is small, for complex scientific problems requiring thousands or tens of thousands of reasoning steps, the cumulative probability of error significantly increases [00:10:04]. Future efforts might focus on creating mechanisms for AI to confirm scientific claims or seek mathematical proofs, though this remains a highly difficult form of reasoning [00:10:27].