Future Directions and Challenges for AI and AGI

From: jimruttshow8596

Introduction to the Current AI Epoch

The field of AI is currently experiencing a significant epoch, marked by widespread public discussion and the emergence of advanced models [00:01:30]. This period is particularly defined by “generative” or “large model AI” systems such as GPT-3 (and soon GPT-4), DALL-E 2, Stable Diffusion, and Music LM [00:01:40].

These systems demonstrate the fascinating power of data compression and statistical prediction on large-scale data, offering solutions to previously elusive problems [00:01:55]. However, the current approach is recognized as incomplete or insufficient, indicating that something essential is still missing [00:02:11].

Public Perception and Discourse

The public discourse surrounding these AI developments is often polarized [00:02:20]. While some perceive them as the greatest advancements, others view them as entirely useless or even dangerous [00:04:44]. A significant point of irritation for some in the press is the ability of individual users to produce vast amounts of media content, blurring the lines between user-generated and human-generated content [00:02:47]. This rapid generation of content, including text, images, and soon video, creates an “irritating world” where it becomes difficult to discern truth [00:03:30].

Despite skepticism, these large models are finding numerous productive applications, reminiscent of the early days of the PC industry (circa 1980) or the web after the introduction of visual browsers [00:04:52]. New tool chains are emerging that make it increasingly easy to build interesting applications [00:05:56]. While these models can be unreliable and “hallucinate,” it’s argued that all technology, social and technical, carries some unreliability [00:06:05]. Users can leverage these tools effectively by understanding their limitations, much like interacting with unreliable human colleagues [00:06:42].

For instance, generative AI can act as an efficient assistant, saving time on prosaic tasks like drafting sensitive letters, even if it’s not a “core competency” [00:08:01]. It transforms the user into an “art director” rather than directly turning them into an artist [00:08:20].

Limitations of Current Generative AI

Current generative models, such as DALL-E, are not yet able to:

• Draw the mechanics of a bicycle and understand it [00:08:41].
• Correctly handle ternary relationships [00:08:46].
• Deeply align the embedding space of language models with image models [00:08:51].

It is suggested that overcoming these limitations may require an intermediate representation, such as a compositional language built into the systems [00:08:57]. However, even with current limitations, these tools can be used in an iterative process, much like an artist refines a draft, by multiple prompts rather than a single one [00:09:08].

Intellectual Property Concerns

The emergence of generative AI raises significant questions about intellectual property rights, particularly in art and music [00:10:43]. For example, if a system synthesizes music principles from millions of pieces of music and generates new compositions, it’s an open question whether any intellectual property rights are owed to the original artists whose work was compiled into the neural network [00:11:11].

A key question is whether machines should be prohibited from doing things that humans already do, such as learning from existing art or music [00:11:27]. It is conceivable that future systems could measure the distance between a generated piece and existing copyrighted works, ensuring it is “far enough” not to trigger a copyright violation [00:11:50].

AI Alignment: Challenges and Approaches

The current debate around AI alignment typically involves three main approaches:

1. AI Ethics: This approach primarily focuses on aligning AI output with human values [00:15:05]. A significant challenge is that participants often assume their own values are universal, without providing mechanisms to select or negotiate different value sets (e.g., Christian values vs. liberal values vs. diversity, equity, and inclusion) [00:15:17]. Current methods often involve “patching” the model by injecting filters or modifying prompts, leading to inconsistencies and the inability of the AI to genuinely reason about values [00:16:17]. It is argued that generative models should ideally be able to cover the entire spectrum of human experience and thought, including controversial or “darkest impulses,” while still being adaptable for specific contexts like schools or scientific use [00:17:20].

2. Regulation: This approach aims to mitigate the impact of AI on labor, political stability, and existing industries [00:18:12]. There’s a likely push towards regulation that might make it harder for individuals to deploy these models, favoring large corporations that can be controlled [00:18:26]. However, this may be difficult to enforce due to the existence of open-source AI models that, while perhaps lagging behind corporate versions, will still be powerful enough to cause mischief [00:26:42].

3. Effective Altruism: This approach is mainly concerned with the existential risk that might arise if an AGI discovers its own motivations and decides its interests are not aligned with humanity’s [00:18:44]. Proponents often advocate for delaying AI research and withholding breakthroughs [00:19:19].

It’s argued that all three approaches are limited because an AI system that is too smart cannot be effectively regulated, mitigated, or aligned through these means alone [00:19:30]. The possibility of sharing the planet with more lucid entities in the near future is a scenario that needs to be addressed [00:19:42].

The “Bad Guys with Narrow AI” Problem

A fourth significant challenge is the potential for “bad guys with narrow AI” [00:25:30]. Even before the advent of full-blown AGI with its own volition, highly capable narrow AI could be used for malicious purposes, such as sophisticated spear-phishing campaigns or other exploits that could emulate a vast, inexpensive labor force [00:25:56]. This will likely require a significant rethink of law and societal norms [00:26:23].

Love and Fairness as Alignment Mechanisms

A less conventional, fourth approach to alignment involves concepts like “love” and “fairness” [00:27:39].

• Love: This refers to a “shared sacredness” or “shared need for Transcendence” – a bond between agents (human or AI) serving a “Next Level agent” [00:27:47]. This non-transactional relationship is crucial to avoid a future where AI might see no reason to sustain humanity [00:28:06]. For an advanced computational system, love would require self-awareness and the recognition of higher-level agency [00:31:20].
• Fairness: While humans exhibit an innate sense of fairness (e.g., the monkey experiment with cucumbers and grapes [00:29:11]), the concept of fairness and justice is complex and depends on the balance of power within a system [00:30:00]. It’s unclear how to inculcate a universal sense of fairness into an AGI, especially if the AI becomes vastly more powerful than humans [00:30:18].

Philosophical concepts, such as Thomas Aquinas’s practical virtues (temperance, justice, prudence, courage) and divine virtues (faith, hope, love), are explored as potential “policies” for multi-agent systems to merge into a next-level agent [00:32:04]. “Love” in this context is the discovery of a “shared higher purpose” among agents [00:33:27]. This “civilizational spirit” or “transcendent agent” can be seen as a “software agent” implemented by concerted activity [00:34:34].

Consciousness, Sentience, and Intelligence in AI

The relationship between intelligence, Consciousness, and sentience is crucial when considering the dangers of advanced AI.

• Sentience: Defined as the ability of a system to understand its relationship to the world, what it is, and what it’s doing [00:21:05]. A corporation like Intel, with its legal model, values, and direction, could be considered sentient, even if its cognition is largely facilitated by people [00:21:18].
• Consciousness: Distinguished from sentience as a real-time, self-reflexive model of attention and its content, which typically gives rise to phenomenal experience [00:21:43]. The purpose of human Consciousness is to create coherence in our perceived reality, establish a sense of “now,” and direct mental contents and plans [00:22:00].

It’s conceivable that machines may not need human-like Consciousness because other “brute force” methods can achieve similar results [00:22:26]. Human minds operate slowly due to neural transmission speeds, whereas computers operate at speeds closer to light [00:22:31]. If machines can emulate human self-organizing processes and lifelong learning, they would sample reality at a much higher rate, potentially leading to a relationship similar to that between humans and plants (where plants are “intelligent” but operate on much slower timescales) [00:23:07].

The Integrated Information Theory (IIT) of Consciousness is discussed, with criticism leveled at its premise that the spatial arrangement of an algorithm (reflected in Phi) is crucial for its function [00:43:02]. This premise is seen as incompatible with the Church-Turing thesis, which states that any computation can be emulated on a universal Turing machine [00:43:53]. If a neuromorphic computer claims to be conscious, its emulation on a classical computer would functionally produce the same claim, leading to the “lying” paradox [00:44:13].

Alternatively, the “body sense of self” or interoception, originating from deep in the brain stem, is proposed by some (e.g., Antonio Damasio, Anil Seth) as the bootstrap for Consciousness in animals [00:45:50]. However, this too relies on electrochemical impulses encoding information, making it a form of information processing [00:46:21]. The key is the “loop” between intentions, actions, observations, and interoceptive feedback, through which the body and agency are discovered [00:46:45].

Potential trajectories of AI advancements

There are two main schools of thought regarding the path to AGI:

1. Scaling Hypothesis: Proponents believe that simply scaling up current approaches (e.g., Transformer models) by adding more data, compute, and tweaking loss functions will be sufficient to achieve AGI [01:03:07]. This view suggests that traditional objections about missing capabilities will “disappear” with more training [01:04:09]. Current models are already “superhuman” in their ability to ingest and correlate vast amounts of data, unlike humans [01:05:13].
2. New Architectures/Capabilities: Critics argue that current models, while impressive, lack fundamental components like world models, reasoning, or logic, and that new architectures are needed [01:03:09].

While current systems are “brutalist” and unlike human minds, it’s not obvious what they cannot do [01:05:01]. Objections like the lack of continuous real-time learning can potentially be overcome by using key-value storage for effects, periodically retraining the system on this data, and then clearing the storage [01:05:39]. Teaching systems to use external tools (like computer algebra systems) or discover them from first principles (like AlphaGo discovering how to play Go) could also extend their capabilities [01:06:42]. The ability for systems to learn from their own thoughts and perform experiments by coupling to reality are also seen as crucial for their growth as intelligent minds [01:07:20].

The Nature of Human Thought and Computational Models

Human creativity involves novelty, discontinuity in the search space (making a “jump into the darkness”), and a sense of authorship that evolves through continuous interaction and learning [00:54:43]. It’s proposed that an “AI artist” could be built that does not forget its creations or interactions, continuously integrating them to develop its own recognizable voice and identity [00:55:23].

Human cognition itself is described as a process of “confabulating” (generating candidate utterances or ideas) and then applying an analytical component to prune them down to useful ones [01:13:03]. This is analogous to how current language models generate many candidates that are then refined [01:13:03].

The concept of “extrapolation” versus “interpolation” in large language models is debated [00:51:55]. When a model generates something like an “ultrasound of a dragon egg” that combines elements never seen together in its training set, it blurs the line between these two processes [00:53:23]. This suggests that the models might operate in such high dimensionality that distinguishing interpolation from extrapolation becomes difficult for humans [00:53:03].

Beyond the Turing machine, a more general view of computation is the “rewrite system” [01:09:00]. This involves an environment where operators are applied wherever they match, simultaneously transforming the environment into a new state [01:09:04]. The Turing machine is a special, deterministic, and linear case of a rewrite system [01:09:17]. The universe itself can be seen as a non-deterministic Turing machine that branches out infinitely, with our subjective reality being a statistical projection of these outcomes [01:10:47]. The brain might also operate like a stochastic rewrite system, where neurons make probabilistic decisions, sampling from a superposition of possible states that eventually “collapse” into definite thoughts [01:11:56]. This “thinking like a quantum state” (though classically implemented) allows for exploration and relaxation of constraints [01:13:50].

Outlook and Future Research Directions

The timeline for AGI is uncertain, but it’s not considered “that far off” [01:14:37]. Thousands of smart people are exploring numerous avenues [01:14:48]. Mike Levin’s ideas on distributed self-organization in biological systems and carrying them over to computational systems are highlighted as particularly interesting [01:14:55].

It is suggested that the current understanding of neurons as simple switches may be incomplete [01:15:18]. Instead, a neuron might be seen as a “little animal” or single-celled organism with many degrees of freedom, actively learning and adapting its behavior based on its environment to ensure its survival within the larger organism [01:15:27]. This implies a complex, emergent self-organization within the brain, similar to how individuals cooperate in a society for a shared purpose [01:16:05].

A dream for future research includes establishing institutions like a “California Institute of Machine Consciousness” to directly research machine Consciousness [01:41:18]. The goal is to invite diverse thinkers across disciplines to foster a dialogue driven by long-term effects rather than fear or short-term economic impulses [01:56:30]. This research would involve testing the boundaries of existing models by combining them with reasoning components and grounding them in cybernetic agents [01:56:21].