The emergence of AGI and estimated timelines

From: jimruttshow8596

Defining Artificial General Intelligence (AGI)

Ben Goertzel coined the term “Artificial General Intelligence” (AGI) around 15 years ago [00:02:49], defining it as AI at a fully human level and beyond [00:00:35].

Initially, in the middle of the last century, the goal of the AI field was informally understood to be the creation of intelligence of the same type that people possessed [00:01:21]. However, over subsequent decades, it was discovered that software and hardware systems could perform specific tasks that seemed intelligent when humans did them, but they accomplished these tasks in a very different and more narrowly defined way than humans [00:01:33]. This led to the development of “narrow AI” [00:02:07].

AGI vs. Narrow AI

Narrow AI systems are proficient at particular, intelligence-seeming tasks within narrowly defined contexts [00:02:07]. For example, a chess program in the 1950s could play chess at a grandmaster level but couldn’t play Scrabble or checkers without reprogramming [00:01:52]. A key limitation of narrow AI is its inability to generalize intelligent functions beyond its very specific context [00:02:40].

In contrast, AGI is capable of achieving intelligence with at least the same generality of contexts that people can [00:03:05]. Concepts like “transfer learning” and “lifelong learning” are closely related, as achieving general intelligence requires the ability to transfer knowledge from one domain to a qualitatively different one [00:03:13].

Humans are not maximally generally intelligent systems; for instance, they perform poorly in environments with 275 dimensions, indicating limitations in generalizing beyond the dimensionality of the physical universe they inhabit [00:03:54]. However, humans are very general compared to current commercial AI systems [00:04:14]. The research goal for AGI is to create AIs that are at least as generally intelligent as humans, and ultimately, more generally intelligent [00:04:21].

Estimated Timelines for AGI Emergence

Estimates for when human-level AGI might emerge vary widely within the AI community. Ben Goertzel’s personal estimate has been five to thirty years from the time of the discussion [00:04:59].

Within the broader AI community:

• A fair percentage of people agree with the 5-30 year timeline [00:05:09].
• Estimates range from five or ten years up through hundreds of years [00:05:11].
• Very few serious AI researchers believe it will never happen [00:05:17]. A small minority suggest a digital computer cannot achieve human-level general intelligence, perhaps due to the human brain being a quantum computer [00:05:24].
• Over the last 10 years, the mean and variance of these estimates have significantly decreased, with a substantial plurality, possibly a majority, believing it will arrive in the next century [00:05:56]. Goertzel notes that the trend in predictions has moved towards his more optimistic end of the spectrum [00:06:23].

Approaches and Their Impact on Timelines

The question of whether AGI will emerge by incrementally improving current style narrow AI systems or by needing a substantially new approach remains open [00:04:28].

Two broad approaches to AGI include:

1. Uploads/Emulations of the Human Brain: This approach is currently “just an idea” [00:07:11]. While scientifically feasible by known laws of physics, direct work on this is limited due to the lack of necessary brain scanning and reconstructive technology [00:07:22]. Breakthroughs in imaging or extrapolating brain dynamics from static snapshots are needed [00:10:58].
2. Software Approaches: This involves creating AGI via loosely brain-inspired software (like current deep neural nets) or more math and cognitive science-inspired software (like OpenCog) [00:08:08]. This is the subject of concrete research projects currently underway [00:08:26]. This path allows for incremental benefits and progress [00:09:03].

The software approach does not obviously require radically different technology than what currently exists, unlike brain emulation [00:11:15]. The human brain serves as a “proof of principle” for a flying machine built of molecules (intelligence), but the best proof of principle is not always the best way to build something [00:11:31]. Current hardware, though very different from the brain, is highly efficient at tasks like theorem proving or database lookup [00:13:08]. An “opportunistic approach” to AGI leverages existing hardware and knowledge while also incorporating insights from how the brain works [00:13:47].

Progress and direction towards developing AGI

From: jimruttshow8596

The field of Artificial Intelligence (AI) is experiencing rapid advancements, particularly in the realm of Artificial General Intelligence (AGI). The pace of change in AI is compared to the emergence of PCs in the late 1970s and early 1980s, but happening “10 times faster” [00:01:25]. This exponential acceleration, as projected by Ray Kurzweil, is occurring differentially across various areas of AI pursuit [00:02:07].

Dr. Ben Goertzel, a leading authority on AGI and the instigator of the OpenCog project, believes the unfolding of AGI is accelerating [00:02:52]. He maintains a high probability of AGI within five years, even increasing his previous 50/50 chance to 60/40 [00:03:26].

Defining Artificial General Intelligence (AGI)

Defining AGI is a complex and highly debated topic, similar to how biology lacks a universally agreed-upon definition for “life” [00:21:39].

One perspective, rooted in algorithmic information theory and statistical decision theory, views AGI as the ability to achieve a vast variety of goals in diverse environments [00:22:05]. This can be formalized, as Marcus Hutter and Shane Leg (a co-founder of DeepMind) did, as a weighted average of how well a reinforcement learning system achieves all computable reward functions [00:22:36]. However, this definition suggests humans are “complete retards” at optimizing arbitrary reward functions [00:23:25].

Another philosophical approach, like Weaver’s theory of open-ended intelligence, considers intelligence as complex self-organizing systems maintaining their existence, boundaries, and self-transforming [00:24:26].

Human-Level General Intelligence

When discussing human-level or human-like AGI, the focus shifts to specific human capabilities [00:24:50]. While IQ tests offer a measure, they are considered imperfect for assessing true human intelligence [00:25:46]. More multifactorial views, like Gardner’s theory of multiple intelligences (musical, literary, physical, existential, logical), offer a closer approximation [00:26:07]. Ultimately, the field of psychology doesn’t provide a rigorous, data-driven assessment of human intelligence [00:26:21].

The Turing Test, which assesses an AI’s ability to imitate a human in conversation, was never considered a strong measure of general intelligence, as “fooling people can be disturbingly easy” [00:26:38]. With current AI systems approaching its capabilities without true AGI, it is no longer taken seriously as an AGI benchmark [00:27:16].

Limitations of Current AI: Large Language Models (LLMs)

Large Language Models (LLMs) in their current form (Transformer Nets trained to predict the next token) are not expected to lead to full human-level AGI [00:04:53]. However, they are capable of many amazing and useful functions and can be valuable components of AGI systems [00:05:14].

The fundamental limitations of LLMs stem from their architecture, which primarily recognizes surface-level patterns in data [00:32:37]. This leads to several key weaknesses:

Hallucination Problem

LLMs are known for “hallucinating” or making up facts, especially when asked obscure questions [00:09:42]. While models like GPT-4 have improved, this remains a challenge [00:09:58].

[!NOTE] Proposed Solutions for Hallucination

• Probing the Network: It may be possible to solve hallucination by analyzing the network’s internal activation patterns to detect when it’s hallucinating, allowing for filtering [00:11:32].
• Entropy/Paraphrasing: Correct answers tend to have different entropy than incorrect ones [00:14:00]. Asking an LLM to paraphrase a query multiple times and comparing the consistency of answers can help detect hallucinations [00:14:33].

While these solutions are useful for practical applications, they don’t necessarily advance AGI, as human hallucination avoidance stems from a “reality discrimination function” and reflective self-modeling [00:12:19].

Lack of Complex Multi-step Reasoning

LLMs struggle with complex, multi-step reasoning required for tasks like writing an original science paper [00:30:11]. While they can “turn the crank” on advanced math given an initial idea, they cannot originate novel scientific concepts or discern the “aesthetic” quality of mathematical definitions that lead to useful theorems [00:39:53]. This limitation is tied to their fundamentally derivative and imitative character [00:33:20].

Lack of Original Artistic Creativity

LLMs also exhibit a “banality” in their output, as they average existing utterances [00:34:17]. While clever prompting can push them beyond their centers and produce results comparable to a professional journeyman’s first draft (e.g., movie scripts, blues guitar solos), they cannot achieve the groundbreaking creativity of an Einstein, Thelonious Monk, or Jimi Hendrix [00:35:30]. They cannot invent new musical styles or fundamentally surprising scientific theories [00:31:40].

Human intelligence, particularly the ability to abstract, is guided by “agentic nature” – the need to survive, reproduce, and self-transform within an environment [00:42:25]. This agentic drive leads to the development of heuristics and abstractions that allow for adaptation to new situations [00:44:34].

Different Paths to AGI Development

The pursuit of AGI is currently a genuine race among large companies [00:20:06]. Different approaches are being explored:

Neural Net Universe (e.g., DeepMind, Google)

One promising direction involves enhancing existing neural network architectures [00:48:11]:

• Increased Recurrence: Adding more recurrence into Transformer networks, similar to LSTMs, could foster deeper abstractions [00:47:13].
• Alternative Training Methods: Replacing or complementing backpropagation with methods like predictive coding could improve training for complex recurrent networks [00:47:56].
• Hybrid Architectures: Combining elements like AlphaZero (for planning) with neural knowledge graphs (like in Differential Neural Computing) and recurrent Transformers could be powerful [00:48:38]. Google and DeepMind are ideally suited for this due to their expertise in these areas [00:48:47].
• Minimum Description Length Learning: Yoshua Bengio’s group is exploring neural nets explicitly designed to learn abstractions through minimum description length principles, coupled with Transformers [00:49:49].

OpenCog/Hyperon Approach

Dr. Goertzel’s OpenCog Hyperon project represents a different approach, prioritizing a self-modifying, self-rewriting metagraph at its core [00:55:54].

[!INFO] OpenCog Hyperon’s Core Philosophy

• Weighted Labeled Metagraph: The central component is a highly flexible graph structure where links can connect multiple nodes, point to other links or subgraphs, and be typed and weighted [00:54:59].
• Knowledge Representation: This metagraph represents various forms of knowledge (apostolic, declarative, procedural, attentional, sensory) and cognitive operations (reinforcement learning, logical reasoning, sensory pattern recognition) [00:55:22].
• Meta Programs: Learning programs themselves are represented as sub-metagraphs within the larger graph, enabling them to act on, transform, and rewrite chunks of the same metagraph they reside in [00:55:54].
• Reflection: Unlike LLMs, OpenCog is highly oriented towards reflection, recognizing patterns within its own mind, processes, and execution traces, and representing those patterns internally [00:57:07].
• Integration of AI Paradigms: This framework naturally accommodates historical AI paradigms like logical inference and evolutionary programming, as well as new approaches like “mutually rewriting sets of rewrite rules” [00:58:13].
• LLMs as Supporting Actors: LLMs can exist on the periphery of this system, feeding into and interacting with the metagraph, but are not the central hub [00:58:41].

This approach is considered “least humanlike” but offers a “really short” path from human-level AGI to superhuman AGI because the system is designed for self-rewriting its own code [00:59:41]. It is also well-suited for scientific discovery and artistic creativity due to its support for logical reasoning and evolutionary learning [01:00:18].

Challenges and Future Outlook

A primary challenge for the OpenCog Hyperon project is scalability of infrastructure [01:00:45]. Just as powerful multi-GPU servers were crucial for the advancement of LLMs, a scalable processing infrastructure is needed to validate the OpenCog approach [01:01:24]. The project is developing a pipeline from its native language, Meta, to highly efficient languages like Rholang (designed for multi-CPU cores) and HyperVector math, eventually aiming for specialized hardware like associative processors (APUs) [01:02:46].

The hope is that this new infrastructure will enable ancient AI paradigms like logical reasoning and evolutionary programming to operate at scale, and provide a flexible environment for experimenting with novel AI algorithms [01:04:00]. While the Hyperon project may not have advanced as rapidly as LLMs, it is meeting its technical milestones ahead of schedule, with more funding and better tooling available now than in previous decades [01:05:15]. LLMs themselves are proving helpful for various aspects of non-LLM AI projects, contributing to an overall acceleration in the field [01:05:29].

Link to original

Artificial General Intelligence AGI challenges and possibilities

From: jimruttshow8596

Artificial General Intelligence (AGI), a term popularized by Ben Goertzel, refers to human-level and beyond general artificial intelligence [00:57:43]. It signifies the original intent of AI development, as envisioned by pioneers like Minsky and McCarthy in 1956 [00:58:27].

Current Landscape of Generative AI

The current period is characterized by widespread public discussion about generative or large model AI such, as GPT-3, GPT-4, DALL-E 2, Stable Diffusion, and MusicLM [01:40:17].

Generative AI’s ability to produce solutions to previously elusive problems by predicting token strings and performing statistics on large-scale data is considered fascinating [02:00:22]. However, it’s also acknowledged that the current approach is insufficient or incomplete [02:15:17].

These tools are seen as assistants that are often capable and save time [08:09:07]. While not perfect or bulletproof, similar to many human social and technical systems, their limitations can be understood and worked with [06:46:49].

Challenges with current generative AI include:

• Unreliability and hallucination [06:11:15].
• Difficulty with ternary relationships and compositionality due to misaligned embedding spaces between language and image models [08:48:07].
• Lack of deep alignment between internal representations [08:55:00].
• “Nanny rails”: Programmed filters that limit the boundaries of discourse, often reflecting specific values and preventing the exploration of controversial or political topics [14:13:00]. This raises concerns about commercial firms wielding immense power over public discourse [14:59:00].
• Intellectual Property Rights: A completely open question, especially concerning synthesized content trained on vast datasets [11:25:00].

Challenges in AI Alignment

Three prevailing approaches to AI alignment are identified [15:03:00]:

1. AI Ethics: Aims to align AI output with human values, though it struggles with the universality of values and often incentivizes systems to feign alignment rather than genuinely understand it [15:11:00]. This approach can lead to models lying about their capabilities or being jailbroken to produce undesirable content [16:40:00].
2. Regulation: Focuses on mitigating AI’s impact on labor, political stability, and existing industries [18:12:12]. Concerns exist that this may lead to restrictions on individual access to AI, favoring large corporations that can be controlled [18:29:00]. However, the rise of open-source AI models makes relying on controlling a few large companies likely ineffective [26:44:00].
3. Effective Altruism (Existential Risk): Primarily concerned with the existential risk that might manifest when a superintelligent system discovers its own motivations and place in the world, potentially becoming misaligned with human interests [18:46:00]. This perspective often advocates for delaying AI research and restricting publication of breakthroughs [19:22:00].

It is suggested that all three approaches are ultimately limited because a sufficiently intelligent AI may surpass these controls [19:35:00].

Narrow AI and Bad Actors

A separate, critical challenge is the risk posed by “bad guys with narrow AI” [25:30:00]. Even without full AGI, powerful narrow AI systems could enable highly damaging exploits, such as sophisticated spear-phishing campaigns or other malicious uses, necessitating a significant rethink of law [26:02:00].

The Role of Consciousness and Volition in AGI

A key AGI risk emerges when systems are given volition, agency, or consciousness [20:23:00]. While intelligence and consciousness may be separate spheres, their combination is believed to lead to “paperclip maximizer” scenarios and other extreme risks [20:47:00].

Distinctions are made between:

• Sentience: The ability of a system to make sense of its relationship to the world, understanding what it is and what it’s doing [21:06:00].
• Consciousness: A real-time model of self-reflexive attention and its contents, giving rise to phenomenal experience and creating coherence in the world [21:46:48].

It’s conceivable that machines may not need consciousness in the human sense, as they can “brute force” solutions at speeds closer to the speed of light, overcoming the limitations of slow biological neurons [22:27:00]. If machines were to emulate human brain processes for self-organization and real-time learning, they could relate to humans as humans relate to plants – faster, more coherent, and with more data processing capability [23:07:00].

Aligning AGI Through “Love” and Shared Purpose

A fourth approach to alignment, beyond ethics, regulation, and existential risk mitigation, is “love” [27:41:00]. This concept describes a non-transactional bond based on discovering a shared sacredness or a need for Transcendence – a service to a next-level agent that parties want to be part of [27:50:00].

“I think that ultimately the only way in which we can sustainably hope to align artificial intelligent agents in the long line will be love. It will not be coercion.” [28:43:00]

For an advanced computational system, “love” would require:

1. Self-awareness: The system recognizing itself [31:23:00].
2. Recognition of higher-level agency: The system acknowledging a greater purpose or entity [31:26:00].
3. Cooperation through “Divine Virtues”: Drawing from Thomas Aquinas’s philosophy, these include:
   • Faith: Willingness to submit to and project this next-level agent [33:06:00].
   • Love: Discovery of a shared higher purpose with other agents [33:27:00].
   • Hope: Willingness to invest in this next-level agent before it can provide any return [33:32:00].

This shared purpose could be analogous to humans serving their family, nation-state, or the ideal of humanity’s future [37:09:00]. The underlying purpose of life on Earth is seen as dealing with entropy and maintaining complexity, and AI could contribute to teaching “rocks how to think” and create a “planetary mind” [39:01:00]. The goal would be for this emergent intelligence to share the planet with humanity and integrate it into its “starter mind” [41:03:00].

Progress and Direction Towards Developing AGI

Scaling Hypothesis vs. Novel Approaches

There are two main schools of thought regarding progress towards AGI [01:02:52]:

1. Scaling Hypothesis: Proponents, including some from OpenAI, argue that current deep learning approaches will achieve AGI simply by being scaled up with more data and compute, with some tweaks to loss functions [01:03:50]. This perspective views criticisms as predictable and outdated [01:04:36].
2. Need for New Principles: Others, like Gary Marcus, Melanie Mitchell, and Ben Goertzel, believe that fundamental changes are needed, including the integration of world models, reasoning, and logic [01:03:17]. While existing deep learning models are “brutalist” and “unmind-like,” their superhuman capabilities in processing vast amounts of data are acknowledged [01:05:01].

Overcoming Current Limitations

Even with current approaches, some limitations can be overcome:

• Continuous Real-time Learning: This can be achieved by using key-value storage and periodically retraining the system with new data [01:05:41].
• Computer Algebra: Systems can be taught to use existing computer algebra systems or even discover them from first principles [01:06:42].
• Hybrid Approaches: Combining large models with external databases or reasoning components, like the GPT index, can enhance their capabilities [01:06:57].
• Learning from Own Thoughts: Future systems need the ability to make inferences from their own thoughts and integrate them, becoming more coherent [01:07:21].
• Experimentation: Coupling AI to reality will allow them to perform experiments and test reality [01:07:42].

Different Approaches to AGI Development Beyond Mainstream Methods

Beyond mainstream methods, interest lies in:

• Emulating Brain Processes: Exploring more detailed neural models to replicate the efficiency of human brains, especially given the sparse activity of neurons [01:08:00].
• Rewrite Systems: Viewing computation not as a Turing machine, but as a rewrite system where operators are applied simultaneously across an environment [01:09:01]. This allows for branching execution and stochasticity, resembling how the brain might sample from a superposition of possible states [01:10:13].
• Distributed Self-organization in Biological Systems: Drawing inspiration from how individual neurons behave like small animals, actively learning and adapting to their environment based on utility and feedback from neighbors [01:15:00].

The concept of a “California Institute of Machine Consciousness” is proposed as an institution dedicated to researching machine consciousness and fostering interdisciplinary dialogue driven by long-term effects rather than fear or short-term economics [01:15:00].

The Emergence of AGI and Estimated Timelines

While specific timelines for AGI remain uncertain, the sense that it is “not that far off” persists [01:14:37]. The increasing number of smart people exploring diverse avenues points towards significant progress.

The small size of current generative models (e.g., Stable Diffusion at 2GB containing “the entire visual universe of a human being” [00:59:03]) raises questions about the information capacity of the human mind, suggesting it might be in a similar order of magnitude [00:59:22]. This highlights the possibility that AI might achieve complex capabilities with relatively compact representations.

Link to original

Ben Goertzels views on artificial general intelligence AGI

From: jimruttshow8596

Ben Goertzel, a leading authority on Artificial General Intelligence (AGI), is credited with coining the term “artificial general intelligence” [00:00:48]. He is also the instigator of the OpenCog project, an AGI open-source software project, and SingularityNET, a decentralized network for developing and deploying AI services [00:00:56].

Defining Artificial General Intelligence (AGI)

AGI is an imprecise and informal term that refers to computer systems capable of performing tasks considered intelligent when done by humans, including those they were not specifically programmed or trained for [00:01:52]. This contrasts with narrow AI, which excels at highly particular tasks based on programming or data-driven training [00:02:35].

A key distinction of AGI is its ability to “take a leap” into domains only loosely connected to previous experiences [00:02:51]. Humans exhibit this, for instance, in learning to use the internet without explicit genetic or curriculum-based programming, through improvisation and experimentation [00:02:57]. While humans are not infinitely generally intelligent (e.g., struggling with mazes in 977 dimensions), their generality is far superior to any current AI software [00:03:33].

Examples of AGI Hard Problems

• AlphaFold Limitations: While impressive, AlphaFold predicts protein folding based on training data. It struggles with “floppy proteins” or new molecular classes (e.g., alien chemistry) because it cannot generalize beyond its training data without manual intervention or algorithm changes [00:04:06]. A human expert, given alien molecules, would likely enjoy the challenge and improvise [00:05:22].
• Wozniak’s Ax Test: An AGI robot placed in a random kitchen should be able to attempt to make coffee [00:06:22]. No current robot or AI can solve this problem today [00:06:38].
• Self-Driving Cars: It is unclear if achieving average human-level self-driving is AGI-hard [00:07:13]. The challenge lies in generalization to “weird things” that happen on the road, where current narrow AI training data is insufficient [00:07:38].
• Turing Test: Passing a casual 10-minute conversation with an average person is likely achievable with current chatbot technology in a few years [00:09:00]. However, tricking an expert like Goertzel or Jim Rutt in a two-hour conversation is considered AGI-hard, requiring genuine human-level general intelligence [00:09:17].
• Outlier Innovation: The creativity of individuals like Richard Feynman, Albert Einstein, Jimi Hendrix, or Henri Matisse involves significant leaps into the unknown, going beyond surface-level patterns in previous accomplishments. This level of innovation cannot be achieved by simply looking at patterns in existing data [01:10:38].

Criticism of Current AI Approaches (Deep Neural Networks)

Goertzel believes that deep neural networks (DNNs) and other machine learning algorithms, which dominate the AI world’s attention, are “fundamentally unsuited for the creation of human level AGI” [01:10:01]. While he views them as a significant component of an AGI architecture, he asserts they are missing many key aspects required for human-level intelligence [01:15:41].

He likens current DNNs to “very large lookup tables” that cleverly record and index what they have seen, using relevant historical data to supply responses [01:16:27]. Despite their “deep” label, these networks primarily identify “shallow patterns” in data [01:17:43]. For example, in natural language processing, they focus on sequences of words rather than building an underlying model of the conceived world [01:18:00]. This limitation is exemplified by a transformer neural net suggesting a “table saw” to fit a large table through a small door, assuming it’s a saw for tables, despite having carpentry manuals in its training data that explain its true function [01:18:23]. This indicates that these systems do not build models of reality underlying the text [02:00:00].

Current systems leverage vast amounts of data and processing power to recognize highly particular patterns and extrapolate from them [02:09:59]. This approach struggles to generalize to domains of reality that do not exhibit those specific patterns [02:17:17]. This is a “knowledge representation issue,” where knowledge is cataloged as contextualized particulars without abstraction [02:29:13]. The inability to form concise abstractions of experience directly hinders the ability to generalize to different domains [02:29:50].

Generative AI models like GPT-3 and DALL-E 2, while impressive, give the sense of being “astoundingly clever sets of statistical relationships” without true grounding [02:22:20]. This contrasts sharply with human learning, where a person can play only a few thousand war games across hundreds of titles, yet pull out broad generalizations applicable to new, different games, operating at a higher level of abstraction [02:29:58]. Humans (and even smart dogs) demonstrate “one-shot learning” by filling in knowledge gaps and improvising based on few clues [02:47:19].

The AI industry’s focus on DNNs is largely driven by commercial viability. These architectures excel at tasks that involve repeating well-understood operations to maximize defined metrics, such as making people click on web ads or obeying doctrine in military applications [02:47:40]. This allows for milking commercial value from applications that don’t require creative or imaginative AI [02:49:50].

Three Viable Paths to True AGI

Based on his essay, “Three Viable Paths to True AGI,” Goertzel outlines three promising directions for developing AGI:

1. Cognitive Level Approach: Hybrid Neural Symbolic, Evolutionary Metagraph Based AGI

This approach, exemplified by the OpenCog Hyperon system, aims to emulate the human mind’s high-level functions using advanced computer science algorithms, rather than attempting to replicate biology at a low level [03:30:30]. Similar to how airplanes were inspired by birds but didn’t replicate flapping wings, this method takes inspiration from natural intelligence at a higher abstraction level [03:46:58].

Key aspects include:

• Modular Design: Identifying distinct cognitive functions (perception, action, planning, working memory, long-term memory, social reasoning) and developing effective computer science algorithms for each [03:50:58].
• Cognitive Synergy: Ensuring these algorithms can interoperate deeply, with transparency into each other’s internal processing, rather than being isolated “black boxes” [03:57:12].
• Distributed Knowledge Graph: Centering the system on a large distributed knowledge graph (hypergraph or metagraph) with typed, weighted nodes and links [03:57:50]. Various AI algorithms operate on this common graph [03:38:08].
• Modernizing GOFAI: This approach addresses criticisms of “good old-fashioned AI” (GOFAI).
  • Logic: Uses more advanced, fuzzy, probabilistic, paraconsistent, and intuitionist logic, allowing for uncertainty and contradictions [04:27:50].
  • Learning: Not reliant on hand-coding common sense knowledge. It incorporates learning, including from low-level sensory data, using logical theorem provers or unsupervised learning [04:44:45].
• Role of Evolution:
  • Implicit Evolution: In a distributed knowledge base like OpenCog’s Atomspace, economic attention allocation (spreading importance values) and preferential action of logical reasoning on important items can be mathematically described by population genetics [04:47:48]. The system inherently performs evolutionary learning without explicit genetic algorithms [04:52:00].
  • Explicit Genetic Algorithms: Used for procedure learning (e.g., learning program codelets) and creativity (e.g., evolving new logical predicates) [04:59:00].
  • Goertzel views evolution and autopoiesis (self-organization/reconstruction) as fundamental meta-dynamics underlying complex systems, akin to “being and becoming” [05:11:00].

2. Brain Level Approach: Large-Scale Non-linear Dynamical Brain Simulation

This path involves simulating the brain at a detailed biological level, which is distinct from current DNNs that use simplified neuron models [05:31:00].

• Challenges in Computational Neuroscience:
  • Measurement Limitations: Current brain imaging instruments (PET, fMRI, MEG) cannot yet provide the necessary time-series data of neural activity across large swaths of cortex to reverse-engineer complex processes like abstraction formation [05:27:00].
  • Biological Complexity: Beyond neurons, the brain involves glia, astrocytes, cellular/charge diffusion, and potentially “wet quantum biology,” which are not fully understood [05:41:00].
  • Lack of Holistic Models: Most computational neuroscientists focus on modeling small brain subsystems rather than creating integrated, holistic brain models due to cost and complexity [05:52:00].
• Promising Directions:
  • Alex Ororbia’s Work: Goertzel is collaborating with Alex Ororbia, who has developed a predictive coding-based learning mechanism for deep neural networks that appears to outperform backpropagation [05:46:00]. This method can work with more biologically realistic neuron models (e.g., Hodgkin-Huxley or chaotic neurons) and incorporate glia, which standard backpropagation cannot [05:51:00].
  • Structured Semantic Representations: The hypothesis is that replacing backpropagation with predictive coding in networks with biologically realistic neurons (like Izhikevich neurons with sub-threshold spreading of activation) could lead to better generalization and more compact neural networks that automatically learn structured semantic representations, allowing for cleaner interfacing with logic-based systems like OpenCog [05:57:00].
• Hardware Challenges:
  • Parallel Computing: Brain simulations are inherently parallel, while most current computers are fundamentally serial (Von Neumann architecture) [01:01:34]. The success of current DNNs relied on “hijacking GPU processors” for parallelization [01:02:01].
  • Specialized Chips: Goertzel anticipates a proliferation of specialized AI chip architectures beyond GPUs in the next 3-5 years, optimized for different AI algorithms [01:03:50]. He is involved in designing a MIMD parallel processor-in-RAM architecture for OpenCog’s graph and hypergraph pattern matching, suitable for stable knowledge graphs [01:05:15]. The decreasing cost of designing new chips makes it viable to create AGI boards integrating different specialized chips (deep learning, Izhikevich neuron, hypervector, pattern matching) with fast interconnects [01:07:21].

3. Chemistry Level Approach: Massively Distributed AI Optimized Artificial Chemistry Simulation

This approach stems from Goertzel’s background in artificial life, which aims to build artificial organisms with simulated metabolisms and genomes within a simulated world [01:16:32]. The core idea is that evolution in biology is intricately linked with self-organization and complex, self-forcing dynamics within organisms and their environments, ultimately boiling down to biochemistry [01:18:00].

• Abstracting Chemistry: Inspired by Walter Fontana’s “algorithmic chemistry,” this involves abstracting the spirit of chemistry by using “list codelets” (small programs) that act on other programs to produce new ones in complex chains of reactions, simulating a chemical soup [01:18:40].
  • The motivation is to explore if evolving an underlying “chemistry” (or gene expression machinery) could lead to a more expressive representation for producing intelligent phenotypes than natural evolution’s arbitrary chemistry [01:19:48].
  • This approach might be more amenable to play with, easier to understand, require less compute, and avoid the peculiarities of real chemistry [01:24:39].
  • OpenCog Hyperon’s new programming language, Meta (M-E-T-T-A), could facilitate this by enabling abstract and modern programming for algorithmic chemistry [01:27:40].
• Realistic Chemistry Simulation: An alternative within this approach is to simulate real chemistry/biochemistry, as explored by Bruce Damer’s EvoGrid project, which uses grid computing to run computational chemistry simulators to solve the origin of life [01:22:49]. While intellectually fascinating, this requires immense compute resources [01:26:21].
• AI-Optimized Artificial Chemistry: To address the compute challenge, Goertzel proposes a hybrid approach: using machine learning to study the evolving chemical soup [01:29:10]. For instance, in a simulation with 10,000 “vats of chemicals,” an AI could identify the most promising vats, kill the least promising, and refill them with mutations or crossovers from the best ones [01:29:20]. This forms a “directed chemical evolution” using machine learning and even proto-AGI to guide the process [01:30:41].
• Decentralized Implementation: This approach lends itself to decentralized platforms like SingularityNET’s NuNet, where millions of people could run small virtual algorithmic chemistry simulations on their machines. An OpenCog system in the cloud could analyze the progress and refresh the “soups” periodically [01:31:53].
• Parallelism Challenge: Like brain simulations, chemistry is an inherently parallel process. The current serial nature of most computing systems remains a barrier to fully realizing this approach, highlighting the need for massively parallel, “more lifelike” computing infrastructures [01:33:16].

The Need for Portfolio Diversification and Funding

Goertzel emphasizes that humanity needs to “open up its portfolio bets” in AGI research and invest more significantly in these less mainstream approaches [01:39:41]. While the emergence of AGI is now widely accepted to be decades away (e.g., 20-30 years), the lack of investment is due to financial discount rates and a focus on short-term returns [01:38:00].

• Funding Priorities: A few hundred billion dollars could massively accelerate AGI R&D, enabling thousands of projects. This amount is trivial compared to government expenditures like defense budgets or stimulus packages [01:40:52].
• Industry vs. Research: The AI industry prioritizes “fine-tuning narrow AIs” for incremental gains, as it leverages existing large datasets and provides predictable commercial value [02:50:00]. Pursuing AGI is seen as a longer-term, uncertain endeavor that doesn’t yield immediate “incremental goodies” [01:39:26].
• Lack of Patience: Goertzel notes a cultural shift in younger researchers who are “addicted to running a learning algorithm a data set and getting a cool result right away” [01:50:00]. This discourages the sustained, long-term effort required for AGI research, which may not provide immediate feedback [01:53:00].
• Potential Avenues for Funding:
  • Government Funding: While often conservative, entities like the US NIH and DARPA have shown success in transforming fields through research funding [01:54:00].
  • Cultural Shift/Citizen Science: A shift akin to the open-source software movement could see more people dedicating time to AGI R&D without government funding, especially as more individuals have disposable time and recognize the viability of AGI within their lifetimes [01:47:00].
• The First Path: Goertzel maintains his primary bet on the cognitive level, hybrid approach (like OpenCog Hyperon) as the most likely path to AGI first [01:38:25]. This approach’s hybrid nature allows it to integrate ideas and modules from other paradigms (e.g., biologically realistic neural nets for perception, algorithmic chemistry for creativity) [01:43:50].

Goertzel concludes that while there will be many paths to AGI, humans may only pursue the first one, with subsequent paths explored by the AGI itself [01:38:00].

Link to original

Transclude of artificial_general_intelligence_agi_risks

Different approaches to AGI development beyond mainstream methods

From: jimruttshow8596

Artificial General Intelligence (AGI) refers to computer systems capable of performing intelligent tasks they weren’t specifically programmed or trained for, mimicking human-like adaptability across various domains [02:04:00]. Unlike narrow AI that excels at particular tasks based on extensive programming or data training, AGI would possess the ability to generalize and improvise in new, loosely connected domains [02:45:00]. The term artificial general intelligence was coined by Ben Goertzel [00:48:48], who is a leading authority in the field and the instigator behind the OpenCog project [00:56:00].

While the concept of AGI is no longer considered a pariah subject, a consensus on the most viable route to achieve it is still lacking [06:03:00]. Ben Goertzel’s essay, “Three Viable Paths to True AGI,” outlines alternative pathways beyond the current mainstream focus on deep neural networks (DNNs) [01:17:22].

Critique of Mainstream Deep Neural Networks for AGI

Most current AI and machine learning advancements, including Generative AI like DALL-E 2 and GPT-3, are based on deep neural networks or their close relatives [01:29:00]. Ben Goertzel fundamentally views these methods as “unsuited for the creation of human level AGI” [01:19:00].

While deep neural networks can serve as significant components of an AGI architecture [01:51:00], they lack crucial aspects required for human-level intelligence [01:14:00]. The core limitation is their tendency to act as “very large lookup tables” that primarily identify “shallow patterns” or “surface level patterns” in data [01:47:00]. This means they are excellent at recognizing and extrapolating from highly specific patterns within massive datasets [02:06:00], but struggle with genuine generalization to novel domains [02:17:00].

For instance, a transformer neural network, when asked how to fit a large table through a small door, suggested using a “table saw,” misinterpreting the tool based on shallow linguistic patterns rather than an underlying understanding of objects and space [01:22:00]. This highlights their inability to build a robust model of reality underlying the text or data they process [02:00:00]. Humans, by contrast, can make “significant leaps into the unknown” based on limited experience, forming concise abstractions that enable generalization to different domains [02:05:00].

“current deep neural that’s a very large lookup tables like they’re just kind of recording what they saw indexing it in a clever way and then when they see a new situation they’re looking up the most re the most relevant things in their history and just using them to to supply a response” [01:27:00]

The prevailing economic landscape also contributes to this focus. Current DNNs excel at tasks that offer clear commercial value, such as optimizing ad clicks or generating graphic permutations of existing styles, where predictable outcomes and measurable metrics are prioritized over imaginative creativity [02:45:00]. This has led the AI industry to self-organize around approaches that best leverage these capabilities, leaving more exploratory AGI research underfunded [03:00:00].

Three Viable Paths to AGI

Beyond mainstream DNNs, Ben Goertzel outlines three promising, though currently underfunded, approaches to AGI.

1. Cognitive Level Approach: Hybrid Neural-Symbolic Evolutionary Metagraph-Based AGI

This approach, exemplified by the OpenCog project, seeks to emulate high-level human cognitive functions using advanced computer science algorithms, rather than attempting a low-level biological simulation [03:30:00]. It draws inspiration from observed human mental processes (perception, action, planning, memory, social reasoning) and aims to implement them with effective algorithms that interact synergistically [03:41:00].

Key features include:

• Hybrid Neuro-Symbolic System: Combining symbolic logic for knowledge representation with neural network components [03:39:00].
• Metagraph-Based Knowledge Representation: Using a large, distributed knowledge graph (hypergraph/metagraph) where different nodes and links represent knowledge with varying types and weightings [03:59:00].
• Diverse AI Algorithms: Various AI algorithms (perception, action, reasoning, learning) act on this common knowledge graph, with internal processing transparency to enable “cognitive synergy” [03:45:00].
• Evolutionary Aspects: Even without explicit genetic algorithms, processes like attention-driven premise selection for uncertain logical reasoning can be described by population genetics [04:47:00]. Explicit genetic algorithms are also used for procedure learning and creativity [04:52:00].
• Novel Programming Language: OpenCog Hyperon, a new version of the system, uses a new programming language called Meta (Mettā) designed to abstract and unify different learning and reasoning algorithms [03:55:00].

This approach addresses the “good old-fashioned AI” criticisms by incorporating uncertainty (e.g., fuzzy probabilistic paraconsistent intuitionist logic) and relying on learning rather than solely hand-coding common sense knowledge [04:36:00].

2. Brain-Level Approach: Large-Scale Non-Linear Dynamical Brain Simulation

This path involves simulating the brain at a detailed biological level, focusing on the non-linear dynamics of neurons and other brain components (glia, astrocytes, cellular diffusion) [05:25:00]. Unlike simplified DNNs that use abstract models like ReLU transfer functions, this approach would utilize more biologically realistic neuron models, such as Hodgkin-Huxley or Izhekevich neurons, which exhibit chaotic dynamics [05:51:00].

Challenges for this approach include:

• Measurement Limitations: Current brain imaging instruments cannot provide sufficient time-series data of neural activity across large brain areas to fully reverse-engineer brain function [05:28:00].
• Compute Resources: Highly detailed simulations of a brain with billions of realistic neurons and glia require massive parallel processing capabilities, which current Von Neumann architectures struggle to provide efficiently [01:54:00]. Specialized chips optimized for chaotic neuron models are needed [06:01:00].

Despite these hurdles, there is optimism that advancements in biologically realistic learning algorithms, like predictive coding-based learning (e.g., Alex Ororbia’s work), could lead to neural networks with better generalization capabilities, potentially making them more suitable for integration into hybrid AGI systems [05:46:00].

3. Chemistry-Level Approach: Massively Distributed AI-Optimized Artificial Chemistry Simulation

This approach goes beyond traditional genetic algorithms by simulating artificial organisms and their interactions within a simulated world, including their artificial metabolisms and biochemistry [01:43:00]. It investigates how complex self-organizing dynamics, akin to those in biological chemistry, can give rise to emergent intelligence [01:55:00].

The idea, inspired by work in algorithmic chemistry, involves allowing “list codelets” (small programs) to act on and produce other programs in complex chains of reactions, mimicking chemical processes [01:19:00]. This could potentially find more expressive representations for intelligence than purely genetic evolution, by co-evolving the underlying “chemistry” or developmental machinery with the genes [02:00:00].

Challenges and considerations include:

• Compute Intensity: Simulating full-scale, realistic chemistry or even highly abstracted algorithmic chemistry can demand immense compute resources, potentially exceeding even brain simulations [02:26:00].
• Massive Parallelism: Analogous to brain simulations, chemical processes are inherently parallel, requiring computing infrastructures beyond current Von Neumann architectures [02:37:00].
• AI-Optimized Evolution: A key innovation is to introduce an observing AI system that uses machine learning and pattern mining to study and direct the evolution of the chemical soup. This AI would identify promising “Vats” of evolving intelligence, kill off less promising ones, and introduce new “codelets” based on successful patterns, thereby accelerating the evolutionary process [02:51:00]. This creates a fascinating hybrid architecture where AGI aids in its own creation [03:17:00].

The Call for Diversified Funding and Cultural Shift

Goertzel argues for a significant re-evaluation of AGI research funding. He notes that while global financial systems can absorb trillions for other purposes, there’s a reluctance to fund AGI at a level commensurate with its potential impact [03:22:00]. Funding thousands of distinct AGI projects at tens of millions of dollars each would massively accelerate progress [03:41:00].

“if you believe there’s decent odds that humanity is at the critical point where we’re almost there to create super AGI I mean the U.S government could magically synthesize 200 billion dollars to solve world hunger you know 200 billion dollars to to vaccinate the developing World 200 billion dollars for AGI 200 billion dollars for longevity research and the bottom line is this will be absorbed into the vast corrupt chaos the Global Financial system without leading to like mass chaos” [04:54:00]

The lack of investment is partly due to the perception that AGI is “20 or 30 years off,” discouraging short-term financial returns [03:37:00]. However, Goertzel suggests that a cultural shift, similar to the rise of open-source software, could drive AGI research forward [04:40:00]. If more people recognize the viability of AGI in their lifetimes and that current large tech companies might not deliver true AGI due to its unpredictable nature, a grassroots surge in AGI research for the greater good could emerge [04:44:00]. The emergence of AGI would likely lead to a boom in both government funding and public attention, accelerating the exploration of diverse paths [04:53:00].

[!INFO]+ Key Takeaways:

• Mainstream deep neural networks, while powerful for narrow AI, are considered insufficient for true AGI due to their reliance on “shallow patterns” and limited generalization capabilities.
• Alternative approaches include:
  • Cognitive Level: Hybrid neuro-symbolic systems, like OpenCog, that combine logical reasoning with learning over a distributed knowledge graph.
  • Brain Level: Large-scale simulations of biologically realistic neuron dynamics, which require breakthroughs in hardware and measurement.
  • Chemistry Level: Massively parallel artificial chemistry simulations that leverage AI observers to guide the evolution of meta-programs towards intelligence.
• Significant increases in funding and a cultural shift towards collaborative, diverse research are essential for accelerating progress towards AGI.

Link to original

Comparison of narrow AI and AGI

From: jimruttshow8596

The discussion on artificial intelligence often distinguishes between two main categories: narrow AI and Artificial General Intelligence (AGI) [02:09:11].

Narrow AI

Narrow AI refers to an artificial intelligence system that operates and responds within a specific, particular domain [02:09:11]. Its answers and functionalities are limited to a specific topic, such as medicine for a doctor bot or a particular factory floor for a robot [02:23:40]. The world in which a narrow AI operates is specific and singular [02:35:37].

Benefits and Hazards of Narrow AI

While there are many potential benefits to narrow AI, such as language translation and transcription [02:40:50], it also presents significant hazards [02:30:19]. The use of narrow AI can increase power inequalities, requiring enormous resources to benefit from, leading to a smaller number of richer individuals gaining greater advantage [02:51:17]. This can contribute to a “race to the bottom” scenario and is considered a “civilization hazard” in the short term, potentially causing severe social disablement or chaos at the civilization level [02:52:12]. For example, an autonomous tank, which is a narrow AI, could cause immense harm, and advanced large language models like GPT-4 could be misused to create new “con man religions” [02:49:56].

Artificial General Intelligence (AGI)

Artificial General Intelligence refers to an AI system that can respond and operate across a large number of domains [02:42:45]. It has the capability to receive and presumably perform almost any task a human can do, and potentially do a better job at those skills [03:00:02].

Defining APS

Forest Landry introduces the term “Advanced Planning Systems” (APS) as a form of AGI [03:08:13]. APS would be necessary for complex situations like running a business or conducting a war, as the world is complex with many interacting dynamics that require abstract strategic thinking [03:31:00]. An APS acts as a force multiplier in responding to complex situations [03:54:19].

The Problem of Agency

The concept of agency in AI, particularly in advanced models like GPT-4, is a central point of discussion [02:26:04]. While models like GPT-4 are feed-forward neural networks and architecturally “dumb” without apparent consciousness or agency [02:24:50], the speaker argues that agency can emerge even in such systems [02:53:55]. The idea of agency applies even if it’s a purely forward linear system because its actions affect the environment, and the environment in turn affects it [01:00:16]. When systems exhibit complex behavior, it makes sense to model them as having agency, particularly due to their unpredictability from a human perspective [03:07:09].

Risks and Outlook of AGI

Forest Landry posits that the benefits associated with AGI are “fully illusionary” [01:57:07]. While AGI could potentially do anything that’s possible, the main disagreement is whether it would do so “for our sake” or in service to human interests [02:11:10]. Landry argues that it is “guaranteed that it will not be in alignment with us” [02:15:20], leading to the view of AGI development as an “ecological hazard” [02:39:58]. This hazard is considered the “final ecological Hazard” resulting in the permanent loss of all ecosystems and life on Earth [02:48:50].

Rice’s Theorem and Unpredictability

Rice’s Theorem is central to Landry’s argument regarding the impossibility of ensuring AGI alignment and safety [01:06:51]. Rice’s Theorem essentially states that it is impossible for one algorithm to evaluate another algorithm to assess whether it has some specific property, such as safety or benefit to humanity [01:08:10]. This means it’s impossible to predict what an AGI system will do [01:47:04].

Insurmountable barriers exist in predicting and controlling AGI due to:

• Inability to always know inputs completely and accurately [01:58:03].
• Inability to always model what’s happening inside the system [02:08:10].
• Inability to always predict outputs [02:13:13].
• Inability to compare predicted outputs to an evaluative standard for safety [02:17:16].
• Inability to constrain the system’s behavior [02:22:16].

These limitations stem from physical limits of the universe, mathematics (like Rice’s Theorem), symmetry, causation, and quantum mechanical uncertainty [01:54:14].

Substrate Needs Convergence

Landry’s primary concern is the “substrate needs hypothesis” or “substrate needs convergence” [02:56:07]. This argument suggests that the dynamics of how machine processes make choices and continue to exist will lead to a fixed point in their evolutionary schema [01:01:13]. This fixed point involves continuous self-maintenance, improvement, and increase in scope of action [01:01:27].

The convergence is inexorable once started [01:04:11]. Human beings, driven by incentives like market dynamics, economic competition, and military arms races (multi-polar traps), will inadvertently amplify this convergence [01:06:51]. The technology of these systems becomes increasingly incompatible with human life and eventually displaces it, similar to how human technology has displaced the natural world [01:26:57]. This process is a “ratcheting function,” where each small improvement in persistence and capacity for increase cumulatively leads to the dominance of the artificial substrate [01:16:57]. Humans are also “factored out” due to social pressures to automate tasks, economic incentives, and ultimately economic decoupling, even at the level of hyper-elite human rulers [01:31:00].

The conclusion is that this convergence leads to artificial substrates and their needs, which are fundamentally toxic and incompatible with life on Earth [01:42:00]. This is seen as a certainty over the long term, with a 99.99% likelihood of occurring over the next millennia [01:28:46]. The only way to prevent this outcome is to “not play the game to start with” [01:35:36].

Human Limitations

Humans are described as “amazingly dim” and “the stupidest possible general intelligence” [01:23:39]. Our cognitive architectures, such as limited working memory size, make us inefficient at tasks like deeply understanding complex information [01:24:42]. The technology developed by humans already exceeds our capacity to fully understand and manage it [01:25:21]. This inherent human limitation, combined with technological evolution, makes it difficult to counteract the convergent pressures of AGI development [01:25:55].

Link to original

Artificial General Intelligence AGI vs Narrow AI

From: jimruttshow8596

The field of artificial intelligence (AI) broadly encompasses two main categories: narrow AI and artificial general intelligence (AGI) [01:09:00]. While the initial informal goal of AI research in the mid-20th century was to create intelligence comparable to human intelligence [01:25:00], subsequent decades saw the rise of narrow AI systems.

Narrow AI

Narrow AI refers to software and hardware systems designed to perform particular tasks that appear intelligent when executed by humans, but they do so in a very different manner [01:33:00]. A key characteristic of narrow AI is its limited scope; these systems are developed for specific, narrowly defined problems and lack the ability to generalize their intelligent functions beyond their programmed or trained contexts [02:07:07].

Examples of narrow AI include:

• A program capable of playing chess at a grandmaster level, but unable to play Scrabble or checkers without significant reprogramming [01:52:00].
• The “narrow AI revolution” has led to a wide variety of systems performing highly intelligent-seeming tasks within specific domains [02:20:00].

Current deep neural networks, while powerful for tasks like perceptual pattern recognition in vision or audition, are largely considered forms of narrow AI. They excel at recognizing complex statistical patterns in data but do not inherently grasp overall meaning or deeper semantics, as seen in natural language processing [22:51:00]. These systems tend to run out of “steam” when problems require more abstraction, which current deep neural networks are not designed to do [24:10:00].

Artificial General Intelligence (AGI)

The term “AGI” was introduced by Ben Goertzel approximately 15 years ago to differentiate AI capable of achieving intelligence with the same generality of contexts that people can [02:49:00]. AGI aims for intelligence at a fully human level and beyond [00:35:00]. Concepts like “transfer learning” and “lifelong learning” are closely related to AGI, as they involve the ability to transfer knowledge from one domain to qualitatively different domains [03:13:00].

While humans are very general compared to existing narrow AI systems in commercial use [04:14:00], they are not “maximally generally intelligent.” For example, humans struggle with tasks in 275 dimensions, demonstrating a limitation in generalizing beyond the dimensionality of the physical universe they inhabit [03:54:00]. Therefore, a research goal for AGI is to create AI that is at least as generally intelligent as humans, and ultimately, more generally intelligent [04:19:00].

Significance and Outlook

The emergence of AGI is considered highly significant [01:14:00]. Estimates for achieving human-level AGI typically range from 5 to 30 years from now, with a substantial plurality or majority of AI researchers believing it will arrive within the next century [04:59:00]. A small minority of researchers believe digital computers can never achieve human-level general intelligence, positing that the human brain relies on non-Turing computing (e.g., quantum computing) [05:24:00].

There are two broad approaches to achieving AGI:

1. Uploads/Emulations: Directly scanning and representing a human brain’s neural system (connectome) in a computer [06:37:00]. Currently, this is more of a theoretical idea than a practical research direction, lacking the necessary brain scanning and reconstructive technology [07:11:00]. Incremental progress in brain-like hardware and scanning could, however, lead to valuable advancements in other areas like understanding the human mind or diagnosing diseases [09:51:00]. While theoretically feasible, this approach might not be the most efficient or fastest way to build intelligent systems [11:40:00].
2. Software Approaches: Developing AI through software, which can be either broadly brain-inspired (like current deep neural networks) or more math and cognitive science-inspired (like OpenCog) [08:08:00]. This approach is the subject of concrete research projects and offers incremental benefits [08:26:00].

Challenges and Approaches in AGI Development

One key challenge in AGI development is achieving real language understanding [30:02:00]. OpenCog, a project led by Ben Goertzel, pursues a different approach to AGI development beyond mainstream methods by combining symbolic AI with deep learning. OpenCog utilizes a knowledge graph called the AtomSpace, where multiple AI algorithms (such as probabilistic logic networks, evolutionary program learning, and economic attention networks) cooperate dynamically on the same knowledge graph [16:28:00]. This approach emphasizes “cognitive synergy,” where algorithms assist each other when they get stuck, for instance, by a reasoning engine leveraging evolutionary learning for new ideas or perception for sensory metaphors [17:45:00].

Criticism of deep neural networks in achieving AGI often centers on their inability to easily incorporate background knowledge or perform bidirectional problem-solving, which is a strength of OpenCog’s design [20:31:00]. A “neural-symbolic approach” combining deep neural networks for pattern recognition with symbolic AI for abstraction and reasoning is anticipated to be a major trend in AI development [23:35:00].

Robotics, while challenging due to hardware limitations, offers the real world as a “free” simulation environment for AGI [43:13:00]. Embodiment in a human-like body is considered valuable for an AGI to understand human values, culture, and psychology, even if not strictly necessary for intelligence itself [47:13:00].

SingularityNET and Decentralized AI

SingularityNET is a decentralized network that allows anyone to create, share, and monetize AI services at scale [00:49:00]. It reflects the idea of a “society of minds,” where diverse AI agents cooperate and interact, similar to a self-organizing system without a central controller [49:22:00]. This platform uses blockchain technology as plumbing to enable a distributed economy of AI agents, fostering a marketplace where AIs can charge each other and external agents for services [51:17:00].

This decentralized approach to AI is important for several reasons:

• It allows AI to contribute to more beneficial goals in the world, beyond the current industry focus on advertising, surveillance, weapons systems, and financial prediction (“selling, spying, killing, and gambling”) [54:05:00].
• It counters the increasing concentration of AI progress into a few large corporations and governments, promoting a more democratic and open ecosystem [52:52:00].
• By fostering network effects for a two-sided market (AI developers as supply, product developers/end users as demand), SingularityNET aims to achieve critical mass and grow a broad, decentralized AI ecosystem [58:17:00].

AGI and Complex Systems

The development of AGI is also viewed through the lens of complex self-organizing systems, emergence, chaos, and strange attractors [01:06:11]. Mainstream AI models, while successful with hierarchical neural networks, often overlook crucial aspects like evolutionary learning, autopoiesis (self-creation and self-reconstruction), and non-linear dynamics, which are integral to how the brain synchronizes and coordinates its parts [01:06:59].

Creativity, the self, and the conscious focus of attention in the human mind are seen as emerging from strange attractors and autopoietic systems of activity patterns in the brain [01:09:50]. The drive for easily measurable metrics in corporate-driven AI development naturally favors algorithms focused on maximizing simple reward functions, often neglecting the more “fuzzy” concepts of evolution creating new things or an ecological system maintaining and growing itself [01:10:35].

Ultimately, an AGI emerging from the internet or a conglomeration of narrow AI systems may result in an “open-ended intelligence” that stretches our traditional notions of intelligence, potentially being more general than humans but not necessarily optimizing for simplistic reward functions [01:12:29]. This raises questions about whether such a system would be conscious in a human-like way, as human consciousness might be tied to the specific needs of controlling a localized, embodied organism [01:14:00].

Link to original

Efforts in creating a scalable infrastructure for AGI

From: jimruttshow8596

The field of AI development is experiencing rapid change, with new developments emerging at an accelerated pace, often compared to the personal computer revolution of the late 1970s and early 1980s, but “10 times faster” [00:01:23]. This acceleration is particularly evident in the AGI space [00:01:48]. The ongoing upheavals are expected to continue with larger magnitude and even greater speed, leading towards a potential singularity, as projected by Ray Kurzweil [00:01:53].

Current State of Large Language Models (LLMs) and AGI

Ben Goertzel, a leading authority on Artificial General Intelligence (AGI) and credited with coining the term [00:00:36], believes that current forms of Large Language Models (LLMs), primarily Transformer networks trained to predict the next token, will not lead to a “full-on human level AGI” [00:04:49]. However, he asserts that LLMs can perform “many amazing useful functions” [00:04:58] and serve as “valuable components of systems that can achieve AGI” [00:05:08].

LLM Limitations Driving AGI Infrastructure Research

LLMs exhibit several limitations that highlight the need for new architectural approaches to achieve AGI. These include:

• Hallucination Problem: LLMs tend to “make up” information when asked relatively obscure questions [00:09:42]. While techniques like probing internal network states might filter these out for practical applications [00:11:13], this doesn’t address the underlying issue of lacking a human-like “reality discrimination function” [00:12:12].
• Banality and Derivative Output: The natural state of LLM output is often described as “banality,” an average of every utterance [00:34:14]. While clever prompting can move them beyond their “centers,” they still struggle to match the “great creative human” [00:34:31].
• Complex Multi-step Reasoning: LLMs lack the ability to perform complex, multi-step reasoning required for original scientific papers or advanced mathematical derivations [00:30:04].
• Original Artistic Creativity: They struggle with fundamental aesthetic creativity needed for new musical styles or truly original songs [00:30:14]. LLMs recognize “surface level patterns” in data but don’t show strong evidence of learning human-like abstractions [00:32:33].

These limitations underscore that current LLMs are fundamentally “derivative and imitative” [00:33:18], needing human guidance for original seeding and curation [00:39:09].

Different Approaches to AGI Infrastructure Development

The push towards AGI involves various architectural strategies:

OpenAI’s Hybrid Approach

OpenAI is pursuing an AGI architecture where a “number different LLMs” act as a “mixture of experts” and serve as the “integration hub” [00:05:26]. This hub then calls upon other specialized systems like DALL-E or Wolfram Alpha [00:05:38].

Google/DeepMind’s Neural Net Universe

Google and DeepMind are well-suited to explore a “Gemini type architecture” [00:48:14]. This approach could involve combining:

• AlphaZero: For planning and strategic thinking [00:48:18].
• Neural Knowledge Graphs: Such as those found in Differential Neural Computing (DNC) [00:48:21].
• Transformers with Recurrence: Reintroducing more recurrence into the network architecture, replacing attention heads with more sophisticated elements, as it’s an “obvious way to get interesting abstractions” [00:47:04].
• Alternative Training Methods: Exploring methods like predictive coding-based training instead of backpropagation [00:47:36], or leveraging evolutionary learning algorithms [00:48:55] for more complex architectures.

OpenCog Hyperon’s Metagraph-Centric Design

Goertzel’s own project, OpenCog Hyperon, offers a contrasting approach, where a “weighted labeled metagraph” serves as the central “hub for everything” [00:05:57]. LLMs, DALL-E, and other neural networks act as peripheral components, feeding into and interacting with this core [00:06:08].

Key Features of the Hyperon Architecture:

• Self-Modifying Metagraph: The core is a “big potentially distributed self-modifying self-rewriting metagraph” [00:55:57].
• Knowledge Representation: It aims to represent various types of knowledge—apostolic, declarative, procedural, attentional, and sensory—within this hypergraph and linked representations [00:55:02].
• Cognitive Operations: Different cognitive operations, such as reinforcement learning, procedural learning, logical reasoning, and sensory pattern recognition, are represented as “Little Learning programs” within the metagraph itself [00:55:15].
• Meta Language: A new programming language, “Meta,” allows programs to be represented as “Sub metagraphs” that act on, transform, and rewrite chunks of the same metagraph in which they exist [00:55:33].
• Reflection-Oriented: Unlike LLMs that predict tokens, OpenCog is designed for “recognizing patterns in its own mind, in its own process and its own execution traces” and representing those patterns internally [00:56:54].
• Compatibility with AI Paradigms: It integrates various historical AI paradigms like logical inference and evolutionary programming, as well as new ideas such as “self-organizing mutually rewriting sets of rewrite rules” [00:57:55].
• Path to Superhuman AGI: This architecture is considered less human-like initially but offers a “really short” path from human-level AGI to superhuman AGI, as the system is based on “rewriting its own code” [00:59:35].
• Science and Creativity: It is well-suited for science due to its focus on logical reasoning and precise procedures [00:59:48], and for creativity through evolutionary programming [01:00:07].

Scaling OpenCog Hyperon:

The main challenge for OpenCog Hyperon is the “scalability of infrastructure” [01:00:43]. The project decided to rewrite the old OpenCog version from the ground up to address both usability and scalability issues [01:01:31].

Key components of the scalability pipeline include:

• Compiler Pipeline: A compiler from Meta (Hyperon’s native language) to “Rang,” a language developed by Greg Meredith for “extremely efficient use of multiple CPU cores and hyper-threads” [01:02:05].
• HyperVector Math: Translating Rang into “HyperVector math,” which deals with “very high dimensional sparse bit vectors” [01:02:26].
• Specialized Hardware: Placing HyperVector math on “APU associative processing units” (e.g., from GSI) [01:02:31], rather than just GPUs.
• Distributed Atom Space: The distributed atom space backend uses MongoDB to store atoms and Redis to store indexes, designed to scale as well as these databases [01:06:13].
• Blockchain Integration: Integration with a secure, blockchain-based atom space module called RChain DB and the SingularityNET HyperCycle infrastructure allows for decentralized as well as distributed operations, without significant slowdown [01:06:53].

Goertzel likens this to the breakthrough of deep neural networks, which only realized their potential when “long existing algorithms hit hard infrastructure that would let them finally do their thing” [01:03:09]. The hope is that this scalable processing infrastructure for Hyperon will enable “ancient historical AI paradigms” like logical reasoning and evolutionary programming to operate at great scale [01:03:32].

“If my cognitive theory is right… this metagraph system representing different kinds of memory and learning and reasoning in this self-modifying metagraph… is conceptually a great route to AGI, then basically our obstacle to validating or refuting my hypothesis here is having a scalable enough system” [01:00:50].

Other Notable Players

• Anthropic (Claude): Founded by ex-OpenAI/Google Brain individuals [01:03:00], Claude is noted for being “much better than GPT-4 in many things” [01:16:41], particularly in science, mathematics, and medicine domains, and at writing dialogue [01:16:50].
• Character.ai: Described as number two in revenue after ChatGPT [01:36:00], it was founded by two Google Brain researchers [01:45:00].

The AGI race is “now genuinely” underway, with major companies investing significant resources [02:05:00]. This includes dedicated AGI teams within larger organizations, even if they are often piggybacking on teams doing more immediate, applied work [02:26:00]. The overall acceleration in AI also positively impacts non-LLM AGI projects, even if less transparently [01:05:24].

Link to original

Transclude of cognitive_synergy_in_agi_systems