From: jimruttshow8596
Defining Mind and Learning in AI
The concept of “mind” in the context of AI is often understood as the “software that runs on the brain” [00:01:50]. This software isn’t a physical entity but a specific physical law, describing how certain macro states with causal structures emerge when elements are arranged in a particular way [00:02:05]. Just as text processors can be described as a physical law of computer gates, mind can be seen as a principle or software running on the brain, emerging from the coherent causal structure of neuron activity [00:02:27]. If brains are Turing complete, minds can be built on different substrates by implementing the same principles [00:03:27].
Embodiment and Mind
While some argue that the human mind is deeply embedded in its bodily substrate, suggesting that artificial minds cannot be the same if not similarly embodied [00:03:45], others contend that a mind can still exist even with a changed interface to the universe. For instance, if an individual is connected to city sensors with aligned incentives, they might “turn into a city at some level” [00:04:20]. Embodiment can be entirely virtual, requiring only a physical substrate realization that implements the necessary principles [00:05:27].
The Role of Emotions
Emotions and motivational impulses, described metaphorically as the “elephant” part of the mind, are accessible to the analytical mind (“monkey”) through feelings [00:07:05]. These feelings are projections into the body map, acting as a communication mechanism between distributed perceptual systems and the analytical, grammatical engine of the brain [00:07:41]. The Psi theory suggests that an agent like humans can be described using homeostasis as a guiding principle [01:21:42]. Needs, when frustrated, produce pain signals and when satisfied, pleasure signals, forming a coherent hierarchy of purposes that define the “soul” or structure of the mind [01:22:07].
Learning Mechanisms in Humans and AI
Human and animal learning rates are remarkably rapid, suggesting a fundamentally different mechanism than the gradient descent methods commonly used in artificial neural networks [00:22:01]. For example, learning to play complex war games to beat AI in only seven games, leveraging transfer learning from thousands of prior game experiences, highlights this efficiency [00:20:45]. In contrast, systems like AlphaZero require hundreds of thousands to millions of plays to learn even stereotyped games like chess or Go [00:21:31].
The Importance of Attention
A key aspect of rapid learning is the ability to “pay attention” in the right way [00:21:55]. Directing a subject’s attention with great acuity, as seen in a tennis coaching example, enables quick behavioral updates in critical areas [00:22:36]. Without knowing what to pay attention to, a system must brute-force the problem, which is wasteful [00:22:54]. The human attentional algorithm is viewed as a “hack” to bypass the combinatorial explosion of options [00:34:31].
GPT-3: Capabilities and Limitations
GPT-3 is a notable example of generative AI, particularly for its ability to produce text that emulates styles and creates plausible completions [00:27:17]. Its underlying algorithm, the transformer, is considered a solution to problems of automatically finding structure in language by making statistics over non-adjacent words [00:25:56], [00:26:10].
Capabilities:
- Style Emulation and Plausible Completions: Remarkable at emulating styles and producing plausible text completions within a short domain (e.g., two or three sentences) [00:27:17], [00:27:31].
- Sentiment Extraction: Can reliably extract sentiments from paragraphs if primed correctly [00:28:17]. This suggests a semantic operation where it understands, at some level, what it means to extract sentiment [00:28:48].
- Arithmetic: Able to perform two-digit arithmetic quite reliably, treating symbols as subject to arithmetic operators [00:31:07].
Limitations:
- Lack of Coherence and Context: Beyond a short text length (e.g., 2,000 characters or two pages of a book), its coherence falls apart [00:27:27], [00:30:24]. It doesn’t have a global model of the universe or a sense of meaning [00:28:56].
- Symbol Grounding Problem: Its symbols are grounded only in language, not in external reality [00:30:52].
- Fixed Working Memory: The transformer algorithm uses a fixed working memory window (2048 adjacent tokens), unable to actively change or construct working memory contexts like human minds [00:26:21], [01:17:44], [01:18:51].
- Offline Learning: GPT-3 is trained offline and stops learning at a certain point (e.g., October 2019 for GPT-3), meaning it doesn’t continuously learn and track reality in real-time [01:19:14], [01:19:31].
- Lack of Relevance Mechanism: It does not explicitly care about relevance; its perceived relevance comes from being trained on human-written texts, which inherently focus on relevant information [01:20:05]. A system interacting with the world needs a motivational system to focus on the most promising parts of its model [01:20:50].
- No Agency: It is not an agent and lacks a fixed context, unable to understand what it is or relate to reality unless explicitly primed [01:13:48].
“I think that the next frontier in thinking about learning and cognitive systems and what we can learn from humans is that the rate at which we learn as animals and not just humans of course, animals learn very rapidly as well have to be somehow fundamentally different than what we’re seeing so far from the world of artificial neural nets” [00:21:53].
Future Directions for AGI and Learning
The next steps for the science of explaining mind emerging from matter should focus on attention-based models, beyond the initial transformer [01:13:30].
Three key areas for improvement in AI learning:
- Larger and Dynamic Attentional Windows: Extending attention to allow active changes and construction of working memory contexts, not just fixed windows [01:17:49], [01:19:08].
- Online Learning: Developing systems that can continuously learn and track reality in real-time, unlike current offline models [01:19:26], [01:19:40].
- Relevance and Motivational Systems: Implementing motivational systems that allow the AI to assign relevance to learning and meta-learning, guiding its focus in a rich sensory environment [01:20:05], [01:20:46]. This requires shifting from content-agnostic models to multi-modal representations that can address various inputs [01:17:59].
The ability of human minds to “rewrite” memories, often through linguistic processing, is also a key difference from current language models like GPT-3, which lack dynamic rewriting capability [01:18:11], [01:18:37]. The OpenCog system, in contrast, incorporates concepts of local and global rewriting [01:18:28]. The focus on perception (akin to deep learning) rather than solely linguistic or symbolic systems is also gaining prominence in cognitive science, suggesting a path forward [01:07:50]. The “realness” of experience is itself a model property that the mind attaches to certain parameter dimensions, indicating its predictive utility for future sensory patterns [01:09:07], [01:09:55].