From: jimruttshow8596
Melanie Mitchell, a professor at the Santa Fe Institute specializing in conceptual abstraction, analogy making, and visual recognition in artificial intelligence systems, joined Jim Rutt to discuss the fundamental differences between human and AI cognition, particularly concerning the role of memory and emotions [00:35:00]. Their conversation also touched upon the broader comparison of human and AI understanding and the debate on AI understanding and consciousness [00:29:18].
Distinguishing Human from AI Cognition
Mitchell highlights that when tests are formulated for humans, assumptions are made about human cognition, such as the human not having memorized all of Wikipedia [00:16:51]. This allows for extrapolation of test performance to real-world concepts [00:13:13]. However, it has not been shown that the same extrapolations can be made from large language models (LLMs) passing these tests [00:17:10]. The core difference lies in how humans and LLMs process and understand information.
The Problem of “Understanding”
The word “understanding” itself is not well-understood in the context of AI [00:31:15]. There’s a significant debate on AI understanding and consciousness, with some arguing LLMs can understand human language similarly to humans and might even be conscious, while others, known as the “stochastic parrot” side, contend these systems merely parrot sophisticated language without genuine understanding [00:29:31]. The goal of a paper Mitchell co-wrote with David Krakauer was to review these perspectives and analyze the notion of understanding within cognitive science and neuroscience [00:30:51].
Historically, AI has forced a clarification of human intelligence [00:31:49]. For instance, early predictions suggested that a chess-playing computer reaching grandmaster level would require full-blown human general intelligence, which proved incorrect [00:31:55]. This pushed people to redefine intelligence beyond brute-force search [00:32:29]. Similarly, LLMs are now pressuring a refinement of the definition of mental terms like intelligence and understanding [00:32:44].
Compression and Working Memory
Humans have a strong, possibly innate, desire to understand via compression [00:35:06]. This involves taking complex information and compressing it into simpler, lower-dimensional representations, such as Newton’s laws or mental models of the world [00:35:14]. This differs from LLMs because humans operate with a very small working memory [00:35:53]. While GPT-4 has a context window of 32,000 tokens, humans cannot retain that much information in their working memory [00:36:03]. This constraint forces humans to build abstractions and compressions, leading to a different kind of understanding that may be more generalizable than that of LLMs [00:36:17]. Some research suggests this constrained working memory might be key to human intellectual abilities that machines may lack [00:36:40].
The Significance of Long-Term Memory
Another key difference is the concept of long-term memory. While LLMs can be said to have a long-term memory in their billions or trillions of weights that store learned data, they lack the episodic memory that forms a human’s sense of self [00:38:21]. They do not “remember” past conversations or experiences in the way humans do [00:38:36]. This lack of episodic memory limits some of the things LLMs can do [00:39:00].
Jim Rutt hypothesized that building hierarchies of memory exterior to the LLM, coupled with an intentional mechanism, could potentially coerce LLMs to act more like the unconscious processes humans use for language understanding and production, without being the sole repository for memory hierarchies [00:37:45].
The Role of Emotions
Emotions play a crucial role in human cognition and decision-making, particularly because humans are a social species [00:40:35]. Emotions provide motivations that facilitate social interactions [00:40:50].
Antonio Damasio’s clinical work shows that patients with damage to their emotional machinery struggle with basic decisions, even choosing breakfast [00:39:33]. This suggests that at the end of the day, a “tipping factor” in human decision-making is often emotion or intuition [00:39:46]. This can be seen as a “hack” to bypass the combinatorial explosion of inference that formal reasoning systems like Prolog faced [00:40:05]. By using a limited working memory and emotion as a “simple-minded picker,” humans can process the world effectively with relatively low computational power [00:40:16].
A significant question arises regarding the concept of “caring” in AI. Philosopher Margaret Boden argued that AI won’t take over the world because it doesn’t care [00:41:04]. While systems like AlphaGo are superior to human Go players, they don’t “care” about winning in an emotional sense [00:41:24]. Jim Rutt proposed that AlphaGo’s “care” could be its definition of the loss function (winning games), but Mitchell views this as an impoverished understanding of caring, akin to a thermostat “wanting” to keep temperature constant [00:41:42]. This leads to the question of whether caring is essential for intelligence in the real world [00:41:36].
Conclusion
The rapid advancement of LLMs is forcing a deeper examination of human cognition and its core components like understanding, memory, and emotion. While LLMs excel at tasks like predicting the next word, their fundamental architecture, currently lacking true online learning, episodic memory, or emotional grounding, sets them apart from human intelligence [00:49:17]. The question remains whether training on language alone, or even with multi-modality, will be sufficient for AI systems to develop human-like intuitive physics or psychology models [00:46:31]. As Mitchell notes, it’s an empirical question that the scientific community is still actively exploring [00:46:53]. The field is ripe for new scientific inquiries into these complex systems and their implications for human understanding [00:43:49].