From: jimruttshow8596
Joshua Bach, Vice President of Research at the AI Foundation and author of “Principles of Synthetic Intelligence: Psi, an Architecture of Motivated Cognition,” views artificial intelligence (AI) as a crucial link between philosophy and mathematics [01:34:00]. His work seeks to understand the nature of AI to better comprehend human existence [01:25:00].
The Philosophical Project of AI
Bach believes that AI, particularly artificial general intelligence (AGI), represents the “capstone” of philosophical inquiry into human nature and our relationship with the mechanical universe [02:57:00]. He argues that the universe is not magic, but rather operates based on understandable, non-conspiratorial processes [02:07:00]. AI is an attempt to build a testable theory of what a non-conspiratorial system capable of reflecting on its environment and its own nature might be [02:29:00].
AGI vs. Narrow AI
The field of AI initially began with great optimism among computer scientists, information theorists, and psychologists, aiming to “teach computers how to think” based on constructive mathematics (computation) [04:06:00]. Early progress was significant, leading to capabilities like decent chess-playing and simple language understanding [04:58:00]. Many programming languages used today originated from these early AI efforts [05:21:00].
However, the daunting nature of building a thinking machine led most researchers to focus on more achievable, short-term goals, resulting in a shift towards “narrow AI” which focused on improving statistical automation [06:05:00]. This practical approach, driven by funding and tenure incentives, sidelined the philosophical project of AGI [06:08:00].
A historical division within AI also occurred, partly due to Marvin Minsky’s claim that cognitive AI was synonymous with symbolic AI, and his active discouragement of cybernetics and neural network research [06:54:00]. This delayed the development of dynamical systems models of cognition and machine learning systems for over a decade, creating a rift between cognitive AI and other approaches [07:11:00]. Despite this, many individuals, including those at major tech companies, secretly remain interested in AGI [08:28:00].
Optimism for Superhuman AI
Bach is optimistic about creating computer intelligences that are at or beyond human level [10:33:00]. He doesn’t believe there are “deep open philosophical questions” that need answering, but rather “a lot of technical details” [10:46:00]. He suggests that humans might be the “stupidest possible general intelligence,” given the limitations of the human brain in areas like working memory size and memory fidelity [12:01:00].
The possibility of creating superhuman AI depends heavily on finding the “appropriate level of representation” [14:55:00]. If the unit of computation is a neuron, it might be very expensive, but if it’s mini-columns (groups of 300-400 neurons), it could be more feasible, perhaps within the cost of a few powerful GPUs [23:38:00].
Large Language Models: GPT-2 and GPT-3
Bach discusses the capabilities of models like GPT-2 and GPT-3, which are trained on massive amounts of filtered internet text [16:37:00]. These models can extract meaningful correlations from enormous datasets in days or weeks, a task far beyond human capabilities [17:15:00].
While not sentient, GPT models demonstrate remarkable coherence in text and image generation, often passing a Turing test for human-like output [17:52:00]. Their “self-attention” mechanism allows them to bind features across dimensions into a relational graph, enabling long-range associations within text or images [35:40:00]. This capability suggests a door opening to embeddings across all modalities [01:36:47].
The Nature of Mind and Reality
Bach delves into the concept of “matter” as a way to describe information and measurable change in a mechanical universe [31:17:00]. Physics, in this view, describes how adjacent states of the universe are correlated, seeking a “causally closed lowest layer” [32:11:00].
He contrasts this with idealism, where conscious experience is primary. While conscious experience is immediately given, he argues that even if we are “dreamt by a mind on a higher plane of existence,” that higher plane could still be modeled by the ideas of physics [34:26:00].
Emergence and Virtual Systems
Drawing from complexity science, Bach explains that “reductionist science” focuses on “dancers,” while the complexity perspective includes the “dance” [39:32:00]. Concepts like a business company are virtual entities—standing waves of coordinated action on signals within boundaries—yet they are “real” because they have traction in the physical world [45:00:00].
Feedback loops are seen as critical in creating higher levels of complexity in systems [40:08:00]. Bach notes that while dynamical systems models are useful, they are “just models” of the behavior of too many parts to count, rather than being “real” in the sense of the underlying physical layer [42:22:00].
He applies this to the mind-body problem: it’s not about two different physical substances, but about making two disparate categories of models (body map and mental states) congruent [44:17:00]. Higher-level emergent structural entities often build from relatively well-defined lower-level units, like neurons, which are relatively fungible units of construction for the mind [47:46:00].
Memetic Viruses
Bach uses the analogy of viruses to explain how ideas operate in society [51:03:00]. A virus, like a text a cell cannot help but read, represents a flaw exploited by “dead chemistry” [49:48:00]. Similarly, “memetic viruses” are radical ideas that challenge the status quo, like the scientific revolution or Darwinism [52:19:00]. While these can increase a group’s fitness and competitive advantage, they don’t necessarily lead to long-term improvements [52:38:00]. Homogeneous societies, particularly those reliant on shared news sources or social media, can be highly susceptible to such “mind viruses” [53:37:00].
The Gap Between Humans and Other Animals
Bach speculates on the factors contributing to the cognitive gap between humans and other animals [01:13:22]. He suggests that the length of human childhood, which is a very expensive period focused on exploration rather than exploitation, allows for “more training data per day” [01:17:09]. This extended maturation period enables the brain to build more layers and better abstractions [01:17:51].
Another key factor might be the ability to do grammatical decomposition, which distinguishes human language use from that of other apes [01:22:46]. This ability, linked to the use of symbols, allows for greater compression and more effective manipulation of information, making the brain exponentially more effective [01:24:20].
Cognitive Architectures
Cognitive architectures are a tradition originating mostly in psychology, influenced by cybernetics and AI [01:29:35]. They aim to identify the underlying structure and principles of the human mind to replicate feats like language learning, social interaction, and symbolic reflection [01:29:48]. Unlike current machine learning approaches that often focus on layered structures, cognitive architectures acknowledge the brain’s complex, interconnected regions, more akin to a city with diverse transport networks [01:31:12]. Examples include SOAR and ACT-R [01:29:22].
The μ (Micro) PSI Architecture
Joshua Bach’s μPSI (MicroPsi) architecture is based on the ideas of German psychologist and cybernetician Dietrich Dorner [01:39:00]. Dorner, initially optimistic about solving human psychology with computers by the late 1970s, independently developed many concepts that became central to AI, including situated and agent architectures with autonomous motivation based on feedback loops [01:40:03].
Bach systematized and translated Dorner’s work into an implementable form, detailed in his book “Principles of Synthetic Intelligence.” The book summarizes Dorner’s ideas, contrasts them with existing fields, and outlines the implementation, before offering critiques and directions for future research [01:43:00].
The first MicroPsi was implemented in Java, while MicroPsi 2 was written in Python [01:44:27]. MicroPsi 2 has been used in AI planning startups and by MicroPsi Industries for controlling industrial robots and basic research [01:45:01]. Bach continues to use MicroPsi internally to experiment with models like spreading activation networks interacting with cellular automata representations [01:45:40].
Looking ahead, future cognitive architectures might leverage very large-scale data processing architectures like Apache Ignite, Flink, and Spark, rather than being confined to single machines [01:46:51]. The focus should shift from monolithic, homogenous representations to understanding how different parts of the architecture implement general principles that allow them to learn to interact, using existing tools like shader programs and graphics engines [01:47:19].