From: mk_thisisit
John J. Hopfield, a recipient of the 2024 Nobel Prize in Physics, was honored for his fundamental discoveries that enabled machine learning using neural networks [00:01:11]. Often referred to as the “father of artificial neural networks,” Hopfield published his seminal model in 1982, which integrated statistical physics with neurobiology, paving the way for modern deep learning [00:01:24].
Distinguishing AI from Biological Processes
Advanced AI neural networks are most interesting because of what distinguishes them most from biology [00:00:07]. One area of focus for improvement is to actually try to incorporate neural networks into biological fields [00:00:17], as well as engineering [00:02:53]. This approach would allow both biology and the abstract world of mathematics to share any concerns or merits regarding these systems [00:03:04].
A key difference lies in the learning algorithms. In biology, changes in synapses do not occur in infinitesimal quantities, and there is inherent noise, with “n or n + 1 proteins” in a synapse [00:13:06]. While the learning algorithm in advanced AI nominally resembles that in biology, it is not the same [00:13:22]. There is a potential future turn in the field of neural networks when the focus shifts to understanding the real biological learning algorithm and how its essence can be included in engineering [00:13:34].
The Essence of Imitation
Hopfield’s work on neural networks aimed to imitate biology as best as possible without slavishly copying every tiny aspect [00:05:51]. This approach, which avoids copying everything verbatim from biology, has proven to be much more advantageous [00:06:03]. Modern physics emphasizes capturing the essence of a phenomenon rather than all its details [00:06:41]. Slavish imitation, conversely, consumes computer resources and provides little insight [00:07:01]. The goal is to achieve something universal, like phase transition systems, which help understand biology, physics, and collective systems [00:08:28].
Understanding Brain Activity and Consciousness
Before Hopfield networks, greater emphasis was placed on tracking the detailed occurrence of action potentials in each neuron over time to describe brain activity [00:09:06]. Hopfield’s realization was the need to understand computation in a physical system, specifically the dynamics in a multidimensional space of coupled variables, rather than just the activities of basic elements [00:09:36]. Computation does not need to be limited to logic alone, unlike modern computer calculations [00:10:25].
Hopfield also recognized that the way biologists and psychologists view memory is isomorphic to certain claims physicists make about the interaction of spin and spin systems with magnetism [00:10:37]. This allowed for combining already similar concepts in a more coherent way, deriving the same equations from completely different perspectives—biology and engineering [00:11:05].
The question of consciousness has been debated for decades [00:00:22]. One key consideration is whether neurons can be thought of as independent systems, or if they must be viewed as exhibiting collective behavior [00:00:29]. Language, for example, might be a collective behavior describable at a higher level, not relying on every lower-level detail [00:17:08].
While Roger Penrose strongly believes human consciousness arises from quantum phenomena in the human brain [00:19:57], Hopfield notes that it cannot be ruled out that consciousness is a problem of classical mechanics rather than quantum mechanics [00:00:39], [00:21:12]. The decision of what role quantum mechanics plays is a secondary issue, and no one can definitively state whether consciousness is purely a matter of classical or quantum physics [00:22:21].
Evolution of Neural Networks
The history of neural networks shows gradual progress [00:14:31]:
- Early networks had one neuron [00:13:53].
- Then came the perceptron network with a forward-flowing information layer [00:13:59].
- The Hopfield network introduced feedback [00:14:05].
- The Boltzmann machine allowed for building more complex topological networks [00:14:11].
- Finally, deep learning emerged, with a method for learning in multi-layered networks [00:14:17].
The shift from Hopfield networks to advanced AI was driven by focusing on understanding knowledge hidden in the network rather than its origin [01:21:13], and the creation of a network structure that could be run through numerous learning algorithms [00:01:28].
Physics and Broader Science
Professor Hopfield views physics as the science of all phenomena in matter [00:23:21]. He believes it is limiting to confine physics to judging whether something is “AI or not AI,” or to prioritize only fields with practical applications [00:24:02]. He cautions against focusing solely on engineering efficiency in the future, advocating for science in a broader context which enables diverse engineering uses [00:24:51].
His own work defies easy categorization, with papers classified as either physical or biological depending on the perspective [00:25:36]. This interdisciplinary nature highlights the intersection of biology and electronics and physics in understanding complex systems.