From: mk_thisisit
Nobel Prize winner in physics, Roger Penrose, argues that the term “artificial intelligence” (AI) is a misnomer, stating that “intelligence requires consciousness” [00:04:00], [02:04:00], [13:56:00]. He believes that current computers, being computational machines, will never gain consciousness [00:24:00], [02:13:00], [22:51:00], as consciousness is not computational [00:30:00], [18:19:00].
Defining “Artificial Intelligence”
Penrose suggests that a more accurate term for current AI systems might be “artificial cleverness” [14:09:00]. He distinguishes between true understanding and mere cleverness, noting that some mathematics students can cleverly repeat what they’ve learned without necessarily understanding it [14:18:00]. Similarly, current AI performs calculations without understanding its own operation [10:45:00].
Computation vs. Consciousness
Computers are fundamentally based on computational mathematics, which is a very limited part of mathematics [00:17:00], [16:06:00]. Mathematical logic reveals that there are “uncomputable things in mathematics” [05:02:00], [17:04:00], meaning they cannot be calculated using an algorithm [16:24:00]. Penrose asserts that physics related to conscious thinking cannot be calculated [12:12:00], suggesting that consciousness operates on principles beyond what current computational models can replicate [13:04:00].
Gödel’s Theorem and Understanding
Penrose’s views stem from his studies in mathematical logic, where he learned about Turing machines, computability, and Gödel’s theorem [03:12:00]. Gödel’s theorem, which he finds “incredible” [03:28:00], indicates that there are things whose understanding goes beyond their use [03:31:00]. It shows that one can construct a statement whose meaning is understood, but which cannot be proven using a given set of accepted computational rules [06:02:00], [06:05:00].
The key distinction lies in “understanding why they are true” [07:31:00], not just using rules. This requires being aware of them [07:50:00], an act of consciousness that allows one to transcend the rules [08:25:00]. AI, on the other hand, cannot create its own rules because “it doesn’t know that they are true” [07:13:00].
Perceived Risks and Ethical Implications
The speaker expresses concern that “people have lost sight of what they are doing” [00:42:00], having become “enchanted by the fact that computers have become so powerful” [18:21:00]. The greatest risk in the development of AI, according to Penrose, is that “people think that machines are in fact that they are conscious” [23:21:00]. The risk would be even greater if machines actually had consciousness [23:30:00].
Penrose emphasizes that current AI is merely a “simple statistical machine” [10:57:00] that performs calculations on vast amounts of data without understanding its operation [10:45:00]. He likens it to mathematics students who are clever but lack true understanding [14:18:00].
The “Black Box” of AI
Current AI is often described as a “black box” where “we know what we put into it and we know what comes out of it, but we have no idea what is inside” [11:47:00]. Penrose suggests that consciousness, as a physical process, might be connected to an as-yet unknown type of physics, possibly involving the mysterious “collapse of the wave function” [23:57:00], which is currently uncomputable [12:12:00], [13:04:00].
Until the quantum world is fully understood, particularly its uncomputable aspects and phenomena like backward causality (as seen in EPR paradox experiments [27:36:00]), it is unlikely that synthetic consciousness can be created or transferred to a machine [19:17:00].
Future Outlook
While conceding that AI can be better than humans at performing many tasks today [23:03:00], Penrose maintains that based on current understanding, AI will not gain consciousness [22:51:00]. The potential future development of a universal quantum computer might lead to synthetic consciousness, but this would still be based on a different kind of physics [19:26:00]. He concludes that the notion that current powerful computers signify the emergence of consciousness reflects a misunderstanding of the fundamental nature of consciousness and computation [18:21:00].