From: mk_thisisit
The development and application of Artificial Intelligence (AI) in medicine present significant opportunities, but also considerable ethical and practical challenges. Experts emphasize the importance of understanding the limitations and implications of AI, especially in critical fields like healthcare.
AI as a Tool in Medicine
AI is viewed as a powerful tool that can assist humans, rather than replace them [00:09:47]. There are “great applications” for AI to monitor human decisions and suggest better alternatives, serving as an “Intelligent Assistant” [00:11:11]. In medical journals, there are many publications on AI, even from institutions without a medical school [00:03:05].
Ethical and Practical Considerations
Verification and Accountability
A significant challenge in AI, particularly in sensitive areas like medicine, is the inability to fully explain how it works or why it makes certain decisions [00:17:11]. Unlike aviation, where planes undergo rigorous certification tests and meet safety standards, AI lacks this verifiable framework [00:16:38]. The question of legal liability is also unclear, similar to issues with autonomous cars that hinder their mass adoption [00:17:03].
When an AI system suggests a critical medical action, such as an amputation, a user would naturally ask why they should trust this “black box” [00:18:10]. While AI can state its confidence (e.g., 95% convinced), this statistical assurance doesn’t answer concerns about the remaining percentage or accountability if something goes wrong [00:18:40], [00:19:08].
It is considered “dangerous to delegate important decisions to machines,” specifically medical decisions in patient care [00:10:11]. From the point of view of ethics and operational properties, humans should remain central to decision-making [00:10:22].
Hallucinations and Reasoning
Large language models, such as those related to ChatGPT, are known to “often make mistakes” or “hallucinate” [00:06:42]. While some argue this imperfection makes AI more human-like [00:07:01], it highlights the inherent risk [00:19:23]. Currently, researchers are exploring the use of mathematical logic to create a “protective layer” around AI technology, verifying the truthfulness of its decisions and mitigating “hallucinations” [00:20:27], [00:22:35]. This approach would allow for mathematical proofs of correctness in certain contexts [00:20:35].
The “Black Box” Problem
The increasing complexity of AI models means the “black box” area, where decisions are opaque, will only grow [00:19:15]. This poses a risk that users are unknowingly taking when using tools like ChatGPT [00:19:26].
Current Projects and Solutions
Distributed Artificial Intelligence
A new paradigm for AI development is emerging: distributed artificial intelligence, also known as federated learning [00:25:17], [00:25:43]. This approach addresses the impracticality of the current paradigm, where large amounts of data must be collected from various sources, harmonized, and then used to train a single large model [00:23:22].
In distributed AI, instead of transmitting large datasets, smaller models are trained locally on the data where it resides [00:26:21]. These trained models are then shared between nodes or with a central aggregator, which makes it practically sensible [00:26:25], [00:26:55]. This resolves issues such as:
- Data sharing reluctance: Organizations like hospitals are often unwilling to share patient data with competitors due to legal and patient safety concerns [00:23:59].
- Data volume and harmonization: Moving and reconciling very large datasets from disparate sources is time-consuming and effort-intensive, leading to outdated models [00:24:15].
- Compatibility: Local models can learn on arbitrarily constructed formats and databases, without requiring databases to be compatible with each other [00:26:38].
This paradigm allows for faster responses to changing situations, as demonstrated by insights from the war in Ukraine [00:24:43]. While the military application was mentioned, this technology has broad applications, including defense [00:27:12].
Intensive Care Applications
AI can significantly improve intensive care by processing the ever-increasing volume of data generated by patient physiological sensors [00:27:50], [00:28:02]. AI can extract “operationally valuable” information from this data, helping doctors and nurses cope with the “human hardness” of their work [00:28:33], [00:28:42].
Future of AI in Medicine
While there are moments where it seems AI development is complete [00:07:22], the journey is ongoing. The goal is to ensure AI acts as a beneficial tool, not a competitor [00:10:00]. Recent research at the University of San Diego indicated that AI-generated medical advice was perceived as more empathetic than human doctors’, suggesting humans can learn from these algorithms [00:10:31], [00:11:06]. However, the overarching risk and ethics of AI development must be carefully managed.