The 510(k) Gap: What Hospital Radiology Departments Haven't Resolved Before Their Next AI Model Update
Viz.ai is deployed in 1,100+ hospitals with 20 FDA clearances for time-sensitive diagnoses including stroke and pulmonary embolism. Aidoc covers 1,200+ health systems across CT, X-ray, and MRI. Nuance PowerScribe 360 with AI is integrated at 10,000+ radiology practices. FDA has cleared 950+ AI/ML-based Software as a Medical Device (SaMD) through early 2025 — approximately 75% in imaging. Most are cleared under the 510(k) predicate pathway, which validates a specific algorithm version against a specific study population. When the vendor updates the model, the hospital continues running it under the original clearance. Most hospitals cannot identify what algorithm version is running in production today or whether it matches their FDA clearance documentation.
Key Numbers
Background
The FDA has cleared more than 950 AI/ML-based Software as a Medical Device (SaMD) products through the first quarter of 2025. Approximately 75 percent of those clearances are in medical imaging and radiology — making FDA-cleared radiology AI the single largest category of clinically deployed AI in the United States healthcare system. The deployment footprint is substantial: Viz.ai operates in more than 1,100 hospitals across more than 40 countries, holding 20 distinct FDA clearances covering time-sensitive diagnoses including large vessel occlusion stroke, pulmonary embolism, aortic dissection, and intracranial hemorrhage. When Viz.ai detects a suspected LVO on a CT scan, it pages the on-call neurovascular team directly — bypassing the standard radiology dictation and sign-off workflow entirely in the initial alert. Aidoc covers more than 1,200 health systems with continuous AI triage across CT, X-ray, and MRI for more than 20 AI pathologies, including incidental findings that would otherwise wait in standard dictation queue. Nuance PowerScribe 360 with AI-assisted findings is integrated into more than 10,000 radiology practices, where AI detection overlays appear inside the radiologist's dictation interface as standard workflow.
The deployment driver is structural. The United States has approximately 35,000 practicing radiologists. Imaging volume has grown more than 40 percent in the past decade — driven by population growth, aging demographics, expanded CT utilization in emergency workflows, and the shift of oncology monitoring toward imaging-heavy surveillance protocols. The math does not close without technology. AI deployment has accelerated not because health systems have resolved the governance questions, but because the operational pressure to deploy was greater than the pressure to govern.
FDA cleared most of these products under the 510(k) predicate device pathway. A 510(k) clearance establishes that a new device is substantially equivalent to a previously cleared predicate device. For radiology AI, the clearance covers the algorithm version that was validated in the study, the imaging modalities and scanner configurations included in the study, the patient population and clinical setting in which performance was measured, and the output format and user interface tested. It does not cover algorithm updates that occur after clearance. When a vendor improves the model — to reduce false positive rates, to correct a demographic performance disparity, to expand coverage to additional pathology types — the hospital continues operating under the original clearance while running a materially different algorithm. The FDA introduced the Predetermined Change Control Plan (PCCP) framework to provide a structured pathway for managing post-clearance algorithm updates. PCCP submission is voluntary, not consistently required by FDA, and not consistently submitted by vendors. Most hospitals have received no formal communication about whether their vendor has a PCCP in place for the product they are running.
The alert fatigue pattern in radiology AI deployment is documented and reproducible. Studies at two academic medical centers found that deploying radiology AI without site-specific threshold calibration increased radiologist alert volume by 35 to 55 percent in the first 90 days of deployment. Within six months at both institutions, radiology staff reported reviewing alerts at reduced attention levels — the behavioral shortcut that defines alert fatigue. An IRB-approved study published in a peer-reviewed radiology journal found that when AI alerts were present in the worklist interface, radiologists' independent detection rate for the same finding class declined by 12 to 18 percent compared to baseline, consistent with automation bias. The AI was deployed to improve detection accuracy. In conditions of alert fatigue and automation bias, the deployment degraded it.
The liability framework was established before most hospitals had deployed radiology AI at scale. FDA-cleared radiology AI is classified as clinical decision support — it informs the radiologist, it does not substitute for the radiologist. The physician who reviews the images and signs the radiology report bears liability for missed findings, regardless of what the AI did or did not flag. When an AI system fails to detect a pulmonary embolism finding on a CT pulmonary angiography study and the radiologist signs the report without independent detection, the malpractice claim is against the radiologist, not the AI vendor. The AI vendor's FDA clearance, the vendor contract, and the clinical decision support classification each provide the vendor with substantial insulation from direct liability. They provide the radiologist and the health system with none. Professional liability insurers have responded: since 2024, several major medical malpractice carriers have added questions about AI tool deployment configuration to their radiology practice questionnaire. Most health system risk management departments have not proactively disclosed their radiology AI deployment configurations to their insurers.
Decision Required
Does the AI model running in your radiology department match the algorithm version on your FDA 510(k) clearance documentation — and if it does not, who is responsible for the gap?
Most health systems that have deployed Viz.ai, Aidoc, Nuance, or a comparable radiology AI product cannot answer the first question. Vendor deployments are managed by the vendor. Model updates are pushed to production environments without a mandatory customer notification requirement under current FDA rules. The hospital procurement team validated an FDA clearance number. The clinical informatics team completed the PACS integration. The radiology department received training on the interface. No one was assigned to track whether the algorithm running in production still matches the version that was cleared.
The governance gap has three distinct dimensions. The model version dimension is the most acute: if the vendor has updated the algorithm since your original deployment, you may be running a model that was not validated on your patient population, your scanner fleet, or your clinical setting — and your compliance documentation does not reflect the current configuration. The liability dimension is the most consequential: your radiologists are signing reports on AI-assisted reads under a liability framework that places the full weight of a missed finding on the physician, and your professional liability insurer may not know you are running AI-assisted reads at all. The contract dimension is the most durable: your PACS integration agreement controls what happens to seven years of AI-assisted read history when you decide to switch vendors, and most agreements do not include data portability or API continuity provisions.
Options
Take the position that vendor management of the AI deployment is sufficient — the vendor holds the FDA clearance, manages the model updates, and maintains the production environment. This is how most health systems are currently operating. The risk is concentration: if the vendor has updated the model without notification, you are running an unverified algorithm under stale clearance documentation, and you have no mechanism to detect the divergence until an adverse event makes it visible. This option is operationally zero-friction and clinically invisible until it is not.
Negotiate a contract amendment requiring the vendor to: (1) provide written 30-day advance notice of any algorithm update in production, including a version number and summary of changes; (2) supply version-specific performance data for your scanner fleet configuration and patient demographics; (3) grant the health system the contractual right to delay an update pending internal validation; and (4) maintain a version change log accessible to the hospital compliance team. This is the governance floor — not an audit program, not a re-validation requirement, just version visibility and advance notice. It is operationally achievable and does not require renegotiating the core commercial terms.
Retain a clinical AI validation firm or deploy an internal IRB-approved study to measure your radiology AI's performance on your specific patient population, scanner configurations, and image quality profiles — not the vendor's published study data. This closes the population validity gap but requires six to twelve months to complete and does not address the model update notification problem. Appropriate as a follow-on step after implementing version governance; premature as a standalone action without first resolving the notification and liability disclosure issues.
Temporarily suspend AI-assisted read workflows while the radiology department and legal team complete model version documentation, liability disclosure, and contract review. Operationally disruptive — removes a tool your radiologists have integrated into their workflow — but creates a clean governance baseline from which to restart. Defensible if an adverse event occurs during the review period. Appropriate for health systems that have discovered material gaps in their existing documentation and need a clean reset rather than a patch.
Recommendation
Implement model version governance now, before a model update event makes the gap visible. The governance action is a contract amendment, not a clinical program — it does not require FDA consultation, IRB approval, or operational disruption to the radiology workflow. The amendment should require the vendor to provide 30-day advance notice of any algorithm update, supply version-specific performance documentation, and confirm in writing the version number of the model currently running in production. If the vendor cannot confirm the current model version in writing, that inability is itself a material finding that should be escalated to the CMO and general counsel.
Separately, notify your professional liability insurer of your current radiology AI deployment configuration. Provide the product name, FDA clearance number, deployment scope (modalities covered, facilities, number of reads per month), and the workflow role the AI plays (pre-triage, concurrent read, retrospective flagging). This disclosure is not a concession of liability — it is standard risk management practice and may affect your coverage terms. Do it before your next policy renewal cycle, not after a claim.
Commission an alert fatigue baseline study within the next 90 days. Measure radiologist alert acknowledgment rates, time-to-review, and override rates by alert category. Compare against the first 30-day baseline from your initial deployment. If you did not capture a deployment baseline, use the current period as baseline and measure again at 90 and 180 days. Alert fatigue is a leading indicator of detection accuracy degradation — and it is measurable before an adverse event makes it visible in your incident data.
Enjoying this brief? The next one ships Tuesday.
One enterprise AI deployment, dissected weekly. Free during beta · No credit card · Unsubscribe anytime
Risks
The vendor pushes a model update. The hospital continues operating under the original FDA 510(k) clearance documentation while running a materially different algorithm. Neither the compliance team nor the radiology department is notified. The divergence between the cleared version and the production version is invisible until either a regulatory audit, an adverse event investigation, or a vendor disclosure triggers a review. This is the most common governance gap in radiology AI deployments and the most difficult to detect without a proactive monitoring protocol.
Radiology AI alert volume grows faster than the department calibrates thresholds. Radiologists develop behavioral shortcuts for alert review — acknowledging without full review, dismissing categories that have historically produced false positives. Detection accuracy for flagged finding classes falls below the department's pre-AI baseline. The adverse outcome is not a single missed finding — it is a systematic, invisible degradation in detection quality across thousands of reads, visible only in retrospective analysis after outcomes data accumulates.
The health system initiates a PACS vendor transition. The radiology AI integration — which required significant IT configuration and clinical workflow change management — is architecturally tied to the existing PACS. The AI vendor agreement has no API continuity provision, no data export right, and no contractual obligation to support integration with the replacement PACS. Seven years of AI-assisted read history, alert logs, and performance metrics stays with the vendor. The PACS transition budget expands by the cost of re-negotiating the AI vendor agreement and rebuilding the integration — a cost that was not in the capital plan.
A missed finding claim proceeds to discovery. The plaintiff's counsel requests documentation of the AI tools used in the radiology workflow at the time of the read. The health system's legal team produces the radiology AI configuration — which the professional liability insurer had not been informed of. The insurer initiates a coverage review. The policy renewal conversation is now framed by a claim in progress and a disclosure gap the insurer characterizes as material. The resolution is expensive regardless of outcome.
Questions Your Team Should Be Answering
These are the questions that distinguish organizations that get this right from those that do not. If your team cannot answer them, that is your first deliverable.
- 1.
What algorithm version is running in your radiology AI production environment right now — and does the vendor have a written record they can provide you?
- 2.
Does the version currently in production match the version number documented in your FDA 510(k) clearance file — and when was the last time someone checked?
- 3.
What is your notification protocol when the vendor pushes a model update — and does your current contract give you the right to delay that update pending internal validation?
- 4.
Has your professional liability insurer been notified of your radiology AI deployment configuration, and does your policy explicitly address AI-assisted clinical decision support?
- 5.
What does your PACS integration agreement say about data portability — specifically, who owns the AI-assisted read history and alert logs at contract termination?
- 6.
Have you measured radiologist alert acknowledgment rates and override rates since deployment — and do you have a baseline from the pre-deployment period to compare against?
- 7.
Which radiology AI alerts in your current configuration trigger direct workflow actions (paging, escalation) without radiologist sign-off — and has your medical staff committee formally reviewed those workflows?
If this memo belongs in your next executive meeting or board pack, send it along. One click opens a pre-drafted email — edit or send as-is.
The ATO Bottleneck: What Federal Agencies Discover When AI Procurement Meets the Authorization Process
Federal agencies are deploying AI tools across procurement, benefits processing, and workforce operations — but the ATO process was written for static systems. FedRAMP authorizes cloud infrastructure, not AI behavior. Most frontier AI APIs lack FedRAMP authorization, and most federal ATOs are stale by the time the model updates.
Read memo →The Algorithmic Underwriting Audit: What NAIC AI Requirements Mean for Every Insurer Using AI in Pricing and Claims
State insurance regulators have moved. The NAIC Model Bulletin on AI has been adopted in 38+ states. Colorado mandates external algorithmic audits for life insurance AI. California CDI has challenged AI-generated property risk scores. Most carriers have deployed AI in claims and underwriting without building the governance documentation regulators are now requiring.
Read memo →The SR 11-7 Blind Spot: What Banks Discover When AI Hits Model Risk Management
Banks are deploying AI in credit underwriting, fraud detection, compliance monitoring, and customer service — but SR 11-7, the OCC/Fed model risk framework, was written in 2011 for statistical models. The validation gap for third-party LLM APIs, the model version change management problem, and what bank examiners are beginning to ask.
Read memo →