Medical AI technologies are changing how doctors make clinical decisions and significantly reducing the time clinicians spend on manual tasks.
However, these AI systems are only as reliable as the data used to train them.
When these models fail, researchers currently must perform slow, manual and nonreproducible investigations to determine why. There is no automated, systematic way to trace a model’s poor performance back to specific errors in the data pipeline. Troubleshooting data errors manually is not only labor-intensive but also costly, and it can slow the entire clinical workflow, causing patients to wait longer for critical treatments.
Unlike general AI, medical data is also precious and expensive to collect because it requires patient consent and high levels of clinical expertise. Simply discarding “bad” data is often not a viable option.
USC researcher Ruishan Liu aims to build a traceable, cost-aware and end-to-end AI framework for data curation in an upcoming study funded by the National Science Foundation (NSF).
Launched this month, the three-year project received nearly $900,000 in funding.
Liu is a WiSE Gabilan assistant professor from USC Viterbi School of Engineering’s Thomas Lord Department of Computer Science and USC Mark and Mary Stevens School of Computing and AI, with joint-appointments at USC Dornsife’s quantitative and computational biology department and Keck School of Medicine of USC‘s radiation oncology department.
Teaming up with Virginia Tech researchers Ruoxi Jia and Wenjie Xiong, Liu will combine her background in computer science, biomedical data science and radiation oncology with her collaborators’ expertise in algorithms and computer systems.
The primary test bed for the study is radiation oncology, a field that relies heavily on complex, multimodal data, including medical imaging such as CT and MRI scans, radiation treatment planning and longitudinal electronic health records. Liu emphasized that the framework could also be applied across a wide range of medical specialties.
“Our goal is to include a human-in-the-loop feature in the tool,” Liu said. “We want to design a framework that allows medical experts to review key decisions.”
How This New AI Tool Works #
Liu said the tool is essentially a framework consisting of three primary components, or “thrusts.”
The first focuses on algorithmic foundations. This part of the framework identifies which data is responsible for a model failure (attribution), why it is responsible (diagnosis) and how best to fix it (intervention selection). It uses reasoning-based large language models (LLMs) to propose root-cause hypotheses. This component replaces manual troubleshooting with an automated pipeline that traces model failures to specific data-level root causes and recommends the most cost-effective repairs. It tackles the challenge of maintaining model reliability in complex medical workflows, where data is scarce and errors are often context-dependent.
The second component is the provenance infrastructure. It acts as a “traceability” layer by logging the complete history and transformation of every data sample. It uses efficient structures such as Bloom filters, a fast probabilistic data structure used to test whether an item belongs to a set, to enable rapid querying of data origins. This provides a scalable, sample-level history of every data point, addressing the information-gap limitations of current medical AI systems. In turn, it transforms the AI “black box” into a transparent system where model failures can be traced to specific scanner protocols, data-processing steps or human errors.
The final component is the Integrated Curation Agent, an end-to-end system that orchestrates the algorithms and infrastructure into a closed-loop workflow. It includes a human-in-the-loop feature that allows medical experts to review and approve high-stakes remediation decisions while managing context-dependent medical data failures that traditional tools cannot detect.
How This Study Could Change the Medical AI Landscape #
The tool developed from this study will be open source, making it available for medical researchers and clinicians to use.
One of the biggest advantages the tool could bring is improved efficiency. It has the potential to reduce the manual labor required for researchers and clinical data scientists to identify and troubleshoot data issues.
The framework introduced in the study also promotes standardization by establishing a replicable methodology for maintaining data integrity that can be applied to high-stakes domains beyond medical imaging.
Ultimately, the tool could help medical institutions deploy AI systems with greater trust, accountability and transparency.
Published on August 6th, 2026
Last updated on August 6th, 2026