{"slug": "actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical", "title": "Actionable Feedback: Improving Model Capabilities from User Edit Data on Clinical Documentation", "summary": "A review of data from a deployed AI clinical documentation product found that over 45% of edited outputs contain user-added information not directly supported by the model's inputs, meaning naive fine-tuning on raw clinician edits risks teaching models to hallucinate, according to an arXiv paper (2611.12904v1). The authors built a taxonomy of clinician-editing behaviors, generated semi-synthetic clinician-edit data for a discharge-summary task based on the public MIMIC-III dataset, and introduced filtering methods for identifying actionable edits evaluated by expert review. On the semi-synthetic dataset, naive fine-tuning on raw edited output increased ungrounded content, while progressively filtering toward grounded edits improved clinician-alignment metrics while retaining faithfulness to the input context.", "body_md": "arXiv:2611.12904v1 Announce Type: new \nAbstract: Large language models (LLMs) are increasingly being deployed in clinical documentation workflows, where clinicians review and edit their outputs before use. In principle, these clinician edits represent a scalable source of supervision for model training. In practice, not all edits represent actionable feedback: for instance, on data drawn from real-world deployment of an AI clinical documentation product, our review suggests that over 45% of edited outputs contain user-added information that is not directly supported by the inputs to the model. Naively fine-tuning on such edits risks teaching the model to hallucinate. In this paper, we develop a method to address this challenge. Given that real clinician edits on deployed documentation tools are not publicly available, we first develop a taxonomy of clinician-editing behaviors, informed by data from a deployed AI clinical documentation product. Then, we develop a method for generating semi-synthetic clinician-edit data, informed by our taxonomy, on a discharge-summary generation task based on MIMIC-III, a public dataset. Finally, we introduce simple filtering methods for identifying actionable edits, with accuracy evaluated by expert review. We show empirically, on our semi-synthetic dataset, that naive fine-tuning on raw edited output increases ungrounded content, while our approach of progressively filtering toward grounded edits enables improvement on clinician-alignment metrics while retaining faithfulness to the input context.", "url": "https://wpnews.pro/news/actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical", "canonical_source": "https://arxiv.org/abs/2611.12904", "published_at": "2026-11-11 05:00:00+00:00", "updated_at": "2026-11-11 05:19:30.330154+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety", "machine-learning"], "entities": ["MIMIC-III", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical", "markdown": "https://wpnews.pro/news/actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical.md", "text": "https://wpnews.pro/news/actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical.txt", "jsonld": "https://wpnews.pro/news/actionable-feedback-improving-model-capabilities-from-user-edit-data-on-clinical.jsonld"}}