The Trump administration released ideas for regulating large language models used in health care as companies roll out chatbots to help patients manage mental health, diabetes and other medical conditions. #
The Food and Drug Administration wants the public to weigh in on whether chatbots should pass “competency-based” tests — similar to exams doctors must take — before they are marketed as medical devices.
FDA’s medical device center “is considering a competency-based approach to premarket evaluation of GenAI-enabled devices that is also inspired, at a high level, by how human clinicians are evaluated and credentialed, but would be adapted for the technical, practical, and legal considerations applicable to the regulation of medical devices,“ according to a discussion draft the agency published Tuesday.
FDA would test the final, user-facing generative AI product rather than foundational models and subcomponents, the agency wrote. The test would specifically check if the generative AI product “demonstrates the clinical knowledge, analytic capabilities, safety behavior, communication, and generalizability to support reasonable assurance of safety and effectiveness of the device for its intended use,” the agency added. The tests could involve scoring rubrics and expert adjudication.
But the rigor of the tests would depend on the risk profile of a generative AI product, the FDA wrote. The product’s risk profile could take into account whether a product steers a user toward an action or provides information and the consequences of a user relying on a faulty generative AI output, according to a proposed FDA framework.
Testing alone might not be sufficient, so FDA could also collect real-world evidence to ensure a generative AI product is working as intended, according to the discussion paper.
FDA is not the first to consider a testing regime for generative AI. Over the past year, multiple academics have proposed policies requiring generative AI products to pass licensure exams like the ones doctors must pass to practice medicine. The American Medical Association, the largest U.S. doctors’ group, is also mulling whether AI licenses are the right path forward.
But FDA’s exploration of generative AI testing is notable because it’s the latest step the agency has taken to determine the best way to regulate the emerging technology.
The fact that large language models can constantly evolve and their outputs can vary wildly are among the reasons FDA has found it difficult to regulate generative AI. An FDA advisory committee met in 2024 and 2025 to help the agency begin addressing those regulatory challenges. FDA has also published several requests for information about AI and recently convened health AI companies to guide its thinking.
The agency is giving the public until Oct. 19 to comment on the generative AI discussion draft. Agency officials maintain that the paper is not a policy statement or representative of future draft regulations.
“FDA intends to lead boldly, but with transparency and genuine intellectual humility. With this paper, we are putting our ideas into the public square and inviting challenge and input from the broader ecosystem so that we can refine these concepts before translating them into policy. The guidance we ultimately develop will be stronger for our having done so,” wrote Rick Abramson, the director of FDA’s Digital Health Center of Excellence, on LinkedIn.