Regulatory processes may need to keep a continuous eye on AI health products, rather than simply providing initial upfront approval, says research commissioned by the agency responsible for doing so
Medical regulatory processes need significant reform to handle products and services using artificial intelligence, according to two reports recently published by the UK’s watchdog for medicines and healthcare devices.
The National Commission into the Regulation of AI in Healthcare – run by the Medicines and Healthcare products Regulatory Agency – recently undertook launched a call for evidence, to which 761 individuals and organisations responded.
Those taking part in the research said that regulatory processes need significant reform – although not a complete redesign – to deal with AI. This will include ways to monitor products and services continuously rather than just through one-off approval.
“The UK’s existing regulatory framework provides a strong foundation for patient safety but was not designed for AI systems that are adaptive, probabilistic, multi-use and capable of ongoing change after deployment,” wrote one industry organisation in its response, included in a newly published MHRA report.
Participants also said that people should continue to oversee and take responsibility for clinical judgements, with AI systems augmenting professionals rather than replacing them. They also called for transparency and explainability of AI systems, better access to data, training on the use of AI for healthcare professionals, standardised incident reporting for AI systems and ways to develop trust, including through engagement with patients and the wider public.
Related content
The best medicine: How the MHRA is using AI to support scientists and thwart threatsScottish body flags ‘significant challenge and huge promise’ of AI for healthcareWHO warns of need for AI healthcare laws
“The challenge is not choosing between innovation and safety; it is ensuring that both are achieved,” wrote the chair of the commission Professor Alastair Denniston and its deputy chair Professor Henrietta Hughes in the foreword to the MHRA report. “This requires a new regulatory framework that is safe, fast and trusted. One that is proportionate to risk and capable of adapting as technologies evolve.”
As well as the call for evidence, the commission also ran three online focus groups with a total of 26 participants, including young people, unpaid carers and those with learning disabilities. They agreed that AI should support rather than replace professional judgement and were concerned about digital exclusion and uneven rollout.
“These findings highlight that inclusive deployment is central to public trust, not an optional add-on,” the report added.
The commission also held an online public meeting, ran other engagement with professional groups on topics including liability, produced a synthesis of published research and commissioned opinion polling, with the results of the last yet to be published. It also included lessons from the AI Airlock, the agency’s regulatory sandbox for AI as a medical device, which has found gaps in regulatory frameworks including over the use of synthetic data, large language models, hallucinations and changes in an AI system’s performance over time known as model drift.