New University of Bristol and HRA AI study

Last updated on 16 Nov 2026

A new study led by the University of Bristol and co-authored by the Health Research Authority (HRA) on using Artificial Intelligence (AI) for routine research approval checks has been published in the Nature Scientific Reports journal today (Monday 16 November).

The peer reviewed early stage proof of concept study, was carried out in 2024 on 6 historical research applications and involved assessing 3 Large Language Models (LLM), Claude 3.5 Sonnet, ChatGPT and Microsoft Copilot, on their ability to evaluate research participant information materials for the corresponding applications against specified quality standards.

The standards set out the criteria that information for research participants need to meet. These are routine checks that take place before a study reaches a Research Ethics Committee or HRA staff for HRA and HCRW Approval. Depending on the study, it can take more than a day to process the checks.

All three models were able to perform the checks, but their performance differed substantially. When reviewing their performance, the study assessed that Claude answered 87.9% of checks fully correctly, compared with 66.5% for ChatGPT and 62.8% for Copilot. Claude was also the fastest, taking an average of 7.17 minutes per application, compared with 13.58 minutes for ChatGPT and 23.25 minutes for Copilot, and it returned an answer to every question.

Failure of logical reasoning was the commonest source of error across the three models. Claude was able to recognise and correct its errors when challenged, whereas ChatGPT and Copilot sometimes defended incorrect answers by citing page numbers that did not exist.

The performance of the LLM was assessed by two HRA researchers with expertise in assessing against these standards, blinded to the specific LLM, and evaluating the answers against their own knowledge of the quality standards.

The study was the first of its kind to explore elements of how AI could be used in this way to streamline routine research approval checks. It is part of wider work the HRA is doing to establish an evidence base and proof of concept on how AI could be used responsibly to drive improvements in speed, quality and consistency.

Jonathan Fennelly-Barnwell

We collaborated on this study to explore and develop an evidence base on how AI could potentially be used to support the streamlining of research approval processes, and to identify the potential risks, limitations, and opportunities.

The study demonstrated how AI, at the time of the research, could be used for specific routine checks before a study is reviewed by a Research Ethics Committee or HRA staff. The potential benefit of using AI in this way is that it could release staff capacity to focus on more complex regulatory issues, making our processes more effective and potentially also helping reduce the timeline for regulatory review while increasing the consistency of how assessment is approached, a common theme in research and sponsor feedback.

While the results of the study are promising, and also gave useful empirical insights into limitations, it is also important to note that the study was small in scale and conducted two years ago. The functionality on the models we tested may have changed since then. This is why we are continuing to test and evaluate the safe and responsible use of AI as part of our wider work to explore its benefits and challenges.

Dr Jonathan Fennelly-Barnwell, Director of Approvals at the HRA, and the co-author of the paper

The study's remit was limited to using AI to support routine checks. It did not involve using AI to inform ethical decisions.

The study was conducted in accordance with the HRA's privacy policy and the sponsors of the applications used in the study gave their permission to take part.

Back to news and updates