To gather views on our draft research transparency, we ran a public consultation which was open from 17 June to 6 September 2019. The consultation consisted of an online survey, open workshops, a patient focus group and webinars and workshops with Research Ethics Committees and the staff who support them.
Contents
- How we analysed the results
- Who took part in the consultation
- The strategy aims and scope
- Prioritising our work
- Making sure all clinical trials are registered
- Reporting results
- Sharing the results of research studies with the people who took part
- Publishing an annual 'transparency league table'
- Considering past performance
- Fining sponsors with very poor transparency rates
How we analysed the results
Some of the questions in the online survey required respondents to choose from a fixed set of options. For the questions which invited a free-text response and for feedback received at the workshops, we used a qualitative content analysis approach to identify, analyse and interpret patterns of meaning (or ‘themes’). Analysis of responses to each question was carried out by one member of HRA staff and checked by another to confirm that the analysis was reasonable, complete and consistent.
Download a CSV file of all the responses to the online survey.
Who took part in the consultation
The online survey received a total of 490 responses; 67 from organisations and the remainder from individuals. Almost half (48%) of responses were from individuals working in research either as researchers, research managers or from sponsors and Contract Research Organisations (CROs). We also received a significant number of responses (36%) from patients, patient advocates and research participants.
Pie chart summary: The pie chart shows how survey respondents described their role in research. Researcher (NHS/Industry/University) occupies 20%, research manager (NHS/Industry/University) occupies 15%, CRO (Contract Research Organisation) occupies 6%, sponsor occupies 7%, patient occupies 15%, patient advocate occupies 12%, research participant occupies 3%, NGO/other advocacy group occupies 1%, REC member occupies 5%, healthcare professional occupies 4%, funder (public/charity) occupies 2%, other occupies 10%.
Download a csv file for pie chart data from figure 17.
The workshops were held in London, Manchester, Cardiff, Belfast and Edinburgh and were open to anyone who wanted to attend. A total of 161 people attended of which 25% were patients, patient advocates and research participants, 18% were research managers and 17% were researchers.
Figure 1 - Pie chart showing how research participants described their role in research.
Pie chart summary: The pie chart shows how the workshop participants described their role in research. Research manager occupies 18%, researcher including industry occupies 17%, patient advocate occupies 11%, patient occupies 11%, sponsor occupies 11%, other occupies 9%, REC member occupies 8%, healthcare professional occupies 7%, funder (public and charity occupies 5%, research participant occupies 3%, industry including CRO, sponsor and manager occupies 1%.
The strategy aims and scope
What we heard
We asked for general views about the draft strategy, its scope and its ambition. Overall, the respondents to the consultation were positive about the draft strategy. They felt that it achieved a good balance between being ambitious yet achievable. They also saw the HRA as the right organisation to lead in this area, in collaboration with patients, participants and research professionals.
We asked for views about our proposal that the draft strategy should initially cover clinical trials - research studies that test the safety and effectiveness of interventions such as medicines, medical devices, surgical techniques, public health measures and behavioural therapies - rather than all types of health and social care research reviewed by the HRA.
Most respondents to the survey agreed with this approach. This view was also expressed by attendees at our workshops and REC members who joined the webinar.
However, in both the survey and the workshops, a significant number of people, even where they agreed with the stated focus, highlighted that this should be the initial and not the sole focus with other types of research included as soon as possible.
Question 1: Please indicate the extent to which you agree with the following statements.
Question 1a. The strategy should focus initially on clinical trials.
Figure 2 - Graph showing responses to question 1a.
Graph summary: The bar chart shows the number of respondents who selected, strongly disagree, disagree, neither agree nor disagree, agree or strongly agree in response to the statement the strategy should initially focus on clinical trials. The most frequent response was agree which 194 respondents selected, followed by strongly agree which 131 respondents selected.
We asked for views about our proposal that the draft strategy should initially cover registration, reporting results and feeding back to participants. We suggested that others in the research system are best placed to continue to enable appropriate sharing of study data and tissue.
Most respondents to the survey agreed with this approach. Whilst many respondents believed that the HRA could have an important role to play in the facilitation of data/tissue sharing, this was not generally felt to be a priority for the HRA given existing high-profile initiatives to promote open access in research.
Question 2. Please indicate the extent to which you agree with the following statements.
Question 2a. The strategy should focus initially on registration, reporting results and feeding back to patients.
Figure 3 - Graph showing responses to question 2a.
Graph summary: The bar chart shows the number of respondents who selected strongly disagree, disagree, neither agree nor disagree, agree or strongly agree in response to the statement the strategy should focus initially on registration, reporting results and feeding back to participants. The most frequent response was agree which 215 respondents selected, followed by strongly agree which 160 respondents selected.
What we decided to do
Whilst all types of transparency are important, we have decided that the focus of this strategy will be on registration, reporting results and feeding back to participants as these are the immediate priority areas for the HRA. We recognise the important facilitative role that we could have with regards the sharing of tissue and data and may explore this in the future.
This strategy covers health and social care research taking place in the UK which involves people, their tissue or their personal data, and which require review by an NHS research ethics committee. We decided that some areas of the strategy – such as informing participants about the findings of a research study – will apply to all types of research but that other areas – such as taking action in cases of non-compliance – will apply only to clinical trials to start with. We will extend the strategy to cover other types of research in a later phase of work.
Prioritising our work
We asked for views about how we should prioritise the activities to improve research transparency that we had already decided to do.
What we heard
Survey respondents and workshop participants thought all the activities were important, but the following were most popular:
- Being clearer what we expect of sponsors and researchers
- Making it clear what information from applicants we will make public and what we will share with others
- Introducing automated reminders for researchers/sponsors to submit transparency data and to view the status of their studies
- Flagging up individual studies where transparency information is overdue.
What we decided to do
We have taken these views on board in the implementation plan. We will start with clarifying our guidance to sponsors and researchers, backed up by online learning. We will introduce automated reminders and public information showing where transparency information is missing into our systems as they are redeveloped during 2020 and 2021.
Making sure all clinical trials are registered
We asked for views about three approaches to ensuring that clinical trials are publicly registered before the study begins:
- Researchers must register their study before seeking approval
- The HRA supplies data directly to a registry
- The HRA becomes a registry itself.
What we heard
The option for the HRA to become a registry itself was the most popular - 34% of survey respondents agreed with this approach. However, there was a relatively even spread across the three options, with 27% of respondents choosing ‘Researchers must register their study before seeking approval’ and 23% choosing ‘HRA supplies data directly to a registry’.
There was general support across all workshops for a single place for registration for all studies and for the HRA to be that place. However, it was emphasised that whatever option is chosen, the process should be straightforward with registration expectations made clear and guidance and training provided.
Question 5. Which of these options do you think is the most appropriate to ensure registration of clinical trials (please select only one)?
Figure 12 - Graph showing responses to question 5.
Graph summary: The bar chart shows the number of respondents who selected don’t know, something else, HRA becomes a registry itself, HRA supplies data directly to a registry or researchers must register their study before seeking approval in response to the question which of the options do you think is the most appropriate to ensure registration of clinical trials (please select only one). The most frequent response was HRA becomes a registry itself which 164 respondents selected, followed by researchers must register their study before seeking approval which 132 respondents selected.
What we decided to do
We have decided to use data submitted by applicants via IRAS to form the basis of registration. However, the decision about where that registration information should be published affects other organisations and has financial and commercial implications. We have therefore decided to carry out a formal options appraisal to determine whether the HRA should set up a registry or data should be supplied directly to an existing registry.
Reporting results
We asked for views about a set of steps for ensuring that information about the results of each study is made public. This would include:
- making it clearer to applicants at the time of study approval that they should send us a final report 12 months after the study has ended
- taking a more proactive approach to prompt sponsors to keep their information up to date and submit final reports
- publishing information we receive on a public platform or provide a link to the information held in a registry or publication.
We asked for views about how effective these steps will be in improving study reporting.
What we heard
81% of survey respondents believed (61%) or believed very strongly (20%) that these steps will improve the reporting of research results.
The need for cultural change across the entire research environment was frequently emphasised in workshops with many supporting the HRA in facilitating this change. It was often suggested that the HRA need to work closely with other stakeholders across the system, particularly funders with whom compliance data might be shared, to help drive improvements in transparency and:
- raise awareness of the social and moral responsibility to report findings
- end the current bias for only reporting positive results
- report research findings in ways other than peer-reviewed journals
In workshops, opinions were split on whether a period of 12 months to submit end of study report/results was appropriate with some attendees suggesting that larger, more complex studies should be allowed more time to report.
Question 6. To what extent do you think that these steps will improve the reporting of results from clinical trials?
Figure 13 - Graph showing responses to question 6.
Graph summary: The bar chart shows the number of respondents who selected I believe very strongly that they will not improve the reporting of research results, I believe that they will not improve the reporting of research results, don’t know, I believe that they will improve the reporting of research results, or I believe very strongly that they will improve the reporting of research results in response to the question to what extent do you think that these steps will improve the reporting of results from clinical trials. The most frequent response was I believe that they will improve the reporting of research results which 297 respondents selected, followed by I believe very strongly that they will improve the reporting of research results which 96 respondents selected.
Download a csv file for graph data from figure 13.
What we decided to do
Based on the feedback from the consultation, we have decided to push ahead with the proposed steps to improve reporting. We will make it clearer to applicants at the time of study approval that they have to submit a final report 12 months after the study has ended. We will take a more proactive approach through systems improvements and monitoring to prompt sponsors and researchers to keep their clinical trials information up to date in registries and to submit final reports for all studies.
We will work with stakeholders to define a data-set for final reports and determine what information we will publish, share with others and use as the basis for reviewing the applicant’s past transparency performance. We will also work with stakeholders to develop a policy for how we take account of past performance when reviewing new studies for approval.
Sharing the results of research studies with people who took part
In the draft strategy, we described plans to reinforce the importance of informing participants about the study findings by changing the question we ask applicants. This is designed to shift the emphasis from asking whether researchers will share study results with participants to how and when they will share them.
We also described plans to require sponsors to submit a lay summary of the study results to us within 12 months of the end of the study, which we will publish. We asked for views about what else would help.
What we heard
Much of the feedback we received strongly supported the importance of informing participants, with several pointing out that information needs to be accessible and easy to understand. The involvement of patients and the public was thought to be essential to informing participants appropriately. This was summed up by one person as ‘Do this with participants, not for them’.
Some felt that participants should be able to choose whether they receive feedback and in what form. Some argued that researchers should build this into their grant applications and funders should consider the applicant’s past record in this area.
In both the survey and workshops, it was often stressed that feedback is an ongoing process and participants should be actively kept informed of the study’s progress.
Additional suggestions included:
- Create feedback templates for researchers to use
- Provide/improve guidance on how to feedback to participants effectively
- Make feedback to participants a mandatory requirement
- Improve sanctions/penalties for non-compliance with proposed feedback
- Promote and endorse patient and public involvement to improve and create effective feedback methods to participants
- Highlight good examples of feedback.
What we decided to do
To ensure better feedback to participants, we will:
- change the questions we ask applicants from whether they will share study results with participants to how and when they will share them
- ask researchers and sponsors to submit a lay summary of the study results to the HRA, as part of the final report, which we will then publish alongside a summary of the research on the Research Summaries tool
- produce new guidance on how to inform participants about study findings, taking into account the types of research which may make this more of a challenge, such as research involving adults without capacity, emergency research and research in which participants are likely to die from their existing illness.
As part of the implementation programme, we will work with stakeholders to determine how this will work in practice and to draft new guidance.
Sponsors’ compliance with transparency requirements, including whether a lay summary of study results have been submitted, will be highlighted in an annual report.
Publishing an annual "transparency league table"
We asked for views about whether we should publish an annual ‘transparency league table’ highlighting those studies where transparency information is due but has not been submitted.
What we heard
The use of league tables received a high level of support, both in the online survey (69%) and in the workshops.
More patients and the public (81%) supported this approach than those employed to conduct research: research managers (61%), researchers (56%) and Contract Research Organisations (58%).
Many who supported the proposal thought this is a good way to incentivise compliance, prompt allocation of more resources to support transparency and increase accountability. Some suggested that such a table should be used to highlight good performance rather than identify poor performers.
Some cautioned that the publication of tables could deter patients from taking part in research and that ‘naming and shaming’ would have negative implications for individuals and organisations even where there were valid reasons for non-compliance. Others noted that transparency data published in a league table must be up to date and accurate.
Question 8. To what extent do you think the following actions would be appropriate?
Figure 14 - Graph showing responses to question 8a.
Graph summary: The bar chart shows the number of respondents who selected not at all appropriate, not appropriate, I don’t know, appropriate or highly appropriate in response to the question to what extent do you think that following actions would be appropriate: publish an annual 'transparency league table' highlighting individual studies which have information that is overdue. The most frequent response was appropriate which 181 respondents selected, followed by highly appropriate which 157 respondents selected.
Download a csv file for graph data from figure 14.
What we decided to do
We will use information submitted in final reports as a basis for measuring research transparency performance and for taking action in cases of non-compliance.
We will publish an annual report describing our own work to improve research transparency and, once we have the appropriate data collection and monitoring system in place, transparency performance in the research community. We will celebrate good practice and highlight poor performance, by publishing transparency performance about individual sponsors.
Considering past performance
We asked for views about an option to take into consideration the extent to which sponsors have fulfilled their transparency responsibilities in relation to their previous studies, when reviewing new studies for approval.
What we heard
The majority of online respondents (75%) supported this approach with significantly more patients and members of the public (92%) agreeing than research managers (67%), researchers (71%) and Contract Research Organisations (CROs) (37%).
At the workshops people generally felt it would be appropriate for past transparency performance to be considered as part of ethical review as this would be a strong motivator and an important catalyst for compliance. This was also supported by sponsors’ representatives who suggested that preventing studies from being approved at an organisational level would give them increased weight in managing researchers.
Those who thought that this approach would not be desirable felt that blocking future approvals would result in less valuable research being undertaken which, in turn, would adversely and unfairly affect potential participants and patients.
Figure 14 - Graph showing responses to question 8b.
Graph summary: The bar chart shows the number of respondents who selected not at all appropriate, not appropriate, I don’t know, appropriate or highly appropriate in response to the question to what extent do you think that following actions would be appropriate: take into consideration the extent to which sponsors have fulfilled their transparency responsibilities in relation to their previous studies, when reviewing new studies for approval. The most frequent response was highly appropriate which 201 respondents selected, followed by appropriate which 165 respondents selected.
What we decided to do
We will use information submitted in final reports as a basis for measuring research transparency performance and for taking action in cases of non-compliance.
We will take into consideration the extent to which the applicant has fulfilled their transparency responsibilities in relation to their previous studies, when reviewing new studies for approval.
As part of the implementation programme, we will work with stakeholders to determine what information we will use as the basis for reviewing the applicant’s past transparency performance. We will also work with stakeholders to develop a policy for how we take account of past performance when reviewing new studies for approval.
Fining sponsors with very poor transparency compliance rates
We asked for views about whether we should impose a financial penalty on sponsors with very poor transparency rates, noting that it would need a change to the law to make this possible.
What we heard
Response to this option was more divded than for other options for sanctions. 47% of online respondents did not support this sanction whilst 39% did. There was greater support from patients and the public (57%) completing the online survey for the use of fines than those employed to conduct research such as research managers, researchers and Contract Research Organisations (CROs) (30%).
At workshops there was generally less support for fines than other sanctions. Whilst there was a strong feeling that the use of fines would ‘focus minds’ many cited a lack of evidence that fines would promote compliance or address current barriers to compliance and, instead, would damage good will, alienate the research community, and potentially make the UK a less attractive place to do research.
All participants of the patient focus group thought, initially, that the introduction of fines was reasonable. However, following a discussion about the disproportionate impact this sanction might have on non-commercial organisations, the practicalities and costs of imposing, collecting and enforcing fines, who the money would belong to and how it would be spent, opinion in the group changed. By the end of the session it was unanimously agreed that, whilst the publication of league tables and considering past performance were appropriate, the introduction of fines was not.
Figure 16 - Graph showing responses to question 8c.
Graph summary: The bar chart shows the number of respondents who selected not at all appropriate, not appropriate, I don’t know, appropriate or highly appropriate in response to the question to what extent do you think that following actions would be appropriate: fine sponsors with very poor transparency compliance rates (this would require a change in legislation). The most frequent response was not appropriate which 156 respondents selected, followed by highly appropriate which 102 respondents selected.
What we decided to do
We have decided not to seek the power to impose financial penalties. Following advice from the Research Transparency Strategy Group, we will focus instead on making performance information public and taking into account past performance when reviewing new studies for approval.