HRA User Satisfaction Report - April to September 2023

Findings from the online user satisfaction survey

the following graphs present quantitative data collected between October 2022 and September 2023 using the HRA online user satisfaction survey. This report does not include the free text qualitative comments - these are reported to the management teams in monthly reports
for the last reporting period (April to September 2023) 303 respondents completed the survey. Response rates have slightly decreased from the last reporting period, with 308 respondents completing the survey for the reporting period October 2022 to March 2023

HRA Overall Service

Figure 1 - overall mean satisfaction scores for the period October 2022 to September 2023.

Figure 1 User Sat Report Apr to Sep 2023 for website.png — Figure 1 - overall mean satisfaction scores for the period October 2022 to September 2023

Figure 1 maps the mean overall service satisfaction score for each month over the last year (October 2022 to September 2023). The chart details a relatively stable mean score - with the highest score in December 2022 (8.4). The level of satisfaction in September 2023 (8.0) was higher than at the beginning of the 12 month period in October 2022 (7.4) which was also the lowest score during the period.

Download a csv file for graph data from figure 1.

Users' experiences of different aspects of HRA Services

Figure 2 - mean satisfaction scores broken down by category for October 2022 to September 2023.

Figure 2 User Sat Report Apr to Sep 2023 for website.png — Figure 2 - mean satisfaction scores broken down by category for October 2022 to September 2023.

Figure 2 details the mean scores for the following areas: queries line, HRA website, IRAS usability, IRAS guidance, online guidance, decision tools and staff, over a 12 month period from October 2022 to September 2023. The mean scores fluctuate during the 12 month period in all areas. The highest mean score was for Staff in December 2022 at 9.3 and Queries Line in March 2023 at 9.3. The lowest mean score was for IRAS Usability in April 2023 at 6.6.

Download a csv file for graph data from figure 2.

Review and action taken in respect of the issues raised in the user satisfaction feedback report for the period April to September March 2023

The following section outlines the review and action taken by the HRA functions in respect of the issues raised by applicants through user feedback

HRA Approval

Trend	Response
Feedback that the Research Ethics Committees (RECs) did not hold expertise in particular topic areas and a perception that the members didn’t understand particular aspects of the study	Options for how the membership of RECs is described are currently being developed as part of the opportunities afforded by the new clinical trials regulation. As part of this, Chairs have discussed what sort of input is required from ‘expert’ members of RECs. Ensuring that our terminology is clear is one of the principles of this work in order to help avoid applicants expecting expert members of RECs to be subject matter experts. In terms of ensuring that relevant expertise is available to the REC when specialist advice is required to support the ethics review, we continue to appoint new members with relevant experience to flagged RECs related to their specialism where it is possible to do so, as well as recording information about topic expertise in the membership record in HARP which can be searched by staff.
Feedback that applicants were surprised at the number of queries raised in correspondence after the meeting as opposed to being discussed in the meeting itself	A new workbook ‘Essential information for REC Chairs’ was launched over the summer and includes information about ensuring that significant questions are raised in the meeting. If less substantial issues such as small document changes will be referred to in correspondence, Chairs are asked to inform the applicant of this. Some of the projects being initiated by the Operations Team (the quality standards and design & review principles for participant information, changes to the ethics review form, and changes to how PR applications are screened to ensure efficient use of full REC meeting slots) could all help with ensuring that the discussion with the researcher is focussed on the ethical issues and could help with structuring requests for further information. This will be considered as part of the evaluation of these pieces of work and when discussing the impact of these projects with REC Chairs.
Specific comments about the conduct and attitude of particular members	Where permission was provided by the respondent to do so, specific comments have been followed up with the REC Chair. The collated comments from the last six months of user satisfaction reports are being tabled at the autumn round of Chairs meetings in order that Chairs can reflect on and consider both the positive and negative comments received.
Communication delayed	Communication for combined review studies was perceived as being delayed because of delays with the Medicines Healthcare products Regulatory Agency (MHRA) issuing REC decisions. To assist with this, the provisional opinion was released to applicants as soon as this was available.
Studies not suitable for PR	Several studies booked for proportionate review (PR) were deemed not suitable and transferred to full meetings which caused frustration with applicants. A pilot was run over the summer to remove PR screening after booking to determine the effects on applicants and REC review, analysis remains ongoing but positive and changes along with guidance updates are being considered.
Requests for information are not clear	Several comments relate to requests for further information not being clear, further analysis and training has occurred with the team to assist with creating greater clarity. Work is also ongoing to streamline the ethics review form completed by REC members in advance of the meeting to further assist with minute taking and clarity in requests for further information.

Integrated Research Application System (IRAS)

Trend	Response
You told us that the question sets on IRAS are long, repetitive, and not always suitable for all types of studies	We are working to update the question set as one element of our wider systems development work. We aim to structure the questions so that users only need to answer questions that are applicable to their study types and eliminate duplication. We are committed to making our systems user-centric and have already carried out user research on the question set. We are using this information to inform further alterations to the question set. We will continue to seek user feedback as it develops
You told us that the IRAS system is not particularly user friendly or intuitive and poses difficulties for those who have not used it before	We have started work on developing a system that supports a simpler, more co-ordinated research journey. Our vision is to create a simple to use, accessible system that meets the needs of all users
You told us that you find uploading documents confusing and time consuming	Back in March 2023, we carried out user research to understand how stakeholders would like the document functionality to work in the future. We will use this feedback to inform future system developments

HRA guidance and advice

Trend	Response
Several people across the six month time period said that they found our guidance easy to use and helpful. In some cases it was highlighted that the guidance helped to support people through complex processes and systems	We appreciate the positive comments about our guidance. As our systems and processes develop and improve in the future we expect that less guidance will be needed to support applicants
You said that our guidance is too wordy and not user friendly. You made some suggestions for improvements that could be made but acknowledged that the platform is old	We are working through our existing guidance in IRAS to try to simplify the language we use. We are also looking at restructuring content to make it easier to use, within the constraints of our current platform. The longer term ambition is to develop a new platform for guidance, which will enable us to deliver more substantive improvements to our content
Some of you said that you found it hard to find the guidance that you needed and that it was not clear where to start with systems and processes	Our ambition to develop a new platform for guidance is so that we can bring all the guidance into one place and improve how it is organised, presented and connected. Alongside this, users will be at the heart of our future developments, which will help us design our systems and processes to be more intuitive

Website

Trend	Response
The HRA's research systems are aging and do not always meet user expectations (comments on accessibility and ease of use)	We will proactively communicate our plans to improve our systems, the change that this will bring and when. We will engage as many users as possible in helping to make sure that the new systems are fit for purpose and meet their needs.
It is confusing when you need to move between websites to complete tasks (for example the student toolkit)	We will monitor feedback on user journeys and proactively promote where to find guidance and advice (for example via a social media campaign). We will make sure that we are using web analytics to identify where improvements could be made to user journeys.

Confidentiality Advice Group (CAG)

Trend	Response
Confusion from HRA staff on CAG outcome and whether HRA Approval can be issued	We are continuing to promote alignment of teams and encourage staff to discuss and resolve specific issues internally. We are also developing some CAG training specific for HRA staff to help embed knowledge on the CAG process and what it means for their role
Duplication of effort relating to CAG and REC information	This was part of a coordinated CAG/REC pilot where the CAG form is testing new questions outside of IRAS. As part of research systems development, we expect that information relevant for both CAG and REC are asked once
Praise of helpful and timely CAG review	Positive feedback is fed back to staff and members

REC and CAG Chairs and members' feedback

Between April and September 2023:

27 Chairs or members submitted completed feedback forms to the Quality Assurance (QA) department for the reporting period; all were forwarded to the Health Research Authority (HRA) / Health & Care Research Wales (HCRW) / Devolved Administration (DA) staff for consideration and response
the comments received highlighted a range of topics including complimentary comments in relation to Staff, Chairs and members, comments relating to remote REC meetings and workload, as well as the reorganisation of administrative staff
all responses to feedback have been sent out for the reporting period