Automated Feedback on Psychological Therapies
Research type
Research Study
Full title
The Feasibility and Acceptability of Automated Feedback on Session Contents in Psychological Therapies for Anxiety and Depression
IRAS ID
338066
Contact name
Sam Malins
Contact email
Sponsor organisation
Nottinghamshire Healthcare NHS Foundation Trust
Duration of Study in the UK
0 years, 4 months, 31 days
Research summary
Background
One psychological therapist can be very different from another in how helpful they are for their patients. The most effective therapists can have recovery rates 10 times the average; the least effective therapists can actually make their patients’ problems worse. Therapists also do not, on average, get more effective with time or experience. This means patients cannot necessarily rely on the overall effectiveness of psychological treatments because there are such big potential differences among therapists they might see. There are few proven methods for helping psychological therapists improve their effectiveness, but the recently developed approach of deliberate practice shows promise in supporting improvement. Deliberate practice involves having skills observed, receiving corrective feedback on individualised skill improvement opportunities and then practicing skills separately from patient care. A current research development has used artificial intelligence to help identify parts of therapy sessions that are helpful for improved outcomes and those that are hindering. This automated tool can support deliberate practice by helping identify the specific skills that require improvement for individual therapists.Aims
This study aims to assess the feasibility and acceptability of a deliberate practice training intervention, supported by automated feedback on therapy session content.Methods
Participants will be 9-12 psychological therapists from NHS services. The study will involve three hours-per-week training over a 10-week period. The deliberate practice intervention will involve a combination of individual practice, one-to-one supervision, and small group practice each week. Patients seen by participating therapists will also be asked to participate in the study. Participation will involve their sessions being audio and/or video recorded then securely and anonymously coded by an automated tool to support identification of helpful and hindering aspects of their therapists’ practice to aid improvement in their skills.The results of this study will be used to inform a randomised controlled trial of the intervention.
Lay summary of study results: Why this study was needed Therapists can differ a lot in how much they help their patients. The best therapists can get much better results than the average, and the least effective therapists can sometimes make things worse for their patients. Time and experience alone does not make therapists better. This means patients sometimes cannot rely on the effectiveness of psychological treatments because there can be such big differences among therapists they might see.There are few proven ways to improve therapist effectiveness, but the recently developed approach of deliberate practice shows promise in supporting improvement. Deliberate practice is where a therapist’s skills are observed, they get focused feedback and then practice specific skills away from patient sessions. A newly developed automated tool uses artificial intelligence (AI) to predict whether a therapy session is likely to lead to a patient’s improvement. This tool could support deliberate practice by helping identify the specific patients and sessions where improvement is needed.
What we wanted to find out
We tested whether a training programme for therapists that included deliberate practice supported by automated feedback on therapy session contents could be used in NHS services, was acceptable to people, and helped therapists improve their skills and patients’ outcomes.Who helped design the study
A Patient and Public Involvement (PPI) group of 6–8 people with lived experience of psychological therapy advised the team at every stage. Two public contributors were members of the research team. We also consulted four stakeholder groups: trainee therapists, therapy trainers, practising therapists and senior managers of therapy services.When and where the study took place
Recruitment began on 16 April 2024 and the last person finished their final study assessment on 3 July 2024. Participants were recruited from an NHS Talking Therapies service (Lincolnshire Partnership NHS Foundation Trust) and specialist therapy services (Nottinghamshire Healthcare NHS Foundation Trust).What we did
Participants were 97 patients seen by nine participating therapists in the 10 weeks before the intervention, and 79 patients seen by the same therapists during the 10-week intervention period. Participating therapists represented a range of professional backgrounds - including clinical, counselling and health psychologists, alongside Cognitive Behavioural Therapists. Therapists invited their patients to consent to have sessions recorded and automatically assessed for predicted prognosis (likely improvement in anxiety and/or depression, or not). The feedback from the automated tool was used as part of the deliberate practice activities. The total amount of intervention activity was 32 hours per therapist. We measured whether the intervention was feasible and acceptable, whether people found it credible, changes in therapist skills, and patient outcomes. We also carried out interviews with therapists at the start, at 5 weeks and at 10 weeks follow-up.What we found
• As therapists used the programme, they found it more acceptable and credible.• Therapists’ therapeutic skills improved significantly over the 10 weeks.
• There were trends of improvement in patient outcomes: the proportion of patients reporting improvement in anxiety increased from 65% before the intervention to 75% during it (a 10% increase). For depression the proportion rose from 58% to 61% (a 3% increase).
• However, relatively few patients agreed to have automated feedback used for their sessions. Many patients had concerns about AI and about trusting the system.
What this means
Using automated feedback together with deliberate practice can become more acceptable as therapists use it. This intervention has potential to improve therapeutic skills and clinical effectiveness. However, addressing patient concerns about how technology is used for automated feedback is essential to increase participation.Future plans: The research team will use these results to improve the intervention and plan a larger, randomised controlled trial to test whether it works for more therapists and patients.
REC name
East Midlands - Leicester Central Research Ethics Committee
REC reference
24/EM/0073
Date of REC Opinion
26 Mar 2024
REC opinion
Further Information Favourable Opinion