Advanced cardiovascular risk prediction in the acute care setting
Research type
Research Study
Full title
Optimising diagnostic efficiency in the Emergency Department by using advanced machine learning methods to update and personalise a contemporary clinical prediction model for early identification and exclusion of acute coronary syndromes and long term cardiovascular outcomes
IRAS ID
244799
Contact name
Richard Body
Contact email
Sponsor organisation
University of Manchester
Clinicaltrials.gov Identifier
NIHR300246, NIHR Doctoral Research Fellowship; G70292, Manchester NHS Foundation Trust - Data Driven Healthcare Award
Duration of Study in the UK
2 years, 0 months, 1 days
Research summary
"Research Summary"
This study will examine retrospective databases and seek to improve the short term and long term treatment of cardiovascular disease.------
The Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aid allows clinicians to rapidly “rule out” the diagnosis of a heart attack when patients present to the Emergency Department with chest pain or similar symptoms. Research from over 4,000 patients has shown the algorithm is accurate, allowing over 40% of patients to be safely reassured following a single blood test that they are not having a heart attack and discharged from A&E. This could save the whole NHS over £100 million per year.
Research suggests that locality and different patient populations may have an important impact on the accuracy of the T-MACS algorithm. In addition, as patient demographics and standards of care evolve with the passage of time, the accuracy of the algorithm is likely to decline (a phenomenon known as 'calibration shift'). T-MACS has taken 14 years and seven clinical studies to be clinically ready. Repeating this would be overwhelmingly expensive, resource intensive and unfeasible.
------
Long term heart disease risk factors are routinely measured in the emergency department, and we hypothesise could be used to predict long term risk. The potential is huge, with 23.4 million patients attending ED each year, ED attenders are less engaged with primary care and Greater Manchester has double the national average of preventable heart disease deaths.
In this work, we will use routinely collected clinical data to refine the T-MACS algorithm and examine long term risk factors. We will determine the outcome of patients by linking the data with Hospital Episode Statistics, a national NHS database. We will use machine learning techniques to identify the best way to continually update T-MACS, and cox and logistic regression to analyse the long term outcomes.
"Summary of results"
Background
People attending the Emergency Department with chest pain may be having a heart attack, but many are not. Doctors need to identify patients who can be safely reassured and discharged, while ensuring that people who are having a heart attack receive prompt treatment.TMACS stands for the Troponin-only Manchester Acute Coronary Syndromes decision aid. It helps doctors estimate the likelihood that a person with chest pain is having a heart attack. The tool combines information about symptoms, examination findings, an electrocardiogram (ECG), which is a recording of the heart’s electrical activity, and a blood test for troponin, a protein released into the blood when heart muscle is damaged.
This research examined how well TMACS worked in routine National Health Service (NHS) care and explored ways to improve its performance over time.
What was done?
The study used routinely collected NHS data from 13,207 episodes of care involving people who attended Emergency Departments in Manchester, Blackburn and Burnley with chest pain where clinicians suspected a heart attack. The TMACS assessments were linked with national NHS records to find out whether patients were diagnosed with a heart attack or experienced another serious heart-related event within 30 days.The work had two parts:
Part 1 evaluated how well TMACS performed in real-world clinical practice.Part 2 tested three ways of updating TMACS to improve its predictions.
Part 1: How well did TMACS perform in clinical practice?
Overall, TMACS was good at separating patients who did and did not go on to have a heart attack.
About 42% of patient episodes were classified as very low risk. Among this group, 99.7% did not have a heart attack within 30 days. This suggests that TMACS can help identify some patients who may be suitable for early discharge, although it should support rather than replace clinical judgement.
However, TMACS often estimated a higher risk of heart attack than was actually observed. This means that some people may have been placed in a higher-risk category than necessary.
The researchers also examined performance in different groups. Results varied according to hospital site, age, sex, ethnicity, kidney function and level of social disadvantage. The clearest difference was seen in people with reduced kidney function, for whom the model performed less well. This may be important because kidney disease can affect troponin levels even when a heart attack has not occurred.
Performance also varied over time. There was no clear evidence that TMACS steadily became worse at distinguishing between patients with and without heart attacks. However, the accuracy of the percentage risk estimates changed over time.
Main finding from Part 1
TMACS remained useful for identifying patients at very low risk, but its percentage risk estimates were often too high and its performance was less reliable in some groups, especially people with reduced kidney function.
Part 2: Can TMACS be improved?
The second part of the research tested three approaches to updating the decision aid.
1. Recalibration
Recalibration means adjusting the existing model so that its risk estimates better match what is now seen in NHS practice. This approach improved the agreement between predicted and observed risks and improved risk estimation across several patient groups, including people with reduced kidney function. However, the recalibrated TMACS would not have allowed any patients to be discharged from hospital after their first blood test, meaning that its use would probably increase length of stay and the number of unnecessary hospital admissions.
2. Adding further patient information
The researchers tested whether adding age, sex, ethnicity, kidney function and level of social disadvantage would improve TMACS. Adding this information improved some aspects of the percentage risk estimates. However, it reduced the model’s ability to distinguish between patients who did and did not have a heart attack.
3. Continuous or “dynamic” updating
Dynamic updating means allowing the model to adjust as new patient data become available. This approach broadly maintained the model’s ability to distinguish between patients with and without a heart attack and improved the agreement between predicted and observed risks over time. Results differed between hospitals, showing that local patient populations and clinical practice may matter. The approach is promising, but safe implementation would require suitable digital systems, monitoring and human oversight.
Overall conclusion
This research evaluated TMACS using more than 13,000 Emergency Department episodes across three NHS hospitals. TMACS remained effective at helping identify people who were unlikely to be having a heart attack, which is the main purpose of the decision aid. However, the study also showed that its percentage risk estimates were often too high and that performance varied in some patient groups, particularly among people with reduced kidney function. This highlights the potential for future improvement.
The research showed that TMACS can be improved through updating. Recalibration is a promising method, but further work will be needed to make sure it doesn't lead to an increase in unnecessary hospital admissions for further tests. Dynamic updating also showed promise as a way for clinical decision aids to adapt when patient populations, tests or clinical practice change.
These findings may support the future development of safer and more accurate decision support for people attending Emergency Departments with chest pain. Further testing, monitoring and appropriate oversight would be needed before an updated or continuously learning version of TMACS could be introduced into routine care.
REC name
Wales REC 7
REC reference
19/WA/0311
Date of REC Opinion
9 Jan 2020
REC opinion
Further Information Favourable Opinion