Five take aways on a history of automated ECG interpretation
Follow-up to: Technical debt: probably the main roadblack in applying machine learning to medicine
As a friend points out, the question "can a computer even accurately diagnose an ECG reading?" is one of the most common questions that medical students and doctors ask when the topic of machine learning in medicine comes up.
With that in mind, I found Pentti Rautaharju's recent take on the history of automated ECG interpretation very enlightening. Here are my five take-aways:
In 1991, a database of 1220 ECGs on clinically validated cases of myocardial infarcations, ventricular hypertrophies, and combinations of the conditions was used to compare 9 ECG programs to 8 cardiologists. Four of the programs were within 10% of the 67% accuracy of the reference cardiologist, and the best program had 3% higher accuracy. Notably, this doesn't mean that the program was actually better, because we need to take into account multiple hypothesis testing. It does seem to warrant a repeat test, however.
In attempting to follow-up on #1, I found the dearth of healthcare provider vs computerized interpretation comparisons in the literature is surprising. I haven't been able to find many in a search on PubMed. Instead, there's a fairly large literature of comparisons of people's ECG interpretation accuracy at different stages of training. Clearly there will be many ways in which computers are worse than providers (e.g., for rare diseases), but it's important to know when, where, and why.
It was interesting to me, although in hindsight not surprising, that the use of most ECG interpretation programs has been through via major ECG manufacturers. It's disappointing that the methods and accuracy of these were often not published, so that best practices couldn't be adopted by other teams far away. Theoretically, software is scalable, but it doesn't seem to have been harnessed in that way in this context. Related question: why is so little software in medicine open-source?
The correction of computer errors in computerized ECG interpretation is called overreading. Medicare only adds $8 for overreading an ECG, and overreaders are often not available locally. Rautaharju suggests, quite interestingly, that it'd be possible for a 24-hour service to exist online where ECG overreading could be performed.
Another area in which computerized approaches have a potential advantage -- because they are faster -- is in serial ECG interpretation. This speaks to a general point: the more longitudinal data available on a patient, the more value a machine learning-inspired approach is likely to be able to provide.
Reference
Rautaharju PM. Eyewitness to history: Landmarks in the development of computerized electrocardiography. J Electrocardiol. 2015;