As part of a group project for one of my graduate courses, I presented an IT solution that automates a highly error prone and time intensive laborious health care process done by pharmacists in the hospital. The IT solution we proposed used Optical Character Recognition to read and recognize faxed medication files.
“As you know Optical Character Recognition is not perfect. It may be 80 / 90% accurate but that is not enough when it comes to health care”
This comment from one of my instructors is what inspired the opinion piece that follows.
This line of reasoning is a theme I have come across in my classes time and time again. The idea that a computer may be able to do something, but since it can still make mistakes, then we can’t automate this process.
Let me begin this piece by saying that I do not think that computers are flawless and acknowledge that they do not make mistakes.
We have all seen news stories of horrible freak accidents that Tesla’s self driving cars have been involved with.
However, let us not forget the thousands of lives that are lost every day by aggressive / drunk / angry / moody human drivers.
Those accidents do not get as much news coverage, because everyday events are not newsworthy.
I would argue that if we can properly study something like self driving cars to understand their crash rate, we need to benchmark their performance not to perfection but to human level performance.
Humans are way more prone to error. We are busy people that live extremely busy lives. For one reason or another we may be running late and driving too fast, on our phones and not paying attention, maybe we had a fight with our significant other earlier on that morning and are extremely angry… The list of reasons goes on.
At the end of the day, a human is way more likely to make an error than a computer is, in many scenarios and if that is the case we need to let computers compete.
Another important reason to keep in mind is, gradual improvement.
If a human driver has been commuting to work for ten years, on any single day he is still quite prone to the many distractions of the world. However, computers, using deep learning, get significantly better with more time and more data being collected.
Let’s go back to the health care setting and the IT solution I presented.
A pharmacist may have to look through 20 pages of faxed documents to compile a list of medications for a single patient. This can take the pharmacist anywhere up to twenty minutes.
A computer can do this exact process in less than a second using optical image recognition and simple processing power.
A quick search of the literature shows that the latest deep learning powered OCR systems are more than 90% accurate when validated.
Is this a reasonable accuracy? Can we trust this system?
I would argue that the most important question to pose when answering that question is:
How accurate is the human alternative?
There has been no scientific studies to look at the error rate of pharmacists in this particular scenario so it is hard to say.
However, we know that there are medication discrepancies found in 76% of cases when a patient is discharged from the hospital. We also know that 1 in 9 ED visits are due to drug-related adverse events.
That is to say, humans are far from perfect.
A pharmacist has an extremely busy day with dozens of patients on a hospital floor. They should be focusing on doing clinical work and not reading faxed information.
They are more likely to make a mistake than a computer is.
The same principle applies to many other health care settings.
A team in Stanford, developed a deep learning based tool that outperforms cardiologists in detecting atrial fibrillation and other heart arrythmias.
The model was only 80% accurate.
In comparison: a single cardiologist was accurate only 76% of the time.
Who would you want to diagnose your atrial fibrillation? The computer, will be many times more consistent than any one individual human. The most expert cardiologist or a team of cardiologist, may outperform the model by discussing the patient results, but this is not a reasonable alternative.
Don’t forget that this was with limited working data. If we put the model to work in diagnosing Afib across the country, it’s recognition will only increase.
I am not arguing that we should automate everything in health care or that computers are going to replace your doctor tomorrow or anytime soon. But, it is always important when evaluating a new technology in the health sector to ask yourself one question:
How accurate is the human alternative?
Source: Deep Learning on Medium