Most Viewed


  • No categories

Augmented Reality in Medicine – Science-Based Medicine

Medicine is often a game of trade-offs. That is why clinical decision-making is often about risks vs benefits; rarely do you get the latter without the former. But there are also occasional win-wins in medicine, where an advance is a clear improvement without any significant downsides. Often technology is what drives such improvements.

It is interesting to look ahead to think about what technologies are going to change the practice of medicine, but it is also very difficult to predict. That doesn’t stop us, however.

One technology that seems poised to have a significant impact is augmented reality, using displays that are worn in front of the eyes and overlay information onto the world in front of you. Google Glass was the first big push for this technology, and it famously flopped, but mainly because of social reasons. Use in a professional context is different.

What is the potential for AR in medicine? A recent article in the Harvard Business Review discusses the potential of AR for surgery. This is certainly one potential use, and may turn out to be a primary use.

One clear technological advance in medicine in the last few decades has been imaging technology, which has further benefited from advances in general computer technology. The real-time detailed images we can produce now have transformed the practice of medicine in many areas. As a neurologist, for example, I can now get an MRI scan on a patient in the emergency room and determine fairly definitively if they are having a stroke.

Surgeons also have lots of anatomical data from images at their disposal, but they are mostly looking at 2-dimensional displays that they review prior to surgery, or have to look away from the patient during surgery. AR has the potential of providing a heads-up display, giving vital information to surgeons overlaid on the patient.

At the very least this allows for accessing information without having to look away from the operating field.

As the technology matures it is also easy to imagine 3-D anatomical information actually overlaid on top of the patient. Now combine this with artificially-intelligent algorithms that can image the patient in real time, and correlate the operating field with prior imaging as well as anatomical information.

For example, vital anatomical structures can be identified in the operating field and highlighted in AR. Even in my very limited experience in the operating room as a medical student, I witnessed errors where a surgeon cut something they weren’t supposed to cut. It happens. The tangle of anatomy, obscured by blood and connective tissue, can be challenging to interpret. Any help can potentially reduce such errors.

Imagine nerves, major blood vessels, and ducts highlighted and color-coded. Imagine the tumor similarly outlined in AR, showing its relationship to surrounding vital anatomy.

None of this is far-fetched. AI algorithms are already doing amazing things with photographs and video. These applications are not out of reach of existing technology, they just need to be developed.

Outside the OR, augmented reality also has potential. The electronic medical record (EMR) is now ubiquitous in medicine. Initially we received many complaints from patients that their physicians were looking away from them to engage with a computer. There was a bit of a learning curve there, for both patient and doctor. Patients learned that this is now the reality of modern medicine, and there are tangible benefits that are worth the small sacrifice. Doctors learned how to interface with the computer without being rude to their patients (it’s a bit of an art).

But imagine if the doctor had access to the EMR through AR, and did not have to turn away from the patient to look at a monitor. I actually don’t know if this would be an advantage or a disadvantage, because a new set of social skills would be required (the kind that doomed Google Glass), but I would be interested in trying it.

Emergency rooms and intensive care units might be more appropriate locations for AR access to the EMR and imaging and other important data. Doctors in those settings are expected to be focused on the task at hand – it is an emergency, after all.

And again, combining this with a medical AI that can anticipate the informational needs of the physician could be huge. While discussing medication choices, the patient’s allergies will display in red, or any potential drug-drug interactions, for example.

Finally the potential for training is also enormous and largely untapped. There are simulated patients for training available, but both AR and VR (totally immersive virtual reality) have untapped potential. The idea would be to simulate patient encounters in AR or VR, and for doctors to essentially make all of their mistakes on virtual rather than actual patients.

The other advantage to this kind of training is that it can be more systematic, rather than random. A training program can ensure that doctors in training see everything they need to see, rather than relying on luck of the draw.

The incredible potential of AR and VR to improve the practice and training of medicine also represents the double-edged sword of technology in medicine. High tech advances have clearly improved outcomes in medicine. Anyone who has had endoscopic surgery rather than open surgery can attest to that.

But at the same time technology has made medicine feel less personal. As a profession we struggle to integrate the human warm-and-fuzzy aspects of medicine with the advantages that modern technologies bring. It is worth integrating the two, and when you really think about it no rational person would forgo the advantages of modern technology in medicine.

But that is actually what is happening. The hard won benefits of science and technology in medicine are often dismissed as being cold and calculating, or part of some big corporate conspiracy, and more “natural” or hands-on alternatives are offered, often with false promises about their efficacy. The hard edges of medical technology are used as an opening for gurus and charlatans to lure patients away and sell them magic.

It is a dilemma we have to face, and we need to be thoughtful about how our patients experience their interaction with the medical system, including all the high-tech gadgets. We also need to communicate to the public and to our patients to make sure they understand what all the technology is about, why we use it, and what advantages it has for them. And of course we have to remember not to neglect the human side of medicine.

None of this is to suggest that pseudoscience in medicine is the fault of science-based practitioners. It is the fault of charlatans, inadequate regulation, and ideology. But we certainly don’t want to give them any openings if we can easily close them.


Forgot Password