Base Hospital #5 by Kentucky National Guard Public Affair’s Office available under Attribution 2.0 Generic (CC BY 2.0).
By Caroline Collins
Consider a world in which each individual possessed x-ray vision. Instead of having to walk to the fridge to open it and view its contents, one could view the inside from his couch to see that, as usual, he was out of coffee creamer. While x-ray vision might evoke a sense of wariness among many, it would still prove to be beneficial in many different aspects of life, such as occupations. Research and advancements throughout the last two decades have resulted in the development of augmented reality (AR), which, in certain scenarios, resembles x-ray vision. Augmented reality is defined as technology that allows a viewer to superimpose computer-generated images on the real world, and view those images through some sort of monitor or lens (Fuchs 2006). Specifically, researchers aim to utilize augmented reality in surgical operating rooms. Within the medical field, augmented reality would allow a surgeon to view the internal anatomy of his patient hovering above the patient’s body in real time, much like x-ray vision (Azuma 2006). Ultimately, the implementation of augmented reality will enhance the operating room by causing surgeries to be safer, providing more comprehensive opportunities for surgeon training, and aiding in the precision and accuracy of operations.
What is Augmented Reality Anyways?
Augmented reality can be broadly defined as the superimposition of computer-generated images on objects in real life; however, in order to fully comprehend the benefits AR can have on the operating room, it is imperative that one has a more meticulous understanding of how augmented reality works. In order to develop the images to project over a patient, technicians enter data that will generate a 3D image on a computer (Shuhaiber 2004). For example, a technician can use the data from an MRI or CT scan to construct a corresponding image on a computer. Then, the data that was used to create the computer image is used to develop stereoscopic images. These stereoscopic images are not only fundamental in the development of AR technology, but also in any 3D imaging system. The images create a 3D illusion by displaying two identical, but slightly offset images on top of one another. This enhances the 2D image’s depth, causing it to appear 3D (Yang 2015). These image are projected into real time, then viewed through an optical see-through lense, which is essentially glasses or goggles that allow the surgeon to view the projections while maintaining full mobility of his head and neck (Shuhaiber 2004). Augmented reality is not a replacement of real images, but rather supplements the information that surrounds a surgeon in the operating room. Despite the complexity of augmented reality, it has the potential to greatly simplify surgeries.
How it Helps
First, augmented reality will reduce many of the risks associated with minimally-invasive surgeries. Although the aforementioned procedures are significantly safer than larger, more complex surgeries, physicians often encounter challenges with less severe surgeries that augmented reality can correct (Azuma 2006). For example, surgeons only need to make small incisions to perform these surgeries; however, this limits their vision of the internal anatomy of the patient they are operating on, complicating the procedure. Augmented reality offers a solution to this dilemma by providing a physician with a 3-D, anatomically correct view of the individual patient’s internal anatomy (Shuhaiber 2004). This image would hover over the patient’s body as the surgeon operates. Essentially, the surgeon can view the inside of his patient’s body as it is projected in front of him to confirm where he is operating and what he is operating on, but he still only needs to make a small incision. The reduced trauma from surgery will result in a shorter hospitalization and recovery period for the patient (Azuma 2006). This is only the first aspect of surgery that augmented reality could improve.
In addition, augmented reality can be used to train novice surgeons. Rather than having to look away from the patient to consult manuals for the steps of surgeries, those steps could be projected through AR into the visual field of the surgeon. This allows the surgeon to remain focused solely on the patient and the monitors that are being projected through his lens. Augmented reality can also be used to identify various organs and tissues during surgery, reinforcing the operator’s memory of the anatomy of an individual. Here is an image of how this might work. More specifically, AR could be used to identify structures that are defective or abnormal. Consider Transposition of the Great Arteries, a birth defect that causes the aorta and pulmonary artery on the human heart to be reversed in their location. AR could project images that identifies the reversed structures, acting as insurance to ensure training surgeons are cognizant of every aspect of an individual’s anatomy, including his or her anatomical differences (Azuma 2006). Ultimately, this will aid a surgeon’s transition from the academic environment to the clinical environment.
Technological Advancements
Researchers have recently developed technologies that recognize pre-programmed code on clothing, and project 3D, anatomically correct images of the structure that is being scanned. This link will show you exactly what this looks like. This technology can be made available to use on cell phones, and exponentially increases both the availability and functionality of information for surgeons to study (Khor 2016). While cadavers have historically been the most viable source of study for surgeons, augmented reality provides a more thorough, readily available alternative. The only limiting factor when studying anatomy using AR is the computing power available (Khor 2016).
Furthermore, researchers have launched an inaugural program that broadcasts surgeries live, from the point of view of the patient. This trial was initiated at the Royal London Hospital, and has brought significant media attention to the benefits of augmented reality. To broadcast these surgeries, images are projected above a patient as in any operation using AR; however, the experience merges virtual reality with augmented reality (Khor 2016). Essentially, the virtual images of the patient’s anatomy and the surgery are recorded from the perspective of the patient, and can then be replayed at any time (click here to see what this looks like). This has significant implications for medical professionals in underdeveloped countries who do not have the resources required for extensive education and training (Azuma 2006). Approximately 5 billion people do not have access to safe and affordable surgeries not only because of inadequate resources, but also because of inadequate surgical training. However, recorded surgeries using augmented reality will allow surgeons in underdeveloped countries to have a step-by-step guide that can be viewed through AR lenses to learn and perform various operations. This will result in increased access to life-saving operations for those who would otherwise have to go without (Khor 2016).
How Augmented Reality Affects Surgical Tasks
Augmented Reality also provides surgeons with the opportunity to superimpose images into a patient’s body for precision tasks like drilling holes into skulls, making incisions, etc. This would allow the surgeon to see exactly where he or she is to make certain incisions, eliminating any need for estimation, and by extension potential error. Not only does this aid surgeons who are still training their own fine motor skills and recognition of where to perform various precision tasks, but it also aids practicing surgeons in perfecting their incisions (Azuma 2006). This aspect of augmented reality would undoubtedly improve the accuracy of surgeries. Additionally, AR has the capability to assist surgeons in registration, or the ability to identify and manipulate spatial information. Registration is a fundamental aspect of surgery, but can be enhanced by the addition of AR. The images projected through an augmented reality lens would provide surgeons with a full view of all the information they are working with and around, and the resources to view it from all angles (Fuchs 2006). This would.
Finally, augmented reality is incredibly useful in pre-operative planning. The ability of surgeons to completely immerse themselves in a patient’s internal anatomy through this 3D modelling allows them to assemble a detailed, anatomically correct plan for any surgery. Surgeons can also anticipate any abnormalities or challenges they might encounter in each individual surgery (Khor 2016). For example, virtual vascular endoscopy allows surgeons to record images of the interior of veins and arteries, viewing all their complexities. If a surgeon is preparing to treat atherosclerosis, or plague buildup blocking arteries, he can use images that are recorded and projected from the virtual vascular endoscopy to view the inside of the arteries he or she will be operating on, before beginning the operation (Khor 2016). This capacity to conduct such thorough, patient-specific pre-operative planning would undoubtedly improve the surgical field.
Augmented reality has the potential to revolutionize surgical medicine. Not only does it provide the technology to make operations safer, but it also radically enhances the training available for novice surgeons, and improves the precisions of surgeries. Medicine is at a crossroads where traditional practices are met by cutting edge technological advancements (Fuchs 2006). It is clear, however, that augmented reality presents numerous benefits to the medical field. The ability to view the internal anatomy of a patient from all angles in real time is unprecedented. Surgeons are better prepared now with AR for both simple, minimally invasive surgeries and complex, high-risk surgeries (Shuhaiber 2004). Although x-ray vision is not possible, it is superseded by augmented reality in the surgical field.
References
Azuma, Ronald T. 2006. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments. 6(4): 356-358.
Fuchs, Henry. Livingston, Mark. Raskar, Ramesh. Colucci, D’nardo. Keller, Kurtis. State, Andrei. Crawford, Jessica. Rademacher, Paul. Drake, Samuel. Meyer, Anthony. 2006. In: Delp, Scott. Augmented Reality Visualization for Laparoscopic Surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer-Verlag Berlin Heidelberg. 933-940.
Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J. 2016. Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. Annals of Translational Medicine. 4(23):454.
Shuhaiber, Jeffery. 2004. Augmented Reality in Surgery. JAMA Network. 139(2). 170-174.
Yang, Jiachen. Lin, Yancong. Gao, Zhiqun. Lv, Zhihan. Wei, Wei. Song, Houbing. 2015. Quality Index for Stereoscopic Images by Separately Evaluating Adding and Subtracting. PLoS One. 10(12). 1-3.