Free-hand, real-time needle guidance for prostate cancer diagnosis with augmented reality

According to the US Centers for Disease Control and Prevention, prostate cancer is the second leading cause of cancer death among men. One of the standard approaches for the diagnosis and treatment of prostate cancer involves transperineal (TP) biopsy. This involves inserting a needle through the perineum wall to collect tissue samples. Current methods for TP biopsy generally include a pre-operation MRI scan and a transrectal ultrasound. These images are then fused together and shown on a monitor to the urologist, who then inserts the needle. The needle insertion can be performed free-hand or through a grid-based method. However, since this approach involves visualizing a 3D region in 2D, visualization and guidance of the needle can be challenging.

To address this issue, a team of researchers led by Dr. Ming Li from the US National Institutes of Health have proposed an approach based on an augmented reality (AR) system called HoloLens. In their recent paper published in the Journal of Medical Imaging, the researchers report the accuracy and efficacy of their AR HoloLens system, which can be used to project the MRI and ultrasound image directly onto the patient, helping guide a needle to its target. According to Li, “The current methods have certain limitations to them. Robot-assisted guidance is costly and adds procedural time, while other methods require a certain path for the needle that leaves out the outer reaches of the prostate. These problems are solved using the HoloLens AR system, which provides the doctor with the ability to use a free-hand approach.”

The HoloLens AR system uses a volumetric 3D scan, such as an MRI scan, to create a view of the patient for the urologist. Using reference data from the patient, the MRI scan can be superimposed onto the patient correctly. This superimposed image is then fed into the HoloLens goggles worn by the urologist, who can see not only the patient but also the MRI of the patient, properly aligned. By moving their head, they can view the image at different angles. HoloLens can also be used to show a preplanned needle path on the patient, the target tissue of the prostate, and the position of the needle in real time.

The researchers tested the image overlay accuracy and needle targeting accuracy for their AR system using a 3D-printed phantom. Both free-hand and planned-path guidance methods were used to guide needles into a gel phantom. Following this, the researchers recorded the placement errors for the needles. Additionally, they also used the system to deliver soft tissue markers onto the tumors of a human pelvis phantom.

They found that the placement errors associated with free-hand and planned-path guidance were similar. Further, the soft tissue markers were all implanted either into or close to the tumors.

“The HoloLens has the potential to provide more flexibility than the current grid-based TP methods and can do so accurately,” highlights Li. “By providing a 3D immersive experience, the HoloLens AR system makes free-hand lesion targeting feasible. As needle procedures move from rectum to the perineum, AR systems could provide great clinical value to doctors and patients by solving the problems associated with prostate intervention procedures,” she concludes.

Read the Open Access article by Li et al., “HoloLens augmented reality system for transperineal free-hand prostate procedures,” J. Med. Imaging 10(2) 025001 (2023), doi 10.1117/1.JMI.10.2.025001.