Motion Planning for Steerable Needles

We are developing steerable needles, a new class of flexible, bevel-tip medical needles that are capable of following curved paths through soft tissue. Steerable needles have the ability to (1) maneuver around anatomical obstacles to reach targets inaccessible by traditional straight needles and (2) correct for perturbations that occur during insertion, thereby increasing accuracy and precision. Steerable needles have the potential to improve patient care for a variety of diagnostic and therapeutic medical procedures, including tissue biopsies, tumor ablation, and brachytherapy cancer treatments in the lungs and in the abdominal and pelvic cavities.

Our steerable needles are composed of a highly flexible material and have a bevel-tip. When inserted in soft tissue, they curve in the direction of the bevel tip and can be steered by axially rotating the needle’s base to re-orient the bevel.

Motion Planning: To assist physicians in guiding steerable needles around anatomical obstacles to clinical targets, we are developing motion planning algorithms for robotic assistance. The motion planners rely on segmentations of the anatomical obstacles (e.g., significant blood vessels, nerves, bones) and the target in pre-operative imaging. The motion planner must consider that steerable needles are under-actuated (the number of control inputs is smaller than the dimension of the configuration space) and nonholonomic (the needle, like a car, cannot translate sideways). We have created interactive-rate motion planning algorithms for needle steering in 3D anatomy using geometric, sampling-based, and optimization-based approaches.

Considering Uncertainty: Handling uncertainty is an important challenge in needle steering; uncertainty in motion arises due to tissue inhomogeneity and patient variability, and uncertainty in sensing arises due to noisy or partial imaging feedback during a procedure. We have developed motion planners that explicitly consider uncertainty in needle motion and sensing in order to maximize the probability of avoiding collisions and successfully reaching the target. Our methods blend geometric motion planning, stochastic modeling, and local optimal control. Explicitly accounting for uncertainty can lead to significantly different motion plans compared to traditional shortest paths, such as longer paths with greater clearance from obstacles or paths that better utilize available sensing in order to increase the probability of success.

Considering Deformations: Inserting needles into soft tissues causes the surrounding soft tissues to deform, and ignoring these deformations can result in substantial placement error. Physically-based simulations that model soft tissue deformations during needle procedures can assist clinicians in preoperative planning. We are developing 2D and 3D simulations of steerable needle procedures by modeling tissue deformations using a finite element method, modeling needle frictional and cutting forces, and using re-meshing to ensure conformity of the mesh to the curvilinear needle path. We also developed motion planners that combine a simulation of needle insertion with optimization to compensate for predicted tissue deformations caused by needle insertion.

This needle steering project is a collaborative effort involving research groups at The Johns Hopkins University, Stanford University, the University of California, Berkeley, Vanderbilt University, Queens University (Canada), and the University of Twente (Netherlands).

Collaborators

• Robert J. Webster III, Department of Mechanical Engineering, Vanderbilt University
• Dr. Richard Feins, Cardiothoracic Surgery, University of North Carolina at Chapel Hill
• Dr. Jason Akulian, Interventional Pulmonology, University of North Carolina at Chapel Hill
• Dr. Yueh Lee, Radiology, University of North Carolina at Chapel Hill

Publications

1. Mengyu Fu, Alan Kuntz, Robert J. Webster III, and Ron Alterovitz, "Safe Motion Planning for Steerable Needles Using Cost Maps Automatically Extracted from Pulmonary Images," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2018, pp. 4942-4949. (Publisher) (Download PDF)
2. Philip J. Swaney, Arthur W. Mahoney, Bryan I. Hartley, Andria A. Remirez, Erik Lamers, Richard H. Feins, Ron Alterovitz, Robert J. Webster III, "Toward Transoral Peripheral Lung Access: Combining Continuum Robots and Steerable Needles," Journal of Medical Robotics Research, vol. 2, no. 1, pp. 1-14, 2017. (Publisher) (Download PDF)
3. Momen Abayazid, Claudio Pacchierotti, Pedro Moreira, Ron Alterovitz, Domenico Prattichizzo, and Sarthak Misra, "Experimental Evaluation of Co-manipulated Ultrasound-guided Flexible Needle Steering," International Journal of Medical Robotics and Computer Assisted Surgery, vol. 12, no. 2, pp. 219–230, June 2016. (Publisher)
4. Alan Kuntz, Philip J. Swaney, Arthur W. Mahoney, Richard H. Feins, Yueh Z. Lee, Robert J. Webster III, and R. Alterovitz, "Toward Transoral Peripheral Lung Access: Steering Bronchoscope-deployed Needles through Porcine Lung Tissue," in Hamlyn Symposium on Medical Robotics, June 2016. (Download PDF)
5. Alan Kuntz, Luis G. Torres, Richard H. Feins, Robert J. Webster III, and Ron Alterovitz, "Motion Planning for a Three-Stage Multilumen Transoral Lung Access System," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2015, pp. 3255-3261. (Publisher) (Download PDF)
6. Wen Sun, Sachin Patil, and Ron Alterovitz, "High-Frequency Replanning under Uncertainty using Parallel Sampling-Based Motion Planning," IEEE Transactions on Robotics, vol. 31, no. 1, pp. 104-116, Feb. 2015. (Publisher) (Download PDF)
7. Wen Sun and Ron Alterovitz, "Motion Planning under Uncertainty for Medical Needle Steering Using Optimization in Belief Space," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2014, pp. 1775-1781. (Publisher) (Download PDF)
8. Gustaaf J. Vrooijink, Momen Abayazid, Sachin Patil, Ron Alterovitz, and Sarthak Misra, "Needle Path Planning and Steering in a Three-Dimensional Non-Static Environment using Two-Dimensional Ultrasound Images," International Journal of Robotics Research, vol. 33, no. 10, pp. 1361-1374, Sep. 2014. (Publisher)
9. Wen Sun and Ron Alterovitz, "Motion Planning under Uncertainty for Medical Needle Steering Using Optimization in Belief Space," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2014, pp. 1775-1781. (Publisher) (Download PDF)
10. Wen Sun, Jur van den Berg, and Ron Alterovitz, "Stochastic Extended LQR: Optimization-based Motion Planning Under Uncertainty," in Proc. Workshop on the Algorithmic Foundations of Robotics (WAFR), Aug. 2014. (Download PDF)
11. Sachin Patil, Jessica Burgner, Robert J. Webster III, and Ron Alterovitz, "Needle Steering in 3-D via Rapid Replanning," IEEE Transactions on Robotics, vol. 30, no. 4, pp. 853-864, Aug. 2014. (Publisher) (Download PDF)
12. Pedro Moreira, Sachin Patil, Ron Alterovitz, and Sarthak Misra, "Needle Steering in Biological Tissue using Ultrasound-based Online Curvature Estimation," in Proc. IEEE International Conference on Robotics and Automation (ICRA), May 2014, pp. 4368-4373. (Publisher) (Download PDF)
13. Sachin Patil, Jur van den Berg, and Ron Alterovitz, "Estimating Probability of Collision for Safe Planning under Gaussian Motion and Sensing Uncertainty," in Proc. IEEE International Conference on Robotics and Automation (ICRA), May 2012, pp. 3238-3244. (Publisher) (Download PDF)
14. Kyle B. Reed, Ann Majewicz, Vinutha Kallem, Ron Alterovitz, Ken Goldberg, Noah J. Cowan, Allison M. Okamura, "Robot-Assisted Needle Steering," IEEE Robotics and Automation Magazine, vol. 18, pp. 35-46, Dec. 2011. (Publisher) (Download PDF)
15. Sachin Patil, Jur van den Berg, and Ron Alterovitz, "Motion Planning Under Uncertainty In Highly Deformable Environments," in Proc. Robotics: Science and Systems (RSS), June 2011. (Download PDF)
16. Edgar Lobaton, Jinghe Zhang, Sachin Patil, and Ron Alterovitz, "Planning Curvature-Constrained Paths to Multiple Goals Using Circle Sampling," in Proc. IEEE International Conference on Robotics and Automation (ICRA), May 2011, pp. 1463-1469. (Publisher) (Download PDF)
17. Jur van den Berg, Sachin Patil, Ron Alterovitz, Pieter Abbeel and Ken Goldberg, "LQG-Based Planning, Sensing, and Control of Steerable Needles," in Algorithmic Foundation of Robotics IX (WAFR 2010), D. Hsu et al. (Eds.), STAR vol. 68, Springer-Verlag, 2010, pp. 373-389. (Publisher) (Download PDF)
18. Sachin Patil and Ron Alterovitz, "Interactive Motion Planning for Steerable Needles in 3D Environments with Obstacles," in Proc. IEEE RAS/EMBS Int. Conf. Biomedical Robotics and Biomechatronics (BioRob), Sep. 2010, pp. 893-899. (Publisher) (Download PDF)
19. Jijie Xu, Vincent Duindam, Ron Alterovitz, Jean Pouliot, J. Adam M. Cunha, I-Chow Hsu, and Ken Goldberg, "Planning Fireworks Trajectories for Steerable Medical Needles to Reduce Patient Trauma," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2009, pp. 4517-4522. (Publisher) (Download PDF)
20. Vincent Duindam, Jijie Xu, Ron Alterovitz, Shankar Sastry, and Ken Goldberg, "Three-dimensional Motion Planning Algorithms for Steerable Needles Using Inverse Kinematics," International Journal of Robotics Research, vol. 29, no. 7, pp. 789-800, June 2010. (Publisher) (Download PDF)
21. Nuttapong Chentanez, Ron Alterovitz, Daniel Ritchie, Jonha Cho, Kris K. Hauser, Ken Goldberg, Jonathan R. Shewchuk, and James F. O'Brien, "Interactive Simulation of Surgical Needle Insertion and Steering," ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 28, no. 3, pp. 88:1-88:10, Aug. 2009. (Publisher) (Download PDF) (Video)
    (Image from article featured on back cover)
22. Ron Alterovitz, Michael Branicky, and Ken Goldberg, "Motion Planning Under Uncertainty for Image-Guided Medical Needle Steering," International Journal of Robotics Research, vol. 27, no. 11-12, pp. 1361-1374, Nov. 2008. (Publisher) (Download PDF)
23. Kris Hauser, Ron Alterovitz, Nuttapong Chentanez, Allison Okamura, and Ken Goldberg, "Feedback Control for Steering Needles Through 3D Deformable Tissue Using Helical Paths," in Proc. Robotics: Science and Systems, June 2009. (Download PDF)
24. Vincent Duindam, Jijie Xu, Ron Alterovitz, Shankar Sastry, and Ken Goldberg, "3D Motion Planning Algorithms for Steerable Needles Using Inverse Kinematics," in Proc. WAFR 2008: The Eighth International Workshop on the Algorithmic Foundations of Robotics, Dec. 2008. (Download PDF)
25. Kyle B. Reed, Vinutha Kallem, Ron Alterovitz, Ken Goldberg, Allison M. Okamura, and Noah J. Cowan, "Integrated Planning and Image-Guided Control for Planar Needle Steering," in Proc. IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Oct. 2008, pp. 819-824. (Download PDF)
26. Vincent Duindam, Ron Alterovitz, Shankar Sastry, and Ken Goldberg, "Screw-Based Motion Planning for Bevel-Tip Flexible Needles in 3D Environments with Obstacles," in Proc. IEEE International Conference on Robotics and Automation (ICRA), May 2008, pp. 2483-2488. (Download PDF)
27. Ron Alterovitz, Thierry Siméon, and Ken Goldberg, "The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty," in Robotics: Science and Systems III (Proc. RSS 2007), W. Burgard et al. (Eds.), MIT Press, 2008, pp. 233-241. (Download PDF)
28. Ron Alterovitz, Michael Branicky, and Ken Goldberg, "Constant-Curvature Motion Planning Under Uncertainty with Applications in Image-Guided Medical Needle Steering," in Algorithmic Foundation of Robotics VII (WAFR 2006), S. Akella et al. (Eds.), STAR vol. 47, Springer-Verlag, 2008, pp. 319-334. (Download PDF)
29. Ron Alterovitz, Andrew Lim, Ken Goldberg, Gregory S. Chirikjian, and Allison M. Okamura, "Steering Flexible Needles Under Markov Motion Uncertainty," in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Aug. 2005, pp. 120-125. (Download PDF)
30. Ron Alterovitz, Ken Goldberg, and Allison Okamura, "Planning for Steerable Bevel-tip Needle Insertion Through 2D Soft Tissue with Obstacles," in Proc. IEEE International Conference on Robotics and Automation (ICRA), Apr. 2005, pp. 1652-1657. (Download PDF)
31. Robert J. Webster III, Allison M. Okamura, Noah J. Cowan, Gregory S. Chirikjian, Ken Goldberg, and Ron Alterovitz, "Distal bevel-tip needle control device and algorithm," US patent pending 11/436,995, May 2006.

This research is made possible by support from the National Science Foundation (NSF) under awards IIS-0905344 and IIS-1149965 and by the National Institutes of Health (NIH) under awards F32CA124138, R21EB011628, R21EB017952, and R01EB024864. Any opinions, findings, and conclusions or recommendations expressed on this web site do not necessarily reflect the views of NSF or NIH.