A Robotic System for Implant Modification in Single-stage Cranioplasty
Abstract
Craniomaxillofacial reconstruction with patient-specific customized craniofacial implants (CCIs) is most commonly performed for large-sized skeletal defects. Because the exact size of skull resection may not be known prior to the surgery, in the single-stage cranioplasty, an oversized CCI is prefabricated and resized intraoperatively with a manual-cutting process provided by a surgeon. The manual resizing, however, may be inaccurate and significantly add to the operating time. This paper introduces a fast and non-contact approach for intraoperatively determining the exact contour of the skull resection and automatically resizing the implant to fit the resection area. Our approach includes four steps: First, we acquire a patient’s defect information using a handheld 3D scanner. Second, the scanned defect is aligned to the CCI by registering the scanned defect to the preoperative CT model. Third, a cutting toolpath is generated from the scanned defect model by extracting the resection contour. Lastly, a cutting robot resizes the oversized CCI to fit the resection area. To evaluate the resizing performance of our method, we generated six different resection shapes for the cutting experiments. We compared the performance of our method to the performance of surgeon’s manual resizing and an existing technique that collects the defect contour with an optical tracking system. The results show that our proposed method improves the resizing accuracy by 56% compared to the surgeon’s manual modification and 42% compared to the optical tracking method.
I Introduction
Cranioplasty is a procedure to treat cranial defects due to trauma, injury, or neurosurgical procedures for brain tumors, aneurysms, or epilepsy [1]. Conventional cranioplasty is a two-stage process that repairs skull deformities in a delayed operation [2]. Such process requires the skull to be partially removed from the patient, who then has to wait for the design and fabrication of the replacing implant for three to four weeks. In contrast, single-stage cranioplasty aims to restore aesthetic appearance immediately following craniectomy within one single operation, therefore, decreasing operative times and speeding up the patient’s recovery [3].
Several approaches are utilized to generate CCIs. The molding technique has been applied to form CCIs in the operating room [4]. This method requires injecting liquid bio-compatible materials such as poly-methyl-methacrylate directly into the defect of the skull or a molding template generated using the autologous bones. However, molding a CCI directly on the defect may release an exothermic reaction damaging nearby tissue[5]. Moreover, autologous bones cannot always be used to create a negative imprint and are limited in their ability to eliminate the discontinuities between the boundaries. Another commonly used approach is a cutting guide, in which a customized implant and a cutting guide are prefabricated, and the surgeon resects the patient’s skull following the cutting guide so that the skull defect can be immediately closed with the prefabricated implant [6, 7]. In addition, some other groups considered using optical navigation systems to achieve planned resections. Although these methods are capable of repairing skull deformities within one operation, they do not consider the possibilities for intraoperative plan changes, limiting neurosurgeons’ flexibility in reaching specific regions of the brain [8, 9].
In practice, cranioplasty replaces the skull defect with an alloplastic implant instead of using a patient’s autologous skull-bone [10]. Although additive manufacturing has been widely used to fabricate CCIs [11, 12, 13], it is not realistic to bring 3D printers into the operating room to intraoperatively print CCIs due to its long material building process and sterilization requirement. Therefore, the subtractive approach with prefabricated and pre-sterilized oversized CCIs is more practical and cost-effective. In [3, 14], a clinical approach using prefabricated oversized CCIs in single-stage cranioplasty is presented. This approach requires a surgeon to intraoperatively modify an oversized CCI by manually resizing it, which is often poor in accuracy and time-consuming.
Computer-assisted single-stage cranioplasty provides a method to help surgeons better visualize the defect contour by directly projecting the defect contour on an oversized CCI. This method utilizes an optical tracking system to collect data points of the defect contour[15]. However, this system is difficult to set up and requires line-of-sight. Moreover, the planar projection from a fixed configuration may not be suitable for implants with complex structures. To address the above-mentioned problems, we previously developed a portable projection mapping device that tracks surgical instruments and projects a 3D defect contour onto the implant in real-time from any angle without information loss [16]. Although this approach improves the accuracy of projection mapping for medical augmented reality, it can only collect one data point per frame with a digitizing instrument. Therefore, this approach takes longer to collect sufficient data points for registration.
Recent advances in 3D scanning technologies provide new venues for extending the application of medical robots in the operating room [17]. 3D scanning generates high-precision 3D models of real-world objects that can be recognized within a robot’s workspace to achieve specific autonomous tasks. Different from the optical tracking system, a 3D scanner can collect thousands of data points per frame without contacting the object. The use of a 3D scanner for skull defect reconstruction can simplify and expedite the identification of the defect’s contour.
In this paper, we present a novel system for generating precise CCIs for patients in single-stage cranioplasty. The system consists of a 3D scanner and a cutting robotic arm. The 3D scanning technique enables fast registration and generation of cutting toolpaths. The cutting robotic arm provides stable and accurate performance compared to the conventional manual cutting approach by surgeons and an existing method using an optical tracking system.
The contributions of this work include:
-
•
We proposed a fast and non-contact approach for acquiring defect contour information using a handheld 3D scanner.
-
•
We developed an algorithm for generating cutting toolpaths by extracting defect contours.
-
•
We integrated a robotic system for automated implant modification.
II Method
We developed a robotic system for resizing oversized CCIs during intraoperative operation. The system consists of a handheld Artec Space Spider 3D scanner (up to 0.1 mm resolution) and a KUKA LBR iiwa 7 R800 robotic arm (repeatability mm). The 3D scanner was utilized to acquire 3D information of the skull defect and to export a refined mesh (Fig. 1, bottom, left). We modified the KUKA robotic arm into a cutting workstation by attaching a spindle tool to the robot’s end-effector (Fig. 1, bottom right).
Our intraoperative CCI modification method includes four steps (Fig. 1, middle): 1) The information of a defected skull is collected using a 3D scanner. 2) The scanned data is registered to the CT model space. 3) A cutting toolpath is generated by extracting the defect contour. 4) A cutting robot resizes the CCI according to the generated toolpath.
II-A 3D reconstruction of a patient’s defect skull
A 3D scanning process was first utilized to acquire intraoperative patient data. During this process, the 3D scanner was held by hand at an approximate half meter away from the skull and moved slowly around it. This process could be terminated when there were sufficient 3D data shown in the visualization software (Artec Studio). This process usually takes less than two minutes.
II-B Patient-CT registration
The 3D-scanned data was then registered to the preoperative CT model of the patient’s skull. This process transformed the scanned data to the CT model space, as shown in Fig. 2. An iterative closest points (ICP) registration method [18] was applied to refine the registration. Three anatomical points were artificially designed on the defect skull and were picked from the 3d-scanned model for ICP initialization.
Since the preoperative CT model has two layers separated by the bone thickness, during the registration process, the 3d-scanned data tends to mistakenly overlap with the inner layer of the skull, because of the similar geometric feature between the inner and outer layer. To prevent this problem, we designed a preprocessing algorithm to convert the closed CT model to an open surface mesh by removing the inner layer.
In this method, we defined the 3D position of each vertex in the CT skull mesh as . The center of the mesh can be approximated by: . Then we constructed vector , which points from the center to each vertex . Since each vertex in the skull mesh is also associated with a normal vector of its own. The vectors of the inner layer point to the hollowed space inside the skull towards the center , while the vectors of the outer layer point to the opposite directions. As a result, the sign of determines whether this vertex is located in the inner layer or the outer layer. Vertices with negative dot products were removed to keep only the outer layer of the CT mesh model.
II-C Toolpath Generation
To generate a toolpath for the subsequent resizing process, the defect contour was first extracted from the 3d-scanned mesh of the skull defect. Then, the 3d-scanned defect was registered to the preoperative CT model. Therefore, the implant was aligned to the 3d-scanned defect model in the CT coordinates. A toolpath consisting of cutting positions and vectors along the extracted contour was generated in the implant coordinate system, as shown in Fig. 3. We implemented the following steps to generate 3D cutting toolpaths with visualization using Pyvista [19]:
II-C1 Curvature Filter
To extract the contour of the defect, a curvature filter was applied to the vertices of the 3d-scanned mesh and followed by manual cleaning. We utilized a curvature filter to determine the local mean curvature along the surface of the defect and extracted the high curvature value above a designed threshold. The mean curvature was calculated as , where and are the maximum and minimum values of the principal curvature on the mesh [20]. This filter was able to identify crease changes by the curvature of the surface mesh. After curvature filtering, only the vertices near the dropping edges were kept (Fig. 3, i). Additional manual cleaning was performed to further remove potential redundant vertices that were far off the edge.
II-C2 Curve Fitting
After removing all the redundant vertices, a group of vertices around the defect edge were remained denoted as (Fig. 3, ii). We defined a local cylindrical coordinate system on a best-fit plane of the extracted contour parameterized as . The center of the cylindrical coordinate system was obtained by the mean coordinates of the remaining vertices. A nonlinear least-squares method was then used to fit a closed polynomial curve to these vertices expressed in the cylindrical coordinates.
II-C3 Spline Projection
The fitted curve was then converted into a spline curve, which consists of several control points along the curve. Although the 3d-scanned defect was registered to the CT model after patient-to-CT registration, the extracted defect contour may not perfectly align to the implant’s top surface due to registration error. To eliminate the error, the control points were projected onto the top surface of the implant mesh so that the spline curve would adjust its shape to match the curvature of the implant’s top surface (Fig. 3, iv).
II-C4 Toolpath Generation
For each discretized point of the spline curve, a unit 3D vector was added to define the tool center point (TCP)’s axis orientation. Each was computed by tilting a constant angle from , the normal vector of the best-fit plane, toward , a vector defined from the center point to each curve point . A cutting toolpath was then obtained by combining each curve point with its associated cutting vector . To compensate the tool radius, the curve points were expanded to by an offset equal to the radius of the cutting bit (Fig. 3, v).
III Experimental Setup
To evaluate the implant-resizing accuracy of our integrated system, we compared our method with the surgeon’s manual-resizing method and an existing optical tracking method. We conducted six experiments with independently generated skull defects with different sizes and shapes. The defect specimens were generated using boolean operations. As shown in Fig. 4 (Top), we first subtracted the mesh inside the red contour from a complete skull to create a defect on the skull. On the same complete skull, the implant mesh inside the blue region was extracted to create its corresponding oversized implant. The defected skull was further cropped by a plane to a 3D printable size and was fabricated using a 3D printer (Stratasys F370, ABS material). Finally, we created three spherical markers on the top surface of each cropped defect and its corresponding implant for Patient-CT registration and implant localization.
III-A Method Comparison
We compared the implants generated by our method with the manual resizing method, as well as the optical tracking method used by Murphy et al. [15]. For robotic cutting, the cut depth was set to 3 mm, which is the thickness of the 3d-printed implants. The cut angle was set to 20 degrees for all the generated cutting toolpaths to generate beveled boundaries.
III-A1 Manual resizing method
We provided the surgeon with pre-designed partial skulls with the generated defects and corresponding oversized implants. The surgeon first outlined the defect contour of each specimen manually on the implant based on his visual judgment. He then resized the implant with a hand-held cutting tool following the outline.
III-A2 Optical tracking method
The optical tracking method used a digitizing instrument and an optical tracking system to trace the defect contour (Fig. 5, a). Instead of manual resizing, the oversized implants were then resized by the same cutting robot and with the same cutting parameters described above.
III-B Tool Center Point (TCP) Calibration
The transformation between the tip of the spindle tool and the robot arm’s end-effector was calibrated using a pivot calibration [21]. In this method, we hand-guided the robotic arm to different poses, such that the TCP always touches the tip of a fixed pin. The accuracy of the TCP calibration, measured by the TCP error, is shown in Fig. 5, and the relative pose from to :
III-C Implant Localization
The oversized CCI was secured on the robot’s working platform with bolts during the resizing process (Fig. 1, bottom, right). In order to obtain the relative position and orientation of the oversized CCI in the robot space (Fig. 2, right), we hand-guided the robot’s spindle TCP to touch the tip of each spherical marker separately so that the locations of the reference markers defined in could be transformed to robot’s frame . The registration between the robot space and CT space was then achieved based on the known locations of the markers, described in the and in the respectively.
III-D Hardware Details
The integrated system was set up on a computer running Intel Core i7-6820HQ @ 2.7GHz CPU. The 3D scanner (Artec Spider) collects data at 15 HZ. The KUKKA robot is operated using online mode via RoboDK 111RoboDK is an offline programming and simulation software for industrial robots. https://robodk.com/. The registration between the 3D-scanned model and the CT model was implemented in Meshlab [22], open-source software for mesh processing. The NDI Polaris optical tracking system operates at 10 Hz (± 0.3 mm tracking accuracy) was used in the comparison experiment.
IV Result
IV-A Registration
1. Optical Tracking Method
Three anatomical markers were artificially added to the original CT models and were 3D printed with the defect specimens. The anatomical points on the printed specimens were localized in the optical tracking system with a tracking instrument (Fig. 5a). Each defect specimen was registered back to the CT coordinate system using point set registration based on singular value decomposition [23]. The registration error was given by the mean Cartesian distance between the registered anatomical points and the original anatomical points defined in the CT space (Table I).
2. 3D Scanning Method
After 3D scanning the defect specimen, we first manually aligned the three anatomical points on the defect with the original anatomical points defined in the CT model as an initialization. Then ICP was used to fine-tune the registration of the scanned specimen to the original CT model. The error was evaluated by calculating the mean distance between all of the valid vertices and their closest vertices in the original CT model (Table I).
Specimen | 1 | 2 | 3 | 4 | 5 | 6 |
Optical Tracking Point Cloud registration (mm) | 0.39 | 0.38 | 0.32 | 0.32 | 0.30 | 0.34 |
3D Scanning ICP (mm) | 0.04 | 0.02 | 0.04 | 0.04 | 0.05 | 0.03 |
IV-B Evaluation of resized implants
The post-completed implants were physically fitted to the defect specimens. First, we scanned the implants sitting on their respective defects with a 3D scanner and registered to their original CT models using the artificial anatomical points on the defect specimens. We then 3d-scanned the post-completed implants individually and registered to the previous 3D scans using the three artificial fiducial points created on each implant so that each post-completed implant model could be positioned correctly relative to the ground truth defect model.
We then evaluated the gap distances between the boundaries of the post-completed implants and their corresponding defect edges. The gap distances were visualized in Meshlab [22] (Fig. 6). We used the gap distance distribution (maximum, mean, and standard deviation) to quantify the error for each method for the 6 specimens. Fig. 7 shows the analysis of the max gap distance. For the manual resizing method, all the resized implants were smaller than the defect edges since the surgeon would repeatedly trim the implant until it fits into the defect. Yet, for optical tracking and 3D scanning methods, the post-completed implant could be slightly larger than the defect boundaries, which would not completely fit into the defect.
Among the six defect specimens, the third specimen was considered the most difficult case due to the complex shape above the eye orbit. Although the gap analysis for our proposed approach showed the fourth and the sixth specimens had larger maximum bone gaps than the third specimen, this corresponds to the fact that the resized implants did not completely fit into the defect and would require slight trimming. Among the mentioned three methods, Fig. 8 shows that our proposed 3D scanning method with robotic integration was the only one with the mean gap distance below 1.5 mm.
IV-C Time cost
The resizing process by our proposed method took about 10-15 minutes, including the setup time, which was similar to the optical tracking method, whereas the conventional manual approach takes a range from ten to eighty minutes by expert surgeons.
V Discussion
We present a novel method for intraoperatively fabricating precise CCI in single-stage cranioplasty. In the proposed method, we first scan the defect to create a mesh model. The mesh model is then registered to the reconstructed 3D model from the preoperative CT in order to define the contour of the defect. Next, a cutting toolpath is generated using a discrete spline curve to represent the defect contour. After localizing the oversized CCI with respect to the robot’s base frame, the cutting robot automatically resizes the implant to generate the final shape to fit the defect.
The proposed method improves the accuracy of the cut by 56% compared to the surgeon’s cut and 42% compared to the optical tracking method. Moreover, the implant cut boundaries as created by the robot were considerably smoother than those created by the expert surgeon. The smooth boundary may contribute to the better fit of implants to the defect area in actual surgical scenarios. Our proposed method significantly reduced the operation time compared to the expert surgeon’s performance time of 10-80 minutes, as reported by Berli et al [3]. The robotic modification of oversized CCIs was shown to be more consistent and accurate when compared to the expert surgeon’s performance. Of note, we used an available seven DOF Kuka robot to perform the cutting tasks. However, a cheaper six DOF robot or a five-axis laser cutting machine (e.g. [24]) can also successfully perform smooth cutting as proposed for this research.
Some of the limitations of the current study are as follows: 1) During the toolpath generation process, the cut angle defining the tool axis attached to the discretized points along the defect contour was constant. In actual surgical scenarios, This may cause problems in fitting the implant if the defect boundary is not beveled uniformly. The extension of this work will include the development of an algorithm that can extract the bevel angle of the defect wall from the scan data. 2) In this study, the manually-tuned, experimentally-determined cutting speed and spinning rate of the tool were not optimized. Additional experiments are needed to evaluate the optimal cutting parameters for smooth cutting of the implant. 3) In the clinical setting, due to the minimally exposed surgical area (Figure 1, Top, iii), the process of registering a patient’s defect scan to the CT model may be challenging. A possible remedy is to use two separate 3D scans at different times during the procedure. Prior to the draping of the patient, fiducial marks will be attached to the patient. The first scan will acquire the full exposed head with fiducial marks attached and register this head model to the preoperative CT scan of the patient’s skull. After draping the surgical area and subsequent skull resection, a second scan containing the defect area information will be registered to the first scan using the information obtained from the fiducial marks. Thereby, the defect scan can be mapped to the CT model through the intermediate scan obtained prior to draping.
References
- [1] S. Aydin, B. Kucukyuruk, B. Abuzayed, S. Aydin, and G. Z. Sanus, “Cranioplasty: review of materials and techniques,” Journal of neurosciences in rural practice, vol. 2, no. 2, p. 162, 2011.
- [2] C. M. Pasick, K. Margetis, G. F. Santiago, C. Gordon, and P. J. Taub, “Adult cranioplasty,” Journal of Craniofacial Surgery, vol. 30, no. 7, pp. 2138–2143, 2019.
- [3] J. U. Berli, L. Thomaier, S. Zhong, J. Huang, A. Quinones, M. Lim, J. Weingart, H. Brem, and C. R. Gordon, “Immediate single-stage cranioplasty following calvarial resection for benign and malignant skull neoplasms using customized craniofacial implants,” Journal of Craniofacial Surgery, vol. 26, no. 5, pp. 1456–1462, 2015.
- [4] A.-R. Fathi, S. Marbacher, and A. Lukes, “Cost-effective patient-specific intraoperative molded cranioplasty,” Journal of craniofacial surgery, vol. 19, no. 3, pp. 777–781, 2008.
- [5] S. Marbacher, L. Andereggen, S. Erhardt, A.-R. Fathi, J. Fandino, A. Raabe, and J. Beck, “Intraoperative template-molded bone flap reconstruction for patient-specific cranioplasty,” Neurosurgical review, vol. 35, no. 4, pp. 527–535, 2012.
- [6] H. Eufinger, A. R. Wittkampf, M. Wehmöller, and F. W. Zonneveld, “Single-step fronto-orbital resection and reconstruction with individual resection template and corresponding titanium implant: a new method of computer-aided surgery,” Journal of Cranio-Maxillofacial Surgery, vol. 26, no. 6, pp. 373–378, 1998.
- [7] G. Gerbino, F. A. Bianchi, E. Zavattero, F. Tartara, D. Garbossa, and A. Ducati, “Single-step resection and reconstruction using patient-specific implants in the treatment of benign cranio-orbital tumors,” Journal of Oral and Maxillofacial Surgery, vol. 71, no. 11, pp. 1969–1982, 2013.
- [8] F. Jalbert, S. Boetto, F. Nadon, F. Lauwers, E. Schmidt, and R. Lopez, “One-step primary reconstruction for complex craniofacial resection with peek custom-made implants,” Journal of Cranio-Maxillofacial Surgery, vol. 42, no. 2, pp. 141–148, 2014.
- [9] P. Dodier, F. Winter, T. Auzinger, G. Mistelbauer, J. M. Frischer, W.-T. Wang, A. Mallouhi, W. Marik, S. Wolfsberger, L. Reissig et al., “Single-stage bone resection and cranioplastic reconstruction: comparison of a novel software-derived peek workflow with the standard reconstructive method,” International journal of oral and maxillofacial surgery, vol. 49, no. 8, pp. 1007–1015, 2020.
- [10] A. M. Shah, H. Jung, and S. Skirboll, “Materials used in cranioplasty: a history and analysis,” Neurosurgical focus, vol. 36, no. 4, p. E19, 2014.
- [11] B.-J. Kim, K.-S. Hong, K.-J. Park, D.-H. Park, Y.-G. Chung, and S.-H. Kang, “Customized cranioplasty implants using three-dimensional printers and polymethyl-methacrylate casting,” Journal of Korean Neurosurgical Society, vol. 52, no. 6, p. 541, 2012.
- [12] E.-K. Park, J.-Y. Lim, I.-S. Yun, J.-S. Kim, S.-H. Woo, D.-S. Kim, and K.-W. Shim, “Cranioplasty enhanced by three-dimensional printing: custom-made three-dimensional-printed titanium implants for skull defects,” Journal of Craniofacial Surgery, vol. 27, no. 4, pp. 943–949, 2016.
- [13] J. A. Morales-Gómez, E. Garcia-Estrada, J. E. Leos-Bortoni, M. Delgado-Brito, L. E. Flores-Huerta, A. Adriana, L. J. Torres-Díaz, and Á. R. M.-P. de León, “Cranioplasty with a low-cost customized polymethylmethacrylate implant using a desktop 3d printer,” Journal of neurosurgery, vol. 130, no. 5, pp. 1721–1727, 2018.
- [14] A. Wolff, G. F. Santiago, M. Belzberg, C. Huggins, M. Lim, J. Weingart, W. Anderson, A. Coon, J. Huang, H. Brem et al., “Adult cranioplasty reconstruction with customized cranial implants: preferred technique, timing, and biomaterials,” Journal of Craniofacial Surgery, vol. 29, no. 4, pp. 887–894, 2018.
- [15] R. J. Murphy, K. C. Wolfe, P. C. Liacouras, G. T. Grant, C. R. Gordon, and M. Armand, “Computer-assisted single-stage cranioplasty,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015, pp. 4910–4913.
- [16] S. Liu, W.-L. Huang, A. Shin, C. Gordon, and M. Armand, “A portable projection mapping device for medical augmented reality in single-stage cranioplasty,” in Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR), vol. 11310. International Society for Optics and Photonics, 2020, p. 1131007.
- [17] A. Haleem and M. Javaid, “3d scanning applications in medical field: a literature-based review,” Clinical Epidemiology and Global Health, vol. 7, no. 2, pp. 199–210, 2019.
- [18] P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures, vol. 1611. International Society for Optics and Photonics, 1992, pp. 586–606.
- [19] C. B. Sullivan and A. A. Kaszynski, “Pyvista: 3d plotting and mesh analysis through a streamlined interface for the visualization toolkit (vtk),” Journal of Open Source Software, vol. 4, no. 37, p. 1450, 2019.
- [20] M. Meyer, M. Desbrun, P. Schröder, and A. H. Barr, “Discrete differential-geometry operators for triangulated 2-manifolds,” in Visualization and mathematics III. Springer, 2003, pp. 35–57.
- [21] W. Birkfellner, F. Watzinger, F. Wanschitz, R. Ewers, and H. Bergmann, “Calibration of tracking systems in a surgical environment,” IEEE transactions on medical imaging, vol. 17, no. 5, pp. 737–742, 1998.
- [22] P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli, and G. Ranzuglia, “Meshlab: an open-source mesh processing tool.” in Eurographics Italian chapter conference, vol. 2008. Salerno, 2008, pp. 129–136.
- [23] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-d point sets,” IEEE Transactions on pattern analysis and machine intelligence, no. 5, pp. 698–700, 1987.
- [24] J. Liu, J. Fang, R. J. Murphy, C. Gordon, and M. Armand, “Design and development of 5-axis cranial implant laser cutting system,” in International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. 58110. American Society of Mechanical Engineers, 2017, p. V001T02A051.