Multi-directional Bicycle Robot for Steel Structure Inspection

Son Thanh Nguyen¹, Hai Nguyen², Son Tien Bui³, Van Anh Ho³, Hung Manh La¹, IEEE Senior Member This work is supported by the U.S. National Science Foundation (NSF) under grants NSF-CAREER: 1846513 and NSF-PFI-TT: 1919127, and the U.S. Department of Transportation, Office of the Assistant Secretary for Research and Technology (USDOT/OST-R) under Grant No. 69A3551747126 through INSPIRE University Transportation Center. The views, opinions, findings and conclusions reflected in this publication are solely those of the authors and do not represent the official policy or position of the NSF and USDOT/OST-R. ¹The authors are with the Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, NV 89557, USA. ²Northeastern University, Boston, MA 02115, USA. ³Japan Advanced Institute of Science and Tech., Japan. Corresponding author: Hung La, email: hla@unr.edu.

Abstract

This paper presents a novel design of a multi-directional bicycle robot, which targets inspecting general ferromagnetic structures including complex-shaped structures. The locomotion concept is based on arranging two magnetic wheels in a bicycle-like configuration with two independent steering actuators. This configuration allows the robot to possess multi-directional mobility. An additional free joint helps the robot naturally adapt to non-flat and complex surfaces of steel structures. The robot has the biggest advantage to be mechanically simple with high mobility. Besides, the robot is equipped with sensing tools for structure health monitoring. We demonstrate the deployment of our robot to perform steel rust detection on steel bridges. The final inspection results are visualized as 3D models of the bridges together with marked locations of detected rusty areas.

I Introduction

Steel structures are indispensable parts of modern civilization. Typical civil infrastructures including bridges, wind turbines, electric towers, oil rigs, and so on; or vehicles, such as ships and submarines are made of steel. Those structures require maintenance frequently to warrant safety and longevity. Most of these inspections are still conducted manually by professional human inspectors with special devices to inspect visual damages and defects on or inside these structures. This procedure is usually highly time-consuming, costly, and unsafe. For instance, it is dangerous for the inspectors to climb up and hang on cables to inspect remotes area of bridges (Fig. 1a), or offshore oil rigs (Fig. 1b). Even the inspection of simpler structures such as wind turbines, ship shells (Fig. 1c), and gas/oil tanks/piles (Fig. 1d) is a big challenge due to their large scales.

Refer to caption — Figure 1: Typical steel structures: a) Sea-crossing steel bridges. b) Offshore oil rigs. c) Ship shells. d) Oil tanks and pipelines. (Source: Google images)

To automate the inspection task, using robots carrying sensing tools is an emerging solution. There are several innovative designs, which have been presented in this field in recent years. Several designs were based on conventional wheeled robots [1, 2, 3, 4, 5], while some made use of tank-like tracks to widen the contacting areas of the robots on steel surfaces [6, 7, 8, 9]. These approaches can work well on structures with large surfaces such as ships or tanks but face difficulties on complex surfaces frequently encountered on bridges and oil rigs. Some unique developments of climbing robots in this field can be seen in [10, 11, 12]. In this direction, many designs imitated the mobility of climbing animals. A spider-like robot with electromagnets on its feet is reported in [13], and a legged robot is developed in [14]. An inchworm-like robot [15] and a hybrid robot [16] are other noted examples. However, the complexity of these robot’s mechanics makes controlling them a challenge for real-world applications. Aerial robots provide an alternative for visual inspection tasks without the need for climbing robots. Recent developments make drone control significantly safer in confined spaces [17, 18]. Nonetheless, in-depth inspections of damages inside structures such as analyzing fatigue cracks or steel thickness that requires touched sensors are still challenging for these inspection drones.

Interestingly, there were a few bike-like robot designs [19, 20]. However, the mobility of these robots is still limited in narrow spaces of complex structures, that often require high flexibility of direction changes such as moving sideways. Our design is originated from [19, 20], but with an improved optimization on the climbing capability and multi-directional locomotion. Our robot can work on any sorts of ferromagnetic surfaces (flat, curved, or rough), and challenging structures for the previous bike-like robots such as narrow bars or structural transition joints. The proposed robot can generate large adhesive forces by using permanent magnets, which allows our robot to firmly adhere to the steel structures while moving. The multi-directional bicycle design makes the robot flexible on non-standard surfaces of steel structures. The robot is able to overcome difficult obstacles including nuts, bolts, and complex joints between bars or surfaces. Moreover, the robot is equipped with measurement and computational devices including a depth camera, a pose tracking camera, a computer for localization, navigation, and visual inspection. To demonstrate the robot’s capabilities and working principles, challenging laboratory and field tests are provided.

II Overall System

The whole system is depicted in Fig. 2. On the robot side, we utilize an Intel RealSense D435i camera, which provides both color and depth images. Poses are tracked by using an Intel RealSense T265 tracking camera, which can provide online poses using visual-inertial odometry. We employ a LattePanda Alpha 864 as an onboard computer. This computer is connected with our ground control station (GCS) computer (Intel NUC i7) via a WiFi router. Two computers form a Robot Operating System (ROS [21]) network, in which the GCS acts as the master. The robot is maneuvered remotely by a joystick on the GCS via a radio connection.

Going into the details of our design, given the challenges mentioned earlier for the locomotion systems for reliably inspecting a wide range of surfaces, we consider the following requirements for our robot:

1.

The robot can climb surfaces with a wide range of outer diameters ( $\geq$ 150mm), which are normally encountered on circular tubes or cylindrical surfaces;
2.

The robot can pass thorny, convex, or concave obstacles at structural transition joints on truss structures;
3.

The robot can travel on steel structures with complex arrangements of obstacles such as bolts, nuts, and gaps;
4.

The locomotion system can maneuver in narrow areas ( $\geq$ 100mm wide) and can move sideways with large changes in orientation.

The design concept of our multi-directional bicycle robot is illustrated in Fig. 3. Inspired by a bicycle, we use two revolute joints equipping the robot two independent steering actuators, which greatly increase the mobility. An additional free joint in the middle of the robot’s body allows its two wheels to fully contact with surfaces of different shapes and sizes. The moving wheels are designed with permanent ring-shaped magnets to generate large adhesive forces. The high mobility of the two steering actuators allows the robot to seamlessly work on two different modes: $(1)$ Mode 1 (Fig. 4a-e): only one steering unit is active, the robot works like a bicycle. The activated steering actuator can be at the front or the back. Working in this mode allows the robot to travel on structures with limited contacting areas (Fig. 4a). It can revert the direction immediately without turning, or even turn on spot by stopping the back wheel as illustrated in Fig. 4b. In addition, the robot can comfortably climb two intersecting surfaces by virtue of its free joint as shown in Fig. 4c. Fig. 4d shows that robot can pass edges which are thicker than the space between its two wheels (4cm in our design). Also in this mode, the robot can be flexible enough to travel on an internal surface of a tube (Fig. 4e). $(2)$ Mode 2 (Fig. 4f-g): Both the two steering units are active and they are controlled independently and in parallel. The robot can move spirally around a cylinder, move sideways (left, right), or rotate around its center.

III Mechanical Design and Analysis

Fig. 5 shows the overall mechanical design of the bicycle robot. The robot’s weight is 1kg (without sensors), while it can carry a 600g of loads (sensors or equipment). The robot is powered by a 3000mAh LiPo battery to allow it to work for up to 1 hour. The robot’s dimension is 150mm $\times$ 80mm $\times$ 70mm. The frame is made of plastic for a lightweight robot. The ring magnets are placed at the cores of the wheels, which are covered by silicone tires. The wheels are driven by two high-torque gear DC motors (100kg $\cdot$ cm torque each), and the steering actuators are controlled by two servos (32kg $\cdot$ cm torque each). The front and the back of the frame are linked by a bearing acting as a free joint. The detail mechanical analysis is presented in the following subsections.

III-A Magnetic Wheel Force Analysis

We analyze the adhesive force that the robot needs to climb reliably in any normally working conditions. We perform the analysis on an extreme situation where the adhesive force between the magnetic wheels and the contacting structures is minimal: the robot climbs a cylindrical tube (Fig. 6a), and encounters a corner (Fig. 6b). Here, $X_{1}$ and $X_{2}$ are two contacting points of the back wheel and front wheel, respectively. Given $P$ is the robot’s weight, $h$ is the distance from the center of mass of the robot to $X_{1}$ . If $F_{2}$ is an adhesive force of the front wheel at $X_{2}$ then $F_{2}$ is at its minimum when the front wheel hits the corner. To keep the robot from falling, the following condition needs to be satisfied:

F_{2}\times X_{1}X_{2}>P\times h\Rightarrow F_{2}>\dfrac{Ph}{X_{1}X_{2}}.

(1)

According to ISO 3691 [22] for safe weight lifting, the safety factor of 5 was selected. Therefore, the real adhesive force $F_{2}$ needs to be at least 5 times greater than the result from the above theoretical calculation (1).

III-B Moving Motors Power Analysis

This analysis is conducted to calculate the necessary motor torque when the robot stands the highest load. It is when the robot passes an internal corner between two surfaces (Fig. 7), the front wheel bears an additional force $F_{2.2}$ , which is the adhesive force of the front wheel at surface 2. Similarly, $F_{2.1}$ is the adhesive forces of the front wheel at surface 1. $F_{f2}$ is the friction of the front wheel on surface 2, $r$ is the wheel’s radius, $k$ is the static friction coefficient (between silicon and steel in our design). The minimum force of the front wheel that allows the robot to be able to pass the corner must satisfy:

\dfrac{M_{moving}}{r}>F_{2.1}+F_{f2}.

(2)

Therefore, the moving motor torque needs to satisfy:

M_{moving}>r\times(F_{2.1}+\dfrac{F_{2.2}+P}{k}).

(3)

According to IEC 60034 [23], the actual torque selected to be at least double that of theoretical calculation in (3).

III-C Steering Servos Power Analysis

An analysis is conducted to investigate the load-torque on the revolute joints. There are two forces as illustrated in Fig. 8: the static friction and the attractive force at the two magnetic wheels, with one affects another. Let $F_{12}$ be the adhesive force of wheel 1 affecting wheel 2, $F_{f}$ be the friction at $X_{2}$ . The measured load-force at point $L$ (Fig. 8a) has to satisfy the following condition:

F_{12}+F_{f}<\dfrac{M_{steering}}{r}.

(4)

Thus, the steering servo torque needs to satisfy:

\Rightarrow M_{steering}>r\times(F_{12}+\dfrac{F_{2}+P}{k}).

(5)

Based on IEC 60034 [23], the actual servo’s torque is chosen to be at least two-fold compared to that of theoretical calculation in (5).

III-D Tire Design

Specialized tires are designed for our robot to improve the friction between the tires and contacting surfaces. They also help better adapt to various encountered surfaces. The thickness of the tires needs to be small enough to maintain the magnitude of the magnetic force while being not too hard for ease of fabrication. Specifically, the tires are designed to adapt well on flat and cylindrical surfaces with the smallest diameter of 150mm. The proposed design of the tires with deformable curved treads is shown in Fig. 9a. We deposited spherical-head thorns and treads for increasing the friction between the tires and the target surfaces and for preventing slipping, especially when the robot operates on a cylindrical surface. Moreover, a dozen of square keys are set to fix and align the tires w.r.t. the inside cores. Due to the deformation of the tire material and trenches, these treads can deform and arrange tidily when traveling on a planar surface, promising a flat surface contacting (Fig. 9b). When the robot travels on curved surfaces, curved treads will also fit well to the target surfaces as shown in Fig. 9c. Regarding the material, it needs to be soft with low viscosity, in order to fill in the mould cavities and to avoid air bubbles. We chose DragonSkin 30 (Smooth On Inc., USA) silicone, which fits our technical requirements.

IV Experimental Evaluation

In this section, we demonstrate an application of our robot to detect rusts on steel bridges. This showcase is general and it can be applied to other steel structure inspection. We first describe the software structure, followed by laboratory and field tests. The hardware is previously described in Section II.

IV-A Software Structure

The software structure is described in Fig. 10. While the onboard computer collects the depth and color images, these images are transferred wirelessly back to the GCS, where localization and rust detection are performed. Unlike a similar system in [25], where all computation was performed onboard, our structure does not require onboard powerful computers and computationally expensive resources, which results in a reduction of the robot’s load.

Localization of the robot is performed w.r.t. a common (world) coordinate known to both T265’s pose coordinate system and a pre-scanned map. The world coordinate is the first pose of T265 and it is manually chosen in the pre-scanned map. Ideally, the mapping should also be done by the robot and it seems that our setting is sub-optimal. However, we found that for a single robot, the onboard camera cannot avoid blind spots due to its limited view, especially at abruptly turning points. Hence, it is challenging to merge neighboring point clouds to create a useful map. To overcome this issue, future work may consider using a team of robots or actively controlling the onboard camera. Rust detection on RGB images is performed by a pre-trained YOLO-v3 [24] network. We use YOLO-ROS [26] to use the trained network in ROS. While each resulting bounding box $[b_{x},b_{y}]$ of detected rusty areas is on the pixel coordinate, we want to localize them on the map. We do that by comparing its coordinate with the corresponding pixel on an aligned color-depth image to retrieve the corresponding depth information $b_{z}$ . Next, $[b_{x},b_{y},b_{z}]$ is fed to a deprojection function to convert to the world coordinate using the intrinsic parameters of D435i.

IV-B Laboratory Tests

An indoor structure comprised of typical parts of general steel structures (cylinder, L, I, U shaped beams) with structural transition joints is built to validate the robot’s locomotion. Our robot can traverse smoothly to any location in the testing structure. In mode 1, the robot can handle well most testing situations. However, mode 2 is necessary when operating in narrow spaces. For instance, as Fig. 11a shown, the robots need to combine mode 1 and 2 for travelling smoothly on a rectangular tube. Fig. 11b illustrates mode 2 while the robot working on a cylindrical shape. Fig. 11c illustrates how the robot changes orientation in mode 1 on an I-shaped beam. Fig. 12 illustrates when the robot crosses convex and concave obstacles of a rectangular tube.

IV-C Field Tests

We deploy our robot to perform a structural inspection for one indoor and one outdoor steel bridge. We first scan the indoor bridge manually using Dot3D¹¹1https://www.dotproduct3d.com/dot3dpro.html software. During experimenting with other approaches, we find this software is easier to use and can produce good 3D maps, which have better qualities than the ones produced by open-source approaches such as RTAB-Map [27] or VoxBlox [28]. The inspection result of the indoor bridge is shown in Fig. 13. In the figure, we can observe that rusty areas, which are marked by collective green spheres, are correctly identified and localized. We also show an example of the robot paths, illustrated by a white trace. Additional color images from the onboard D435i camera are shown with detected rusts marked by pink boxes. For the outdoor bridge, we only test the locomotion capability of our robot due to the difficulty of scanning this much bigger structure. Fig. 14 shows our robot operating on surfaces of this bridge. Regarding the online performance, please refer to our supplementary video at https://youtu.be/g35J7auf6oM.

V Conclusions and Future Work

This paper presents a novel development of a multi-directional bicycle robot, which is capable of agilely climbing different steel structures to perform the structural inspection. The robot design is implemented and validated on laboratory testing structures and a cylindrical steel bridge. During all tests, the robot is able to firmly adhere to steel structures with various challenging levels. A rigorous theoretical analysis of the magnetic adhesive forces has been performed to confirm that the robot is able to work reliably in challenging situations of real applications. The robot can also operate well in limited spaces. A demonstration of deploying our robot to perform rust detection on steel bridges is shown. The visual data is collected and transferred to a ground station for visualization and processing to create a 3D map of the structures with marked locations of rusts. In the future, the robot can be further equipped with non-destructive testing (NDT) sensing modules for more in-depth inspections (steel thickness, fatigue cracks, and so on).

From the autonomy perspective, future directions can consider performing autonomous map construction, localization, navigation, and object detection all onboard. Given the known maps, it is also possible to perform path planning for autonomous inspections or even learning-based methods such as reinforcement learning to train the robot to carry the inspection task more efficiently.

References

[1] A. Sirken, G. Knizhnik, J. McWilliams, and S. Bergbreiter, “Bridge risk investigation diagnostic grouped exploratory (bridge) bot,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 6526–6532.
[2] D. Zhu, J. Guo, C. Cho, Y. Wang, and K. Lee, “Wireless mobile sensor network for the system identification of a space frame bridge,” IEEE/ASME Trans. on Mechatronics, vol. 17, no. 3, pp. 499–507, June 2012.
[3] J. Guo, W. Liu, and K.-M. Lee, “Design of flexonic mobile node using 3d compliant beam for smooth manipulation and structural obstacle avoidance,” in 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014, pp. 5127–5132.
[4] S. Kamdar, “Design and manufacturing of a mecanum sheel for the magnetic climbing robot,” Master Thesis, Embry-Riddle Aeronautical University, May 2015.
[5] N. H. Pham and H. M. La, “Design and implementation of an autonomous robot for steel bridge inspection,” in 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2016, pp. 556–562.
[6] W. Shen, J. Gu, and Y. Shen, “Permanent magnetic system design for the wall-climbing robot,” in IEEE International Conference Mechatronics and Automation, 2005, vol. 4. IEEE, 2005, pp. 2078–2083.
[7] S. T. Nguyen and H. M. La, “Development of a steel bridge climbing robot,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 1912–1917.
[8] G. Lee, G. Wu, J. Kim, and T. Seo, “High-payload climbing and transitioning by compliant locomotion with magnetic adhesion,” Robotics and Autonomous Systems, vol. 60, no. 10, pp. 1308 – 1316, 2012.
[9] T. Seo and M. Sitti, “Tank-like module-based climbing robot using passive compliant joints,” IEEE/ASME Transactions on Mechatronics, vol. 18, no. 1, pp. 397–408, Feb 2013.
[10] S. T. Nguyen and H. M. La, “Roller chain-like robot for steel bridge inspection,” in the 9th International Conference on Structural Health Monitoring of Intelligent Infrastructure (SHMII-9), Aug 2019.
[11] H. Eto and H. H. Asada, “Development of a wheeled wall-climbing robot with a shape-adaptive magnetic adhesion mechanism,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 9329–9335.
[12] Y. Takada, S. Ito, and N. Imajo, “Development of a bridge inspection robot capable of traveling on splicing parts,” Inventions, vol. 2, 2017.
[13] T. Bandyopadhyay, R. Steindl, F. Talbot, N. Kottege, R. Dungavell, B. Wood, J. Barker, K. Hoehn, and A. Elfes, “Magneto: A versatile multi-limbed inspection robot,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 2253–2260.
[14] A. Mazumdar and H. H. Asada, “Mag-foot: A steel bridge inspection robot,” in 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2009, pp. 1691–1696.
[15] P. Ward, P. Manamperi, P. R. Brooks, P. Mann, W. Kaluarachchi, L. Matkovic, G. Paul, C. H. Yang, P. Quin, D. Pagano, D. Liu, K. Waldron, and G. Dissanayake, “Climbing robot for steel bridge inspection: Design challenges,” in Austroads Publications Online, ARRB Group, 2015.
[16] S. T. Nguyen, A. Q. Pham, C. Motley, and H. M. La, “A practical climbing robot for steel bridge inspection,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 9322–9328.
[17] “Elios2,” https://www.youtube.com/watch?v=hW1Fn32JBls/.
[18] “Case study: Advancing bridge inspections with intel’s drone solutions, accessed on February 1, 2021,” https://www.intel.com/content/www/us/en/products/docs/drones/advancing-bridge-inspections-case-study.html.
[19] F. Tâche, W. Fischer, G. Caprari, R. Siegwart, R. Moser, and F. Mondada, “Magnebike: A magnetic wheeled robot with high mobility for inspecting complex-shaped structures,” Journal of Field Robotics, vol. 26, no. 5, pp. 453–476, 2009.
[20] G. Caprari, A. Breitenmoser, W. Fischer, C. Hürzeler, F. Tâche, R. Siegwart, O. Nguyen, R. Moser, P. Schoeneich, and F. Mondada, “Highly compact robots for inspection of power plants,” Journal of Field Robotics, vol. 29, no. 1, pp. 47–68, 2012.
[21] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in ICRA workshop on open source software, vol. 3. Kobe, Japan, 2009, p. 5.
[22] “Iso3691,” https://www.iso.org/obp/ui/#iso:std:iso:3691:-4:ed-1:v1:en/.
[23] “Iec60034,” https://global.ihs.com/doc_detail.cfm?document_name=IEC%2060034%2D1.
[24] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.
[25] T. Dang, F. Mascarich, S. Khattak, H. Nguyen, H. Nguyen, S. Hirsh, R. Reinhart, C. Papachristos, and K. Alexis, “Autonomous search for underground mine rescue using aerial robots,” in 2020 IEEE Aerospace Conference. IEEE, 2020, pp. 1–8.
[26] M. Bjelonic, “YOLO ROS: Real-time object detection for ROS,” https://github.com/leggedrobotics/darknet_ros, 2016–2018.
[27] M. Labbé and F. Michaud, “Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation,” Journal of Field Robotics, vol. 36, no. 2, pp. 416–446, 2019.
[28] H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart, and J. Nieto, “Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 1366–1373.