Autonomous Electric Vehicle Battery Disassembly Based on NeuroSymbolic Computing

Disassembly strategy study is one of the earliest researches for battery disassembly tasks, which currently are primarily carried out by humans [2, 3, 4]. From 2014 to 2015, researchers designed a disassembly workstation and conducted in-depth research on the Audi Q5 battery pack [5]. Recent research work is to further refine the problem, mainly according to the geometric structure of power battery, using genetic algorithm, matrix or graph theory to make decision and optimization on disassembly behavior, disassembly sequence and even disassembly trajectory [6, 7]. The research in this field has been mature, which provides scientific analysis and guidance for the efficient disassembly of power batteries. However, the specific implementation links still need to be completed manually. Some researchers tried to use pre-programmed robots to automatically accomplish the task [8, 9], which cannot be deployed extensively due to the highly dynamic disassembly environment and battery pack variations.

In addition to the above work, a lot of research has been done on robot task planning. Robot task planning refers to selecting a proper sequence of actions and constraints according to the initial state and target state of the manipulated object without interference, changing the state of the robot and the manipulated object, and realizing the transition from the initial state to the target state [10, 11]. Most of these methods carry out task planning based on artificial intelligence. First, they define different tasks and expand PDDL [17, 12] to achieve different solutions. For instance, in multi-stage complex tasks, the hierarchical planning method is adopted to decompose the overall robot task into simple tasks [13]. Furthermore, some scholars focus on motion planning and decision-making under uncertainty [14], task planning and decision-making of robot pushing away from object under chaotic environment [15], real-time online task planning and decision-making of intelligent robot operating object behavior [16] and so on. Although such research results are remarkable, they all have a common problem: the planner requires inputting the symbolic representation of the system state, which often needs to be pre-defined by semiotics experts, which significantly limits the application scope of task planning. At present, such research is still in the laboratory stage.

Along with the development of deep learning, some researchers have tried to use computer vision neural networks to identify target types and locations, and then do motion planning for robots [4]. In addition, for task planning these methods usually assume accurate perception results which are either expensive to obtain or impractical due to stringent geometric accuracy requirements. Above all, the major obstacle towards disassembly is the inability of robots to plan based on perception results under an unstructured environment autonomously.

NeuroSymbolic computing is a new technique that has the potential of organically integrating high-level inference with lower-level perception. The framework is of two levels: The high-level conducts knowledge-based logic inference, while the low level carries out probabilistic learning-based perception and control, etc. [21]. On one hand, it is imperative to design explicable and reliable task planning based on real-time information, which demands symbolic inference capabilities. On the other hand, it is impractical to symbolize all disassembly status [22], which is also due to complicated battery disassembly scenarios. This has made probabilistic learning methods indispensable to learning symbolic representation from visual inputs. The high-level symbolic inference system can narrow down contexts for low-level perception, which greatly reduces the perception model complexities and expands its applicability in practical industrial scenarios [23, 24]. Based on these understandings, we use NeuroSymbolic methods to shed light on autonomous battery disassembly solutions.

Disassembling bolts are straightforward to humans while challenging for robots to reliably and autonomously accomplish with the presence of environmental uncertainty. The goal is to design an algorithm to let robots autonomously carry out bolt disassembly tasks. To simulate environment dynamics, we use random obstacles around the bolts to test if the robot can choose the right action under different circumstances. This is critical to achieving the autonomous disassembly goal.

Through analyzing the characteristics and focuses of the bolt disassembly process [25], we designed five action primitives for the task: Approach, Mate, Push, Insert, Disassemble.

(1)Approach This primitive directs the robot arms’ nut runner to approach the bolt from above. This primitive requires the bolt’s general location, which could be coarse, and only changes the robot arm’s position with no interactions with the target bolt.

(2)Mate This primitive adjusts the robot arm’s nut runner to get its centerline aligned with the bolt’s centerline.

(3)Push If perception indicates that there is not enough room for the nut runner to conduct disassembly, this primitive will be invoked to push away obstacles around the target.

(4)Insert This primitive directs the nut runner to rotate around the bolt axis to ensure valid contact, normally with a contact depth of 1mm to the gasket under the bolt. Meanwhile, the nut runner slowly rotates clockwise, ready to stop if a torque larger than 5Nm is detected to avoid anomalies.

(5)Disassemble This primitive directs the nut runner to rotate counterclockwise and move away from the bolt at a speed equivalent to the amount of pitch that the bolt moves per second.

To achieve autonomous disassembly under dynamic and unstructured environments, the robot needs to choose appropriate primitives and sequences autonomously based on the current situation.

We formulate the bolt disassembly task as a planning issue ($S_{0}$,$S_{G}$,A), $S_{0}$ as the initial status of the disassembly system, $S_{G}$ as its target status, A as a sequence of disassembly primitives A = a, which consists of primitives from the aforementioned primitives set (Approach, Mate, Push, Insert, Disassemble).

Each primitive is defined by a set of descriptors: action(a), pre(a), eff(a), param(a). Action(a) denotes the name of the primitive, with its required parameters described in param(a). Pre(a)defines this primitive’s prerequisite conditions, while eff(a) represents this primitive’s effects after being carried out.

Planning techniques focus on finding a disassembly primitive sequence (plan = ($a_{1}$,$a_{2}$,…,$a_{i}$,…,$a_{n}$)), and by executing these primitives in turn, the system moves from its initial status $S_{0}$ to the target status $S_{G}$. PDDL is a standard language to describe this formulation. This paper uses PDDL to define different disassembly status and actions, and based on PDDL implements autonomous battery disassembly.

Predicate logic has been extensively adopted owing to its strong descriptive capabilities [26]. Predicates, as the core concept of predicate logic, are used to describe relationships between events and objects. The classical method refines the predicate definition continuously until the predicate corresponds to the sensor data to complete the relational representation. Defining predicates in this manner requires expert knowledge and is only applicable in restricted scenarios, which makes it unsuitable for dynamic and open environments.

To address these challenges this paper introduces a neural predicate, which is both a predicate and a neural network. It is responsible to convert continuous status to symbols that could be exploited by primitives. This method does not require complicated rules defined by experts beforehand. We use p(sensor,O)to denote the neural predicates. Neural predicates take sensory data as the input. If a neural predicate has n groundings, then we use an n-dimensional vector as the output to represent this predicate’s probabilistic distribution across n categories. For example, if a neural predicate $clear$ could be $True$ or $False$, then the output will be $[p_{1},p_{2}]$ that corresponds to the probabilities of clear(sensor,True) and clear(sensor,False), respectively. Deep learning’s softmax output matches well with the n-dimensional probabilistic distribution, hence adopted as the output.

We define two neural predicates for the bolt disassembly system (shown in ”Figure 3”): ‘$target\_aim$’ to indicate target aligned and ‘$target\_clear$’ to indicate that there are no obstacles.

It is worth mentioning that since neural predicates classifications are defined for a specific scenario, neural network training becomes much easier. In this paper’s tests, we use 400 pictures to train each neural predicate and start with the pre-trained VGG-16 [27] for the training. Predicate ‘$target\_aim$’ achieves an accuracy of 98% after 40 epochs of training, while predicate ’$target\_clear$’ achieves an accuracy of 96% after 40 epochs of training. Such accuracy well suffices to serve the following inferring tasks.

In robot task planning scenarios, it’s the primitives that bridge logic planning and the robot’s physical movements. PDDL can accurately describe in symbolic space each primitive and its prerequisite (:pre), effects after execution (:eff). It also unambiguously describes the system’s initial and goal states. In the symbolic space, the PDDL-based planner infers the primitive sequence from the initial state (Init) to the target state (Goal) as the action plan. Battery disassembly, specifically bolt disassembly tasks, involves primitives including Approach, Push, Mate, Insert, and Disassemble, etc. Their corresponding PDDL definitions are as follows:

The system defines for each primitive its controller. Primitive controllers control robots to execute in the physical environment which is prescribed by the primitives, i.e. when prerequisite (:pre) is satisfied, using parameters (:param) as input, to control the robot to transit from its current status to status described in (:eff). For the sake of testing, this paper uses ROS Moveit to develop primitive controllers. Note that primitive controllers can be implemented by many other ways.

With abstractions supported by neural predicates and disassembly primitives, the original complicated planning problem in the continuous space is converted to finding the best action plan in the symbolic space. By following the action plan sequentially, the disassembly system makes the robot transfer from its initial status $S_{0}$ to its target status $S_{G}$. We use forward searching to address this issue. Algorithm 1 shows the specific algorithm.

Even though the neural predicates and probabilistic representations of status are introduced to avoid inconsistencies caused by manually defined thresholds, there still exist planning uncertainties. For example, after approaching the target object by executing initial locating primitives, is it still necessary to adjust robot poses? While executing the “Push” primitive, is it ensured that obstacles are fully cleared up? To minimize human interventions, after the execution of each primitive, the system will ensure that it has achieved the desired results. If not, the system will re-execute the planning algorithm to obtain a new action plan based on the current status and the goals.

Figure 4a shows the test environment. We use a visual simulation environment Gazebo with a 7-DOF robot arm xMatePro7 ,a product of ROKAE, as the testing platform. The robot arm is equipped with a nutrunner at its end, which could be controlled by ROS MoveIt!. The nutrunner is actually a pneumatic torque actuator(as shown in Figure 4b) which is composed of visual input module, power subject, sleeve, flange base and limit shell, etc [28]. In more detail, the visual module uses Intel RealSense LiDAR Camera L515 to estimate the position and pose of the target object, the power subject similar to the internal structure of the pneumatic wrench converts the kinetic energy of the high-pressure air into the kinetic energy of the sleeve rotation, the sleeve is mounted on the extension rod (the sleeve can be replaced in different sizes), the flange base of the actuator is fixed at the end of robot arm, high-pressure air enters the power subject through the air inlet, the limit shell aims to prevent the extension rod from swinging in a wide range. The overall structure is mainly used to realize the passive compliant disassembly of the bolt due to insufficient accuracy in the position and pose estimation of the bolt, compensating for the error by using mechanical structure.

In addition, the simulation platform is also equipped with a global camera (on top, overseeing the whole environment) and a regional camera (at the end of the robot arm). Camera parameters are consistent with Intel RealSense Depth Camera D435i. The simulator is also configured with a Chang’an motors EADO EV460 battery pack, bots, and a woodblock as the obstacle. All tests are run on an Intel NUC.

First, we observe if the proposed method could autonomously adapt to different scenarios with different task plans. Next, we use SR (Success Rate) to evaluate the algorithm’s effectiveness against perception errors. The baseline is a classic method that treats perception and control as separate tasks and the control module assumes accurate enough target positions from perception. Note that the accurate target positions could be from either symbolic or neural methods. Here we intend to compare how perception errors affect the proposed and classical solutions.

Figure 5 shows three different situations that the robot might face during the task:

1. 1.

   The robot arm has accurate location information of the battery bolt;

2. 2.

   The robot arm does not have accurate location information of the battery bolt;

3. 3.

   There are obstacles near the bolts.

From Figure 5, it can be seen that the proposed method autonomously adjusts the task plan to adapt to different situations. For case 1 it takes only 3 steps to accomplish the task. For case 2 since the robot does not have an accurate bolt location, it takes an extra step to make sure the nut runner and the bolt are lined up with each other properly. For case 3 due to both inaccurate location information and obstacles around the bolt, the system takes two extra steps to first line up the nut runner with the bolt and then clear the area around the bolt. The test result shows that the proposed method can autonomously plan and execute based on the perceived situation.

To test the robustness of NeuroSymbolic TAMP against inaccurate target locations for battery disassembly task, we randomly position the target bolt on the platform and then introduce an error with $N(0,\sigma^{2})$ distribution to the position. The tests are repeated with different $\sigma$ to obtain SR.

Figure 6a shows the test results: The classic method [10] achieves a decent SR while the variance is less than 1mm, which its performance quickly deteriorates as the variance increases. The baseline method achieves decent performance in a static and structured environment, which reflects the current industry status. Unstructured environments pose a completely new set of challenges to the classic methods, though. In comparison, the proposed method can adapt and adjust based on online scenarios to achieve a nearly 100% SR. Among the 400 trials, the 3 Failures are due to inadequate error handling of certain primitives (deadlock for the system cannot find the bolt’s 6 corners under poor lighting conditions). The high success rate of the proposed method is mainly due to its ability to execute the $Mate$ action autonomously (as shown in Figure 6c). The experimental results show the broad prospect of man-machine cooperation due to high success rate and enhanced security compared to the existing solution [4].

This is to test the proposed method’s effectiveness with obstacles. The tests are conducted randomly initializing the target bolt’s position on the platform, then use this location with a deviation of $N(0,\sigma^{2})$ distribution as the obstacle’s location. Obstacles include bolts, nuts, and wooden blocks. The tests are conducted with different $\sigma$ to calculate the SR.

Figure 6b shows the test results. The classic method is not capable of cleaning up obstacles, hence its poor SR when the obstacles are of smaller $\sigma$ (the obstacles are highly likely to block the bolts) and better SR with large $\sigma$. This is exactly what existing solutions need to be improved and what the proposed method intends to address. The method proposed by this paper achieves a high SR consistently, regardless of the $\sigma$ value. It should be emphasized that the proposed method does not execute ’$Push$’ every time, as shown in Figure 6c. It autonomously decides whether ’$Push$’ is necessary based on the current status. When $\sigma$ is small, i.e., it is more likely that the obstacles are in the way, the system will execute this action sequence: $Approach\to Push\to Insert\to Disassemble$. When $\sigma$ is large, i.e., it is less likely that the obstacles are in the way, the system will execute this sequence of 3 actions: $Approach\to Insert\to Disassemble$. From this perspective, we can highlight the flexibility of the proposed method with better adaptation to the real site compared to the pre-programming solution [5].