Wake-Informed 3D Path Planning for Autonomous Underwater Vehicles Using A* and Neural Network Approximations

For underwater applications, and specifically those with two or more interacting vehicles, an understanding of the best path through a fully realized 3D field is important. These operations are common and include, but are not limited to, launch and recovery, formation flying and multi-vehicle underwater cable and pipe inspection operations. In these applications, the 3D flow field, including wake structures and shed hydrodynamics each of the vehicles may pose control, safety and energy efficiency effects that are important to account for (Bhattacharyya, 1978).

To date, limited research has sought to model these effects within a path planner, at a level sufficient to encapsulate the complex 3D hydrodynamic physics, including the wake structures of nearby vessels. Existing methods either simplify the flow field in relation to its dimensionality (3D to 2D (Garau et al., 2005; Petres et al., 2007b)), include only the current, avoiding turbulent regions altogether (Pensado et al., 2024b), or use approximated hydrodynamic models (Zhang et al., 2022; Ronghua et al., 2024; Zadeh et al., 2018; Zeng et al., 2018; Li and Yu, 2023). Research focusing on UAV navigation in turbulent 3D aerial domains using the A* algorithm opted to completely avoid turbulent regions entirely to ensure flight safety (Pensado et al., 2024b). However, when conducting close proximity operations between vehicles, such as docking, formation flying or collaborative tasks, avoiding these regions is not always a practical or possible option. Vehicles may be required to navigate in close proximity to achieve their goals. These existing models may be inadequate to capture the fluid dynamic impacts present in multi-vehicle operations, where strong 3D wake interactions can significantly impact AUV control, safety, and energy consumption (Cooper-Baldock et al., 2023, 2024; Bhattacharyya, 1978).

To investigate whether a detailed knowledge of these wake hydrodynamics is required, and the benefits of doing so, two A* path planners will be developed for comparison – one in which only the global current is considered, and one where the detailed 3D wake structure is accounted for. Additionally, the use of machine learning based planners, trained on the A* trajectories, will also be investigated. Such an approach may enable the rapid prediction of feasible, energy-efficient paths without the need for computationally expensive A* searches, potentially bringing real-time 3D wake-informed navigation within reach.

To address these scientific gaps, this paper makes the following primary contributions:

1. 1.

   Proposes a wake-informed A* path planning algorithm that integrates full 3D hydrodynamic data, including complex wake structures, to achieve more energy-efficient and safer AUV navigation. Bridging the gap between simplified flow models and the complex realities of 3D underwater environments is crucial for operations involving significant wake effects (Cooper-Baldock et al., 2023, 2024).

2. 2.

   Introduces a machine learning framework trained on wake- and current-informed A* trajectories, enabling rapid real time path generation that approximates the near-optimal paths in real-time, levering existing techniques for ML informed planning (Tai et al., 2017; Pfeiffer et al., 2017).

3. 3.

   Develops a robust set of metrics to holistically evaluate energy consumption, path length, turbulence exposure, and computational efficiency, advancing the understanding of AUV path planning performance in complex 3D domains.

To contextualize this work, Section 2 reviews the current state of path planning in the underwater domain and identifies key gaps that motivate the study. Section 3 introduces the computational domain and the specific AUV configurations considered. In Section 4, the development of the wake- and energy-informed planning algorithm is presented, while Section 5 discusses the neural network approach used to approximate the planners for real-time applications. The performance of both the A* planners and their neural network counterparts is evaluated in Section 6, followed by a detailed discussion of the findings in Section 7. Finally, Section 8 concludes the paper and outlines potential directions for future research.

The environment is first discretized into a three-dimensional grid $G\subset\mathbb{Z}^{3}$, where each cell (voxel) represents a discrete location in 3D space. The grid dimensions are defined to be ($[1,128],\ [1,128],\ [1,128]$) representing the 3D $128^{3}$ grid structure. Each individual cell in the grid corresponds to a node $n$. These nodes are connected to their neighbors based on a 26-connected grid, allowing movement to any adjacent voxel sharing a face, edge, or corner. For a node $n$, its set of neighbors $N(n)$ includes all adjacent nodes that are within the grid bounds and are traversable (not occupied by obstacles). The Euclidean distance between two nodes $n=(x_{n},y_{n},z_{n})$ and $n^{\prime}=(x_{n^{\prime}},y_{n^{\prime}},z_{n^{\prime}})$ is denoted by $d(n,n^{\prime})$ and is given by Eq. (1):

The cost of moving from node $n$ to an adjacent node $n^{\prime}$ is defined based on the estimated energy required to overcome hydrodynamic drag force $F_{D}$ during that movement. This can be determined, as the velocity throughout the local environment is known (or estimated) within each cell of the grid $G$. The drag force ($F_{D}$) acting on the AUV is determined via Eq. (2). The AUV particulars and values used to enable accurate drag calculation for Eq. (2) are provided in B.

In Eq. (2) $\rho$ denotes the fluid density, $v$ denotes the magnitude of the relative velocity between the vehicle and the fluid, $C_{D}$ denotes the coefficient of drag for the AUV, and $A$ denotes its cross-sectional area. The drag can be used in conjunction with the Euclidean distance to determine the energy cost associated with moving to $n^{\prime}$. To calculate this cost of traversal ($n$ to $n^{\prime}$), we evaluate the drag force using the fluid velocity conditions at the destination node $n^{\prime}$, denoted $F_{D}(n^{\prime})$. The subsequent energy $E_{D}(n,n^{\prime})$ required to overcome this drag, over the distance $d(n,n^{\prime})$, is approximated as by Eq. (3).

The energy cost $E_{D}(n,n^{\prime})$ is then used to define the movement cost $c(n,n^{\prime})$, also known as the edge weight, for traversing from node $n$ to node $n^{\prime}$. A weighting factor $\omega_{1}$ (typically set to 1 unless scaling is desired) can be included as denoted in Eq. (4) which states the movement cost as a function of the energy ($E_{D}$) which is comprised of both the fluid forces ($F_{D}$) and the movement distance ($d$).

The A* algorithm searches for a path by maintaining the cost of the path found so far from the start node $n_{0}$ to the current node $n_{curr}$. This is known as the $g$-score, denoted by $g(n_{curr})$, and is calculated by summing the edge costs along the path given by $P=[n_{0},n_{1},\dots,n_{curr}]$. The equation for this is provided in Eq. (5).

The path planning problem is formulated as an optimization task. The design variable is the path $P$, represented as a sequence of connected nodes $[n_{0},n_{1},\dots,n_{k}]$ from a given start node $n_{0}$ to a specified goal node $n_{k}$. The objective is to find the optimal path $P^{*}$ that minimizes the total cumulative cost, $g(n_{k})$, required to reach the goal $n_{k}$, as stated by Eq. (6).

Here, $\overline{E_{D}}$ represents the global average of the energy term, which consists of the weighted drag force term, $\omega_{1}\cdot F_{D}(n^{\prime})$ (derived from Eq. 4). This heuristic combines the geometric distance to the goal with an overall estimate of the environmental energy resistance. Crucially, this heuristic is admissible because the average energy cost $\overline{E_{D}}$ will not overestimate the actual minimum energy cost per unit distance along any specific optimal path segment. Since the actual cost on the optimal path from $n$ to $n_{k}$ involves traversing specific cells with potentially lower-than-average or higher-than-average costs, using the global average to guide the search prevents systematic overestimation of the remaining cost. Admissibility is a required property for A* to guarantee finding the minimum cost path (Hart et al., 1968). While simple, this heuristic provides a computationally inexpensive way to focus the search, balancing distance and the energy landscape of the grid $G$.

The A* algorithm explores nodes based on the priority function $f(n)=g(n)+h(n)$. It maintains an ’open set’ (priority queue) of nodes to be evaluated, ordered by their $f$-score, and a ’closed set’ of nodes already evaluated. Starting with $n_{0}$, it iteratively selects the node $n$ with the lowest $f$-score from the open set, expands its neighbors $n^{\prime}$, calculates their tentative $g$-scores and $f$-scores, and updates them if a cheaper path is found. The process continues until the goal node $n_{k}$ is selected for expansion, at which point the optimal path is reconstructed by backtracking from the goal.

To assess the importance of knowing the wake structures, two variants of the planner will be assessed. The first variant, $P^{*}_{C}$ will only be informed of the current within the computational domain. The second variant, $P^{*}_{W}$, will be informed of both the current and the vehicle’s wake structure. $P^{*}_{W}$ will be applied to the flow field across a range of wake fields behind a large underwater vehicle. The current and wake dataset was generated from prior work by the authors and is comprised of high fidelity computational fluid dynamics (CFD) data interpolated onto the discrete grid $G\subset\mathbb{Z}^{3}$ (Cooper-Baldock et al., 2024) (Cooper-Baldock et al., 2023). More details about this are provided in A. The data contains a local current from 0.10 m/s to 5.00 m/s in increments of 0.10 m/s. Additionally, a strong propeller wake is included, with wake structures at 0° to 60° degrees of separation, simulating a turn of the vehicle in question. This was done via the simulation of an INSEAN E1619 propeller. The wake free velocity field ($v_{C}$) containing only the current, will be the same field, with the wake structure’s components filtered out and only global current retained.

Fig. 3 details the A* path planner methodology. The positions of the XLUUV and AUV (x, y, z) are used to discretize a 3D domain in the $128^{3}$ structure as discussed. This grid structure is then augmented with either the current or wake flow data as denoted (blue and purple respectively). Following this, the path planner is started, with the start and goal locations ($n_{0},n_{k}$) defined. A priority queue is initialized before the assessment process for the neighboring nodes ($n^{\prime}$) using the search heuristic ($h$) is undertaken. This process is iterated, with the best node selected based on the cost until the goal location is reached. At this point, the planner is stopped, as a trajectory has been generated.

To compare the effectiveness of the wake-informed A* planner and the current-informed A* planner, many trajectories needed to be generated for assessment. This was achieved through the use of the CFD dataset (A). This dataset contained 500 distinct flow fields, each of which were characterized by a different speed and angle pairing, and resulted in a different wake flow field behind the XLUUV.

For each of these 500 instances, the location of the AUV was randomly generated on the rear boundary of the 3D domain (depicted in Fig. 4). If the current-informed planner was being assessed, the grid velocities $G(v)$ would be overwritten with the mean velocity value ($\overline{v}$) in all cells, resulting in a domain comprising of only the global current. If the wake-planner was being assessed, then the wake and detailed velocity structure of the CFD data would be retained.

The randomized start points of the AUV, of which 36 were generated for each of the 500 fields, were held consistent for both the current- and wake-informed planners. From these starting positions, the path planning process (outlined in Fig. 3) would be followed, until the AUV had been navigated to the center of the payload bay onboard the XLUUV (the goal $n_{k}$). Using 36 trajectories for each of the 500 flow conditions resulted in 18,000 paths being determined for the current- and wake-informed planners respectively.

An example instance, of 1 of the 500 distinct flow fields, is provided in Fig. 4. In this image, the 36 paths generated for the wake-informed A* planner are shown. The 36 starting locations of the AUV on the rear boundary are denoted by the green spheres. The trajectories towards and into the XLUUV are denoted by the red lines, and the wake velocity data ($v_{wake}$) is provided in blue, with the XLUUV located in the center of the wake field.

The energy capacity onboard an AUV is extremely limited. This is due to their small physical size and inability to house large battery systems (Cooper-Baldock et al., 2023). Due to the limited energy budgets, there are several important performance metrics that should be assessed for any trajectory.

A successful trajectory should be energy efficient, short in physical length and also minimize the traversal of adverse high velocity or turbulent regions of the domain $G$. To assess these metrics, several performance metrics have been designed. These are the method to determine the energy cost of the path ($E$), the length of the path ($L$), the number of high velocity cells traversed $N_{\text{high-velocity}}(P)$) and the number of turbulent cells traversed ($N_{\text{high-velocity}}(P)$).

To make these assessments, each generated path, $P$, is decomposed into a set of $N$ nodes such that:

Each individual node ($n_{i}$) in the set corresponds to a 3-dimensional coordinate $(x_{i},y_{i},z_{i})$ in the grid. The velocity at node $n_{i}$ is $v(n_{i})$. $\tilde{v}$ denotes the median velocity of the entire grid $G$, and $\sigma_{v}$ denotes the standard deviation of the velocities in the grid. Crucially, whilst the current-informed planner operated without knowledge of the wake flow in $G$, it will be assessed in a domain containing the wake flow, in order to assess the impact of the wake on the metrics identified above and the impact of accounting for the wake in the trajectory planning stage. $\sigma_{v}$ is used to identify high velocity cells, where we define a high velocity cell to be 1 standard deviation or more from the median velocity ($\tilde{v}$). Where an individual node ($n_{i}$) encounters a cell that is more than 1 standard deviation from the velocity field’s mean value in the grid $G$, it is recorded, indicating the traversal of a high velocity cell. This threshold is given by Eq. 9

The number of cells along the trajectory that exceed this threshold is denoted by Eq. 10. This equation tracks the number of high velocity cells via $N_{\text{high-velocity}}(P)$ and is calculated via Eq. (10), where the cutoff threshold for a high velocity cell is denoted by the indicator function in Eq. (11) and given via $\delta_{\text{V}}$ as mentioned.

To assess the degree of turbulence encountered over the trajectory, a similar method is applied. Where the velocity is found to change at a given node (via $v^{\prime}(n_{i})$), it is summed, and thus tracks the number of velocity fluctuations recorded over the path of the trajectory (through $N_{\text{turbulent}}(P)$). The CFD flow field data, determined via ANSYS Fluent, may have slight fluctuations due to numerical precision, which can result in small changes in velocity despite uniform freestream velocity conditions. To stop these cells from being incorrectly tracked, a small offset is applied $(\varepsilon=0.005)$ ensuring that the velocity fluctuation, sufficient to be recorded as a turbulent cell transition, must be more than 0.5% of $v$. The two equations to enable this tracking are Eq. (12) and (13) respectively.

To determine the energy cost of a path, the drag force ($F_{D}$) as denoted by Eq. 2 and distance ($d$) denoted by Eq. 1 of each node ($n_{i}$) and the next ($n_{i+1}$) are summed along the trajectory via the application of 14. This yields the energy cost of a path $E(P)$ in joules.

The total path length ($L$) along the set of nodes in $P$, denoted $L(P)$, is the sum of the distances between consecutive nodes where $d_{i}$ is as previously defined by 1.

To train the networks to predict the trajectories of the current and wake-informed A* planners, a dataset of their trajectories will be used. Each dataset consists of 18,000 trajectories generated by the wake-informed and current-informed A* planners counterparts, as described in Section 4.3. Each trajectory comprises up to 130 waypoints, with each waypoint represented by its three-dimensional coordinates $(x_{i},y_{i},z_{i})$. The input features for the neural network include the vehicle’s starting position $(x_{\text{start}},y_{\text{start}},z_{\text{start}})$, goal position $(x_{\text{goal}},y_{\text{goal}},z_{\text{goal}})$, flow speed $v\in[0.1,5.0]$ m/s, and flow angle $\theta\in[0^{\circ},60^{\circ}]$. The network output is a vector of sequential waypoint coordinates representing the predicted trajectory. Due to varying path lengths, all trajectories were padded to a fixed length of 130 waypoints, and a mask was applied during training to handle the variable lengths, that may be shorter than this length of 130 cells.

Both neural network architectures are fully connected feedforward networks, designed to regress the sequence of waypoint coordinates directly from the input features. The architecture consists of an input layer, two hidden layers and an output layer as detailed in Fig. 5.

Further details are provided in Table 7 in C. The choice of layer sizes was determined empirically to balance model capacity and computational efficiency. ReLU activation functions were used to introduce non-linearity and mitigate the vanishing gradient problem (Glorot et al., 2011). A fully connected architecture was chosen for its simplicity and to serve as a baseline, ensuring that future studies can easily compare advanced architectures (e.g., RNNs, Transformers) against this initial model.

Both input features and output waypoint coordinates were normalized using the standard score as represented by Eq. (16).

The network was trained using the Adam optimizer (Kingma and Ba, 2015) with a learning rate of $1\times 10^{-4}$. Adam was chosen for its adaptive learning rate capabilities and robustness in training deep neural networks. The batch size was set to 64, and the model was trained for 50 epochs. Weight initialization was performed using the Xavier initialization method to ensure stable gradients at the start of training (Glorot and Bengio, 2010).

To prevent overfitting, three regularization techniques were implemented: (1) Early Stopping: The validation loss was monitored, and training was halted if the loss did not improve for a set number of epochs (10). (2) Dropout: Although not included in the final model, dropout layers were experimented with during hyperparameter tuning but did not yield significant improvements. (3) L2 Regularization: L2 weight decay was applied implicitly via the Adam optimizer’s default settings.

The model’s performance was evaluated using the root mean squared error (RMSE) between the predicted and true waypoint coordinates on the validation set only. Additionally, the trajectories generated by the neural network were assessed using the same metrics as the A* planners, including total energy consumption, path length, number of high-velocity cells, and number of disparity cells, as defined in Section 4.4 and presented in Section 6.

The neural network was implemented using PyTorch (Paszke et al., 2019), a widely used deep learning framework that provides dynamic computation graphs and efficient GPU acceleration. Training was performed on a workstation equipped with an NVIDIA RTX2070 GPU to expedite the computational process. Further details, if required, are provided in Table 7 in C.

While the neural network provides rapid trajectory predictions, it inherently relies on the quality and diversity of the training data. As the model is trained on trajectories generated by the A* planners, its performance is directly tied to the scenarios represented in the training set. Extrapolation to unseen flow conditions or starting positions not represented in the training data may result in decreased accuracy, and will be assessed within this analysis.

Energy efficiency is critical in AUV operation due to their limited onboard energy resources. The wake-informed A* planner ($W.I._{A*}$) consistently demonstrates the lowest total energy expenditure, as determined via Eq. 14, across all speed ranges, as shown in Table 2. At the lowest speed range of 0.1–0.5m/s, $W.I._{A*}$ achieves an average energy expenditure of 51.49 joules with a standard deviation of 52.82 joules, outperforming the current-informed A* planner ($C.I._{A*}$) by approximately 11.3%. This persists at higher speeds, with $W.I._{A*}$ maintaining lower energy consumption compared to the other models. At the maximum tested speed range of 4.0–5.0m/s, $W.I._{A*}$ records an average energy expenditure of 11,952.30 joules, 7.2% less than $C.I._{A*}$.

The neural network models, $C.I._{NN}$ and $W.I._{NN}$, based on their counterpart A* implementations, exhibit higher energy expenditures compared to their calculated A* trajectories. $W.I._{NN}$ performs better than $C.I.{NN}$, but does not match the energy efficiency of the wake-informed A* planner. The increased energy expenditure (ranging between 10.03–13.12%) in the neural network counterparts may be attributed to approximation errors inherent in learning-based approaches, where it remains challenging to capture the optimality of the A* algorithm in path planning, especially in the presence of complex wake structures. Provided in Fig. 6 is a depiction of the variation in standard deviation against speed and angle for both planners and networks. It can be seen that the current-informed NN has significant variation in path energy, indicating it may struggle to understand how best to traverse the domain, particularly at high speeds.

The path length metric ($L$), as given in Table 3, and determined via Eq. (15), reflects the total distance traveled by the AUV along the planned trajectory. Shorter paths generally result in quicker LAR missions, for a given speed, but may traverse more hazardous areas. The current-informed A* planner ($C.I._{A*}$) achieves the shortest average path lengths across all speed ranges, with consistent trajectory lengths of 104.22–104.31m and a standard deviation of approximately 9.5m. This suggests that $C.I._{A*}$ is able to generate the most direct paths, but without considering wake structures and associated hazards. In contrast, the wake-informed A* planner ($W.I._{A*}$) shows slightly longer path lengths, averaging around 107 m, indicating that it deliberately avoids certain areas to minimize energy expenditure and exposure to high-velocity regions. The neural network models, particularly $C.I._{NN}$, exhibit significantly longer path lengths with higher variability. For example, $C.I._{NN}$ records an average path length of 133.27m in the 4.0–5.0m/s speed range, approximately 27% longer than $C.I._{A*}$. This suggests that the neural networks may not approximate the optimal path lengths as effectively, potentially leading to suboptimal routing. Fig. 7 provides an assessment of the median path length of the models against speed and angle. It can be seen that the A* current-informed method is the most consistent, with the A* wake-informed method also exhibiting similar performance. The neural networks result in a higher median path length, with suboptimal path generation at low speed and angles for the wake-informed NN.

Navigating through high-velocity cells poses increased risk and potential for energy expenditure, due to increased drag and control challenges associated with traversing the larger vehicle’s wake structure. This data is provided in Table 4. The wake-informed A* planner ($W.I._{A*}$) exhibits the lowest average number of high-velocity cells encountered ($N_{\text{high-velocity}}(P)$) across all speed ranges, as it has knowledge of the structures. At speeds between 0.5–1.0 m/s, $W.I._{A*}$ encounters an average of 2.73 high-velocity cells, significantly lower than $C.I._{A*}$ (7.32) on average. This indicates that $W.I._{A*}$ sufficiently avoids the high-velocity regions it is informed of, enhancing operational safety. Both neural network models exhibit less consistent performance in this metric. $C.I._{NN}$ is observed to reduce high-velocity cell encounters compared to $C.I._{A*}$, whilst $W.I._{NN}$ generally encounters more high-velocity cells than $W.I._{A*}$. This suggests that the neural networks may not fully capture the nuances of avoiding hazardous regions, possibly due to limitations in the training data or model capacity. Fig. 8 details the median number of high velocity cells encountered by each method. Similar performance is observed for the current-informed A* and NN, and the wake-informed NN. The wake-informed A* planner exhibits beneficial performance in comparison to the other models, with typically 0 - 5 cells encountered irrespective of speed and angle.

The turbulent cell count ($N_{\text{turbulent}}(P)$), as given in Table 4, reflects the number of cells along the trajectory where any velocity fluctuation occurs, which would pose an additional control challenge for the AUV. It should be noted that no thresholding is applied to this, so small and large fluctuations are treated equally. The current-informed A* planner ($C.I._{A*}$) records the lowest turbulent cell counts across all speed ranges, with an average of 31.29 cells in the 0.5–1.0 m/s range. The wake-informed models, both $W.I._{A*}$ and $W.I._{NN}$, exhibit higher turbulent cell counts. This suggests that while the wake-informed planners effectively avoid high-velocity cells, they may traverse areas with higher turbulence. One possible explanation is that in avoiding high-velocity wake regions, the wake-informed planners route the AUV through areas with greater velocity disparities. The neural network models again display higher turbulent cell counts compared to their A* counterparts, indicating a potential area for improvement in modeling turbulence avoidance.

Computational efficiency is critical for real-time path planning on resource-constrained AUVs. As shown in Table 5, the neural network models offer a significant advantage in computational time. The neural networks compute trajectories in approximately 0.30 milliseconds, whereas the A* planners require 37 to 43 seconds on average, depending on the method. This is a compute time reduction of approximately 5 orders of magnitude - a significant decrease. The wake-informed NN, $W.I._{NN}$, achieves the shortest computation time across all speed ranges, with an average of 0.29 milliseconds. The substantial reduction in computation time makes the neural network models highly optimal for real-time applications. This speed, however, comes at the cost of reduced path optimality compared to direct computation via A*, as evidenced by the higher energy expenditures and longer path lengths compared to both A* planners.