Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigation

To extract object-related noun features from the instruction, we begin by choosing word embeddings describing objects from $\mathcal{W}$. Specifically, our process begins by obtaining labels for all objects from the MatterPort3D simulator [47]. These labels are then compiled into a noun database, denoted as $D$. For a given natural language instruction $\mathcal{W}$, we iterate through each word. If a word is found within $D$, it is selected as an object-related token.

Upon acquiring these object-related embeddings, we enhance them with positional embeddings as described in [48]. These positional embeddings correspond to the respective word’s location within the sentence. Additionally, a type embedding specific to text, as outlined in [49], is also incorporated. Next, we input all noun tokens into a text encoder that consists of a multi-layer transformer. This process generates contextual noun representations, which we refer to as ${\mathcal{\hat{W}}}=\{\boldsymbol{\hat{w}}_{1},\boldsymbol{\hat{w}}_{2},\dots,\boldsymbol{\hat{w}}_{\hat{L}}\}$.

In the study of object relations, to circumvent the issue of inadequate learning capabilities resulting from the shallow nature of GCN, we have designed a temporal object relations (TOR) module. This module employs a cross attention mechanism to focus on objects observed during navigation and noun embeddings in the instruction. Through this approach, we learn a relationship matrix accurately as the agent progresses along its exploration trajectory. This ensures temporal continuity in the agent’s learning process.

Fig. 3(a) depicts how the agent, when arriving at a new location, establishes connections between the objects it perceives and the relevant nouns from the navigation instruction. Specifically, when the agent reaches a location, it obtains a panoramic view of that position. The agent employs a cross attention mechanism to learn the associations between all objects discovered at this location and the nouns mentioned in the instruction. This process is consistently applied at each location along the agent’s navigational path. It sets the stage for the agent to learn and progressively refine the inter-object relationships across temporal dimensions.

In our approach, we treat the object features, denoted as $O_{t}$, as the query, and the noun features, $\hat{W}$, as the key. We employ a cross attention mechanism to compute the relationship matrix $T$, which can be expressed in the following way:

where $\rm FC$ is a fully connected layer.

We leverage $\mathbf{T}$ to derive the temporal object features $\mathcal{M}_{t}$. It is formalized as follow:

Due to the gap between external knowledge and the navigation environment, the agent is unable to accurately learn the relationships between objects based on external knowledge. To bridge this gap, we introduce the spatial object relations (SOR) module. This module considers all viewpoints in the environment and ensures complete spatial coverage, which help the agent learn a more precise object relations.

During the establishment of spatial object relations, the agent proceeds to update an initially zero-valued relationship matrix $\mathbf{E}$ based on the objects identified at each respective location. As illustrated in Fig. 3(b), for objects $x$ and $y$ observed concurrently at the same location, a shorter distance between them correlates with a stronger association. Consequently, we update the relationship matrix based on the distances between objects as seen by the agent at each location within the environment. The update rule for $\mathbf{E}$ is formalized as follows:

where $\boldsymbol{v}_{x}$ and $\boldsymbol{v}_{y}$ denote the perspectives from the current position in relation to objects $x$ and $y$, respectively, and $\boldsymbol{d}_{x}$ and $\boldsymbol{d}_{y}$ represent the respective depths of objects $x$ and $y$ from the current position. Here, $k_{1}$, $k_{2}$, and $k_{3}$ are predefined constants, and $\|\cdot\|_{2}$ denotes the L2 norm.

In the course of the agent’s training, the pertinent matrix $\mathbf{E}^{\prime}$ is retrieved from $\mathbf{E}$, guided by the objects detected at the agent’s current location and the nouns encapsulated within the instruction. Assuming the agent currently discovers $c_{1}$ objects, and the current instruction contains $c_{2}$ nouns, the calculation of $\mathbf{E}^{\prime}$ is as follows:

where $p(i)$ is the index of the i-th object and $q(j)$ is the index of the j-th noun. Subsequently, matrix multiplication is employed to derive the environmental object feature $\mathcal{N}_{t}$, as per the following equation:

Upon obtaining both the temporal object features $\mathcal{M}_{t}$ and the spatial object features $\mathcal{N}_{t}$, the final object relationship feature $\mathcal{Q}_{t}$ is computed utilizing the equation:

where $\alpha_{1}$, $\alpha_{2}$, and $\alpha_{3}$ are three learnable parameters, summing to 1. Subsequently, a concatenation of $\mathcal{Q}_{t}$ and the image feature $\mathcal{R}_{t}$ is performed to yield the panoramic feature $\mathcal{F}_{t}=\left[\mathcal{R}_{t},\mathcal{Q}_{t}\right]$

Building upon the foundational work presented in [50, 51, 39], the advantageous impacts of pretraining transformer-based models within the domain of Vision-and-Language Navigation (VLN) have been well-established. In this work, we employ four auxiliary tasks to pretrain our proposed model.

In the phase of fine-tuning, we employ a triad of loss functions: Single-step Action Prediction (SAP), Object Grounding (OG), and Turning Back Penalty (TBP), to meticulously guide and refine the learning process of the agent. Distinct from the approach of utilizing a demonstration path in the pre-training phase, during fine-tuning, the model is supervised by a pseudo-interactive demonstrator. This pseudo-interactive demonstrator operates by employing the shortest path algorithm at each decision point, selecting a node as the ensuing location. This selection is made such that it adheres to the criterion of minimizing the overall path length from the agent’s current location to the target destination. The cumulative loss function that governs the fine-tuning process is formulated as follows:

In this expression, $\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$ serve as balance factors, ensuring a harmonious integration of the individual loss components.

For inference, the agent predicts one action at each step. If the action is not a stop action, it executes the action and moves to the appropriate position. Otherwise, the agent stops at the current position. When the number of actions executed by the agent exceeds the maximum number of actions specified, it is forced to stop and the node with the maximum stop probability is taken as the final position. When the agent stops, it will take the object with the maximum prediction score as the target.

For the extraction of object features within the REVERIE dataset, which conveniently offers bounding boxes, we employ the ViT-B/16 model [53], pretrained on the ImageNet dataset. In contrast, when dealing with the SOON dataset, object bounding boxes are extracted using the BUTD object detector [46] due to its absence of provided bounding boxes. For the R2R dataset, we do not employ the spatial object relations module because of the unavailability of object annotations.

Further delving into our architecture, we incorporate a dual-scale graph transformer [16]. The specific configuration of this transformer includes setting the number of layers for the language encoder, panorama encoder, coarse-scale cross-modal encoder, and fine-scale cross-modal encoder to 9, 2, 4, and 4, respectively. The parameters for this segment of our model are initialized using the pretrained LXMERT model [49].

In the process of computing the relationship matrix $\mathcal{E}$, we assigned specific values to the parameters $k_{1}$, $k_{2}$, and $k_{3}$, setting them at 2, 2, and $5e^{-4}$, respectively. Furthermore, in order to compute the object relationship feature $\mathcal{Q}$, the weight parameters $\alpha_{1}$, $\alpha_{2}$, and $\alpha_{3}$ were meticulously initialized to 0.8, 0.1, and 0.1, respectively. Due to lacking spatial object relations module, we set $\alpha_{1}$, $\alpha_{2}$, and $\alpha_{3}$ as 0.8, 0.2, and 0 when using R2R dataset. Analogously, for the precise calculation of the loss function $L$, we established the values of the weight parameters $\lambda_{1}$, $\lambda_{2}$, and $\lambda_{3}$ at 1, 1, and 0.2, respectively.

For pretraining, we set the batch size as 32 using 1 NVIDIA RTX3090 GPU. For the REVERIE dataset, we combine the original dataset with augmented data synthesized by DUET [16] to pretrain our model with 100k iterations. Then we fine-tune the pretrained model with the batch size of 16 for 20k iterations on 1 NVIDIA RTX3090 GPU. For the SOON dataset, we only use the original data with automatically cleaned object bounding boxes, sharing the same settings in DUET [16]. We pretrain the model with 40k iterations. Then we fine-tune the pretrained model with the batch size of 4 for 40k iterations on 1 NVIDIA RTX3090 GPU. For the R2R dataset, additional augmented R2R data in [39] is used in pretraining. We pretrain the model for 200k iterations with batch size of 64 and then fine-tune it for 20k iterations with batch size of 8.

To explore the effects of our proposed modules, TOR and SOR, on the agent’s navigation skills, we integrated them separately into the baseline method. We then conducted experiments with these integrations on the val unseen split of the REVERIE and SOON datasets. As illustrated in Table IV, both TOR and SOR notably enhance the navigation performance of the agent. However, we observed that SOR contributes to a more modest improvement in navigation performance compared to TOR. This is attributed to the limited scale of the current datasets used for training the agent. Solely relying on the objects observed at each position within the environment is inadequate to accurately capture the relationships among different objects. When both modules are employed in tandem, the navigational prowess of the agent is further amplified.

In Table IV, we observes an intriguing phenomenon. The inclusion of object relationships does indeed significantly enhance the agent’s success rates in navigation (OSR, SR, and RGS). However, this enhancement also leads to an increase in the trajectory length (TL) of the agent. As a result, there is no pronounced improvement in metrics such as SPL and RGSPL. This suggests that the object relationship module leads the agent to engage in excessive redundant exploration, resulting in elongated navigation paths. Upon integrating the TBP loss function, we observes a significant reduction in the agent’s revisits to the same location. This is illustrated in Table V. This change leads to a more efficient task completion and shows clear improvements in metrics such as TL, SPL.

Additionally, Fig. 4 depicts the distribution of navigational path lengths corresponding to successful navigation instances (specifically when SR is 1) for both the val seen and val unseen splits of the REVERIE dataset, comparing scenarios with and without the integration of the TBP loss function. The figure reveals that with the TBP loss, there’s a notable increase in the proportion of agent paths falling between 0 and 10, while paths longer than 20 significantly diminish. This solidly validates that our TBP loss function effectively curtails the navigation path length of the agent.

It is also a intuitive way punish turning back during inference. To assess whether penalizing the agent’s repetitive visiting behavior during inference improves its navigational abilities, we have designed several experiments. The experimental outcomes are presented in Table VI.

Our findings indicate that penalizing the agent’s repetitive visiting behavior during inference does not enhance its navigational performance. As evidenced in the last five rows of Table VI, the navigation success rate of the agent decreases with increasing penalty intensity. This decline in performance is attributed to the fact that such penalties during inference prevent the agent from correcting its navigational errors. Additionally, the first two rows of Table VI reveal that when repetitive visits by the agent are encouraged, there is a more significant drop in navigational ability. This is due to the agent engaging in more unproductive exploration, substantially increasing the length of the navigational path.

To demonstrate that our method can accurately learn the various relationships between objects, we visualized attention heatmaps of different object interactions after training. Fig. 5(a) showcases the object relationships learned by both the TOR and SOR modules. From this, we observes that both modules are capable of accurately modeling object relationships. The spatial object relations provide a detailed and comprehensive depiction of inherent connections between objects, such as chairs and rooms, or decor and thermostats being closely associated. The spatial object relations supplement the temporal object relations. Certain information not captured by the TOR module is correctly learned in SOR. For instance, the environmental object relationships suggest a connection between decor and bathroom, which aligns with real-world scenarios but is not precisely represented in the trajectory-object relationships.

To elucidate the effectiveness of our proposed approach, we have rendered a comparative visualization of the navigation trajectories generated by both DUET and our method. As illustrated in Fig. 5(b), both techniques can accurately reach navigation targets. However, DUET tends to involve the agent in excessive exploratory actions. This often results in repeated visits to the same places, hindering navigational efficiency. Conversely, our method substantially diminishes the agent’s inclination to backtrack, facilitating the selection of more direct routes that expedite the completion of the navigation tasks. Furthermore, our analysis reveals that at the onset of navigation, our method opted for a proximal route, in contrast to DUET which embarked on a relatively longer path with a greater number of actions. This indicates that our method can effectively understand the relationships between objects encountered by the agent and the specified targets. Consequently, it identifies more efficient paths. This aids in completing navigational tasks more effectively.