Robust Change Detection Based on Neural Descriptor Fields

Previous works conduct change detection based on inputs in various 2D and 3D environment representations. Given 2D image inputs, Derner et al. [7] build a visual database and compare classical 2D feature descriptors extracted from greyscale images for query and reference feature matching. For works with 3D representations, Finman et al. [1] discover new objects as changed parts of multiple depth point clouds while Herbst et al. detect changes through movement of surfaces [8]. Ambrus et al. [2] compute a meta-room static reference map based on point clouds, and discover new objects from changes via spatial clustering. Fehr et al. [3] propose a multi-layer TSDF grid structure and perform volumeric differencing for object discovery and class recognition. Langer et al. [4] combine semantic information and supporting plane information to discover objects newly introduced to the scene. Schmid et al. [5] propose a multi-TSDF panoptic mapping approach and conduct online change detection based on TSDF value comparison.

Most previous works focus on point-wise or voxel-wise differencing on pre-aligned reconstructions in an offline fashion, demanding high observation overlap and decent localization results. They therefore do not always satisfy the need for online change detection under various viewing angles, as commonly encountered during mobile robot operation.

Neural implicit representations have been proposed as a continuous, differentiable, and parameterized representation of 3D geometry  [9, 10, 11], appearance [12, 13], and tactile properties [14] of both objects and scenes. Neural implicit representations represent shapes as continuous functions, enabling the principled incorporation of symmetries, such as $\text{SE}(3)$ equivariance [15, 6], as well as the construction of latent spaces that encode class information as well as 3D correspondences [16]. Owing to their continuous parameterized nature, several works have applied [6, 17] them to robotics tasks as intermediate object representations directly inferable from raw perception. Simenov et al. [6] show that symmetries incorporated from such a representation can generalize demonstrations of objects to novel poses, while [17, 18, 19] demonstrate their ability to be integrated with online robotic mapping tasks.

In this paper, we explore how the $\text{SE}(3)$-equivariance of NDFs [6] enables them to represent objects from partial observations and thus allows for robust online change detection with disparate viewing angles and localization noises.

NDFs consist of an encoder function $f_{\theta}({\bf{P})=z}$, which maps a partial object point cloud $\bf P$ into a global latent code $\bf z$, and a decoder function, $\Phi(\bf{x},f_{\theta}(\bf{P}))$, which maps an input point $\bf{x}$ to its predicted occupancy value:

		$\displaystyle f_{\theta}({\bf{P)=z}}:\mathbb{R}^{n\times 3}\rightarrow\mathbb{R}^{k\times 3}$		(1)
		$\displaystyle\Phi({\bf{x},f_{\theta}({\bf{P}}))}=\Phi{\bf(\bf{x},{\bf{z}}}):\mathbb{R}^{3}\times\mathbb{R}^{k\times 3}\rightarrow[0,1].$

The encoder function $f_{\theta}(\cdot)$ is constructed such that by zero-centering the input $\bf P$, the inferred global latent code $\bf z$ is represented as a vector of points that is equivariant with respect to $\text{SO}(3)$ rotations of the input point cloud $\bf P$. This means that if a point cloud is rotated by $R$, the inferred latent will be equivalently rotated by $R$, as ensured via the Vector Neuron encoder layers [15]. By subtracting from $\bf x$ the point cloud center of $\bf P$ as the translation, we then make $f_{\theta}(\cdot)$ an $\text{SE}(3)$-equivariant shape occupancy predictor for different points $\bf x$ on and off $\bf P$.

NDFs are trained with partial object point clouds recovered from semantic-segmented RGB-D images and corresponding 3D occupancy voxelgrids of objects’ complete geometry. During training, the full model $[f_{\theta},\Phi]$ is trained to predict the complete 3D occupancy of an object using the standard cross-entropy classification loss $L_{occ}$:

$$L_{occ}=\mathcal{L}(\Phi({\bf{p}},f_{\theta}({\bf{P}}),v),$$

(2)

where $\bf{p}$ is a point sampled from the object occupancy grid and $v$ is the ground truth occupancy value at point $\bf{p}$.

By feeding $\Phi(\cdot,\cdot)$ with a query point cloud $\mathcal{X}$, which is obtained via uniform sampling within a large bounding box centered around $\bf P$, the full shape point cloud $\mathcal{S}$ of the object can be reconstructed in terms of the predicted occupancy values:

$$\mathcal{S}=\{{\bf x}|\Phi({\bf x},f_{\theta}({\bf x}|{\bf{P}}))>v_{0},{\bf x}\in\mathcal{X}\},$$

(3)

where $v_{0}$ is the occupancy threshold to mark a point location as occupied.

Thanks to the $\text{SO}(3)$-equivariance of ${\bf z}$, a shape code invariant of view direction, ${\bf{s}}\in\mathbb{R}^{k}$, can then be extracted from its rotation invariant portion:

$${}s_{i}=||{\bf{z}}_{i}||_{2},i=1,2,...,k.$$

(4)

Ideally, the shape code, serving as a compact representation of the full object shape, should be consistently close among partial observations of the same shape from various viewing perspectives, while discriminatively far apart across observations of different shapes. While NDFs enable identical shape codes given identical point clouds that are rotated, shape consistency across partial inputs of the same shape is not inherently guaranteed.

To enforce $\bf{s}$ to be a discriminative yet consistent representation for shapes seen from various viewing angles, as commonly seen during mobile robot operation, we further formulate a shape similarity loss, $L_{shape}$, fashioned after the idea of the triplet loss with [anchor, positives, negatives] for person re-identification tasks [20]. Taking a partial point cloud from object shape $i$ as the anchor ${\bf{A}}_{i}$, we assign it with a positive sample ${\bf{P}}_{i}$ as the point cloud obtained from another perspective of the same object, and any other partial observations of a different object as the negative sample ${\bf{N}}_{i}$. With the distance metric $D(\cdot,\cdot)$ chosen as the cosine similarity, $L_{shape}({\bf{A}}_{i},{\bf{P}}_{i},{\bf{N}}_{i})$ then tends to pull the shape codes of ${\bf{A}}_{i}$ and ${\bf{P}}_{i}$ closer, while pushing those of ${\bf{A}}_{i}$ and ${\bf{N}}_{i}$ further apart:

$$L_{shape}=-D({\bf{s}}_{{\bf{A}}_{i}},{\bf{s}}_{{\bf{P}}_{i}})+D({\bf{s}}_{{\bf{A}}_{i}},{\bf{s}}_{{\bf{N}}_{i}}).$$

(5)

Moreover, to ensure that $L_{shape}$ always finds the more informative $(\bf{A},\bf{P},\bf{N})$ triplet, we adopt the batch hard way for triplet forming [20], i.e., using the most dissimilar $(\bf{A},\bf{P})$ and the most similar $(\bf{A},\bf{N})$ within each batch to guide training.

We hence populate each training batch $B$ with randomly generated observations of $({o_{A}},o_{P})$ pairs of different (but likely repetitive) object shapes, which ensures that at least two observations exist for each object within the batch. Every sample within the batch is then treated as an anchor, and paired batch-wise for the most dissimilar positive and the most similar negative samples. Hence the final batch-hard shape similarity loss is formulated as:

	$\displaystyle L_{b\_shape}=\frac{1}{|B|}\sum_{i=1}^{N}\sum_{j=1}^{N_{i}}($	$\displaystyle-\min_{k\in[1,N_{i}]}D(o_{ij},o_{ik})$		(6)
		$\displaystyle+\max_{m\neq i}D(o_{ij},o_{mn})),$

where $N$ is the number of objects whose observations are included in the current batch, $N_{i}$ the number of samples of object $i$ and $o_{ij}$ the $j$th observation of object $i$ within batch.

The final training objective is therefore formulated as the weighted combination of the cross entropy loss and the shape similarity loss as:

$$L=L_{occ}+\alpha L_{b\_shape},$$

(7)

where $\alpha$ is the weighting coefficient set as $\alpha=$ 0.01.

To overcome the data availability issue, we train our category-level shape-consistent NDFs fully in simulation using depth images rendered with Pybullet [21]. To consider the commonly encountered variation in viewing angles, observation distances, and occlusions during operation, for each training sample, we place a randomly-posed object on the table, along with 2-4 randomly drawn objects around it to create occlusion. The camera locations are sampled in a hollow rectangular region with the table at the center, thus accounting for both nearby and longer distance observations seen in real-world operations (see Fig. 2).

We hence base our object representation on the shape code in (4) and the object center (translation) $\bf{t}$ recovered from NDFs as $(\bf{s},t)$, where $\bf{t}$ is simply the predicted center of the reconstructed full point cloud $\mathcal{S}$ in (3):

$${\bf{t}}=\frac{1}{|\mathcal{S}|}\sum_{{\bf{x}}\in\mathcal{S}}{\bf{x}}.$$

(8)

By converting $\bf{t}$ into the world frame with the given camera pose, the object is characterized by $\bf{s}$ for its full shape and $\bf{t}$ for its global location. With the shape completion ability of NDFs, the current object representation is expected to provide a compact as well as robust way to distinguish object instances, even under varying viewing angles during robot motion.

To enable the online execution of our approach, we construct and maintain a spatial object tree in the world coordinate to organize the incoming data stream.

Built on top of the coordinate interval tree proposed in [22], our object tree comprises two translation interval trees, $T=(T_{x},T_{y})$, in the $x$ and $y$ direction, respectively, which are initialized and updated simultaneously in the same fashion each time a new object measurement $(\bf s,t)$ arrives. Considering that adjacent objects are located mostly on the same plane, for our case, it suffices to maintain trees only in the $x$ and $y$ direction for neighborhood query (while an extension to the $z$ direction should be straightforward).

Tree Construction. Consider $T_{x}$ for instance. $T_{x}$ divides the $x-y$ plane into several equi-distant interval slices in the $x$ direction (Fig. 1(b)), where each instantiated interval $[x_{min},x_{max}]$ of fixed length $l=x_{max}-x_{min}$ is represented by a node $n$. Each node stores a set of object instances whose translation component ${\bf t}$ follows ${t}_{x}\in[x_{min},x_{max}]$. The $x_{min}$ and $x_{max}$ are determined by the first ${t}_{x}$ that initializes this new node as $[{t}_{x}-l/2,{t}_{x}+l/2]$. The new node is then inserted in the tree such that the upper interval limit of a left child $n_{l}$ is always smaller than the lower interval limit of its parent node $n_{p}$, i.e., $x_{l,max}<x_{p,min}$ and similarly for the right child $n_{r}$, we have $x_{r,min}>x_{p,max}$.

Intra-tree Update. We associate the incoming object measurement $\bf m=(\bf s,t)$ with existing object instances in the tree through spatial proximity and the shape cosine similarity used in (5). We first traverse through $T_{x}$ and $T_{y}$ to locate the corresponding coordinate interval nodes $n_{x}$ and $n_{y}$ that ${t}_{x}$ and ${t}_{y}$ land in, respectively. By finding the intersection of the objects within $n_{x}$ and $n_{y}$, we obtain the neighborhood $\mathcal{N}$ that $\bf m$ should be adjacent to as $\mathcal{N}=\{o|o\in n_{x}\wedge o\in n_{y}\}$. The measurement is then successfully associated with an object instance ${o}_{0}=({\bf s}_{0},{\bf t}_{0})$ in the neighborhood when:

$\displaystyle||{\bf t}_{0}-{\bf t}||<\delta_{d}\wedge D({\bf s}_{0},{\bf s})>\delta_{s},$

(9)

where $\delta_{d}$ and $\delta_{s}$ are the maximum distance threshold and minimum shape similarity threshold for valid association. We then replace the old shape code and the recovered object center of the object with $\bf m$ as an update if the average occupancy value of the reconstructed full shape point cloud for $\bf m$ (calculated in (3)) is higher than that of $o_{0}$.

Given two sequences as the source and the target, we construct the source object tree $T_{s}$ in advance and perform object change detection during construction of the target tree $T_{t}$.

Sequence Registration. We wish to apply our approach in an online mobile operating scenario where post-alignment tools or motion capture systems are not always readily available. Therefore, we first conduct a rough alignment between the target and the source sequence at the beginning of the target traverse, in the hope that target objects can correctly locate the corresponding neighborhood patch in the source so as to enable valid neighboring object comparison.

With all the shape codes of the source objects easily obtainable from $T_{s}$, after intra-tree update for $T_{t}$, we determine each corresponding source object ${o}^{\prime}$ for the current target object ${o}$ as the one sharing the highest shape code similarity among all the objects (if exist) whose shape similarity with $o$ is above $\delta_{s}$ (similar to (9) amid intra-tree measurement-object association).

After $N$ pairs of object correspondences ${(o^{\prime},o)}$ have been accumulated, we apply Single Value Decomposition (SVD) with RANSAC onto the $N$ object center pairs $({\bf t^{\prime}}_{i},{\bf t}_{i})$ and obtain the relative transform $T_{rel}$ between the two sequences. This considers the potential inclusion of changed object instances, while assuming that they only take up a small portion of the environment to allow for alignment when the target sequence starts. Here, we set $N=6$ to account for transform accuracy versus timing to start change detection.

Change Detection. Rather than examining object-wise correspondences through spatial proximity and shape similarity (as in (9)), we draw support from neighboring objects and detect changes through the relative spatial layout consistency of the object within its target neighborhood and that of its corresponding neighborhood (if exists) in the source.

When an object measurement $\bf m=(s,t)$ arrives, we update the $T_{t}$ by finding/instantiating the associated object instance $O$. Considering the online operation setting, change detection is then performed on $O=({\bf s}_{0},{\bf t}_{0})$ that has not been marked as changed before.

Given $O=({\bf s}_{0},{\bf t}_{0})$ along with its neighborhood $\mathcal{N}_{0}$ in the target, we find the projected location in $T_{s}$ with $T_{rel}$ and query the corresponding source neighborhood $\mathcal{N}^{\prime}=\{o^{\prime}_{i}=({\bf s}_{i},{\bf t}_{i})\}$. Potential object matches are found based on shape code similarity between ${\bf s}_{0}$ and each ${\bf s}_{i}$ in $\mathcal{N}^{\prime}$.

The most straightforward change cases are that either $\mathcal{N}^{\prime}$ is empty or no matched $o_{i}$ is found in the source neighborhood, as both indicate newly added objects to new locations or else new shapes.

Then with object matches found within $\mathcal{N}^{\prime}$, we conduct local neighborhood comparison for each matched pair $(O,o^{\prime}_{i})$ (see Fig. 3 for a concrete example). This also serves as a validation step for rejecting false positives in dealing with noise-corrupted localization. When object position changes at a scale similar to cross-session localization errors, it can be hard to determine if the absolute object center difference between $(O,o^{\prime}_{i})$ is brought by actual changes (e.g., object moves within the plane) or merely localization/projection drift, while the latter can be validated through the relatively more stable inter-object spatial layout.

For each of the matched pairs, neighborhood comparison is accomplished via object graph matching.

A local object graph $G_{o}=(V,E)$ for object $o$ in neighborhood $\mathcal{N}_{0}$ is constructed as a directed graph with vertices as all the objects in $\mathcal{N}_{0}$, $V=\{o_{i}|o_{i}\in\mathcal{N}_{0}\}$, and directed edges as the object center difference pointing from $o$ to its neighbors, $E=\{{\bf e}_{oi}|{\bf e}_{oi}={\bf t}_{i}-{\bf t}_{o},\forall i\in\mathcal{N}_{0},i\neq o\}$. Considering the limited number of objects within a neighborhood (size set similar to a tabletop), the relatively small graph size makes it possible for edge-wise matching. We find corresponding edges ${\bf e}_{oj}$ and ${\bf e}_{o^{\prime}j^{\prime}}$ through vertex shape matching. Similar edges, indicating unchanged inter-object layout, is determined by the Euclidean distance between them:

$\displaystyle\sum\limits_{j}\mathds{1}(||{\bf e}_{oj}-{\bf e}_{o^{\prime}j^{\prime}}||_{2}\leq\delta_{e})=$

(10)

If at least one pair of edges is found to be closer than $\delta_{e}$, i.e., at least one out of the few neighboring objects remains the same, the local object layout is marked as consistent. This implies an unchanged object, and vice versa. Note that in the rarer case when either $O$ or $o^{\prime}_{i}$ is the only object in the neighborhood, we revert back to the object-wise comparison in terms of spatial proximity and shape similarity.

Lastly, for detecting objects removed from the source, during target streaming we label each source object as observed if it ever participates in shape matching with any target objects, then removed objects can be found after the target sequence finishes as those observed but never matched with a target object.

To better examine the effectiveness of our category-level object representation and the change detection approach built on top of it, we choose to have our testing sequences composed of objects from the same category. We hence design two pairs of testing sequences, in simulation and in real world, respectively, featuring coffee mugs of diverse shapes observed from distinct viewing angles. Considering the extensive multi-category results reported in relevant works [6, 23], we argue that the effectiveness of our approach should be extendable to the multi-category case by incorporating more object categories into the training data.

Synthetic Sequences. We set up an environment in Pybullet with 35-40 instances of 20 ShapeNet [24] mugs models scattered around eight tables and obtain RGB-D images along with segmentation masks from two preset camera trajectories. The mug models are unseen by NDFs during training. Each table supports 3-5 mugs with mugs newly added, removed, or switched locations, creating in total 12 changes between the two sessions. The two camera trajectories are designed such that the camera always faces towards the nearest table, prompting less observation overlap between sequences around table 1,2, and 5 (see Fig. 4).

Real-world Sequences. We collect two sequences featuring 14 mugs of diverse shapes randomly placed on three rectangular tables. We mount an Intel RealSense L515 camera onto a Jackal robot close to mugs’ height and collect RGB-D data with the trajectories shown in Fig. 5. The camera trajectories are recovered using ORB-SLAM3 [25]. Since the camera is mounted facing sideways, while the camera circles around table 1, 2, 3 in the source, it only weaves through table 1 and 3 in the target, thereby creating observation disparity on mugs along the inner side of table 1 and 3.

We adopt the commonly used precision and recall rate at the object level, i.e., the number of objects, for evaluation. We report the number of correctly detected changes (true positives, TP), falsely detected changes (false positives, FP), and undetected changes (false negatives, FN). Precision and recall rate are computed as $\frac{TP}{TP+FP}$ and $\frac{TP}{TP+FN}$.

To train NDFs using an occupancy network, we set the camera-table distances as $d_{c}=$ 0.2 m and $d_{f}=$ 5 m and generate 50,000 RGB-D partial observations with 94 ShapeNet [24] mug models following the sample generation strategy in III-C. Partial object point clouds are obtained by extracting corresponding depth points in the camera frame indicated by the segmentation masks on RGB images. We build our model upon the network implementation provided by [6] and train it on two NVIDIA RTX 3090 GPUs using a learning rate of $6\times 10^{-4}$ with the Adam optimizer. The length of the interval tree $l$ is set to be 1.2 m and 1.6 m for the synthetic and real-world sequence, respectively, intending to group most objects on the same table plane. For parameters in IV, we set $\delta_{s}=$ 0.9, $\delta_{d}=$0.02 m, and $\delta_{e}=$ 0.03 m.

To demonstrate the robust generalization of the shape-consistent NDFs to unseen object instances under various viewing angles, in Fig. 6, we render the reconstructed 3D shapes of five mugs drawn from the synthetic and real-world testing sequences. The three views are selected in the hope of representing some of the most typical perspectives when observing a mug, e.g., handles visible in different directions and handle obscured. We can conclude that despite the variety of the selected mug shapes, the shape-consistent NDFs still demonstrate satisfactory performance in encoding and completing the full shape under varying viewing angles, justifying the adoption of NDF-derived object representation in our change detection task.

This can be attributed to the fact that by learning purely from geometric structures embedded in the partial point clouds, NDFs are able to transfer this structural knowledge to unseen instances regardless of object color and texture. We also include cases with occlusion and shape ambiguity. For occluded observations, as shown in the first two columns of “View 2” and “View 3” , we can still see decent reconstruction results by virtue of the occlusion scenes incorporated in the training data. For the case of shape ambiguity, i.e., the handle is obscured in the second and last column of “View 2”, the major body parts of the mugs are still reconstructed reasonably well. While the handle location was incorrectly predicted, such mistakes only result in negligible errors to object center recovery.

We further present the change detection results on the two sets of testing sequences based on our proposed approach and compare them to two baselines.

The first baseline is the typical nearest neighboring point search (NN) commonly used for offline scene differencing [4, 2]. Given two spatially aligned observations $S$ and $T$, the change point set $C$ of $T$ w.r.t $S$ is found as $C=\{p|p\in T,\forall s\in S,||p-s||_{2}>d\}$. We adapt it to the object-level and define that an object is changed if a certain proportion $r$ of its target point cloud finds no neighbors in the source. To make the results interpretable to online object-level change detection evaluation, we mark an object as changed the first time during data streaming it is detected as changed, as during real-world operation, corresponding actions will be taken right after changes are detected and usually no correction can be made for false positives.

The second baseline is the recently proposed panoptic multi-TSDFs (PMT) mapping method by Schimd et al. [5], which represents panoptic entities as TSDF submaps and captures online object-level scene changes based on voxel-wise weight counting informed by TSDF value differences. After tuning the default parameters for better performance, we count the total number of changed objects based on the number of conflicting submaps determined by the baseline.

Since both baselines require pre-aligned sequences, we feed the baselines with ground truth camera poses from Pybullet and those aligned by ORB-SLAM3, respectively. For NN, the reference partial point cloud of each source object is fused from the whole sequence of the source depth images. Data associations between the target partial mug observations and the source point clouds are determined by the instance ID in Pybullet for synthetic sequences and manual labeling based on the panoptic masks produced by Detectron2 [26] (used in PMT) for real-world sequences.

All results are obtained on a laptop with an Intel Core i7-9750H CPU and an Nvidia GeForce RTX 2070 GPU. The network inference speed for NDF is around 0.019 s per frame, and the change detection operation with an expanding spatial tree executes at an average speed of 0.023 s per frame given the magnitude of the object number in our experiment setting (up to 40 mugs). These result in an overall speed of 24 FPS for the complete pipeline, thus showing no obvious latency for online operation.

Results on Synthetic Sequences. We present the change detection results of the three methods in Table. I. Here for NN, we set $d=$ 0.002 m and $r=$ 0.3. We apply a 6 DoF random transform to the target camera poses when running our approach so as to simulate the unavailability of motion capture systems during common mobile robot operations.

We can see that our method accurately detects all the changes without any false positives. On the one hand, NN and PMT respectively ignore four and one changes for the two pairs of mugs switching positions at table 5 and 7, as the similar viewing angles of these mugs during the source and target traverse induce high overlap between the target and source point clouds, thereby confounding the baselines by the existence of neighboring points and similar local geometry.

On the other, both baselines falsely mark the four mugs on table 1 and 2 as changed, which is due to the remarkable viewing angle difference on these four mugs between the source and target trajectory. As explained in Fig. 7, the two mugs highlighted in red and green are observed in almost opposite directions, which then leads to little/no overlap between the two partial point clouds (blue and yellow). Therefore, few neighboring points can be found between the source and target observations, while at the same time leads to drastic differences in the resulted TSDF values. In contrast, our method is capable of encoding the full mug shape with a compact shape code from the partial observations and hence produces no false positives.

Results on Real-world Sequences. We report change detection results of the three methods on the real sequence in Table. II and present the reconstructions obtained from PMT and our method (for mugs) in Fig. 8. For fair comparison, we obtain the mug point clouds for all the three methods using the panoptic mask adopted in PMT, which are generated by Detectron2 [26] and preprocessed by DBSCAN [27]. Here, the parameters of NN are set as $d=$ 0.002 m and $r=$ 0.4 considering the noises in the real-world point clouds. Due to localization drift between two sessions, for PMT, the final number of changed objects are counted after manual merging of the few submaps instantiated for the same object. In order to mimic the real operating scenarios, the camera poses to our approach are obtained through running ORB-SLAM3 separately on the two traverses.

When confronted with noisy point cloud inputs and localization results, despite the reconstruction failure of the beige mug on table 1 and the blue mug on table 2 (as shown Fig. 8(a)) due to poor point cloud and panoptic mask quality, our method still maintains better detection efficacy in terms of both precision and recall rate. From Fig. 8(b), we see that the confusion brought by disparate viewing angles persists, as both NN and PMT wrongly mark the black and blue mug on table 3 (highlighted in green) as changed. This matches the PMT reconstruction results shown in the second column, as the two mugs have just one side partially reconstructed during each session. Furthermore, another false detection emerges as the dark blue mug on table 3 (highlighted in yellow in Fig. 8(a)), which results from the larger localization error at the starting point of the target trajectory. Our method reduces its reliance on the absolute accuracy of camera pose estimation by conducting local neighborhood layout matching, i.e., the relative spatial relationship among neighboring mugs, thereby successfully recognizing the mug through its unchanged relative positions with its neighbors.

Limitation and Extension. The proposed approach works well under the assumption that the changed objects do not take up a predominant portion of all the objects in the scene, which is a common case for doing frequent visits of the same environment whose changes occur incrementally. For scenes with drastic changes between the two scans, our approach may not work well with the absence of a consistent local layout to refer to. Our approach can be further extended to detect objects that only rotate but do not move by utilizing the $\text{SO}(3)$ equivariance of NDFs, i.e., computing an extra $\text{SO}(3)$ transform between the two $\bf{z}$’s after the shape-similarity-based registration step. This situation was not included in the experiment as it is rare for changed daily objects to have pure rotation but no translation.