TableDC: Deep Clustering for Tabular Data

The basic DC architecture consists of two phases: (i) representation learning and (ii) clustering. The joint optimization of both phases helps each to improve the algorithm’s performance iteratively. A precursor to (i) and (ii) is (iii) cluster initialization, which contributes to the algorithm’s performance by providing the initial data shape. With better cluster initialization, a DC algorithm is less likely to get stuck in local minima, and requires fewer converging iterations.

Regarding (i), the most widely used architecture is the basic Auto-encoder (AE) (Yang et al., 2017), where the encoder function $f_{e}$ encodes an input representation $x_{i}$ into a latent representation given by $h_{i}=f_{e}(x_{i})=\frac{1}{1+e^{-(Wx_{i}+b_{i})}}$. Similarly, the decoder function $f_{d}$ decodes $h_{i}$ back into the reconstructed input $x^{\prime}_{i}=f_{d}(h_{i})$. Here, $W$ and $b_{i}$ represent the weights and biases of the neural network, respectively. The optimization function of AE with $N$ samples is defined as $f_{min}=\min\frac{1}{N}\sum_{i=1}^{N}\|x_{i}-x^{\prime}_{i}\|^{2}$. Many recent DC proposals use generative models in representation learning, especially Variational Auto Encoders (VAEs), which combine an AE architecture with probabilistic graphical models. In VAEs, the encoder maps the input data into a probabilistic distribution over the latent space while the decoder reconstructs the data from the latent probabilistic representation. Some recent proposals with VAEs in representation learning are DGG (Yang et al., 2019), LTVAE (Li et al., 2019), VaDE (Jiang et al., 2017) and MFCVAE (Falck et al., 2021).

To benefit from the local and global structure of data in representation learning, several recent DC proposals combine basic AE learning with a Graph Convolutional Network (GCN) to assist the model in learning the structural information. Some recent examples are DCRN (Liu et al., 2022), DFCN (Tu et al., 2021) and SDCN (Bo et al., 2020). Subspace representation learning is another recent technique that identifies subspaces that optimize cluster separation, unlike AEs and VAEs, which require additional clustering mechanisms. Subspace learning seeks to identify and represent underlying subspaces within high-dimensional data, while AEs and VAEs focus on reconstructing the input data as accurately as possible. The latest DC methods based on Subspace representation learning are EDESC (Cai et al., 2022), DMCAG (Cui et al., 2023), SAGSC (Fettal et al., 2023) and OMSC (Chen et al., 2022b).

DC is inherently more complex for the tabular data embeddings obtained by Large Language Models (LLMs) than widely used sparse image or textual datasets like CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011), ImageNet (Russakovsky et al., 2015), USPS (LeCun et al., 1990) and Reuters (Lewis et al., 2004), primarily due to data structure and feature representation differences. Image datasets include spatial and visual coherence that DC methods can better differentiate, even when the resolution is low or the number of classes is high. DC has been used widely on complex and varied datasets unlike ImageNet, where, despite the vast diversity, the visual patterns (such as extracted from different textures, shapes, objects, and backgrounds) retain a level of consistency

On the other hand, the primary issue with data management datasets is the high density and significant overlap in the feature space of their embeddings; specifically, tabular data embeddings represent a complex relationship among different features, including categorical and numerical data.

In relation to (ii), clustering, the aim of existing work on DC is to use the low dimensional representation from the representation learning module and obtain clustering assignment probabilities for soft clustering. In the related work, several clustering mechanisms are employed to jointly work with representation learning for optimal clustering. The most widely used one is self-supervised clustering (Dizaji et al., 2017; Guo et al., 2017; Peng et al., 2017; Xie et al., 2016; Bo et al., 2020; Tu et al., 2021), which aims to group data points based on learned representations without explicit labels; the approach uses two distributions: the cluster assignment distribution $Q$ representing the probability that a data instance belongs to a particular cluster, and an auxiliary distribution $P$ which is a modified version of $Q$ that emphasizes more confident cluster assignments and minimizing the Kullback-Leibler (KL) divergence between $Q$ and $P$. $Q$ is calculated using the distance between the data point $x_{i}$ and each cluster centroid $\mu_{k}$. The closer $x_{i}$ is to $\mu_{k}$, the higher the probability $Q$. Self-supervised clustering focuses on high-confidence instances due to the squaring of probabilities in $P$ and preventing cluster dominance by normalizing the probabilities, ensuring balanced clustering assignments.

Pseudo-labeling is another clustering technique recently utilized in DC (Niu et al., 2022; Zhou et al., 2022), where the model generates labels that are then used to fine-tune the model, effectively creating a feedback loop where the model iteratively refines its clustering assignment. The recent success of contrastive learning has led to its use in DC (Sun et al., 2023; Zhang et al., 2023; Liu et al., 2023; Ma and Kim, 2022), where the data points from low dimensional representations are paired in a way that the same cluster (positive pairs) is contrasted with the different clusters (negative pairs). Further improvements in contrastive deep clustering include the contrast comparison between instance-to-instance, instance-to-cluster, and cluster-to-cluster (Zhou et al., 2022).

TableDC employs a self-supervised clustering approach by comparing the encodings of tables, rows or columns within a latent space. The selection of an appropriate distance function is critical in this context, as it determines how the distances between each table, row or column and the cluster centroids in the latent space are measured. A simple distance metric like Euclidean distance is beneficial when dealing with data where features exhibit limited variance and correlation, as in image vectors or sparse data matrices (Bo et al., 2020; Tu et al., 2021). However, a more sophisticated approach is required in dense embeddings, where the data points show significant feature correlation. Here, the Mahalanobis distance is particularly advantageous (Mahalanobis, 2018). Its ability to account for feature correlation enables the model to discern even subtle differences among clusters, mainly when a cluster shows high similarity across multiple features. Mahalanobis distance is particularly relevant when comparing tables, columns or rows, as it allows for a more nuanced interpretation and grouping of data points based on their inherent relationships and similarities.

In relation to (iii), most existing works use K-means to initialize clusters, whether self-supervised or subspace clustering such as EDESC (Cai et al., 2022), DCRN (Liu et al., 2022), DFCN (Tu et al., 2021) and SDCN (Bo et al., 2020). The quality of the initial clusters significantly affects the final clusters. To our knowledge, there is no related work on different cluster initialization approaches in the DC environment. To address this, we compare different cluster initialization approaches in an ablation study in Section 5.1.

Clustering is important in data management for several data integration and cleaning tasks, including those discussed here.

Schema Inference identifies the raw data’s types, relationships and attributes to infer an overarching schema (Kellou-Menouer et al., 2022). Schema inference has been explored to infer a schema that fully describes some closely related datasets  (Baazizi et al., 2019; Bex et al., 2007) or to summarise structures that appear in more diverse datasets. A common first step in the latter case is the identification of recurring structural patterns, which can be used as types in an inferred schema. In this setting, a variety of SC algorithms have been used to group similar structures (e.g., (Bonifati et al., 2022; Kellou-Menouer and Kedad, 2015; Tsuboi and Suzuki, 2019)).

Entity Resolution is the well-explored problem of identifying two or more records that represent the same real-world object (Christophides et al., 2021). Several proposals use clustering, especially for entity matching and duplicate detection, for which empirical comparisons have been carried out (Hassanzadeh et al., 2009; Costa et al., 2010; Saeedi et al., 2018; Draisbach et al., 2020; Hassanzadeh et al., 2009). Although many entity resolution proposals focus on pairwise comparisons, some proposals incorporate clustering because the transitive closure of pairwise similarity may not always lead to robust clusters (Fisher et al., 2015; Mandilaras et al., 2021).

Domain Discovery involves identifying columns that draw values from the same collection of values. Most existing works have used bespoke algorithms (Li et al., 2017; Ota et al., 2020; Piai et al., 2023). However, we cast domain discovery as a clustering problem, where the goal is to cluster a set of columns that share semantic types. RaF-STD (Piai et al., 2023) uses clustering to discover semantic types for heterogeneous sources. Similarly, D4 (Ota et al., 2020) provides a column-based clustering approach to discovering local and strong domains from a set of columns, while handling the challenge of incomplete columns. Building on language models, Starmie (Fan et al., 2023) provides column clustering based on the learned column representation through contrastive learning.

Since $m$ acts as clustering probabilities and can be further optimized, we integrate KL-divergence to refine the clustering probabilities $m$ to be more accurate and representative of the underlying data. KL divergence provides a flexible way to guide the model toward the expected clustering assignments. We use the following objective function to optimize the $m$.

$ce_{loss}$ is clustering loss, where $p_{ij}$ is the target distribution, calculated by adjusting the soft assignments $q$. This step aims to make the soft assignments more robust during the clustering process. We use the following reconstruction loss ($re_{loss}$) to maintain the inherent structure and essential features in a compressed form.

where $\bar{x}$ is the reconstructed input. $ce_{loss}$ and $re_{loss}$ work together to achieve effective clustering while maintaining the quality of the data reconstruction. The total loss is given by:

where $\alpha$ is a weighting factor that balances the contribution of the $ce_{loss}$ and the reconstruction loss ($re_{loss}$) to the total loss. Combining $ce_{loss}$ and $re_{loss}$ enables autoencoder learning to generate meaningful latent representations that can be effectively clustered while preserving the structure and information in the input data. We show the pseudo-code for training of TableDC in Algorithm 1.

The reconstruction loss, $re_{loss}$ (Equation 12), of TableDC is consistently reducing over epochs during the training (see ablation study in Section 5.2), which shows that TableDC is learning a meaningful representation of the data, capturing essential features and structures. In $ce_{loss}$ (Equation 10), m is a softmax-ed q consistently diverging from the target distribution p (see ablation study in Section 5.2). After applying softmax, q becomes a probability distribution m over clusters. Each row of the probability matrix has a sum of 1, representing the probability for each point to belong to each cluster. The softmax function emphasizes the significant values in a probability matrix m while suppressing smaller values, making TableDC soft assignments sharper. The choice of the hyperparameter $\alpha$ allows for fine-tuning the trade-off between clustering performance and reconstruction quality. We use $\alpha$ = 0.9 (Equation 13), so TableDC gives more importance to the clustering loss, $ce_{loss}$ (Equation 10) than $re_{loss}$. We determined the weight of $\alpha=0.9$ for the $ce_{loss}$ through empirical evaluation.

The presence of noise or exact probabilities towards more than one cluster can help provoke the model towards more confident cluster assignments despite having inherent ambiguities. p is artificially derived from q by re-normalizing and making high-probability assignments higher and diminishing low-probability assignments. However, the high divergence of m from the sharped p shows that TableDC is more confident with the actual soft assignments q becoming sharper over time. Moreover, the data points in target p need more intra-cluster distance instead of high cohesion. A consistent increase in divergence of m from q indicates that TableDC’s internal clustering, as represented by m, is moving in a direction that better aligns with the true data structure. p is an auxiliary target to guide the clustering process and does not represent the ground truth; in the presence of an overlapping cluster, the divergence from p is not inherently poor. Considering this phenomenon, we can consider the KL-divergence as a maximization problem where we prefer m to have a maximum divergence from p and can be defined as a sequence of KL-divergences over $N$ iterations: $\{D_{KL}(p\parallel m^{1}),D_{KL}(p\parallel m^{2}),\ldots,D_{KL}(p\parallel m^{N})\}$ where $D_{KL}(p\parallel m^{k})$ represents the KL-divergence between $p$ and $m$ at the $k^{th}$ iteration. The KL divergence is increases over $N$ if: $D_{KL}(p\parallel m^{k+1})>D_{KL}(p\parallel m^{k})$ for all $k$ in $\{1,2,\ldots,$N-1$\}$.

In the training phase of DC algorithms, the quality of the initial centroids can directly affect the quality of the final clusters. Poor initialization can lead to poorly formed clusters, such as unevenly sized, too small, or too large. Existing DC methods (Bo et al., 2020; Liu et al., 2022; Tu et al., 2021) use K-means to initialize cluster centers c. However, in data integration problems dealing with highly dense and overlapping embedding representation, K-means may not be a suitable initialization (see ablation study in Section 5) for the following reasons: (i) In densely packed and overlapping data dimensions, the Euclidean distance between c and z becomes uniform, which leads to poor cluster initialization. (ii) In a dense data space, many sub-optimal solutions exist, and K-means is not optimal enough to avoid local minima. (iii) K-means initialization is based on hard assignment, where the centers are randomly initialized. However, dense data with overlapping clusters requires soft assignments based on probabilities. K-means initialization is often considered suitable in existing DC proposals due to the nature of the data, which is mainly sparse or semi-dense, particularly in image representations. The Euclidean distance between different points becomes more distinct in a sparse data space, resulting in well-separated clusters. This distinct separation enhances the effectiveness of K-means initialization in these contexts, facilitating the clustering process by providing a clear starting point for the algorithm.

In TableDC, we integrate the Birch algorithm  (Zhang et al., 1996) to initialize the clusters. Birch is a traditional hierarchical clustering algorithm that handles large and noisy databases. Birch uses a Clustering Feature tree (CF-tree) to process each data point. A CF-tree is a set of nodes, and each node consists of a triplet that efficiently stores information on data points in a compressed format to the cluster instead of processing a complete dataset. The triplet includes the number of data points per cluster, squared, and linear sum. In a dense space, each node of the CF-tree represents aggregated properties of each data point instead of the distance between data points, which enables Birch initialization to avoid close proximity and high overlaps. Since Birch is hierarchical, the multiple levels of the CF-tree can effectively capture the different granularities of clusters even if the overlaps exist on one level. We compare different cluster center initialization techniques in the ablation study (see Section 5.1). The pseudo-code for the initialization of c using Birch is given in Algorithm 2).

We adopted two widely used standard clustering evaluation metrics, Accuracy (ACC)  (Yang et al., 2010) and Adjusted Rand Score (ARI)  (Wu et al., 2019) to test the performance of TableDC and existing benchmark methods. ARI ranges from -1 to 1, where 1 indicates perfect matching, 0 indicates random labeling and negative values indicate worse than random labeling. ACC ranges from 0 to 1.

TableDC consists of four AE layers and is trained using the Adam optimizer. The loss function combines KL-divergence (to minimize the Mahalanobis distance in self-supervised learning) and mean squared error loss for reconstruction. The latent space size is fixed at 100. In order to be consistent with the existing DC benchmarks, we pre-trained TableDC with AE on 30 epochs for schema inference and domain discovery. Pretraining provides a good starting point for the network parameters, avoiding random initialization and local minima during the clustering phase, which leads to faster convergence. Since entity resolution involves many clusters, the CF-tree in Birch initialization needs more nodes with detailed cluster summaries, each representing a smaller and more defined cluster. We have pre-trained TableDC on 100 epochs for entity resolution. Training epochs for schema inference are fixed to 200, domain discovery to 100, and entity resolution to 50. We run all existing benchmark methods on the same number of epochs adopted for TableDC, excluding the internal parameters for which we use the originally published values for each existing method, such as the number and size of each layer. We tuned the threshold T (the maximum range of radius of a cluster in the CF-tree) when initiating the centroids in TableDC for those experiments where the number of subclusters found on each CF-tree node is less to be able to summarize cluster information. We adopted a grid search on threshold T and divided the number of sub-clusters unless the CF-tree becomes stable. Some existing methods need K-means for centroids initialization and clustering assignments. To avoid extreme cases, we initialize 20 times and choose the best solution.

Schema inference as a clustering problem can be defined as the clustering of tables that share a similar schema. Table 2 shows the clustering results of schema inference on Table Union Search (TUS) and web tables datasets encoded with several representations.

We observe the following: (i) TableDC obtained the best results over both datasets when considering schema-level data (SBERT, FastText, USE) compared to all benchmark methods, including SC. TableDC improves the clustering performance (with SBERT) by 0.28 and 0.27 ARI and 0.30 and 0.15 ACC, compared to SDCN and SHGP, respectively. (ii) TableDC exhibits good but not always best performance on instance-level data (TabTransformer, SBERT) and was outperformed by SDCN by a small margin (0.01 ARI and 0.03 ACC) on the TUS dataset and by EDESC on the web tables dataset using TabTransformer. However, TableDC outperformed the other methods with SBERT, improving clustering performance by 0.31 average ARI and 0.23 average ACC on the TUS dataset. (iii) TableDC handles overlapping clusters due to its heavy-tailed Cauchy distribution as a similarity kernel. For example, TableDC assigns two tables that share similar schemas, RadioStation.(Radio, Country, Language) and Country.(Country, Language, Broadcasters) with a 0.79 cosine similarity score in the representation space to two different clusters, compared to SDCN and DFCN, which failed to handle high overlap and put them in one cluster, leading to misclassification. (iv) TableDC provides effective distance measures in a dense space through the Mahalnobis distance measure when calculating the distance between data points. Two instances from different GT clusters are closely packed in the latent space when the distance measured is Euclidean compared to the Mahalbonis distance, which helps the model by placing both instances apart. For example, the Mahalanobis distance between different attributes of two tables Book.(rank, title, genre, creator, date, freq) and Film.(rank, title, year, director, overall rank), is 0.99 compared to the Euclidean distance, which is 0.71, indicating that during self-supervision in TableDC, both instances are further apart despite having highly overlapping values. Experimentally, TableDC correctly placed the two instances in different clusters whereas EDESC and DCRN misclassified them into one cluster due to a low Euclidean distance score. (v) TableDC focuses on semantic similarity more than distributional frequency. For example, when considering instances on TUS with SBERT, attributes (Day, Month) with different frequencies (i.e., the frequency of each day) present in a column should be in the same clusters, ignoring the synthetic properties or the count of a particular day repeated in a column. However, SDCN and DCRN consider the syntactic similarity and the frequency (Tuesday and Wednesday are the highest counts in both tables) and incorrectly place the two tables in different clusters, in contrast with TableDC.

Entity resolution as a clustering problem can be defined as clustering multiple records that refer to the same real-world entity. Table 3 shows the clustering results of entity resolution on GeoSet and Music Brainz datasets encoded with several representations.

We observe the following: (i) The best results are with TableDC on both datasets, outperforming all other methods by 0.63 average ARI and 0.53 average ACC on Music Brainz and 0.31 average ARI and 0.24 average ACC on GeoSet using SBERT. (ii) TableDC performed well with SBERT compared to EmbDi, delivering good semantic similarity performance among different records of the same real-world entity. TableDC is better with SBERT than EmbDi by 0.20 ARI and 0.17 ACC with Music Brainz, while 0.05 ARI and 0.15 ACC with GeoSet. (iii) TableDC more effectively adopted the semantic mapping by SBERT than SHGP and DFCN by clustering contextually similar records. For example, two records of geographical locations (name: Manchester (England), longitude: -2.23743, latitude: 53.4809) from data.nytimes.com and (name: manchester united kingdom, longitude: -2.24, latitude: 53.48) from rdf.freebase.com have a low syntactic similarity (pairwise Euclidean distance) of 0.47 and high cosine similarity (pairwise cosine distance) of 0.86. SHGP and DFCN misclassified both instances and grouped them into different clusters compared to TableDC, which forms one cluster with contextually similar records. (iv) TableDC with EmbDi on GeoSet created fewer unary clusters (50) than SHGP (101) and EDESC (90), indicating that TableDC effectively balances cluster purity and cluster size. A higher ARI score with fewer unary clusters implies that TableDC is better at avoiding unnecessary fragmentation of the rows into too many small clusters. (v) Like schema inference, in entity resolution, existing clustering methods failed to form clusters based on the context of a particular text. For example, two different songs with some similar values of attributes are clustered together by SDCN and SHGP with EmbDi, i.e. (title: Jabberwock, length: 292706, year: 2010, language: French) and (title: Bubble Star, length: 244346, year: 2004, language: French). Both records share the same language and numeric values with a similar range encoded by EmbDi despite having different titles. TableDC managed to put them in different clusters based on the context of both records despite them having several similar attribute values.

Domain discovery as a clustering problem can be defined as a clustering of the columns from a collection of datasets that refer to the same domain. Table 4 shows the clustering results of domain discovery on Camera and Monitor datasets encoded with several representations.

We observe the following: (i) TableDC obtained the best results with SBERT and T5 when considering column values of the Camera dataset outperforming other methods by an average of 0.32 and 0.18 with SBERT and 0.30 and 0.20 with T5 on ACC and ARI, respectively. (ii) TableDC also led on the Monitor dataset by an average of 0.18 and 0.11 with SBERT and an average of 0.14 and 0.09 with T5 on ARI and ACC, respectively. (iii) TableDC shows a superior capacity for learning and interpreting the column’s context in the latent space compared to other methods. TableDC’s ability to learn and utilize complex patterns shows the better integration of the embeddings obtained using T5 on the Camera dataset. For example, the cosine similarity of the T5 vectors of two columns (image sensor: CMOS) and (optical sensor: CMOS) is 0.61, and TableDC learns the context and assigns both columns to the same cluster, compared to SDCN and DFCN, which incorrectly assigns them to different clusters. (iv) The clustering performance of existing methods is affected by the length of columns compared to TableDC, which handles the columns uniformly regardless of the length. For example, two columns of the Camera dataset with different lengths (camera color: silver gray/ black/ red/ silver) and (color: black) are categorized in different clusters by TableDC and DCRN with SBERT; however, TableDC overrides the column structure and correctly puts them together in the same cluster. In the experimental evaluation, some experiments with DCRN have not managed to scale on available resources and are reported as N/A.

Previous experiments have compared TableDC with other SC and DC algorithms, using the same representations. This section complements such results by comparing TableDC with existing state-of-the-art solutions.

We observe that existing DC proposals struggled to scale to large numbers of clusters $\mathbb{K}$ (see Figure 3). To compare the scalability of different DC methods, we used Music Brainz, which was scaled to large numbers of clusters, up to $\mathbb{K}=2400$. The times are reported running on Nvidia A100 GPU with 80GB GPU RAM.

Most existing DC algorithms use GCN to create an adjacency matrix to capture underlying relationships among different features. This leads to a sparse adjacency matrix in scenarios where the representation is not dense. The complexity of GCN-based representation learning, assuming a sparse adjacency matrix, is proportional to the number of vertices and edges. We often deal with dense representations with overlapping distances in data management applications. For dense representations, the multiplication between an adjacency matrix $\mathbb{A}$ (size $\mathbb{N}\times\mathbb{N}$) and a feature matrix (size $\mathbb{N}\times\mathbb{D}$) becomes more computationally intensive as $\mathbb{K}$ and $\mathbb{D}$ increase, leading to greater complexity.

In contrast, TableDC does not rely on GCN to capture relationships but instead uses Mahalanobis distance to determine correlations between different features. TableDC employs an autoencoder with a linear relationship between the number of data points $\mathbb{N}$ and the size of the hidden layer $\mathbb{H}$. The self-supervised module, which calculates the differences between cluster centroids from dense representations, maintains a manageable computational load because the calculation is around the distance from each data point to every cluster centroid. TableDC’s efficiency is optimized using Cholesky decomposition to compute the Mahalanobis distance. This method efficiently handles dense representations by decomposing the covariance matrix, allowing for faster computation of distances (Golub, 2013). Additionally, the integration of a Cauchy distribution kernel normalizes these distances and applies softmax operations, ensuring that the computational overhead remains low even as the number of clusters $\mathbb{K}$ increases.

We compare several cluster centroid initialization approaches with Birch initialization to investigate how cluster initialization affects clustering performance. We record ARI using different initialization methods to evaluate the cluster quality. Figure 4 shows a bar chart comparison of TableDC trained over different initialization methods, including K-means, which has been used significantly in existing DC algorithms.

We can observe from Figure 4 that TableDC with Birch initialization provides the best results for all problems. The higher ARI score of TableDC with Birch initialization indicates the accurate identification of homogeneous clusters and clear separation between different clusters. K-means++ initialization appeared as the second best in domain discovery. From the comparative analysis, we also observed that with Birch initialization, the points are more widely dispersed around the centroid, indicating enhanced cluster compactness (how tightly the data points are grouped around the centroid of the cluster) and homogeneity (the similarity of data points within the same cluster).

To empirically assess the impact of loss functions in TableDC and existing benchmarks for data management problems, we use schema inference and web table data to plot (i) $re_{loss}$ and (ii) $\text{KL}_{\text{div}}$ (see Figure 5). A well-trained AE with effective $re_{loss}$ minimization, leads to embeddings where clusters are more distinct and separable, specifically in the cases of high density and overlap in the original embeddings. We can observe from Figure 5 that the $re_{loss}$ of TableDC is significantly reduced and more consistent compared to the benchmark methods, which exhibit fluctuations in loss during the learning of dense embeddings. For $\text{KL}_{\text{div}}$, the relationship between p and q (see Section 3.1) is investigated to show if q deviates from p. This divergence is necessary because the data points in the target p require greater intra-cluster distance rather than high cohesion. We only consider those benchmarks using p and q distribution and $\text{KL}_{\text{div}}$ loss to align the comparison with TableDC. TableDC shows a high divergence (see Figure 5) of q from p, which means that TableDC is more confident with the actual soft assignments q becoming sharper over time, and over highly packed p, the q needs to minimize the cohesiveness.

We evaluate the impact of difference distance measures and similarity kernels on the performance of TableDC, which utilizes a self-supervised module employing the Mahalanobis distance and Cauchy distribution to manage densely overlapping data effectively. We record the ARI and ACC (see Table 5) of TableDC by keeping the Cauchy distribution as a similar kernel and replacing the Mahalanobis distance with Euclidean (Bo et al., 2020) and Cosine distances (Salton et al., 1975), each providing different geometric properties. The Euclidean distance measures the straight-line distance between data points and their centroids, and the Cosine distance evaluates the cosine of the angle between vectors, emphasizing orientation over magnitude. Similarly, we kept the Mahalanobis distance constant while varying the similarity kernels. We replaced the Cauchy distribution with the Student’s t-distribution (Bo et al., 2020) (with an additional degree of freedom parameter that adjusts the distribution’s kurtosis.) and the Normal distribution (that provides a standard Gaussian decay of similarity).

From Table 5, we can observe that the choice of Mahalanobis distance combined with the Cauchy distribution appears optimal for all three problems. Euclidean distance with a Cauchy distribution performs relatively better in schema inference and domain discovery than cosine distance, showing that the straight-line distance between points is more reasonable than the angle when we have dense and overlapping clusters. With Student’s t distribution, TableDC shows a high ARI for schema inference but significantly lower ACC than the Normal distribution. The higher ARI suggests that it can effectively group similar items into clusters, but the low ACC indicates that it failed to handle overlapping clusters and mismatches predicted labels against the actual labels.