S²MVTC: a Simple yet Efficient Scalable Multi-View Tensor Clustering

For clarity, we present notations frequently used in Tab. 1.

According to the T-product operation in Definition 1, this decomposition can be obtained by Algorithm 1, where $\mathrm{fft}$ and $\mathrm{ifft}$ are the fast Fourier transform (FFT) operator and the inverse FFT, respectively.

In this section, we will focus on how to quickly learn the consistency among embedding features well, which involves the exploration of intra-view graph similarity (IGS) and ensuring inter-view semantic consistency (ISC). With anchor graphs $\phi(\mathbf{X}_{v})\in\mathbb{R}^{M\times N},v=1,\cdots,V$ obtained in advance, we can first learn $V$ embedding features by projection matrices $\mathbf{U}_{v},v=1,\cdots,V$, as follows:

where $\lambda$ serves as a weight aiming to attain a stable solution for the mapping matrix $\mathbf{U}_{v}$. $\mathbf{B}_{v}(:,n)\in\mathbb{S}$ represents the $z$-score normalization constraint, ensuring that each feature from different views has a mean of 0 and a standard deviation of 1. Here $\mathbf{b}_{v}^{n}=\mathbf{B}_{v}(:,n)$, and $\mathbb{S}=\{\mathbf{b}_{v}^{n}:\sum_{k=1}^{K}\mathbf{b}_{v}^{n}(k)={0},\frac{\sum_{k=1}^{K}(\mathbf{b}_{v}^{n}(k))^{2}}{K-1}=1,n=1,\cdots,N\}$.

To efficiently explore the intra-view information of embedding features, we initially stack the embedding features from each view into a tensor. Subsequently, we introduce a newly tensor low-frequency approximation $\|\mathcal{B}\|_{\text{TLFA}}$, applied to its rotated form $\mathcal{B}\in\mathbb{R}^{K\times V\times N}$, aiming to capture the IGS.

Besides, the term $\sum_{v=1}^{V}\|\tilde{\mathbf{B}}-\mathbf{B}_{v}\|_{\operatorname{F}}^{2}$ is included to ensure ISC [40], where $\tilde{\mathbf{B}}$ is the average embedding feature.

Overall, the framework of S²MVTC is formulated as:

Here $\mathcal{B}=\Omega(\mathbf{B}_{1},\cdots,\mathbf{B}_{V})$, where $\Omega$ represents the operation of stacking each feature and rotating and the inverse operator $\mathbf{B}_{v}=\Omega_{v}^{-1}(\mathcal{B})$.

To make the above optimization problem separable, we introduce an auxiliary variable $\mathcal{Y}$, leading to the reformulation as follows:

This optimal problem in Eq. 10 can be solved through an alternating optimization approach, where each parameter is updated individually while keeping the others fixed.
Update $\mathbf{U}_{v}$. Fixing other variables, the subproblem of $\mathbf{U}_{v}$ can be rewritten as:

By taking the derivative of the equation and setting it equal to zero, we can obtain:

where $\phi(\mathbf{X}_{v})\phi^{\operatorname{T}}(\mathbf{X}_{v})$ can be pre-calculated outside the main loop to decrease the computational cost. Therefore, the storage and computational complexity are $O(MN)$ and $O(\max(KMN,M^{3}))$, respectively.
Update $\mathbf{B}_{v}$. The subproblem of $\mathbf{B}_{v}$ can be rephrased as:

Similarly, by taking the derivative of the equation and setting it equal to zero, we can obtain

where $\Omega_{v}^{-1}(\mathcal{Y})=\mathbf{Y}_{v}\in\mathbb{R}^{K\times N}$. $\operatorname{normalize}$ is the $z$-score normalization function, which can be achieved by the Matlab command “normalize”. In this case, both the storage and computational complexity of updating $\mathbf{B}_{v}$ are $O(KN)$.
Update $\mathcal{Y}$. The subproblem of $\mathcal{Y}$ is rewritten as:

According to the definition of $\|\mathcal{Y}\|_{\text{TLFA}}$, a closed-form solution for $\mathcal{Y}$ can be obtained by choosing the low-frequency component in the Fourier domain. The details are outlined in Algorithm 2. The computational and storage complexity are $O(KVN\log(N))$ and $O(KNV)$, respectively.
Update $\tilde{\mathbf{B}}$. The subproblem of updating $\tilde{\mathbf{B}}$ is:

The solution of $\tilde{\mathbf{B}}$ is:

which needs computational and storage complexity of $O(KN)$ and $O(KNV)$, respectively.

After obtaining the consistent fused embedding feature $\tilde{\mathbf{B}}$. The final clustering results can be obtained by the following clustering structure learning framework:

where $\mathbf{D}$ is the clustering center and $\mathbf{G}$ is the clustering indicator matrix.

Overall, the method S²MVTC is summarized in Algorithm 3. According to the parameters analysis in the algorithm, the main storage and computational complexities of S²MVTC are $O(\max(KVN,MN))$ and $O(\max(KMNT,M^{3}T))$, respectively, where $T$ represents the total number of iterations in the S²MVTC.

In addition, the theoretical convergence can be well guaranteed. Our problem is bounded due to the summation of norms with positive penalty parameters. Furthermore, the exact minimum points of each subproblem can be achieved, implying that each subproblem exhibits a monotone decrease. The proposed algorithm converges according to the convergence theorem in [28] (theorem 7.29).

Multi-view Datasets: Six well-known large-scale multi-view datasets, including CCV  [13], NUS-WIDE-OBJ [7], Caltech102 [19], AwA [18], Cifar-10 [17], YoutubeFace_sel [41] evaluate the effectiveness of S²MVTC, where the statistical information is presented in Tab. 2.
Compared Clustering Algorithms. To compare clustering performance, we select $k$-means and seven state-of-the-art methods: binary multi-view clustering (BMVC) [2018, TPMAI] [45], fast multi-View anchor-correspondence clustering (FMVACC) [2022, NeurIPS] [38], one-pass multi-view clustering (OPMC) [2021, ICCV] [23], fast multi-view clustering via ensembles (FastMICE) [2023, TKDE] [10], scalable and parameter-free multiview graph clustering (SFMC) [2020, TPMAI] [20], fast self-guided multi-view subspace clustering (FSMSC) [2023, TIP] [6], fast parameter-free multi-view subspace clustering with consensus anchor guidance (FPMVS-CAG) [2021, TIP] [37]. One low-rank tensor-based multi-view method: scalable low-rank MERA based multi-view clustering (sMERA-MVC) [2023, TMM] [24]. All tests were conducted on a desktop computer equipped with a 3.79GHz AMD Ryzen 9 3900X CPU and 64GB RAM, using MatLab 2021b.

Evaluation Metrics. To assess the performance in our experiments, eight standard evaluation metrics are utilized, including Accuracy (ACC), Normalized Mutual Information (NMI), Purity, F-score, Precision (PRE), Recall (REC), Adjusted Rand Index (ARI), and CPU Time. These metrics, except for CPU Time, are designed such that higher values signify superior clustering performance.
Parameter Sensitivity Analysis. In our experiment, we employed a grid search strategy to determine the optimal choices for all parameters. S²MVTC has a fixed parameter $M$, which is the size of the anchor, and three free parameters: low frequency parameter, denoted as $L$, and two balance parameters, $\lambda$ and $\beta$. We fixed two of these parameters and adjusted the third through an exhaustive search. For instance in dataset Caltech102, we initially fixed $L=16$ and $M=1000$, and adjusted parameters $\beta$ and $\lambda$, where $\beta$ and $\lambda$ took values in {$10^{-4}$, $10^{-3}$, …, $10^{2}$}. The clustering results are shown in Fig. 33(a), where the performance remains stable with $\beta$ in the range of $[10^{-4},1]$, with ACC close to 55.47%. We observed that the best clustering performance can be achieved within a wide range of $\lambda$, specifically $\lambda\in[10^{-1},10^{1}]$. Additionally, we fixed $\beta=1$ and $\lambda=10$, and selected parameter $M$ from {500, 1000, …, 3000} and parameter $L$ from {10, 16, …, 40}. In Fig. 33(b), performance is stable for $L$ within the $[16,22]$ range, achieving an ACC of approximately 55.47%. Notably, $M$ was consistently set to 1000 for all test datasets. To ensure more consistent anchor selection in random sampling, the dataset is sorted in ascending order.

Tab. 3 reports the clustering performance of various methods on six large-scale multi-view datasets, where the best and second-best are highlighted in bold and underlined, respectively. From the table, it is evident that sMERA-MVC and S²MVTC significantly outperform others across various metrics such as ACC, NMI, Purity, F-score, PRE, REC, and ARI. sMERA-MVC, leveraging advanced tensor networks, excels in exploring both inter-view and intra-view correlations of anchor graphs among views simultaneously. This allows sMERA-MVC to utilize the most compact structure for identifying global correlations among anchor graphs, which could be a key factor contributing to its superior performance.

In comparison with sMERA-MVC, S²MVTC consistently outperforms it in most scenarios. Notably, S²MVTC demonstrates a significant improvement in clustering performance on the CCV dataset. For instance, the clustering performance of S²MVTC has shown remarkable enhancements of 9.25%, 11.17%, and 13.71% in terms of ACC, NMI, and ARI, respectively. In the case of the Caltech102, S²MVTC exhibits a performance advantage, achieving a 10.24% higher ACC compared to sMERA-MVC. For the NUS-WIDE-OBJ and AwA datasets, the performances of both methods are similar. Regarding the YoutubeFace_sel dataset, which is larger than other large-scale datasets, sFSR-IMVC demonstrates the best performance compared to sMERA-MVC.

Significantly, in terms of CPU time, , S²MVTC outperforms sMERA-MVC across all large datasets, especially with larger sizes. For instance, it achieves an approximation speedup of 39 times on AwA and 20 times on Cifar-10. Furthermore, BMVC, relying on binary feature representation for clustering, exhibits a CPU time performance at a comparable scale to S²MVTC on large datasets. However, the clustering performance of S²MVTC surpasses BMVC, with ACC improvements of 32.16% on CCV, 25.47% on Caltech102, 48.22% on NUS-WIDE-OBJ, 52.04% on Cifar-10, and 29.18% on YoutubeFace_sel, respectively.

In summary, based on the analysis of these metrics, it can be concluded that the S²MVTC algorithm excels not only in accuracy for multi-view clustering tasks but also significantly outperforms traditional algorithms in terms of computational efficiency, making it suitable for fast large-scale multi-view clustering tasks.

Model Analysis. Fig. 4 illustrates the embedding feature learning process of S²MVTC on the CCV dataset. Compared the 1-st and 3-rd columns, it is evident that after the TLFA operation, the dispersion of sample points increases. This is because that TLFA aims at identifying the compact subspace representation of embedding features across views, known as a smooth representation. During the iterative process, the complementary information captured in this compact subspace further propagates across views. Simultaneously, the exploration of complementary information facilitates the discovery of improved and more smooth subspace representations, ultimately resulting in enhanced clustering performance. For instance, after the 6-th TLFA operation, the clustering performance reaches 54.79%.

Ablation Study. The S²MVTC model consists primarily of three components: 1. Exploration of ISC; 2. Exploration of IGS; 3. Adoption of a nonlinear anchor graph. To further investigate why S²MVTC performs well, we systematically removed each component while keeping the other two fixed, naming these methods as w/o ISC, w/o IGS, and linear anchor graphs. The results are presented in Tab. 4. From the table, it can be observed that removing the exploration of ISC leads to a slight decline in clustering performance, while the removal of the exploration of IGS results in a significant drop in clustering performance. This indicates that the comprehensive exploration of graph similarity among intra-view features effectively, thereby improving clustering performance. Furthermore, when the anchor graph is transformed into a linear mapping between anchors and samples, clustering performance decreases with an increase in the number of samples, especially in Cifar-10 and YoutubeFace_sel. This is because, with a fixed number of anchors, a nonlinear anchor graph is better at capturing relationships between samples when the sample size is exceptionally large.