Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds

The study of class difficulty is most relevant to our work. The methods in the three domains presented above almost all assume that classes with few samples are the most difficult classes to be learned, and therefore more attention is given to these classes. However, recent studies [30, 31] have observed that the performance of some tail classes is even higher than that of the head classes, and that the performance of different classes varies on datasets with perfectly balanced samples. These phenomena suggest that the sample number is not the only factor that affects the performance of classes. The imbalance in class performance is referred to as the “bias” of the model, and [30] defines the model bias as

where $A_{c}$ denotes the accuracy of the $c$-th class. When the accuracy of each class is identical, bias = 0. [30] computes the difficulty of class c using $1-A_{c}$ and calculates the weights of the loss function using a nonlinear function of class difficulty. Unlike [30], [31] proposes a model-independent measure of classification difficulty, which directly utilizes the data matrix to calculate the semantic scale of each class to represent the classification difficulty. As with the sample number, model-independent measures can help us understand how deep neural networks learn from data. When we get data from any domain, if we can measure the difficulty of each class directly from the data, we can guide the researchers to collect the difficult classes in a targeted manner instead of blindly, greatly facilitating the efficiency of applying AI in practice.

In this work, we propose to consider the classification task as the classification of perceptual manifolds. The influence of the geometric characteristics of the perceptual manifold on the classification difficulty is further analyzed, and feature learning with curvature balanced is proposed.

A perceptual manifold is generated when neurons are stimulated by objects with different physical characteristics from the same class. Sampling along the different dimensions of the manifold corresponds to changes in specific physical characteristics. It has been shown [33, 34] that the features extracted by deep neural networks obey the manifold distribution law. That is, features from the same class are distributed near a low-dimensional manifold in the high-dimensional feature space. Given data $X=[x_{1},\dots,x_{m}]$ from the same class and a deep neural network $M\!odel=\{f(x,\theta_{1}),g(z,\theta_{2})\}$, where $f(x,\theta_{1})$ represents a feature sub-network with parameters $\theta_{1}$ and $g(z,\theta_{2})$ represents a classifier with parameters $\theta_{2}$. Extract the p-dimensional features $Z=[z_{1},\dots,z_{m}]\in\mathbb{R}^{p\times m}$ of $X$ with the trained model, where $z_{i}=f(x_{i},\theta_{1})\in\mathbb{R}^{p}$. Assuming that the features $Z$ belong to class $c$, the $m$ features form a $p$-dimensional point cloud manifold $M^{c}$, which is called a class perceptual manifold [56].

We measure the volume of the perceptual manifold $M^{c}$ by calculating the size of the subspace spanned by the features $z_{1},\dots,z_{m}$. First, the sample covariance matrix of $Z$ can be estimated as $\Sigma_{Z}=\mathbb{E}[\frac{1}{n}\sum_{i=1}^{n}z_{i}z_{i}^{T}]=\frac{1}{n}Z\!Z^{T}\in\mathbb{R}^{p\times p}.$ Diagonalize the covariance matrix $\Sigma_{Z}$ as $U\!DU^{T}$, where $D=diag(\lambda_{1},\dots,\lambda_{p})$ and $U=[u_{1},\dots,u_{p}]\in\mathbb{R}^{p\times p}$. $\lambda_{i}$ and $u_{i}$ denote the $i$-th eigenvalue of $\Sigma_{Z}$ and its corresponding eigenvector, respectively. Let the singular value of matrix $Z$ be $\sigma_{i}=\sqrt{\lambda_{i}}(i=1,\dots,p)$. According to the geometric meaning of singular value [57], the volume of the space spanned by vectors $z_{1},\dots,z_{m}$ is proportional to the product of the singular values of matrix $Z$, i.e., $V\!ol(Z)\propto{\textstyle\prod_{i=1}^{p}}\sigma_{i}=\sqrt{{\textstyle\prod_{i=1}^{p}}\lambda_{i}}$. Considering $\lambda_{1}\lambda_{2}\cdots\lambda_{p}=\det(\Sigma_{Z})$, the volume of the perceptual manifold is therefore denoted as $V\!ol(Z)\propto\sqrt{\det(\frac{1}{m}Z\!Z^{T})}$.

However, when $\frac{1}{m}Z\!Z^{T}$ is a non-full rank matrix, its determinant is $0$. For example, the determinant of a planar point set located in three-dimensional space is 0 because its covariance matrix has zero eigenvalues, but obviously the volume of the subspace tensed by the point set in the plane is non-zero. We want to obtain the “area” of the planar point set, which is a generalized volume. We avoid the non-full rank case by adding the unit matrix $I$ to the covariance matrix $\frac{1}{m}Z\!Z^{T}$. $I+\frac{1}{m}Z\!Z^{T}$ is a positive definite matrix with eigenvalues $\lambda_{i}+1(i=1,\dots,p)$. The above operation enables us to calculate the volume of a low-dimensional manifold embedded in high-dimensional space. The volume $V\!ol(Z)$ of the perceptual manifold is proportional to $\sqrt{\det(I+\frac{1}{m}Z\!Z^{T})}$. Considering the numerical stability, we further perform a logarithmic transformation on $\sqrt{\det(I+\frac{1}{m}Z\!Z^{T})}$ and define the volume of the perceptual manifold as

where $Z_{mean}$ is the mean of $Z$. When $m>1$, $V\!ol(Z>0$. Since $I+\frac{1}{m}(Z-Z_{mean})(Z-Z_{mean})^{T}$ is a positive definite matrix, its determinant is greater than 0. In the following, the degree of separation between perceptual manifolds will be proposed based on the volume of perceptual manifolds.

Euclidean or cosine distances between class centers are often applied to measure inter-class distances, and these two distances are also commonly used as loss functions when constructing sample pairs. However, maximizing the distance between proxy points or samples cannot keep a class away from all the remaining classes at the same time, and the distance between class centers does not reflect the degree of overlap of the distribution. In this section, we propose a measure of the separation degree between perceptual manifolds.

Given the perceptual manifolds $M^{1}$ and $M^{2}$, they consist of point sets $Z_{1}=[z_{1,1},\dots,z_{1,m_{1}}]\in\mathbb{R}^{p\times m_{1}}$ and $Z_{2}=[z_{2,1},\dots,z_{2,m_{2}}]\in\mathbb{R}^{p\times m_{2}}$, respectively. The volumes of $M^{1}$ and $M^{2}$ are calculated as $V\!ol(Z_{1})$ and $V\!ol(Z_{2})$. Consider the following case, assuming that $M^{1}$ and $M^{2}$ have partially overlapped, when $V\!ol(Z_{1})\ll V\!ol(Z_{2})$, it is obvious that the overlapped volume accounts for a larger proportion of the volume of $M^{1}$, when the class corresponding to $M^{1}$ is more likely to be confused. Therefore, it is necessary to construct an asymmetric measure for the degree of separation between multiple perceptual manifolds, and we expect this measure to accurately reflect the relative magnitude of the degree of separation.

Suppose there are $C$ perceptual manifolds $\{M^{i}\}_{i=1}^{C}$, which consist of point sets $\{Z_{i}=[z_{i,1},\dots,z_{i,m_{i}}]\in\mathbb{R}^{p\times m_{i}}\}_{i=1}^{C}$. Let $Z=[Z_{1},\dots,Z_{C}]\in\mathbb{R}^{p\times{\textstyle\sum_{j=1}^{C}}m_{j}}$, $Z^{\prime}=[Z_{1},\dots,Z_{i-1},Z_{i+1},\dots,Z_{C}]\in\mathbb{R}^{p\times(({\textstyle\sum_{j=1}^{C}}m_{j})-m_{i})}$, we define the degree of separation between the perceptual manifold $M^{i}$ and the rest of the perceptual manifolds as

The following analysis is performed for the case when $C=2$ and $V\!ol(Z_{2})>V\!ol(Z_{1})$. According to our motivation, the measure of the degree of separation between perceptual manifolds should satisfy $S(M^{2})>S(M^{1})$.

If $S(M^{2})>S(M^{1})$ holds, then we can get

We prove that $V\!ol(Z)<V\!ol(Z_{1})+V\!ol(Z_{2})$ holds when $V\!ol(Z_{2})>V\!ol(Z_{1})$, and the details are as follows.

The above analysis shows that the proposed measure meets our requirements and motivation. The formula for calculating the degree of separation between perceptual manifolds can be further reduced to

Next, we validate the proposed measure of the separation degree between perceptual manifolds in a 3D spherical point cloud scene. Specifically, we conducted the experiments in three cases:

• (1)

  Construct two 3D spherical point clouds of radius $1$, and then increase the distance between their spherical centers. Since the volumes of the two spherical point clouds are equal, their separation degrees should be symmetric. The variation curves of the separation degrees are plotted in Fig.3, and it can be seen that the experimental results satisfy our theoretical predictions.

• (2)

  Change the distance between the centers of two spherical point clouds. Observe their separation degrees, the separation degrees of these two spherical point clouds should be asymmetric. Fig.3 shows that their separation degrees increase as the distance between their centers increases. Also, the manifold with a larger radius has a greater separation degree, and this experimental result conforms to our analysis and motivation.

• (3)

  As shown in Fig.3, as the volume difference between the two spherical point cloud manifolds becomes larger, the difference in separation between the two increases, a result that is entirely consistent with our motivation.

The separation degree between perceptual manifolds may affect the model’s bias towards classes. In addition, it can also be used as the regularization term of the loss function or applied in contrast learning to keep the different perceptual manifolds away from each other.

Given a point cloud perceptual manifold $M$, which consists of a $p$-dimensional point set $\{z_{1},\dots,z_{n}\}$, our goal is to calculate the Gauss curvature at each point. First, the normal vector at each point on $M$ is estimated by the neighbor points. Denote by $z_{i}^{j}$ the $j$-th neighbor point of $z_{i}$ and $u_{i}$ the normal vector at $z_{i}$. We solve for the normal vector by minimizing the inner product of $z_{i}^{j}-c_{i},j=1,\dots,k$ and $u_{i}$ [59], i.e.,

where $c_{i}=\frac{1}{k}{\textstyle\sum_{j=1}^{k}}z_{i}^{j}$ and $k$ is the number of neighbor points. Let $y_{j}=z_{i}^{j}-c_{i}$, then the optimization objective is converted to

${\textstyle\sum_{j=1}^{k}}y_{j}y_{j}^{T}$ is the covariance matrix of $k$ neighbors of $z_{i}$. Therefore, let $Y=[y_{1},\dots,y_{k}]\in\mathbb{R}^{p\times k}$ and ${\textstyle\sum_{j=1}^{k}}y_{j}y_{j}^{T}=YY^{T}$. The optimization objective is further equated to

Construct the Lagrangian function $L(u_{i},\lambda)=f(u_{i})-\lambda(u_{i}^{T}u_{i}-1)$ for the above optimization objective, where $\lambda$ is a parameter. The first-order partial derivatives of $L(u_{i},\lambda)$ with respect to $u_{i}$ and $\lambda$ are

Let $\frac{\partial L(u_{i},\lambda)}{\partial u_{i}}$ and $\frac{\partial L(u_{i},\lambda)}{\partial\lambda}$ be $0$, we can get $YY^{T}u_{i}=\lambda u_{i},u_{i}^{T}u_{i}=1$. It is obvious that solving for $u_{i}$ is equivalent to calculating the eigenvectors of the covariance matrix $YY^{T}$, but the eigenvectors are not unique. From $\left\langle YY^{T}u_{i},u_{i}\right\rangle=\left\langle\lambda u_{i},u_{i}\right\rangle$ we can get $\lambda=\left\langle YY^{T}u_{i},u_{i}\right\rangle=u_{i}^{T}YY^{T}u_{i}$, so the optimization problem is equated to $\mathop{\arg\min}_{u_{i}}(\lambda)$. Performing the eigenvalue decomposition on the matrix $YY^{T}$ yields $p$ eigenvalues $\lambda_{1},\dots,\lambda_{p}$ and the corresponding $p$-dimensional eigenvectors $[\xi_{1},\dots,\xi_{p}]\in\mathbb{R}^{p\times p}$, where $\lambda_{1}\geq\dots\geq\lambda_{p}\geq 0$, $\left\|\xi_{i}\right\|_{2}=1,i=1,\dots,p$, $\left\langle\xi_{a},\xi_{b}\right\rangle=0(a\neq b)$. The eigenvector $\xi_{m+1}$ corresponding to the smallest non-zero eigenvalue of the matrix $YY^{T}$ is taken as the normal vector $u_{i}$ of $M$ at $z_{i}$.

Consider an $m$-dimensional affine space with center $z_{i}$, which is spanned by $\xi_{1},\dots,\xi_{m}$. This affine space approximates the tangent space at $z_{i}$ on $M$. We estimate the curvature of $M$ at $z_{i}$ by fitting a quadratic hypersurface in the tangent space utilizing the neighbor points of $z_{i}$. The $k$ neighbors of $z_{i}$ are projected into the affine space $z_{i}+\left\langle\xi_{1},\dots,\xi_{m}\right\rangle$ and denoted as

Denote by $o_{j}[m]$ the $m$-th component $(z_{i}^{j}-z_{i})\cdot\xi_{m}$ of $o_{j}$. We use $z_{i}$ and $k$ neighbor points to fit a quadratic hypersurface $f(\theta)$ with parameter $\theta\in\mathbb{R}^{m\times m}$. The hypersurface equation is denoted as

further, minimize the squared error

Let $\frac{\partial E(\theta)}{\partial\theta_{a,b}}=0,a,b\in\left\{1,\dots,m\right\}$ yield a nonlinear system of equations, but it needs to be solved iteratively. Here, we propose an ingenious method to fit the hypersurface and give the analytic solution of the parameter $\theta$ directly. Expand the parameter $\theta$ of the hypersurface into the column vector

Organize the $k$ neighbor points $\left\{o_{j}\right\}_{j=1}^{k}$ of $z_{i}$ according to the following form:

The target value is

We minimize the squared error

and find the partial derivative of $E(\theta)$ for $\theta$:

Let $\frac{\partial E(\theta)}{\partial\theta}=0$, we can get

Thus, the Gauss curvature of the perceptual manifold $M$ at $z_{i}$ can be calculated as

Up to this point, we provide an approximate solution of the Gauss curvature at any point on the point cloud perceptual manifold $M$. [60] shows that on a high-dimensional dataset, almost all samples lie on convex locations, and thus the complexity of the perceptual manifold is defined as the average $\frac{1}{n}{\textstyle\sum_{i=1}^{n}}G(z_{i})$ of the Gauss curvatures at all points on $M$. Our approach does not require iterative optimization and can be quickly deployed in a deep neural network to calculate the Gauss curvature of the perceptual manifold. Taking the two-dimensional surface in Fig.5 as an example, the surface complexity increases as the surface curvature is artificially increased. This indicates that our proposed complexity measure of perceptual manifold can accurately reflect the changing trend of the curvature degree of the manifold. In addition, Fig.5 shows that the selection of the number of neighboring points hardly affects the monotonicity of the complexity of the perceptual manifold. In our work, we select the number of neighboring points to be $40$.

Learning typically leads to greater inter-class distance, which equates to greater separation between perceptual manifolds. We trained VGG-16 [61] and ResNet-18 [62] on F-MNIST [63] and CIFAR-10 [64] to explore the effect of the learning process on the separation degree between perceptual manifolds and observed the following phenomenon.

As shown in Fig.7, each perceptual manifold is gradually separated from the other manifolds during training. It is noteworthy that the separation is faster in the early stage of training, and the increment of separation degree gradually decreases in the later stage.

We conducted experiments on CIFAR-10, CIFAR-100, and SVHN by training ResNet-18, SeNet-34, and ShuffleNetV2 models. We extracted embeddings for each class of images at different training stages of the models. For visualization purposes, we averaged the curvature of perceptual manifolds corresponding to each class and plotted them in Fig.8. It can be observed that the curvature of perceptual manifolds decreases rapidly in the early stages of training, but as training progresses, the rate of decrease gradually slows down. Compared to the initial curvature, the degree of decrease appears less pronounced, for example, the curvature of perceptual manifolds decreased by less than $10\%$ on CIFAR-10. We speculate that this is due to the lack of curvature constraints in the optimization objective, leading to the rapid initial decrease because of the widespread information compression ability in deep neural networks. To confirm this hypothesis, we plotted the curves of loss decrease and curvature imbalance as a function of epochs in Fig.9. It can be observed that on all three datasets, as the loss gradually converges, the rate of decrease in curvature and imbalance also gradually decreases.

The above experiments indicate that deep neural networks, driven by optimization objectives lacking curvature constraints, are still capable of reducing the curvature of perceptual manifolds through information compression. This is understandable, as without information compression, achieving classification would be challenging. However, we must consider whether optimization objectives without curvature constraints are sufficient to address model bias caused by curvature imbalance. In the next subsection, we visualize the curve of the correlation between the curvature of the class perceptual manifold and the class accuracy with epoch to answer this question.

Although existing models separate class perceptual manifolds from each other during the learning process and also flatten the perceptual manifolds, do existing models have enough power to adequately mitigate the model bias caused by these two factors? We trained VGG-16 and ResNet-18 on Fashion MNIST and CIFAR-10 and plotted the correlation between the separation and curvature of class perceptual manifolds and class accuracy as a function of epoch.

The experimental results, as shown in Fig.10, reveal a decrease in the correlation between the separability of perceptual manifolds and the accuracy of corresponding classes as training progresses, while the negative correlation between curvature and accuracy increases. This suggests that existing methods can only alleviate the impact of the separability between perceptual manifolds on model bias while overlooking the influence of the complexity of perceptual manifolds on model bias. Additionally, we further trained ResNet-18, SeNet-34, and ShuffleNetV2 on CIFAR-100 and SVHN to thoroughly observe the trends of curvature and class accuracy of class-aware manifolds as a function of epochs (see Fig.11). The experimental results demonstrate that existing models lack constraints on curvature during the training process, resulting in a highly negative correlation between curvature and class accuracy after model convergence.

The proposed curvature regularization needs to satisfy the following three principles to learn curvature-balanced and flat perceptual manifolds.

(1) The greater the curvature of a perceptual manifold, the stronger the penalty for it. Our experiments show that learning reduces the curvature, so it is reasonable to assume that flatter perceptual manifolds are easier to decode. (2) When the curvature is balanced, the penalty strength is the same for each perceptual manifold. (3) The sum of the curvatures of all perceptual manifolds tends to decrease.

In order to propose curvature regularization in a reasonable way, we start from softmax cross-entropy loss to inspire our method. Given a $C$ classification task, suppose a sample $x$ is labeled as $Y_{k}$ and it is predicted as each class with probabilities $P_{1},P_{2},\dots,P_{C}$, respectively. The cross-entropy loss generated by sample $x$ is calculated as $L(x)={\textstyle\sum_{i=1}^{C}}-Y_{i}\log(P_{i})$, where $Y_{k}=1,Y_{i}=0,i\neq k$. The goal of $L(x)$ is to make $\log(P_{k})$ converge to $0$, i.e., $P_{k}$ converges to $1$, at which point $P_{i}(i\neq k)$ converges to $0$. Unlike cross-entropy loss, which can pull apart the difference between $P_{k}$ and other probabilities, we expect the mean Gaussian curvature of the $C$ perceptual manifolds to converge to equilibrium.

Assume that the mean Gaussian curvatures of the $C$ perceptual manifolds are $G_{1},G_{2},\dots,G_{C}$ (Algorithm 3), and perform the maximum normalization on them. The $-\log(G_{k})$ loss can make $G_{k}$ converge to $1$. Therefore, perform a negative logarithmic transformation on the curvature of all perceptual manifolds and use it as loss, which can make each curvature converge to $1$ and thus achieve curvature balance. However, the above operation violates the third design principle of curvature regularization, which is that the sum of curvatures of all perceptual manifolds tends to decrease. As shown in Fig.12, all curvatures smaller than $G_{1}$ gradually increase driven by the loss function, and the smaller the curvature, the greater the resulting loss. To solve this problem, we update each curvature to the inverse of itself before performing the maximum normalization of the curvature. Eventually, the curvature penalty term of the perceptual manifold $M^{i}$ is denoted as $-\log(\frac{G_{i}^{-1}}{\max\{G_{1}^{-1},\dots,G_{C}^{-1}\}})$. Considering the differentiability of the loss function, we use a smoothed form of the $\max$ function, resulting in the final form of the curvature regularization: