On Generalized Degree Fairness in Graph Neural Networks

Modern GNNs usually resort to multi-layer neighborhood aggregation, in which each node recursively aggregates information from its neighbors. Specifically, in the $l$-th layer the representation of node $v$, $\mathbf{h}^{l}_{v}\in\mathbb{R}^{d_{l}}$, is constructed as

where $d_{l}$ is the dimension of node representations in the $l$-th layer, $\textsc{Aggr}(\cdot)$ denotes an aggregation function such as mean-pooling (Kipf and Welling 2017) or self-attention (Veličković et al. 2018), $\sigma$ is an activation function, $\mathcal{N}_{v}$ denotes the set of neighbors of $v$, and $\omega^{l}$ denotes the learnable parameters in layer $l$. Node representations in the input layer are given by the input node attributes, i.e., $\mathbf{h}^{0}_{v}\equiv\mathbf{x}_{v}$.

The overall framework of DegFairGNN is illustrated in Fig. 1. We first design a strategy of structural contrast in Fig. 1(b), to split nodes into two groups by their degree. The two groups contrast each other to facilitate the learning of a debiasing function, which generates node-specific debiasing contexts to modulate neighborhood aggregation, as shown in Fig. 1(c). More specifically, the modulation aims to distill high-degree nodes to remove bias, and complement low-degree nodes to enrich their degree-biased neighborhood structures. Thus, by trying to balance the two groups, the modulation is able to debias neighborhood aggregation. Finally, the debiasing function and GNN are jointly optimized by the task loss and the fairness loss in Fig. 1(d).

To debias nodes with varying contextual structures, we propose to learn the debiasing function by contrasting between two groups of nodes, namely, low-degree nodes $\mathcal{S}_{0}$ and high-degree nodes $\mathcal{S}_{1}$, as illustrated in Fig. 1(b). Note that in the layer-wise neighborhood aggregation in Eq. (4), each node can only access its one-hop local contexts in each layer. Thus, it is a natural choice to construct the two groups based on one-hop degree: $\mathcal{S}_{0}=\{v\in\mathcal{V}\mid\deg_{1}(v)\leqslant K\}$ and $\mathcal{S}_{1}=\mathcal{V}\setminus\mathcal{S}_{0}$. That is, $\mathcal{S}_{0}$ is the low-degree group, $\mathcal{S}_{1}$ is the high-degree group, and $K$ is a threshold hyperparameter. Note that having two groups can provide the most significant contrast for model training.

To contrast between the two groups, nodes in the same group would share the learnable parameters for the debiasing mechanism, whereas nodes across different groups would have different parameters. This strategy enables the two groups to eliminate the degree bias in different ways, which is desirable given their different degrees.

To fundamentally eliminate degree bias, we propose to debias neighborhood aggregation, the key operation in GNNs in which degree bias is rooted. As shown in Fig. 1(c), we leverage a learnable debiasing function to generate the debiasing contexts, which modulate the neighborhood aggregation for each node and in each layer of the GNN encoder.

The debiasing function is designed to generate a debiasing context for each node in each layer, to modulate the neighborhood aggregation in a node- and layer-wise manner. To achieve ideal modulations, the debiasing contexts should be comprehensive and adaptive.

Comprehensiveness. First, debiasing contexts need to be comprehensive, to account for both the content and structure information in the neighborhood. To be comprehensive, we resort to the context embedding of node $v$ in layer $l$, denoted $\mathbf{c}^{l}_{v}\in\mathbb{R}^{d_{l-1}}$, which can be calculated as

where $\textsc{Pool}(\cdot)$ is a pooling function. Here we use a simple mean pooling, although it can also be made learnable. The context embedding $\mathbf{c}^{l}_{v}$ aggregates the layer-($l$-$1$) contents in node $v$’s $r$-hop local context $\mathcal{C}_{r}(v)$, and thus embodies both the content and structure information. The debiasing function is then a function of the context embedding, i.e.,

which is parameterized by $\theta^{l}_{c,*}$, where $*$ is $0$ or $1$ depending on the node group. We use a fully connected layer as $f$.

Adaptiveness. Second, debiasing contexts need to be adaptive, to sufficiently customize to each node. While the two groups of nodes, $\mathcal{S}_{0}$ and $\mathcal{S}_{1}$, are already differentiated by group-specific parameters for the debiasing function, the differentiation is coarse-grained and cannot sufficiently adapt to individual nodes. In particular, even for nodes in the same group, their degrees still vary, motivating the need for finer-grained node-wise adaptation. However, letting each node have node-specific parameters is not feasible due to a much larger model size, which tends to cause overfitting and scalability issues. For fine-grained adaptation without blowing up the model size, inspired by hypernetworks (Ha, Dai, and Le 2017; Perez et al. 2018; Liu et al. 2021), we adopt a secondary neural network to generate node-wise transformations on the debiasing contexts in each layer. Concretely, the debiasing function can be reformulated as

where $\gamma^{l}_{v}$ and $\beta^{l}_{v}\in\mathbb{R}^{d_{l}}$ are node-specific scaling and shifting operators in layer $l$, respectively. Here $\odot$ denotes element-wise multiplication, and $\mathbf{1}$ is a vector of ones to ensure that the scaling is centered around one. Note that $\gamma^{l}_{v}$ and $\beta^{l}_{v}\in\mathbb{R}^{d_{l}}$ are not directly learnable, but are respectively generated by a secondary network conditioned on each node’s degree. Specifically,

where $\phi_{\gamma}$ and $\phi_{\beta}$ can be any neural network, and we simply use a fully connected layer. The input to these secondary networks, $\delta^{l}(v)$, is the degree encoding of $v$ to condition the transformations on the degree of $v$, which we will elaborate later. Thus, the learnable parameters of the debiasing function in Eq. (7) become $\theta^{l}_{*}=\{\theta^{l}_{c,*},\theta^{l}_{\gamma},\theta^{l}_{\beta}\}$ in layer $l$, and $\theta_{*}=\{\theta^{1}_{*},\theta^{2}_{*},\ldots\}$ denotes the full set of learnable parameters of the debiasing function for nodes in group $\mathcal{S}_{*}$.

Finally, the input to the secondary network in Eq. (8) is the degree encoding $\delta^{l}(v)$, instead of the degree of $v$ itself. The reason is that degree is an ordinal variable, implicating that degree-conditioned functions should be smooth w.r.t. small changes in degree. In other words, nodes with similar degrees should undergo similar transformations, and vice versa. In light of this, inspired by positional encoding (Vaswani et al. 2017), we propose to encode the degree of a node $v$ in layer $l$ into a vector $\delta^{l}(v)\in\mathbb{R}^{d_{l}}$, such that $[\delta^{l}(v)]_{2i}=\sin(\deg_{1}(v)/10000^{2i/d_{l}})$ and $[\delta^{l}(v)]_{2i+1}=\cos(\deg_{1}(v)/10000^{2i/d_{l}})$. Note that, although nodes at the periphery of the two groups $\mathcal{S}_{0}$ and $\mathcal{S}_{1}$ (e.g., nodes with degrees $K$ and $K+1$) have different debiasing parameters ($\theta_{c,0}$ and $\theta_{c,1}$), the usage of degree encoding in Eqs. (7) and (8) will assimilate their debiasing contexts to some extent given their similar degrees.

We focus on the task of node classification. Apart from the classification loss and the fairness loss associated with the classification, we further introduce several useful constraints to improve the training process, as shown in Fig. 1(d).

For model fairness, the group-wise DSP and DEO (Sect. 2) essentially require that the outcomes are independent of the degree groups. In particular, we form two groups from the test set: $\mathcal{G}_{0}$ and $\mathcal{G}_{1}$, containing test nodes with low and high generalized degrees, respectively. A two-group granule is a first major step toward degree fairness (resource-poor group vs. resource-rich group), where the fairness issue is the most serious. As degree usually follows a long-tailed distribution (Liu et al. 2020), based on the Pareto principle (Newman 2005) we select the top 20% test nodes by generalized degree as $\mathcal{G}_{1}$, and the bottom 20% nodes as $\mathcal{G}_{0}$, which may present more prominent biases. To thoroughly evaluate the fairness, we also test on alternative groups with top and bottom 30% nodes. Moreover, generalized degree fairness is defined w.r.t. a pre-determined number of hops $r$. We employ $r=1$ as the default, and further report the evaluation with $r=2$. We only select $r\leq 2$ for evaluation as GNNs usually have shallow layers, which implies a small $r$ can sufficiently cover the contextual structures where biases are rooted. Hence, we adopt the following metrics $\Delta_{\text{DSP}}$ and $\Delta_{\text{DEO}}$, which evaluate the mean difference between the distributions of the two groups ($\mathcal{G}_{1}$ and $\mathcal{G}_{0}$) in the test set. For both metrics, a smaller value implies a better fairness.

We first evaluate our model using the default base GNN (i.e., GCN) and fairness settings, and further supplement it with additional fairness settings and base GNNs.

Firstly, DegFairGCN consistently outperforms the baselines in both fairness metrics, while achieving a comparable classification accuracy, noting that there usually exists a trade-off between fairness and accuracy (Bose and Hamilton 2019; Dai and Wang 2021). The only exception is on the Squirrel dataset, where FairWalk obtains the best $\Delta_{\text{DSP}}$ at the expense of a significant degradation in accuracy, while DegFairGCN remains a close runner-up ahead of other baselines. Secondly, degree-specific models DSGCN, Residual2Vec and Tail-GNN typically have worse fairness than fairness-aware models, since they exploit the structural difference between nodes for model performance, rather than to eliminate the degree bias for model fairness. Thirdly, the advantage of DegFairGCN over existing fairness-aware models including FairWalk, CFC, FairGNN, FairAdj and FairVGNN implies that degree fairness needs to be addressed at the root, by debiasing the core operation of layer-wise neighborhood aggregation. Merely treating degree as a sensitive attribute cannot fundamentally alleviate the structural degree bias. Interestingly, GCN can generally achieve comparable and sometimes even better fairness than these methods, which again shows that degree fairness cannot be sufficiently addressed without directly debiasing neighborhood aggregation.

Altogether, the three base GNNs employ different neighborhood aggregations, e.g., mean or attention-based mechanisms, but none of them employs a fairness-aware aggregation. The results show that our models can generally outperform their corresponding base GNN models across the three datasets in terms of fairness, demonstrating the flexibility of DegFairGNN when working with different GNN backbones.

We conduct further model analysis on DegFairGNN, using the default base GNN and fairness settings.

We report the results in Fig. 2 and make several observations. Firstly, without scaling and shifting, we often get worse accuracy and fairness, which means node-wise adaptation is useful and the model can benefit from the finer-grained treatment of nodes. Secondly, without structural contrast, the fairness metrics generally become worse. Thus, structural contrast is effective in driving the debiasing contexts toward degree fairness. Lastly, without the modulation of neighborhood aggregation, the fairness metrics become worse in most cases, implying that simply adding a fairness loss without properly debiasing the layer-wise neighborhood aggregation does not work well.