Graph Neural Networks for Wireless Networks:
Graph Representation, Architecture and Evaluation

The development of smart radio environment (SRE) enabler and high-frequency communication complicates the wireless network architectures and propagation of radio frequency signals. It becomes increasingly crucial and challenging to facilitate wireless resource allocation (including interference management and signal processing) to satisfy the quality of service (QoS) requirements of intelligent applications [1]. Traditional convex (CVX) optimization methods are computationally complex on account of their iterative frameworks and instance-by-instance property, which are hard to meet the time-intensive demand. In addition, when it comes to complicated wireless optimization, the mathematical approaches may rely on frequent relaxation and approximation and thus, mismatch the original problem. In the view of that the artificial intelligence (AI) achieves huge success in many application fields, AI empowered wireless networks has the potential to give birth to innovative and novel wireless optimization technique[2, 3]. Recently, some existing works have attempted to apply DL to solve the resource allocation problem in wireless communication[4]. Although some classic neural networks such as multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) can be trained to realize near-optimal and real-time optimization under learning-to-optimize paradigm, they may suffer from poor generalization performance in dynamical wireless networks. The problem-driven discipline of DL motivates the development of specialized neural network architectures for wireless networks. By taking the graphical topology of wireless networks into consideration, the graph neural networks (GNNs) tailored to operate directly on graph-structured data have drawn increasing attention [5, 6].

There are good advantages of GNNs over other neural networks from the perspective of resource allocation in wireless networks. First, the permutation invariance and equivariance of GNNs simplify the input and improve the utilization efficiency of samples such that the training cost is reduced [7]. Second, the node/edge-level readout operation and weight sharing of the GNNs enhance the generalization capability over the number of nodes/edges and the application to unseen problem sizes. Besides, the node/edge-level readout operation is the key factor to facilitate the parallel execution and the distributed implementation. Third, the GNN provides a paradigm to explore the underlying relationships by the interaction among elements in graphs which provides more degrees of freedom to utilize the prior knowledge of theoretical results in optimization of wireless networks. Forth, by regarding GNN as a feature extracting module, optimizing different system utilities can reuse the same extracting module under the unsupervised learning framework. Nevertheless, most existing works on GNN-enabled wireless optimization employs the vanilla GNN in wireless networks or problems. Some limitations restrict the development of GNN-based models in more complicated scenarios. Therefore, an in-depth study on GNNs for wireless networks is required.

There are some existing overviews on GNNs which focus on the application of the GNN in wireless networks[8, 9], while this article pays more attention to improve the capability of GNN-based models for optimizing wireless networks. To the best of our knowledge, little effort has been made on graph representation, model architectures and evaluations. The three aspects are fundamental to develop and improve the GNN-based models for wireless networks.

• •

  The graph representation is to organize the wireless network parameters into graph-structured, which has a great impact on the scalability of the GNN-based models. Besides, the definition of node/edge features and the interaction over graph is determined by the graph representation.

• •

  The model architectures play the central role in developing GNN-based models to learn and exploit relational patterns between the different elements within wireless networks. Designing model architecture by taking the task-specific properties into account contributes to not only training efficiency but also expressive performance.

• •

  The evaluation for DL-based methods are different from the theoretical approach, as they cannot guarantee stationary-point solutions. Besides, the comprehensive evaluation is the basis to compare DL models, which helps to promote the DL-enabled resource allocation schemes.

Based on the three aspects, the article intends to give a uniform framework to facilitate the application of GNN in wireless networks as shown in Figure 1. Besides, we list the challenges in the key steps of the framework. Some can be tackled well by this article while some require further investigation. For example, the model capacity of GNNs is required to be improved such that they can be used in various wireless networks. The over-smoothing issue is an intrinsic problem of GNN-based models to stack deep layers [10], while the complicated constraints and complex input and output are the characteristic of wireless resource allocation problem. To adapt to dynamics and scalability of wireless environment is the advantage of GNN-based models, but to quantify this capability is not easy. For large-scale networks, the distributed implementation of the resource allocation scheme is crucial and the over-the-air computing is very suitable to be integrated into the message passing mechanism in GNN to facilitate the distributed implementation. Transfer learning and meta learning can improve the expressive performance as well as reducing training overhead. At last, the high-quality dataset is necessary as the DL-based method is data-driven.

The wireless networks are usually with topological structure, which can be represented by graphs. Formally, a graph denoted by ${\cal G}=({\cal V},{\cal E})$ is defined by a set of notes ${\cal V}$ and a set of edges ${\cal E}$ between the nodes. In general, each node is with a node-level feature while the relationship between nodes is represented by the adjacency matrix and the edge features. Therefore, the graph representation is to model the elements (such as antenna, user, wireless link) in wireless networks as nodes and build the relationship between nodes to be edges. Note that different graph representations result in not only different node/edge features and adjacency matrices but also different interaction between nodes. Therefore, the graph representations of wireless networks play an important role in GNN-enabled wireless optimization.

Based on the types of nodes, we divide the existing graph representations of wireless networks into two categories, i.e., the isomorphic graph representation and the heterogeneous graph representation.

With the graph representation, a GNN-based model¹¹1The resource allocation problem can be tackled by Node-GNN for node-level task and Edge-GNN for edge-level task[13]. As most edge-level tasks can be equivalently transferred into node-level tasks, this article focuses on the architecture design of Node-GNN. is to map the graph-structured network parameters to the desirable resource allocation scheme. The most popular GNN-based model is message passing neural network (MPNN) or graph convolutional network (GCN). However, these vanilla GNNs face difficulties of inflexible relationship between nodes and stacking deep layers. To improve the model capability of GNN-based models, some mechanisms such as attention and residual can be taken into account for designing the model architecture. Besides, different from the applications of DL in other fields, one of the challenges of DL-enabled wireless optimization is that the output of the neural networks are required to satisfy the constraints (such as power budget, QoS requirements, etc). To guarantee feasible output, the specialized activation functions (AFs) or/and loss functions are required.

To describe the detailed processes including massage passing, attention and residual, we refer to an illustrative example as shown in Figure 3 where the input graph of the GNN is the isomorphic graph representation of the MU-MISO network.

The main goal of the DL-based methods for wireless networks is to realize near-optimal and real-time optimization. Therefore, the evaluation of the DL-based methods is of high importance. However, the existing evaluation metrics, such as convergence behavior, optimality and computational complexity, are mainly used to evaluate the CVX-based algorithms and may not be enough to evaluate the DL-based methods. On the other hand, as a widely used methodology, the DL has its specific evaluation metrics, such as ablation experiment and generalization performance. Therefore, we intend to develop some new metrics to comprehensively evaluate the DL-based wireless optimization methods by taking both wireless communications and DL into account.

• 1)

  Optimality: The ratio of the average objective value (with feasible solutions) by the DL-based method to the average optimal objective value (by the CVX-based methods or exhaustive search methods) over the test samples. Actually, the optimality performance is equivalent to the generalization performance when the training set and the test set have the identical setting, and the optimal objective value can be regarded as the label of the test sample. The demand on the feasible solutions is necessary since the infeasible solution shall result in better objective value and induce unfair comparison. As the DL-based methods cannot always yield global optimal solutions, the ratio should be lower than $100\%$. However, for some complicated problems, the optimal objective value may be unavailable for all test samples. In this case, the results by CVX-based methods or existing schemes can be approximately utilized as labels and the ratio may exceed $100\%$.

• 2)

  Feasibility rate: The percentage of the feasible solutions to the considered problem by the DL-based methods over the test samples. The feasibility rate is an important metric especially when the optimization problems involving complicated constraints which are not handled by activation. A DL-based method with high optimality performance but low feasibility rate is unacceptable. Besides, the value of the QoS threshold has a great impact on the feasibility rate during the testing.

• 3)

  Inference efficiency: The average running time required to calculate the feasible solution for the considered problem by the DL-based methods. Generally, the well-trained DL models are with much faster inference speed than the CVX-based methods. Nevertheless, the wireless networks require real-time resource allocation, which makes lightweighting DL models a central consideration. Besides, a trade-off exists between inference efficiency and model capability.

• 4)

  Scalability: Evaluate the optimality performance of the DL-based methods on the test set with different settings from the training set, such as different numbers of users and different values of power budget. The scalability is a key requirement in the dynamic wireless networks which is also the advantage of the GNN-based models.

• 5)

  Training efficiency: The training samples and the epochs required to coverage. Most DL models are data-driven. Although the unsupervised learning alleviates the demand on labels of the training samples, it still requires unlabeled samples to train the models. Via model architecture design and prior knowledge, the training efficiency can be enhanced.

• 6)

  Stability: The percentage of the test samples (with feasible solutions by the DL methods), which suffer from $\leq n\%$ ($n\in(0,100)$) performance loss to the corresponding labels, in all test samples with feasible solutions. The optimality reflects the average performance while the stability reflects the variance performance. A poor stability performance may be caused by unsuitable dataset setting or hyperparameter selection.

An example of the multi-dimension evaluation metrics is illustrated in Fig. 4 to compared different DL models and CVX-based method, from which it is observed that it is hard to find a model that outperforms other models in terms of all metrics. Therefore, the model should be selected based on the requirements of specific application. Namely, the DL-enabled wireless optimization is to develop the “best-effort” DL models.

This section provides numerical results by a case study on MU-MISO networks to verify the effectiveness of the message passing, attention-assisted aggregation and residual-assisted updating as well as validating the scalability of the GNN-based models. Three classic system utility optimization tasks are considered i.e., sum rate maximization (SRM), energy efficiency maximization (EEM) and max-min rate (MMR). Besides, the AF method, PM and LDM for handling sum-power constraint are compared, and the supervised learning and unsupervised learning are also compared.

Figure 5(a) provides an ablation experiment to show the effectiveness of the message passing, attention and residual in an $16$-antenna $8$-user MISO network in terms of the three system utilities. It is observed that the message passing, attention and residual could improve the optimality of the models, especially for MMR, which validates the feature extracting capability of the GNN-based models over wireless networks. Due to involving more computation processes, the message passing, attention and residual also slightly increase the inference time. Furthermore, the ablation experiment on multi-head attention mechanism and the residual are respectively presented in Figure 5(b) and Figure 5(c). It is observed that with increase of number of attention heads, the optimality performance is improved while the performance gain becomes smaller. Besides, the optimality performance is improved with depths with the assistance of residual while degraded with depths without the residual.

The scalability is the key advantage of the GNN-based models over other DL models in the dynamic wireless networks. Figure 5(d) shows the scalability performance to users of the three GNN-based models, i.e., GCN, GAT [17], RGAT[18], for the optimization of considered system utilities. All GNN-based models are trained with $8$ users while tested with $7$ and $9$ users. It is observed that all GNN-based models are able to generalize to unseen problem sizes in the training set. The reason is that the trainable weights of the GNN-based models can be independent of the graph size. Moreover, with sophisticated architecture design, the GAT and RGAT achieve better scalability performance.

Figure 5(e) compares three methods, i.e., AF method, PM and LDM, to make the output of the DL models satisfy the sum-power constraint for beamforming vectors. It is observed that the AF method always guarantees $100\%$ feasibility rate while the PM and LDM may fail to yield feasible solution for some realizations. Besides, in most cases, the increment of the power budget induces optimiality loss as the enlarged feasible set complicates the learning task, and the AF method slightly outperforms the other methods in the term of the optimiality. Nevertheless, the AF method may be hard to implement for complex constraints which however, can be handled by the PM and LDM.

Figure 5(f) shows that the supervised learning and the unsupervised learning achieve very close optimality and scalability performance fot the tested DL models. The reason is that the DL model the utilize gradient decent method (like the CVX-based method) to lower the loss function. To minimize the difference between the objective value and the optimal value (via supervised learning) is equivalent to minimize/maximize the objective function (via unsupervised learning). The unsupervised learning not only simplifies the training set (compared with supervised learning) but also prevent complicated problem reformulation (compared with deep reinforcement learning), which makes the focus of the DL-enabled optimization in wireless networks to be developing more powerful neural networks.

In this article, we provided a comprehensive overview of applying GNNs to optimize wireless networks and discussed its key steps. In particular, the graph representation of the wireless networks, the principal and design frameworks, as well as evaluation metrics are detailed presented. Numerical results demonstrated the advantages of the GNN-based models in comparison with CVX optimization method and other DL models.

Though GNN-based wireless optimization showed significant performance advantages in providing near-optimal performance, scalability to network typologies and real-time inference, there are still some key issues which need to be addressed as described in the following.

Graph representation of complicated wireless networks: Although there is a trend on applying the GNN to wireless networks, most of existing works consider the classic application scenarios. For multi-hop, multi-service and multi-element networks, the graph representation still remains an open issue. Besides, instead of directly using the system parameters as node/edge features, how to manually design the node/edge features based on the prior knowledge is also an important question to investigate in graph representation.

Efficient approach to handle complicated constraints: Yielding feasible solutions challenges the DL-enabled resource allocation schemes, especially when there are multiple complicated constraints and the output involves complex values, e.g., beamforming vectors and phase shift elements. For GNN-based models, the feasible rate may be degraded when they face unseen problem sizes. One promising way is to reformulate the constrained optimization problem into multi-objective problem and leverage the multi-task learning to enhance the robustness of GNN-based models. Besides, the transfer learning is also a choice to improve the feasible rate at little fine-tuning cost.

Powerful GNN-based structure for ultra-dense networks: Most of existing GNN-based models are designed for the single-cell or D2D scenarios, while these models are hard to extend to the practical ultra-dense networks. Some new mechanisms (like [19, 20]) are required to develop more powerful GNN-based models. Then, the trade-off between inference performance and inference speed should be the central consideration for designing the architectures of GNN-based models. Another promising way is to employ the over-the-air computing framework to enable the distributed implementation of the GNN-based models. Besides, the message passing mechanism requires to re-deigned by taking the signaling overhead exchanging process into account.

Engineering-awareness dataset and evaluation metrics: Different from the CVX-based methods which aims for theoretical results, the DL-based methods are responsible to meet the practical implementation requirements. Training DL models often requires huge amounts of samples with enough diversity to abstract deep insights, which makes the engineering-awareness dataset necessary. Besides, the evaluation metrics in terms of practical implementation play the key role to connect the theoretical research and the practical demands. The standard datasets and evaluation metrics are also helpful to build up the open source DL libraries and platforms for wireless networks.

GNN-enabled deep reinforcement learning: Deep reinforcement learning (DRL) is an important branch of DL, which has been widely adopted to solve the complicated resource allocation problems in RIS- and UAV-assisted networks. Different from unsupervised learning, the DRL enables the online training of the model. Most existing DRL methods train MLPs to generate state value or action. As the GNN-based models have been proved to be with more powerful expressive capability, training GNN via DRL shall be a hot research spot. Note that the inputs, i.e., graph representations, of GNN-based modes trained by the unsupervised learning and the DRL may be different even for the same problem. Besides, the comparison between DRL-based GNN and unsupervised learning based GNN remains an open issue.