Deep Learning based Radio Resource Management in NOMA Networks: User Association, Subchannel and Power Allocation

In this paper, a NOMA based mmWave heterogeneous wireless networks with an intelligent control center (Fig. 1) is studied, where small cells are distributed uniformly in a macrocell. We concentrate on issues of downlink user association, subchannel allocation and power control. Denote $\mathcal{B}=\left\{{1,2,...,b,...,B}\right\}$ as the set of all BSs including macro BS and small BSs, $\mathcal{N}=\left\{{1,2,...,N}\right\}$ is the set of all subchannel, $\mathcal{M}=\left\{{1,2,...,m,...,M}\right\}$ is the set of users, and $X=\left\{{{x_{1,1}},{x_{1,2}},\cdots,{x_{b,m}}}\right\}$ is used to represent the user association strategy between BS $b$ and user $m$. If user $m$ is assigned to the BS $b$, then, ${x_{b,m}}=1$. Otherwise, ${x_{b,m}}=0$.

It is assumed that a subchannel in NOMA can be utilized by up to two users through successive interference cancelation (SIC). The set $S=\left\{{{s_{1,1}^{1}},{s_{1,2}^{1}},\cdots,{s_{b,m}^{n}}}\right\}$ stands for the subchannel allocation strategy between user $m$ and subchannel $n$ of BS $b$. If the subchannel $n$ of BS $b$ is occupied by the user $m$, then, ${s_{b,m}^{n}}=1$. Otherwise, ${s_{b,m}^{n}}=0$.

The signal to interference plus noise ratio (SINR) of user $m$ on subchannel $n$ of BS $b$ can be given as

where ${p_{b,m}^{n}}$ is the power of user $m$ on subchannel $n$ of BS $b$, and $g_{b,m}^{n}$ is the gain of user $m$ on subchannel $n$ of BS $b$ and ${\sigma^{2}}$ is the variance of AWGN. The capacity of user $m$ on subchannel $n$ of BS $b$ can be expressed as

where $W$ is the system bandwidth.

The total rate of the network is

and the total transmit power is

where ${p_{c,b}}$ denotes the circuit power of BS $b$.

The total EE of the system can be formulated as

It’s assumed that each user can be serviced only by one BS at time. Thus,

This work also assumes that at most two users can occupy one subchannel of each BS. Therefore hold that

with the constraint:

where $K_{b}$ is the the maximum number of users assigned to BS $b$. The total power constraint is:

where ${p_{\max}}$ is the maximum transmit power of BS $b$, and the QoS constraint is:

which ensures that each user has the minimum QoS requirement of ${R_{t}}$. Finally, the cross-tier interference constraint can be written as:

where ${I_{b}}$ denotes the maximum interference constraint.

Aiming to achieve load-balance and maximum EE, the problem of user association, subchannel and power optimization can be formulated as:

where C1 represents the user scheduling constraint, C2 is the constraint on maximum users of the each subchannel, C3 is the constraint on the maximum associated users of each BS, C4 is the maximum power constraint on each BS, C5 is the QoS constraint on each user and C6 is the cross-tier interference constraint on each BS.

In this subsection, an user association scheme based on the Lagrange dual decomposition method is proposed.

The Lagrangian dual function for user association can be written as

where $\mu{\rm{,}}\lambda{\rm{,}}\nu{\rm{,}}\tau$ are the Lagrange multipliers to decouple constraint. Then the original problem can be divided into two sub-problems by dual method,

For the sub-problem of user association the partial derivative of ${f_{X,S,P}}\left({\mu,\lambda,\nu,\tau}\right)$ is given by (16) at the top of next page.

The decision reference value of user association of iteration $t$ is given by

Then, the user association strategy can be determined as:

In this subsection, a novel co-training semi-supervised learning method for subchannel allocation is proposed. In order to solve the channel allocation problem, we transform problem (12) into a loss function in the deep learning model, and get the optimal power allocation by minimizing the loss function. The loss function can be written as

where ${\widehat{S}}$ means the predicted subchannel allocation strategy.

In this paper, the subchannel allocation scheme is initially generated by two-sided matching scheme as shown in Algorithm 1. Users and subchannels are regarded as two players pursuing the maximization of their respective utility.

The proposed deep learning scheme for subchannel allocation is based on semi-supervised learning regression algorithm. In order to improve the accuracy of DNN, semi-supervised learning is used to improve the performance of NOMA heterogeneous network. Given a labeled data set ${L_{S}}=\left\{{\left({{G_{1}},{S_{1}}}\right),\left({{G_{2}},{S_{2}}}\right),\cdots,\left({{G_{\left|L\right|}},{S_{\left|L\right|}}}\right)}\right\}$ generated by Algorithm 1 and an unlabeled data set $U=\left\{{G_{1}^{{}^{\prime}},G_{2}^{{}^{\prime}},\cdots,G_{\left|U\right|}^{{}^{\prime}}}\right\}$, where ${G_{i}}$ is matrix representing the i-th group of gains, which contains all $g_{b,m}^{n}$ generated by the i-th initialization, ${S_{i}}$ is is the i-th group of channel allocation matrix, which contains all $s_{b,m}^{n}$, ${\left|L\right|}$ and ${\left|U\right|}$ are the number of data samples $L_{S}$ and $U$, respectively. We aim to get a learner ${H_{S}}:G\to S$ that can accurately predict the real label of an unlabeled input. The input of the learner is $G$, and the output of the learner is $S$.

D. J. Miller and H. S. Uyar analyzed the feasibility and rationality of using unlabeled data to improve learners performance based on data distribution estimation theory [31]. Assuming that all data follow a mixture of $L$ Gaussian distributions, then the distribution is

where $\sum\limits_{i=1}^{L}{{\alpha_{i}}}$ is a mixing coefficient and $\theta=\left\{{{\theta_{i}}}\right\}$ is the parameter. The random variable is a label that is determined by the selected mixture component $m_{i}$ and feature vector $g_{i}$ in the probability of $P\left({\left.{{c_{i}}}\right|{g_{i}},{m_{i}}}\right)$. According to the maximum posteriori probability hypothesis, the optimal classification formula can be written as

where $P\left({\left.{{m_{i}}=j}\right|{g_{i}}}\right)=\dfrac{{{\alpha_{j}}{H_{S}}\left({\left.{{g_{i}}}\right|{\theta_{j}}}\right)}}{{\sum\limits_{l=1}^{L}{{\alpha_{i}}}{H_{S}}\left({\left.{{g_{i}}}\right|{\theta_{l}}}\right)}}.$

According to the description above, the purpose of semi-supervised learning can be interpreted as using training data to estimate ${P\left({\left.{{c_{i}}=k}\right|{g_{i}},{m_{i=j}}}\right)}$ and $P\left({\left.{{m_{i}}=j}\right|{g_{i}}}\right)$. Obviously, the former is related to labels while the latter is not related to labels. Therefore, if there is massive unlabeled data, the generalization ability of learners is improved. The analysis results of T. Zhang and F. J. Oles [32] showed that if a parameterized model can be decomposed into the form of $P\left({\left.{x,y}\right|\theta}\right)=P\left({\left.y\right|x,\theta}\right)P\left({\left.x\right|\theta}\right)$, the simulated model parameters can be better supported by unlabeled data, thus improving the performance of the learning model.

Co-training semi-supervised learning algorithm is the most important semi-supervised learning method [30]. It initializes more than two learners based on clustering hypothesis or popular hypothesis. In the process of learning, the labeled data with the highest confidence after labeling is selected according to specific criteria. After labeling, the labeled data are put into each other’s labeled data set of the other’s learner to update the model. Co-training method is also based on the compatibility and complementarity of multi-view data [33]. Assuming that the data has two sufficient redundant and conditionally independent views. Sufficient means that each view contains enough information to produce the optimal learner, conditionally independent is that in given labels, each view is independent.

The co-training semi-supervised deep learning based subchannel allocation scheme is shown in Algorithm 2. The key of the co-training is to define the rule for selecting high confidence labeled data. To solve this problem, the criterion defined by Algorithm 2 is that the unlabeled data with the most consistent labeled training set is the data with the highest confidence. The rules for calculating confidence are as follows:

where $H$ represents learning model, ${g_{u}}\in U$, $H^{{}^{\prime}}$ represents the retrained learning model using the original labeled data set adding labeled data generated by model $H$.

According to Zhou’s research results [33], in order to start a co-training process with good effects, the two initialized learners must have relatively large differences. In extreme cases, if two learners are very close, the added labeled data will be very similar, and eventually it will evolve into a single learner, that is, self-training algorithm. When designing learners of Algorithm 2, we ensure the largest difference between learner $H_{1}^{S}$ and learner $H_{2}^{S}$. The differences of the models are mainly obtained by using different hidden layers, different network parameters or different number of nodes on the hidden layer.

Two network models are designed as 3 hidden layer network and 4 hidden layer network, the dropout coefficients are 0.8 and 1, batch-size are 200 and 500, respectively. We run the initial learners with different learning rates and choose the better learning rates of 0.01 and 0.05 respectively, as shown in Fig. 2. The Algorithm 2 runs under different number of hidden layer neurons from 600 to 1000 and from 60 to 10 in order to choose the optimal parameters, and the accuracy of the learner is evaluated as shown in the Fig. 3. By comparing different accuracy rates, the neurons number of hidden layer in the two networks is $n_{1}=700$ and $n_{2}=80$, respectively.

In this subsection, a power optimization scheme based on DNN is proposed to maximize the EE. DNN scheme has been proved to have high approximation rate and computing performance for power control problem in [18]. In this scheme, the complex iteration process is replaced by the non-linear mapping. Its output operation is realized only by simple matrix multiplication and addition, which guarantees the real-time performance of the network. DNN is a multi-layer network model, which works mainly through the connection between neurons. Specifically, denote $\xi$ as the activation function, denote $n_{j}$ as the number of neural of layer $j$, denote $w_{i}^{j}$ as the weight coefficient and $b^{j}$ as the bias of layer $j$. The output of the ${i^{{\rm{th}}}}$ neural of layer $j$ is given by

Consider the loss function of power optimization problem, problem (12) can be rewritten as

where ${\widehat{P}}$ shows predicted matrix of power optimization. We translate the maximization problem of non-deterministic polynomials into the problem of minimizing the error between the predicted value and the sample value, which is also directly expressed as the problem of the mean square error between the predicted value and the sample that represents the accuracy of the network model.

An iterative gradient algorithm is used to generate training data. Let ${p_{m,b}}=\sum\limits_{n=1}^{N}{s_{b,m}^{n}p_{b,m}^{n}}$, and the power optimization function is given by:

In this case, the power ${p_{b,m}}$ is updated shown below:

where ${\delta}\left(t\right)$ is the step and $\Delta{p_{b,m}}$ is shown as

In training stage of DNN based power optimization scheme, we update the weight $w$ and bias $b$ to minimize lost function. The process of DNN based power optimization scheme is shown as Algorithm 3. The rectified linear unit (RELU) is used as the activation function and the optimization algorithm we use is the Adam algorithm. The related neural network structure and simulation environment are discussed in the next section.

In this subsection, the asymptotic time complexity of proposed radio resource management is discussed.

• •

  The complexity of user association algorithm.

The proposed scheme is based on the Lagrange dual decomposition method. The complexity performance of the Lagrange dual decomposition method for user association is discussed in [2]. Thus, user association needs $O\left(MB+3B+M\right)$ operations at each iteration.

• •

  The complexity of subchannel allocation algorithm.

The proposed deep learning scheme for subchannel allocation is based on semi-supervised learning regression algorithm, which contains two learner $H_{1}^{S}$ and $H_{2}^{S}$ of different number of hidden layers and nodes per layer. The time complexity of DNN can be represented by floating-point operations (FLOPs). For each layer of neural network, the number of FLOPs can be expressed as

where $I_{i}$ is the input dimension of the $i$th layer and $O_{i}$ is the output dimension of the $i$th layer. Therefore, for our semi-supervised learning scheme, the number of FLOPs is:

While the time complexity of two-side matching algorithm would be $O(BN!{2^{M/B}})$ discussed in [5]. So the complexity comparison between learning-based subchannel allocation and matching subchannel allocation is as follows:

Therefore, the proposed learning model for subchannel allocation has a lower complexity than matching algorithm.

• •

  The complexity of power allocation algorithm.

For our learning scheme for power allocation, the number of FLOPs is:

The complexity performance of the iterative gradient algorithm for power allocation is discussed in [2]. Thus, data generation of Algorithm 4 needs $O\left(MB+B\right)$ operations at each iteration. We suppose that the iterative gradient algorithm requires $t_{p}$ iterations to converge.

So the complexity comparison between learning-based power allocation and iterative gradient algorithm is as follows:

Therefore, the proposed learning model for power allocation has a lower complexity than the iterative gradient algorithm.