Stackelberg Game Based Power Control with Outage Probability Constraints for Cognitive Radio Networks

This paper firstly investigates the problem of uplink power control in cognitive radio networks (CRNs) with multiple primary users (PUs) and multiple second users (SUs) considering channel outage constraints and interference power constraints, where PUs and SUs compete with each other to maximize their utilities. We formulate a Stackelberg game to model this hierarchical competition, where PUs and SUs are considered to be leaders and followers, respectively. We theoretically prove the existence and uniqueness of robust Stackelberg equilibrium for the noncooperative approach. Then, we apply the Lagrange dual decomposition method to solve this problem, and an efficient iterative algorithm is proposed to search the Stackelberg equilibrium. Simulation results show that the proposed algorithm improves the performance compared with those proportionate game schemes.


Introduction
The high energy consumption and exponential growth in wireless communication networks face serious challenges to the design of more energy efficiency and spectrum efficiency green communications that should deal with the scarcity of radio resources.A promising approaches technique called cognitive radio networks (CRNs) is proposed as a key design to improve spectral efficiency of the LTE-advanced standard.Game theory is an effective tool for resource allocation in multiuser communication systems, which has been applied to CRNs, and [1] summarized the advance of game theory for CRNs.Recently, Stackelberg game has been formulated for resource allocation which addresses the relationship between PUs and SUs in [2][3][4][5][6][7][8][9].In these two-tier femtocell networks, the PUs were served by the mobile operators, whereas the SUs are deployed by indoor users for their own interests.Their different service requirements and design objectives motivate us to adopt the framework of hierarchical game to the power allocation problem with the leader-follower structure, which is actually a multiple-leader multiple-follower Stackelberg game.Specifically, the PUs compete with each other in a noncooperative manner to maximize their utilities, all the time anticipating the response of the followers.This subgame is referred to as the upper subgame, while after the PUs apply their strategies, the SUs update their power allocation strategies in response to the PUs' strategies.The SUs also compete with each other in a noncooperative manner to maximize their own utility function.Moreover, this subgame is referred to as the lower subgame.
Reference [2] presented a joint pricing and power allocation scheme for CRNs with Stackelberg game; PUs and SUs can benefit from the channel sharing model by achieving the Stackelberg equilibrium (SE).In [3,4], a Stackelberg game is used to study the joint utility maximization of the PUs and the SUs with a maximum tolerable interference power constraint (IPC) at the primary base station (PBS).However, the algorithm is suboptimal because it does not consider how to control the IPC among PBS.Therefore, [5] proposed an optimal price-based power algorithm for the PBS and SUs maximize their revenues by Stackelberg game with IPC.To maximize users' energy efficiency of CRNs, a new green power control scheme was studied in [6], where PUs and SUs aim at maximizing their energy efficiency with  Stackelberg game.In [7], the authors formulated a Stackelberg game model to maximize the payoff of both SUs and PUs by jointly optimizing transmission powers of SUs and subband allocations of SUs.Moreover, in [8], the authors analyzed the cooperation interaction between the PUs and SUs transmitters from a Stackelberg game theoretic perspective with secrecy constraints.Moreover, [9] proposed a game theoretic framework to model the interactions among multiple PUs and multiple SUs under different system parameter settings and under system perturbation.However, the above papers aim to maximize users' revenue without considering the outage probability of multi-PUs and multi-SUs.It is clear that both PUs and SUs need to update their transmission power frequently to maintain their Signal to Interference plus Noise Ratio (SINR) level (usually referred to as QoS) due to channel fading and interferences from other users when they transmit on the same channel.Therefore, in CRNs, the fundamental performance traits of multiple SUs power control should be investigated with multiple PUs; also the issues of multiple PUs power control should be studied in the presence of multiple SUs in fading channels and interferences.In the game problem formulation, the outage probabilities of both PUs and SUs represent their own utility below the required target SINR used to guarantee their desired QoS.Hence, the channel outage probabilities with respect to both PUs and SUs transmissions should be required in the multiple users' game framework, if users' SINR falls below a certain threshold.
In this paper, we proposed a new Stackelberg game to model the hierarchical competition between PUs and SUs in CRNs with global interference and outage constraints.We theoretically prove the existence and uniqueness of robust Stackelberg equilibrium for the noncooperative approach.Then, we employ the Lagrange dual decomposition method to solve the game problem by decomposing the game problem into independent suboptimal solution.Furthermore, we develop an efficient iterative algorithm to converge the SE.

System Model
We consider a CRN composed of a single PBS with a set of PUs K fl {1, . . ., } having priority to use a set of channels N fl {1, . . ., } and a secondary base station (SBS) with a set of SUs L fl {1, . . ., } allowed to share  channels from the PBS, shown in Figure 1.We define the channel gains of the PU  and the SU  on the channel  as ℎ   and ℎ   , respectively.The channel gain between the th PU transmitter and the SU  receiver is denoted by ℎ   ; the channel gain from the th SU transmitter to the th PU receiver is ℎ   .The transmit power of the th PU and the th SU on the channel  is denoted by    and    .We assume each user can transmit simultaneously over multiple channels, and one channel can be served only by one PU but can be used by multiple SUs.
The SINR at the th PU receiver on the th channel can be defined as where if the PU  and the SU  transmit on the same channel , For each PU , taking (3) into (4), we can get After rewriting (6), we can obtain ) , ∀, ∀.(7) International Journal of Distributed Sensor Networks 3 Similarly, taking (3) into (5), rewrite (5) as follows: In addition, the global aggregate interference from all SUs to each channel should not be larger than the maximum interference threshold    to ensure the SUs' transmission would not cause unendurable interference on every channel of each PU.Mathematically, this can be written as

Stackelberg Game Theoretic Approach
The PUs (leaders) price the SUs (followers) to control the interference power made by the SUs under the IPC.Each PU will offer a suitable price to maximize its revenue by selling resource to SUs.Based on the interference price provided by PUs, each SU will adjust its transmission power to maximize its revenue.PUs have higher priority than SUs, we use Stackelberg game to model the strategy between the PUs and SUs.Then, as the leaders, PUs will maximize their utility (SINR performance plus the payment from the SUs occupying the channels).The utility function of the PU  is as follows: where if the PU  transmit on the channel ,    = 1; otherwise,    = 0. p  = ( 1  ,  2  , . . .,    ) is the transmission power vector over the transmitted channels of the PU ; p − represent the transmit power vector of all users except the PU .  denotes that the PU  charges the price over all SUs if the SUs transmit on the channel of the PU.
Hence, the revenue utility optimization problem for the PU  is as follows: where  max  is the maximum transmission power of the PU .Then, for SUs (followers), we set the revenue utility of the th SU with two parts: the first one is the income from the SINR achieved from the PUs.The second one is the payment for the PUs.Then, the revenue utility function of the th SU is as follows: where if the SU  transmit on the channel ,    = 1; otherwise,    = 0. p  = ( 1  ,  2  , . . .,    ) is the transmission power vector over the transmitted channels of the SU ; p − represent the transmission power vector of all users except the SU .w  = ( 1 ,  2 , . . .,   ) denotes that the SU  pays the price vector for all PUs; if the SU  does not transmit on the channel of the PU ,   = 0.
Hence, the optimization problem for the SU  is as follows: where  max  is the maximum transmission power of the th SU.

Solution of the Proposed Stackelberg Game
The optimization problems P1 and P2 taken together are the proposed Stackelberg game problem with several constraints.The goal of the proposed Stackelberg game is to achieve the SE, in which point both PUs and SUs have no incentive to deviate [3], so, the distributive algorithms convergence to the SE is difficult.Thus, we depart the game problem into suboptimal independent solution and employ an iterative algorithm to achieve the SE.

Global Efficiency of the RSE.
In noncooperative games, the existence and uniqueness of equilibrium are not always achieved [11] due to multiple players' competition.Hence, for the multifollower subgame, we should study the existence and uniqueness of the global SG response to the leaders' price.In particular, a variational equilibrium (VE) [12] is applied to analyze the global SG for our case.This is because a VE is more stable than any other generalized Nash equilibrium under parameter uncertainty [13].Particularly, a number of SUs aim to achieve their QoS requirement through applying the resource from PUs in cognitive radio networks; VE is regarded as an appropriate solution.
For a market fixed price at the PUs, all the SUs aim to maximize their own utility by buying the resource through PUs.Thus, we formulate a new objective function for all SUs; the new utility is the sum utilities of all SUs and can be expressed as follows: Therefore, in order to achieve the actable outcome of the proposed SG, our goal is guarantying the existence and uniqueness of the SG when maximizing (14).The corresponding Lagrangian function and the KTT conditions for the SU  expressed in and the KTT conditions for the SU  are given by We note that the robust followers' game shows a jointly convex generalized SE problem; therefore, the solution of the SE problem with constraints in ( 13) is a variational inequality VI(P, F), where P is the set of joint convexity.It is important to determine a vector  * ∈ P ⊂   , such that ⟨F(z * ), z − z * ⟩ ≥ 0, for all  ∈ P and F(p) = −(∇  Ũ (p  ))  =1 [13].Then, the solution of VI(P, F) is a variational SE.
In this paper, we only focus on the power control in cognitive radio networks by assuming the channel assignment has already been done.Then, we can divide the variational inequality VI(P, F) into  subproblems; each subproblem denotes VI(P  , F  ) on the subchannel  and they are independent.Therefore, on the subchannel , the KKT conditions can be expressed as [12] Now from the definition of [12], we have . . .
Therefore, the Jacobian of F  is . . .
Each F  J  is a diagonal matrix and all the diagonal elements are positive.Therefore, F  J  is positive definition on P  , and so F  is strictly monotone.Hence, the global SG problem admits a unique global variational equilibrium solution [12].Due to the jointly convex nature of the global SE problem, the variational equilibrium is the unique global maximizer of (14) [12], which completes the proof in the literature [12].

Solution of the Optimization for SUs (Followers).
For the P2, the utility function of each SU is a concave function of    and the constraints are all linear, so a partial Lagrange dual decomposition method (LDDM) [4] for the problems is used.
For the problem in ( 13), the corresponding Lagrangian function for the SU  on the subchannel  can be expressed as where    ,   , and    are the nonnegative dual variables of the constraints in (13).
We decompose the optimization problem into  independent subproblems.Then, on the subchannel , taking the Karush-Kuhn-Tucker (KKT) condition [4], Simply, in (20), we assume that the th user causing the interference power    ℎ   to the SU  on the subchannel  can be denoted as the average interference power except the SU : where    = (  +    )ℎ   +   − ℎ   /(p  − ).The transmission power of the SU  is zero if the interference price for it is larger than payoff threshold    on the channel .Then, setting    * = 0, we can get the payoff threshold of the SU  if it transmits on the channel : ) . ( From ( 23), if the price   >    , the price is above the payoff threshold of the SU , and it will stop transmitting on the channel  without buying the interference power.

Solution of the Optimization for PUs (Leaders).
In order to maximize its own utility, each PU needs to adaptively offer an interference price to SUs based on transmit power response of the SUs.P1 can be decomposed into two subproblems: fix   to get the optimal transmission power of each PU , and then search the optimal   .The optimal transmission power of the PU  can be applied by the previous LDDM.
Thus, for P1 in (11), the corresponding Lagrangian function can be given as where    ,   , and ]   are the dual variables of the constraints in (11).
Similarly, in (24), we assume the th SU causing the interference ℎ      to the PU  on the channel  can be denoted as the average interference power: According to the KKT conditions, we obtain the optimal transmission power of the PU  if it transmits on the channel : Since L  (p  ,   ,   ,   , ^) is a stepwise function with breakpoints at    for the SU , we should discuss the existence of the optimal price   first.So we divide (24) with respect to   with two parts on each channel ; we have L , (   ) =    (   ) and L , (  ) = (  − ]  )   ℎ  .From ( 22), it can be easily observed that    (   ) is a concave function of   .Therefore, we only need to discuss the situation of L , (  ).For the SU , we first sort    ( = 1, . . ., ) in ascending order and have  intervals Note, if the SU  is not allocated on the channel , ( −1  ,    ) must be taken out of the order.We take (0,  1  ) for an example.When   → 0, we can derive that L , (  ) Taking the second derivative of L , (  ) with respect to   is International Journal of Distributed Sensor Networks (01) Initialization: set  = 0, set initial   () and p  () for  ∈ .Set initial p  () for  ∈ .
Set , where  is positive and sufficiently small.(02) For each SU  (03) Use SM to find the optimal step sizes  * ,  * and  * , and update    ,   and   according to (28), respectively.(04) For the given   () and p  () of all PUs, each SU  responds with its transmit power vector p *  ( + 1) according to (22).(05) If    <   (), the SU  stops transmitting on the channel  of the PU .(06) End (07) For each PU  (08) Use SM to find the optimal  * and  * , and update    and   by (29).(09) For the responded p *  () of all SUs, each PU  updates its transmit power vector as p *  ( + 1), according to (25).( 10 L , (  ) is a concave function whether < 0 except at the nondifferentiable point  1  .Through the above analysis, L  (p  ,   ,   ,   , ^) is a concave function with respect to   except at    .The ellipsoid method [4] can be employed to solve the convex optimization in each interval.

Iterative Algorithm to Find the SE.
For the above discussion, we propose an iterative algorithm to search the SE.Due to the fact that computation of the dual variables is a complicated task, the subgradient method (SM) [14] where  is the iteration index and  > 0,  > 0,  > 0,  > 0, and  > 0 are sufficiently small.The SM guarantees the convergence of the above optimal dual variables if the step sizes are chosen by following the step size policy [14].
We then design the iterative algorithm to achieve the SE shown in Algorithm 1.

Simulation Results and Their Analysis
In this section, several numerical examples are presented to evaluate the performances of the proposed SG by comparing the optimal price-based SG considering global interference in [5] and the nominal SG without considering global interference and outage constraints.The cell radius of 500 m with the PBS centered at the original CRN.The simulation parameters are as follows:  2 = 10 −12 W,  = 3,  = 5, and  = 10.The SINR threshold of PUs and SUs is set as 7 dB and 4 dB, respectively.All PUs and SUs deploy the maximum power:  max  = 100 mW and  max  = 50 mW.The channel gain in this system is ℎ  =  −4   , with   being their corresponding distance.The outage probability thresholds of both PUs and SUs are 0.001.We set the interference-to-noise ratio (INR) as / 2 .
Firstly, we illustrate the convergence of the proposed algorithm for achieving an SE of the proposed game.From Figure 2, the three PUs and five SUs iteratively update their utilities and obviously converge to the SE.The proposed algorithm converges quickly in terms of PUs, only about ten times.In addition, due to the larger number of SUs, the convergence of the SUs is slower than that of the PUs.

Impact of INR.
In this subsection, we set the number of PUs and SUs as 3 and 5, and INR changes from −20 dB to 20 dB which means that IPC changes from 10 −14 W to 10 −10 W.
We then consider the sum rate of PUs and SUs for the three solutions with different tolerant interference constraints shown in Figures 3 and 4. For the performance of sum rate of SUs, Figure 3 shows the nominal SG scheme outperforms other two schemes when the interference temperature level is stringent but is inferior to the two schemes when it is loose.The proposed SG performs the worst, because of the demand of satisfying the outage probability constraint.
Once the interference constraints are loose enough to be not active, accordingly, our proposed solution works better than the others.For the leaders, the sum rate of the nominal SG performs the worst because of not including the global interference constraints.This is because the performance of PUs may be degraded with the increases of the interference.Figure 5 presents the outage probability of the system with different INR.It is observed that the proposed scheme achieves much lower outage probability than other schemes; in particular, the performance gap becomes larger with the increase of INR.This is because our proposed algorithm works best by considering the outage probability constraints of users, which prevents the outage events well.

Impact of Different Number of SUs.
In this subsection, the INR is set to be 10 dB.The number of SUs changes from 2 to 20.All the other simulation parameters are the same as the beginning part of this section.
For the PUs, from Figure 6, the sum rate of the SG solution performs the worst because of not including the global interference constraints.So the PUs may refuse to sell more spectrum resource because of protecting their own communication QoS.The proposed scheme outperforms most in terms of sum rate of SUs because the algorithm allows more SUs to share their radio resource so that it increases PUs' utilities with considering the channel uncertainty and global interference constraints.
Figure 7 shows the sum rate of SUs versus the number of SUs.The sum rate of SUs performance of our proposed algorithm works better than other two schemes; this is because the proposed scheme is able to support more SUs at the BS shown in Figure 7, so that more SUs have opportunities  to transmit, which increase the sum rate.In addition, the performance gap between the proposed scheme and the other algorithms increases when the network grows larger, by which it can be concluded that the proposed scheme is more suitable for application in larger networks.Because the scheme sets the adaptive punishment parameter among all served SUs to control their behavior, more SUs can be served at the BS, thus achieving a higher sum rate.
Figure 8 shows the number of outage probability comparison versus the number of SUs for different algorithms.The proposed algorithm is able to support more SUs than  other schemes.This is because we develop the channel assignment scheduling scheme to decrease the probability of the unserved SUs, so more SUs are admitted to serve at the BS without causing unendurable interference to PUs.In addition, the interference among SUs is taken into the revenue utility function to void serious interference for some SUs who have bad channel condition.

Conclusion
In this paper, we propose a Stackelberg game for power control problem in CRNs with channel outage constraints and global interference constraints.We employ LDDM to solve the problem by decomposing it into independent subproblems and develop an iterative algorithm to achieve SE.Simulation results show that the proposed algorithm improves the performance compared with other game algorithms.

Figure 2 :
Figure 2: The convergence of utility of PUs (leaders), setting price of PUs, and utility of SUs (followers).

Figure 6 :
Figure 6: Sum rate of PUs versus number of SUs.

Figure 7 :Figure 8 :
Figure 7: Sum rate of SUs versus number of SUs.
is applied to obtain the global optimum SE of this problem.Then, dual variables are updated as follows: