Dynamic incentive mechanism in mobile crowdsourcing networks by combining reputation and contract theory

By utilizing the mobile terminals’ sensing and computing capabilities, mobile crowdsourcing network is considered to be a promising technology to support the various large-scale sensing applications. However, considering the limited resources and security issue, mobile users may be unwilling to participate in crowdsourcing without any incentive. In this work, by combining reputation and contract theory, a dynamic long-term incentive mechanism is proposed to attract the mobile users to participate in mobile crowdsourcing networks. A two-period dynamic contract is first investigated to deal with the asymmetric information problem in the crowdsourcing tasks. Reputation strategy is then introduced to further attract the mobile users to complete the long-term crowdsourcing tasks. The optimal contracts are designed to obtain the maximum expected utility of service provider with reputation strategy and without reputation strategy, respectively. Simulation results demonstrate that the long-term crowdsourcing tasks can be guaranteed by combining the contract’s explicit incentive with the reputation’s implicit incentive. The incentive mechanism can gain a higher expected utility, the more implicit reputation effect factor.


Introduction
Nowadays, with the popularity of smartphones and wearable sensing devices, most mobile devices are equipped with a wide range of processors, sensors, and enormous memories. 1 These devices can be applied to gather various data about human society, surroundings, and individuals. Then, the various mobile crowdsourcing applications appear around the world, such as CrowdTracker 2 for object tracking and PERIO 3 for industrial Internet of Things.
In mobile crowdsourcing networks (MCNs), 4 the service provider (SP) always recruits the mobile users (MUs) to participate in the crowdsourcing tasks. Most of the MUs provide crowdsourcing services based on voluntary participation. When completing 1 crowdsourcing tasks, each MU may consume its resource (i.e. memory, battery, and time). 5,6 Certain private information (i.e. location information) may also be contained in the collected crowdsourcing data, leading to the potential privacy threats to MUs. [7][8][9] Considering the limited resources and security issue, the MUs may be reluctant to offer crowdsourcing services or have privacy concerns. 1 Thus, the effective incentive mechanisms are essential to accomplish the mutual benefit in the long-term corporation.
However, it is challenging to design an effective incentive mechanism in MCNs. Due to MUs' mobility and crowdsourcing environment's dynamic characteristics, the SP may not be able to obtain the MUs' exact crowdsourcing efforts, which leads to the asymmetric information problem between the SP and the MUs. 10 In this case, even if the MUs are willing to offer their help, the MUs may deviate from the crowdsourcing tasks or provide poor performance in tasks. Fortunately, contract theory 11 was first proposed to deal with asymmetric information in the economic issues. Thus, we try to investigate a contract-based incentive mechanism to address these challenging issues under the asymmetric information scenario. Moreover, in order to motivate the MUs to participate in the longterm crowdsourcing tasks more effectively, the reputation strategy is introduced to provide the implicit incentive by combining with the explicit contract incentive.
Motivated by the above issues, this work proposes a dynamic incentive mechanism to create the mutual benefits in the long-term crowdsourcing tasks. Our contributions are summarized as follows: New solution technique: we investigate a twoperiod dynamic contract under the asymmetric information scenario. A parameter named incentive coefficient is introduced to attract the MUs to offer the effective crowdsourcing efforts. The MUs' basic salary paid by the SP is determined by the crowdsourcing efforts. Moreover, by introducing the implicit reputation parameter, reputation strategy is designed to motivate the MUs to provide the long-term crowdsourcing efforts effectively. As far as we know, the dynamic incentive mechanism by combining reputation and contract theory has not been investigated for MCNs. Optimal incentive mechanism design: the optimization problem is formulated to achieve the maximum expected utility of the SP subject to the constraints of individual rationality (IR) and incentive compatibility (IC). The necessary and sufficient conditions of the optimization problem are systematically presented. The optimal contract designs are achieved to maximize the SP's expected utility for both non-reputation and reputation strategies. The performance of the optimal reputation-based dynamic contract incentive mechanism is demonstrated through simulations.
The key notations summarized in this article are shown in Table 1. The remaining of this article is organized as follows. The system model and problem formulation are introduced in section ''System model and problem formulation.'' Then, the detailed design for the optimal two-period dynamic contract incentive methods is derived in section ''Two-period contract-based incentive mechanism without reputation strategy.'' In section ''Two-period contract-based incentive mechanism with reputation strategy,'' we present the optimal two-period dynamic contract with the reputation strategy. Section ''Results and discussion'' presents the experimental results. The final section summarizes this article.

Related works
Incentive mechanisms for MCNs were classified into three categories, which are service-based, monetarybased, and entertainment-based mechanisms. 12 A monetary-based multi-round incentive mechanism was investigated by alternating between task information diffusion and task allocation operations. 13 Jiang et al. 14 developed a crowdsourcing incentive mechanism for  truth discovery of textual answers with copiers. Among these works, auction 15,16 or other game-theoretic methods 17,18 have gained most attention to cope with workers' strategic behaviors. However, there are always much more signaling overheads in most game-based mechanisms. Long-term incentive problems under the asymmetric information scenario have not been taken into consideration by most of these research works.
Recently, contract theory has been applied to solve the incentive problems in many other applications, for instance, cooperative relay, 19-21 cognitive radio, 22,23 mobile crowdsourcing, 24,25 computing, 26,27 and federated learning. 28,29 In mobile crowdsourcing scenario, most works designed a one-shot static contract with the static relationship between the SP and MUs. Practically, contracts between the two parties may repeat over time. The SP can continue to recruit the MUs in the next crowdsourcing tasks. In our prior work, 30 a dynamic contract mechanism was introduced into the long-term relay incentive for both the independent asymmetric information and the correlated asymmetric information. However, most existing works considered the issue of private information with little attention paid on the problems of hidden action. Considering the MUs' crowdsourcing action may not be monitored by the SP all the time, the MUs will deviate from the incentive mechanism. Moreover, due to the dynamic characteristics of mobile crowdsourcing environment, the crowdsourcing efforts of MUs may be different in various crowdsourcing tasks. Therefore, it is necessary to design a contract to attract MUs to offer the long-term helps in such dynamic crowdsourcing environments.
Apart from the above widely used monetary incentive mechanisms, reputation theory 31 is a non-monetary incentive strategy to measure the people's actions or efforts. A reputation-based multi-auditing algorithm was proposed for reliable mobile crowdsensing. 32 An online incentive mechanism based on reputation was proposed for mobile crowdsourcing systems. 33 Yu et al. 34 proposed a fog-blockchain distributed approach for crowdsourcing reputation management. However, in these works, the asymmetric information problems have not been taken into consideration. There is also very little research to combine the monetary and nonmonetary incentive strategy for MCNs.

System model and problem formulation
In a typical MCN, there are three essential parts as follows: an SP, a set of MUs N = f1, . . . , N g, and end users, shown in Figure 1. Each end user first sends its crowdsourcing requirement to the SP for help. Then, the requirements of all end users are divided into several small tasks by the SP. Here, the crowdsourcing incentive process is modeled as a labor market. The SP designs the contract to recruit the MUs to participate in these small crowdsourcing tasks. The contract is composed of the various items related with basic salary and performance bonus. Each MU chooses one item of the contract when willing to participate in crowdsourcing tasks.

System model
Assume that the MU i makes a crowdsourcing effort e i to complete the crowdsourcing task, where i 2 N . Then, the SP will obtain the profit p i from the help of the MU i. Considering the impact of the dynamic environment and certain measurement errors, 24 the SP's real profit p i is assumed to be a noisy signal, which is defined as where u i is the profit per unit crowdsourcing effort, and e is a normally distributed random variable with e;N (0, s 2 ). Without loss of generality, we assume that the SP pays the MU i with a linear payment, 35 which is given by where a i is the basic salary, and b i 2 ½0, 1 is the incentive coefficient related with the achieved profit p i of the SP. Then, by subtracting the crowdsourcing cost C(e i ) from the payment s i , the reward w i to the MU i can be obtained Note that the crowdsourcing cost of MUs increases with the resource consumption increasing. The larger crowdsourcing effort e i is, the more rapidly the crowdsourcing cost C(e i ) will grow. Then, C 0 (e i ).0 and C 00 (e i ).0. Thus, for simplicity, we assume that the crowdsourcing cost C(e i ) is quadratic, that is, where c i is the crowdsourcing cost coefficient of the MU i to describe the cost information (i.e. battery, computing power, and memory). Considering that the profit p i is a random signal, the reward w i is approximately normally distributed with mean and variance Moreover, in order to describe the willingness of MUs to participate in crowdsourcing tasks, the crowdsourcing behaviors of MUs can be categorized as riskaverse and risk-neutral. 36 A risk-averse MU does not want to achieve too much profit from crowdsourcing tasks, whereas a risk-neutral MU only wants to maximize its own profit. Here, each MU is assumed to have a constant absolute risk aversion preference. The negative exponential utility function of the MU i can be written as f (w i ) = À e Àh M w i , where h M .0 is the parameter of absolute risk aversion. A larger value of h M indicates more incentive for the MU to offer a crowdsourcing effort. Then, the expected utility U M i of the MU i can be defined as Then, the expected utility of the MU i can be rewritten as the MUs' expected utility can be simplified as Then, the utility of the SP is defined as the difference between the total achievable profit and the total payment to the MUs, which is written as with the expectation means

Problem formulation
In this article, a dynamic incentive mechanism is proposed in MCNs by combining reputation and contract theory. The optimal contract mechanisms are investigated with the following two incentive strategies, twoperiod contract without reputation strategy and twoperiod contract with reputation strategy. The timing of two-period contract-reputation dynamic incentive mechanism is described in Figure 2. The whole process of the two incentive strategies mainly consists of three similar phases: contract confirmation phase, crowdsourcing task complete phase, and contract realization phase. Two-period contract without reputation strategy (Figure 2(a)) is presented in the third section. Twoperiod dynamic contract incentive mechanism is proposed to capture the dynamic characteristic of the MUs' efforts in the long-term crowdsourcing tasks. The whole process is described as follows: Phase I: Contract confirmation phase. At the beginning stage of contracting, the SP broadcasts a long-term contract with a set of contract items fp (1) i (2) i )g to the nearby potential MUs, where p (1) i and p (2) i represent the SP's profit in Period 1 and Period 2, respectively. s (1) i (p (1) i ) and s (2) i (p (2) i ) are the SP's payment in Period 1 and Period 2, respectively. When receiving the contract, the MUs evaluate and inform the SP their choices if they accept certain items. Phase II: Crowdsourcing task complete phase of Period 1. After receiving the MUs' confirmations, the SP offers the crowdsourcing tasks to the employed MUs. Then, the MUs offer the effort e (1) i to participate in crowdsourcing tasks of Period 1. Phase III: Contract realization phase of Period 1. At the end of Period 1, after checking the received data, end users will inform the SP of the MUs' crowdsourcing performance by feedback. When crowdsourcing tasks succeed, the SP offers payment s (1) i (p (1) i ) to the employed MUs according to contracts. However, if certain crowdsourcing tasks fail, the corresponding employed MUs will obtain no payments. Phase IV: Crowdsourcing task complete phase of Period 2. The process of crowdsourcing task is similar to that of Phase II.
Phase V: Contract realization phase of Period 2. The process is similar to that of Phase III. If crowdsourcing tasks are completed, the SP offers payment s (2) i (p (2) i ) to the employed MUs according to contracts.
Two-period contract with reputation strategy ( Figure  2(b)) is proposed in the fourth section. Considering that reputation strategy can bring the implicit incentives to MUs, a two-period contract-based incentive mechanism with reputation strategy is investigated to further attract MUs to participate in the long-term crowdsourcing tasks. As shown in Figure 2(b), at the contract confirmation phase of each period, the SP broadcasts a contract with a set of contract items fp (1) i When receiving the contract, the MUs evaluate and inform the SP their choices if they accept certain items. The processes of crowdsourcing task complete phase and contract realization phase are similar to those of the incentive mechanism without reputation strategy.

Two-period contract-based incentive mechanism without reputation strategy
In this section, a two-period dynamic contract is investigated to solve the asymmetric information problems. According to the basic model of the SP given in equation (8), the SP's utility of Period 1 can be given by Similarly, the SP's utility of Period 2 can also be defined as Then, the SP's total utility in the two periods can be obtained as where d.0 is a discount factor. d.1 indicates that Period 2 lasts longer than Period 1.
Based on the basic model of the MUs in equation (7), the expected utility of the MU i in Period 2 can be defined as Then, the expected utility of the MU i in the two periods can be obtained as

Contracting design in Period 2
Based on backward induction, 18 the contract of Period 2 should be considered first. In order to motivate the MUs to participate in the crowdsourcing tasks of Period 2, we need to guarantee that each MU's achieved utility u (2) i in equation (12) should not be lower than its retained utility U . That is, the following IR constraint 18 should be satisfied Moreover, to motivate MUs to complete the crowdsourcing tasks effectively in Period 2, a contract should be designed to ensure that each MU can maximize its utility u (2) i in equation (12) by offering the optimal effort e (2)Ã i . Then, the IC constraint 18 should be designed as follows max e (2) i ø 0 Then, based on the above IC and IR constraints, the optimization problem is designed to maximize the expected utility of the SP E½u (2) s , which can be defined as (14) and (15) ð16Þ From the above IC constraint in equation (15), the optimal crowdsourcing effort e (2)Ã i of Period 2 can be obtained as Moreover, in order to achieve the SP's maximum expected utility E½u (2) s , we need to obtain the minimum basic salary a (2) i from equation (10). While from the IR constraint in equation (14), we can see that each MU achieves its retained utility U with the minimum basic salary a (2) i . Thus, the optimal basic salary a (2)Ã i of Period 2 can be obtained with the optimal crowdsourcing effort e (2)Ã i in equation (17), that is Then, by substituting equations (17) and (18) into equation (16), the optimization problem of Period 2 in equation (16) can be simplified as Any optimal local solution of Period 2 (denoted aŝ b (2) i ) to the optimization problem in equation (19) And the second-order derivative of the optimization problem in equation (19) is given by Therefore, the optimal solution of the incentive coefficient b (2)Ã i to equation (19) is obtained as Contracting design in Period 1 Here, the two-period dynamic contract design is considered in both periods. In order to assure that each MU's expected utility u i in equation (13) is no less than its retained utility U in the two periods, the IR constraint should be satisfied Moreover, to attract MUs to complete crowdsourcing tasks effectively in two periods, the contract should be designed to ensure that each MU can maximize its expected utility u i in equation (13). That is, the IC constraint in the two periods should be satisfied max e (2) i ø 0 Therefore, considering a (2)Ã i in equation (18) and b (2)Ã i in equation (22), the optimization problem of the two periods is designed to obtain the SP's maximum expected utility E½u s in equation (11), which can be written as From the IC constraint in equation (24), we can have the optimal effort e (1)Ã i of Period 1, that is Moreover, from the optimization problem of two periods in equation (25), we can achieve the SP's maximum expected utility E½u s only if the minimum basic salary a (1) i is obtained. From the IR constraint in equation (23), each MU achieves its retained utility U with the minimum basic salary a (1) i . Thus, the optimal basic salary a (1)Ã i of Period 1 can be obtained as Furthermore, by substituting equations (26) and (27) in equation (25), the optimization problem of two periods in equation (25) can be further simplified as Then, the optimal incentive coefficient b (1)Ã i of Period 1 can be obtained as Two-period contract-based incentive mechanism with reputation strategy In the above two-period contract-based incentive mechanism, the SP can obtain certain information about MUs through the achieved profit p (1) i in Period 1, such as the crowdsourcing performance e (1) i of the MUs. Then, the SP may renegotiate the contract offered to the MUs in Period 2, which will be a voluntary act to benefit both two parties. Therefore, in this section, by combining the contract's explicit incentive with the reputation's implicit incentive, the MUs can be motivated to participate in the long-term crowdsourcing tasks more effectively. In order to obtain the more payment in Period 2, each MU may strive to offer the more effort e (1) i of Period 1. Then, the performance of the MUs in Period 1 may affect the utility of the current period as well as that of Period 2. That is the mechanism by which the implicit reputation incentive can work.
Moreover, assume that the implicit achievable utility of reputation strategy to be lp (1) i , where l.0 is the implicit reputation parameter. While the MUs perform better in Period 1, their reputations' implicit effect will be greater. Considering that the SP determines the payment in Period 2 based on the achieved profit p (1) i in Period 1, we denote s (2) i (p (2) i p (1) i ) as the payment of Period 2. The expected utility of the MU i in Period 2 can be defined as i andê (2) i are the estimated efforts of the MU i in Period 1 and Period 2, respectively.
Similarly, the conditional variance can also be obtained as Then, by substituting equations (34) and (35) into equation (30), the expected utility of the MU i in Period 2 can be given by Next, in order to assure that each MU's achievable utility in equation (36) is no less than its retained utility U , the following IR constraint should be satisfied Moreover, to motivate the MUs to participate in the crowdsourcing tasks effectively, the contract design of Period 2 also needs to satisfy the following IC constraint max e (2) i ø 0 Thus, based on the above IC and IR constraints, the optimization problem is designed to obtain the SP's maximum expected utility E½u (2) s , which can be written as Under the assumption of rational expectation, 18 when equilibrium is achieved, the crowdsourcing effort e (1) i , e (2) i chosen by the MU will be equal to the estimated value of crowdsourcing effortê (1) i ,ê (2) i . That is, e (1) i =ê (1) i and e (2) i =ê (2) i . Then, from the IC constraint in equation (38), the optimal effort e (2)Ã i of Period 2 can be obtained Furthermore, from the optimization problem of Period 2 in equation (39), we can achieve the SP's maximum expected utility E½u (2) s only if the minimum basic salary a (2) i is obtained. Thus, from the IR constraint in equation (37), we can have the optimal basic salary a (2)Ã i of Period 2 Next, by substituting equations (40) and (41) into equation (39), the SP's utility maximization problem can be further simplified as Then, the optimal incentive coefficient b (2)Ã i of Period 2 can be obtained as Contracting design in Period 1 In this section, considering that each MU's utility u i in equation (31) is no lower than its retained utility U in the two periods, the IR constraint should be satisfied Similarly, in order to motivate MUs to complete crowdsourcing tasks effectively in two periods, the following IC constraint should be satisfied to ensure that each MU can maximize its expected utility. That is Therefore, based on the above IR and IC constraints, the optimization problem is designed to obtain the SP's maximum expected utility E½u s in equation (33), which can be written as Similar to Period 2, from the IC constraint in equation (45), we can have the optimal effort e (1)Ã i of Period 1 From the IR constraint in equation (44), the optimal basic salary a (1)Ã i of Period 1 can be obtained Accordingly, by combining equations (47) and (48) with equation (46), we can further simplify the SP's expected utility maximization problem, that is Then, the optimal incentive coefficient b (1)Ã i in Period 1 can be obtained as Since there are two periods in the dynamic incentive mechanism, it is not necessary to consider the influence of reputation effect in Period 2. Then, the MUs' optimal effort e (2)Ã i is only related to the incentive coefficient b (2)Ã i of Period 2. While in Period 1, considering the reputation effect, the MUs' optimal effort e (1)Ã i is not only related to the current incentive coefficient b (1)Ã i , but also affected by the implicit parameter l of reputation effect.
in Period 1 becomes smaller than that of Period 2. The smaller the optimal incentive coefficient given by the SP to MUs, the more utility SP can obtain in Period 1. Considering the reputation effect, the incentive mechanism can achieve the SP's optimal incentive coefficient leading to the increasing utility of SP.

Results and discussion
Simulation results are presented to evaluate the performance of the proposed dynamic incentive mechanism. All experiments are performed using the MATLAB platform. The profit per unit crowdsourcing effort of N = 5 MUs u i follows the uniform distribution. The absolute risk aversion parameter of each MU is h M = 0:3. The crowdsourcing cost coefficient of each MU is c i = 0:4. The deviation of the SP's real profit s 2 = 0:81. We assume that the retained utility of each MU is U = 0:2. The discount factor of the two-period contract is d = 0:6. The implicit reputation parameter of MUs is assumed to be l = 0:2.
Optimal contract design of two periods i is required to achieve the optimal utilities of the two parties.
Moreover, the two-period dynamic contract designs with reputation strategy in Figure 3(c) and (d) show the similar performance with those of Figure 3(a) and (b), respectively. By combining the reputation's implicit incentive with the contract's explicit incentive, the MUs will be motivated to participate in the long-term crowdsourcing tasks more effectively. Then, the crowdsourcing effort (e (1)Ã i =e (2)Ã i ) is larger than that of Figure 3   Effort incentive of two-period contract design Figure 4 describes the performance of the effort incentive with the three MUs selected from Figure 3. All the simulation parameter settings are the same as those of Figure 3. Figure 4(a) and (b) presents the optimal crowdsourcing utilities of MUs without reputation strategy in Period 1 and Period 2, respectively. As u i increases, the MU's profit per unit effort becomes large, leading to the more optimal crowdsourcing effort (e (1)Ã i =e (2)Ã i ). Moreover, only when the MUs select the optimal crowdsourcing effort e (1)Ã i and e (2)Ã i , do they achieve their maximum utilities U = 0:2. Thus, twoperiod contract-based dynamic incentives can solve the asymmetric network information problems to motivate MUs to make crowdsourcing efforts.
Moreover, Figure 4(c) and (d) shows the optimal crowdsourcing utilities and effort of MUs with reputation strategy in Period 1 and Period 2, respectively. We can observe that only if the MUs offer their optimal crowdsourcing effort e (1)Ã i and e (2)Ã i , they can obtain their maximum utilities U = 0:2. Moreover, considering the implicit incentive of reputation strategy, the optimal crowdsourcing efforts of Figure 4(c) and (d) are always larger than those of Figure 4(a) and (b) without reputation strategy. Thus, by combining the reputation's implicit incentive with the contract's explicit incentive, the two-period incentive method can attract MUs to offer crowdsourcing efforts more effectively.

Feasibility of the proposed incentive scheme
Finally, the feasibility of the proposed incentive strategy is evaluated. Figure 5 shows the optimal expected utility of the SP with the different incentive mechanisms. As the retained utility of MUs U becomes large, the basic salaries (a (1)Ã i and a (2)Ã i ) decrease, which results in the lower expected utility of the SP for all the incentive mechanisms. Moreover, by integrating the implicit incentive of reputation strategy with the explicit incentive of contract, the incentive mechanisms with reputation strategy always achieve the higher expected utilities of the SP than that of the incentive mechanism without reputation strategy. Furthermore, with the more implicit reputation effect parameter l, the SP achieves the higher expected utility. Specifically, the incentive mechanism with reputation strategy (l = 0:2) can gain an obvious performance improvement of about 20% comparing with that of the incentive mechanism without reputation strategy. Figure 6 shows the optimal expected utility of the SP with the different numbers of MUs. As expected, when the number of MUs N increases, the achieved expected utility of the SP will also increase in all the incentive mechanisms. Moreover, due to the implicit incentive of reputation effect, the incentive mechanisms with reputation strategy always achieve the higher expected utilities of the SP than that of the incentive mechanism without reputation strategy. Furthermore, as the implicit reputation effect parameter l increases, the SP achieves the higher expected utility. The incentive mechanism with reputation strategy (l = 0:6) always obtains the most expected utility of the SP, which improves over 80% utility comparing with that of the incentive mechanism without reputation strategy.
In Figure 7, the optimal expected utility of SP is studied with the various discount factors d. As the discount factor d increases, the SP obtains the more utility with the long-term crowdsourcing help of MUs. Specifically, when the discount factor is d = 0, the contract between SP and MUs is designed for one period. However, even with one-period crowdsourcing, the SP's expected utility is more than that without MUs' participation. Moreover, Figure 7 also shows that the SP's optimal expected utility is an increasing function of the reputation effect parameter l. The incentive   mechanism with reputation strategy (l = 0:2) always achieves over 20% utility than that of the incentive mechanism without reputation strategy.

Conclusion
In this work, the crowdsourcing incentive method between the SP and the MUs is proposed in dynamic environments. Two-period dynamic contract incentive mechanism is investigated to cope with the information asymmetric issue. The SP designs the contract to describe the basic salary and incentive coefficient of MUs. Each MU chooses one contract item when participating in crowdsourcing tasks. Moreover, in order to attract the MUs to complete the long-term crowdsourcing tasks, reputation information is introduced into the two-period dynamic contract. The optimization problem is formulated to maximize the SP's expected utility based on the IR and IC constraints. The optimal contract schemes are derived for both non-reputation and reputation strategies. Simulation results demonstrate that the proposed two-period dynamic contract method effectively increases the SP's expected utility by breaking information asymmetry. By combining explicit contract with implicit reputation, the MUs can be motivated to participate in the long-term crowdsourcing tasks more effectively with the less payment and the more crowdsourcing effort.