Multi-Leg Seat Inventory Control Based on EMSU and Virtual Bucket

Expected marginal seat revenue (EMSR) is a well-known method for airline seat inventory control airlines. However, this method employs a static model to study the dynamic reservation process, and does not take into account the risk tolerance of policy makers. Expected marginal seat utility (EMSU) replaces revenue by utility, which addresses the real situation of seat inventory control. However, there is still a lack of multi-leg seat control algorithms based on EMSU. Therefore, using EMSU and bucket algorithms, this paper applies the Markov decision-making process to simulate the flight reservation process and builds a dynamic multi-leg seat inventory control model. Experimental results validate the effectiveness of the proposed method.


Introduction
Airline seat inventory control is about "selling the appropriate seat to the right person at the right time" [1]. If all the airline's seats are sold at a cheap price, travellers on urgent business cannot buy short-notice tickets just before the plane takes off. On the other hand, if too many seats are allocated for travellers able to pay high prices, this may lead to many empty seats. Therefore, optimization of airline seats aims to allocate reasonable numbers of seats for the different levels of fare, in order to maximize flight revenue.
The most well-known airline seat inventory control method today is the expected marginal seat revenue (EMSR) method proposed by Belobaba [2]. Because this method is easy to understand and easy to implement, it was quickly adopted by many airline companies. EMSR has become the classic airline seat inventory control method and the normal foundation of seat optimization algorithms. But EMSR has two disadvantages. First, a reservation limit produced by EMSR cannot be changed as the reservation process progresses. In other words, the method is static. Secondly, the method does not consider the risk in the decision-making process [3]. To address these two shortcomings, the research community has recently proposed the expected marginal seat utility (EMSU) method [3][4]. EMSU has been applied in the single-leg seat control algorithm; however, for the multileg seat control problem there is not yet a sound solution. A popular method to address the multi-leg problem is the virtual bucket [2]. In this paper, a model based on the virtual bucket and EMSU ideas is proposed in order to solve the multi-leg problem.

Introduction to EMSU
EMSU uses the expected marginal utility to replace the expected marginal revenue in the EMSR model [3]. The utility function is: (1) The model determines the level of risk. EMSU is defined as below: (2) In Equation (2), c represents the number of remaining seats, i shows the requested class level, represents the fare of class i, means the risk factor, indicates the probability of requesting the level of class j in stage n, and a means the accepted number of passengers in stage n.

Virtual Bucket
To solve the multi-leg seat allocation problem, we should consider not only the fare level (F), but also the interaction between routing origin (O) and destination (D). The virtual bucket method is potentially important to solve multi-leg seat allocation effectively [4].

Model and Policies
The assumptions of this study include: 1. Each passenger can only request one seat.
2. There is no overbooking and there are no cancellations.
3. There are several levels of ticket from 1 to k, where the corresponding fare is , .
4. The aircraft capacity at each stopover point remains the same. The model of the booking stage is divided into N stages.
In each stage, one person at most will arrive. The probability of one passenger arriving is found in the Poisson process. jn p shows the probability when class j is requested in stage n. The probability of no passenger arriving is: .

model description
Markov decision processes are used to choose an action from the decision-making space, according to the various stages of decision-making. Decision-makers can also make a new decision according to the new state observed. This study uses Markov for dynamic decision-making in the booking process.
A Markov decision-making process includes the state space, action set, decision epochs, rewards and transition probabilities. In the model study herein, these are defined as follows. 1. State space: S={0,1…C}×{0,1…k}, where the first dimension stands for the remaining seat capacity and the second dimension is the fare class, with artificial fare class 0 with fare F0 = 0. A state (c, i) says that as c seats are remaining, we have a request for fare class i. Aircraft capacity is C, and the remaining capacity is the variable c, c ≤ C. 2. Action set: (4) This represents the "reject" and "accept" decision for a given state. 3. Decision epochs correspond to the time periods: T={0,1,..N}, where N represents the departure of flight and 0 is the start of booking. 4. Transition probabilities: (5) 5. Rewards: To combine EMSU and virtual bucket, first we should divide the bucket according to the multi-leg fares; second, we attach the bucket to the corresponding leg, and then we calculate the EMSU values for every leg using Equation 1.
Then, we determine the structured protection number of various stages. The structured protection number is different from the protection number in the EMSR model: it means that the air company can accept reservations only when the remaining capacity is greater than the protection number. The calculation method is as follows: (7) With the protection number, it is then possible to make a decision when a request arrives. The expression is: In Equation (8), only when the remaining capacity is larger than the number of seats left at stage n can the reservation for the long leg ODF be accepted.

The division of stage and the transition probabilities' setting
As stated above, we require N-stage booking stages and transition probabilities based on the Poisson arrival process. These requirements are met as follows.
1. The booking time is divided into a series of short time intervals, and in each time interval the passenger arrival process follows a Poisson distribution. 2. Each interval is set to a number n, n=1,2,3…h. Time interval 1 represents the end of the booking time; time interval h represents the beginning of the booking. 3. Since the model assumes that each decision interval has at most one passenger arrival, each time interval described above is divided into intervals related to the decisions required by the model. µ � � means the expected arrivals in stage n at the required class level k. The expected arrivals in each class level in every stage corresponds to the following equation: In each time interval, passengers arrive according to the Poisson distribution with a mean µ � . 4. The interval n is divided into � � short decision intervals of equal length. Passenger arrivals in each short interval meet the Poisson distribution with a mean µ � v � � ; the most important constraint is that at most one passenger arrives in each interval. 5. ε is a very small probability value, which can be negligible; v � must then satisfy the equation 6. The probability of passenger arrival in each decision interval is calculated. Passenger arrival meets the Poisson distribution with mean , and p � ��� stands for a passenger arriving, requiring class level k at stage n.

Numerical Simulation and Results
This study uses data from a Chinese domestic airline route, Nanjing (A) ---Shenzhen (B) ---Sanya ( Tables 3  and 4.  Table 3. Probabilities of passenger arrival ODF Arrival at interval t ABY 48,55,67,71,84,93,173,200,208,217,232,247,260,262,333,398,413,435,444,461,504,521,533,544,547,560,594,642,661,703,783,869,999,1004,1071,1103,1120,1163,1217,1221,1276,1302,1322,1383,1438,1536,1618,1693,1737,1763,1828,1919  In this case, the risk factor was set to 0.002. Then, the protection number was obtained in each stage. The protection number of the AB leg is expressed in Fig. 1, and that of BC in Fig. 2. The risk factor was changed to 0.001, and the results are expressed in Figs. 3 and 4. (See Table 2.) In Figs. 1 and 3, buckets 2, 3, and 4 mark the protected seats for ACY and respectively represent ODFs ABY, ACB and ABB. In Figs. 2 and 4, buckets 2,3, and 4 mark the protected seats for ACY and respectively represent ODFs ACB, BCY and BCB.   A comparison of the four graphs shows that a different risk factor may lead to different protected seats. It also indicates that the smaller the risk factor, the bigger the number of protected seats in general, and vice versa.
On the other hand, how does risk factor affect flight revenue? Table 5  simulation flight using the above method. The last column about the flight revenue shows that the smaller risk factor may bring more revenue; of course, this is on the premise that, no matter what the risk factors set, the flight requests should be consistent. Risk factor equates to risk tolerance, and is always set by the decision-maker or seat control employees in airlines. The decision-maker may thus control different seat allocation schemes at every stage over the sales period of a flight by setting different risk factors.

Conclusion
This paper combines the EMSU and the virtual bucket methods using a Markov decision process to approach the multi-leg seat inventory control problem. The authors simulated the arrival procedure of passengers with nonhomogeneous Poisson distribution. The theoretical analysis and the experimental results show that with the reduction of the risk factor, the number of protected highclass seats rises. With the addition of risk-sensitive conditions that surpass the subjective rationality of decision-makers, the decision-making process better corresponds to real situations.