DRNN-based shift decision for automatic transmission

In research on intelligent shift for automatic transmission, the neural network selected has no feedback and lacks an associative memory function. Thus, its adaptability needs to be improved. To achieve this, an automatic shift strategy based on a deep recurrent neural network (DRNN) is proposed. First, a neural network framework was designed in combination with an eight-speed gearbox that matches a particular type of vehicle. Then, the working principle of the DRNN was applied to the shifting process of an automatic gearbox, and the implementation model of the shift logic was established in MATLAB/Stateflow. A data sample obtained from the model was used to train the DRNN. Training and evaluation of the DRNN were accomplished in Python. Finally, a simulation comparison of the DRNN with a back-propagation (BP) neural network proved that after the epochs have been increased, the DRNN has higher precision and adaptation than a BP neural network. This research provides a theoretical basis and technical support for intelligent control of automatic transmission.


Introduction
For decades, the proportion of automatic transmission (AT) vehicles in the market has been increasing significantly because they are easy to operate, make drivers comfortable, and reduce driver fatigue. 1 Consequently, many alternative methods and theories have been proposed to solve the attendant problems. A key issue with conventional AT is low transfer efficiency. Many novel shift strategies have been proposed to improve driving comfort while taking vehicle dynamics and fuel economy into account.
Yi et al. 2 researched the control system of automatic vehicle transmission and used fuzzy control theory to achieve more reasonable shift occasion for the tracked vehicle. In 1992, Bavarian Motor Works of Germany developed an adaptive control system of four-speed and five-speed ATs that appropriately adjusts shift strategies by identifying the driver style and the environment and then shifting automatically. Deok-Hokim et al. 3 proposed an adaptive neuro-fuzzy inference system combined with Lyapunov stability and robust adaptiveness for smooth and precise gear shifting.
Based on precedent, the driving environment, 4,5 and the driver's intention, 4,6 an advanced driver assistance system (ADAS) 7 was further considered for avoiding or reducing unnecessary shift actions, and consequently, achieving better performance. Based on the shift strategy of fuzzy control, Kahlbau and Bestle [8][9][10] researched how to avoid energy waste, and their simulation test demonstrated that their proposed shift 1 strategy enhances fuel economy. Furthermore, some researchers optimized the shifting process of automatic transmission in other ways, such as minimizing the jerk 11 and reducing greenhouse gas emission. 12 Other researchers presented dynamic analyzes of mathematical models that were used to study the control strategy of hydromechanical continuously variable transmission (HMCVT) 13 to improve shifting quality.
However, the control effect is related to design experience, for which it is hard to offer an optimal solution. 14 The emergence of the neural network (NN) provides a novel solution to this contradiction. NN has a highly parallel structure and a capacity for parallel implementation that results in better fault tolerance and faster overall processing. The NN possesses an inherent nonlinear property that stems from its theoretical ability to approximate arbitrary nonlinear mapping. It brings a new promise to nonlinear control problems. A suitably trained NN has the ability to generalize all data. Thus, an NN can solve control processes that are hard to deal with using mathematical models or descriptive rules. 15 Kondo and Goka et al. 16 introduced the NN into shift strategy and set the road gradient, vehicle speed, braking force, and steering angle as input data, which made their model cover more extensive road conditions, particularly uphill and downhill conditions. However, more sensors are required to acquire several different types of data concurrently. The extra cost of placing these sensors in the vehicle would raise production costs, which would be unacceptable to both manufacturers and customers. 17 Li et al. 14 researched a high-speed, accurate NN optimization method for a shift schedule that meets real-time control requirements for construction vehicles in complex working environments. Li et al. 17 proposed an adaptive downhill shift control system that considers vehicle state and driver intention in order to provide a reliable, effective solution to the downhill problem, and for comprehensive improvement of drivability, the strategy of uphill, sports, and heavy brake application.
However, although the NN selected in the above shift strategy is superior in comparison with a backpropagation NN (BPNN), it remains a feedforward NN with the implication that data are isolated from each other at different moments. DRNN is a kind of powerful artificial NN algorithm proposed based on the premise that human cognition is based on past experience and memory. DRNN is different from the traditional NN as it not only considers the input from the previous moment but gives the network a new function to remember the previous content. The name, deep recurrent neural network, implies that the current output of a sequence is also linked to the previous output.
Notably, the network will remember the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment. Hence, it is particularly useful for processing sequential data such as sound, time series (sensor) data, or written natural language.
Because DRNNs have these advantages, the DRNN was applied to the gear shifting decision process in this study, and consequently, enhanced its use of data. In the subsequent sections of this paper, the principle of the DRNN is first introduced to explain how it works. Then, the automatic shift strategy based on DRNN is proposed, including data acquisition, preprocessing of training data, and training of DRNN. Finally, the BPNN gear shifting prediction model is set as the control group, and the simulation result shows that DRNN is indeed superior to BPNN in prediction accuracy. In the next phase, more data collected from varied vehicle models under complex conditions will train new neural network to adapt corresponding models and various road conditions. This research can provide technical guidance and theoretical basis for vehicle automation and intelligence.

Structure of DRNN
The eight-speed gearbox matching a particular type of vehicle was selected as the research object. The relative fuel injection volume (ratio of the current fuel injection volume to the maximum fuel injection volume) and the speed were taken as the input of the DRNN. Thus, the format of the input layer is 2 3 1. Then, the data of the current step and the previous two steps are set as the input data for each input. Because gear selection is a classification problem, the number of neuron nodes in the output layer is the type (label) of the output vector, which is also the number of gear positions. Thus, the number of neuron nodes in the output layer is eight. To improve the adaptability of the network and speed up the rate of convergence in training, a hidden layer with three recurrent layers is designed. Through experience and experimentation, we find that DRNN works best when the number of hidden layer nodes is 13.

RNN training algorithm: Back-propagation through time
Forward compute. The output of the recurrent layer s t and the output of the output layer o t at time step t are defined as: . . .
Where f(Á) is the activation function of the current layer, G 2 < 13 3 2 is the weight of the input layer to the 1th recurrent layer, x t 2 < 2 3 1 is the input, W i 2 < 13 3 13 is the weight of the ith recurrent layer at time step t21 to the ith recurrent layer at time step t, s i t 2 < 13 3 1 is the output of the ith recurrent layer at time steps t, U i 2 < 13 3 13 is the weight of the ith recurrent to the i + 1th recurrent layer, g(Á) is the activation function of the output layer, and V 2 < 8 3 13 is the weight of the last recurrent layer to the output layer.
Calculation of error terms. BP through time propagates the error of the layer h at time step t in two directions. In one of the directions, back-propagation through time (BPTT) transfers the error to the upper layer to obtain d hÀ1 t , which is related to only the weight matrix U. In the other direction, BPTT transfers it to the initial time step through time to obtain d h 1 , which is related to only the weight matrix W.
The weighted input of the neuron at time step t is defined as: Therefore, the output of the recurrent layer at time step t21 is defined as: Partial derivatives of net t with respect to net tÀ1 are defined as: where w j, i is the weight from the neurons in recurrent layers at time step t21 to the jth neurons in recurrent layers at time step t, s t k are the kth neurons in the recurrent layer at time step t, and net t j is the input of the jth neurons in the recurrent layer at time step t.
Thus, the final expression of equation (5) is defined as: This formula describes the rule that error terms propagate to the last moment through time. Based on this formula, an error term at any time step k is defined as: where d T k is the matrix transpose of the error term at time step k. This formula is used to calculate the error term of the BP through time.
The relationship between the weighted inputs of the neighboring two cyclic layers are as follows: Where net l t is the weighted input of the lth layer at time step t, a lÀ1 t is the output of the l-1th layer at time step t, and f lÀ1 is the activate function of the l-1th layer.
Thus, partial derivatives of net l t with respect to net lÀ1 t are defined as: Hence, the matrix transpose of the error term of the previous layer is: This equation describes the rule that error terms propagate from one layer to the above layer. Based on this, the error terms of mth layer are defined as: Weight gradient calculation. Because the partial derivatives of loss function E with respect to weight matrix W, which has a connection with only the present input of the recurrent layer has no connection with U and X t , partial derivatives of loss function E with respect to each weight matrix W is defined as: Weight modification. Because the stochastic gradient descent (SGD) has the advantage of automatic escape from the saddle point and local optimum, the SGD is chosen as the optimization algorithm. The expression of weight modification is: where m is batch size, E is the loss function, f x i ð Þ ; w À Á is the predicted output, y(i) is the actual output, and h is the learning rate.

Prediction principle of shift
Considering the two-parameter shift situation for vehicle speed and relative fuel injection volume and for better consideration of vehicle dynamics and fuel economy, the realization flow chart of an intelligent shift program based on DRNN is provided in combination with the aforementioned theoretical model of the DRNN. The flowchart of the DRNN-based automatic shift strategy is presented in Figure 1.
In Figure 1, the Stateflow model of shift logic is first constructed using MATLAB to determine the critical speed and relative fuel injection volume of different shift positions. Then, the data between the shift critical value is filled completely as input data and expected output. Because the dimensions of the input data are different from the dimensions of the expected output, and a DRNN is sensitive to the data, the data needs to be normalized as training samples. Subsequently, the number of neuron nodes of the input layer and the output layer are determined based on the dimensions of the input data and the actual output in the training sample, and the number of neuron nodes of the current layer is determined based on an empirical formula experiment. The structure of the NN is determined concurrently. After the NN is trained using the training data, the output and theoretical output of the recurrent layer is obtained. By comparing the actual output with the theoretical output and calculating their difference, we obtain the loss function and error term, which are used to modify the weight matrix between layers in the NN. Finally, training of the NN is stopped when the epoch reaches the maximum. After the training is completed, this DRNN can be used to assist transmission with automatic shift control.

Establishment of shift logic Stateflow
The Stateflow model can realize gear selection for transmission, as shown in Figure 2. The input is defined as the relative fuel injection volume and vehicle speed by the Model Explorer, and the output is defined as the gear required. Selection-state (always active) starts by calculating the during function.
Based on the gear and the instantaneous value of the relative fuel injection volume, the velocity threshold of the gear is calculated. When the model is in steadystate, it compares these values with the current speed to determine whether to shift or not. If the speed does not meet the shift condition, the model will not shift and moves back to steady-state to avoid the shift caused by external noise, see Table 1.

Preprocessing of training data
DRNN is sensitive to data, and the dimensions of input data are different from the dimensions of expected output. Hence, data needs to be normalized, and its normalization formula is defined as: where x' is the normalized data, x is the data waiting to be normalized, x min is the smallest data in each column, and x max is the biggest data in each column.

Training
To highlight the superiority of DRNN in processing time series data, 18 a BP NN (BPNN) is designed as the control group. This BPNN uses the hidden layer and output layer with the same number of neurons as the DRNN, and the output layer of the BPNN selects the same activation function and loss function as the DRNN. The structure diagram of the DRNN and BPNN are presented in Figure 3.
After the data was normalized, the required libraries were imported into Jupyter, and super parameters in the NN were set based on experience. When the data is inputted to both NNs, the weights in both NNs will be modified based on the algorithm introduced earlier, which makes both neural networks achieve the expected level of precision. Simultaneously, the image of loss varying with epochs was obtained and is presented in Figure 4. The loss of DRNN was eventually lower than 0.2; thus, achieving the expected accuracy.
Apparently, when compared with a DRNN, the BPNN has limitations dealing with complex problems, especially when dealing with time series problems.
Finally, the image of accuracy varying with epochs of the two neural networks is shown in the figure below. As shown in Figure 5, the accuracy of the BPNN tends to become flat after being trained 8000 times and finally fluctuates around 80%, which is not an ideal level of accuracy. It is easy to see that the BP neural network has some limitations in dealing with a two-parameter shift. After being trained, the accuracy of the DRNN   will slowly increase, vary with epochs, and eventually reach 93%, which is far higher than the accuracy of the BPNN.

Simulation
After the training of the DRNN is completed, five groups of test data are selected as the input for the DRNN and BPNN. Then, the prediction results for gear position can be obtained, as shown in Figure 6 and Table 2. Simulation results show that the DRNN has higher accuracy than the BPNN in predicting gear position. In addition, the prediction accuracy of DRNN is significantly higher than that of BPNN when the relative fuel injection volume is 0. Conversely, a comparison of Figure 6(b) and (c) shows that at the same   Note: '' + +'' means that the prediction accuracy of DRNN is much higher than that of BPNN. '' +'' means that the prediction accuracy of DRNN is slightly higher than that of BPNN. ''-'' means that the prediction accuracy of DRNN is slightly lower than that of BPNN. ''0'' means that the prediction accuracy of DRNN and BPNN are nearly equal.
relative fuel injection volume and speed, the gear position in the downshift process is not lower than the gear position in the upshift process, which is in agreement with the test data provided. However, the BPNN, whether accelerating or slowing down, will give the same result when the input does not change. Finally, only through relative fuel injection volume and speed could the state of the vehicle (upshift or downshift) be judged, and the two parameters for high precision shift also be achieved. Importantly, the Table 2 shows four things. First, the prediction accuracy of DRNN will increase with the decrease of the relative fuel injection volume when the vehicle is running at low speed. Second, the prediction accuracy of DRNN is slightly higher than that of BPNN on the whole when the vehicle is running at high speed. Third, the DRNN model has more prediction accuracy for high-speed vehicles when the relative fuel injection volume is large. Last, the prediction accuracy of DRNN is significantly higher than that of BPNN when the vehicle is slowing down and in process of shift down.
Compared with previous research, the similarities between researches lies in that the excellent nonlinear fitting ability of neural network was sufficiently utilized. The difference lies in that a more advanced and more applicable neural network was applied, which further improved the prediction accuracy of neural network with its unique advantages in processing time series data. In addition, compared with other neural networks, DRNN can achieve the same function with fewer features.
Although there are important discoveries revealed by these studies, there are still shortcomings in this study. For example, the training data of vehicles under complex condition (such as mountains and muddy roads) are not obtained, which makes it difficult to guarantee the prediction accuracy of this model under complex condition.
As the current simulation results are relatively ideal, it has the prospect of being applied to other vehicles. If the data under complex working conditions can be obtained, the coverage of the model will certainly be wider.
This study does not apply the shifting model in a validated vehicle model with different driving cycle. Therefore, in the next phase of the study, this problem will be firstly solved and the data obtained from different vehicle under complex pavement will be used as extra training samples to train the new neural network to improve its adaptability and practicability.

Conclusion
For a transmission control system built with DRNN, we only need to obtain real-time data on relative fuel injection volume and speed to judge the vehicle's state (acceleration/deceleration) and determine the corresponding gear position that overcomes the shortcomings of the traditional two-parameter neural network shift.
By comparing the simulation results for the BPNN and DRNN, we find that the prediction accuracy of a DRNN reaches more than 93%, which is much higher than the 80% for the BPNN. This indicates that a DRNN has superior adaptability to that of a BPNN when applied to gear shifting in vehicles.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The National Natural Science Foundation of China