A long short-term memory neural network model for knee joint acceleration estimation using mechanomyography signals

With the growth of the number of elderly and disabled with motor dysfunction, the demand for assisted exercise is increasing. Wearable power assistance robots are developed to provide athletic ability of limbs for the elderly or the disabled who have weakened limbs to better self-care ability. Existing wearable power-assisted robots generally use surface electromyography (sEMG) to obtain effective human motion intentions. Due to the characteristics of sEMG signals, it is limited in many applications. To solve the above problems, we design a long short-term memory (LSTM) neural network model based on human mechanomyography (MMG) signals to estimate the motion acceleration of knee joint. The acceleration can be further calculated by the torque required for movement control of the wearable power assistance robots for the lower limb. We detect MMG signals on the clothed thigh, extract features of the MMG signals, and then, use principal component analysis to reduce the features’ dimensions. Finally, the dimension-reduced features are inputted into the LSTM neural network model in time series for estimating the acceleration. The experimental results show that the average correlation coefficient (R) is 94.48 ± 1.91% for the estimation of acceleration in the process of continuously performing under approximately π/4 rad/s. This approach can be applied in the practical applications of wearable field.


Introduction
With the aggravation of the population aging process, the number of elderly people is growing rapidly. The world's population aged over 60 will grow from 12% in 2015 to 22% by 2050. 1 On the other hand, the number of amputees has increased due to vascular diseases, traffic accidents, work-related injuries, and accidental injuries. According to the Sixth National Census, the total number of people with disabilities was approximately 85.02 million in China by the end of 2010, of which 24.72 million were physically disabled, ranking first among all types of disabilities. 2 Therefore, it is urgent to solve the problems of inconvenience of life and travel caused by the weakening of the limbs of the above people, which is a major social problem that needs to be solved at present.
Many scholars and institutions have used wearable robot technology to research wearable power assistance robots in the elderly or the disabled fields. Wearable power assistance robots are developed to provide athletic ability of limbs for the elderly or the disabled who have weakened limbs to better self-care ability. 3 It is great significance and application prospects to meet the growing current situation of elderly and physically disabled people.
With the development of signal detection, signal processing, and data fusion technology, wearable power assistance robots have gradually evolved from a passive way of accepting human instructions to a way of actively recognizing and understanding human motion intentions. Early active and passive wearable power assistance robot products mainly use angle, force, acceleration, and other physical information as control sources. Since the external kinematic parameters can only passively reflect people's motion intentions, the actual information obtained lags behind human motion. 4 Biological signals can intuitively reflect the intention and state of human motion and precede the occurrence of macromotion. If the physiological signals of the human body can be used as control sources, the control of the wearable power assistance robot and the human body will be more coordinated and smoother. The research of modern human anatomy has shown that the surface electromyography (sEMG) signal is closely related to muscle activity. The sEMG signal can be generated about 30-100 ms before muscle activity. The sEMG signal can be used to estimate and evaluate muscle function, muscle strength, and fatigue state. 5 Through collecting and processing the sEMG signals of the skin surface, the features of the signals can be extracted as the source of control signals for the wearable power assistance robot control system. 6 However, most of the sEMG sampling electrodes require to be attached on the skin, which restricts its application. 7 The research has shown that not only sEMG signals are produced but also a low-frequency mechanical vibration signal is produced following the muscle activity. 8,9 Researchers use mechanomyography (MMG) to describe the mechanical vibration signal. MMG signal can provide information about the number of muscle motor units and the excitation rate. 10 The MMG signal reflects the characteristics of muscle activity during motion. Through effective features extraction and classification of MMG signal, the intention of human motion can be predicted in advance.
Researchers have tried to control the wearable power assistance robots based on the MMG signals. 11 MMG is a vibration signal generated by muscle contraction during motion. 12 The MMG signal acquisition does not need to be very accurate with the placement of sensors. 13 At the same time, using the MMG signals has such advantages as the low cost of acquisition system, anti-interference, and robustness. In addition, the data can be effectively collected by tying the sensor to the relevant parts through clothes without directly contacting with the skin so that data collecting is not affected by the change in resistance caused by sweat or the temperature of the skin surface. 14 MMG signals are more applicable to applications in sports, wearables, and so on.
Alves and Chau 15 made use of two acceleration sensors to detect the MMG signals of forearm muscle activity, and the recognition rate of the classifier reached 89 + 2% for three types of hand motion. Silva et al. 16 constructed classifiers based on the MMG signals for controlling the closing and opening of a prosthesis. The accuracy reached approximately 70%. Zeng 17 used only a single acceleration sensor to detect the MMG signals of forearm muscle activity. Through principal component analysis (PCA) and the construction of a quadratic classifier, the recognition rate of hand motion reached approximately 80% in real-time prosthetic hand control experiments. Ibitoye et al. 18 built a support vector regression model based on MMG signals to estimate the knee torque induced by neuromuscular electrical stimulation. When using the Gaussian support vector kernel, the decision coefficient between the actual torque and the estimated torque reached 94%. Dzulkifli et al. 19 constructed a neural network model based on the quadricep MMG signals to predict the knee torque to solve the situation that the muscle torque could not be quantified independently. It was found that the average accuracy of the predicted knee joint elongation torque reached 79 + 14%, which provided a safer automatic control for the standing of patients with a complete spinal cord injury. Youn and Kim 20 used acceleration sensors (ADXL202JE) to record the MMG signals of the brachioradialis and biceps muscles. Meanwhile, they extracted features of the MMG signals as inputs to the neural network model. Then, the bend force of the elbow joint was estimated by the neural network algorithm.
In this study, we take the knee joint as the research object and design a long short-term memory (LSTM) neural network 21 that uses MMG signals for estimating the motion acceleration of knee joint. The acceleration can be further calculated by the torque required for movement control of the wearable power assistance robots for the lower limb. We detect the MMG signals on the clothed thigh. Some time-and frequency-domain features of the MMG signals are extracted, and then, PCA 22 is used to reduce the features' dimensions. Finally, the dimensionreduced features are inputted into the LSTM neural network model in time series for estimating the acceleration of knee joint.
The rest of the article is organized as follows. In the second section, we introduce the LSTM neural network. In the third section, the experiment and signal preprocessing are described, including the experimental procedure, the MMG signal preprocessing, the feature extraction, the features dimension reduction, and the implementation of the LSTM neural network model. The experimental results and discussion are presented in the fourth section. Finally, the conclusion is shown in the fifth section.

Long short-term memory neural network
Recurrent neural network (RNN) is a neural network for processing sequence data. 23 RNN remembers the previous information. The previous information is applied to the calculation of the current output. Figure 1 shows the structure of the RNN. The neurons of the hidden layer are connected. The input of the current time hidden layer includes not only the input of the current time but also the output of the previous moment hidden layer. Figure 2 shows the unfolding of a neuron of the hidden layer on the input sequence. The output value O of a certain neuron of the hidden layer at the current moment not only depends on the input x of the neuron at the current moment but also depends on the output value O tÀ1 and the memory value S tÀ1 of the hidden layer at the previous moment.
The LSTM neural network is a deformed structure of RNN. The LSTM neural network combines long-time and short-time series-related information through subtle gate control to better preserve the long-time series-related information and to control the gradient flow, which solves the problem of gradient disappearance to a certain extent. The unit composition of the LSTM block is shown in Figure 3. Each LSTM block is equal to a neuron in the hidden layer of RNN. The LSTM neural network uses a gate structure to remove or increase the memory information of a block, to retain the important information, and to remove the unimportant information. The LSTM block consists of the forget gate, input gate, and output gate. These units are connected in series to learn and store long-term and short-term seriesrelated information.
The first unit is the forget gate, which decides how much information of unit memory S tÀ1 from the previous block is retainedS tÀ1 in the current block. The forget gate includes a sigmoid function and a pointwise multiplication operation. The sigmoid function is as follows where W f is the weight parameter, O tÀ1 is the output of the previous hidden layer, x t is the input for the current block, and b f is a bias term. 24 S tÀ1 is expressed as follows: where f t is the result from formula (1), S tÀ1 is the unit memory from the previous block, andS tÀ1 is the memory preservation from the previous block to the current block. 25 The second unit is the input gate, which decides how much information of the input x t is saved to the unit memory S t at the current moment. 26 The input gate includes a sigmoid function, a tan h function, and a pointwise multiplication operation. The sigmoid function is as follows where i t is the decision vector of the unit memory at the current moment, W i is the weight parameter, and b i is a bias term.
The tan h function is as follows where S t is the candidate information of the unit memory at the current moment, W c is the weight parameter, and b c is a bias term. 26 The pointwise multiplication operation is as follows where S t is the output of unit memory at the current block. The third unit is the output gate, which decides how much information of the unit memory S t is outputted to the output O t at the current block. 26 The output gate includes a sigmoid function, a tan h function, and a pointwise multiplication operation. The sigmoid function is as follows where h t is the part of the unit memory output at the current block, W h is the weight parameter, and b h is a bias term. The tan h function and the pointwise multiplication operation are as follows where O t is the output at the current block. Finally, the output O t and the output of the unit memory S t at the current block are inputted into the next moment hidden layer. The process is repeated. The difference between the output of LSTM neural network and the real training samples is minimized by learning and optimizing the weight parameters and bias terms of the model.

Experimental procedure
The right knee joint acceleration is estimated while the right knee joint is performing isokinetic knee extension and flexion motions in this experiment. Meanwhile, the Medical Ethics Committee of Hefei Institutes of Physical Sciences authorized the experiment. We select four healthy young people as experimental participants, who aged between 24 and 30 years. We refer to the knee joint swing angle (approximately 60 ) during normal walking. In the sitting position, participants are required to continuous extension and flexion of the knee joint under approximately p/4 rad/s (SV) and p/2 rad/s (FV) for 30 s, as shown in Figure 4. Each participant has a complete rest between two different speed experiments.
According to the anatomy of human motion, we select the superficial muscles that control the continuous motion of the knee joint for MMG signal detection. We select semitendinosus, biceps femoris, vastus medialis, vastus lateralis, and rectus femoris from the thigh, as shown in Figure 5. Figure 6 shows the collection and store system of data. According to the previous experiment, the MMG sensor is made by a triaxial accelerometer (model ADXL335). 27 The positive direction of the z-axis is placed perpendicular to the skin outward. The output of z-axis measures the MMG signal. Five MMG sensor positions are selected to be in the corresponding muscle's central region and are bound to the clothes by kneepad-like straps. The acceleration sensor is made by a dual-axis accelerometer evaluation board (model ADXL203EB), which measures the real linear acceleration of the knee joint to calculate the angular acceleration and the torque. 28 The acceleration sensor is placed on the clothes against the medial ankle joint of the shin (the X-axis is perpendicular to the shin and points forward in front of the body), and it is bound to the leg by kneepad-like straps for following the rotation of the knee joint. The acceleration is calculated according to change in the acceleration sensor data of the motion of knee extension and flexion, which marks the acceleration category label. A collector (model NI USB-6215) collects signals of the MMG sensors and acceleration sensor. The collection frequency of the data is 2000 Hz. The collector is connected to the host computer through the USB interface. Meanwhile, the data are shown and saved by the program of the host computer.

Signal preprocessing
We use sliding window and stepping methods to continuously read signal flow. With reference to the previous  related research, 14 we chose a 250 ms (500 samples) sliding window and 62.5 ms (150 samples) increment. For the MMG signals, we use a 5-100 Hz third-order Butterworth band-pass filter to filter out the human motion trajectory and environmental noise. Figure 7 shows the filter effect on the MMG data of the rectus femoris. For the acceleration signal, we use a 1.5-Hz fourth-order Butterworth low-pass filter to filter out high-frequency noise. Figure 8 shows the filter effect on the acceleration data.

Feature extraction and dimensions reduction
In this experiment, we extract features from the set sliding window of each MMG signal. Root mean square, mean absolute value, zero-crossing rate, slope sign change, and waveform length of the MMG signals are selected as the time-domain features. 29 Median frequency, 30 mean power frequency, 30 and mean frequency 29 of the MMG signals are selected as the frequency-domain features. Sample entropy and Spearman's correlation coefficients of the MMG signals are selected as other features. 31 To improve the speed of data processing, the 48-dimensional time and frequency domain feature data constructed from five MMG sensors data are reduced dimensionally by the PCA. To implement the PCA, we call the PCA function in MATLAB 2018A. Principal components of the training samples whose cumulative contribution rate is more than 90% are obtained as new input features of the training model. The new features contain 18 principal components. Figure 9 shows the contribution rate of the principal components. Meanwhile, the corresponding transformation matrix of training samples is obtained. The test samples are multiplied by the transformation matrix to obtain the new features of the test samples. The new feature data of the training samples are normalized using the Z-score method 32 and then are inputted into the LSTM neural network model. The new   feature data of the test samples are normalized using the mean and standard deviation of the training samples.

Implementation of the long short-term memory neural network model
The implementation process of the LSTM neural network model is shown in Figure 10. The five-channel original MMG and acceleration signals are filtered by Butterworth filter. The feature extraction of the MMG signals is performed by the sliding window method. The feature data are reduced dimensionally by the PCA and then normalized using the Z-score method. In addition, the corresponding category labels are generated by the mean value of the acceleration data in each sliding window. Each participant can yield nearly 390 sets of valid time-series data (acceleration-MMG). According to the time series, we select the first 80% of valid data as the training samples. Then, we select the last 20% of valid data as the test samples.
To implement the LSTM neural network, we call deep learning toolbox in MATLAB 2018A. The designed LSTM neural network is shown in Figure 11. The LSTM neural network that we design contains four layers. First, the timeseries features are inputted into the input layer according to the time sequence. Second, the input layer is connected to the LSTM layer. Third, the LSTM layer is connected to the fully connected layer. Fourth, the fully connected layer is connected to the regression layer. Finally, the regression layer outputs the estimated acceleration. The input layer contains 18-dimensional feature data according to the result of the PCA algorithm. The LSTM layer contains 400 neurons. During model training, we set the number of iterations is 400, and the initial learning rate is 0.005. After 200 iterations, the learning rate decreases according to a fading factor of 0.2. Meanwhile, to compare the superiority of the LSTM neural network model, we set up 50 and 100 neurons in the fully connected layer to carry out comparative experiments.
According to the time series, we input the training samples of each participant into the LSTM neural network to train the neural network model. According to the time series, we input the test samples of each participant into the trained model, respectively. We, respectively, calculate the correlation coefficient (R) and root mean square error (RMSE) of the test samples to assess the accuracy. Finally, the neural network model with the highest R is selected. The calculation formulas of R and RMSE are as follows whereŷ i is the estimated value of the i'th test sample; y i is the observed value of the i'th test sample; D ðŷ i Þ is the variance of the estimated value of the test samples; D(y i ) is the variance of the observed value of the test samples; and Cov ðŷ i ; y i Þ is the covariance between the estimated value and the observed value of the test samples.

Experimental results and discussion
Tables 1 and 2 present R values on the test samples under SV and FV for four participants, when the fully connected layer contains 50 and 100 neurons, respectively. The mean of R is 88.43% under SV and FV for four participants, when the fully connected layer contains 50 neurons. The experimental results with 50 neurons are better than the experimental results with 100 neurons in the fully connected     sliding window and stepping methods from approximately 11,700 MMG signal values. We observe that the waveform of MMG signals change similarly with knee flexion and extension motion. The selected MMG signals can effectively reflect the motion information of knee joint. Figure 8(

Conclusions
In this study, we design an LSTM neural network model based on the MMG signals to estimate the motion acceleration of knee joint. Performance of the model is compared under SV and FV for four participants. The experimental results show that the model can better estimate the motion acceleration of knee joint under SV. The whole results show that the LSTM neural network model performs well. This approach promotes the application of the angular acceleration and the torque required for movement control of the wearable power assistance robots for the lower limb. Meanwhile, it improves the flexibility, comfort, and wearable ability of the wearable power assistance robots for the lower limb.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: