A Dai–Yuan conjugate gradient algorithm of linear equation constrained optimization approach for optimal robust controller of bipedal robots

In this article, combined rapidly exponential control Lyapunov function with hybrid zero dynamics, a sufficiently descent projected Dai–Yuan approach is proposed, investigated, and analyzed for online solving optimal robust control problems with linear equation constraints of bipedal robots. Moreover, a new approach is developed for designing optimal robust controller. To demonstrate the effectiveness and feasibility of the proposed method, we will conduct numerical simulations on the model of three-link robot with nonlinear, impulsive, and under-actuated dynamics. Numerical results show that the bipedal robot can walk effectively and stability on the ground though the optimal robust controller when the parameters of the hybrid system model are disturbed three times. Furthermore, under the random noise condition, the bipedal robot can walk stably and effectively through online computing the nonlinear optimization problem for optimal robust controller. In addition, some classical control methods are compared with the developed approach in this article, numerical results are reported and analyzed to substantiate the feasibility and superiority of the proposed method for linear equation constrained optimization problem. Last, this article develops a systematic approach on exploiting optimal robust control technique to design hybrid system models for robustly and accurately via online solving linear equation constrained optimization problems.


Introduction
At present, with the enhancement of human's ability to detect deep sea and space, more and more robots are utilized to accomplish the impossible tasks. In the process of completing the task, the bipedal robot generates an unstable walking gait due to external operating condition information or internal parameter disturbances, therefore, it hopes that the bipedal robots can deal with the interference better and realize the high efficiency and stable working gait.
Hybrid zero dynamics is one of the effective methods to solve the dynamic and stable walking of bipedal robots. [1][2][3][4][5][6] It can realize dynamic stability of bipedal robots by selecting a suitable output function and using feedback linearization technology to design feedback controller. Combined rapidly exponential stabilizing control Lyapunov function with hybrid zero dynamics and Poincaré mapping principle, Ames et al. designed a rapidly exponential stabilizing controller and achieved a highly efficient and stable walking gait for bipedal robots. 7,8 However, in the process of controller design, it was necessary to adjust the parameters of the controller, moreover, if the parameters were not selected at real time, the bipedal robot could not produce stable walking gait or even fall down. According to the rapidly exponential stable theorem and control torque saturation condition, Gallaway et al. designed a feedback controller by solving the quadratic programming problem to realize the dynamic walking of the bipedal robots. 9 However, the disadvantage is that the objective function must be quadratic, and the constraint conditions can only be linear, which cannot better satisfy the requirements of designing controller. For the uncertainty mathematical model of bipedal robots, none of the controllers mentioned above has been realized in the dynamic and stable walking of bipedal robots.
The input-state-stability has been developed, investigated, and analyzed for the problem of robust stability of hybrid systems. 10,11 Therefore, a series of results have been obtained in recent years. However, the main drawback of the input-state-stability is that there is no zero-tracking error, that is to say, the system error can only converge to a certain neighborhood of the origin, which brings trouble to the system analysis and synthesis. By designing the control Lyapunov function, Sontag constructed the explicit robust feedback controller and analyzed the robustness of the nonlinear system via combining the input-state-stability. [12][13][14] Therefore, the robust stability of the nonlinear system was first developed by using the control Lyapunov function method. In addition, sliding mode control can solve the problem of model uncertainty, which is widely utilized to analyze the robust stability of nonlinear systems. [15][16][17] The advantage of sliding mode control technique is that the state of nonlinear system can quickly converge to the cycle orbit without error. To stabilize the unstable limit cycle of a compass-like bipedal robot, an active disturbance rejection control was developed, investigated, and analyzed through a transverse coordinate transformation technique. 18 Moreover, to obtain different walking gaits, an interconnection and damping assignment passivity-based control technique is proposed for a simple compass-like bipedal robots. 19 However, to ensure the rapidly convergent stability of the nonlinear system and keep the nonlinear system attract to the sliding surface, the principal drawback of sliding mode control technology is the chattering phenomenon of discrete switching models.
In this article, considering a class of hybrid systems with uncertainty represented by bipedal robots, the Lyapunov theorem and nonlinear optimization method of rapidly exponential stability are developed and analyzed under some suitable conditions. Moreover, a novel nonlinear optimization algorithm with rapidly convergent property is designed to solve the optimal robust controller to realize the dynamic and stable walking of bipedal robots. The article is organized as follows: in the second section, we will review the model of bipedal robot with uncertainties. In order to solve optimal controller, combining the rapidly exponential control Lyapunov function and control torque saturation conditions, we transform the problem of solving optimal control into nonlinear optimization problem with linear constraints in the third section. Optimal robust controller is proposed via an online solving optimal algorithm and its global convergence in the fourth section. It is emphasized that all of the above will be illustrated on a bipedal robot with uncertainties and random noise condition in the fifth section. Finally, the sixth section addresses the conclusions and future work for bipedal robots.

Dynamic model of bipedal robots
The dynamic model of bipedal robots is a kind of hybrid system with the properties of nonlinear, strong coupling, and impulsive. It mainly includes the nonlinear differential system represented by swinging foot and the collision mapping represented by supporting foot of bipedal robots. In general, the hybrid system model of bipedal robots can be seen as follows where the vector fieldsf ;g are locally Lipschitz continuous functions, q À ; _ q À ð Þis the pre-impact state variable, q þ ; _ q þ ð Þ is the post-impact state variable, S is the switching surface when the swing leg contact with the ground, D is an impact mapping, q means the state variable of bipedal robot, u 2 R m represents the control input. Furthermore, the output function is defined as follows where q q ð Þ represents a monotone function in variable q. y d q q ð Þ ð Þ means the desired trajectory of the robots. In order to convenience, given a zero-dynamic surface as follows where L f is denoted by Lie derivative. 20 The swinging leg of bipedal robot is not allowed to impact with the ground before touching the ground; therefore, bipedal robots should satisfy unilateral constraint condition in the process of dynamic walking, that is According to the assumption that the output function y 2 R m has a relative order of 2, Ames et al. obtain the following second-order system for hybrid system equation (1) Due to the idea of feedback linearization, a feedback controller is designed with the following equation Substituting equation (3) into equation (2), the following equation can be obtained as To improve the convergence rate of the control approach, a proportion-differential (PD) controller is designed by adopting parameter e. Let h ¼ y; _ y ½ T , it yields that Replacing equation (5) with equation (4), the following equation can be generalized as In fact, equation (6) can be converted to the following input-output linear system by introducing state variables The nonlinear system for variables h and z 2 Z transforms into as follows Therefore, a hybrid model of bipedal robots with uncertainties is considered as follows HU : ð Þ T are the state variables of the discrete system before and after the impact, Þ are uncertain terms. For convenience, equation (9) is converted into the following hybrid system HU : where Substituting equation (3) into equation (10), it can be described as Optimal robust controller for hybrid system According to the definition of the exponentially stable control Lyapunov function in the literature, 6 a rapidly exponential stabilizing control Lyapunov function is constructed for the nonlinear system equation (10) as follows where P is the solution of the following Lyapunov equation where K P and K D represent the gain matrices and Q is a positive definite matrix. For a hybrid system with uncertainties equation (9), an optimal robust controller is designed as follows. First, according to the rapidly exponential stabilizing control Lyapunov function, it will be directly calculated the derivative of equation (10) as follows Therefore, the following equation can be computed as To satisfy the decreasing condition of the rapidly exponential stabilizing control Lyapunov function, the following inequality can be generalized as Taking the energy consumption as the performance functional, the optimal robust controller is solved via a nonlinear optimization method, and then the dynamic walking of bipedal robot is realized and analyzed, the details are similar to the literature. 20 However, in this article, the algorithm is designed by fully considering the saturation constraints, unilateral constraint condition and maximum ratio and minimum normal ground reaction force of the control torques in the solving solution process. Therefore, the nonlinear optimization model is considered as follows where J u ð Þ is a continuous differentiable function in u, u min and u max represent the minimum and maximum torques for the motors, respectively. h q t ð Þ ð Þ means unilateral constraint condition, s s denotes the maximum ratio of tangential to normal ground reaction forces experienced by the stance leg end. t N 1 represents minimum normal ground reaction force experienced by the stance leg end. t T 1 means tangential to normal ground reaction forces.
Define (15) can be transformed into the linear equality constrained optimization problem represent variable and objective function, respectively, and A is a linear constraint matrix and b is a constant vector. The block diagram of the proposed control method is shown in Figure 1.

Linear equality constrained optimization algorithm and its global convergence
A modified projective Dai-Yuan nonlinear conjugate gradient method is proposed, analyzed, and investigated for online solving the optimal robust controller of the linear equality constrained optimization problem equation (16).
Assumption 3.2. The objective function f : R n ! R is continuously differentiable and bounded. The gradient of objective function satisfies the following Lipschitz continuous condition, that is, there is a positive constant L > 0 such that jjg x ð Þ À g y ð Þjj Ljjx À yjj; 8x; y 2 O Lemma 3.1. Assumption 3.1 and Assumption 3.2 are held.
The projected matrix is defined as follows P ¼ I À A T AA T À Á À1 A d k is generated by the step 2 of the modified projection Dai-Yuan conjugate gradient algorithm 1. If d k 6 ¼ 0, then d k is the feasible and descent direction.
Proof. A similar proof can be found in the study by Sun et al., 20 we will omit it.  Proof. According to the algorithm 1 and Assumption 3.1 and Assumption 3.2 in this article, we further assume that the following inequation holds, that is If there exists a positive constant e > 0, then jjPg k jj ! e; 8k ! 0 Now, the following two cases will be considered in this section.
Case 1. Due to Assumptions and liminf k!1 a k > 0, the equation can be obtained as It is obviously contradiction. The conclusion is true.
Moreover, due to the mean value theory and Taylor expansion theory, there exists a constant t k 2 0; 1 ð Þ, we have x k þ t k r À1 a k d 2 N , therefore Because of the search direction sequence d k f g is

Numerical simulation
The parameters of model with different constraints.
The three-link bipedal robot with the upper limb is considered as the numerical simulations, the physical structure of bipedal robot is shown in Figure 2.
The parameters of the three-link bipedal robot model are selected as the following The desired joint angle of upper limb is expected to be q d 3 ¼ p 9 and the ground angle is p 6 and a ¼ 0:5. The desired trajectory is as follows Grizzle et al. adopted the nonlinear optimization algorithm to solve the parameter values of desired trajectory and obtained a set of optimal parameter values as follows 1 a Ã ¼ 0:512 0:073 0:035 À 0:819 À 2:27 3:26 3:11 1:89 ½ Furthermore, we define the angular velocity q 1 as an initial angular velocity ! 1 ¼ 1:55 rad=s: The minimum energy consumption is defined as a performance functional for bipedal robots as follows 21,22 and the determination is defined as e < 10 À6 : First, solving the Lyapunov equations where A is calculated by equation (15), and Q is an arbitrarily positive definite matrix, and here it will be set as Q ¼ 100I 4Â4 . Using Matlab software toolbox to solve equation (15), the following equation can be computed as P ¼ for control input versus time. Let u 1 ¼ t 1 ; u 2 ¼ t 2 ; as shown in Figure 3, and using the Dai-Yuan projection conjugate gradient optimization algorithm to solve the optimal robust controller makes the control input increase with the number of walking steps of the robot, and the robot can achieve dynamic walking with minimum energy. Figure 4 represents the functional relationship between the output function and the time. As shown in Figure 4, when the foot of bipedal robot collides with the ground, the error of the output function changes greatly. Otherwise, it will change a little. From the Figure 4, it can be seen that, since the output function y 1 slight changes, it is also within the controllable for the stable walking. Therefore, the bipedal robot can achieve stable motion. The output function y 2 has a large variation when the collision occurs, and the error almost equal to zero in other time. Figures 5 and 6 represent the angular displacements q 1 ; q 2 and angular velocities ! 1 ; ! 2 of bipedal robots with upper limbs in the fifth step, respectively. It can be seen from the figures that the bipedal robots with upper limbs can achieve stable periodic motion. Angle Displacement/rad Figure 5. The limit cycle of angular displacement and angular velocity .q 1 with no parameters' perturbation. θ2 Angle Displacement/rad Angle Velocity/ (rad/sec) θ 2 Figure 6. The limit cycle of angular displacement q 2 and angular velocity .q 2 with no parameters' perturbation.   The other parameters are consistent with the case of nonperturbation. The numerical simulation results are shown in Figures 7 to 10. Figure 7 represents the control input versus time for bipedal robots. Let

Parameters' perturbation three times
it is known from the figure that although the parameters of dynamic mathematical model are changed three times for the bipedal robots, the robust and stable walking of the bipedal robots is achieved by using the optimal robust controller to effectively reject the perturbation of parameters. Figure 8 represents the functional curves between the output functions and the time t. From the output function y 1 , it is known that the error of the output function is not equal to zero, but the bipedal robot can still achieve dynamic walking. From the output function, it can be easily seen that the error of the output function y 2 changes only when the collision occurs, and the error remains at around 0 during the process of dynamic walking. Figures 9 and 10 represent the limit cycles of angular displacements q 1 ; q 2 and the angular velocities ! 1 ; ! 2 of the various joint angles within the five-step cycle for the bipedal robots. The numerical simulation results show that the bipedal robots with upper limbs can achieve stable cycle motion.

Numerical results with random noise
To investigate the feasibility and efficiency of the proposed approach, the nonlinear hybrid system with random noise is considered in this subsection. The details can be seen as follows Figure 9. The limit cycle of angular displacement q 1 and angular velocity .q 1 with three times parameters' perturbation. Figure 10. The limit cycle of angular displacement q 2 and angular velocity .q 2 with three times parameters' perturbation.  HUN : where HUN means the hybrid system with random noise, d t ð Þ denotes the random noise. The numerical simulation results are shown in Figures 11 to 14. Figure 11 means the control signals versus time for bipedal robots under the random noise condition. As shown in Figure 11, with the disturbance of random noise, the bipedal robot can walk smoothly and dynamically. It infers that the optimal robust controller which designed through nonlinear optimization technique can effectively reject the perturbation of random noise. Figure 12 shows that the output curves between the output functions and the time t. As shown in Figure 12, the error function of the output function and real function is not equal to zero while the foot of robot impacts with the ground, however, the bipedal robot can still achieve stable walking under the random noise condition. Figures 13 and 14 represent the limit cycles of angular displacements q 1 ; q 2 and the angular velocities ! 1 ; ! 2 of the various joint angles within the five-step cycle for the bipedal robots under the random noises condition. The numerical simulations also show that the bipedal robots with upper limbs can achieve stable cycle motion. In addition,  it infers that the feasibility and effectiveness of the proposed method can be utilized to the other classical nonlinear systems, for instance, the lower limb rehabilitation robot, manipulator, and upper limb rehabilitation robot.

Comparison with classical methods
Comparison with numerical algorithms. In this subsection, the performance of Dai-Yuan approach proposed in this article is to be compared with classical Hestenes-Stiefel method and Fletcher-Reeves method for all the testing problems in CUTE library. 21,23 The methods are implemented by Matlab 7.1 code in double precision arithmetic. The numerical examples are performed on a PC computer with CPU Pentium 4, 2.40 GHz, and Windows XP operation system. Due to the Dai-Yuan method of this article, the fraction of optimization minimization problems is plotted as a factor t for the best time. In Figure 15, the performance will be described as a comparison the CPU-time with the other classical methods. 20,23 As shown in Figure 15, first, the developed method of this article is better than the other classical methods in CPU-time. Second, the computational effect is more superior than the Hestenes-Stiefel method and Fletcher-Reeves method. 20,23 Last, the proposed approach of this article is more robust than the classical methods for all numerical problems. 20,23 It can be seen from numerical results that the proposed Dai-Yuan conjugate gradient method is feasible, effective, and superior than the classical approaches which can be represented as the Hestenes-Stiefel method and Fletcher-Reeves method. 20,23 In addition, on the one hand, the bipedal robots are high dimensional nonlinear dynamic multibody system, and furthermore, the walking legs are continued to accept to irregular shock. On the other hand, the motion control of bipedal robots involves the internal dynamics, the high order, nonlinear and strong coupling characteristic. Therefore, the high speed and high mobility mean the gait of a high standard of motor control and advanced algorithms. Therefore, as shown in Figure 15, it also infers that the method of this article may be suitable for the real-time control systems.
Comparison with different control methods. To analyze and investigate the superiority and efficiency of the projected Dai-Yuan method, the traditional control methods (which can be called as PID controller and sliding mode control (SMC) technique) will be compared with the developed approach of this article. The details can be shown in Figures 16 and 17. As shown in Figures 16 and 17, the sum error of angles/angular velocities performances of PID, SMC, and modified Dai-Yuan method are illustrated for online solving the robust controller of bipedal robot. It can be seen that the sum errors of PID and SMC are considerably large for the optimal robust controller. In other words, we can see that the PID and SMC perform worse in handling the optimal robust controller. However, the modified Dai-Yuan method could converge to the exact solution. Therefore, comparing with the PID controller and SMC controller, the numerical results further infer that the superior performance and effectiveness of the modified Dai-Yuan method for the optimal robust controller of bipedal robot.

Conclusions and future work
For bipedal robot model with uncertainty, the exponential stabilizing control Lyapunov function, hybrid zero dynamics, and control torque saturation conditions were utilized to transform the problem of online solving rapidly exponential stabilizing robust controller into a nonlinear equality constrained optimization problem. A modified projective Dai-Yuan conjugate gradient method was designed to online solve the optimal robust controller, which improved the computational efficiency of the algorithm and developed a new approach for the design of optimal robust controller. The numerical results showed that the optimal robust controller solved by the optimization algorithm which could effectively reject the model perturbation problem and random noise condition and realize the efficient and stable walking of the bipedal robots. Furthermore, some comparisons were reported which means the proposed method is feasible and effective in this article.
In the future, some rapidly and effectively algorithm should be considered for online solving linear equation constrained optimization problems for optimal robust controller of bipedal robots. The proposed method may open a door to the performance improvement of the related applications, such as, the applications of neural network, 24,25 rehabilitation robots, 26,27 and optimization problems, 28,29 with great capacity in real-time property and computing accuracy. Furthermore, the reset map function of nonlinear hybrid system will be considered and designed a general controller in a unified framework for the bipedal robots. In addition, we will consider the effect of the bipedal robot which is walking on the uneven ground.