Optimized stable gait planning of biped robot using multi-objective evolutionary JAYA algorithm

This article proposes a new stable biped walking pattern generator with preset step-length value, optimized by multi-objective JAYA algorithm. The biped robot is modeled as a kinetic chain of 11 links connected by 10 joints. The inverse kinematics of the biped is applied to derive the specified biped hip and feet positions. The two objectives related to the biped walking stability and the biped to follow the preset step-length magnitude have been fully investigated and Pareto optimal front of solutions has been acquired. To demonstrate the effectiveness and superiority of proposed multi-objective JAYA, the results are compared to those of MO-PSO and MO-NSGA-2 optimization approaches. The simulation and experiment results investigated over the real small-scaled biped HUBOT-4 assert that the multi-objective JAYA technique ensures an outperforming effective and stable gait planning and walking for biped with accurate preset step-length value.


Introduction
Nowadays, humanoid robots have attracted the attention from numerous scientists. 1 The fact is that biped robot research have frequently met with big obstacles not only in how to realize the humanlike locomotion but also in how to effectively and efficiently control. On study of Kajita,2 as to have biped to naturally walk, it needs a walking pattern generator (WPG) through which biped can realize the required step.
WPG is created using zero moment point (ZMP) criteria in which it exists two principal design methods: the first relies on an inverse pendubot structure and the second improves from biped foot-hip trajectorial tracking. The first approach model is introduced by  in which numerous studies have investigated the 3D inverse pendubot as to use in stable biped control. The second model is initiatively proposed by Huang 6 in which various constraints have been applied to biped feet-hips and then implementing the biped step trajectory description. Using the obtained hip-leg-feet joint trajectories, a ZMP-based computation procedure is created to choose the parameters, so that the biped stably walks in equilibrium. Nevertheless, it seems too hard to end the ZMP point right at the base center in case biped formed in the single-surface-support single support phase (SSP) process. As a consequent, humanoid can be fallen in staggering in its walking, while the fact shows that the ZMP orbit in SSP stage usually reaches forward. [7][8][9] As to surpass such drawback, Erbatur et al. 10 suggested a method in which referential trajectory is built using the inverse pendubot-based ZMP orbit, linking with Fourier analysis to design an appropriate resolution. Wei Xu et al. 11 also improved the reference trajectory algorithm relied on ZMP orbit, so that humanoid robots achieve a more natural gait during the SSP process. Dip et al. 12 introduced a gait planning by only relying on four principal coefficients efficiently tested on small-scaled bipeds. 12 Unlike previous approaches, in Park and Oh, 13 the proposed pattern generation method only takes into account a current capture point (CP), a target CP (initial and final conditions), a step time, and a double-support phase time as walking parameters. Then Park and Oh successfully suggested a new real-time pattern generation for biped robot using an analytic method based on CP.
Recently, research that focused on advanced biped gait planning identification and control have increasingly developed. 14-16 Jeong et al. 17 introduced a robust walking controller optimizing step position and step time that exploit advantages of footed biped. Kim et al. 18 proposed a new stability control method for dynamic walking of bipedal robot with real-time CP trajectory optimization. Jeong et al. 19 continually developed a new robust walking controller based on online optimization of ankle, hip, and stepping strategies. Especially, meta-heuristic optimization algorithms and soft-computing tools have been successfully applied in optimal biped walking control. Elhosseini et al. 20 proposed a novel approach for biped stable walking based on an alternative characterization (AC) parametric whale optimization algorithm. Yang et al. 21 suggested an energy efficient walking control for biped robots using interval type-2 fuzzy logic systems and optimized iteration algorithm. Furthermore, multi-objective (MO) optimization approaches have also been effectively used in developing biped optimum gait WPG planning methods. Raj et al. 22 proposed a new way for MO optimized bipedal locomotion. Pentosa-Garcia et al. 23 introduced a new synergistic design of biped lower limb using MO differential evolution algorithm, among them.
Those studies above mentioned show that these recent results that focused on the biped WPG design and implementation have been considered as an MO optimization task in which every objective is presented as a relation with respect to determined variables. The difficulty relates to the fact that those objectives seem to contradict one another, then the optimum Pareto front of solutions should be used to solve. As to efficiently and satisfactorily solve such MO optimization problems, the studies of Rao 24 and Berrouk et al. 25 are worth mentioning. In Rao, 24 a new metaheuristic called JAYA (a Sanskrit word meaning victory) is developed. JAYA is a population-based optimization algorithm inspired from the idea that an optimal solution of a certain problem tends to move away from the worst solution and, simultaneously, finding its way to the best solution. Moreover, JAYA has a major distinction over other optimization algorithms because it has no specific parameters which must be carefully selected and needs only common parameters like population size and the maximum number of iterations. 24 A multi-objective JAYA algorithm noted as MO-JAYA is developed in Berrouk et al. 25 Improving the results abovementioned especially the idea of using MO-JAYA in Berrouk et al. 25 for developing a biped WPG planning based on four key parameters, 12 in this article, an attempt has been made to formulate the optimal biped walking gait planning WPG for a 10-degree of freedom (DOF) biped robot with preset step length using an MO optimization algorithm. The biped robot is modeled as a kinetic chain of 11 links connected by 10 joints. The inverse kinematics of the biped is derived using the specified positions of the hips and feet. Two objectives related to the biped walking stability and the biped to follow the preset step-length magnitude have been investigated, using three comparative MO optimization algorithms, including MO-NSGA-2, 26 MO-PSO, 27 and proposed MO-JAYA techniques, are used to issue the optimum Pareto front. A comparative analysis of three MO optimization approaches is satisfactorily illustrated.
The organization of this article is structured as follows: the second section deals with the mathematical formulation of the biped which is in investigation. The third section presents the new MO-JAYA-based WPG technique to solve the optimal biped walking gait planning problem. Simulation and experiment results are adequately presented and analyzed in the fourth section. The fifth section concludes the article.

Mathematical formulation of the problem
This article proposes an investigation of a 10-DOF (comprising four at hips, two at knees, and four at ankles) humanoid robot, via a biped gait WPG optimized by MO-JAYA algorithm for ensuring a robust walking with preset steplength value. The biped walking configuration in one step cycle is illustrated in Figure 1.
The link mass is supposed to be lumped at the joint ends. For simplicity, the movement of the robot is considered in one direction. As illustrated in Figure 1, the three main orbits of biped locomotion are composed of hip orbit P 5 ¼ [P 5x , P 5y , P 5z ], supporting leg's ankle orbit P 1 ¼ [P 1x , P 1y , P 1z ], and swing leg's ankle orbit P 10 ¼ [P 10x , P 10y , P 10z ].
Furthermore, Figure 1 also shows that the biped gait WPG planning, developed in this article, is designed relying on four key variables (S ¼ step length of biped, H ¼ the height of foot lift, h ¼ the vertical variation of hip, n ¼ the horizontal variation of biped hip) regarding to both of the frontal (YZ-frontal) and sagittal (XZ-sagittal) view. The three chosen orbits P 1 , P 5 , and P 10 are investigated sine-time dependently and fully presented in the equation (1) (consulted more in Huan et al. 28 ).
with T denotes biped step-cycle time and w denotes the distance between two legs. Based on the biped inverse kinetic equations, the orbits of 10 rotating joint angles of biped in one step-cycle locomotion are fully illustrated using P 1 , P 5 , and P 10 variables. The geometric method developed from biped rotating joint-angle orbits will be used to calculate and describe in the equation (2), in which x l , y l , z l , x r , y r , z r , l l , l r , q A , q B , q C , and q D represent the inverse kinetic parameters (consulted more at Huan and Huy Anh 29 ).
In this study, the biped keeps balancing walking based on ZMP principle; in more detail, during ZMP, point locates within the support-foot surface. Then the calculation of ZMP location regarding to the rotating joint ankle values is determined in equation (3) (consulted more in Huang et al. 6 ).
where m i denotes the ith link mass, (P xi , P yi , P zi ) represents the location of the ith link center; I ix , I iy denotes inertia component values; € q ix and € q iy represent the rotating angular accelerating components of the ith joint; grepresents gravitation force; € P xi , € P yi represents the acceleration of link ith in x-and y-direction, respectively; (x ZMP , y ZMP ) represent the ZMP co-ordination.
The position of each link, P i ¼ (P ix , P yi , P zi ), is computed from the 10 angles of rotation (q 1 , q 2 , q 3 , q 4 , q 5 , q 6 , q 7 , q 8 , q 9 , and q 10 ) along with biped step cycle based on support-foot center coordination using direct kinematics (see additional consults at Huan et al. 29 ). The angular velocity _ q i ðtÞ and acceleration € q i ðtÞ are calculated from the q i ðtÞ derivation to time. Similarly, the velocity _ P i ðtÞ and acceleration € P i ðtÞ of the ith link are calculated using the P i ðtÞ derivation to time.
The biped stable walking performance is quantitatively verified by the sum of difference of ZMP with foot-center position in a step-cycle time. Hence, biped gait with best robustness are achieved by minimally optimizing the objective equation f 1 determined in (4) 12 where T represents step-cycle time; ðx ZMP ; y ZMP Þ represents the location of ZMP in the biped walking process. Additionally, the second objective of this study is to force biped to track precisely the preset step-length magnitude (S ref ) which is quantitatively verified by the sum of deviation of calculated step length (S) magnitude with preset step length (S ref ) one described in equation (5) Then the constraint-based MO optimization task can be eventually described in equation (6)

Proposed algorithm
Proposed gait planning WPG design and implementation for the biped stable walking with preset step length has been adequately modeled as shown in Figure 2(a).

Explain algorithm diagram
The WPG is composed of the biped gait provider of the twofeet orbit (P 1 and P 2 ), biped hip orbit (P 5 ), and the full biped inverse kinetic equations, described in equations (1) and (2). The MO-JAYA algorithm is proposed to find out the best optimum gait coefficients (S, H, h, n) which generate the desired gait WPG planning including 10 joint-rotating values (q 1 , q 2 , q 3 , q 4 , q 5 , q 6 , q 7 , q 8 , q 9 , q 10 ) which ensure biped stable and robust walking with selected accurate step length. The optimized WPG ensures the biped ZMP locomotion as presented in equation (3). The MO-JAYA objective function is applied to optimally select the four gait coefficients, which guarantee biped stable and effective walking with precise chosen step-length magnitude, as illustrated in equation (6).

MO-JAYA optimization algorithm
The novel MO-JAYA is improved from JAYA 14 which firstly suggested in 2016 by RV Rao. The main steps of JAYA are clearly presented in Table 1. The main steps of MO-JAYA algorithm, initiatively developed from F. Berrouk et al., 15 are clearly described in Table 2 and in the flowchart illustrated in Figure 2(b). In this flowchart, the candidates of the solution are moved and updated using the following expression: X New ðiÞ ¼ X Old ðiÞ þ r 1 Â X Best ðiÞ À X Old ðiÞ À r 2 Â X Worst ðiÞ À X Old ðiÞ 8i where r 1 and r 2 are two random numbers in the range [0,1], X Best (i) and X Worst (i) are the best and worst candidates obtained at the ith iteration, respectively. For a given candidate, if the new solution (after moving) is better than the old one (before moving), the solution is accepted and the Table 1. The main steps of JAYA algorithm.
Step 1 Initialization: In this step, a population of N candidates that has a predefined size is randomly generated in the search space Step 2 Identification of best and worst solutions: After the initialization of the population, the best and worst candidates among the population in terms of objective functions are identified Step 3 In this step, the candidates of the solution are moved using the following expression: where r 1 and r 2 are two random numbers in the range [0,1], X Best (i) and X Worst (i) are the best and worst candidates obtained at the ith iteration, respectively. For a given candidate, if the new solution (after moving) is better than the old one (before moving), the solution is accepted, and the corresponding candidate is updated. Otherwise, the new solution is discarded and the old one is kept The process in step 2 and step 3 is repeated until a termination criterion is reached. The termination criterion chosen for JAYA is the maximum number of iterations and this criterion can be changed and any other criterion can be implemented Table 2. The main steps of MO-JAYA.
Step 1 Initialization: In this step, a startup population noted as Pop (0)  and Pop archive (ith) is calculated using the following procedure: For each candidate i, in the population Pop(ith) and in the archive Pop archive (ith), a strength S(i) is assigned, which represents the number of candidates it dominates. Based on S(i), the raw fitness R(i) of a candidate i is determined by following expression It is worth to mention that, since the multi-objective pareto front (MOPF) problem is formulated as a minimization one, R(i) ¼ 0 corresponds to a nondominated candidate (a solution from the Pareto front), while a high value of R(i) is synonym to that this candidate is dominated by many candidates After that, a density D(i) is calculated using the following expression: where s k is the kth nearest neighbor (based on the distance in the objective space . The 2 value is added in the expression of D(i) to ensure that this one is comprised between 0 and 1; N denotes number of particles and N A is archive size. Finally, fitness function F(i) is calculated based on followed expression: Step 3  responding candidate is updated. Otherwise, the new solution is discarded and the old one is kept.

Results and analysis
Image and model of the small-scaled biped robot (HUBOT-4) are illustrated in Figure 3.
The small-sized biped HUBOT-4 composes of links with masses (in gram) as follows: m ¼ 70 g and m 0 ¼ 100 g. The links are designed with following lengths (in centimeter): cm. The step-cycle time T is selected 1 s. The step length of biped swing foot is chosen 10 cm. In simulation and experiment, the values of, S min , S max , H min , H max , h min , h max , n min , n max (in centimeter) have been set as 8, 12, 1.0, 12, 0.1, 12, 0.1, and 12 cm, respectively. Simulation results are carried with MATLAB platform run with 2.49 GHz CPU and 8 GB RAM.

Results of constrained MO optimization
Test case 1. Survey the performance of MO-JAYA algorithm in optimization biped walking with preset step length and gait WPG planning of biped robot by changing the population number of particles (NP). Selecting Generations (GEN) ¼ 1000; NP ¼ 30, 32, 34, and 40, respectively. Figure 4 illustrates the resulted optimum Pareto fronts achieved with the MO-JAYA method for the biped stable walking with preset step length. These results show that the MO-JAYA technique attains the best optimization performance with NP ¼ 32.
Test case 2. Survey the performance of MO-JAYA algorithm in optimum biped stable walking and gait WPG planning with preset step length by comparing MO-NSGA-2 and MO-PSO algorithms. Constraint-based optimal process combined with reward factor is used to find out the global optimum solution. The following MO-NSGA-2 coefficients are selected: stochastic crossover factor p c ¼ 0.7, stochastic mutation factor p m ¼ 0.4, mutation rate mu ¼ 0.02, maximal amount of iterations ¼ 1000, and number of particles ¼ 32 for the biped stable walking with preset step length. Similarly, the following MO-PSO coefficients are chosen for the global optimum solutions: inertia weight w ¼ 0.4, individual confidence factor c 1 ¼ 2, swarm confidence factor c 2 ¼ 2, maximal amount of iterations ¼ 1000, and number of particles ¼ 32 for this test case.
In the same way, the following MO-JAYA coefficients are carefully selected for the global optimum Pareto front of solutions: maximal amount of iterations ¼ 1000,  population size ¼ 32 for the biped stable walking with preset step length. Figure 5 illustrates the optimum Pareto fronts of solutions optimally computed via MO-NSGA-2, MO-PSO, and proposed MO-JAYA algorithm for the biped stable walking with preset step length and optimum gait WPG planning. In this case, it is clear to notice that MO-JAYA algorithm has outperformed the MO-NSGA-2 and MO-PSO methods. It is important to explain these results related to the fact that MO-JAYA is able to simultaneously provide both global and local optimum solutions, while the MO-NSGA-2 and MO-PSO are worse in searching global optimum solutions.
The results from Figure 5 show that optimum Paretofront solutions of multiple-objective optimization (minimizing the sum of difference between ZMP with foot-center location in a step cycle and minimizing the deviation of calculated step length S with preset step length S ref values) does not pose a unique solution. Table 3 presents Pareto-optimal fronts of solutions for optimization walking gait planning by MO-JAYA algorithms. The best solution of MO optimization is tick in Figure 5. Based on Table 3 and Figure 5, the best solution is S ¼ 10 cm, H ¼ 1 cm, h ¼ 0.76 cm, n ¼ 7.12 cm.

Results of tested Pareto-optimal front of solutions by MO-JAYA algorithm
This subsection performs simulation and experiment test of biped HUBOT-4's walking process with a set of gait parameters optimized by MO-JAYA algorithm in one step cycle. Figure 6 illustrates the 3D gait of the HUBOT-4, regarding to preset step-length distance. Results from Table 3 and Figure 6 show that the biped HUBOT-4 attains the step length in terms of the preset value. Figure 7 illustrates the resulted ZMP point trajectory for preset step-length amplitudes. This result clearly confirms that the ZMP trajectory is always inside the supporting foot area and then it ensures that the HUBOT-4 biped robot surely keeps stable walking.    Figure 9 demonstrates the 10 rotary angular trajectories in one stepping cycle of the two legs of biped HUBOT-4 during walking with an optimally resulted set of gait parameters with respect to selected step-length value. Figure 10 shows the 10 rotary angular trajectorial errors within one stepping cycle of the experimental biped two legs during walking with optimally resulted set of gait parameters in terms of preset step-length value S ref ¼ 10 cm.
Based on the results of the optimization and simulation presented in Table 3, Figure 6 and Figure 7 and the experimental results presented in Figures 8 to 10, which once more demonstrate that the work of preset step-length parameter (S ref ) using four optimally selected parameters (Sstep length, H-foot lifting, h-kneeling, and n-hip swinging)       ensuring the HUBOT-4 biped robot to stably walking without falling apart and keeping pace with desired step-length value. As a consequent, the proposed algorithm with gait parameters optimized by MO-JAYA algorithm is convincingly feasible.

Conclusions
This article proposes an optimal gait planning WPG for a 10-DOF biped able to robustly walking with preset steplength value. The proposed WPG is considered with two objectives, namely, minimum the difference of ZMP with foot center in a step cycle and minimum the deviation of the calculated step length with the preset step-length values. Then this constrained optimization problem has been separately solved using the MO-NSGA-2, MO-PSO, and MO-JAYA optimization algorithms as to achieve separate optimum Pareto fronts. The results in this article prove fit with the general human experience. The simulation and experiment results demonstrate that the proposed MO-JAYA performs convincingly better than the MO-NSGA-2 and MO-PSO, as the MO-JAYA simultaneously gives both global and local solutions, while the MO-NSGA-2 and MO-PSO only show a potential ability for global solution searching. The resulted optimum Pareto fronts will be quite useful for designer to choose optimum MO solutions available regarding to the complicated MO constraints which would be investigated in further studies.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the financial support for the research, authorship, and/or publication of this article: This work belongs to the project grant No: T2021-75TÐ funded by Ho Chi Minh City University of Technology and Education, Vietnam.