Knee-stretched walking with toe-off and heel-strike for a position-controlled humanoid robot based on model predictive control

Knee-stretched walking is considered to be a human-like and energy-efficient gait. The strategy of extending legs to obtain vertical center of mass trajectory is commonly used to avoid the problem of singularities in knee-stretched gait generation. However, knee-stretched gait generation utilizing this strategy with toe-off and heel-strike has kinematics conflicts at transition moments between single support and double support phases. In this article, a knee-stretched walking generation with toe-off and heel-strike for the position-controlled humanoid robot has been proposed. The position constraints of center of mass have been considered in the gait generation to avoid the kinematics conflicts based on model predictive control. The method has been verified in simulation and validated in experiment.


Introduction
Most biped walking studies use a knee-bent posture to avoid the problem of singularities. Nevertheless, kneebent walking is not only extremely unnatural but also high cost in terms of power consumption compared with human walking. 1,2 Knee-stretched walking is more human-like and energy-efficient than knee-bent walking, 3 though the problem of singularities arises. It can also achieve longer strides than knee-bent walking. 2 To date, knee-stretched walking has been extensively discussed in passive walking studies, [4][5][6][7] but few studies about knee-stretched walking have been conducted in fully-actuated humanoid robots. 8 Recently, utilizing whole-body dynamics control on the fully-actuated torque-controlled robot is a big trend in this field, but these methods based on accurate modelling and their validity have not been confirmed on the positioncontrolled robot yet. 9,10 Therefore, the position-based method is still worth being researched for its fundamental simplicity and robustness. 11 Knee-stretched walking with fully-actuated positioncontrolled robots has been achieved by exploiting special mechanisms to avoid the problem of singularities. Ogura et al. developed a pelvis with an extra roll joint and a foot with a toe joint. [12][13][14] Handharu et al. designed a foot with both toe and heel joints. 15 Sadedel et al. equipped active toe joints on robot feet. 16 All of them solve the problem of singularities by increasing the degrees of freedom (DOFs) of mechanisms to increase the control authority. Unfortunately, this increases manufacturing costs and control difficulties. To avoid these shortcomings, some studies have solved the problem of singularities by predefining vertical center of mass (CoM) trajectory and then controlling dynamics with the zero moment point (ZMP). 17,18 Nonetheless, an appropriate vertical CoM trajectory is always computationally complex since it must carefully avoid singularities.
To overcome the difficulty of complex vertical CoM trajectory planning, the strategy of fully extending the robots' legs within a limited knee range of motion was introduced. With such strategy, an appropriate vertical CoM trajectory can easily be obtained, and the neighborhood near singularity point is avoided in the limited range of motion. Some of the researchers that utilized this concept failed to exploit the toe-off and heel-strike. [19][20][21][22] Toe-off and heel-strike are important characteristics of human-like walking, and they are helpful to enhance step length and energy efficiency. [23][24][25] Kajita et al. first designed the knee-stretched walking pattern with toe-off and heel-strike in the spatial domain and then transformed the pattern into a time-domain trajectory based on an optimized CoM velocity trajectory. 26 Successful long-stride knee-stretched walking was accomplished with humanoid robotics platform HRP-2Kai, 27 and yet this method is necessary to sacrifice speed tracking accuracy to ensure walking stability because of the trade-off between ZMP tracking accuracy and speed tracking accuracy in the cost function of CoM velocity optimization. Fortunately, this problem does not exist if gait generation is designed in the time domain. Miura et al. designed the walking pattern in time-domain and utilized toe-off and heel-strike motion while forcing the legs to extend with given horizontal CoM trajectory. A stable knee-stretched walking has been accomplished on the HRP-4C robot. 28 The horizontal CoM trajectory generation of this method is based on an analytical solver. 29 In the analytical solver, the relative horizontal positions between the CoM and the midpoint of the corresponding step at the transition moment between single support and double support phases are stochastic. Nevertheless, kinematic conflicts will occur at the transition moment if using the strategy of extending legs with the stochastic relative horizontal position. Therefore, this method requires extra optimization to lower the vertical CoM generated by extending legs so as to overcome the kinematic conflicts. As a result, the optimized vertical CoM trajectory cannot guarantee the desired knee-stretched posture. Moreover, the method was based, to some extent, on extra toe links.
To solve these deficiencies, this article proposes a timedomain knee-stretched gait generation with toe-off and heel-strike for position-controlled humanoid robots. The position constraints of horizontal CoM have been considered in the proposed gait generation based on model predictive control (MPC) to avoid the kinematic conflicts while using the strategy of extending the legs. The content of this method can be summarized as follows. The CoM trajectory planning strategy of the proposed knee-stretched walking is decoupled into the sagittal plane and the lateral plane. In the sagittal plane, gait generation is further divided into vertical and horizontal CoM trajectory planning. For vertical CoM trajectory planning, legs are extended as much as possible within a limited knee range of motion using toe-off and heel-strike motion under a desired horizontal CoM trajectory. In addition, the desired horizontal CoM trajectory ensures the walking stability in the sagittal plane, and the limited knee range of motion does not include the neighborhood near singularity point. For the desired horizontal trajectory planning, the MPC is utilized to constrain the horizontal CoM position, particularly at transitions between single and double support phases. The closed-form solution of preview control 30 instead of constrained MPC is exploited to generate the lateral CoM trajectory, which accelerates computations. Moreover, available ankle trajectories planning is also provided. In the end, the presented method is validated both in simulation and in experiment on a position-controlled humanoid robot BHR6.
The rest of the article is structured as follows. The second section explicates knee-stretched gait generation with toe-off and heel-strike motion. The third section presents simulation and experimental validation. The fourth section concludes the article and suggests future research.

Knee-stretched gait generation
The CoM trajectory planning strategy is shown in Figure 1, in which the lateral, horizontal, and vertical trajectories are along x-axis (left-right axis), y-axis (forward-backward axis), and z-axis (upward-downward axis), respectively. Generally, the horizontal CoM position is equal to the horizontal position of the hip joint, the lateral CoM position is equal to the lateral position of the midpoint between two hip joints, and the CoM remains a constant height difference with the vertical position of the hip joint. Since the vertical position of the hip joint is used in inverse kinematics instead of the vertical position of the CoM, we regard these two vertical positions as equivalent in this article.

Vertical CoM trajectory planning
In vertical CoM trajectory planning, the legs fully extend, exploiting the toe-off and heel-strike, under a desired horizontal CoM trajectory; the desired horizontal CoM trajectory ensures the stability in the sagittal plane. The vertical CoM position is determined by the minimum of the maximum CoM height range that legs can achieve. If the leg has full contact with the ground, the maximum available CoM height is z Max a (Figure 2(a)). If the leg has line contact with the ground, the leg exploits the available extension range by stretching with heel-strike and toe-off motion, and the maximum available CoM height is z Max b (Figure 2(b) and (c)). The maximum available CoM height is given in the following equation where H Ankle is the vertical distance from the ankle joint to the sole of the foot, d is the horizontal distance between the CoM and the heel or toe, L Max a is the maximum length between the hip and the ankle in the sagittal plane corresponding to the knee range of motion, L Max b is the maximum length between the hip and the toe or heel in the sagittal plane corresponding to the knee range of motion, and y CoM is the desired horizontal position of the current desired CoM. Note that the neighborhood near singularity point is not within the knee range of motion.
To determine the minimum value of the maximum available CoM height in a single support phase (Figure 3(a)), the supporting leg is required to have full contact to maintain stability, while the swinging leg does not need to be considered (because it is off the ground). Therefore, the vertical CoM position z CoM during the single support phase is determined only by the supporting leg and is given as follows To determine the minimum value of the maximum available CoM height in the double support phase (Figure 3 CoM in the sagittal plane, the corresponding leg has full contact with the ground. If the ankle is farther from the projection point of the current desired CoM in the sagittal plane, the corresponding leg has line contact with the ground. Therefore, the vertical CoM position during the double support phase can be expressed as follows Note that this vertical CoM position must be modified when used in inverse kinematics because it is calculated with legs fully extended in the sagittal plane, that is, there may be no inverse kinematics solution if the desired lateral and sagittal CoM positions are directly combined together as the desired CoM position. In addition, the desired lateral CoM position ensures the stability in the lateral plane, and the desired sagittal CoM position ensures the stability in the sagittal plane and the kneestretched posture. As shown in Figure 4(a), the robot outlined in black is conducting the desired sagittal CoM position with no lateral movement. The robot outlined with gray-dotted line is conducting the desired sagittal CoM position and the desired lateral CoM position without modification of vertical CoM position, of which the maximum length limitation of the leg is violated. The robot outlined in red is conducting the desired knee-stretched motion which is stable in the sagittal plane and the desired lateral CoM position with modification of vertical CoM position, of which the maximum length limitation of the leg is satisfied. Therefore, to realize both the desired knee-stretched motion which is stable in the sagittal plane and the desired lateral CoM position, the vertical CoM position z CoM m used in inverse kinematics has to be modified as follows where x CoM is the desired lateral CoM position which ensures the stability in the lateral plane. A geometric explanation of equation (4) is shown in Figure 4(b).

Horizontal CoM trajectory planning
The horizontal CoM position is required to be constrained, if the vertical CoM trajectory is obtained by extending the knees with toe-off and heel-strike. Otherwise, the vertical CoM position will have a sudden change at the transition moments between single and double support phases as shown in Figures 5 and 6. Time t a indicates the transition moment from a single support phase to a double support phase, and time t b indicates the transition moment from a double support phase to a single support phase. Point Mid is the midpoint between current desired footsteps during a double support phase and lies on the ground. In Figure 5, the robot is at transition moment t a and its CoM position is in front of point Mid. In Figure 5(a), the hind leg is the support leg and the front leg is the swing leg. The vertical CoM is determined by the support leg. The swing leg has to bend to avoid the collision with the ground. In Figure 5(b), the vertical CoM is determined by both legs. Both legs stretch as much as possible with the help of the toe-off motion conducted by the hind leg. Therefore, the vertical CoM position will have a sudden upward change at the transition moment t a . In Figure 6, the robot is at transition moment t b and its CoM position is behind point Mid. In Figure 6(a), the vertical CoM is determined by both legs. Both legs stretch as much as possible with the help of the heel-strike motion conducted by the front leg. In Figure 6(b), the vertical CoM is determined by the support leg. The front leg becomes the support leg, so it has to be fully contacted with the ground at once. On the other hand, the hind leg becomes the swing leg, so it has to bend at once to avoid the collision with the ground. Therefore, the CoM position will have a sudden downward change at the transition moment t b .
To sum up, the horizontal CoM position must be behind the horizontal position of point Mid at transition moment t a and in front of point Mid at transition moment t b to ensure the smoothness of the vertical CoM trajectory. The ideal horizontal CoM positions of the robot at the transition moments are shown in Figure 7. With these horizontal CoM position constraints, the robot can achieve the desired kneestretched posture generated by extending legs during the whole walking procedure.
Therefore, the constraints of horizontal CoM position for the entire duration of every step can be given as follows y CoM y Mid when t ¼ t dst y CoM ! y Mid when t ¼ t ded (5) where y Mid is the horizontal position of point Mid, t dst , and t ded are the start and end times, respectively, of the double support phase.
To satisfy the position constraints in equation (5), a linear constrained MPC is used to generate the horizontal CoM trajectory. The linear inverted pendulum model is selected as the simplified dynamics model for kneestretched walking in the sagittal plane. A state vector that comprises horizontal CoM position, velocity, and acceleration of horizontal CoM is defined as follows where T is a constant time interval. The input vector, comprising horizontal CoM jerk, is given as follows The output vector, comprising horizontal ZMP position and horizontal CoM position, is given as follows Thus, the discrete system dynamics is given as follows where where h CoM is an approximate value of the actual average vertical CoM position, which has a constant height difference with the average vertical CoM position generated, and g is gravitational acceleration. The position constraints of horizontal ZMP and horizontal CoM are given as follows where p max yk , p min yk , y max k , and y min k are the maximum and the minimum values of horizontal ZMP position and the maximum and the minimum values of horizontal CoM position, respectively, which are determined by the desired footsteps at time kT .
The cost function minimizes the jerk of the horizontal CoM trajectory, which prevents the horizontal CoM position from diverging. 31 Therefore, this corresponds to solving the following constrained optimal control problem at kT under the constraint shown in equation (12), which is a standard formulation of linear constrained MPC problem

Lateral CoM trajectory planning
The same simplified dynamics model and similar state and input vectors as those used for the sagittal plane were selected for the knee-stretched walking in the lateral plane, thus the discrete system dynamics is given as follows ð Þ, x kT ð Þ, and p x kT ð Þ are the lateral positions of CoM and ZMP at kT, respectively, A x ¼ A y , B x ¼ B y , and C x ¼ 1 0 Àh CoM =g ½ . To accelerate computations, we exploited the closedform solution of preview control 30 instead of using linear constrained MPC. Therefore, to track the lateral reference ZMP trajectory px r which was determined by the desired footsteps, the cost function J and the manipulate variable u xk at every kT can be given as follows where px r i is the lateral ZMP reference position at iT ; 4u xi and 4x i are the increments of u x and x at iT , respectively; N is the prediction horizon; and G i , G x , and G p can be calculated from Q e , Q x , R, and the system parameters of equation (13).

Ankle trajectory planning
Ankle trajectory planning can also be divided into the sagittal plane and the lateral plane. In the lateral plane, the positions and postures of the ankles are directly equivalent to the positions and postures of desired footsteps; therefore, we only need to discuss the ankle trajectory planning in the sagittal plane.
In the double support phase shown in Figure 8  ankle are determined by L Max b and the heel or toe position of the desired footstep. In the single support phase shown in Figure 8(c), the desired position and posture of the supporting ankle are determined by its desired footstep. For the position and posture of swinging ankle, we first interpolate the hip pitch joint from joint angle b st to joint angle b ed with a cubic spline curve and set the distance between hip and ankle in the sagittal plane as L Max a . Therefore, the horizontal trajectory y Ankle swing and a temporary vertical trajectory z Ankle tmp1 of the swinging ankle position can be obtained by geometric calculations, which guarantees an inverse kinematics solution. Second, we interpolate another temporary vertical trajectory z Ankle tmp2 of the swinging ankle position from ankle height z Ankle st to ankle height z Ankle ed and halfway through height z Ankle mid with a cubic spline curve. Note that z Ankle mid > max z Ankle st ; z Ankle ed À Á . Then, we interpolate the posture angle in the sagittal plane a of the swinging ankle from posture angle a st to posture angle a ed with a cubic spline curve. Finally, the vertical trajectory of the swinging ankle position z Ankle swing is given as follows which guarantees clearance between the swinging foot and the ground.

Humanoid robot BHR6
The validity of the presented method is verified both in simulation and in experiment on humanoid robot BHR6. BHR6 is a 21-DOF, position-controlled robot; its appearance and configuration are shown in Figure 9. In addition, the vertical distance between the ankle joint and the sole of the foot is 0.112 m in the simulation and 0.126 m in the experiment. This vertical distance difference is because of the rubber pads that are installed under the actual robot feet to absorb the landing impact.

Simulation
The simulated robot built-in simulation platform V-REP has the same physical parameters as the actual robot, except for the vertical distance between the ankle joint and the sole of the foot. The coefficient of static friction of the ground is set to 0.8. A six-step knee-stretched walking with toe-off and heelstrike has been conducted and its snapshots are shown in Figure 10. Note that an ankle compliance control strategy similar to the one based on double spring damping model 32 was used in this simulation for better walking performance. The step duration was 0.6 s, the step length was 0.4 m, and the walking speed was 2.4 km/h. To avoid singularities, the maximum leg length was achieved when the angle of knee joint is À4 . For better stability, the sagittal ZMP position constraint in equation (11)   where yzmp max and yzmp min are determined by the boundaries of the range of the robot support polygons; and L fb and L ff are the back and the front lengths of foot, respectively. The desired trajectories of CoM, ZMP, and knee joints generated from the proposed method are shown in Figures 11 to 13, respectively. As shown in Figure 13, the knee joints only bent while the according legs being the swing legs in single support phases, which means the desired knee-stretched posture generated by extending legs can be obtained from the proposed method.
The ZMP trajectories measured during knee-stretched walking are shown in Figure 14. Both sagittal and lateral ZMP trajectories are within the boundaries of the range of the robot support polygons; therefore, the results  demonstrate that stable knee-stretched walking with toe-off and heel-strike has been achieved, verifying the proposed gait generation in simulation.
Moreover, a comparison of energy consumption between a six-step knee-stretched walking and six-step knee-bent walking is also conducted in simulation, and the result is shown in Figure 15

Experiment
Snapshots of a four-step knee-stretched walking experiment with toe-off and heel-strike motion are shown in Figure 16. Note that a torso compliance control strategy similar to trunk position compliance control 33,34 was used in this experiment for better walking performance. The step duration was 0.8 s and the step length was 0.25 m, thus the walking speed was 1.125 km/h. To avoid singularities, the maximum leg length is achieved when the angle of knee joint was À4 . For better stability, the sagittal ZMP position constraint in equation (11) becomes yzmp min þ 0:87L fb p y yzmp min À 0:90L ff (17) where yzmp max and yzmp min are determined by the boundaries of the range of the robot support polygons; and L fb and    L ff are the back and the front lengths of foot, respectively. The desired trajectories of CoM, ZMP, and knee joints are shown in Figures 17 to 19, respectively. As shown in Figure 19, the knee joints were stretched except for the period of their according legs being the swing legs, which means the desired knee-stretched posture generated by extending legs can be obtained from the proposed method. ZMP trajectories measured in the experiment are shown in Figure 20. Although the ZMP values around time 5 s exceed the boundaries, which are mainly caused by the modeling errors and joint flexibility, the subsequent ZMP values return within the boundaries due to the utilized compliance controllers. The results still show that successful knee-stretched walking with toe-off and heel-strike is fulfilled by the position-controlled humanoid robot BHR6, thus the validity of the proposed method in experiment is confirmed.
The walking speed and number of steps in the experiment were less than those in the simulation. The performance of stretched-knee walking with toe-off and heel-strike will be improved in future experiments.

Conclusion and future plan
Knee-stretched walking is generally considered as a natural and energy-efficient type of walking. This article focused on realizing a knee-stretched walking with toe-off and heelstrike on a position-controlled humanoid robot. The contributions of this article are summarized as follows: Time-domain gait generation for knee-stretched walking with toe-off and heel-strike for positioncontrolled humanoid robots is proposed. The horizontal CoM trajectory with position constraints is generated by MPC to ensure that the desired kneestretched posture generated by the strategy of extending legs can be obtained. The validity of the proposed method has been confirmed both in simulation and in experiment. The position-controlled humanoid robot BHR6 achieved successful knee-stretched walking with toe-off and heel-strike at a speed of 2.4 km/h in simulation and at a speed of 1.125 km/h in experiment. The energy consumption of the knee-stretched walking generated by the proposed method is reduced compared with the knee-bent walking, which is confirmed in simulation.
In the future, the performance of knee-stretched walking with toe-off and heel-strike motion in experiment is intended to be improved, and some online footstep adjustment strategies are intended to be integrated to allow the robot walk through terrain with knee-stretched posture or maintain stable knee-stretched walking under large external disturbance.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This