of Advanced Robotic Systems Stable Formation Control of Multi-robot System with Communication Delay Regular Paper

In  this  paper,  a  distributed  formation  control algorithm  with  delayed  information  exchange  is discussed.  The  algorithm,  which  is  derived  from  the flocking behaviour of birds and consensus theory, enables robots to move in formation at a desired velocity. After a series  of  orthogonal  transformations  to  the  original formation  system,  the  upper  bound  tolerable  delay  is obtained  by  using  matrix  theory  and  the  Nyquist criterion.  According  to  the  results,  the  upper  bound tolerable  delay  depends  on  the  control  parameters  and eigenvalues of the Laplacian matrix. Therefore, the effect of  the  parameters  on  the  maximum  tolerable  delay  is analysed, obtaining  the  following conclusions:  the upper bound  tolerable delay  is proportional  to  the parameters associated with the velocity, inversely proportional to the parameters  associated  with  the  position,  and  inversely proportional  to  the difference between  the eigenvalue of Laplacian matrix and 1. The simulation results of a four‐ robot  formation  system  with  different  communication delays  verify  the  effectiveness  of  the  formation  control algorithm and the correctness of the theoretical analysis.


Introduction
The distributed formation control of large groups of autonomous robots has been a topic of great interest in the last few years, partly due to broad applications in cooperative search and rescue missions using multiple robots, the space exploration of coordinated minisatellites, and mine countermeasures employing multiple autonomous underwater vehicles [1] [2].At present, numerous results involving the distributed formation control of multi-robot systems with different models of individual dynamics, formation requirements and application domains presented in the recent literature [3]- [8].In many swarming systems, the robots form and maintain a certain formation through limited interactions, using only relative information from a subset of the group.For example, in [3] the authors study a bioinspired formation feedback scheme which mimics the collective motion of birds and fish.Flocking is a typical coordinated motion of a network of agents in a selforganized way.There has been great interest among control scientists and robotics scientists in analysing this phenomenon and the derived consensus problem [9] [10].
However, within the enlarged application range of a multi-robot system, one particular problem becomes increasingly apparent, namely that communication constraints will greatly depress the performance of the formation system in some practical applications.These negative factors involve delayed information exchange, communication failure or interruption, limited communication range and a high error bit rate.For example, adverse underwater environments lead to poor communication when using underwater sound communication, which is not good for a multiple autonomous underwater vehicles system.In this paper, we mainly investigate one of these aspects-the effect of communication delays on formation control.It is well known that communication delays will not only reduce the performance of the formation system, but they may also even lead to instability where the delays are too large.Thus, the design of an effective control strategy in the presence of communication delays and the investigation of its stability is an important issue.
Following the research on collective flocking and the consensus theory, the consensus protocol has become a powerful tool for designing a control law of formation with delayed information exchange.There are some remarkable results, where each individual is modelled as a double integrator [11]- [17], other forms of linear systems [18] and nonlinear systems [19].Owing to the delays contained in the system, the stability analysis becomes more difficult and has been a hot topic in recent years.Until now, the problem of formation stability has mainly been investigated by the frequency domain method [11]- [13], the eigenvalue analysis method [14] and the Lyapunov stability theory [15] [16].
The previous research has usually been concern with static formation rather than with formation using a nonzero desired velocity.Sometimes, we expect that robots will not only reach the desired positions, but also converge on a desired velocity.Considering the non-static predefined velocity, Munz et al. [16] introduce an additional term in the control law that contains the reference velocity so as to get more accurate predictive position information.Delay-dependent stability conditions are obtained based on the Lyapunov stability theory.However, the construction of the Lyapunov function partly depends on the authors' experience.In this paper, we investigate a distributed formation control law for a second-order multi-robot system with the delayed exchange of information between neighbouring robots.Each robot is assumed to have instantaneous access to its own state information, but delayed state information of its neighbours.In addition, a predictive term is involved in the control law, as in [16].We assume that the communication graph or formation graph is undirected and connected, and that the communication delays between any two neighbouring robots are constant, homogeneous and symmetric.Based on these simplified conditions, we apply frequency domain arguments to gain an accurate expression of the upper bound tolerable delay which guarantees that the formation becomes stable at a certain point.Furthermore, we study the effect of initial conditions on the equilibrium of the multi-robot system.In addition, the relationship between the upper bound delay and the parameters is also studied.This paper is organized as follows: Section 2 gives the background of algebraic graph theory and the mathematical model of formation control.Section 3 presents the delay-dependent stability analysis method of the multi-robot formation.Section 4 shows some simulation results, and this is followed by a summary and conclusions in Section 5.

Formation Graph
The formation graph G is used to denote communication between neighbouring robots.The undirected graph G=(V,E,A) is defined with a vertex set V={1,2,…,n} to represent the robots, an edge set EVV to represent the communication links, and a adjacency matrix A=[aij],where AR n×n is given by aij>0 if (i, j)E, and aij=0 otherwise.Suppose that each vertex has no self-loop, i.e., aii=0 for all i.In this paper, we use the weighted adjacency matrix, which is much more general than the 0-1 adjacency matrix.In particular, we let the sum of aij in the row i be 1.If there exists an edge from vertex i to vertex j, i.e., robot j can receive data from robot i, then i and j are called adjacent.All the adjacency robots of i are defined as Ni={j|(i,j)E}.The cardinality of the set Ni is called the degree of i.The diagonal degree matrix is Δ=diag(|Ni|).The Laplacian of G is a matrix L=Δ-A.For an undirected graph, the Laplacian matrix L is symmetric positive semidefinite.If the graph G is connected, the Laplacian matrix L has a simple zero eigenvalue, and the corresponding eigenvector is the vector of ones, 1.In addition, the Laplacian matrix L is diagonalizable by the orthogonal matrix U with the orthogonal similarity transformation U T LU=λ, where λ=diag(λ1…, λn,), λn≥…λ2>λ1=0 are ordered eigenvalues of L, and U consists of eigenvectors, with each column associated with an eigenvalue.In particular, the last column and the last row of U are the right eigenvector and left eigenvector of the zero eigenvalue of L respectively, that is 1/ n .This property is very important in discussing the equilibrium of the formation.The details can be found in [17].A path from i to j is a sequence of edges of the form (i, j1), (j1,j2),…,(jm, j)E.If there is a path between any two nodes of the graph G, then G is said to be connected in the case of undirected graphs, and strongly connected in the case of directed graphs.
In order to achieve a stable formation for the multi-robot system, the formation graph must meet one requirement, i.e., the graph G must contain a spanning tree or else the graph G must be connected, no matter whether there are communication delays.

Formation Control
Consider a multi-robot system that consists of n robots.The dynamics of each robot is described by a double integrator: where xi(t)R N is the position, vi(t)R N is the velocity of a robot i, and ui(t)R N is the control input of that robot.
The robots adjust their control inputs to achieve and maintain a stable formation, moving with the desired relative positions and orientations.In this paper, we also hope that the robots can move at a desired velocity.
, the robots are said to converge on formation H.
Suppose that the communication delay from robot j to i is τij, τijR + and that the communication is symmetric, i.e., aij=aji, τij=τji.In this paper, a distributed formation control algorithm is considered based on the consensus theory.
Since each robot can access its own state information instantaneously through embedded sensors, robot i uses its own current information in the control law in this paper.However, communication delays between neighbouring robots cannot be neglected in some applications.Taking communication delays into account, robot i receives out-dated state information of its neighbours.Inspired by the work presented in [16], the control law is designed thus: where c0 is the damping coefficient, c1 and c2 are position and velocity feedback gain respectively, the parameters c0, c1 and c2 are positive scalars, Ni represents the neighbour set of robot i, and vd(t) represents the desired velocity of vi(t).If vd(t)=0, the robots form a stationary formation.Otherwise, the robots move at the velocity vd(t).As distinct from previous controllers, the controller (2) contains an additional predictive term vd(t)τij due to the non-zero desired velocity.Therefore, robot i needs to know the value of τij, which can be solved by adding a time stamp in message transmission.If the desired velocity is time-varying, the term should also be added to the velocity coupling term, which is omitted in this paper due to the constant desired velocity.
The proposed formation protocol is distributed in the sense that each robot only needs information from its neighbours.This distributed mode reduces the complexity of connections between robots significantly.

Analysis of the Delayed Multi-robot Formation
This section focuses on the stability of the formation with control law (2).The control of formation and stability analysis are particularly difficult because of communication delays.Usually, there is an upper bound delay τ * which, when exceeded, will see the system become unstable.In this section, expressions of the upper bound delay will be given for a multi-robot system on the assumption that all of the neighbouring robots have the same constant communication delay τ and a fixed reference velocity vd.

Formation Stability
The closed-loop dynamics of a robot i are: According to the Newton-Leibniz formula: If the dynamics of every robot can be decoupled in all coordinates for 2D and 3D formations, the problem can be discussed in a 1D space.Thus, we discuss formation stability in a 1D space.Now, let  i x and  i v be the generalized position and velocity of a robot i, . The formation model of the multi-robot system is written in the vector form: where A is the adjacency matrix of the weighted graph G.
Here, the degree matrix of G is a unit matrix In×n, and the Laplacian matrix is L=I-A.It is possible to find an orthogonal matrix U to diagonalize the adjacency matrix A: The following system can be derived from system (5): Now, the closed-loop control system (6) can be decoupled into n subsystems, each with the following form: where i=1,2,…,n, ( ) Taking the Laplace transform of Eq. ( 7), we get: According the Eq. ( 8), we have: ( ) where It is obvious that the feedback loop contains the eigenvalue λi of the Laplacian matrix and the communication delay τ.The characteristic equation of the system is: The necessary and sufficient condition of forming a stable formation is that all of the characteristic roots of Eq. ( 10) are located in the left-half complex plane. Consider: ) Because c0, c1, and c2 are positive and λi≥0, the roots of Fi(s,e -τs ) are located in the left-half complex plane when τ=0.Therefore, the system ( 6) is stable for τ=0.As a result, the formation with the control law ( 2) is stable for τ=0.
Next, we analyse the formation stability for τ>0.Here, we discuss the stability of the system (6) by analysing the stability of ( ) F s e G s , where: Let s=jω.The necessary and sufficient condition of the stability of the ith subsystem ( 7) associated with λi is that the trajectory of Gi(jω) does not enclose the point (-1, j0) for all ω[0,∞).
Eq. ( 11) can also be expressed as:

� χ≥0
If χ≥0, there exists a root ω=0 of Eq. ( 12), otherwise there does not exit any positive real root ω>0 of Eq. ( 12), which for any τ>0.Therefore, the subsystem (7) associated with λi is stable independent of the delay τ.In a word, if for all λi there is χ≥0, the formation of the multi-robot system (3) is stable for any τ>0.
, the real roots of Eq. ( 12) about ω are: Where arctan2 is an arc tangent function valued on (-π,π]. Combining the results of formulas ( 13) and ( 14), we obtain the upper bound delay of the ith subsystem, marked by τi.The upper bound delay of the whole system is τ * =min{τi}. Remarks: 1) The subsystem (7) associated with λi=0 or other eigenvalues has the characteristic root s=0, which means that the subsystem will be marginally stable.If, for any τ[0,τ * ), all of the characteristic roots of each subsystem are located in the left-half complex plane, the multi-robot system is stable.
2) This paper only considers the case of a homogenous constant time delay.The problem of formation control with time-varying delays can be solved by substituting the time-varying delays with the maximum constant delay and utilizing the buffering technique, as in [11].

Formation Equilibrium
It is also important to estimate the effect of the initial conditions.However, there are some differences due to communication delays.The following part will discuss the effect of the initial conditions on the formation properties.
All of the  i x come to a common value eventually.It is also obvious that the final formation equilibrium is not affected by the delay if there is priori knowledge of the neighbours of every robot.Since: when t goes to infinity.
We infer that Remarks: 3) Assume that a robot i has no access to the state of its neighbours within the beginning interval (0, τ) because of the communication delay, i.e.,  ( ) . The equilibrium 1eq x will be: The equilibriums of other subsystems can still be  0 ieq x .
From these two different cases of initial conditions, it can be inferred that although different initial conditions may lead to different equilibriums, all the  ( ) i t x can still reach a common value.However, the equilibrium of the latter situation is related to the communication delay.These subsystems achieve the common value if, and only if, they have the same communication delay.As to a formation with different communication delays, we will not discuss it in this paper.
The limit of  ( ) is for all robots will converge to the desired velocity, i.e.,   Now, we have the following theorem.
Theorem 1: Given a multi-robot system consisting of n robots with dynamics (1) and a control law (2), where the communication network is fixed, connected, symmetric (i.e., aij=aji), a normalized adjacency matrix A, a fixed constant delay τ, a reference velocity vd, and a specified formation H, if τ[0, τ * ) holds, where τ * =min{τi},τi is calculated by formulas ( 13) and ( 14), the desired multi-robot system formation will be asymptotically achieved and hold stable, i.e., From theorem 1, the formation will hold stable for small delays but become unstable for large delays.

Simulations and Analysis
In this section, we illustrate our conclusions through several simulations and analyse the simulation results to get a deeper understanding.
To concentrate our attention on the relationship between the upper bound delay τ * and parameters c0, c1, c2 of the eigenvalue λi, here we only discuss combinations of parameters c0, c1, c2, for which the upper bound delay τ * is computable according to the formula ( 13) and (14).To ensure χ<0 when λi>0 and χ<0 when λi=0, we set c0=1, c1=2, c2=1 as the basic combination, and then change c0, c1, and c2 separately in the computable range of τ * .We obtain the results as shown in Fig. 1 and Fig. 2. From the two figures, it can be seen that τ * is proportional to the value of c0 and the value of c2, and inversely proportional to the value of c1.Therefore, we can set a larger value for c0 and c2, and a smaller value for c1 when the multi-robot system has large communication delays.
Comparing Fig. 1 and Fig. 2 for the same combination of c0, c1 and c2, the system with topology A2 is much more tolerant with the delay than the system with topology A1.This is attributed to the effect of the eigenvalues of the Laplacian matrix on the value of τ * .To investigate the relationship between λi and τ * , set c0=0.1, c1=10 and c2=0.1.Fig. 3 shows the value of τ * with a different λi.As can be seen from Fig. 3, the upper bound delay τ * is inversely proportional to the difference between the value of λi and 1.For an extreme case, where λi is equal to 1, τ * will go to infinity.Thus, the subsystem (7) associate with λi=1 is stable for any delay.This assertion can also be obtained by analysing equation (10) directly, of which both characteristic roots s=0, -(c0+c2) have nothing to do with τ.
We define the formation average error function as: The simulation time is 50s and the simulation step is 0.01s.Fig. 4 shows the process of the velocity change of the four robots and the formation average error within t=0~50s when τ=1.5s.It is clear that the speed on the X-direction and Y-direction of every robot can converge to the desired value within t=30~40s.Moreover, the error of this formation goes to zero asymptotically.Therefore, we conclude that the four robots reach the desired formation and move at the desired velocity thereafter.The system is stable for τ=1.5s.
The formation process within t=0~50s in Fig. 5 also demonstrates that the multi-robot system can converge on the desired formation and desired velocity when τ=1.5s.
When increasing the communication delay, it is not only the stabilization time of the system that increases, but equally the system may become unstable.Larger delays bring in long-time oscillation which impedes the progress of the stabilization of the system.When τ>τ * , the four robots cannot reach the desired formation.Fig. 6 depicts the process of the velocity change of the four robots and the formation average error within t=0~50s when τ=2.5s.It can be seen that the persistent oscillation of velocity occurs both on the Xdirection and Y-direction.Moreover, the formation average error of the multi-robot system diverges over time.The formation process within t=0~50s in Fig. 7 also demonstrates that the multi-robot system cannot converge on the desired formation or desired velocity when τ>τ * .
In summarizing these examples, it can be seen that the formation conditions derived from this paper are correct and not conservative because of the frequency domain method adopted.Next, we switch the fixed topology of four robots to the other one A2= [0 1/3 1/3 1/3; 1/3 0 1/3 1/3; 1/3 1/3 0 1/3; 1/3 1/3 1/3 0], as mentioned before.The initial conditions and configurations are the same as with the first simulations about topology A2.The upper bound delay is calculated as τ * =5.6397s.Fig. 8 and Fig. 9 show the simulation results when τ=3.0s with the topology A2.The results are consistent with our conclusion.To observe the long-time tendency of the formation further, we get the simulation results within t=0~200s when τ>τ * .Fig. 10 shows the velocity and formation average error of the four robots when τ=5.7s.The multi-robot system cannot converge with the desired formation any more.The trajectory of the robots is not clear when the running time is too long, and we do not give the figure for the robots' trajectory here.The movement tendency can also be inferred by the formation average error.
In comparing these two groups of simulations with the same initial configuration, it can be seen that the formation with topology A2 has a quicker convergence rate than the formation with topology A1.

Analysis
The control strategy used in this paper is derived from flocking behaviours, and the multi-robot formation performs in a distributed manner.The control strategy and conclusions in this paper can also be applied to a multi-robot system with more robots.Once the parameters c0, c1 and c2 are fixed, the upper bound delay is only dependent on the eigenvalues of the Laplacian of the formation graph.Through multiple simulations of a different number of robots and different topologies, we find the rule that the denser the formation graph is, the larger the upper bound delay is.Obviously, it can also be concluded from the above simulation results of a fourrobot formation.In a practical application, especially for multiple autonomous underwater vehicles, it is not appropriate to realize highly frequent communication, and so it is still necessary to develop some compromise measures in order to get an optimal state according to the specific situation.

Conclusion
In this paper, we present the formation stability of a multirobot system with a homogeneous constant communication delay.We assume that the communication network is connected and symmetric.The matrix theory and Nyquist criterion are used to conduct the stability analysis.The results show that the multi-robot system can reach the desired formation and move at a desired velocity when the delay is smaller than a certain value.The derived stability conditions are supported by the simulation results.The relationship between the upper bound delay and parameters is also investigated.Further research will involve consideration of the convergence rate with respect to the delay and its parameters, formation with time-varying delay and multiple different delays.

Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant No. 60975071.
the ith components of x and v , respectively.Note that the last variables nx and n v are defined by nth row of U T .Therefore, the variable n x corresponds to the average of the position of all the robots, and n v represents the average of velocity of them.Notice that this result holds on the basis of a symmetric connected communication network.

Figure 4 .Figure 5 .Figure 6 .Figure 7 .
Figure 4. Velocity on the X-direction (a) and Y-direction (b) and the formation average error (c) over time when τ=1.5s with topology A1

Figure 8 .Figure 9 .Figure 10 .
Figure 8. Velocity on the X-direction (a) and Y-direction (b) and the formation average error (c) over time when τ=3.0s with topology A2