The consensus of non-linear agents under switching topology using dynamic inversion in the presence of communication noise and delay

In this study, a consensus protocol for non-linear multi-agent systems using the non-linear dynamic inversion (NDI) technique is presented. It is named as distributed NDI or DNDI. The agents are considered to be working in a randomly changing environment which is realistic. The randomness in the operating environment is introduced by random switching communication topology, delays, and noise. The consensus protocol is obtained as a closed-form expression, which is a critical requirement for real-time implementation. Also, various cases regarding the communication issues have been considered to study the performance of the DNDI controller. The simulation results are found to be satisfactory.


Background
The consensus of multi-agent systems (MASs) is a wellstudied research topic. Multi-agent systems are used to execute critical and complex tasks which involve life risk, or urgent attention is required. Examples of these tasks include search and rescue, surveillance, delivery, firefight, and other civil or military operations. Such complex tasks would be completed more effectively or performed cooperatively, but for this, each agent must share information with its neighbour and compute the control action accordingly. A common goal can be achieved if cooperation among agents is possible or otherwise if they could reach a consensus. For consensus to be achieved, the agents must be connected in a meaningful communication topology. Therefore, the consensus is widely dependent on the communication topology.
While there exist several issues associated with communication topologies that can affect the consensus among the agents, the most critical of them is switching topology, communication noise and delay. These events are handled by modifying the consensus protocols designed considering the agent dynamics. Some of the works related to communication issues in MASs are mentioned in the following paragraphs.

Literature review
In MAS communication, switching topology is a common event that occurs primarily due to the changing surrounding, communication range limitation, or nodal link failure. Switching topology has been considered in many research studies. The distributed consensus tracking problem is presented for multi-agent systems having Lipschitz-type node dynamics. 1 In this study, the topology among followers is considered to be changing with time. Wen et al. 2 described the consensus tracking problem where the agents are having second-order non-linear dynamics tracked a time-varying reference state. They considered the time-varying communication topology. The consensus problem for heterogeneous multi-agent systems involving firstand second-order dynamics was addressed by Kim et al. 3 The communication among the agents has probabilistic link failure, which is represented using a Bernoulli probability sequence. Liu et al. 4 solved the leaderless consensus problem of multi-agent systems agents to be switching and noisy. The switching of the topologies is deemed to be random. Li et al. 20 discussed the consensus considering Markovian switching topologies and communication noises. Moreover, necessary and sufficient conditions for consensus of continuous-time multi-agent systems are derived. Ming et al. 21 also presented the communication channel as Markovian switching topologies along with stochastic communication noise.
Next, the works having the combination of switching topology and communication delay are discussed. Leaderfollowing consensus stability of multi-agent systems is discussed by Park et al. 22 Also, the stabilization is studied with interval time-varying delays. The communication among the agents is Markovian switching and contains delay. Pei et al. 23 considered switching topologies and time delay to solve the consensus of multi-agent systems having discrete-time linear dynamics. Ding et al. 24 described sampled-data leader-following consensus of multi-agent systems having non-linear dynamics. Communication is characterized by random switching network topologies and communication delay. Shang et al. 25 discussed the consensus problem of MASs considering Markovian switching network with time-varying delays. The topologies are also uncertain in nature. Jiang et al. 26 also studied a consensus control problem for multiple robotic manipulators under directed switching topologies with and without time-varying delays.
A few works have been reported where communication delay and noise have been considered to study the consensus among multi-agent systems. Liu et al. 27 showed that uncertainties and communication delays affect the consensus. Zong et al. 28 investigated stochastic consensus conditions of multi-agent systems. In this research article, both time delays and measurement noises are considered to achieve consensus among MASs.
All of the studies discussed so far consider one or two events among switching delay, communication delay, and noise. It can be mentioned that there exists a research study that was conducted by Ming et al. 29 (to the best of the author's knowledge) in which all the issues related to communication among the agents are discussed. However, this work assumed the agent dynamics to be linear timeinvariant (LTI), and the consensus protocol was designed accordingly. Therefore, the consensus protocol does not apply to non-linear systems. However, in reality, the agents have non-linear dynamics. Therefore, a protocol is required, which can be applied to the non-linear systems and can handle the communication issues. The scope of finding the consensus protocol of multi-agent systems having non-linear dynamics is explored in this study. Communication among the agents is considered to have all the issues related to communication, that is, switching topology, communication delay, and noise.
The consensus of agents is achieved when the error among similar states converges to zero or the consensus error zero. Therefore, the consensus controller aims to drive error zero. In this context, it can be mentioned that, along with other robust controllers, there exists one control strategy which enforces the error dynamics and includes plant non-linearities in the control expression. This controller is based on the philosophy of non-linear dynamic inversion (NDI), which uses feedback linearization theory to eliminate the plant's nonlinearities. Moreover, the closed-loop plant responds similarly to a stable linear system. Non-linear dynamic inversion has many advantages, like simple and closed-form control expression, easily implementable, global exponential stability of the tracking error, use of non-linear kinematics in the plant inversion, minimize the need for individual gain tuning or gain scheduling. Non-linear dynamic inversion has been used to design the controller for various purposes. The theory of NDI is used by Enns et al. 30 to design flight controller. Singh et al. 31 used NDI to develop a controller for autonomous landing of unmanned aerial vehicle (UAV). A partial integrated guidance and control (PIGC) is designed for reactive obstacle avoidance of UAVs using neuro-adaptive augmented dynamic inversion by Padhi et al. 32 Another application of NDI is presented by Mondal et al., 33 where a formation flying scheme is proposed. In this research study, the NDI is used to track the leader's desired position, velocity and orientation. Caverly et al. 34 described the use of NDI for the attitude control of a flexible aircraft. Horn et al. 35 described control design using dynamic inversion. Lombaerts et al. 36 showed NDI-based attitude control of a hovering quad tiltrotor electric vertical take-off and landing vehicle.
The robust non-linear NDI has been widely used for designing controllers for many applications, but for the first time, in this study, we have reformulated NDI to solve the consensus problem of non-linear agents and named it as distributed NDI (DNDI). It inherits all the advantages of NDI discussed in the last paragraph. It is tested in the presence of communication issues like switching topology, delay, and noise. The switching topology is random and changes at each time step. Moreover, the number of switching topologies is not limited, but it can be scaled as the user wants, as it is often in real cases. Also, not all the topologies may have a spanning tree, and thus they may be uncertain when the stability is concerned. The delay is considered in various scenarios like random delays for randomly selected agents. The performance of DNDI to handle the communication issues is studied and discussed in detail in the following sections.
The contribution of this work is summarized as follows.
1. We have addressed the communication issues together, which are rare. Moreover, all issues with non-linear dynamics are not reported. We wanted to explore the scope and found that NDI is an appropriate control strategy that could be beneficial in this context. 2. The application of NDI to the consensus problem is new. We have presented the new derivation in the study and named it distributed NDI or DNDI. We have highlighted the features of NDI along with applications in the introduction. We wanted to study the performance of DNDI in the consensus problem.
3. To highlight the effect of the issue 'switching topology', we have considered unlimited topologies in contrast to other studies where limited topologies are considered. Moreover, we considered the switching of the topologies in a completely random manner to make the simulation scenario more realistic. Communication noise is also random. The communication delay is considered in four cases which are realistic.
The rest of the study is organized as follows. The section 'Preliminaries' gives a brief description of the consensus of MASs, graph theory, switching topology, communication noise, delay, and NDI. The section 'Problem Description' gives a formal problem statement. The next section, 'NDI Controller for Consensus of MASs', presents the mathematical details of the DNDI. The 'Result' section presents the simulation results and their analysis. Finally, the conclusion is given.

Preliminaries
The topics which are relevant to the problem considered in this study are given in this section.

Consensus of multiple agents
The consensus of MASs on communication network is discussed in this section. The definition of the consensus is given as follows.
Definition 1. Let us consider a MAS with N agents, where X i , (i = 1, 2, 3, …, N) denotes the states of the ith agent. The MASs will achieve the consensus if kX i À X j k → 0, "i ≠ j as t → +∞.
The primary goal of designing a consensus protocol is to minimize the error in similar states of each individual agent with its neighbour by exchanging information among them through the communication network, which is designed and explained by graph theory.

Graph theory
The networked MAS is represented by a weighted directed graph written by G = {V, E}. The vertices V = {1, 2, …, N} of the graph denote the agents, and the set of edges is denoted by E 4 V × V represents the communication among the agents. e ij denotes the information flow along the edge from j to i. The neighbour of agent i is denoted by N i = {j 2 V: (i, j) 2 E}. The Adjacency matrix A ¼ ½a ij 2 ℜ N ×N which denotes the connectivity among the nodes or agents. a ij denotes the elements of A. There is no self-loop in the graph. This fact is expressed by selecting the diagonal elements of the adjacency matrix A as zero, that is, i 2 V, a ii = 0. The off-diagonal elements, that is, " i ≠ j, e ij 2 E, a ij > 0 represents the weight associated to edge e ij , while a ij = 0 otherwise. The degree matrix is denoted by The Laplacian matrix is written as L = D À A. All the matrices describe the connections and properties of the consensus among MASs.
There exist several issues with communication among MASs, which affect the consensus. The issues like switching topology/link failure, communication noise and delay are common in practice. They are discussed in the following sections.

Time delay in communication channel
The communication delay is a practical event in a MAS platform. The MASs exchange information among themselves to achieve the consensus. The agents in the synchronous network receive information from the neighbours at the same time instant; that is, they share the same clock frequency. But in a practical scenario, the situation arises where some of the agents or all of the agents may have delay. The delay may be counted by time stamping the incoming data from the neighbours. The ith agent updates its own and neighbour's states at time instants t i 0 ,t i 1 ,…,t i k ,…. The agent i receives the state information of jth neighbour If a neighbour has η = 1, then the information of jth agent at t kÀ1 , that is, Pictorial representation of the delay is given in Figure 1.

Noise in communication channel
In this section, the effect of communication noise on the consensus of agents is highlighted. The noise is added to state information transmitted by the agents. Therefore, the noisy information is received by ith agent from its neighbours. The noisy information received by ith agent from jth neighbour j 2 N i can be modelled as X ji ¼ X ji þ σ ji ω ji , where X i ,X j 2 ℜ n are states, ω ji ; i, j 2 1, 2, …, N are independent standard white noises and σ ji is the noise intensity.

Switching topology
In this study, communication topologies are considered to be changing in a random manner. This scenario is presented by generating many topologies, and one of them is selected at each random time interval. The number of switching topologies can be selected by the user; that is, topologies are not limited. The topologies are defined by the adjacency matrices. The procedure for generating random graphs (G i ) or adjacency matrix is given in Algorithm 1. 37 A k defines the kth topology, k = 1, 2, …, N p , and N p is the number of topologies. i, j defines the position of an element in the adjacency matrix A k . The algorithm generates N p adjacency matrices which are square in dimension. The value at each i, j position is dependent on a random variable x. If the value of x is greater than 0.5, then A k (i, j) = 1, otherwise A k (i, j) = 0. The threshold value 0.5 is considered to obtain 0 and 1 with equal probability, but the value can be changed. It is important to note that the graphs should contain a spanning tree to achieve consensus. But the graphs generated may not contain a spanning tree always. Therefore, the performance of an NDI-based controller is evaluated considering this factor.
The selection of a topology among N p topologies at each time instant is also random, as mentioned earlier. The selection procedure is shown in Algorithm 2. A S denotes the selected topology. The variable i denotes the time instants. The switching depends on the value of x. The variable x takes a random value between 0 and 1. If the value is greater than 0.5, the topology will change. The function random_integer ([1 N p ], 1) generates a random integer ind, in the range [1 N p ]. The selected topology A S is the topology corresponding to the integer ind, that is, A S = A ind . If the value is less than 0.5, that is, x ≤ 0.5, the topology does not change.

Non-linear dynamic inversion
Non-linear dynamic inversion is a potential candidate for designing the state feedback controller for the nonlinear systems. A brief description of NDI is presented here. Let us consider a class of affine non-linear dynamics as follows.
where X 2 ℜ n , U 2 ℜ n and Y 2 ℜ n are state, control and the output, respectively. f is a continuously differentiable vector-valued function representing the intrinsic nonlinear dynamics.

Remark 1.
It can be noted that the system in equations (1) and (2) represents a specific class of non-linear systems. However, they are used in many practical applications.
A few examples include the control design of decoupled three-loop guidance and control design of autonomous vehicles. In these cases, the number of states, output and control inputs is same for each loop. 38,39 Assumption 1. The system is perfectly known, and the matrix g(X) is invertible for all time.
The objective of NDI is to design a state feedback control U such that the output of system Y to track a bounded and smooth reference signal Y * with time t. The tracking error is given by The first-order error dynamics is written as It can be mentioned that the gain is considered to be positive definite, that is, K > 0. The expression of the tracking error, that is, e, is substituted in equation (4) to obtain the control expression as follows Assumption 2. g(X) is non-singular for all the time.

Problem description
The problem objective in this study is to design a consensus control for MASs using NDI. The MASs consist of N agents. They are connected by the network topology. All the agents are homogeneous, that is, they have similar dynamics. In this study, the dynamics of each agent is the same as given in equations (6) and (7). The dynamics for i th agent is given by The state vector of ith agent is given by X i 2 ℜ n . The output of ith agent is given by The agents are operating in a randomly changing environment. The communication among the agents faces issues like switching topology, noise and delay, which are described in the preliminaries section.

Non-linear dynamic inversion controller for consensus of MASs
The derivation of NDI-based control for MASs is presented in this section. In case of consensus of MASs, the reference signal or desired output of each agent is the outputs of its neighbours. First, the error for scalar output is obtained. The error in output of ith agent can be written as where j denotes the jth agent of ith agent's neighbourhood N i . Equation (9) can be simplified as follows The error in equation (10) is written for vector output of ith agent, that is, Y i 2 ℜ n ; n > 1 as where d ¼ ðd i Ä I n Þ 2 ℜ n×n , a ¼ ða i Ä I n Þ 2 ℜ n×nN and Y 2 ℜ nN . I n is n × n identity matrix. 'Ä' denotes the Kronecker product. The Kronecker product of A = [a ij ] and B is given by We define a Lyapunov function V i as follows Differentiating equation (12) yields According to the Lyapunov stability theory, let the time derivative of the Lyapunov function should be where K 2 ℜ n×n is a positive definite diagonal matrix. The expression of _ V i in equations (13) and (14) is equated to obtain Equation (15) is simplified as follows Differentiation of equation (11) yields Substitution of the expressions for e i and _ e i in equation (16) gives The expression of control U i for ith agent is obtained by simplifying equation (18) as follows It is clear that the control expressions of conventional NDI are different from what is obtained for consensus of MASs. This control expression in equation (19) is used to generate results which shown in the following section.

Results
The simulation study is performed to evaluate the performance of the NDI-based controller in achieving consensus in the presence of different issues related to communication discussed before. The performance is evaluated under randomly changing behaviour of communication. Five cases are considered as follows. These cases are simulated with the same initial conditions and for the same simulation duration. The simulation study is performed on PC with AMD Ryzen five processor and 8 Gb RAM. The gain of each agent K i is same, that is The gain is selected in a random manner. The user can change the gain and obtain the response accordingly. Any specific method is not used to find the gain because the aim of this work is to focus on how the NDI is used for the consensus of agents having non-linear dynamics. The dynamics are given in the following section.

Agent dynamics considered for simulation study
The non-linear dynamics considered for the ith agent is given by Putting the dynamics of equations (20) and (21) in the form given in equation (6) gives and and The output Y i is chosen as follows where Y i 2 ℜ 2 . In this simulation study, 10 agents are considered. The global state vector of the agents is given by and Also, the global control vector is given by and The initial conditions of the states X 1 and X 2 are generated using MATLAB command 'randi' in the range [À10, 10].

Case 1. Consensus using NDI controller
In this section, the results obtained using NDI are shown. In this case, communication delay, noise, and switching topologies are not considered. The states X 1 and X 2 achieved a common value or consensus in a few seconds, as shown in Figures 2 and 3, respectively. It can be mentioned that the time of convergence of the states depends on the gain K. The trajectories of the agents are shown in different colours. The controls for the consensus of ith agent, that is, U 1 and U 2 , are calculated using DNDI. Both control signals of all the agents are shown in Figures 4 and 5, respectively. The controls have driven the states of all agents to the consensus, which can be understood from the error plot. The consensus error e i in states X 1 and X 2 of all the agents is shown in Figures 6  and 7, respectively. It is clear that the error converges to zero in few seconds.

Case 2. Consensus using NDI with switching topology
The consensus of the agents is discussed in this section considering switching topology. Other events, like communication delay and noise, are not included. It is important to note that in this work, random switching topologies are considered, as discussed in section. The number of switching topologies considered in this work is     100, that is, N p = 100. At each time instant, one of these topologies is selected.
The state trajectories of all the agents, that is, X 1 and X 2 , are shown in Figures 8 and 9, respectively. The DNDI is able to make the states converge to a common value successfully in the presence of the effect of switching topology. The difference between the state trajectories for Case 1 and Case 2 can be observed. The state trajectories in Case 2 are not as smooth as those obtained in Case 1. The corresponding control signals U 1 and U 2 are shown in Figures 10 and 11, respectively. It is clear that the control signals have the effect of switching topology. The switching signal is shown in Figure 12. The 'y' axis shows the topology number selected at each time instant, and the 'x' axis shows the time in second. The topologies are selected according to Algorithm 2. It can be observed that the topologies toggle randomly over time. The control signals are calculated considering these switching topologies. Therefore, the difference in control signals between Case     1 and 2 is obvious. It can be noted that each of the selected topologies may not contain a spanning tree, but NDI can achieve consensus among the agents.

Case 3. Consensus using NDI with communication noise
In this section, the effect of communication noise on the consensus of agents is studied; other effects are not considered. The noise considered in this study is assumed to be white Gaussian. The noise is generated using the MATLAB function 'WGN'. The intensity is selected as σ ij = 0.015. The random noise added to both the states of all the agents at a time instant is shown in Figure 13. The effect of the noise can be observed in the state trajectories shown in Figures 14 and 15. The trajectories are affected by the noise, but the NDI controller is able to achieve the consensus in finite time. The control signals corresponding to the state trajectories are shown in Figures 16  and 17.

Case 4. Consensus using NDI with communication delay
The effect of communication delay on consensus is studied considering different scenarios which are described in following sections.

Case 4.1. All agents with same constant delay
In this scenario, all the agents are assumed to have the same constant delays, that is, each agent receives information about states of its neighbours not in real time but with a delay of two time steps, that is, at t i k , agent i receives the state information X j ðt i k À τ k ij Þ with η = 2. The state trajectories X 1 and X 2 are shown in Figures  18 and 19, respectively. The effect of the delay can be observed in the state trajectories as ripples appear. The control signals corresponding to the state trajectories are shown in Figures 20 and 21. Although the delay is present, the NDI controller is successful in achieving the consensus.            Figure 28. It shows the agent delay toggles between one and two-unit time steps.
The control signals of the agents are shown in Figures  29 and 30.

Case 4.4. Variable agents with variable delay
In this case, some agents are selected randomly at each time instant with a random delay ranging between one and two unit time steps. At time instant t k , a set of agents are selected in a random manner, and a set of delays are considered. In the next time instant t k+1 , another set of agents are selected randomly with a different set of delays. It can be mentioned that the number of agents that would be selected at each time instant is also random. This number is generated by using the MATLAB function 'randi'. Also, the delays are selected in a similar way. For example, let us assume that at a time instant t i k , the selected agents are 2 nd and 4 th , with a time delay of one and two unit time steps, respectively. In the next instant, the agents selected are 3 rd , 4 th and 7 th with the delays of 1, 2 and 1 unit time steps, respectively. The state trajectories are shown in Figures 31 and 32.
The control signals are shown in Figures 33 and 34. The agents selected at each time instant is shown in Figure 35. The number of agents is allowed to vary between two and eight.
In general, the state and control history of the agents looks different from each other. This is due to the different cases that we have considered.             neighbouring agents, and therefore, it can handle the communication issues.

Conclusion
The proposed distributed non-linear dynamic inversion controller (DNDI) is designed with a robust non-linear control theory dynamic inversion. This is for the first time dynamic inversion is applied to design the distributed consensus protocol among MASs. The agents' communication is subjected to practical issues like switching topology, communication delay, and noise. To make these issues more realistic, randomness is mixed with them. The results generated prove that the controller efficiently handles the communication issues among MASs. Moreover, the expression of consensus protocol comes in closed form, making it possible to implement in real-time. Therefore, DNDI is a potential candidate for designing consensus protocol. We consider applying DNDI in consensus problem to handle faults in actuators as our future work and compare the result with other existing controllers.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.