An intern-sufficient cloud for large-scale multi-robot systems and its application in multitarget navigation

Due to generally limited computing capability of an individual robot, cloud-based robotic systems are increasingly used. However, applications in large-scale multi-robot systems will be hindered by communication congestion and consequent lack of computing resources. In this study, an intern-sufficient cloud is investigated to alleviate the burden of communication and thus support more robots. At the same time, it enables heterogeneously idle computing resources of robots inside the system to be shared on demand, instead of relying on cloud servers and communication infrastructures, to make the scope of application wider. To this end, a hierarchical communication mechanism and a resource schedule algorithm are proposed. In the mechanism, the transmission power, signal-to-noise ratio, available bandwidth, and other relevant features are taken into account to estimate link quality for data transmission. Then, the constrained communication conditions and heterogeneous computing resources are balanced by the resource scheduling algorithm, so that the most appropriate computing resources of the robots are contributed to the mobile cloud. Furthermore, a multitarget navigation task is applied on the cloud to validate the work. Thereby, simulations and experiments are performed. The results show that the proposed intern-sufficient cloud can provide stable resources of communication and computation for a multi-robot system with 20 physical robots while achieving more effective multitarget navigation.


Introduction
In recent years, the multi-robot system gains a lot of attention due to its high efficiency and robustness. 1 Although the computing capability of processors installed on robots has been getting stronger, many algorithms for better robotic performance are more complicated with the development of robotic technology, such as the application of deep learning. 2 It means that robots have always been under constrained computing resources. Furthermore, advanced hardware represents high financial costs, which cannot be ignored in multi-robot systems. A possible solution is cloud computing. 3,4 robot system was constructed to investigate communication mechanism with a large number of robots and the algorithm complexity is discussed with robots increasing. 22 So it is hard to say its effectiveness in physical large-scale multi-robot systems.
In this study, an intern-sufficient cloud is investigated to improve the ability to support more robots in large-scale multi-robot systems without relying on cloud servers nor communication infrastructures. Furthermore, a multitarget navigation task is applied on the cloud to test the availability of the guaranteed communication links and the contributed computing resources. The major contributions can be summarized as follows: 1. A hierarchical network is proposed, in which communication mechanisms for heavyweight data and lightweight messages are constructed respectively to ensure available communication links. 2. A resource scheduling algorithm is proposed to ensure that only the most appropriate computing resources will be shared, which is achieved by reallocating communication resources and then matching. 3. A multi-robot multitarget navigation task is applied on the intern-sufficient cloud, and experiments with a physical multi-robot system, which has 20 robots, are performed to show the effectiveness.
The rest of this article is organized as follows. The second section is the problem description. The third section details the intern-sufficient cloud. The fourth section shows the implementation of the multitarget navigation task in the cloud. In the fifth section, simulations are performed. In the sixth section, experiments are carried out and results are analyzed. The seventh section gives conclusions.

Problem description
In this study, the large-scale multi-robot system is composed of N ðN 2 N; N ð20Þ robots. An intern-sufficient cloud is constructed by the robots inside the system. Available communication links and computing resources are provided by the cloud, supporting the large-scale multirobot system to perform tasks in outdoor scenes without relying on the developed communication infrastructures and cloud servers.
During the process of tasks, it is assumed that not all of the robots perform complex computing algorithms all the time, which is reasonable. So there are sufficient idle computing resources inside the system to be scheduled. For individual robot iði 2 1; Á Á Á ; N f g Þ , the idle computing resources at time t can be represented by C i ðtÞ, which is related to CPU-take-up rate. However, for robot j who wants to use the computing resources of robot i, the transmission rate T ij ðtÞ between robot i and robot j is supposed to be considered. For example, the C i ðtÞ can process an image of 100 Kb within 0.5 s, while T ij ðtÞ is merely 10 Kb/s. To meet the requirement to complete processing within 1 s, only those images smaller than 10 Kb can be transmitted, which results in a waste of computing resources. In contrast, if there is a large transmission rate but few idle computing resources, a waste of communication resources has appeared. Thereby, the communication resources should be reallocated to keep stable communication links and obtain more available computing resources. The intern-sufficient cloud is composed of these available computing resources.
Further, to better show the effectiveness, not only the performance of the cloud has been validated but a multitarget navigation task combined with the cloud has also been tested. During the task, because the required resources and available resources are constantly changing according to their actual situation, the successful application can definitely indicate the robustness of the cloud.

The proposed intern-sufficient cloud
The intern-sufficient cloud is constructed by a hierarchical network and a resource scheduling algorithm. The multirobot system is abstractly divided into three layers by the hierarchical network, as shown in Figure 1.
The first (top) layer is a virtualization of the contributed computing resources. The second and third layers are composed of the physical robots, in which the multi-robot system is divided into many subgroups. In one subgroup, there is one leader robot. The second layer is composed of all leader robots, which are selected according to spatial positions and communication conditions to better cover all the robots, which is clearly shown in Figure 2 from a top-down perspective. The resource scheduling algorithm is run in the leader robot to schedule the heterogeneously idle computing resources inside the subgroup, then the scheduled resources are contributed to the cloud. Thereby one subgroup is regarded as one minimal unit of the cloud. The situation of only one subgroup has been researched in our previous publication. 23 The complete cloud and implementation of a multitarget navigation task in the large-scale multi-robot system are emphasized in this study.

The hierarchical network
The hierarchical network is generated by a topology control (TC) algorithm and a subgroup generation (SG) algorithm, which are run sequentially in pairs. Generally, the data that need to be transmitted can be divided into two categories, heavyweight data, for example, images, and lightweight messages, for example, coordinates. So, each robot is assumed to be equipped with two communication devices of different frequency bands, for example, 5 GHz and 2.4 GHz, for those two categories, respectively. The TC algorithm uses 2.4 GHz for fast long-distance transmission of lightweight data among all robots, whereas the SG algorithm uses 5 GHz for heavyweight data in each subgroup.  In the TC process, the robot that firstly received start signals sends messages to all other robots with as few hops (relays) as possible. Fewer hops mean higher success rate of transmission of lightweight messages, which can be represented by where P is the failure probability of a hop, and m is the number of hops. Assumed P ¼ 0:1, the transmission success rate is 59% after five hops. So, the purpose of the TC is to minimize the number of hops when lightweight messages arrive in the last robot, and the process of which is shown in Algorithm 1. Here, e ij is the signal strength of robot j received by robot i, which can be calculated by where x is a system coefficient related to the wavelength of signal, transmitter, and receiver antenna gains, etc. P j is the transmission power, d ij is the distance, and d is an attenuation factor of the wireless channel, which is equal to 2 in a common environment in this study. Then, the TC will be run again after a fixed-time interval to update network topology, and lightweight messages can be transmitted via links within the topology.
In the SG process, the robot that firstly received the start signal becomes the first leader robot. To be noted, the leader robot is responsible for scheduling resources of the robots inside its subgroup, unlike traditional leader-follower strategy, for example, formation control. 24 Other leader robots are selected according to their positions and communication conditions and then cover as many robots as possible while ensuring available communication links, as shown in Algorithm 2. The SG is run after the TC, which means that R 2:4 k ðk ¼ 0; 1; 2; Á Á Á ; K 2:4 Þ is known. The E 5 is much larger than E 2:4 because a larger transmission rate is required. For example, the execution time of object recognition in an image of 100 Kb is required to be less than 1 s. It can be completed within 0.5 s when computing resources in the cloud are used, so the time for transmission of the 100-Kb image is less than 0.5 s, which implies that a Algorithm 1. Topology control Algorithm 2. Subgroup generation transmission rate larger than 200 Kb/s is required. In addition, the distance corresponding to signal strength E 2:4 should be twice that of signal strength E 5 to achieve better coverage. According to Eq. (2), the relationship between E 2:4 and E 5 can be represented by where superscripts "(2.4)" and "(5)" indicate parameters from communication devices with 2.4 GHz and 5 GHz, respectively. The SG will be run again only when it is detected that changes of communication conditions and idle computing resources are greater than certain thresholds.
To be noted, all robots use the 2.4 GHz devices to communicate before subgroups are divided. After the hierarchical network is generated, the 5 GHz devices are used for communication between the second layer and the third layer (inside subgroups), and the 2.4 GHz devices are used for communication within the second layer (between leader robots), respectively, shown as Figure 1.

The resource scheduling algorithm
As soon as the hierarchical network is generated, the resource scheduling process is carried out. The resource scheduling algorithm is run on every leader robot to contribute available computing resources inside the subgroups to the intern-sufficient cloud, as shown in Figure 2.
Inside a subgroup, firstly, idle computing resources of the robots in R iÀ are reported to leader robot i. Due to the idle computing resources are not equal to the available computing resources, the resource scheduling algorithm is used to schedule the communication resources, already obtained in the process of generating hierarchical network, and the idle computing resources to enable idle computing resources to match appropriate transmission rate, which is achieved by reallocating bandwidth because the transmission rate is mostly determined by bandwidth. It has where R ij is signal-to-noise ratio (SNR), and B j is the allocated bandwidth to robot j. After executing the resource scheduling algorithm, the reallocated bandwidth of robot j can be represented by where T required j is the transmission rate required to use all idle computing resources C i , and T 0 j is the estimated transmission rate in the process of generating hierarchical network. The detailed process of reallocating bandwidth can be seen in our previous publication. 23 At the system level, messages of available computing resources of all subgroups are shared among leader robots, which means the information of available computing resources of any robot is known by each robot, that is, available computing resources have been contributed to the cloud. At this time, if there is a computing requirement in a robot, available computing resources in the internsufficient cloud can be requested.
When robot i prepares to use the intern-sufficient cloud, firstly, the computation task is supposed to be divided into m sub-computations. Secondly, the required data of the sub-computations will be transmitted to appropriate robots and calculated. Finally, the subresults will be reassembled on robot i. In the process, the computation division time in robot i is t i dc , the data transmission time of a subcomputation for robot j is t ij T ¼ S ij data =Tr ij , the data processing time in robot j is t j p ¼ HðS ij data Þ in which H represents the time function of processing S ij data with available computing resource of robot j, the data transmission time back to robot i of a result is t ji dr , the recombination time in robot i is t i cr , and the time constraint of the task is assumed to t. Finally, the execution time of robot i needs to satisfy The robots who share available computing resources need to satisfy e.g. robot j where t 0 is the time for message queuing, retransmission, and other mechanisms.
Up to now, the whole intern-sufficient cloud is presented, by which the robots are allowed to share computing resources on demand while keeping stable connections.

Implementation of the multitarget navigation task
A multi-robot multitarget navigation task is applied on the intern-sufficient cloud. All of the robots are equipped with normal performance processors, electromagnetic compasses, infrared obstacle avoidance modules, and other cheap but necessary hardware. Some of them are equipped with cameras. During the navigation process, algorithms with high computational complexity, such as image processing, are executed on those robots equipped with cameras while relatively simple algorithms are run in other robots. It is assumed that extra computing resources are needed by the robots equipped with cameras, while surplus computing resources are wasted in other robots.
In the task, a physical target and a virtual target are set, the positions of which are not known by the robots. At the beginning, the robots move randomly to detect and localize positions of the targets. Once the target positions are localized, the robots will be navigated to their target positions.

Navigated to physical target
The robots that are equipped with cameras are supposed to be navigated to the physical target. The navigation rule to the physical target is simple. After the camera perceives the target, the robot moves towards the target. An example is shown in Figure 3. Robot 6 is equipped with differential global positioning system (DGPS), which has high localization accuracy, but other robots can only use the camera for relative localization. Once the connected perception graph as shown in Figure 3 is formed, all robots can obtain absolute position information. In Figure 3, robots 2 and 6 have found the target, so all the robots will move towards the target from now on. In the process, the visual measurement algorithm should be run all the time. A single-camera can intuitively only measure angle information, and navigation using only bearing measurements is challenging. 25 Therefore, the height of the robot is used to calculate depth information where K is camera internal parameters, and Y is bounding box information.
Obviously, an error estimation algorithm is indispensable to improve the accuracy of localization, which is essentially a data fusion algorithm.
The data fusion algorithm is based on extended Kalman filter, which includes two steps, propagation and update. The propagation process can be represented bŷ The update process can be represented by The^, À , and þ mean an estimation, not optimal, and optimal, respectively. Here f represents the nonlinear function of state transition, h represents the nonlinear function of observation, t represents time step, x is the state, u is the control value, and z t is the observation vector. The y is different between measurement and estimated output, and S is the covariance. Here, P is the covariance matrix of state-error, F is the Jacobian matrix of function f called system propagation matrix, and Q is the covariance matrix of process noise. And G is Kalman gain, H is the Jacobian matrix of function h, that is, observation matrix, R is the covariance matrix of observation noise, and I is an identify matrix.
Obviously, extra computing resources are needed by these robots due to the object perception algorithm and the data fusion algorithm, while the communication resources are not in short supply because of that these robots communicate only when perception occurs.

Navigated to virtual target
The robots without cameras are supposed to be navigated to the virtual target, which can be assumed to be the position of the gas leakage source. These robots can sense the concentration of the smell, which is assumed to be normalized reciprocal of distance from the robot to the position of the virtual target. The navigation algorithm for these robots is based on partial swarm optimization, which is a simple version of our previous publication. 26,27 The core equations are where p t i and _ p t i are position and velocity of robot i at time t, respectively. Parameter x t b;i represents individual best position, which is obtained from moving experience of robot i. Parameter x t g is the global best position, which is the position of the robot with the highest signal concentration in the multi-robot system. Parameter ! is an inertia weight, c 1 is self-learning factor, c 2 is a social factor, and r 1 and r 2 are two independent random parameters.
Obviously, idle computing resources are excessive in robots without cameras.

Implementation on the intern-sufficient cloud
When the multitarget navigation task is carried out, the robots equipped with cameras will encounter the issue of communication congestion and lack of computing resources. It can be solved by the proposed internsufficient cloud. The process of using computing resources from the cloud is shown in Figure 4. Simultaneously, in step 1, the signal concentration is also transmitted to calculate the global best position, which is then informed to each robot in step 4. An example of the message format is shown in Figure 5.
Therefore, the key to implement the navigation task in the cloud is to make the task decomposable. In the process of being navigated to the physical target, the task of robot i is divided into six subtasks, which are object detection D t i and components for data fusionx t À i ; G t i ; y t i ; H t i ; P tjtÀ1 i . Firstly, the robots report to their leader robots the requirements of computing resources. Secondly, a task allocation algorithm is run on the leader robots to match the required computing resources and the available computing resources. Thirdly, the leader robots inform the robots of their subgroup that calculations can be offloaded to the corresponding robots. Fourth and finally, the resultŝ x t þ i ; P tjt i will be reassembled on robot i. In this process, a Hungarian algorithm-based task allocation algorithm is proposed. Under the constraints of Eqs. (16) and (17), the required computing resources of robot i is C required i . Inside the subgroup, the available computing resources of robot j is C j . The matching process is shown in Algorithm 3. Here, is a sum of required computing resources of q (q 8 K) robots, where K m , m 2 1; 2; Á Á Á ; l f g , is one of the l permutations, calculated by C q K . This C is an equation of permutation and combination.

Simulation
Two steps, that is, generating intern-sufficient cloud and using resources from it, are simulated sequentially to validate the effectiveness.

Parameters setting
In this case, the large-scale multi-robot system is composed of a hundred robots. These robots and two targets are randomly initialized in a two-dimensional square area (100 m Â 100 m). The bandwidth of the leader robot is assumed to be 40 MHz (20 MHz of 2.4 GHz device and 20 MHz of 5 GHz device). The assumed range of signal power (À90 dBm to À40 dBm) and noise power (À140 dBm to À110 dBm) is closed to the actual condition extremely, which ensures the SNR ranges from 20 dB to 100 dB.
Twenty-five robots inside the system are equipped with cameras. It is assumed that extra computing resources are needed by the robots equipped with cameras, whereas idle computing resources are excessive in the robots without cameras. Specifically, 30% of the processor has been taken up to guarantee the basic functions of the robot. Then, the idle computing resources of the robots without cameras are assumed to be 50% + 10%. The computing resources needed by the robots equipped with cameras are assumed to be 120% + 10%, which, together with the 30% of basic function, is a total of 150%. The +10% is measurement error that follows Gaussian distribution. Overall, in the large-scale multi-robot system, the idle computing resources are 50% Â 75 ¼ 3750%, whereas the extra computing resources needed by the robots equipped with cameras are 50% Â 25 ¼ 1250%.

Effectiveness of the intern-sufficient cloud
The effectiveness of hierarchical network is firstly validated. Then, together with the resource scheduling algorithm, the effectiveness of the intern-sufficient cloud is validated.
Validation of the hierarchical network. The hierarchical network is constructed by TC and SG algorithms. Firstly, as long as an signal to construct the intern-sufficient cloud is received by a robot, TC is run on this robot. After the communication topology is constructed, a subgroup of the intern-sufficient cloud is generated. Then, via SG, the leader robot of this subgroup searches other candidate leader robots according to signal strength, and the selected leader robots generate their subgroups. Go on this loop. Finally, the hierarchical network is constructed. The process is shown in Figure 6.
Furthermore, simulations of the TC process are performed. The results are shown in Figure 7, in which six hops are the maximal in the system with a hundred robots, which can effectively ensure the transmission success rate of lightweight messages.  It can be easily concluded from Figures 6 and 7 that the hierarchical network can effectively connect the robots, in which SG is able to cover all robots and TC can minimize the number of hops.
Validation of resource scheduling. The size of the data to be processed on the intern-sufficient cloud is decided by available computing resources according to HðS ij data Þ, and the data S ij data to be transmitted is constrained by communication conditions, that is, the use of the shared computing resources C shared i requires specific communication resources, called shared transmission rate T shared i . Therefore, it is vital to reasonably schedule the communication resources and the computing resources. In this study, ratio T shared i =T actual i of the shared computing resources to the actual transmission rates is discussed, three kinds of which are proposed, such that where N, n, and m are numbers of all robots, the robots that share all computing resources, and the robots that share    parts of computing resources, respectively. Then, simulation with 10,000 runs is performed, and the results are compared with round robin algorithm and max-SNR algorithm in the mobile cloud, 28,29 as shown in Figures 8 and 9. The U 1 , U 2 , and U 3 in Figure 8 are overall parameters, which show better performance of the proposed method in matching computing resources and communication resources. The distribution of the ratio in Figure 9 is detailed parameters of U 1 , which further proves the effectiveness of the proposed algorithm.
These results are obtained under the condition that the idle computing resources are set to 3750% (parameters setting). Next, how much available computing resources can be contributed is discussed. For more versatile, the idle computing resources within [2300%, 4900%] are simulated, as shown in Figure 10. It can be concluded that computing resources contributed to the mobile cloud can reach 1872% in the specific case of idle computing resources of 3750% and different resource scheduling algorithms lead to different available computing resources, and the proposed algorithm is the best.

Effectiveness of the multitarget navigation
From the data of the intern-sufficient cloud alone, computing resources of 1250% can be easily supported when there are idle computing resources of 3750%. However, the cloud can be proven effective only if it can adapt to the changes of spatial positions (corresponding communication conditions) caused by tasks. Thereby, a multitarget navigation task that applied on the cloud is simulated in this subsection. The scene of 100 m Â 100 m square area with a hundred robots, a physical target, and a virtual target is shown in Figure 11.
The time step is assumed to be 1 s, the task is assumed to be divided into six parts (20% is needed by each part), and the data (0.08 Mb) of each part should be transmitted to the appropriate robot within 0.1 s to meet Eq. (8), so the required transmission rate of the 25 robots can be calculated by 0:08Mb 0:1s Â 4 Â 25 ¼ 80Mb=s, in which "4" represents four parts should be transmitted and calculated on the intern-sufficient cloud. It also presents the relationship that sharing computing resources of 10% needs 0.4 Mb/s. Therefore, the application of the navigation task is validated by whether the total transmission rates of 80 Mb/s can be guaranteed and whether the task requirements of 1250% can be successfully calculated by the internsufficient cloud.
However, during the process of the task, there may not be enough available computing resources for each leader robot inside its subgroup. When the failure happens, the robots keep their former states and keep requesting   computing resources. At each time step, the CPU-take-up rate above 90% is regraded as a failure. Specifically, the failure rate can be represented by where T is total time steps, N t is the number of leader robots at time step t, and f n equals 1 if the CPU-take-up rate exceeds 90%, otherwise it is 0. For example, the dotted line in Figure 12 maybe a failure due to insufficient robots inside the subgroup. In simulation, totally 1000 runs are executed under exactly the same initial setup of navigation and differently varying conditions of communication. A navigation process is shown in Figure 12. The time of completing this process is 18 s. The CPU-take-up rate above 90% occurred 85 times, that is, the failure rate is 4.6%. The results of total 1000 runs are presented in Table 1 and Figure 13.
It can be seen that the multitarget navigation task can be completed even if the cloud maybe ineffective in some time steps, that is, the failure cases only make the completion time longer. We can conclude that the multitarget navigation task can be successfully supported by the internsufficient cloud.
All in all, according to Figures 6 to 10, the proposed hierarchical network is constructed and then combined with the proposed resource scheduling algorithm to construct an   Secondly, when executing a multitarget navigation task in a large-scale multi-robot system, the common problems of communication congestion and corresponding lack of computing resources are solved by deploying it on the internsufficient cloud, as presented in Table 1 and Figures 11 to 13. All the results prove that the computing resources can be simultaneously contributed to and used from the internsufficient cloud efficiently.

Experiment
The physical large-scale multi-robot system is shown in Figure 14, which is composed of 20 mobile robots. An individual robot is shown in Figure 15. The robot uses a Raspberry Pi 3 Model B running Linux OS fixed underneath of the plastic disc with a DGPS, a camera, and a compass on the top, to process the position and signal information and an STM32F407 microcontroller for motor control. The Raspberry Pi communicates with the STM32 controller via a serial port.

Experimental parameters
The operation system, RAM, and frequency of the Raspberry Pi 3 Model B are Raspbian 4.14, 1 GB, and 1.2 GHz, which shows a low-computing capability. The object detection algorithm running on the robots equipped with cameras is NanoNets after transfer learning. During the navigation process, the time step is set to 1 s as in simulation, the intern-sufficient cloud is updated per 5 s, and the information of idle computing resources and estimated transmission rate is updated every time step.
Next, the computing resources, that is, CPU and RAM, needed by the object detection algorithm, the data fusion algorithm, and particle swarm optimization (PSO)-based navigation algorithm are tested. The results after 100 runs are shown in Figure 16.  Here, the PSO-based navigation algorithm, data fusion algorithm, and object detection algorithm are represented by A1, A2, and A3, respectively. The frames per second is 5 when the object detection algorithm runs. It can be seen that the required RAM is relatively stable and sufficient, while the required CPU is unstable and insufficient. Therefore, the required computing resource is represented by the CPU-take-up rate, which are medians, that is, A1 is 5%, A2 is 14%, and A3 is 63%. In addition, a safety factor of 10% is set, that is, if the estimated CPU-take-up rate is higher than 90%, the data fusion algorithm is required to be decomposed and then calculated on the intern-sufficient cloud.
The message to be transmitted shown in Figure 5 is about 0.5 Kb. So, the required transmission rate is defined to be 10 Kb/s, which also has a safety factor 2.

Experimental settings
The field experimental area is a square with a size of 10 m Â 10 m, in which 20 robots, a physical target, and a virtual target are initialized in the area, see Figure 17.
Before the experiment, the robots are randomly initialized in the area, and target positions are the same as in the simulation. Then, the five robots use cameras to detect other robots, to localize themselves, and to discover the physical target, while the 15 robots will be navigated to the virtual target using distance as the signal strength. In this process, extra computing resources are needed by the robots equipped with cameras, but the computing resources of other robots are in surplus. So, the multi-robot navigation task cannot be completed without the intern-sufficient cloud.

Effectiveness
The effectiveness of the intern-sufficient cloud has been theoretically validated in the previous section. However, during the process of the task, there may not be enough available computing resources for each leader robot inside its subgroup. The robots will keep their former states if the   computing requirements cannot be offloaded successfully. The navigation process is shown in Figure 18, and the navigation results are shown in Figure 19.
It can be qualitatively concluded that the internsufficient cloud is effective since all robots have been navigated to their targets. The navigation task lasts 19 s, during which there are robots failed to offload computing requirements. The results are shown in Figure 20. The failure rate is 18.9%, higher than simulation 4.7%. There are two reasons for such a large gap. The first is the update process of the intern-sufficient cloud. At 5 s and 10 s, none of the five robots can offload their computing requirements successfully since the internsufficient cloud is updated per 5 s. At 15 s, four of the five robots have arrived in their target positions, that is, only one robot needs computing resources, and this robot still cannot offload its computing resources. Subtracting these failures, the failure rate is reduced to 7.4%. The second is because the CPU-take-up rate of the physical robot is unstable.
For comparison, the proposed communication mechanism is replaced by least common multiple (LCM). 30 The results are shown in Figure 21. During this process, there are two robots equipped with cameras that have not even moved. Neither communication nor computing resources can be guaranteed under LCM.
Finally, it can be concluded that the proposed internsufficient cloud can provide stable resources of communication and computation for a multi-robot system with 20 robots while effectively achieving a multitarget navigation.

Conclusion
An intern-sufficient cloud for large-scale multi-robot systems is investigated in this study. On the one hand, it does not rely on developed cloud servers and communication infrastructures compared to traditional cloud methods. On the other hand, it maximizes available resources inside the robotic system from the perspective of performing tasks, which is more practical compared to traditional ad hoc cloud methods. To some extent, the intern-sufficient cloud has breakthrough the limitation of the number of robots in multi-robot applications in the real-world, as the simulation and the experiment proved. Specifically, the communication congestion can be alleviated by the hierarchical network, and a robot that needs computing resources is allowed to use available computing resources of the nearby robots. Nevertheless, the small experimental area (10 m Â 10 m) results in generating only one subgroup. In the future, larger experimental areas and more complex tasks are the mean work in the next step.