Bio-inspired artificial pheromone system for swarm robotics applications

Pheromones are chemical substances released into the environment by an individual animal, which elicit stereotyped behaviours widely found across the animal kingdom. Inspired by the effective use of pheromones in social insects, pheromonal communication has been adopted to swarm robotics domain using diverse approaches such as alcohol, RFID tags and light. COSΦ is one of the light-based artificial pheromone systems which can emulate realistic pheromones and environment properties through the system. This article provides a significant improvement to the state-of-the-art by proposing a novel artificial pheromone system that simulates pheromones with environmental effects by adopting a model of spatio-temporal development of pheromone derived from a flow of fluid in nature. Using the proposed system, we investigated the collective behaviour of a robot swarm in a bio-inspired aggregation scenario, where robots aggregated on a circular pheromone cue with different environmental factors, that is, diffusion and pheromone shift. The results demonstrated the feasibility of the proposed pheromone system for use in swarm robotic applications.


Introduction
Pheromone is one of the most effective communication mediums widely used by animals (Camazine et al., 2003;Wyatt, 2014). It is a complex chemical substance that allows intraspecific communication by being secreted into an environment and detected by agents eliciting behavioural and developmental changes. From yeast and small insects such as ants and bees to mammals such as dogs and humans, a wide range of insects and animals adopt pheromone as a communication medium (Fields, 1990;Ho¨lldobler & Wilson, 1990;Preti et al., 2003). Those species benefit from pheromone that has distinct characteristics compared to visual, audio and tactile communication. Especially, social insects use pheromone communication widely (Ho¨lldobler & Wilson, 1990).
In swarm robotics which is the field of study that how large numbers of simple robots coordinate for a shared goal inspired by nature (S xahin, 2005), pheromonal communication has been adopted by researchers. Pheromonal communication has a number of useful characteristics for swarm robotics as well as in nature. For example, pheromone allows a group of agents, which consist of extremely simple sensory-motor and memory capabilities, to achieve fully decentralised control (Khaliq & Saffiotti, 2015). Furthermore, pheromone optimises behaviour of an entire group of robots. Researchers have implemented pheromonal communication with various methods such as chemical substances, radio-frequency identification (RFID) and virtual environments.
COSF is a low-cost light-based artificial pheromone system which ensures high degree of flexibility and fidelity (Arvin et al., 2015). This work and its extension (Na et al., 2019) investigated collective behaviours of a robot swarm with environmental factors including evaporation, diffusion and wind effect on artificial pheromone as they are ever-present phenomena in nature. The work is one of the early studies that embodied environmental effects and investigated impact of environmental factors on collective behaviours of robotic swarm. However, bidirectional communication between the pheromone system and the robots was not achieved; in other words, the robots reacted the given pheromone without releasing any additional pheromone into the system.
In this article, we proposed a novel artificial pheromone system for swarm robotic applications. This article improves the state-of-the-art as (1) highly precise pheromone release mechanism is implemented due to the high resolution of LCD screen. (2) High flexibility of the system is achieved since the characteristics of the artificial pheromones can be easily modified in the software.
(3) Cost-effectiveness of the system enhances the ease of utilisation in swarm robotics studies. (4) Environmental effects, for example, wind effect and diffusion, are implemented in the system based on the reliable mathematical models to make it realistic and having high fidelity. We presented a detailed study of impact of environment factors involving bidirectional pheromonal communication on collective behaviour of robots, which we called it pheromone injection. We specifically investigated influence of diffusion in diverse settings including static and dynamic cue settings and pheromone injection on aggregation behaviour of robots that operate based on F-Clust algorithm (Arvin et al., 2018), which was a pheromone-based biomimetic simulated swarm.

Pheromone in biology
Pheromone is defined as chemical substance secreted to the outside of an individual and detected by conspecifics triggering them to conduct stereotyped behaviour and/or have developmental changes (Karlson & Lu¨scher, 1959). Pheromones are essential factors in animal communication and trigger physiological change in a wide range of behavioural and ecological contexts in nature (Baracchi et al., 2017;Chalissery et al., 2019;Hostachy et al., 2019;Okosun et al., 2019;Tateishi et al., 2020). Pheromones play important roles as communication means for diverse taxonomic groups from yeast and insects to mammals (Fields, 1990).
Pheromonal communications were found in vertebrate animals (Brennan & Zufall, 2006). For example, 2-methylbut-2-enal, which is contained in rabbit milk, improves nipple-search behaviour of rabbit pups (Schaal et al., 2003). Several studies have found that the physiological and psychological effects elicited by chemosignals arise also in humans (Mutic et al., 2017;Preti et al., 2003;Stern & McClintock, 1998;Wedekind & Furi, 1997). For example, it is found that female reproductory state is affected by male axillary extract which mediates the hormonal change in female brain (Preti et al., 2003). These results showed that pheromonal communication is widely used across diverse taxonomic groups including human beings.
Although a broad spectrum of animals use pheromonal communication as their communication mechanism, social insects utilise pheromone most effectively (Ho¨lldobler & Wilson, 1990). Pheromone enables a whole group of social insects to communicate effectively as an externalised and spatialised shared memory although the individuals have limited memory and sensory-motor capability. Also, agents only need local sensing ability to detect pheromone rather than global sensing. Moreover, a collective behaviour in a group of social insects can be optimised appropriately deploying pheromones (Denny et al., 2001;Goss et al., 1989). For example, the Argentine ant Iridomyrmex humilis finds the shortest path from its colony to the food source using pheromone and its feedback mechanisms (Goss et al., 1989;Ho¨lldobler & Wilson, 1990).
Monomorium pharaonis, called Pharaoh's ants, which are usually found in human habitats, utilise multiple pheromones that are vital for food-foraging behaviour in dynamic and competitive environments . They create pheromone trails to fetch to and return from the food source using three types of trail pheromones: non-volatile attractive pheromone, volatile attractive pheromone and repellent pheromone. The three different kinds of pheromones work, respectively, as long-term memory of the trail, attraction leading to the currently rewarding trail and a stop-sign. The sophisticated use of the three types of pheromones enables the ant colony to create optimal pheromone trails from the nest to the food source in dynamically changing environments. Bombus hortorum, known as bumblebee, leaves chemical cue on flowers which allows detection and avoidance of recently depleted flowers (Eltz, 2006). Similarly, the use of chemical cue enhances the efficiency of foraging behaviours of bee colonies since it prevents meaningless visits of depleted flowers. Majority of social insects utilise queen pheromones. This type of pheromones characterise queen and other reproductive individuals and are extremely important to maintain the whole colony (Holman, 2018;Princen et al., 2019). For example, a queen pheromone of Lasius niger, known as the black garden ant, regulates worker sterility so that the reproduction behaviour of the colony is controlled by the queen (Holman et al., 2010). As another example, if the queen of a colony fails by any reasons including viruses and pesticides, the secretion of queen pheromone in the colony decreases, and as a result the colony might collapse (Steinhauer et al., 2018).

Pheromone in robotics
Swarm robotics is the study of how large number of simple agents for a collective behaviour achieved by local interactions among the agents and/or between the agents and the environment (S xahin, 2005). The effectiveness of pheromonal communication is significantly aligned with the needs of swarm robotics. First of all, pheromonal communication requires a simple capability for individual robot since it only needs local sensing (Khaliq & Saffiotti, 2015). Second, regardless of the size of the environment, the individuals only need to have a limited memory since the environment contains information about the state and actions via released pheromone . Third, its potential to optimise the performance of a group task via use of combination of multiple types of pheromone and feedback mechanisms is highly desirable for robot swarms (Denny et al., 2001). Finally, no external control is needed with pheromone-based self-organisation; therefore, a fully decentralised swarm system can be achieved (Jackson et al., 2011). Several researchers have noticed the potential of pheromonal communication in swarm robotics; thus, they adopted pheromonal communication in the simulation and physical robot systems. One of the early works inspired by pheromone communication was introduced by Russell (1999). This work employed Cinnamomum camphora, known as Camphor, as implementation of trail pheromone for trail following behaviours for the robotic systems which embodied odour releasing and sensing functionality. The results of this work demonstrated feasibility of using pheromone in robotic systems as a communication medium. Similar to the work introduced above, Fujisawa et al.
(2014) used ethanol as a pheromone for the robotic system. The work proved that a complete autonomous robotic system that conducts cooperative behaviours with trail following can be achieved using pheromone communication. Despite the successful implementation of chemical-based pheromone communication for robotic systems introduced above, practical difficulties in sensors and actuators are incurred such as cost and resolution of chemical detecting sensors (Purnamadjaja & Russell, 2010).
A number of studies proposed robotic systems using RFID tags as a medium for pheromonal communication (Herianto & Kurabayashi, 2009;Khaliq & Saffiotti, 2015). In these studies, RFID tags were distributed on the floor in the environment where the robots operate. The tags store data transmitted by robots passing above and elicited the corresponding behaviour depending on the data transmitted previously, which works like pheromones in nature. One of the recent works using RFID tags (Alfeo et al., 2019) show its usability in real-world application. This work utilised RFID tag-based pheromone communication for autonomous waste management in urban environment. The work demonstrated pheromone-based communication robotic system outperformed the conventional communication method. Although the approach that uses RFID tags is reliable and usable, it requires appropriately equipped environments with RFID tags, which might challenge users when preparing the environments are not feasible.
Recent studies involved virtual environments to implement artificial pheromone into swarm robotic systems. Campo et al. (2010) proposed a mechanism for path selection by a foraging robot swarm using virtual ants. In this mechanism, robots locally transmitted and received messages. The messages were referred to as virtual ants and they deposited the virtual pheromone on the robots, thereby indicating path. While this work implemented virtual pheromone only within robots, the other works created virtual map to mark deposited pheromone accessible to all the robots. As an example, Reina et al. (2017) introduced the augmented reality for Kilobots (ARK) system that creates a virtual environment where Kilobots swarm (Rubenstein et al., 2012), which has been designed for large-scale robotics experiments, transmits and senses information in real-time.
Through the system, the robots equipped the virtual sensors and actors and they accessed to the virtual environments shared by all robots integrating overhead tracking and control. In the virtual environments, the robots could release and detect the virtual pheromones. One of the following works utilised this system to investigate quality-based foraging of robots (Font Llenas et al., 2018). Similar to ARK system, Kilogrid (Valentini et al., 2018) was proposed as a virtualisation environment system that primarily uses bidirectional infra-red (IR) communication between Kilobots and the grids mounted under the arena. It also created virtual pheromones in the virtual environments accessible to the robots and the remote PC in real-time. Although the two virtualisation systems covered large-scale swarm and are flexible to utilise than other methods, the systems have technical limitations. It is stated that both ARK and Kilogrid implicitly involves low communication frequency that leads to considerable delay in the communication.
Besides the works that are grouped with similar works, several researchers have implemented pheromone communication system with diverse means. Garnier et al. (2007) projected light from a video projector mounted on the ceiling to realise artificial pheromone trail. This was the first work utilising light as artificial pheromone. Although this work is one of the early works using light, it did not replicate pheromone in real-life as it implemented evaporation in every 5 s. In another work, Mayet et al. (2010) used phosphorescent paint that was only observable by projecting ultra violet (UV) light. This work demonstrated that use of phosphorescent paint and the robot that embodied appropriate sensors can be alternative of existing methods like chemical substances. However, its use with multiple robots was not investigated as this work only experimented with a single robot rather than swarm robots.
COSF is the state-of-the-art artificial pheromone system for swarm robotics that used light-based pheromone trails (Arvin et al., 2015). It used LCD screen as the arena for robots to interact with the pheromones which are the light spots displayed on the screen. With the tracking system via a USB camera mounted above the arena, the system updates the position, orientation and ID of the robots and generates virtual pheromone accordingly. Several advantages of COSF as an experimental platform were reported. First of all, this system has a significantly high resolution for implementing pheromone compared to the other works. Unlike the systems using fixed size grids, it offers high-resolution field on which the artificial pheromone can be injected with high precision. Second, diverse environments can be implemented since it is a highly flexible system. From thickness of pheromone trails to evaporation rate of pheromone, all characteristics can be easily modified as needed. Third, it uses a low-cost configuration with a basic digital camera (Krajnı´k et al., 2013(Krajnı´k et al., , 2014 and a flat LCD screen, which makes it accessible to be utilised by many researcher. While the only basic leader-follower scenario was demonstrated in the first work using COSF system to test its validity, following studies have extended the system and investigated collective behaviour of robots. Sun et al. (2019) utilised multiple colours to implement multiple types of pheromone based on the COSF system and investigated food-foraging and aggregation behaviours applying both attractive and repellent pheromones. Moreover, this system extended COSF system to realise the diffusion effect. The use of multi-colour artificial pheromone and adding diffusion highly improved the system; however, the movement of pheromone was not implemented. The movement of pheromone is essential for the pheromone system to be reliable since it realises wind and other causes of pheromone shift. Na et al. (2019) extended the system to include diffusion and wind effect, which are the phenomena almost ever-present in nature. The work demonstrated the performance of collective behaviour of robot swarm with different diffusion and wind rates affected considerably. It is found that when the robots aggregate on the pheromone cue, diffusion helped the robots approach the centre of the cue closer than without it. Also, wind affected the robots' individual behaviour to stay longer on the cue than without wind. To the best of our knowledge, it was the first work that investigated direct impacts of both diffusion and wind effect on collective behaviours of robot swarms. However, this work did not accomplish replicating pheromone communication in real world since pheromone injection was not implemented during the experiments. In other words, the communication between the pheromones and robots was only uni-directional. Since complex swarm behaviours are more likely to emerge from bidirectional interplay between robots and pheromones, the work presented here aims to demonstrate the ability of the proposed artificial pheromone system to emulate realistic swarm-pheromone interactions under natural conditions of variable diffusion and advection.

Artificial pheromone system
The whole set of the artificial pheromone system proposed in this work consists of two components: (1) pheromone system that computes pheromone and displays on the horizontally placed flat LCD screen, representing the scenario arena and (2) tracking system that tracks robots and sends their data to the pheromone system. The two components enable the system to implement real-time localisation and pheromone injection on the precise locations.

Pheromone system
The pheromone system simulates the several types of artificial pheromone and their interaction simultaneously. The resultant pheromone is displayed on an LCD screen, and it triggers predefined responsive behaviours of the employed robots. The system displays the resultant pheromone and the rest part of the arena as a grey-scale image with the size of the screen. The brightness of grey-scale image is represented as I, which is a two-dimensional matrix with the size of resolution of the LCD screen. It is determined by F, which is a twodimensional matrix representing the intensity of pheromone having same size with I. Each element of I is equivalent to brightness of the corresponding pixel. The brightness of the image at position (x, y), I(x, y), is defined as where F i (x, y) represents the intensity of the ith pheromone at position (x, y) and c i denotes the influence of the ith pheromone on the screen. I(x, y) is determined by the summation of multiplication of F i (x, y) and c i , describing that n number of pheromones can be overlapped. As an example of how the model works, the combination of three different pheromones having different influence to the screen can be displayed on a single pixel after the calculation of I(x, y) by equation (1).
While the system is running, the intensity of pheromone released on the screen is constantly updated with discrete time steps. The intensity of updated pheromone is given by where is a twodimensional vector quantity that characterises the gradient of pheromone intensity at a given position (x, y), where it is mathematically defined by equation (3); u represents the velocity vector which linearly shifts the pheromone on the arena; e iF determines the evaporation rate of ith pheromone which is characterised by half-life; k i is the diffusion constant of ith pheromone; and i i (x, y) corresponds to newly injected pheromone at the position (x, y) on the screen. The model of spatiotemporal development of pheromone intensity was derived from the simplified version of Navier-Stokes equation, which characterises the model of fluid flow (Ferziger et al., 2020;Stam, 1999) Recomputing equation (2) for every values of x, y allows to calculate new intensities of pheromones from their previous state, which, in turn, determines the new pixel values of the grey-scale image displayed on the screen.
The parameters shown in the right-hand side of equation (2) can be divided into two categories: (1) environmental effects and (2) pheromone injection. Environmental effects include evaporation rate, e F , diffusion constant, k, and velocity vector, u, where i is omitted to generalise. They influence in the pheromone released on the arena unconditionally and constantly while the system is running. Their effects are described in the next subsection. While the environmental effects have constant influences in the arena, the injection of pheromone, i(x, y), affects the intensity of pheromone in the arena only when the pheromone is injected by the predefined conditions, for example, injecting pheromone only when robots stop. Under the conditions, the pheromone is injected with a circular shape with a given intensity. The injection of pheromone, i i (x, y), is defined as where (x r , y r ), respectively, represent the x and y coordinates of a robot in the arena; s F is the intensity of injected pheromone at a time; and l F is the diameter of injected pheromone. Within a circle with the diameter, l F , where the centre of the circle is the position of the robot, (x r , y r ), the pheromone is uniformly injected with s F .

Environmental effects on pheromone
The environmental effects contribute to the versatility of the system in emulating realistic conditions that affect the distribution of the pheromones in the environment over time.
3.2.1. Evaporation. Evaporation is the process by which the surface of a liquid turns into the gas phase. As volatile chemical substances, evaporation occurs in secreted pheromone. In several works on kinetic properties of pheromones, half-life of pheromones, which is the time required for pheromones to decay by half, are investigated as a metric (Vogt et al., 1985). Conforming this practice, the half-life of the pheromone is adopted, that is, e iF represents the half-life of the pheromone, see equation (2).

Diffusion.
Diffusion is a movement of molecules from a region of higher concentration to a region of lower concentration. Implementing diffusion in the pheromone system is indispensable to be a realistic system since diffused pheromone from the source has a great impact on swarm behaviour (Wyatt, 2003). In this work, diffusion is implemented using Gaussian blur instead of directly using the term, k i r 2 F i (x, y), which is a mathematical definition of diffusion. The advantage of Gaussian blur over the original definition is that it can emulate faster diffusion with lower computational costs. The intensity of pheromone at the position (x, y) after application of the Gaussian blur is given by where F k + 1 i (x, y) is the intensity of ith pheromone at the discrete time k + 1, F k i (x, y) is the intensity of ith pheromone at the discrete time k and v is a twodimensional kernel matrix with the size of (2a where s is a standard deviation of the Gaussian distribution of the kernel matrix. Equation (6) shows that the elements of v are determined by the Gaussian distribution. Apart from the computational efficiency, the Gaussian blur allows more intuitive control of the diffusion rate and area. Adopting Gaussian blur into the pheromone system does not principally violate properties of diffusion for two reasons: (1) the higher intensity of pheromone decreases and the lower intensity of pheromone increases after computation of Gaussian blur and (2) the total amount of pheromone is preserved after every computation.
3.2.3. Pheromone shift. In real and dynamic environments, the position of the released pheromone can be shifted. A natural cause of the shift in nature is advection, which is the flow of any fluid, for example, air, that transfers pheromone from one position to another. The movement of released pheromone in the same direction is modelled as u � rF(x, y). The twodimensional velocity vector is defined as where u x , u y , respectively, represents the speed along x-axis and y-axis.

Tracking system
The fast and precise open-source localisation system (Krajnı´k et al., 2014) is used to track the position, orientation and ID of the robots, thereby precisely injecting pheromone at the position of the robots on the LCD screen in real-time. The system captures images using a digital camera mounted above the screen, searches for black-and-white roundel patterns and converts their image coordinates into the realworld coordinates in the Cartesian plane. By attaching unique patterns on the individual robots, each robot can have unique ID. Krajnı´k et al. (2014) claimed that the localisation precision can achieve millimetres scales and hundreds of patterns can be tracked simultaneously in real-time. By transforming the captured images into the coordinates of the tracked patterns in the defined area and sending the data to the pheromone system, the tracking system allows the pheromone system to release the pheromone at precise positions of the robots in real-time. Precise localisation of the patterns can be achieved through auto-calibration of the system that makes the pheromone system robust under any circumstances, for example, external disturbance during experiments. The system defines the area of the arena in the coordinate system from the image by setting the four corner tags attached on the corners of the frame placed on the screen, and determine the coordinates of patterns inside of the defined area. Since the system requires less than 50 s to calculate one robot position, the difference between the pheromone injection position and the actual robot position is determined by the time required to transfer the image via the USB interface and the delay caused by the graphic interface driver. Nevertheless, the difference between the expected pheromone positions and the actual positions is negligible.

Experimental setup
4.1. System configuration 4.1.1. Arena. To implement the artificial pheromone system, a high-definition (HD) 42$ LCD screen with the size of 92 3 50 cm 2 was used as an arena on which the pheromones were displayed and robots operated. The light sensors of the robots were faced to the screen; hence, the sensors allowed the robot to read the illuminance at the current position of the robot. On the top of the screen, the aluminium arena frame was set on the edge. The frame allowed the robots to detect the boundary of the arena; therefore, they can turn to another direction when they are close to the boundary. The four corner tags were attached on the frame that allowed the tracking system to define the arena. A low-cost digital camera was mounted on the frame above the centre of the arena. By receiving the images from the arena in real-time, the tracking system determined the robots' positions and status if the robots are randomly moving or waiting. Figure 1 shows the arena setup with a PC which controls the pheromone system, connected to the LCD screen.
4.1.2. Robotic platform. Colias micro-robot (Arvin et al., 2014) which was developed for swarm robotic applications was used as a robotic platform. The front and bottom view of the robot is shown in Figure 2. It is a small robot with a diameter of 4 cm having simple functionalities. The robot is a differential wheeled robot which movement is determined by two micro DC gear-head motors directly connected to wheel with the diameter of 2.2 cm. The speed of the robot in the forward motion is maximum 35 cm/s. The rotational speed of each motor is controlled by pulse-width modulation (PWM). Each motor is driven by the embedded H-bridge DC motor driver which draws an average current of 35 6 5 mA when there is no-load and 150 6 20 mA at maximum in stall conditions. For its sensing, the robot has three IR proximity sensors which include pairs of IR emitter and receiver in front of the robot. It is used to detect objects, obstacles or other robots, within a distance of approximately 2 6 0.5 cm. In addition, the robot has two light (illuminance) sensors at the bottom next to the wheels. The light sensors are used to read light intensity on the ground where the robot is located, that is, read the intensity of the pheromone in this work. The robot's power consumption is approximately 800 mW.
In this work, the behaviour of the robot was determined for achieving pheromone-based aggregation adopted from one of the previous studies (Arvin et al., 2018). The state machine of the scenario is described in Figure 3. The robot begins to move forward after it is switched on. The rotational speed of the left and right wheels, N l and N r , are defined as where a is the velocity sensitivity coefficient; b is the biasing speed; and s l and s r are the sensor reading from the left and right light sensors. b is defined as b is modelled in order for the robot to have slower speed where the average intensity detected by both the sensors is higher so that the robot stops where the robot reaches the area with the considerably high intensity of pheromone. As depicted in Figure 3, the robot has two different states transited after it detects an object. If the object is an obstacle, it rotates to the opposite direction at which the obstacle is. The robot distinguishes whether the object is an obstacle or another robot by checking if IR read by a sensor is emitted from other robots, not from ones embodied in the robot itself. If the object is another robot, it transits to the waiting state and starts to inject pheromone at the position of the robot. The waiting time t w is defined as where t wmax is the maximum waiting time, 20 s, which happens at the highest pheromone intensity; s avg is the averaged value of s l and s r . Depending on the illuminance on the bottom of the robot, the robot can wait between 0 and 20 s. While the robot is waiting, pheromone of a circular shape with a diameter of 2.5 cm is injected to the environment. The rate of pheromone injection is approximately 20%/s, which suggests that it takes approximately 5 s to reach the maximum intensity.

Experiments
We designed an experiment plan to systematically investigate the proposed pheromone system. The set of experiments adopted the pheromone-based aggregation scenario. The experiments were implemented with two configurations: (1) without pheromone injection and (2) with pheromone injection by the robots -called as without F and with F, respectively, for simplicity in the following sections. Each configuration of the experiments has different cue conditions -static and dynamic. In every set of experiments, the identical initial setting was provided. A circular cue with the diameter of 25 cm was generated at the beginning of the experiments at the position (x c , y c ) = (70, 25) cm of the arena, where the coordinates (0, 0) refers to the bottom left corner of the arena. On the left-half of the screen, the position and orientation of the robots were determined in a stochastic manner with markers appearing at random positions on the screen. Before the experiment began, the robots were placed on the markers, and when the experiments started, the markers disappeared, the robots then started to move and the main circular cue was generated on the specified position on the right-half of the arena simultaneously. The duration of an experiment was T = 300 s. Note that, in the experiments in the configuration without F, pheromone was not injected while the robot was waiting after colliding with another robot. The purpose of the experiments was to investigate the impact of pheromone injection on collective behaviour of the swarm. The screenshots from experiments with different configuration is shown in Figure 4. The parameters and their values are listed in Table 1.

Static cue configuration.
In static cue configuration, the position of the cue (x c , y c ) remained the same during the experiments. In other words, the cue did not move (u = 0 cm=s). In this configuration, three different diffusion coefficients, k 2 f0%, 50%, 75%g=T , which are named as 'No Diffusion', 'Slow Diffusion' and 'Fast Diffusion', respectively, with two swarm sizes of N 2 f4, 6g robot were investigated. To show the effect of diffusion intuitively, how much pheromone at the centre of the cue is diffused over the duration of experiment, T , is given as a constant k. Each diffusion coefficient k 2 f0%, 50%, 75%g=T, described in equation (2) is equivalent to s 2 f0, 6, 20g, a, b = 7 for the Gaussian blur kernel matrix, which is characterised in equations (5) and (6). For each different diffusion setting in this configuration, five independent runs of experiment for T were carried out.

Dynamic cue configuration.
In dynamic cue configuration, the position of the cue (x c , y c ) was moved horizontally with a constant speed. Moreover, injected pheromone during the experiments also moved with the same speed with the cue. In this configuration, two different cue speeds were applied: (1) the medium speed (the centre of the cue moves with speed of u = 0:1 cm=s on the arena) and (2) the fast speed (the centre of the cue moves with speed of u = 0:2 cm=sÞ. Similar to the static cue configuration, five independent runs of experiments with three different diffusion, k 2 f0%, 50%, 75%g=T , and two swarm sizes, N 2 f4, 6g, for duration of T were conducted.

Metrics.
To evaluate the aggregating behaviour of swarm, two parameters were defined: (1) size of aggregate, n a , and (2) cohesiveness, d coh . The size of aggregate determines how many robots are aggregated, which is equivalent to the number of the robots are waiting on the circular cue. The robots that are waiting on the outside of the cue because of the injected pheromone are not counted. Cohesiveness determines the quality of the aggregation behaviour of the swarm. Cohesiveness is the reciprocal of the averaged value of the distances of the robots from the centre of the cue. The cohesiveness is defined as where (x i , y i ) is the Cartesian coordinates of the ith robot of N robots.

Statistical analysis.
To statistically analyse the observed results from experiments, analysis of variance (ANOVA) test was conducted. ANOVA test is a test used to analyse the difference among groups caused by difference in factors (Scheaffer et al., 2010). F-statistic (F) in ANOVA indicates how the factor makes difference between the means of the samples of different groups. If F-statistic is high, the factor is regarded as a significant factor for sample means. Typically, F . 1 is considered as high F-statistic. In addition, p-value, which is the smallest significance level at which the hypothesis that the factor does not significantly impact on samples is rejected. When p\0:05, it is regarded that the impact of the factor is significant. In this work, F and p are calculated for the factors: (1) population, (2) diffusion and (3) time to analyse their impacts on the swarm behaviour and determine the strongest factor that affects the swarm behaviour in a more statistical manner.

Results
The results of experiments are presented in this section.
The results are depicted with line plots. The plots show the aggregation performance, both size of aggregate and cohesiveness with different factors: population, diffusion rate and with different configurations: cue configurations and pheromone injection. In the line plots, a line represents the median of the observed data from five repetition and the shaded region surrounding the line represents the inter-quartile range of the data.

Static cue configuration
Here, we depict how the aggregation performance of robots varies with different diffusion rates in static cue configuration (u = 0 cm=s). In each plot, the observed data from experiments with no diffusion, medium diffusion and fast diffusion are represented as red, blue and green lines, respectively. Figures 5 to 8 show the size of aggregate and cohesiveness with N 2 f4, 6g robot.
5.1.1. Diffusion. First, we investigated the impact of diffusion on swarm behaviour. Figures 5 and 6 show the size of aggregate with three different diffusion rates k 2 f0, 50, 75g%=T in two different population sizes N 2 f4, 6g both without F and with F. In both figures, a decrease in the size of aggregate was observed when medium and fast diffusion was applied without F. In Figure 5(a), the size of aggregate with medium and fast diffusion began to decrease after certain time (t = 70 s) while the size of aggregate with no diffusion stably stayed in the range from 2 to 4 throughout the experiment. Moreover, the decrease in the size of aggregate with fast diffusion was rapid than the decrease with medium diffusion. In Figure 5 (4) with N = 6 robots. The size of aggregate with medium and fast diffusion decayed after a certain time whereas the size of aggregate with no diffusion stayed in a range from 3 to 5 quite stably. Fast diffusion caused a radical decrease in the size of aggregate from t = 30 s while medium diffusion led less steep decrease than fast diffusion. Different diffusion rates led different cohesiveness in the robots. In Figure 7(a), cohesiveness with fast diffusion before t = 50 s was considerably higher than with the two other diffusion rates. Similar phenomenon was observed in Figure 8(a), where cohesiveness with fast diffusion was the highest among with three different diffusion before t = 30 s.
As it was expected, diffusion led the gradual decay in pheromone intensity in the system; therefore, robots less decreased their speed when the cue was diffused than when there was no diffusion. The degree of decay in intensity was higher in the outer parts of the cue than in the inner parts. When the robots reached the edge of the diffused cue, they kept moving forward while they stayed on the edge of the cue that was not diffused. As a result, the robots approached closer to the centre of the cue in experiments with diffusion than experiment without diffusion. Therefore, diffusion led high cohesiveness of the swarm while the intensity of pheromone was sufficient for robots to stay on the cue. Figures 5(a) and (b), it is shown that robots aggregated in higher probability on the cue throughout the experiments regardless of diffusion rates with F than without F. Moreover, the decrease in size of aggregate in experiments with N = 4 robots in case of fast diffusion with F was considerably delayed. This difference caused by pheromone injection  was also observed in experiments with N = 6 robots. In Figure 6, it is shown that the decay in the size of aggregate with medium and fast diffusion was slower with F than without F. The impact of diffusion that increases the cohesiveness of robots was amplified by pheromone injection. In Figure 7, it is observed that the range of cohesiveness was remarkably higher with F than without F. For the cases without being subjected by amount of diffusion -no diffusion and medium diffusion -the cohesiveness was above 6 with F while most of the time it ranged from 2 to 4 without F. The increase in cohesiveness with F in N = 6 is also shown in Figure 8, although it seems not as influential as in N = 4.

Pheromone. Comparing
The results suggested that the pheromone injection offsets the impact of diffusion on the size of aggregate. Since the intensity of the cue increases when pheromone is injected, the robots are likely to stay on the cue. Hence, the size of aggregate is higher with F than without F. Due to the higher size of aggregate, the cohesiveness also increases. In experiments with diffusion, robots are more likely to inject pheromone while they are waiting close to the centre; therefore, the cohesiveness is higher.

Statistical analysis.
To statistically analyse the results, a fully nested ANOVA test with factors of population, diffusion and time was carried out to find significance of the factors. Also, the most effective factor on the size of aggregate and cohesiveness in both configurations: (1) with F and (2) without F was determined. Tables 2 and 3 show the results of ANOVA tests on the size of aggregate and the cohesiveness with different cue configuration, respectively. For the size of aggregate, diffusion and time were the significant factors while population had no effects on the size of aggregate in both configurations (p . 0:05). Diffusion had the most significant influence in both configurations. The significance of time suggests that the size of aggregate was time-variant. In accordance with the results of ANOVA test on the size of aggregate, diffusion and time were the significant factors on the  cohesiveness. The most influential factor for the cohesiveness is also diffusion in both configurations (F = 12.66 for with F and F = 30.58 for without F). Population size did not have a significant impact on the cohesiveness, and time had a weaker impact than diffusion as identical as the results in the ANOVA test on the size of aggregate.

Dynamic cue configuration
In this section, the aggregation performance of the robots with three different diffusion rates (k 2 f0, 50, 75g%=T ) and different cue speeds (u 2 f0:1, 0:2g cm=s) is presented in the dynamic cue configuration. The dynamic cue configuration with two different speeds: (1) medium speed and (2) fast speed. Figures 9 to 12 show the size of aggregate and cohesiveness with N 2 f4, 6g robot with medium cue speed, respectively. The subsequent four figures, Figures 13 to  16, show the size of aggregate and cohesiveness with N 2 f4, 6g robot with fast cue speed, respectively. Figures 9(a), 11(a), 13(a) and 15(a) display the impact of diffusion on the size of aggregate in dynamic cue configuration with two population sizes (N 2 f4, 6g robot) and two cue speeds (u 2 f0:1, 0:2g cm=s). The decrease in the size of aggregate appeared in fast diffusion in all the four figures regardless of the cue speed and population. Moreover, this effect only arose in fast diffusion. The   size of aggregate with medium diffusion with different population size and cue speed which is displayed in Figures 9(a), 11(a), 13(a) and 15(a) did not have notable different trends compared to the size of aggregate with no diffusion in both cue speed. The results shown in Figures 10(a), 12(a), 14(a) and 16(a) demonstrated the impact of diffusion on the cohesiveness in dynamic cue configuration with two population (N 2 f4, 6g robot) and two cue speed (u 2 f0:1, 0:2g cm=s). In Figure 10(a), it is shown that with medium speed, the cohesiveness with diffusion was higher than with no diffusion. The cohesiveness with medium diffusion was higher than with no diffusion throughout the experiment. Although the median of the cohesiveness with fast diffusion was lower than with no diffusion after t = 80 s, the inter-quartile range suggests that the cohesiveness with fast diffusion was higher than with no diffusion in two sets of experiments. Especially, it was high with fast diffusion when the cue still remained. Likewise, with population N = 6 robots, the cohesiveness was higher with diffusion than no diffusion (see Figure 12(a)). When the cue was still not diffused over a certain amount, the cohesiveness with fast diffusion surpassed other two diffusion rates. With medium diffusion, the cohesiveness was higher than other two diffusion rates in the end of the experiments. The increase in the cohesiveness resulted from increasing diffusion rate does not clearly appear in experiments with fast speed. In Figure 14, the difference of the cohesiveness between experiments with no diffusion and medium and fast diffusion is not as clear as seen in medium speed cases.

Diffusion. The results shown in
The results showed that the effect of diffusion led in decay in the size of aggregate and increase in the cohesiveness as seen in static cue configuration; however, the degree of the effect was smaller in dynamic cue configuration than in static cue configuration. As the cue speed increased, the diffusion effect became less  influential. It is because the moving cue dragged the robots with the same direction with the cue and it hampered robots staying closer to the centre of the cue.

5.2.2.
Pheromone. The impact of pheromone injection on the size of aggregate is displayed in Figures 9, 11, 13 and 15. In the three Figures 9, 11 and 15, the size of aggregate in experiments with fast diffusion slowly decreased later with F than without F. This effect of pheromone injection seemed greater in experiments with N = 6 robots than N = 4 robots. Figures 10, 12, 14 and 16 depict the difference in the cohesiveness in experiments with three diffusion rates (k 2 f0, 50, 75g %=T ) between with F and without F. Pheromone injection had a greater impact on the cohesiveness with N = 4 than N = 6 regardless of cue speed. The impact of pheromone injection seemed stronger in experiments with N = 4 robots than N = 6 robots with smaller difference in different cue speeds. The reason that the effect of pheromone injection that cancelled the effect of diffusion is greater in experiments with N = 6 robots than N = 4 robots is that the probability of collision rose as the population increased. Similarly, the impact of pheromone injection on the cohesiveness in N = 4 can be explained that rather than they collided with each other outside of cue, they collided with higher probability when they are close to the centre of the cue; therefore, the cohesiveness was higher with F. However, with N = 6 robots, the robots collided with each other more frequently in the outer part of the cue than with N = 4 robots; thus, there was no remarkable increase in the cohesiveness.   of aggregate and the cohesiveness in experiments with N 2 f4, 6g robots with medium cue speed and the last four figures show the size of aggregate and the cohesiveness in experiments with N 2 f4, 6g robots with fast cue speed. We can observe that the increase in the cue speed caused the decreased effect of pheromone injection on the size of aggregate. Comparing Figures 9(b) and 13(b), the decay of the size of aggregate with fast diffusion occurred faster with fast cue speed than medium cue speed. Similarly, the smaller impact of pheromone injection on the size of aggregate with fast cue speed than medium cue speed was observed in experiments with N = 6 robots comparing Figures 11  and 15. Whereas the pheromone injection with medium speed seemed to have an impact only with fast diffusion, the pheromone injection with fast speed affected the size of aggregate with no diffusion. We can also see the increase in the cue speed causes the decrease in the cohesiveness. This observation is not clearly shown in N = 4. Comparing Figures 10 and 14, the cohesiveness rather increased as the cue speed increased with F, whereas the cohesiveness slightly decreased without F for all diffusion rates. However, the cohesiveness considerably decreased in experiments with N = 6 robots (see Figures 12 and 16). The difference was notably featured the cohesiveness with fast diffusion, and the difference was not as remarkable in experiments with no diffusion and medium diffusion.
The difference observed between the experiments with two different cue speeds was likely to occur because the cue moved away from the robots which were in the waiting phase more quickly with the faster cue speed. The position at which the pheromone was injected was always where the robot was waiting. Therefore, the position of injected pheromone and the cue moved away further from each other in experiments with fast cue speed than medium cue speed. As a result, the greater number of robots tended to stay on  the injected pheromone outside the cue with fast cue speed, thereby the decrease in the size of aggregate. The decrease in the cohesiveness by increasing the cue speed was also caused by the same reason why the impact of pheromone injection decreased. Since the cue passed the robots too fast while they were waiting after collision at the centre of the cue, the cohesiveness rapidly decreased especially when it was high. This observation also supports why there was no considerable difference of the cohesiveness between experiments with medium speed and fast speed, where in both cases of no diffusion and medium diffusion was applied.

Statistical analysis.
The results from the experiments in the dynamic cue configuration with two different cue speeds were also analysed using ANOVA test (see Tables 2 and 3). The statistical analysis revealed that diffusion and time were two significant factors (p\0:05) for both experiments with and without F. Diffusion was the most significant factor on both the size of aggregate and cohesiveness, both with and without F regardless of the cue speed. It is also shown that this system was time-variant (p\0:05). Although both size of aggregate and cohesiveness with two different cue speeds were influenced by diffusion the most, the impact of diffusion on cohesiveness with medium cue speed is the lowest. F = 13.6 and 13.0 for experiments with and without F, respectively, while F is greater than 30 in the test of size of aggregate with medium cue speed and both the metrics with fast cue speed.

Discussion
The observation made from the results suggests that the environmental effects and pheromone injection have an impact on the performance of collective behaviour of robots: size of aggregate and cohesiveness. The impact of the environmental effects and pheromone injection of robot swarm across different population sizes is discussed in detail here.

Static cue configuration
In static cue configuration, we have found that the trend of swarm behaviour did not noticeably vary between two population sizes (N 2 f4, 6g robot). Statistical analysis of the results showed that the population size did not affect the size of aggregate and cohesiveness. It means that the ratio of the size of aggregate given the population size did not differ by the population size. In other words, the aggregation performance increased linearly with the population size. This relationship between the population and swarm performance was also reported in several works (Arvin et al., 2016(Arvin et al., , 2018. While the decrease in swarm performance using pheromone-based communication with highdensity robot swarms was reported in Hamann (2013) and Hunt et al. (2019), this work did not show the decrease in the performance due to relatively small sizes of robot swarm.
In the set of experiments without F, we investigated the effect of diffusion on the swarm aggregation performance when there was no feedback provided by bidirectional communication. In the plots, the decrease in the size of aggregate and increase in the cohesiveness by increasing diffusion rate was observed. It was revealed that diffusion had the most significant impact both on the size of aggregate and cohesiveness. This result is identical with the impact of diffusion on the swarm behaviour of robots reported in Arvin et al. (2018). Such impact of diffusion can be exploited for desired swarm behaviour (Payton et al., 2001).
In the set of experiments with F, we studied the impact of pheromone injection as a feedback mechanism on the swarm aggregation performance with different diffusion rates. The results revealed that activation of the feedback mechanism had a substantial impact on the swarm behaviour. The effect of diffusion that reduces the size of aggregate was diminished by the feedback via pheromone injection. The statistical analysis confirmed that pheromone injection was one of the significant factors affecting the swarm aggregation performance (see Table 4). The similar impact of the pheromone injection that increases the swarm behaviour was reported in several works both in biology (Sumpter & Beekman, 2003;Wyatt, 2003) and robotics (Arvin et al., 2018;Liu et al., 2007).

Dynamic cue configuration
In dynamic cue configuration, we investigated the impact of dynamic environment as well as the factors in static cue configuration. As identical in static cue configuration, the population size did not have significant impact on the swarm performance according to the statistical analysis (see Tables 2 and 3).
In the set of experiments without F, similar results in the experiments with static cue configuration were observed for both two cue speeds. The increase in diffusion rate led the decrease in the size of aggregate and the cohesiveness. Although the same impact of diffusion appeared in both static and dynamic cue configuration, the amount of the impact was differently observed. In dynamic cue configuration, it is shown that the time taken for the robots to completely leave the cue was delayed than in static cue configuration. This delay indicates the moving cue dragged the robots to the cue rather than randomly moving. Nevertheless, the statistical analysis confirmed that diffusion was the most influential factor of the swarm performance.
The effect of pheromone injection in dynamic cue configuration was investigated in the same manner used in static cue configuration. The results demonstrated that the pheromone injection affected the swarm performance. Although the observed impact was identical with static cue configuration, the observation in dynamic cue configuration was not as clear as in static cue configuration. In Figures 9 and 13, the trend of the size of aggregate in experiments with F was not clearly distinguished from experiments without F whereas the impact of pheromone injection weakening the diffusion effect was clearly shown in the other configurations. Despite of this vagueness of the impact of pheromone injection shown in the mentioned figures, the results in corresponding figures displaying cohesiveness still showed that the pheromone injection had impact. As seen in Figures 10 and 14, the increased cohesiveness in experiments with F compared to without F is described. This result posited that pheromone injection allowed the robots to stay close to the cue rather than randomly roaming around. The impact of pheromone injection was significant on the swarm performance according to the statistical analysis. The robustness of the swarm using the feedback via pheromone injection against the environmental factors was similarly reported in Liu et al. (2007) and Stewart & Russell (2006). Figure 17 shows the overall experiments designed to show the impact of cue speed primarily. The results depicted that the cue speed did not deteriorate the swarm performance. Interestingly, in experiments with the medium cue speed, the size of aggregate reached the highest regardless of diffusion and pheromone injection. This observation suggests that the gradient made by moving pheromone with a moderate speed leads the highest responsiveness on pheromone for robots. This guidance effect using the gradient in pheromone trails was reported in Nieh et al. (2004). As well as the observation of maximised dragging effect with the medium cue speed, it is worth noting that the size of aggregate with the fast cue speed fluctuated shown in Figure 17(d). It was caused by the definition of the cue location. As mentioned in 'Results' section, it is shown that the swarm aggregate was formed around the cue, rather than roaming. Therefore, the swarm was still robust with this extreme environmental effect. The statistical analysis showed the cue speed was significantly influential to the robot swarm. This phenomenon that the increase in cue speed sustains the size of aggregate against the environmental effects was also reported in Na et al. (2019).

Conclusion
In this study, we proposed a state-of-the-art artificial pheromone system, which, unlike the previous systems, allows efficient and realistic emulation of environmental effects on the pheromone distribution. We demonstrated that emulation of realistic spatio-temporal development of pheromone, involving pheromone diffusion and advection, can bring new insights into the interplay of the swarms, released pheromones and the environment. One of the interesting findings is that the ability of the swarm to release pheromones counters adverse effects of moving cue, which can displace environmental cues. In future works, we will realise scenarios with more pheromone types implemented by different colours. Using pheromones with different diffusion, evaporation and advection rates will allow investigation of richer pheromone-swarm-environment interactions emulating complex biological swarms in the real world. Since the effect of time is paramount to the swarm behaviours, we will combine the pheromone field with the system for on-the-fly recharging systems for swarm robotics.