Controlling two-dimensional collective formation and cooperative behavior of magnetic microrobot swarms

Magnetically actuated mobile microrobots can access distant, enclosed, and small spaces, such as inside microfluidic channels and the human body, making them appealing for minimally invasive tasks. Despite their simplicity when scaling down, creating collective microrobots that can work closely and cooperatively, as well as reconfigure their formations for different tasks, would significantly enhance their capabilities such as manipulation of objects. However, a challenge of realizing such cooperative magnetic microrobots is to program and reconfigure their formations and collective motions with under-actuated control signals. This article presents a method of controlling 2D static and time-varying formations among collective self-repelling ferromagnetic microrobots (100 μ m to 350 μ m in diameter, up to 260 in number) by spatially and temporally programming an external magnetic potential energy distribution at the air–water interface or on solid surfaces. A general design method is introduced to program external magnetic potential energy using ferromagnets. A predictive model of the collective system is also presented to predict the formation and guide the design procedure. With the proposed method, versatile complex static formations are experimentally demonstrated and the programmability and scaling effects of formations are analyzed. We also demonstrate the collective mobility of these magnetic microrobots by controlling them to exhibit bio-inspired collective behaviors such as aggregation, directional motion with arbitrary swarm headings, and rotational swarming motion. Finally, the functions of the produced microrobotic swarm are demonstrated by controlling them to navigate through cluttered environments and complete reconfigurable cooperative manipulation tasks.


Introduction
Untethered mobile miniature robots without on-board actuators and power sources could be actuated by external stimuli, such as light (Palagi et al., 2016;Sridhar et al., 2018), acoustics (Ahmed et al., 2017), and electrostatic and magnetic fields (Liu et al., 2017;Palagi and Fischer, 2018). In particular, mobile microrobots actuated and controlled by external magnetic fields can access distant, enclosed, and small spaces, such as inside microfluidic channels and the human body, making them appealing for minimally invasive applications, such as biomedicine (Ceylan et al., 2017;Erkoc et al., 2019;Nelson et al., 2010;Sitti, 2017Sitti, , 2018Sitti et al., 2015;Yasa et al., 2018). While single magnetic miniature robots have already shown promising functionalities Hu et al., 2018;Li et al., 2017;Martel, 2017;Park et al., 2017;Singh et al., 2017;Yan et al., 2017), their simplicity increases considerably when scaling down to submillimeter-or micro-scale, limiting their capabilities. Despite the simplicity of individual magnetic microrobots, a collective of these microrobots that can work closely and cooperatively would significantly enhance their functionalities. Such collective microrobots could together exert higher forces and have more advanced functionalities by cooperatively transporting multiple objects using different formations compared with individual microrobots, providing high throughput and fast parallel distributed operation. Moreover, when equipped with advanced programmability of formations and collective motions, these collective magnetic microrobots could also have unprecedented applications in programmable self-assembly (Whitesides and Grzybowski, 2002), swarm systems (Rubenstein et al., 2014;Yang et al., 2018), and modular robotics (Daudelin et al., 2018).
However, enabling a large number of magnetic microrobots to work cooperatively is challenging because magnetic microrobots are easy to cluster due to magnetic interactions and their individual motions are highly coupled. Many methods have been proposed aiming at controlling magnetic microrobots to work closely and cooperatively in large numbers (Chowdhury et al., 2015), e.g., as microrobotic swarms, with programmable and reconfigurable functions.
One strategy in existing works is to individually address and control multiple microrobots by planning their heterogeneous dynamics or kinematics in 2D and 3D environments with global actuation signals (Diller et al., 2012(Diller et al., , 2013aFrutiger et al., 2010), where a small number of magnetic microrobots (up to three) have been controlled independently. Another strategy is to sequentially and spatially select specified microrobots to move, while anchoring other robots by auxiliary forces such as electrostatic forces (Pawashe et al., 2009) or friction induced by magnetic gradient pulling forces (Rahmer et al., 2017). The design and control of these microrobots are limited to individuals or small robot teams due to the disability of these methods for the design and parallel control of large numbers of microrobots. Other methods use local magnetic forces created by miniature coils (Chowdhury et al., 2016;Pelrine et al., 2012) to trap multiple millimeter-sized magnetic robots. It is difficult to scale down the robot size due to the coil resolution and close operation of multiple robots is also difficult because of magnetic interactions.
Instead of individually addressing and controlling microrobots in a team, other works have proposed to simultaneously assemble ferromagnetic microrobots (Diller et al., 2011;Han et al., 2017;Miyashita et al., 2013;Torres and Popa, 2015;Wang et al., 2017) into programmed formations or structures. The realized formation can be reconfigured using a proper disassembly method but the formation is relatively simple. Moreover, other works actuate paramagnetic microrobots such as helix swarms (Servant et al., 2015;Wu et al., 2018), magneto-bacteria clusters (De Lanauze et al., 2014;Felfoul et al., 2016), and paramagnetic microparticles or nanoparticles in formations of asters (Snezhko and Aranson, 2011) and ribbons (Yu et al., 2018a, b). These assembled structures rely on a rotating or vibrating external magnetic field and usually work as a soft body with limited morphologies.
Although these existing methods all represent advancements towards programmable and reconfigurable cooperative magnetic microrobot swarms, it is still a challenge to produce versatile desired formations among a large number of magnetic microrobots, owing to the lack of systematic methodology for actuation and control.
On the other hand, it is also challenging to enable complex collective motions and reconfigurable cooperative functions into these magnetic microrobots. Although cooperative manipulation using collective magnetic matter has been shown before in micro-scale systems in experiments (Snezhko and Aranson, 2011;Torres and Popa, 2015;Wang et al., 2018a, b ;Xie et al., 2019) and simulations (Becker et al., 2013;Shahrokhi et al., 2018), they cannot complete versatile manipulation tasks on demand owing to the limited reconfigurability. For example, dynamically assembled magnetic asters have been shown to trap objects at a liquid-liquid interface (Snezhko and Aranson, 2011). In the approach of Torres and Popa (2015), submillimeterscale ferromagnetic microrobots in a disk-shaped formation have been controlled by non-uniform external magnetic gradient forces to collectively transport objects on 2D solid substrates (up to 14 robots). Magneto-bacteria clusters have been used to assemble 2D structures via a pushing-type manipulation strategy by Martel and Mohammadi (2010). Ferromagnetic microparticles in chains, ribbons, and ''vortex''-like patterns controlled by rotating or vibrating magnetic fields have been shown to collectively push objects on 2D solid substrates (Xie et al., 2019).
In this article, to overcome these challenges, we report a method of controlling desired 2D static and time-varying formations among self-repelling collective ferromagnetic microrobots at the air-water interface. These microrobots are from 100 mm to 350 mm in diameter and up to about 260 in number, referred as microrobots as is the convention (Diller et al., 2013b). The method is potentially also applicable on 2D solid surfaces (see Appendix B for preliminary results) or with other type of physical constraints, but we focus on the air-water interface in this work as a proof of concept.
In this work, for the static formations, our method is to program the system equilibrium of collective self-repelling magnetic microrobots by patterning desired magnetic potential energy maps. The external magnetic potential energy maps are created by designing external ferromagnet arrays with a model-based design method. A theoretical analysis of functional basis is presented to explain the programmability of the external magnetic potential energy distribution function. With such a method, versatile complex static formations are created among collective magnetic microrobots experimentally. Scaling analysis of the static formations as a function of robot sizes, magnetic properties, shapes, and their population size, has been provided.
For time-varying formations, external magnetic potential energy distribution is programmed both spatially and temporally, allowing bio-inspired collective motions and reconfiguring collective formations among collective magnetic microrobots. Although the individual magnetic moment and position of each ferromagnet in an external magnet array cannot vary on-site, we show that the magnetic potential energy landscape can indeed be varied spatially and temporally by controlling the rigid-body motions of external magnet arrays. Here, we demonstrate the collective mobility of these magnetic microrobots by controlling them to exhibit bio-inspired collective behaviors such as aggregation, directional motion with arbitrary swarm headings, and rotational swarming motion.
Finally, the functions of the produced microrobotic swarm are demonstrated by controlling them to navigate through cluttered environments and complete reconfigurable cooperative manipulation tasks, such as cooperative ''pushing,'' ''caging,'' and ''grasping,'' demonstrating efficient cooperation with fast parallel distributed operation. The ability of reconfiguring formations with different morphologies and stiffness further enhances the functionality of these collective magnetic microrobots for diverse tasks, compared with existing work.
A preliminary version of this work has been presented as a conference paper (Dong and Sitti, 2019), where the concept of programming static collective formations and cooperative functions into microrobotic swarms has been introduced. This work expands on the conference paper with the general theory and design rules for programming external magnetic fields and collective static formations. At the same time, this work also presents the methodology of programming time-varying formations demonstrating reconfiguring collective formations and controlling bioinspired collective motions among collective magnetic microrobots. Moreover, more details are also added including the analysis on floating conditions, modeling of interaction between collective magnetic microrobots and the manipulated objects, experimental procedures, and the preliminary result of collective formations on solid surfaces.
The article is organized as follows. Section 2 introduces the concept of controlling static collective formation, bioinspired collective motions and reconfigurable cooperative functions. Section 3 presents the dynamic model of collective magnetic robots at the air-water interface, the theory and methodology of programming static formations and controlling time-varying formations. Section 4 shows the experimental characterization, setup, and results of collective static formations. Section 5 gives the experimental results of bio-inspired collective motions. Section 6 presents the experimental results of reconfiguring collective formations and reconfigurable cooperative manipulation of objects. The article is concluded in Section 7.

Programmable collective formation
The key concepts of programming collective formations include two parts. First, to avoid magnetic robots clustering when close to each other, their motions must be constrained such that their inter-modular reaction is always repulsive. Second, the equilibrium positions of these microrobots must be programmable and reconfigurable. Here, we propose to create collective self-repelling magnetic microrobots and program their equilibrium positions by designing an external magnetic potential energy distribution ( Figure 1). The air-water interface is employed as a physical constraint in this work as a proof of concept, although the method is potentially also applicable on 2D solid surfaces, which are discussed briefly in the discussion section. The air-water interface allows surface tension to constrain the orientations of magnetic microrobots vertically and balance forces in the z direction, therefore, magnetic microrobots avoid clustering into each other owing to the inter-modular repulsive magnetic forces (Figure 1(b) and (c)).
For multiple magnetic microrobots exposed to an external magnetic field B ext produced by an external magnet, the magnetic potential energy of the ith magnetic microrobot at a position r mi with a magnetic moment M mi is a superposition of the potential energy resulting from both external and inter-modular magnetic fields given by At an equilibrium state, magnetic microrobots transform to a static pattern to minimize the total magnetic potential energy P J i = 1 U i (r mi ) (J is the number of robots) assuming other types of potential energy at the air-water interface remain relatively constant. During the transition phase, e.g., when the microrobots are far from each other, the magnetic potential energy from the inter-modular interaction U mag, inter is negligible compared with that from the external magnetic field U mag, ext . All the microrobots will move towards a position with a local or global minimum of the external magnetic potential energy. When they approach each other, the inter-modular magnetic potential energy increases and modifies the total magnetic potential energy distribution. Individual equilibrium positions of magnetic microrobots depend on both their initial positions and the distribution of the total magnetic potential energy.
At the submillimeter scale, independently controlling individual agents is difficult owing to the severe underactuation of external magnetic fields. Instead of independently addressing each microrobot to a specific position, we propose to concurrently control the overall formation of the magnetic microrobots by programming U mag, ext spatially in the Eulerian space. When the magnetic moment of each microrobot is vertical and invariant of its position, U mag, ext can be equivalently programmed by designing an external magnetic field B ext spatially. As an illustrative example, to create a ring-shaped formation, we produced an external magnetic potential energy landscape on the image plane as shown in Figure 1(d), of which the spatial gradient is the desired external magnetic force (Figure 1(e)). The achieved ring-shaped formation (Extension 1) of microrobots minimized the total magnetic potential energy of the system.

Bio-inspired collective behavior and cooperative functions
If we can program a set of collective formations sequentially, one can imagine a swarm of microrobots constantly change their formations depending on the tasks and work cooperatively similar to small animal collectives such as ants and other animals (Mlot et al., 2011;Sumpter, 2010).
Here we can further spatially and temporally vary an external magnetic potential energy landscape U mag, ext , to produce bio-inspired collective motions, such as aggregation, directional motion with arbitrary swarm headings and rotational swarming behaviors. Moreover, animal collectives can reconfigure their formations depending on the tasks, such as foraging and transportation (Berman et al., 2011), where the reconfiguration of formations allows diverse functions within the same group. Inspired by such behaviors, we demonstrate cooperatively manipulating and transporting single and multiple objects in different shapes via ''pushing,'' ''grasping'', and ''caging'' manipulation strategies using the same collective magnetic microrobots.
Compared with other existing microrobotic systems using self-assembled magnetic matter, our proposed collective microrobotic system has two advantages. First, the collective microrobots can reconfigure their static collective formations, i.e., changing shapes, on demand, to work as a cooperative microrobotic swarm capable of reconfigurable cooperative manipulation. Second, the collective magnetic robots in a static formation show soft-body-like behaviors owing to the inter-modular repulsion allowing intrinsic compliance within the group for morphological adaptability when collectively navigating through a cluttered environment and cooperatively manipulating objects.

Theory and methodology
This section presents the theory of programming static and time-varying formations.

Dynamics of collective magnetic microrobots
As shown in Figure 2, assuming the forces in the vertical direction are always balanced, the motion of each magnetic microrobot on the image plane z = z 0 emerges from horizontal magnetic forces and capillary forces (owing to surface tension) while damped by the fluid drag. The system can be modeled as a mass-spring-damper network, where the 2D transient dynamics of each microrobot at time t is given by where i = 1, . . . , J , m i is the mass of the ith magnetic microrobot, r mi, xy 2 R 2 is the 2D position vector of ith magnetic microrobot (r mi, z = z 0 ), D is the 2D fluid damping coefficient matrix, F mag, ext i, xy =À ½ ∂U mag, ext ∂x , ∂U mag, ext ∂y T is the planar external magnetic force exerting on the ith magnetic microrobot, and F mag, inter ij 2 R 2 and F capillary ij 2 R 2 are the interactive magnetic force and capillary force in the x-y plane applied from the jth to the ith magnetic microrobot, respectively.
At an equilibrium state, the horizontal force balancing condition for the ith magnetic microrobot is thus given by

Static formation as system equilibrium
At a static state, the overall formation depends on the system equilibrium under a spatially distributed external magnetic field. The system equilibrium can be derived from the total potential energy, which is given by where r m 2 R 3J (concatenated position vectors of magnetic microrobots, r mi, z = z 0 for i = 1, . . . , J ) are the states of the system, U s represents the potential energy induced by surface tension, and U o represents all other types of potential energy induced by buoyancy and gravity, which are assumed to remain constant on the image plane.
It should be noted that magnetic robots tend to be as far away from each other to minimize the first term in (4). However, because of the existence of the second term, the system equilibrium is given by minimizing the total potential energy according to where Equation (6) means H e m is positive semi-definite. If we look at the system equilibrium regarding individual positions of these magnetic microrobots, the system equilibrium does not have a unique solution because the individual positions of microrobots are exchangeable, by switching which the total potential energy will not change. Instead of directly designing the system equilibrium, we design the external magnetic potential energy distribution U mag, ext or equivalently the external magnetic field distribution B ext in the global Eulerian space. The desired external magnetic field at a position (x, y, z = z 0 ) can be further approximated with N external magnetic sources where each external magnet has a position r ak 2 R 3 × 1 and a magnetic moment M ak 2 R 3 × 1 .

Programming static formations
To understand the programmability of U mag, ext , we look at the external magnetic potential energy distribution at P points on the image plane produced by K discrete magnetic sources.
At a point r i 2 R 3 × 1 on the image plane, the external magnetic potential energy function is given by where M mi 2 R 1 × 3 is the magnetic moment vector of a magnetic microrobot, the matrix BB ext ik 2 R 3 × 3 is derived using magnetic field models, e.g., the magnetic dipole model assuming the potential energy from higher-order terms of M ak are negligible, which is given by where m 0 is the magnetic permeability of vacuum and I 3 2 R 3 × 3 is an identity matrix. By concatenating the equations in (7) at the P points, we have the external magnetic potential energy as a vector given by where U mag, ext 2 R P × 1 is a vector concatenating the magnetic potential energy values at P sampled positions, M m 2 R P × 3P is a matrix mapping magnetic fields to magnetic potential energy, BB ext 2 R 3P × 3K is a matrix mapping magnetic moments to magnetic fields, M a 2 R 3K × 1 and r a 2 R 3K × 1 are vectors concatenating the magnetic moments M ak and 3D position vectors r ak of K external magnets, respectively, and r 2 R 3P × 1 is a vector concatenating 3D position vectors r i of P sampled points. The external magnetic potential energy distribution that we can create can be clarified by a linear basis analysis of the matrix BM ext = M m Á BB ext . The columns of the matrix BM ext are the linear basis of the achievable external magnetic potential energy vector. The components in the vector M a are the coefficients of these linear bases. Theoretically, the achievable external magnetic potential energy vector is the span of these linear bases in BM ext . With more external magnets, more function bases are encoded in BM ext providing more degrees of freedom. In reality, the achievable external magnetic potential energy distribution depends on the programmability of BM ext and M a together, which need to be designed jointly by controlling (1) the relative distance from external magnets to the image plane, (2) the magnetic moment vectors of external magnets, and (3) the magnetic moments of magnetic microrobots.
On the other hand, the desired external magnetic potential energy distribution on the image plane needs to be encoded according to a desired formation. An external magnetic potential energy distribution on the image plane can be expanded using Taylor expansion, given by where p 2 R 2 is the position vector on the image plane and p 0 = p + Dp is a neighbor point around it. While encoding the desired external magnetic potential energy map, its derivatives need to satisfy ∂ U ∂p = 0 and ∂ 2 U ∂p 2 .0 at points where the formation is desired, such that equilibrium points in terms of external magnetic potential energy can be produced.

Design external magnets
External magnets can be implemented with electromagnets or ferromagnets. Compared with electromagnets (Kummer et al., 2010), ferromagnets can produce a more complex magnetic potential energy distribution within a similar range owing to a higher energy density, which allows more degrees of freedom for programming complex static formations. Existing work on programming ferromagnets have been mainly focusing on producing uniform magnetic fields using optimized Halbach arrays (Choi and Yoo, 2008;Cooley et al., 2018). Here we present a method of programming ferromagnets for a desired non-uniform distribution of magnetic fields and magnetic potential energy. As formulated in (7), both the 3D magnetic moments M ak and 3D positions r ak of modular or bulk-shaped magnets can be design variables for matching a desired external magnetic potential energy distribution. For ease of manufacture and without losing generality, we assume the magnetic moments of all modular or bulk-shaped ferromagnets are in 6z directions.
The desired magnetic potential energy distribution is matched by programming the positions r a 2 R 3 × K (a matrix concatenated by the position vector r ak of each external magnet) and numbers N a 2 R K × 1 (N ak 2 fÀ1, 0, or 1g) of K discrete external modular or bulk-shaped ferromagnets given by where BB ext 2 R 3P × K is a matrix mapping the number of discrete magnets N a to the magnetic field at P points on the image plane. The general programming steps in Figure 3 for ferromagnets with discrete magnetization profiles are given as follows.
(a) The desired magnetic potential energy distribution is encoded from the desired formation pattern, where the desired pattern is coded as low energy and other parts as high energy (zero by default). The value of low energy is given by a negative nominal value U 0 =À 2:5 × 10 À9 J scaled by a positive number a ø a min as we can only select N ak from a limited set of discrete values. (b) The potential positions of modular or bulk-shaped magnets are sampled in a predefined 3D volume, which is under the image plane and has similar sizes with the formation pattern for x, y, and z dimensions. Therefore, r a are fixed after sampling. (c) The magnetic moment at each potential position is iteratively optimized by choosing the number of modular or bulk-shaped magnets N ak (À1, 0, or 1) at a sampled position. A zero magnetic moment means there is no magnet, and a positive or negative magnetic moment indicates an upward or downward magnetic moment. (d) A finite element model (Multiphysics 5.3a and Livelink for MATLAB, COMSOL Inc.) of ferromagnets is used for computing the matrix BB ext based on the iteratively designed magnetization profile in (c). Then, step (c) is revisited to iteratively optimize the magnetization profile manually for matching the desired potential energy distribution. The procedure from step (c) to step (d) typically takes 5-10 iterations for the formations demonstrated in this article. The magnetic dipole model can replace the finite element model for more efficient computation when the distances from external magnets to the image plane are relatively long (.1.5 body length) compared with the size of an individual ferromagnet (Petruska and Abbott, 2013). As future work, a high-dimensional integer-optimization algorithm (Bertsimas and Shioda, 2007) can potentially be incorporated for steps (c) and (d) to automatically optimize the magnetization profile but how to ensure an optimal solution for such a high-dimensional discrete optimization problem needs to be further studied.
(e) The formations of magnetic microrobots are simulated based on the obtained magnetic potential energy distribution in (d) and the predictive dynamic model given by (2). (f) The assembly of external magnet arrays are assisted by laser-cut wood fixture jigs and adhesive tapes (Figure 3(c)).
The design method used here is an iterative optimization approach based on predictive models, as it is difficult to solve a high-dimensional integer optimization problem of matching a desired magnetic potential energy distribution using modular magnets with discrete magnetization profiles. In contrast, with external magnets with continuous magnetization profiles, the optimization problem can be solved using a quadratic programming algorithm (Boyd and Vandenberghe, 2004).

Programming time-varying formations
Collective motion can be further controlled with a spatially and temporally varying external magnetic potential energy distribution U mag, ext (r i , t). Although ferromagnets as external magnetic sources cannot be flexibly changed, i.e., the individual positions and magnetic moments of each magnet in the array cannot be altered on site, we can still change the magnetic potential energy landscape by controlling the rigid-body motions of single or multiple external magnet arrays.
With a single external magnet array, the total external magnetic potential energy at the position r i is given by where R(t) 2 SO(3) and b(t) 2 R 3 represent the rigid-body rotation matrix and translation vector of the external magnet array. Below we report the methods of producing collective aggregation, translation and rotation of magnetic microrobots, as well as other time-varying formations produced by coordinating multiple external magnet arrays.
3.5.1. Collective aggregation. Magnetic microrobots can aggregate into a compact formation, by decreasing the size and increasing the strength of the external magnetic potential energy well. With ferromagnets as external magnetic sources, aggregation can be realized by decreasing the distance b z (t) from an external magnet to the image plane.
3.5.2. Collective translation and rotation. Magnetic microrobots can translate and turn in a desired trajectory while still maintaining their formation shape, by translating the potential energy distribution in the x-y plane and rotating it about its z 0 axis (in the body frame). With ferromagnets as external magnetic sources, collective translation can be realized by moving an external magnet in parallel with the image plane without rotation. Collective rotation can be realized by rotating an external magnet in parallel with the image plane without translation. During collective translation and rotation, magnetic microrobots also experience fluid drag forces. With smaller magnetic microrobots, fluid drag is more dominant compared with both magnetic forces and inertia, which makes the dynamic response slower and the formation more difficult to be maintained when moving external magnets.

Generalized time-varying formations.
With multiple predesigned external magnet arrays, a more complex spatially and temporally varying external magnetic potential energy distribution can be produced by coordinating their rigid body motions, as given by where R n (t) 2 SO(3) and b n (t) 2 R 3 represent the rigidbody rotation matrix and translation vector of the nth external magnet array. M n a 2 R 3 × K n and r n a 2 R 3 × K n represent the magnetic moments and 3D positions of the K n magnets in the nth external magnetic array.

Collective static formations
This section presents the experimental setup, experimental results of creating collective static formations, and the analysis of scaling effect, pairwise interactions and floating conditions.

Microrobots characterization
The magnetic microrobots were made of microparticles of neodymium-iron-boron (NdFeB) alloy (MQP-15-7, Magnequench, Inc.; average diameter 5 mm, density 7.61 g/cm 3 , magnetic relative permeability 1.05) and casting resins (Cast 310, Smooth-On, Inc.; density 1.05 g/cm 3 ) with 20-50 wt% of NdFeB magnetic microparticles. NdFeB microparticles were selected for fabricating magnetic microrobots because they have high remanent magnetization (remanence B r : up to ;896 mT) for exhibiting strong magnetic interaction and large magnetic coercivity (intrinsic coercivity H ci : ;591 kA/m) for keeping such a magnetization. We chose cast resin because of its low density and large water contact angles for better floating of magnetic microrobots at the air-water interface. These two materials were mixed together and casted into a polydimethylsiloxane (PDMS) mold obtained from a two-step molding method (Chung et al., 2015) as shown in Figure  4(a). The magnetic microrobots were molded in thin disk shapes as shown in Figure 4(b) and (c). The top surface of the magnetic material was scraped with a razor carefully, which may cause a convex shape on the top surface. After polymerization, all these magnetic microrobots in disk shapes were magnetized perpendicular to their top surfaces by a 1.0-1.6 T uniform magnetic field generated by a vibrating sample magnetometer (EZ7 VSM, Microsense, LLC). The magnetization of magnetic microrobots were controlled by applying different magnetic fields and using different mass ratios of NdFeB particles as shown in Figure 4(d) and (e).
The magnetic microrobots in disk shapes in all the demonstrations had an average diameter of 350.5 mm 6 4.0 mm (standard deviation for n = 5 measurements) and an average thickness of 100.2 mm 6 20.0 mm (n = 5). The static water contact angles were measured using the sessile droplet method through an automated goniometer routine (DSA100, KRACESS GmbH). For the magnetic composite material with 20 wt% NdFeB, the static water contact angles were measured as u a = 958638, u r = 658668 (n = 3) for the back surface, and u a = 98628, u r = 708648 (n = 3) for the top surface. For the magnetic composite material with 50 wt% NdFeB, the static water contact angles were measured as u a = 96628, u r = 69658 (n = 3) for the back surface, and u a = 958628, u r = 678668 (n = 3) for the top surface. The fabricated magnetic microrobots were not subjected to any treatment before the experiments. The magnetic moments of magnetic microrobots were measured in a vibrating sample magnetometer (EZ7 VSM, Microsense, LLC). The magnetic moment density (magnetization) was then obtained by dividing the corresponding volumes measured by a laser scanner (VK-X200, Keyence Corp.).

Experimental setup
Discrete ferromagnets were assembled into wood jigs to fix their positions and orientations according to the designed profile. The wood jigs were cut by a laser cutting machine (ProtoLaser 3D, LPKF Laser & Electronics AG) with an accuracy of 625 mm in x and y dimensions. Adhesive tapes (Tesa SE) were attached at the bottom of the jigs for fixing the z positions of these magnets. The discrete ferromagnets assembled are commercial NdFeB magnets (N45, Supermagnete.de) in cylindrical (f1 mm × 1 mm) and cubic (1 × 1 × 1 mm 3 ) shapes. For each desired static formation, a template jig fixture is manufactured accordingly. The number of needed template fixtures depends on the number of desired static formations. As the external magnet array can also be translated and rotated to induce reconfiguration of formations, only a limited set of external magnet arrays are required for a given task.
As shown in Figure 5(a), the prototyped system comprised a Petri dish container in two different sizes (inner diameters 137 and 89 mm; height 18 and 14 mm, respectively) filled with deionized water. We used deionized water to avoid dusts, although in general other clean water could also work for our purposes. An artificial arena was added inside the bigger Petri dish. The containers were supported by customized sample holders ( Figure 5(b)). Magnetic microrobots were manually placed at the air-water interface by tweezers with the back surface touching water. This procedure was assisted by an external ferromagnet under the container. The distance from the air-water interface to the bottom of the container was adjusted by adding or retrieving water. In all the experiments of static formations as well as the navigation and manipulation experiments, the external magnet arrays were manipulated manually. In the experiments of rotational swarming, fast flocking, gathering, the external magnet arrays were remotely manipulated using a customized x-y-z-u motorized stage ( Figure 5(c)) adapted from a commercial x-y-z motion stage (AxiDraw V3, Axidraw.com) by adding one rotational degree of freedom about the z axis. In the experiments of automatic reconfiguration of formations, the external magnet arrays were remotely manipulated using a customized motorized z positioner ( Figure 5(d)) assembled from plastic gears and servo motors controlled by Arduino Uno (arduino.cc).

Versatile static formations
The versatility of formations arises from the excessive programmability of external magnetic potential energy maps.
In Figure 6, we demonstrated creating a variety of desired complex static formations among 146 magnetic microrobots at the air-water interface, including letters ''M,'' ''P,'' ''I,'' ''A,'' ''B,'' ''C,'' and a ''wrench''-shaped pattern ( Figure  6(a) and Extension 1). To create the desired external magnetic potential energy maps, we programmed arrays of cubic or cylindrical modular ferromagnets as external magnets (Figure 6(b)). Predictive models of magnetic potential energy maps and collective formations given by (9) and (2) were also used to guide the design of external magnets (Figure 6(c) and (d)). Here, NdFeB ferromagnets were chosen for building the external magnets because their small sizes (down to f1 mm × 1 mm) and high energy density provide more degrees of freedom for matching the desired potential energy map.
The proposed method allows both homogeneous and heterogeneous magnetic microrobots of various sizes and populations to form desired 2D patterns. First, to create a full formation pattern, the population size of magnetic microrobots must be large enough to cover the region of interest (encoded by the desired pattern) in the potential energy map (Figure 7(a)). Next, formations can be created with homogeneous magnetic microrobots of various magnetic moments, sizes, and shapes. Exposed to the same external magnetic field, the formation packing density (number of microrobots per unit area) is higher with magnetic microrobots of smaller magnetic moments ( Figure  7(b)) owing to weaker inter-modular repulsion. Formation pattern is independent of the sizes and shapes of magnetic microrobots but centrosymmetric shapes (Figure 7(c)) and smaller sizes (Figure 7(d)) are preferred for smoother contours of formations. Moreover, heterogeneous magnetic microrobots with different magnetic moments, shapes, and other domain properties can also produce a desired formation (Figure 7(e)). Finally, the formation stiffness is also programmable, allowing advanced manipulation capability as microrobot collectives. The stiffness of a formation depends on the inter-modular distance and determines the compliance of collective magnetic microrobots when moving together and interacting with other objects. Compared with existing methods of creating self-assembled magnetic structures, our method allows collective magnetic microrobots to have programmable and considerably more versatile formations simply by designing different external magnets.

Pairwise interaction
The pairwise equilibrium distances for magnetic microrobots of different diameters were studied to understand the un-clustering conditions. Assuming the body axes and magnetic moments of microrobots are in vertical directions, pairwise capillary force exerted on a pair of magnetic microrobots in the lateral directions are modeled as follows where g = 72 mNÁm À1 is the surface tension of water at a room temperature, C is the menisci slope angle, R is the radius of the microrobot, and r ij = r mi À r mj is the intermodular distance vector. The pairwise inter-modular magnetic force is given by where m 0 is the magnetic permeability of vacuum, and M and h are the thickness and magnetization of the microrobot, respectively. The external magnetic force is given by Assuming C is constant along the contact line for ease of scaling analysis, we have sin (C) = F z =(2pgR) based on a monopole model (Kralchevsky and Nagayama, 2000), where F z ' À F mag, ext i, z is the vertical component of surface tension. The inter-modular equilibrium distances must satisfy jr ij j.2R to avoid touching or clustering of microrobots. Without losing generality, we analyzed the pairwise interaction between a pair of magnetic microrobots subject to an external magnetic field produced by nine modular external magnets (f1 mm × 1 mm, N45, Supermagnete.de) under the image plane (distance ;8 mm). As shown in Figure 8, both experimental results and simulation using the predictive model in (2) suggested that smaller magnetic microrobots would have a larger equilibrium distance ratio (the inter-modular distance over the diameter of a microrobot). Analysis also suggests the capillary forces between magnetic microrobots are less than 10% of magnetic repelling force at the equilibrium states.

Floating condition
The maximal balancing force for the microrobot from surface tension is derived by adapting a model from Extrand and Moon (2008) assuming the body axis aligns vertically. We also assume the distribution of contact angles along the boundary is uniform (monopole) for ease of scaling analysis, although high-order terms can be added to compensate for the multipole effects in surface tension. The maximal lift force from the air-water interface is given by where r l is the density of water, R and h 0 are the radius and thickness of the disk-shaped magnetic microrobot, u a is the advancing water contact angle, and L max is the maximal distance from the top surface of the microrobot to the image plane z = 0 (Figure 9(a)). Based on the model reported by Extrand and Moon (2008), we have Hence, we have the condition of floating for a magnetic microrobot, given by where the variables r m and M represent the density and magnetization of the magnetic microrobot, respectively, jrBj max is the maximally allowable magnetic field spatial gradient. Therefore, magnetic microrobots with smaller R and h 0 are expected to float better when exposed to the same external magnetic field (Figure 9(b)). The maximal balance torque from surface tension is given by which is further is given by t surface max = pR 2 g( cos u a À cos u r ); assuming the contact angles along the contact line can be simplified as where u A and u R are the static advancing and receding water contact angles, and s is the perimeter length along the circular contact line starting from s 0 . To hold the magnetic microrobot at the air-water interface, the magnetic torque given below in (24) should be less than t surface max , where f B and f are the deviating angles from the positive z direction of the net magnetic field and magnetic moment of the microrobot, respectively. The condition suggests magnetic microrobots should have thinner thickness (Figure 9(c)) to better hold the magnetic microrobot vertically.

Bio-inspired collective motion
This section presents versatile collective motions of magnetic microrobots and their soft-body-like collective navigation through cluttered environments, demonstrating the collective mobility of magnetic microrobots enabled by the proposed method.

Collective behavior
Inspired by the motions from small-scale animal collectives (Sumpter, 2010), in Figure 10 we demonstrated controlling the external magnetic energy distribution to create a microrobotic swarm capable of complex collective behavior. First, like the aggregation behavior of ants (Mlot et al., 2011), magnetic microrobots aggregated into a compact formation as shown in Figure 10(a), by decreasing the size and increasing the strength of the external magnetic potential energy well when moving the external magnet in the + z direction. Second, similar to the migration behavior of bird flocks (May, 1979), magnetic microrobots were able to translate and turn in a desired trajectory while still maintaining their formation shape (Figure 10(b) and Extension 2), by translating the external magnet in the x-y plane and rotating it about its z 0 axis (in the body frame). Fast directional collective motion of these microrobots in a designed formation was realized by controlling the motion of an external magnet in a planned trajectory (Figure 10(c) and Extension 2). During the collective movement, magnetic microrobots also experienced fluid drag forces by which their formations were also distorted. Such damping effect limits the maximal navigation speed of collective magnetic microrobots while they still maintain the desired formation. The maximal speed generally depends on the strength of external magnetic potential energy   9. Critical floating condition of a magnetic microrobot at the air-water interface as a function of its size. (a) Schematics of a disk-shaped magnetic microrobot floating at the air-water interface. (b) Simulated maximally allowable applied spatial gradient of external magnetic fields for a single disk-shaped magnetic microrobot floating at the air-water interface. (c) Simulated maximally allowable applied magnetic field for a single disk-shape microrobot floating at the air-water interface. Parameters other than the dimensions of magnetic microrobots are same to that of the microrobots in Figure 6(e) in the main text. distribution and has been experimentally found to be around 42 mmÁs À1 for a disk-shaped formation. Automated gathering of free microrobots out of the formation was also realized by controlling the external magnetic potential energy well to follow a planned path (Figure 10(d) and Extension 2).
In addition, like the milling behavior of fish schools (Herbert-Read et al. (2011)), these magnetic microrobots also exhibited collective rotational behavior (Figure 10(e) and Extension 2) by creating a rotational external magnetic force distribution when spinning a designed external magnet about its z 0 axis. The physical modular interaction in the collective system allows emulating animal-like behaviors, which may be helpful for researchers to use it as a platform to study swarm systems such as collective animal behaviors (Sumpter (2010)) at small scales.

Soft-body-like adaptive navigation
Small animals can take advantage of their formation as a group for different tasks such as foraging, migrating, and escaping from danger (Sumpter, 2010;Tan and Zheng, 2013). For example, an army of fire ants (Mlot et al., 2011) can survive from a water flood by collectively floating at the air-water interface in a disk-shaped formation. In addition, soft organisms (Brackenbury, 1997) can navigate through cluttered environments exploiting their morphological adaptation and compliance when physically interacting with the environment.
Inspired by these behaviors, we created a microrobotic swarm capable of transforming into a formation and collectively navigating through confined spaces taking advantage of their actively designed formation and intrinsic compliance. The collective magnetic microrobots were capable of navigating through complex cluttered environments by adapting the formation morphology during the interaction with the environmental boundaries (Figure11(a) and Extension 3). First, the designed formation, such as an ellipse-shaped formation, provided a level of adaptation in morphology to the environmental boundaries by minimizing the contact area with channel walls. Moreover, the compliance from inter-modular repulsion further enhanced the adaptability and flexibility of negotiating through narrow channels and holes and over posts (Figure 11(b)-(d)). For  Figure 17(b)). All magnetic microrobots have an average diameter of 350 mm and thickness of 100 mm (20 wt% NdFeB in (a) and 50 wt% NdFeB in (b)-(e); magnetized in a 1.0 T magnetic field in (a) and 1.2 T in (b)-(e)). In the color bars: U min = À 1:6 × 10 À9 J and U max = 3:54 × 10 À10 J. Scale bars: 5 mm. example, by pushing the boundaries and each other, magnetic microrobots could pass through narrow holes as shown in Figure 11(d). Finally, the redundancy in the collective microrobots also provides robustness for fault tolerance, similar to their counterparts in nature. For example, during the navigation, these magnetic microrobots still maintained their formation even when they lost some agents (Extension 3). Therefore, compared with a large monolithic robot, the designed microrobotic swarm have more flexibility of accessing confined space by adapting to the morphological constraints in the environments, and are more robust against agent failure.

Reconfigurable cooperative manipulation
This section presents the experimental results of reconfigurable cooperative manipulations, demonstrating example applications of the proposed design and control methods for collective magnetic microrobots. Figure 12 shows several representative examples of reconfiguring formations of collective magnetic microrobots. First, rotating an external magnet about its x 0 or y 0 axis (in the body frame) can induce formation alterations among collective magnetic microrobots, such as transforming from a ring-shaped formation to a ''C''-shaped formation ( Figure  12(a) and Extension 4). Next, coordinated motions of multiple external magnet arrays can yield more complex timevarying formations. For example, a size-programmable ring-shaped formation was created by coordinating the z positions of two external magnet arrays (Figure 12(b)). The size reconfigurable ring-shaped formation is useful for adjusting the range of caging-type manipulation. Finally, by switching among multiple external magnet arrays, reconfiguration of completely different formations were realized among the same collective magnetic microrobots ( Figure  12(c) and Extension 4). The ability to reconfigure formations can enable metamorphosis (Miyashita et al., 2017) in collective magnetic microrobots for versatile cooperative functionalities.

Reconfigurable cooperative manipulation
Compared with single robots, swarms of microrobots working together can exert higher force for manipulating large objects. The collective formation also allows more efficient manipulation of objects, such as transporting single large objects or multiple objects simultaneously. For example, by using caging-type manipulation (Rodriguez et al., 2012), microrobots can transport many objects at the same time. Although cooperative manipulation using collective magnetic matter has been shown before in micro-scale systems in experiments (Snezhko and Aranson, 2011;Torres and Popa, 2015;Wang et al., 2018a,b;Xie et al., 2019) and simulations (Becker et al., 2013;Shahrokhi et al., 2018), they cannot complete versatile manipulation tasks owing to the limited reconfigurability. In contrast, animal collectives can reconfigure their formations depending on the tasks, such as foraging and transportation (Berman et al., 2011), where the reconfiguration of formations allows diverse functions within the same group. Adaptation to environmental constraints from the inter-modular compliance of the microrobotic swarm while navigating around posts and through channels and holes and (ii) the modeled total potential energy maps from both external magnetic fields and environmental constraints (environmental boundary energy was modeled with artificial potential field from Khatib (1986)). Colored segments connecting magnetic microrobots in (i) in (b)-(d) indicate normalized inter-modular stiffness (K ij }jr ij j À5 ; six-nearestneighbor connections are selected within a 6R maximal Euclidian distance). In the color bars in (ii) in (b)-(d): U min = À 1:06 × 10 À9 J and U max = 1:06 × 10 À9 J. All magnetic microrobots have an average diameter of 350 mm and thickness of 100 mm (50 wt% NdFeB, magnetized in a 1.2 T magnetic field). Scale bars: 5 mm.
Inspired by such behaviors, in Figure 13, we demonstrated cooperatively manipulating and transporting single and multiple objects in different shapes via ''pushing,'' ''grasping,'' and ''caging'' (Extension 5) manipulation strategies using the same collective magnetic microrobots. The applied 2D manipulation forces and torques from collective magnetic microrobots depend on the formation and the object being manipulated, which are modeled by where r c 2 R 2 is the center of mass (COM) of the manipulated object, S denotes a set including the indexes of magnetic microrobots that have contact with the manipulated object, and F interact i 2 R 2 is the force exerted from the ith microrobot in contact to the object. At static states, the manipulation forces on the object are given by When microrobots and objects are moving at a constant speed, the manipulation forces on the object are given by By reconfiguring specific formations on demand, the collective magnetic microrobots can exert distributed forces and torques on the manipulated objects, thus allowing them to complete different tasks for an optimal performance. As shown in Figure 13(a), with an ellipse-shaped formation, the magnetic microrobots could uniformly and cooperatively push a long rectangular object and transport it to a desired location. The inter-modular stiffness of the  Figure 17(c)) with u 1 = 0 8 , u 2 = À 10 8 , and u 3 = 10 8 . (ii) Schematics of the rotational motion of external magnets. (b) Size-programmable formation for reconfigurable trapping of objects. (i) Top: video (Extension 4) snapshots of size-programmable ring-shaped formation of 144 magnetic microrobots by coordinating two external magnets in the z direction. Bottom: the modeled external magnetic potential energy maps (using a finite element method for the magnet in Figure 17(c)) are displayed in parallel. The differences of distances from the top surfaces to the image plane: Dd 1 = 0 mm, Dd 2 = 1:5 mm, and Dd 3 = 2:5 mm (d 2 = 8 mm; d 1 is varying). (ii) Schematics of the relative motion of two external magnets. (c) Switching between different formations among magnetic microrobots. (i) Video (Extension 4) snapshots show reconfiguring formations of 144 magnetic microrobots by coordinating three external magnets. (ii) Relative distance from each external magnet to the image plane. All magnetic microrobots have an average diameter of 350 mm and thickness of 100 mm (50 wt% NdFeB; magnetized in a 1.2 T magnetic field). In the color bars: U min = À 1:6 × 10 À9 J and U max = 3:54 × 10 À10 J. Scale bars: 5 mm.
formation was kept relatively high allowing the collective microrobots maintaining the formation while transporting objects (Figure 13(d)). After that, the same collective magnetic microrobots further transformed into a circular and more compliant formation, and adjusted the opening and closing states of the formation to trap and transport multiple objects gently and simultaneously (Figure 13(b) and (e)). Sequentially transforming into circular and diskshaped formations was then employed for demonstrating tight grasping of multiple objects with force closures (Figure 13(c)). In this task, caging was first used to surround the targeted objects so that magnetic microrobots could disperse uniformly around the target objects. Then, a disk-shaped formation was engaged to aggregate magnetic microrobots so that they could grasp objects in diverse shapes compactly and reliably as shown in Figure 13(f). The grasping forces applied on the objects at static states are difficult to be experimentally measured but can be estimated using (27), as shown in Figure 14. The maximal applied grasping force applied by a microrobot in contact was estimated to be about 0.2 mN.
The cooperative manipulation by collective magnetic microrobots in different formations allow higher forces and torques and more efficiency compared with single robots. Moreover, the ability of reconfiguring formations with different morphologies and stiffness further enhanced the functionality of these collective magnetic microrobots for diverse tasks.

Conclusion and discussion
In summary, this article has presented a generic methodology to program 2D collective static and time-varying formations of collective magnetic microrobots, as well as control their cooperative behaviors. This work demonstrated such a concept on submillimeter ferromagnetic microrobots at the air-water interface by programming external ferromagnet arrays. Despite the simplicity of the demonstrated system, remarkable programmability and reconfigurability have been shown using the proposed methodology. In our demonstrations, although the individual magnetic moment and position of each ferromagnet in an external magnet array cannot vary on-site, we showed that the magnetic potential energy landscape can indeed be varied spatially and temporally by controlling the rigidbody motions of external magnet arrays. Moreover, for formation on 2D solid surfaces, a large uniform magnetic field can be applied to align magnetic microrobots in addition to that produced by the designed external magnets, as the formation only depends on the spatial gradient of external magnetic field (see Appendix B for preliminary results).
Potential limitation of the proposed method is the relatively short actuation range (\10 mm). The proposed system can be potentially used for bio-medical applications where a short actuation range is allowed. To increase the actuation range, stronger external magnets, for example, using superconducting materials (Cao et al., 2018) with efficient cooling in the extreme case, must be used for applications requiring a larger actuation range (.10 mm).
In addition to robotic applications, the proposed method and system can be potentially used for manufacturing, such as efficient 2D contactless material assembly (Tasoglu et al., 2014). Furthermore, the physical modular interaction in the collective system allows emulating animal-like behaviors, which may be helpful for researchers to use it as a platform to study swarm systems and animal behaviors (Sumpter, 2010) at small scales.
As future research directions, collective formations at the air-water interface and on 2D solid surfaces for specific applications will be explored. In addition, 3D formation is extremely challenging but may also open up new opportunities for wider applications. There are two critical challenges preventing creating 3D formations among collective magnetic microrobots. First, it is impossible to control ferromagnetic robots in 3D in an open-loop stable manner, according to the Earnshaw's theorem. Instead, feedback control of collective magnetic microrobots in 3D are required to stabilize their 3D positions. Second, it is challenging to create self-repelling effects among magnetic microrobots in three dimensions, as the omnidirectional magnetic repulsion in 2D cases is no longer applicable in 3D cases. One possible direction is to explore other types of physical constraints to enable omnirepulsion of individual magnetic microrobots in 3D, such as using electrostatic double-layer forces, while controlling time-varying 3D potential energy landscapes with feedback control.
Programmable formation of collective magnetic microrobots is potentially achievable on 2D solid surfaces. Compared with the air-water interface, solid substrates can still balance the magnetic pulling force in the z direction but cannot provide the torques to align magnetic microrobots vertically. Inter-modular magnetic forces are not necessarily repulsive, which may cause clustering of magnetic microrobots. To address such an issue, we propose to add a large DC external magnetic field on top of that from the designed external magnet. In this case, the orientations of magnetic moments can be aligned in the z direction, so that magnetic microrobots still repel each other as illustrated in Figure 15. The external magnetic force distribution remains the same because it only depends on the non-DC portion of external magnetic fields.
In Figure 16(a) and (b) we compare the simulated external magnetic force distributions when applying and not applying a large DC magnetic field, respectively. The simulated magnetic force distributions on the 2D substrate assume that the external DC magnetic field is dominant over inter-modular magnetic fields so that the magnetic   moments of microrobots align vertically with a small deviation. The simulation result shows that with an added high DC magnetic field along the negative z axis, the force distribution is more desirable because it yields a desired ringshaped pattern. For a proof of concept, we experimentally created a ringshaped formation (Figure 16(c)) on a 2D substrate (Teflon) using a designed external magnet. An extra external DC magnetic field about À10 mT, across an area of 10 × 10 mm 2 with a 95% uniformity, was produced by a large NdFeB magnet (f60 mm × 5 mm, N45, Supermagnete.de). The DC magnetic field was used for aligning magnetic microrobots in the z direction. The collective motion of magnetic microrobots in the experiment was limited by the friction forces induced by both gravity and the magnetic pulling force in the z direction. To reduce surface friction, one possible solution is using two external magnets on both sides of the image plane to compensate for the pulling forces in the z direction. Other potential methods include coating the surfaces of magnetic microrobots or the substrate with lubricant materials (Timonen et al., 2013) to reduce the friction coefficient or using magnetic microrobots in sphere shapes to reduce contact areas.

Appendix C. Assembled external magnets
The assembled external magnets used in the experiments are shown in Figure 17.  Figure 6. (b) Assembled external magnet to produce rotational external magnetic force fields when rotating about its z axis in Figure 10(e). (c) Assembled external magnet for reconfiguring the radius of ring-shaped formations in Figure 12(b). The handle is used for tuning the z position of the inner magnet (Magnet 1). All dimensions of the external magnets are in millimeters. Scale bars: 5 mm.