A Communication Framework in Multiagent System for Islanded Microgrid

Microgrids are integrated energy systems consisting of interconnected loads and distributed energy sources which as a system can operate in parallel with the grid or in an island mode. By fault occurrence in the connected power system or geographical isolation, microgrids operate in the islanded mode. In the islanded mode, microgrids should be operated to meet a power balance between supply and demand without power trade. In recent years, multiagent systems have been proposed to provide intelligent energy control and management systems in microgrids. In this paper, we design a communication framework to control and operate distributed sources and loads in the islanded microgrids. The framework reliably delivers microgrid control frame between agents by employing wireless mesh network as an advanced topology of the wireless sensor network. From results of experiments, we show that our framework outperforms other conventional one, with respect to the rate of success on the transmission of frames among agents.


Introduction
A microgrid is an energy community having clean energy sources such as solar power, wind power, and fuel cells and energy storage devices such as batteries.The energy sources and energy storage devices are distributed in the community, and they are called distributed generation systems (DGs) and distributed energy storage devices (DSs), respectively.Recently, attention on the microgrid has been growing as an eco-friendly power system reducing climate change.Since Professor Lasseter proposed the concept of the microgrid in 2001 [1], many technologies such as power control [2][3][4][5], protection schemes [6,7], simulators, and field tests [8][9][10] for microgrids have been studied.In addition, multiagent system applications have been studied for efficient and economic control and operation of microgrids [11][12][13][14][15][16][17][18][19][20].
The microgrid can be operated by two operation modes: the grid-connected mode and the islanded mode.In the grid-connected mode, a microgrid is connected to a power system, especially a distributed system.On the other hand, the islanded mode means an isolated operation mode from any power system for the case of fault occurrence in the connected power system or geographical isolation such as a small island.In the islanded mode, microgrids should be operated to meet a power balance between supply and demand without power trade.Whenever a power imbalance occurs, the output of DGs is decreased and load shedding is used to solve the power imbalance.
Since DGs and DSs are distributed geographically, microgrids can be operated and controlled using communication links such as the Internet, the power line communication (PLC), and fiber-optic lines [19].In particular, in the case of geographically islanded microgrids located at an island, the PLC and the wireless sensor network (WSN) can be considered as economical communication links.The PLC is used for the pilot microgrid of Kythnos Island (Greece) [12].However, the propagation problem and the limited data rates of the PLC are well-known problems.Besides, there are many ways in which error has been introduced into the communication signals.Interference, cross chatter, some active devices, and some passive devices introduce noise or attenuation into the signal.When error becomes significant, the devices controlled by the unreliable signal may fail, become inoperative, or operate in an undesirable fashion.For this reason, the WSN was considered basically for a communication link in geographically islanded microgrids as explained in our previous work [15].
In this paper, we propose a communication infrastructure based on the WSN for geographically islanded microgrid operated and controlled by a multiagent system.As an advanced topology of the WSN, we employ wireless mesh network (WMN) that needs only a few access points for wireless connections among agents and also reduces the infrastructural costs.To improve the performance in terms of the success on the transmission of frames among agents, we customize the routing protocol for adjusting routes according to the link quality.Also, we verify that our protocol improves the success on the transmission irrespective of dynamics of link quality.
The remainder of this paper is structured as follows.Section 2 describes geographical islanded microgrid operation based on the multiagent system as backgrounds.Section 3 explains our communication infrastructure and discusses the proposed routing protocol.Following this, we verify the designed system by NS-2 simulations in Section 4. Finally, Section 5 summarizes our study results.

Islanded Microgrid Operation Based on Multiagent System
2.1.Islanded Microgrid Operation and Control.The microgrid should maintain a constant frequency such as 50 Hz or 60 Hz.In practice, some deviation such as ±0.2 Hz is allowed.The frequency affects a power balance between supply and demand.Figure 1 shows an operation scheme for power balance in the islanded microgrid [17].The information of power supply, power demand, and status of storages is collected and a condition of power balance is checked.If the power supply is greater than the power demand, DSs are selected for charging.Otherwise, DSs decided to discharge and the load shedding is used.Figure 2 illustrates the operation procedure of the microgrid simply where an operation plan prepared in the previous interval is implemented in the next interval.The operation is related to planning action for operational intervals and is composed of two steps: planning and implementation [15][16][17][18][19].In general, the interval period is determined by microgrid operation rules, for example, a few minutes or a few dozen minutes.The planning action as a control reference is established in each interval.And then the control action is followed.Table 1 shows the features of the operation and control.

Multiagent-Based Islanded
Microgrid.An agent is considered as an intelligent agent which senses the changes of environment and acts by its design purpose.A multiagent system is composed of multiple agents.In our previous works [15][16][17][18][19], a multiagent system for microgrid was defined as follows: where Ag MGOCC is the Microgrid Operation and Control Center (MGOCC) agent, AG DG is a set of DG agents (Ag DG ), AG DG is a set of storage device agents (Ag DS ), and AG L is a set of load agents (Ag L ).The MOGCC agent manages entire operation and control in the microgrid.Each agent operates and controls its DG, DS, and load.The agents communicate with the agent communication language (ACL) and share their knowledge for cooperation.An example of the message for communication among agents is as follows: (<performative>: from <agent name>: to <agent name>: content <OAV type data>), where OAV (objective attribute values) type data is composed of an object, an attribute of the object, and the value of the attribute.Table 2 is the communication protocols and Tables 3 and 4 are the designed performatives for the protocols [17].
Here, P1 is used for interactions between the MGOCC agents and the DG agents and P2 is used for between the MGOCC agents and the Load/DS agents.

WSN-Based Communication Infrastructure
3.1.Design of WMN Structure.As an extension of the WSN, the WMN has been recently developed to provide high-quality services and applications over wireless personal area networks, wireless local area networks, and wireless metropolitan area networks [21].Its applications and services include wireless home Internet access, community and neighborhood networking, public safety and security surveillance systems, intelligent transportation systems, and emergency and disaster networking.The WMN has a hybrid network infrastructure with a backbone and an access network.It is operated in both ad hoc and infrastructure modes with self-configuration and self-organization capabilities.
The WMN is the ideal solution to provide both indoor and outdoor broadband wireless connectivity in urban, suburban, and rural environments without the need for extremely costly wired network infrastructure [22].The WMN has been envisioned as the economically viable networking paradigm to build up broadband and largescale wireless commodity networks.Installing the necessary cabling infrastructure not only slows down implementation but also significantly increases installation cost.Therefore, the wired architecture is costly, unscalable, and slow to deploy.On the other hand, building a mesh wireless backbone enormously reduces the infrastructural cost because the mesh network needs only a few access points for connection.This reduction of network installation cost ensures rapid deployment of a metropolitan broadband network even in rural or scarcely populated urban areas.Thus, we employ the WMN to design communication infrastructure for the multiagent system.The WMN is a group of mesh clients and routers interconnected via wireless links.Mesh clients (MCs) can be various devices with wireless network interface cards such as PCs and laptops.In this paper, a MC represents an agent.Hereafter, MC and agent are exchangeable for convenience.Agents have limited resources and capability in terms of processing ability, radio coverage range, and so on.Mesh routers (MRs) are usually powerful in terms of computation and communication capabilities and have continuous power supply.They normally stay static and act as access points to supply network connections for the agents.Due to limited radio coverage range and dynamic wireless channel capacity, message from an agent usually is transmitted through a multihop path to its destination.Ad hoc mode interconnections of the MRs construct the wireless mesh backbone network.

Planning Implementation
When a new or existing router joins or leaves the backbone, the network self-organizes and self-configures accordingly.In a wireless mesh access network, there are usually one static agent and a number of sensors.
Our WMN structure is illustrated in Figure 3. On each MR, one wireless channels is assigned for access network communication, while the other channel is assigned for the backbone network interconnection.Adjacent access networks should be set to operate on separated channels in order to avoid interference with each other.In the backbone, when directed traffic travels towards the destination, the backbone provides redundant paths between each pair of MRs significantly increasing communications reliability, eliminating single points of failure and potential bottleneck links within International Journal of Distributed Sensor Networks   [24], yet allows vendors to operate using alternate protocols.HWMP is inspired by a combination of an on-demand AODV [25] and a proactive tree-based routing.The proactive mode requires one MR to be configured as one root MR and we configure the MR connected with MGOCC as the root MR.The root MR constantly propagates routing messages that either establish and maintain routes to all MRs in the mesh or enable MRs to initiate a path to it.In Figure 4, MR K uses the root MR C to establish an initial path (dotted arrow) to MR J. Once In on-demand AODV, in order to construct a routing path from source MR to destination MR, the source broadcasts a path request message (PREQ) to the mesh.Upon receiving the PREQ, the destination responds to the source with a path response message (PREP) through unicast transmission.The routing path from the source to the destination is created after the source receives the PREP.Once established, this path is used before it times out.The AODV works on the link layer and adopts a radio-aware metric [26,27] to calculate the link cost.
The route determined by using the link cost improves the probability of success on the transmission of frames, compared to the route determined by using the hop distance.However, once the route is established, the route is not updated except route failure or route timeout.This route setup procedure decreases the probability of success on the transmission.Since, in our environments, microgrid control messages of agents are exchanged between the MRs, the degradation of the transmission success is fatal.Thus, the dynamic adjusting scheme of routes according to the link quality is needed.However, this scheme suffers from the route flap problem that the MRs constantly change their routes toward the destinations.When the MR detects the degradation of current link quality for the routes, the MR chooses new link having the smallest link cost as its new route.When multiple MRs simultaneously switch to the new link, traffic input rate to the link increases abruptly and congests the link, while the previous link becomes unloaded.Then, the MRs detect the status change and switch back to their previous link.The switch results in unloading the new link, and the process repeats.This route-flap problem causes frequent packet reordering and increases packet loss.We extend HWMP to adjust the route between MRs according to the link quality with no route-flaps.According to the relative The MR detects the degradation of the link quality of its current link i when the link cost c i is larger than a certain threshold (c th ).Then, the MR selects a new link as follows: the MR calculates Γ(c i ) of its links whose c i s are smaller than c th .Γ(c i ) is the min-max normalization function of c i .The normalized link cost represents the relative link quality levels among them.As Γ(c i ) of a link is smaller, the link quality of the link is better, compared to the other links having larger values.Given Γ(c i ), the MR sorts Γ(c i ) in an ascending order and we denote Then, with probability p, the MP selects link i as the new link as long as p satisfies the condition as in the following equation: (2)

Performance Evaluations and Discussions
To quantitatively evaluate the performance of the proposed routing protocol, we use NS-2 network simulator [28].Our WMN delivers the microgrid-related command between agents in a reliable manner.We evaluate the rate of success on the transmission of frames as a performance metric.In a 1500 m by 1500 m grid, we deploy 20 MCs that is, agents and 36 MRs.The MCs are randomly placed and MRs are equally spaced.The root MR sends a control message every 0.5 seconds.MCs exchange 1024-byte CBR packets with randomly selected other MCs through MRs every second.Table 5 is the list of the parameters as well as the selected values of the physical (PHY), MAC, and HWMP layer applied for the simulation environment.The values for the PHY layer are those for the Extended Rate PHY (IEEE 802.11g).The RTS/CTS mechanism of the MAC layer is disabled by the default settings of the IEEE 802.11 standard.The total simulation time is 3600 seconds.
Figures 5-7 show the probability of success on the frame transmission between MCs.The probability is mostly affected by the frame error rate (i.e., the probability of success on the transmission of frames), data rate, and threshold c th [26,27].The numbers in each legend of Figures 5 and 6 present how to set the value of the c th .For example, "Our (c th : 20%)" means our protocol with the c th being 20% of the difference between min and max of the estimated link costs.Since the frame error rate fluctuates in time according to contention and interference, we vary the frame error rate with standard deviation of Gaussian distribution N(0, σ 1 ) in Figure 5.The results indicate that our protocol delivers more frames than HWMP by about 5% on average.When the σ 1 is small, the performances of the both protocols are similar.However, when the σ 1 is large, our protocol delivers more frames up to 11% than HWMP.
Figure 6 shows the performance with varying data rate r when we set σ 1 to 0.3.Since IEEE 802.11 g standard supports data rates of 6, 9, 12, 18, 24, 36, 48, and 54 Mb/s, we vary the data rate according to them.As the data rate increases, the performance usually increases.In our scenario, since the number of delivered frames between MCs is not large, for example, one frame for each second, the performance improvement is not large.Figure 7 indicates the performance with varying the c th .As seen in the figure, the performance of HWMP is not affected by the c th .In our protocol, as the c th is changed between 10% and 90%, the performance is changed up to 6%.When the c th is small, our protocol is too highly sensitive to the dynamics of the link cost and the overhead of changing the route increases.When the c th is large, the protocol becomes dull and the benefit of adjusting routes according to the link quality decreases.From the results, we conclude that the performance is good when the c th is 30% or 40%.

Conclusions
In this paper, we designed a communication infrastructure for multiagent-based islanded microgrid.There are several contributions in our design: we designed the infrastructure to deliver grid control frame between agents by employing WMN.In order to deliver the frame reliably, we extended the traditional HWMP to adjust the route between MRs serving agents according to the link quality.For the development, we selected a new link in a statistical manner to avoid the routeflap problem.In addition, we showed the feasibility of our protocol from the comparison of our protocol with HWMP.
From the comparison, we can conclude that the performance of our protocol is better than that of HWMP with respect to the rate of success on the transmission of the frame.
In our protocol, one of the factors affecting the performance is the threshold c th .To optimize the performance, we need any threshold decision scheme in time-varying environments.We will consider developing the scheme as a future research direction.In addition, when the islanded microgrid needs to be connected to the external network, for example, the Internet, our infrastructure uses one or more MRs to connect the external network.As another research direction, we will develop the load-balancing scheme between the MRs to avoid the traffic concentration.

Figure 1 :
Figure 1: An operation scheme for the islanded operation.

Figure 5 :Figure 6 :
Figure 5: Experimental results obtained by varying frame error rate with standard deviation of Gaussian distribution N(0, σ 1 ).

Figure 7 :
Figure 7: Experimental results obtained by varying c th threshold.

Table 1 :
The features of operation and control of microgrids.
the mesh.Network resilience and robustness against potential problems (e.g., node failures, and path failures due to temporary obstacles or external radio interference) are also ensured by the existence of multiple possible routes to the destination.