A Novel Quad Harmony Search Algorithm for Grid-Based Path Finding

A novel approach to the problem of grid-based path finding has been introduced. The method is a block-based search algorithm, founded on the bases of two algorithms, namely the quad-tree algorithm, which offered a great opportunity for decreasing the time needed to compute the solution, and the harmony search (HS) algorithm, a meta-heuristic algorithm used to obtain the optimal solution. This quad HS algorithm uses the quad-tree decomposition of free space in the grid to mark the free areas and treat them as a single node, which greatly improves the execution. The results of the quad HS algorithm have been compared to other meta-heuristic algorithms, i.e., ant colony, genetic algorithm, particle swarm optimization and simulated annealing, and it was proved to obtain the best results in terms of time and giving the optimal path.


Introduction
Throughout the past few decades, the global interest of researchers has been in the grid-based graph representations and path-planning problems, since they have been shown to be of great significance for many practical applications and research studies. Several of them were utilized in response to problems in the areas of computer vision, medical informatics, CAD/CAM-design, gaming and robotics. All these problems and applications become more challenging if they need to be solved in real-time. The algorithms employed for grid-based path finding can fall into two categories: deterministic and meta-heuristic. Because deterministic algorithms have many computational drawbacks, such as gradient information, great dependency on the initial value and often huge memory requirements, meta-heuristic algorithms have been considered as a feasible alternative. Meta-heuristic algorithms can be formulated as an iterative process, utilizing rules and randomness to improve the candidate solutions in order to efficiently find a near-optimal solution. Some of the meta-heuristic algorithms employed to solve the path-finding problems are: tabu search (TS), artificial neural network (ANN), genetic algorithm (GA), particle swarm optimization (PSO), ant colony optimization (ACO) and simulated annealing (SA). Although it has not always been possible to find an optimal path in grid-based environments using such algorithms, these researches have served as a great example and a catalyst for finding alternatives to existing algorithms, thus providing either better time or memory complexities -key points of exploration to be seriously considered. Such research proved that these algorithms obtain poor results and a lack optimality when they have either very high or very low occupancy with obstacles in the working environment [1].
A new meta-heuristic algorithm called harmony search (HS), intended to overcome optimization and search problems, has been proposed by Geem et al. [2]. This algorithm utilizes the musician's experience in jazz improvisation to obtain an optimal solution through an iterative process. When HS has been evaluated against other meta-heuristic algorithms, a conclusion has been drawn that the HS algorithm performs better in terms of diversification and intensification concepts [3].
This means that HS has a randomization phase that is more efficient in exploring the search space, whilst not moving too far from possible good solutions and avoiding convergence with local optimal solutions. It is also noted that the greater the probability of the harmony memory consideration rate (HMCR) parameter, the faster the convergence to the best solution will be. The implementation of the HS is a lot easier and simpler than that of the other meta-heuristic algorithms, and the sensitivity to the chosen solutions will not be affected by fine tuning the parameters of the algorithm to obtain good solutions. It is also possible to parallelize the algorithm, which can provide better implementation with better efficiency [3]. This algorithm has been widely used in many engineering problems, such as internet routing [4], robotics [5] and web page clustering [6], but also in other real-world applications [7], electrical engineering [8], civil engineering [9], mechanical engineering [10], bioinformatics [11] and medical applications [12]. It can be easily adapted to any optimization matters according to one's needs. There have been several uses of this algorithm in motion planning and visual tracking [13], but to the best of our knowledge none of the previous researches have applied this algorithm to the grid-based graph search problem.
In mobile robotics, the path-finding problem in a gridbased environment could be defined as follows: where there is a set of vertices V, a set of edges E between these vertices, a start state and a goal state, an optimal route ought to be established between the start state and the goal state in a finite period of time -the planning time.
The edges between the vertices in the environment have a concrete cost assigned to them. A solution cost is the cost obtained by traversing the vertices of the computed path from the initial point to the destination. The algorithm that will provide this functionality ought to give a feasible or optimal path between two points in a finite time, supposing this path exists. To perform in such way, the hypothesis of safe exploration of the search problems needs to be established. Concretely, edges have finite costs; furthermore, where there are states s1, s2 and s3, if s1 and s2 are connected and s2 and s3 are connected, then s1 and s3 are connected.
There have been many attempts to obtain acceptable solutions which perform equally well with reduced time. Formally, all meta-heuristic algorithms mentioned in this chapter are applicable to path-finding problems in gridbased environments. However, they present various drawbacks depending on the environment structure. Many of them tend to become stuck in a local optimum of some sort, so that the result given is not optimal. This is especially evident in grids with the decreasing of the free space for robot movement. This fact makes them impractical for real-time applications as well as for remote control.
The main aim of this work is to propose a new algorithm capable of finding the optimal path (if one exists) in a grid-based environment at reduced time, compared to other meta-heuristic algorithms. This novel algorithm will prove its adaptability towards various types of realworld environments, having various percentages of obstacles and different sizes, and to prove its speed of execution and accuracy as well.
The proposed solution first separates the search space into smaller rectangles using the quad-tree (QT) algorithm, and then calls itself on these sub-problems recursively. During the search space division, rectangles in the search space are marked (coloured) and the fields with the same number (colour) are furthermore treated as a single node. This helps to create a reduced graph together with an adjacency list of the neighbouring rectangles, on which the HS approach is applied in order to find the shortest path between two given cells in the grid. The performances of the proposed algorithm have been evaluated empirically and using simulations. The obtained results confirm that the proposed quad harmony search (QHS) algorithm is superior compared to known meta-heuristic algorithms applied in the same environment and conditions.

Related work
Many meta-heuristic algorithms have been developed and applied for solving NP-complete problems such as the path-finding problem, mainly because they are robust enough and capable of producing acceptable solutions very quickly.
Particle swarm optimization (PSO) [14] is a populationbased meta-heuristic algorithm inspired by the swarm behaviour of fish and bird schooling in the natural world. The algorithm explores the space of a fitness function by regulating the paths of individual searching agents, commonly known as particles. Besides the original algorithm, many PSO variants have been recently developed and applied to the grid-based robot path planning. For example, in [15], the paper proposes an approach using PSO in a shortest path-finding problem, which gave better results than algorithms that used the genetic algorithm (GA) as an alternative to obtain the optimal route. This approach also has the advantage of removing loops in providing the best possible path. The path-planning algorithm presented in [16], where PSO is applied in the path-planning problem for a mobile robot, the MAKLINK graph is firstly constructed to describe the search space of the robot, then the Dijkstra algorithm is applied to provide the shortest distance from the start node to the end node, and finally the PSO is used to give the optimal path. Results have shown that this approach is applicable in real-time mobile robot navigation. The same authors later presented a modified version of the approach in the previously published paper [17]. In order to escape the plunging into the local minimum, they added a mutation operator, which led to greater speed in performance in the early phase of the algorithm. The hybrid algorithm described in [18], known as CIPSO, includes a combination of several techniques, including artificial immune system (AIS), chaos operator and the PSO algorithm. This approach has been shown to give better results than the GA and the PSO in terms of optimality of route and the time execution of the algorithm.
Simulated annealing (SA) [19] is a meta-heuristic algorithm that is often utilized in problems that require the obtaining of a solution in a certain amount of time, rather than requiring the optimal solution. The slow cooling that appears in the process of annealing (a process known in metallurgy) is implemented in the algorithm, as it performs a slow decrease, respecting the probability of accepting worse solutions in the search space. It has also found its use in several mobile robot path-planning problems, and it has also been combined with other techniques. In [20], the SA algorithm is applied to the robot path-planning problem to boost the artificial potential field (APF) approach and avoid plunging into local minima, a method frequently used in real-time problems. This algorithm has been shown to be greatly effective in local and global path finding. The paper [21] describes the same technique and applies this method to soccer robots' path planning, and has also proven the validity of this approach. Also, a simulated annealing neural network has been introduced in [22], which is used to describe the obstacles in the robot's environment. This path planner has been effectively applied to several kinds of robots, like flying robots and snake robots.
The ant colony (AC) algorithm's probabilistic nature gives well-approximated solutions and is often used as an optimization technique [23]. It is mainly based on the behaviour of ants as they search for food and find a path from their colony. After finding food, they return to their colony, leaving pheromone trails. These pheromone trails tend to evaporate over a given period, which is due to the concrete time an ant has to travel to the food. Hence, the shorter the path, the less evaporation will happen and the pheromone density will increase. This phenomenon is the reason why this algorithm cannot get stuck in a locally optimal solution. When one ant finds food by following a given (shorter) path, the other ants will be more likely to follow the same path. The paper [24] shows the application of the AC to the two-dimensional robot pathplanning problem, which solves the problem of the local optima and increases search speed. In the method proposed in [25], called SACOdm, the decision-making process is improved by storing the existing distances in memory, since the ant colony algorithm itself cannot remember the visited nodes, which produces a speed-up of around 10 in several cases [25]. Then, the optimal path is selected using fuzzy interference systems, applying the simple tuning algorithm. An improved augmented ant colony algorithm has been introduced in [26], which decreases the time for the execution of the initialization phase by adding the heuristic probability function in this phase, which solves the precocity of the algorithm. Results have shown that this technique has better performance and greater optimization capabilities than the traditional augmented ant colony algorithm. The AC algorithm has also been applied to the robot pathplanning problem in a dynamic environment with dynamically appearing obstacles [27]. This implementation uses two different re-initialization schemes.
Another meta-heuristic algorithm which is also used for search and optimization is the genetic algorithm [28], which is a search heuristic mirroring process of natural evolution, such as the processes of crossover, mutation, selection and inheritance. Here, the evolution is headed towards better solutions, typically encoded with 0s and 1s, but there can be other types of encodings. This algorithm finishes when a certain number of iterations is reached, or a given value of the fitness function is obtained; hence, the solution to the problem is the finally generated generation. A knowledge-based genetic algorithm (GA) applied to the mobile robot pathplanning problem has been introduced in [29], applying its domain knowledge to its operators. This effective technique has demonstrated its utility in obtaining the optimal or near-optimal path of a mobile robot both in static and dynamic environments. The global path planning using neural network and genetic algorithm presented in [30] uses the neural network to construct the search space (environment) of the robot in order to establish a connection between a collision avoidance path and the output of the neural network, and then this information, along with the information for obtaining the optimal route, is fed to the fitness function of the genetic algorithm. Also, an effective method to obtain the optimal route of a mobile robot is the chaos genetic algorithm [31], so that when the chaos operation is added to the genetic algorithm, it decreases the time needed for convergence and performs well in avoiding collisions with obstacles. The paper [32] has described the implementation of the GA on the path-planning problem by using four neighbour movements in a grid-based environment.

The harmony search algorithm
Harmony search (HS) is a music-inspired technique which mimics the process of improvisation of jazz musicians; it is analogous to the musicians' examination of all the possible combinations of musical pitches they have remembered and their recalling them from memory. In such a fashion, this algorithm is applied to exploring such combinations with the aim of solving various optimization problems. It is a probabilistic technique which finds the optimal solution in several different stages.
1. First, the harmony memory (HM) is initialized. This process is performed by generating random numbers in a specified range, and this memory would look as follows: where n is the dimension of the candidate vectors and N is the total memory size.
2. Secondly, a new candidate solution is improvised, , based on the harmony memory consideration rate (HMCR), which is defined as the probability of the new candidate solution being chosen from the HM. So, in a similar way, the probability of the candidate solution to be selected randomly is (1-HMCR). If a member of the candidate solution is chosen from the harmony memory, it can be changed further due to the pitch adjustment rate (PAR), which is the probability of the candidate being mutated.
3. Thirdly, the new candidate solution is compared to the other candidate solutions stored in the HM. If the new candidate solution provides a better result than the worst candidate solution whilst evaluating the function, it is stored in the memory and the worst solution is discarded from the memory. Otherwise, this new candidate solution is not considered in future, and the previous processes are repeated. The algorithm stops when a certain termination criterion is reached, such as number of iterations.

Quad harmony search (QHS) algorithm description
In this paper, we introduce a novel approach, based on the utilization of the HS algorithm, to the problem of graph search in grid-based environments [1,33]. The grids that have been examined are of a rectangular shape, having square-sized cells of fixed height and width, and the agent was able to move in four directions, marked as north (N), east (E), south (S) and west (W). For the purposes of algorithm optimization, we use a search space reducing technique in order to speed up the process of finding a solution, and provide a better convergence in fewer iterations and with smaller memory size. This technique has been widely used in geometry-related problems, but for the main goals of this paper, it proved to raise an important issue in terms of time and space complexity. It is commonly known as a quad-tree algorithm.
The quad-tree (QT) algorithm is based on a dividing technique which separates the search space into four blocks, i.e., sub-problems, and then calls itself on these sub-problems recursively. The algorithm stops with the recursive calls when it meets certain defined criteria. For the main goals of this research, the QT technique is used in a way which marks (colours) certain squares in the search space, and the fields which bear the same number (colour) are furthermore treated as a single node. The technique is adapted to this research as follows: 1. Take as input the full grid to be examined. The running of this algorithm on an example grid and its result are shown in Figure 1 and Figure 2.
After the grid is labelled with the proper numbers, a reduced graph is constructed using the previous separation, and an adjacency list is obtained using the neighbouring rectangles (squares). This approach is extremely efficient when it comes to searching through a maze that has a large amount of free areas and the percentage of obstacles in the grid is relatively small. Once the adjacency list is constructed, the proper input data is ready to be fed to the HS algorithm. The memory consideration rate of the algorithm (HMCR), as defined in the previous section, is the probability for a candidate solution to be chosen from the harmony memory (HM). The greater the consideration rate, the greater the probability for a candidate solution to be selected from the harmony memory (HM), as in Eq. 2 [2]: where Xi is the set of possible values in range [1;maximumNumberOfRectangles], which means all possible values are dependent on the number of rectangles generated by quad-tree, denoted as maximumNumberOfRectangles. The equation states that, with a probability of HMCR, a candidate solution xi' will be generated from the HM, and with a probability of (1-HMCR) it will be generated randomly from the set of possible values Xi.
Since the parameters are set correctly, the fitness function needs to be defined, which is used to obtain convergence to a desired and acceptable solution. In a single iteration of the evaluation, the following process is implemented (the pseudocode is available in Figure 3): 1. Declare a variable prevRect to track the previously visited rectangle. 2. Set prevRect to the start node in the candidate vector. 3. For each member in the vector: a. If we have reached the end node, increase the value of the fitness function by one and terminate this iteration. b. Declare variable now and set it to the value of prevRect. c. From all of the neighbours of the current rectangle, choose the next rectangle for exploration in the following manner: the next value in the candidate vector modulus number of the neighbours of the current rectangle. This number is the index of the next rectangle in the adjacency list. d. Increase the fitness function by one and if the end of the vector is not reached, return to step 3.a.
Thus, given the algorithm previously described, the formula for the fitness function ready to be minimized would be simply defined as: where d is a direction the algorithm is taking (a node in the maze), D is the set of directions (nodes) given by the random candidate solution (with the destination node as final element), and PathCost(d) is a predefined cost for taking the direction d and can be defined as follows: When the end of the HS stage is reached, the path from the best resulting candidate vector previously obtained is constructed. This is done by retrieving the positions of the upper-left corners and dimensions of each of the rectangles present in the obtained solution, and by simple calculation of the minimum Manhattan distances between them, which greatly improves the performance of this algorithm.

Simulation
The developed QHS algorithm has been evaluated for global path planning in grid environments using simulations. Under the same conditions it was compared with other known meta-heuristic approaches: ant colony (ACO), genetic algorithm (GA), particle swarm optimization (PSO) and simulated annealing (SA). The simulation environment has been developed and implemented in Java programming language. All the evaluated algorithms were implemented as separate modules. An additional module was developed for generation of grid environments with a random filling percentage of obstacles. All the simulations were executed on a PC with an Intel® Core i5 processor with a frequency of 2.53 GHz, 4GB of RAM and 64-bit Windows 7 operating system.
In the simulation experiments, we used the following values for the QHS parameters: HMCR of 0.9, BW (bandwidth) of 0.2, PAR of 0.4, size of a candidate vector set to the number of labelled rectangles (squares), size of HM set to 10 and number of iterations (the termination criterion) set to 50.
The planning time, i.e., the time required to find the optimal path from a given start to a given end position was considered as a main evaluation parameter.
In the first simulation test, all the algorithms have been run on the example grid shown in Figure 1. For the given grid and the same start and goal position (marked with green and red respectively in Figure 1), the obtained results are shown in Table 1.

Algorithm
Planning Simulated Annealing 37958 Table 1. Comparison of the planning times of the evaluated algorithms for the grid given in Figure 1 From the results, one can observe that the QHS algorithm has given the best results compared to the other algorithms whilst obtaining the optimal solution, while the genetic algorithm gave the worst results.
To The measured planning times for grids of a specific size and obstacle percentage were averaged and calculated for each of the algorithms as: The average planning times calculated using Equation 5, given in milliseconds, are presented on Table 2, Table 3, Table 4, Table 5 and Table 6 for each algorithm, respectively. The comparison of the tested algorithms for the generated grid variants is shown in Figures 4-8       The developed algorithm has also been evaluated experimentally. The main aim of the experimental evaluation was to confirm the ability to generate an optimal and collision-free path of the developed algorithm. For experimental evaluation of the developed algorithm, a specific laboratory setup for indoor motion planning was used.
For this purpose, besides the path-planning phase, the system should also be able to perform robot localization and motion control. The system consists of three main components: mobile robot, control workstation and ceiling-mounted cameras for localization of the robot, obstacles and the target.
The three-wheel mobile robot used in our study is a modified version of the ARobot (Figure 9) [34]. The robot contains one Basic Stamp II controller from Paralax [35], and two coprocessors: PIC16F84 for motor control. The robot has the following sensors: sonar, two light sensors, temperature sensor, whisker sensors, PIR passive infrared motion detector, digital compass, R283-HOKUYO-LIDAR and sound output transducer. The robot has two 12-volt DC drive motors. These motors are regulated independently using PWM-controlled H-bridges. The robot also has optical encoders that enable determination of speed and position of the robot's wheels. All these components are placed in a lightweight aluminium construction with these dimensions; 10" x 10", 5" tall, with a payload capacity of 3 lbs. The mobile robot is connected to the control workstation via wireless link using RF two-way radio modules. The workstation is running a program that is responsible for robot localization, path planning and motion control planning.
Considering the dimensions of the working environment and the necessity for real-time control, a multi-camerabased system for localization tasks has been employed. The working environment was virtually split into four overlapping regions and each of them has been fully covered by a ceiling-mounted web camera with a 90degree field of view, sending 640 x 480 real-time videos to the command workstation for further processing. This way, all the obstacles (marked with white or green colour), the target (marked with blue) and the robot, marked with QR code, were captured with satisfactory resolution, and thus became easily detectable ( Figure 10). The image processing task has been performed at the control workstation by the localization module, which takes video as input, detects the obstacles and generates the grid-based map. The obstacles were detected using an algorithm based on colour filtering and edge detection with a localization error of less than 2 cm.
Once created, the grid-based map is sent to the pathplanning module, which employs the developed quadtree harmony search algorithm to find the path from the start to the end point. At the end, in order to control the mobile robot, the control program running on the workstation sends direct commands to the robot actuators to perform the actual movement. In the experimental evaluation, the planning time is considered as the time required to find the optimal path from a given start to a given end position, after obtaining the grid-based map of the environment until the path calculation.
For the experiments, two grid sizes, 64 x 64 and 128 x 128, with 30 %, 40 % and 50 % obstacles were considered. Ten grid samples of each grid type were generated, and for each of them two different start and goal positions were analysed. In all the cases, QHS found the optimal path. The average planning times obtained experimentally are shown in Table 7.  All the results were verified for correctness using a deterministic algorithm, namely the breadth-first search algorithm. It was easily proved that all of the paths obtained by QHS have the exact same length with the paths provided by the breadth-first search algorithm.

Discussion
Analysing the simulation results provided by QHS algorithm, we can easily infer that at a lower percentage of obstacles (10 %-20 %) we mainly get faster planning times, because we use the QT algorithm to determine the free areas in the grid-based graph. This is also the case for a greater percentage of obstacles (70 %-90 %). However, in the range of 30 %-60 % of obstacles, we get longer planning times. This can also be explained in terms of the greater number of rectangles that have to be examined; therefore, we get greater memory size (HM), more possibilities to consider, and all these lead to a longer period spent exploring the optimal solution.
Comparing the results we have obtained with QHS to ant colony (ACO), genetic algorithm (GA), particle swarm optimization (PSO) and simulated annealing (SA), several crucial conclusions have been drawn. Ant colony proved to always give the optimal path, though it took longer to execute than QHS in 100 % of the test cases. Using the GA, PSO and SA, results show that the algorithm can always find a feasible solution, but on the examined test cases it never obtained the optimal solution. This means that GA, PSO and SA can easily be stuck in a local optima and thus not give the best possible solution. Also, in terms of time execution compared to QHS and ant colony, these three algorithms (GA, PSO and SA) gave poor results.
Thus, one can simply infer that the proposed QHS is the best fit to the problem of graph search in maze-like environments.
The results obtained with the experimental evaluation correspond to the simulated ones. Experimentally obtained planning times for optimal path calculation make the QHS suitable for real-time robot control and applicable to various real-world applications. Relatively small standard deviations prove the stable behaviour of the algorithm and its robustness.

Conclusion
In this research, we introduced a technique to find an optimal solution to a grid-based graph search using quad-tree decomposition to reduce the search space, and harmony search to find the optimal route between the connected areas, finishing by extracting the path using the Manhattan distance between the maze fields that belong to these areas. This algorithm was compared to ant colony, genetic algorithm, particle swarm optimization and simulated annealing and was proven to give the best results correlated to obtaining the optimal path. It has also been shown that our approach gave best results in mazes with lower percentages of obstacles, which means that it is suitable to use when one needs faster performance in these concrete cases. This gives an open opportunity for further researches and examinations of the application of this technique in various graph searches, and for the expansion of this approach to greater dimensions.