AN EFFICIENT Hybrid Swarm Intelligence Technique for Solving Integer Programming

In this paper, a hybridization of two different swarm intelligent approaches, stochastic diffusion search, and particle swarm optimization techniques is presented for solving integer programming problems. The hybrid implementation allows us to avoid certain drawbacks and weaknesses of each algorithm, which means that we are able to find an optimal solution in an acceptable computational time. Our hybrid implementation allows the IP algorithm to reach the optimal solution in a considerably shorter time than is needed to solve the model using the entire dataset directly within the model. Our hybrid approach outperforms the results obtained by each technique separately. It is able to find the optimal solution in a shorter time than each technique on its own, and the results are highly competitive with the state-of-the-art in large-scale optimization. Furthermore, according to our results, combining the PSO with SDS approach for solving IP problems appears to be an interesting research area in combinatorial optimization.


INTRODUCTION
Optimization can be viewed as one of the major quantitative tools in network of decision making, in which decisions have to be taken to optimize one or more objectives in some prescribed set of circumstances. In view of the practical utility of optimization problems there is a need for efficient and robust computational algorithms, which can numerically solve on computers the mathematical models of medium as well as large size optimization problem arising in different fields. Heuristics and bioinspired techniques have become efficient and effective alternatives for researchers in solving several complex optimization problems. These techniques are not able to reach the optimal solution for large-scale combinatorial optimization problems in spite of their effectiveness. But these techniques are able to provide satisfactory solutions for most of the applied problems within acceptable computational times. In contrast, mathematical programming techniques, particularly the Integer Programming, have been studied and developed by scholars over several decades with the main goal of obtaining optimal solutions to difficult problems using as little CPU time as possible. For these reasons, a hybrid swarm intelligence technique has been suggested. In recent years, swarm intelligence, which can be considered as a branch of Artificial Intelligence techniques, has attracted much attention of researchers, and has been applied successfully to solve a variety of problems. A swarm can be viewed as a group of agents cooperating with certain behavioural pattern to achieve some goal [10]. There are a number of different models of swarm intelligence that have been proposed and investigated, and among the most commonly used swarm intelligence models include ant colony optimization [3], [6], particle swarm optimization [15], [4], honey bee swarming [29], [30], stochastic diffusion search, and bacterial foraging [24], [25]. These algorithms have proved their mettle in solving complex and intricate optimization problems arising in various fields. The paper is organized such that the next section 2 provides a brief overview of integer programming. Section 3 describes Particle Swarm Optimization technique. Section 4 describes Stochastic Diffusion Search technique. Section 5 describes the method of the proposed hybrid swarm intelligence technique used. Section 6 discusses the computational results. In section 7, a conclusion is introduced.

INTEGER PROGRAMMING
It is often impossible to represent certain features of many real-world problems using only linear constraints and continuous variables. In modeling a real world problem, it is often necessary to represent discrete activities by variables which are restricted to take only integer values. The general mathematical form of integer programming problems is: This problem is called the linear integer-programming problem. It is said to be a mixed integer program when some, but not all, variables are restricted to be integer, and is called a pure integer program when all decision variables must be integers. If the constraints are of a network nature, then an integer solution can be obtained by ignoring the integrality restrictions and solving the resulting linear program. In general, though, variables will be fractional in the linearprogramming solution, and further measures must be taken to determine the integer-programming solution. We consider the 0-1 integer programming problem.
We will restrict our attention to the case where a is 0-1, and where b is integer. Also with these restrictions, the problem is NP-hard, and several well known NP-hard problems, such as the set partitioning, covering and packing problems, are conveniently stated in this way. Common solution methods for ILP are based on solving LP, which can usually be done efficiently. If LP happens to give a 0-1 solution, this is also an optimal solution to ILP, and for certain common and nontrivial subclasses of 0-1 problems, LP always have an integer solution. For more difficult problems particular instances may also be easy in this sense. If however the LP solution has many noninteger values, very little information about the solution to the 0-1 problem is obtained in this way, and typically techniques such as branch and bound have to be used to resolve the solution to integrality. This can work very well for small problems and also for larger problems with special structure, but nevertheless strongly limits the range and size of problems that can be solved. For a full presentation of existing methods, see for example [12], [27]. Optimization techniques developed for real search spaces can be applied on Integer Programming problems and determine the optimum solution by rounding off the real optimum values to the nearest integer [23], [26]. One of the most common deterministic approaches for tackling Integer Programming problems is the Branch and Bound (BB) technique [11], [22], [26]. The technique is based on the observation that the enumeration of integer solutions has a tree structure. The main idea of the branch and bound algorithm is to find an optimal solution and to prove its optimality by successively partitioning the feasible set of the solution, or the original problem, into subproblems of smaller size. These subproblems are investigated by computing lower/upper bounds of the objective function. These lower/upper bounds are used to avoid exhaustive search of the solution space. Evolutionary and Swarm Intelligence algorithms are stochastic optimization methods that involve algorithmic mechanisms similar to natural evolution and social behavior respectively. They can cope with problems that involve discontinuous objective functions and disjoint search spaces [8], [14], [28]. Genetic Algorithms (GA), Stochastic Diffusion Search (SDS), and the Particle Swarm Optimization (PSO) are the most common paradigms of such methods. Early approaches in the direction of Evolutionary Algorithms for Integer Programming are reported in [9], [13].

PARTICLE SWARM OPTIMIZATION
The particle swarm optimization (PSO) was inspired by the observations of birds flocking and fish schooling. It differs from other well-known Evolutionary Algorithms (EA) [1], [7], [8], [14], [28]. As in EA a population of potential solutions is used to probe the search space; but, no operators, inspired by evolution procedure, are applied on the population to generate a new promising solution. Instead, in PSO, each individual (named particle) of the population (called swarm), adjusts its trajectory towards its own previous best solution (called pbest) and the previous best solution attained by any member of its topological neighborhood. There are different kinds of sharing information between particles. In the global variant of PSO, the whole swarm is considered as the neighborhood. Thus, global sharing of information takes place and the particles benefit from the discoveries and the previous experiences of all other companions during the search for promising regions of the landscape [17]. Alternatively, there are some local variants of PSO wherein particles only make use of their own information and that of the best of their adjacent neighbors. Each particle in PSO has two main characteristics: its position and its velocity. Assume that the current position and velocity vector of the i-th particle in the ddimensional search space are denoted as Xi = (xi1, xi2,…., xid) and Vi = (vi1, vi2,…., vid), respectively. The best earlier position of the i-th particle is represented as pbesti = (pbesti1, pbesti2,…., pbestid). There are different kinds of PSO including global vision of PSO with inertia weight (GWPSO), local vision of PSO with inertia weight (LWPSO), global vision of PSO with constriction factor (GCPSO), and local vision of PSO with constriction factor (LCPSO) [31]. In GWPSO, which is very popular among researchers, there are two methods for updating position and velocity of each particle. The best position of entire group at k-th iteration is used in the first method while in the second method; the best position of entire group up to the current search is employed. In the first method, the position and velocity of particle i in the k-th iteration are updated as follow: In the second equation w is the inertia weight, c1 and c2 are positive constants called cognitive and social parameters, respectively, and r1 and r2 are random numbers selected in the interval [0 1]. The constants c1 and c2 represent the weighting of the stochastic acceleration terms that pull each particle towards pbest and gbest positions and usually are set c1=c2=2 . In the second method gbest is replaced by gbestk. As will be shown later, in the numerical examples of mixed-variables or in the problems that only have discrete variables, usage of gbestk is more suitable compared to the use of gbest. In other words, the success rate of gbestk is higher than that of gbest. The reason is firstly due to the fast convergence of gbest and secondly, the inability of particles to escape from local minima in gbest method. In other words, since the discrete variables are rapidly converged the continuous variables will be obliged to search in a limited specific area which might not be the optimum area. The role that inertia weight w plays in the convergence behavior of PSO is very important. The inertia weight is employed to control the effect of the previous velocities on the current velocity. This way, the parameter w makes a compromise between global and local exploration abilities of the swarm. In PSO, when the search continues, the inertia term decreases linearly as: Where and are the maximum and minimum values of the inertia term, respectively, and is the maximum number of iterations. These parameters are assumed to be: Sometimes as particle oscillations become wider, the system will gain tendency to explode [14]. The usual means of preventing explosion is simply to define a parameter and curb the velocity of every individual i from exceeding that velocity on each dimension d. In the case that velocity violates, it will be modified as follows: The effect of this is to allow particles to oscillate within the bounds [14].

Fig 1. The pseudocode of PSO algorithm
Although PSO has been used mainly to solve unconstrained, single-objective optimization problems, PSO algorithms have been developed to solve constrained problems, multi-objective optimization problems, problems with dynamically changing landscapes, and to find multiple solutions. There are some disadvantages of PSO Algorithm as follow:  The method easily suffers from the partial optimism, which causes the less exact at the regulation of its speed and the direction.  The method cannot work out the problems of scattering and optimization.  The method cannot work out the problems of non-coordinate system, such as the solution to the energy field and the moving rules of the particles in the energy field.

STOCHASTIC DIFFUSION SEARCH
This section introduces Stochastic Diffusion Search (SDS) [2], a multi-agent global search and optimization algorithm, which is based on simple interaction of agents. A high-level description of SDS is presented in the form of a social metaphor demonstrating the procedures through which SDS allocates resources. SDS introduced a new probabilistic approach for solving best-fit pattern recognition and matching problems. SDS, as a multi-agent population-based global search and optimization algorithm, is a distributed mode of computation utilizing interaction between simple agents [5]. Unlike many nature inspired search algorithms, SDS has a strong mathematical framework, which describes the behavior of the algorithm by investigating its resource allocation [19], convergence to global optimum [20], robustness and minimal convergence criteria [18] and linear time complexity [21]. The SDS algorithm commences a search or optimization by initializing its population. In any SDS search, each agent maintains a hypothesis, h, defining a possible problem solution.
After initialization two phases are followed, test Phase, and diffusion phase. In the test phase, SDS checks whether the agent hypothesis is successful or not by performing a partial hypothesis evaluation which returns a boolean value. Later in the iteration, contingent on the precise recruitment strategy employed, successful hypotheses diffuse across the population and in this way information on potentially good solutions spreads throughout the entire population of agents. In the Test phase, each agent performs partial function evaluation, pFE, which is some function of the agent's hypothesis; pFE = f(h). In the diffusion phase, each agent recruits another agent for interaction and potential communication of hypothesis.

Fig 2. The pseudocode of SDS algorithm
Passive recruitment mode is employed In SDS algorithm. In this mode, if the agent is inactive, a second agent is randomly selected for diffusion; if the second agent is active, its hypothesis is communicated to the inactive one. Otherwise a completely new hypothesis is generated for the first inactive agent at random. The main disadvantage of the SDS is in the case of search spaces distorted heavily by noise, diffusion of activity due to disturbances will decrease an average number of inactive agents taking part in random search and in effect will increase the time needed to reach the steady state.
Initialise particles While ( stopping condition is not met ) For all particles Evaluate fitness value of each particle If (current fitness < pbest ) pbest = current fitness If (pbest<global (or local ) best) global (or local ) best = pbest Update particle velocity Update par t icle position End End Initialising agents() While ( stopping condition is not met) Testing hypotheses () Diffusion hypotheses () End for ag = 1 to No of agents if (ag. activity () == false ) r ag = pick a random agent () if (r ag. activity () == true) ag. setHypothesis ( r ag .getHypothesis()) else ag. setHypothesis (randomHypothsis()) end

PROPOSED SDS-PSO TECHNIQUE
Because of the drawbacks of PSO algorithm, a hybrid swarm intelligence technique called SDS-PSO technique has been proposed for solving integer programming problems to produce better solution by improving the effectiveness and reducing the limitations. The motivating thesis justifying the merging SDS and PSO is the partial function evaluation deployed in SDS, which may mitigate the high computational overheads entailed when deploying a PSO onto a problem with a costly fitness function. In the hybrid algorithm, each PSO particle has a position, and a velocity; each SDS agent, on the other hand, has hypothesis and status. Every PSO particle is an SDS agent too together termed psAgents. In the psAgent, SDS hypotheses are defined by the PSO particle positions and a status which determines whether the psAgent is active or inactive (see Figure 4). Figure 5 shows the pseudocode of the proposed SDS-PSO technique. In the test-phase of a stochastic diffusion search, each agent has to partially evaluate its hypothesis. The fitness of each psAgent's particle's personal best is compared against that of a random psAgent; if the selecting psAgent has a better fitness value, it will become active, otherwise it is flagged inactive. On average, this mechanism will ensure 50% of psAgents remain active from one iteration to another. In the Diffusion Phase, each inactive psAgent picks another psAgent randomly, if the selected psAgent is active, the selected psAgent communicates its hypothesis to the inactive one; if the selected psAgent is inactive too, the selecting psAgent generates a new hypothesis at random from the search space. In the proposed technique, after each n number of PSO function evaluations, one full SDS cycle is executed.  Computed results show that in small problems; with 10, and 20 variables, execution times were approximately equal in both algorithms (SDS-PSO, and PSO). However moving to largest scale problems, the execution time of SDS-PSO was obviously smaller than that in PSO. It may be included from the results that as the number of variables increase the difference between execution time of the two algorithms assures the better performance of the SDS-PSO algorithm with a clear reduction of execution time dealing with large problems. From the previous results we can conclude from the author's view, the efficiency improvement between SDS-PSO and PSO as shown in the following equation