Combining of Differential Evolution and Implicit Filtering Algorithm Applied to Electromagnetic Design Optimization Leandro dos Santos Coelho1 and Viviana Cocco Mariani2 1

Production and Systems Engineering Graduate Program, PPGEPS

2

Mechanical Engineering Graduate Program, PPGEM

Pontifical Catholic University of Parana, PUCPR Imaculada Conceição, 1155, Zip code 80215-901, Curitiba, Parana, Brazil Abstract Differential evolution (DE) is a population-based and stochastic search algorithm of evolutionary computation that offers three major advantages: it finds the global minimum regardless of the initial parameter values, it involves fast convergence, and it uses few control parameters. This work presents a global optimization algorithm based on DE approaches combined with local search using the implicit filtering algorithm. The implicit filtering algorithm is a projected quasi-Newton method that uses finite difference gradients. The difference increment is reduced as the optimization progresses, thereby avoiding some local minima, discontinuities, or nonsmooth regions that would trap a conventional gradient-based method. Problems involving optimization procedures of complex mathematical functions are widespread in electromagnetics. Many problems in this area can be described by nonlinear relationships, which introduce the possibility of multiple local minima. In this paper, the shape design of Loney’s solenoid benchmark problem is carried out by DE approaches. The results of DE approaches are also investigated and their performance compared with those reported in the literature. Keywords: evolutionary computation, electromagnetic optimization, differential evolution. Introduction The literature about electromagnetics contains several optimization algorithms for solving Loney’s solenoid design [1]-[3]. With the development of artificial intelligence, some alternative methods for the nonlinear, nonconvex, and discontinuous optimization problems in electromagnetics are revealed. Recent attempts to overcome the limitations of the traditional mathematical programming approaches include the application of evolutionary algorithms [4], taboo search [5], and particle swarm optimization [6]. In this context, Evolutionary Algorithms (EAs) are a class of stochastic search and optimization methods. These algorithms, based on the principles of natural biological evolution, have received considerable and increasing interest over the past decade. EAs operate on a population of potential solutions, applying the principle of survival of the fittest to produce successively better approximations to a solution. EAs encompass a range of different ‘nature-inspired’ methods,

2

Leandro dos Santos Coelho1 and Viviana Cocco Mariani2

including genetic algorithms, evolution strategies, evolutionary programming, genetic programming, and their variants. DE is one of the best EAs, and has proven to be a promising candidate to solve real valued optimization problems. DE is a population-based algorithm introduced by Storn and Price [7], [8] is a very simple but very powerful stochastic global optimizer. The crucial idea behind DE is a scheme for generating trial parameter vectors. DE combines simple arithmetic operators with the classical operators of crossover, mutation and selection to evolve from a randomly generated starting population to a final solution. This paper contributes by presenting hybrid approaches of DE combined with implicit filtering algorithm for the optimization of Loney’s solenoid design. The Loney’s solenoid design problem is an ill conditioned objective function with a very narrow valley that makes it very difficult to handle. Differential evolution The fundamental idea behind DE is a scheme whereby it generates the trial parameter vectors. In each time step, DE mutates vectors by adding weighted random vector differentials to them. If the cost of the trial vector is better than that of the target, the target vector is replaced by trial vector in the next generation. A number of alternative versions of basic configurations of DE have been proposed by [9], [10]. However, there is a variant of DE that has been reported, DE/best/1/exp. The different variants are classified according to the following notation: DE/α/β/δ, where α indicates the method for selecting the parent chromosome that will form the base of the mutated vector, β indicates the number of difference vectors used to perturb the base chromosome, and δ indicates the recombination mechanism used to create the offspring population. The bin acronym indicates that the recombination is controlled by a series of independent binomial experiments. The exp indicates a series of independent exponential experiments. The variant implemented here was the DE/best/1/exp, which is given by the following steps: A. Initialization of the parameter setup The user must choose the key parameters that control the DE, i.e., population size, boundary constraints of optimization variables, mutation factor (fm), recombination rate (CR), and the stop criterion (Gmax). B. Initialization of individual populations Set generation t=0. Initialize a population of i=1,..,M individuals (real-valued n-dimensional solution vectors) with random values generated according to a uniform probability distribution in the n dimensional problem space. Initialize the entire solution vector population in the given upper and lower limits of the search space.

Combining of Differential Evolution and Implicit Filtering Algorithm Applied to Electromagnetic Design Optimization 3 C. Evaluation of individual populations Evaluate the fitness value of each individual (in this work, the objective of the fitness function is to minimize the cost function). D. Mutation operation (or differential operation) Mutation is an operation that adds a vector differential to a population vector of individuals, according to equation (1): z i (t + 1) = x best (t ) + f m (t ) [ x i,r2 (t ) − x i,r3 (t )]

(1)

where i=1,2,...,M is the individual’s population index; j=1,2,...,n is the position in n of the dimensional individual; t is the time (generation);

[

]

xi (t ) = xi1 (t ), xi2 (t ), ..., xin (t ) T stands for the position of the i-th individual of a

population

[

of

N

z i (t ) = z i1 (t ), z i2 (t ), ..., z in (t )

]

T

real-valued

n-dimensional

vectors;

stands for the position of the i-th individual of a

mutant vector; xbest(t) is the best individual of the population in generation t; r2 and r3 are mutually different integers and are also different from the running index, i, randomly selected with uniform distribution from the set {1, 2, L , i − 1, i + 1, L , N } ; fm(t) > 0 is a real parameter called a mutation factor, which controls the amplification of the difference between two individuals so as to avoid search stagnation and is usually a constant value taken from the range [0.1, 1]. The mutation operation using the difference between two selected randomly individuals may cause the mutant individual to escape from the search domain. If an optimized variable for the mutant individual is outside of the domain search, then this variable is replaced by its lower bound or its upper bound so that each individual should be restricted within the search domain. E. Recombination operation Following the mutation operation, recombination is applied to the population. Recombination is employed to generate a trial vector by replacing certain parameters of the target vector by the corresponding parameters of a randomly generated donor vector. For each vector, zi(t+1), an index rnbr (i ) ∈ {1, 2, L , n } is randomly chosen using uniform distribution, and a trial vector, ui (t +1) = ui1(t +1), ui2 (t +1),..., uin (t +1) T , is generated with

[

]

zi j (t +1), if (randb(j) ≤ CR) or ( j = rnbr(i) ), ui j (t +1) =  xi j (t), if (randb(j)> CR) or ( j ≠ rnbr(i) )

(2)

where randb(j) is the j-th evaluation of a uniform random number generation with [0, 1]; CR is a crossover or recombination rate in the range [0, 1]; and F is the

4

Leandro dos Santos Coelho1 and Viviana Cocco Mariani2

evaluation of the cost function. Usually, the performance of a DE algorithm depends on three variables: the population size N, the mutation factor fm(t) and the CR. F. Selection operation Selection is the procedure whereby better offspring are produced. To decide whether or not the vector ui(t+1) should be a member of the population comprising the next generation, it is compared with the corresponding vector xi(t). Thus, if f denotes the objective function under minimization, then u (t + 1), if f (u (t + 1)) < f ( x i (t )), xi (t + 1) =  i  x i (t ), otherwise

(3)

In this case, the cost of each trial vector ui(t+1) is compared with that of its parent target vector xi(t). If the cost, f, of the target vector xi(t) is lower than that of the trial vector, the target is allowed to advance to the next generation. Otherwise, the target vector is replaced by a trial vector in the next generation. G. Verification of the stopping criterion Set the generation number for t = t + 1. Proceed to Step C until a stopping criterion is met, usually a maximum number of iterations (generations), Gmax. The stopping criterion depends on the type of problem. Combining of DE and implicit filtering method A fundamental and practical remark is that after a certain amount of time, the population in DE is quite uniform and the cost function of the population is no longer decreasing, the odds to produce fitter solutions (individuals) being very low. That is, the process has fallen into a basin of attraction from which it has a (very) low probability to escape [11]. It is experimentally clear that the exploitation of the basin of attraction that has been found may be more efficiently performed by another algorithm than by a DE. Hence, it is much more efficient to use a local search algorithm such as a hillclimbing. This schema of algorithm is qualified hybrid. DE and IF methods have potentialities that complement each other. The proposed combination of DE with Implicit Filtering (IF) for local search consists of a form of sequential hybridization. Basically, in this combined method, the DE is applied to the optimization problem and the best solution obtained by DE is used as starting point for the IF method. In this work, this hybrid approach is called DE-IF method. The quasi-Newton implicit filtering algorithms differ from other methods in the literature that use either inaccurate gradient information, only samples of the function, or difference or interpolatory approximations to gradients and/or Hessians. Implicit filtering, originally proposed in the context of computer aided design of semiconductors [12] is a generalization of the gradient projection algorithm of

Combining of Differential Evolution and Implicit Filtering Algorithm Applied to Electromagnetic Design Optimization 5 [13] in which derivatives are computed with difference quotients. The step sizes (called scales) in the difference quotients are changed as the iteration progresses with the goal of avoiding local minima that are caused by high-frequency, low amplitude oscillations. Real filtering could be performed, but this requires sampling and filtering the entire solution space and thus, is computationally quite expensive. Implicit filtering is very similar to adaptive meshing schemes used by the computational fluid mechanics community to avoid unwanted harmonics. The algorithm is fully described in [14] and [15]. Case study: Optimization of Loney’s solenoid design Loney’s solenoid problem presents an outstanding potential as a benchmark in the field of magnetostatic inverse problems [1]-[3], [16]. The design problem of Loney’s solenoid is to determine the distribution of the current density J which generates a uniform magnetic flux density B in a given interval on the axis of the solenoid. The upper half plane of the axial cross-section of the system is presented in figure 1. The interval of the axis, where the magnetic flux density must be uniform, is (-zo, zo). The current density J0 in the coils is assumed to be constant [6]. The position s and the length l of the correcting coils are unknown. The other dimensions are presented in figure 1.

Figure 1. Axial cross-section of Loney’s solenoids (upper plane). The variables s and l are optimized in this work by DE approaches. In this case, the optimization problem to be solved is the following minimization problem: min F(s,l)

(4)

where the objective function F of the minimization problem is given by: F=

B max − B min B0

(5)

6

Leandro dos Santos Coelho1 and Viviana Cocco Mariani2

where Bmax and Bmin are the maximum and minimum values of the magnetic flux density in the interval (-zo, zo) and are functions of s and l. In particular, three different basins of attraction of local minima are placed in the domain of F with values of F > 4·10-8 (high level region), 3·10-8< F < 4·10-8 (low level region), and F < 3·10-8 (global minimum region). Further details of Loney’s solenoid problem are discussed in [1]-[6]. Optimization results The DE and DE-IF approaches were calculated using MathWorks’ Matlab software. For each of the above described optimization algorithm, a total of 50 independent runs (using different seeds to generate random numbers in each run) were made, using the aforementioned parameters and different initial random seeds. All the runs were terminated after Gmax = 50 generations. A total of 1,000 cost function evaluations were made by each DE and DE-IF approach in each run. The variables were allowed to span (space search) within the range of 0 ≤ s ≤ 20 and 0 ≤ l ≤ 20. The DE/best/1/exp were selected, along with the values of CR=0.8 and a population size N of 20 individuals was selected.

Other specific parameters and design procedures used in the standard optimization methods, which were set empirically, were: • DE(1): classical DE with a constant mutation factor given by fm(t) = 0.4;

• DE(2): fm(t) was generated with uniform distribution in the range of [0.1; 0.7]; • DE(3): fm(t) was generated with uniform distribution in the range of [0.3; 0.8]. Table 1 summarizes the statistics of experimental results obtained by applying the DE and DE-IF approaches. An analysis of Table 1 reveals that IF (Implicit Filtering) presented difficulty and does not provide a good solution. According to the results obtained with IF, the solution presented a relatively high standard deviation compared with the other two methods. DE(2)-IF presented the best minimum of the results listed in Table 1. The best solution for the Loney’s solenoid benchmark problem was s= 11.3108 cm and l = 1.3484 cm. However, DE(3)-IF found mean values inferior to those resulting from the other methods. All DE(1-3)-IF methods found a solution very close to the best solution of DE (1-3) methods. However, the DE(1-3)-IF improve all the solutions obtained by DE(1-3) methods. The mean time of each run in a PC-compatible with Pentium IV 3.2 GHz processor and 2 GB RAM using Matlab 6.5 was 0.02 seconds for IF, 0.05 seconds for DE, and 0.08 seconds for DE-IF approaches.

Combining of Differential Evolution and Implicit Filtering Algorithm Applied to Electromagnetic Design Optimization 7

Table 1: Results (50 runs) for objective function minimization of Loney’s solenoid using DE and DE-IF approaches. Optimization Method Tested Approaches IF DE(1) DE(1)-IF DE(2) DE(2)-IF DE(3) DE(3)-IF

Objective Function in 50 Runs, F(s, l)·10-8

Minimum

Mean

28254.3503 3.4223 3.2245 2.7010 2.0593 3.1521 3.0564

51371.1260 3.8729 3.8428 3.8732 3.8336 3.8586 3.8214

Standard Deviation 49774.1623 0.1703 0.2052 0.2926 0.3376 0.1774 0.2287

Maximum 153427.5173 4.4179 4.4179 4.7663 4.7663 4.1812 4.1812

Conclusion and future research

In this work, Loney’s solenoid design was optimized using DE, IF and DE-IF approaches. DE algorithms offer potential advantages: they find the true global minimum regardless of the initial parameter values, they display a fast convergence, and use few control parameters. In this paper, the efficiency of the DE(2)-IF approach was validated, as shown in Loney’s solenoid benchmark (see Table 1). In comparison with the latest results reported in the literature [5], [6], the result listed in Table 1 for DE(2)-IF is better for Loney’s solenoid design. In future research, more detailed studies relating to fm(t) setup using DE approaches will be carried out to optimize Loney’s solenoid design. References [1] Di Barba, P.; Savini, A. (1995). Global optimization of Loney’s solenoid by means of a deterministic approach, International Journal of Applied Electromagnetics and Mechanics, Vol. 6, No. 4, pp. 247-254. [2] Di Barba, P.; Gottvald, A.; Savini, A. (1995). Global optimization of Loney’s solenoid: a benchmark problem, International Journal of Applied Electromagnetics and Mechanics, Vol. 6, No. 4, pp. 273-276. [3] Di Barba, P.; Dughiero, F.; Trevisan, F. (1997). Optimization of the Loney’s solenoid through quasi-analytical strategies: a benchmark problem reconsidered, IEEE Transactions on Magnetics, Vol. 33, No. 2, pp. 1864-1867. [4] Borghi, C. A.; Fabbri, M. (1999). Loney’s solenoid multi-objective optimization problem, IEEE Transactions on Magnetics, Vol. 35, No. 3, pp. 1706-1709. [5] Cogotti, E.; Fanni, A.; Pilo, F. (2000). A comparison of optimization techniques for Loney’s solenoids design: an alternative tabu search algorithm, IEEE Transactions on Magnetics, Vol. 36, No. 4, pp. 1153-1157. [6] Ciuprina, G.; Ioan, D.; Munteanu, I. (2002). Use of intelligent-particle swarm optimization in electromagnetics, IEEE Transactions on Magnetics, Vol. 38, No. 2, pp. 1037-1040. [7] Storn, R.; Price, K. (1995). Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical Report TR-95-012, International Computer Science Institute, Berkeley, USA. [8] Storn, R. (1997). Differential evolution  a simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, Vol. 11, No. 4, pp. 341-359.

8

Leandro dos Santos Coelho1 and Viviana Cocco Mariani2

[9] Price, K. (1999). An introduction to differential evolution, in Corne, D., Dorigo, M. and F. Glover (editors), New ideas in optimization, London, McGraw-Hill, UK, pp. 79-108. [10] Storn, R. (1999). Designing digital filters with differential evolution, in Corne, D., Dorigo, M. and F. Glover (editors), New ideas in optimization, London, McGraw-Hill, UK, pp. 109-125. [11]Preux, Ph.; Talbi, E. -G. (1999). Towards hybrid evolutionary algorithms, International Transactions in Operational Research, Vol. 6, pp. 557-570. [12] Stoneking, D.; Bilbro, G.; Trew, R.; Gilmore, P.; Kelley, C.T. (1991). Yield optimization using a GaAs process simulator coupled to a physical device model, Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Devices and Circuits, Ithaca, NY, USA, pp. 374-383. [13] Bertsekas, D. B. (1976). On the Goldstein-Levitin-Polyak gradient projection method, IEEE Transactions on Automatic Control, Vol. 21, pp. 174-184. [14] Gilmore, P.; Kelley, C. T. (1995). An implicit filtering algorithm for optimization of functions with many local minima, SIAM Journal of Optimization, Vol. 5, No. 2, pp. 269-275. [15] Gilmore, P.; Kelley, C. T. (1994). An implicit filtering algorithm for optimization of functions with many local minima, Tech. Report CRSC-TR94-23, North Carolina State University, Center for Research in Scientific Computation. [16] Coelho, L. S.; Mariani, V. C. (2006). Use of cultural particle swarm optimization for Loney’s solenoids design, Proc. 20th Biennial IEEE Conference on Electromagnetic Field Computation, Miami, FL, USA.