GENETIC-ALGORITHM BASED IMAGE COMPRESSION

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999 GENETIC-ALGORITHM BASED IMAGE COMPRESSION Merlo, G....
Author: Theodora Morris
3 downloads 2 Views 177KB Size
4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

GENETIC-ALGORITHM BASED IMAGE COMPRESSION Merlo, G.1; Caram, F1; Fernández V. 1, Britos, P. 2 Rossi, B. 2 & García-Martínez R. 1,2 2.

1. Intelligent Systems Laboratory. School of Engineering. University of Buenos Aires Software Engineering & Knowledge Engineering Center. Buenos Aires Institute of Tecnology Av. Madero 399. (1106) Buenos Aires. ARGENTINA [email protected]

Abstract. In this paper we analyze the image compression problem using genetic clustering algorithms based on the pixels of the image. The main problem to solve is to find an algorithm that performs this clustering efficiently. Nowadays the possibility of solving clustering problems with genetic algorithms is being studied. In this paper we make use of genetic algorithms to obtain an ordered representation of the image and then we perform the clustering to obtain the compression. With this purpose in mind, we have developed different clustering methods to apply to the ordered representation. Keywords: Genetic algorithms, image compression. 1

INTRODUCTION

We can split the clustering algorithms in two categories: constructive or iterative algorithms. In the constructive methods, the object assignment to various clusters is determined by developing a complete possible solution from a partial solution. In the iterative methods, an initial complete solution is always possible and one tries to improve this solution by different ways from one iteration to other. Popular clustering algorithms like K-MEANS belong to this category. An improvement to this algorithm has recently been developed by Ismail & Kamel [1989] (AFB y ABF) that alternates between a first depth search and a first breadth search to minimize the objective function. In this algorithms objects are sistematically moved to different clusters in each iteration, whenever this action decreases the value of the objective function. The greedy nature of these algorithms can make them get stuck to a local minimum. This problem can be avoided by taking different random initial configurations and then applying the transformation procedure to each of them. This type of evaluation is too ad hoc and the results quality strongly depends on data properties and on the objective function. Recently Klein & Dubes [1989] have applied simmulated annealing , but the main disadvantages of this method are the great amount of execution time and that an efficient schedule for a simmulated annealing algorithm is very difficult to achieve.

Our work is based on the paper of Bhuyan [1991]. These author considers the problem of partitioning N objects in M disjoint clusters using genetic algorithms to obtain a suitable object permutation. In this work, we consider a solution to the clustering problem based on genetic algorithms in which we search for the optimum solution by simultaneously considering and manipulating a set of possible solutions (population). It constitutes an alternative to the classical methods [Aldenferder & Blashfield,1984; Everitt, 1980]. We have defined a fitness function that minimizes the disorder among the elements we are ordering. In this way we do not mix the fitness calculation with the clustering procedure. We apply the clustering as the last step of the algorithm when the ordered representation has already been obtained. According to the results obtained in our experiments we think that the genetic algorithm solution to the clustering problem is very promising.

2

GENETIC ALGORITHM DESCRIPTION

The genetic algorithm used to obtain the ordered element representation is based on the algorithm introduced in [1].

2.1

Cost Function

In this work we did not consider the clustering in the cost function. We simply pretend to obtain an ordered element representation using the genetic algorithm, so what we intend to minimize is the disorder of them. In order to achieve this we propose: Given an ordered element representation:

X = [ X i ] N ordered elements , where each element of the vector is another vector containing p bytes.

The cost function is defined as follows: N −1

F ( xi ) = ∑ dist ( xi , xi +1 ) i =1

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

nearest object to the most recently added to the ordered list. This constructor, instead of searching among all the objects that are not included in the list, searches only in k objects, where k is a constant. The temporal complexity, in this case, is considerable reduced.

where: p

dist ( xi , xi +1 ) = ∑ ( xi , j − xi +1, j ) 2 j =1

The problem is now to minimize F, the distance sum from each element of the representation to the following one.

2.2

Clustering Solution Representation: Ordered Representation.

In this work we use the ordered representation to represent the solutions. In this representation each partition is represented by an object permutation. This kind of ordered representation gives information about the similarities among the objects instead of directly giving the optimum clusters. For example, let's assume that the chromosomic representation of the objects from 1 to 6 is the permutation (2 3 5 1 6 4). We assume that in a permutation that leads to an optimal solution similar objects are placed one close to the other. Based on this hypothesis the order ( 2 3 5 1 6 4 ) shows that 2 is more similar to 3 than to 5, and that 5 cannot be included in the same cluster as 2 without including 3 too. In the particular case of partitioning N objects in M clusters let's assume that (O1,O2,...,ON) is a particular permutation of N objects. Then each cluster consists in an object interval (Oi,Oi+1,...,Oj)

In this work we are going to use the C Constructor in which we allow the user to entering the parameter k.

2.4

Parent Selection operator

The problem we are facing is a function minimization, so the objective function f(x) must be mapped to a fitness function f(x). In this fitness function the probability of choosing a string xi to be crossed is given by:

f ( xi ) p

∑ f (x ) i

i =1

Where p is the population size. If f(x) is taken as the inverse of F(x) there will not be much difference between a good string and a bad one. This problem can be solved using another transformation: f(x)=Cmax-F(x)

where 1≤i, j≤N. With this constraint the number of different possible clusters is:

1 ⋅ N ⋅ ( N + 1) 2

where Cmax corresponds to the value of the worst string in the population. In our case Cmax was computed like t N −1

2.3

Initial Population Constructors.

∑ dist

máx

( xi , xi +1 ) = ( N − 1) * distmáx ( xi , xi +1 )

i =1

In [1] the authors explore three different initial population constructors for clustering algorithms based on GAs. For simplicity they are called A, B, and C. Constructor A places the objects in an ordered list at random. The use of this strategy to obtain an initial population allows testing genetic algorithms in the most adverse circumstances.The GA is completly ignorant about the good regions in the search space. Constructor B uses and heuristical algorithm based on the minumun distance between two objects. It generates an object label at random and places it in the first position of the ordered list. Then it searches among all the objects absent from the ordered list, to find the nearest object to the most recently added to the list. It repeats this procedure until all the objects are included in the ordered list. The complexity of this constructor is θ(N2). Constructor C uses a greedy and probabilistic heuristic to contruct the initial population. The primary intention in this constructor is to balance the knowledge about the problem with randomness. The diference between this constructor and constractor B is in the way they find the

where: p

distmáx ( xi , xi +1 ) = ∑ ( max ( xi , j − xi +1, j )) 2 j =1

As each xi,j is a byte, then

max ( xi , j − xi +1, j ) = 255 and p

Cmáx = ( N − 1) * ∑ 2552 j =1

The problem encountered here was that due to the great value of the maximum constant computed for the fitness function, all the fitness values for the Chromosomes of a population were too close each other. This produces a generation evolution based practically on an equal probability basis and not based on the selection of the most suitable components of the old population. In order to solve this problem, we scaled the fitness function in the following way:

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

f ' = a⋅ f +b; We determined the constants a and b using the following conditions:

a ⋅ f min + b = 0;

a ⋅ f prom + b = f prom

Solving this equations we obtained:

a=

f prom f prom − f min b=

parent1 = (5, 3, 8, 2, 27, 6, 4) and parent2 = (6, 5, 1, 2, 4, 7, 8, 3). Supposse that x=2 and xw=2 are selected randomly.Then, the corresponding string in parent2 to be cut is :X={5, 1, 2, 4, 7} and the corresponding sequence in parent1 is (5, 1, 2, 7, 4). In the first step of the method, the objects 3 and 8 are placed in the first and in the second positions of the offspring. In the second step, the objects 5, 1, 2, 7, and 4. are copied to the offspring starting from the third position. In the last step, the object 6 is added in the last position of the offspring, which results in (3, 8, 5, 1, 2, 7, 4, 6).

f prom ⋅ f min 5

f min − f prom

3

8

1

2

7

6

4

Padre Dominante X={5,1,2,4,7}

2.5

6

Cross Operators

1

3

Then the objective is to place the X-{x} objects sorrounding xp1 and the other (Xt - X) objects in the remaining positions so that the order of Xt-X and X in parent1 is mantained in the ofspring. This objective is achived by a three step process: • Place the objects Xt-X ∩{parent1[1], parent1[2],...,parent1[xp1]} in the offspring, the first object in the first position and so on, keeping in this way the sequence of parent1. • The objects present in X are placed in the offspring begining in the most left empty position and in the same sequence of parent1. • The objects Xt-X ∩ {parent1[xp1], parent1[xp1+1] ,..., parent1[N]}, where N is the string length, are placed in the vacant positions of the offspring by keeping the sequence in parent1. It is easy to see that the complexity of this algorithm is θ(N) Let's consider an example of this method: the set of objects to be partitioned is Xt = {1, 2, 3, 4, 5, 6, 7, 8}, and the parents:

4

7

8

3

8 Paso 1

Cross-operator 1

Let's consider Xt as the object set to be partitioned.We randomly choose a window size xw and an object x belonging to Xt. The position of x in the dominant parent ( parent1) is xp1 and the position of x in the support parent ( parent2 ) is xp2. We will call X the set of all the objects between xp2-xw and xp2+xw in the support parent. The first action made by the algorithm is to place the x in the xp1 position of the offpring.

2

Padre soporte

In this work we have used two cross operators that were explained in [1]. We are going to name them as cross-operator 1 and cross-operator 2. 2.5.1

5

3

8

5

3

8

5

1

2

7 4 Paso 2

1 2 7 Paso 3

4

6

Fig1 - Cross operator 1 (Translation: Padre Dominante = Dominant Father; Padre soporte = Suporting Father; Paso 1 = Step 1; Paso 2 = Step 2; Paso 3 = Step 3)

2.5.2

Cross-operator 2

In this crossing operation the first object in the offspring corresponds to the object in the first position of any of the parents. Then we determine the distance of the recently included object to the nearest one in both parents. The object which has the shortest distance is added in the offspring in the most left empty position. If the calculated distances are equal, then one object is selected at random. This process continues until all the offspring positions are filled. Let's see an example with 8 objects, in which the parents are: P1=(5, 3, 8, 1, 2, 7, 6, 4) and P2 =(6, 5, 1, 2, 4, 7, 8, 3). We take the first position in the offspring at random from the first position of any of the parents.Let's asume that in this case we select randomly P2, so the object 6 will be placed in the first position in the offspring. The objects nearest to 6 in parent P1 with a distance of 1 are {7, 4}. and in parent P2 is 5. Therefore, we select at random on object from 4, 5 and 7. Let's assume that the one selected is 7,which will occupy the second position in the offspring. The closest objects to 7 in parent P1 and in parent P2 are {2, 6} and {4, 8} respectivelly. Because of the object 6 has already been included, we choose an object at random from 2, 4 and 8 for the third position in the offspring.

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

Let's assume that the object selected is 2. In a similar manner all the remaining positions in the offspring are filled. The result chromosome is (6, 7, 2, 4, 8, 1, 5, 3). 5 3 8 1 2 7 6 4 Padre 1

6 5 1 2 4 7 8 3

3.1

This method is based on clustering the elements by assigning the same quantity of elements per cluster. Once we have obtained the final ordered representation we divide the amount of elements by the amount of clusters, obtaining in this way the amount of elements that each cluster will contain.

E=

Padre 2

6 7 2 4 8 1 5 3

Fixed Clustering

N M

where: N = amount of elements M = amount of clusters E = amount of elements per cluster.

The first E elements of the representation are assigned to the first cluster, the following E elements to the second one and so on until we assign the last E elements to the M cluster.

Fig2 - Cross operator 2

3.2 2.6

Replacement Operator

This operator is used to select a fixed size population based on an old population P(old) and on the offspring P(offspring) which was created using the cross operators. A parameter X, given by the user, determines that the new population P(new) has to be created using the X best strings from the combination of P(old) and P(offspring). The remainder of the strings from the new population P(new) are selected randomly from P(offspring). You can notice that when the parameter X is zero, the whole new generation is selected only from P(offspring). Otherwise, if the parameter X equals the population size then P(new) is formed by the best strings among all the strings in P(old) and P(offspring).

2.7

Mutation Operator

This operator tries to solve some inherent problems of the genetic algorithms as regards local minima. These problems appear because of GAs pay attention to the population as a whole instead of identifying the best individual. As is natural in genetic algorithms, they progress identifying high performance regions inside the search space. The genetic algorithms would be more useful in combinatorial optimization problems if they were adapted to invoke a local search strategy to optimize the final population members. The mutation operator function is select at random two objects of the string and exchange their positions.

3

In this method, the amount of elements inside each cluster is not fixed as in the previous one. Once we have obtained the final ordered element representation, we compute the euclidean distances between each element and the following one. After that we select the M-1 greater distances. The idea in this method is that the elements whose distances belong to this set of greater distances must be assigned to different clusters. Therefore we intend that the places in the vector where these greater distances appear were the limits between clusters.

3.3

Dynamic Clustering with Variable Clusters

This method is similar to the previous one; the difference is that the amount of clusters is not fixed (M). Instead it is determined by the algorithm using a parameter entered by the user: the compression percentage. Using this parameter the algorithm computes the amount of clusters needed and then performs the clustering process using them. The amount of clusters is computed:

 (100 − P) ⋅ N  C = trunc   100  Where: C = Amount of clusters to be used. P = Compression percentage entered by the user. N = Amount of elements to be clustered.

IMPLEMENTED CLUSTERING METHODS

Once the ordered element representation was obtained by means of the genetic algorithm, we proceed to make the clustering of it. With this purpose in mind, we developed four different clustering methods that are explained below.

Dynamic Clustering with Fixed Clusters

3.4

Local Tuning

This is a clustering method in which the amount of clusters is fixed, but not the amount of elements assigned to each of them. This method begins by making a fixed clustering as it was explained in 4.1. Once these fixed clusters are determined we go through the vector ascendently, and for each last element of a cluster we compute the distance to the centroid of the

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

cluster(Dcp) and the distance to the centroid of the next cluster(Dcs). If Dcs < Dcp, then we change the cluster limit placing this element in the next cluster. If Dcs >= Dcp, we take no action. In this way we go through the whole vector until the last elements of all its clusters have been analyzed. After that we go through the vector descendently and we analyze in the same way the first element of each cluster, computing the distance to the centroid of their own cluster (Dcp) and to the centroid of the previous cluster (Dca). If Dca < Dcp, then we change the cluster limit placing this element in the previous cluster. If Dca >= Dcp, we take no action. In this way we go through the whole vector until the first elements of all its clusters have been analyzed. It is worth mentioning that each time an element is changed we have to recalculate the centroid of the cluster it belonged to and the centroid of the cluster it belongs after the change. The procedure of going through the vector ascendently and descendently is performed R times, where R is a parameter entered by the user.

4

RESULTS

In order to perform comparative testing, the following parameters were used: Amount of chromosomes : 50 Amount of generations : 100 Mutation probability : 0.01 X: 10 K: 400 R: 100 % Compression : 7--++65 The results of the testing are summarized in the table below: Cross Operator 1 1 1 1 1 2 2 2 2 2

5

Fixed Clustering Error 15,55 15,97 15,69 15,72 15,72 15,73 15,67 15,31 15,60 15,75

Dynamic Clustering Error 20,42 27,62 20,96 20,88 29,99 28,50 22,11 28,28 26,82 22,05

Local Tuning Error 14,61 14,88 14,94 14,39 14,65 14,54 14,92 14,44 14,60 14,70

CONSTRAINTS

The main constraint encountered in this work is that all the testing has been performed on the same image. In consecuence, all the evaluations and performance comparisons among the different methods are valid in this context. We consider that it would be important to perform tests using other images with different sizes and characteristics to confirm or complete the results exposed in this paper.

6

CONCLUSIONS

Based on the results obtained and detailed in the previous sections, we conclude the following: • The genetic algorithms constitute a powerful tool that can succeed in constructing a clustering algorithm • Among the methods proposed to perform the clustering, Local Tuning and Dynamic Clustering with Variable Clusters turned to be the best ones. • In spite of its simplicity, the Fixed Clustering method produced aceptable results. Anyhow, we do not recomend its utilization because it can be widely improved by the Local Tuning method with almost no increment in computing time. • The Local Tuning method appears to be the best one when we have a fixed amount of clusters. As we mentioned before, this method improves in a considerable manner the Fixed Clustering method performance. We think that an adequate value for the R parameter, should be between 50 and 100. Below these values, its action is not so noticeable and above them we do not obtain any further improvements. This can be explained with the concept that with a fixed amount of clusters the image cannot be improved further than a certain limit. Therefore, when there are no changes in one cycle, there is no sense in continuing applying this method because the mentioned limit has been reached. • The Dynamic Clustering with Fixed Clusters did not produce good results. In most of the cases, the results obtained with this method were worse than the ones obtained with the Fixed Clustering method. We observed that in the images compressed by this method the clusters created were too big, so there were much average in them. This characteristic can be observed in the images backgrounds which appeared to be practically uniform. Otherwise, object details are kept in a perfectly manner. Another problem that arises in this method, concerns the definition of the object edges which are not sharply defined. In spite of what we have exposed here, we do not want to completely discard this method because we think that it could reach good results with images of other characteristics such as uniform colors and well defined edges. • The results obtained with the Dynamic Clustering with Variable Clusters were excellent and by analyzing them we can observe a great improvement in the image sacrifying a little of compression. • As a final conclusion we propose the use of Local Tuning method when the clustering is constrained to a fixed number of clusters, and the Dynamic Clustering with Variable Clusters method when the amount of clusters can be increased to obtained a better representation of the image.

4o. SBAI – Simpósio Brasileiro de Automação Inteligente, São Paulo, SP, 08-10 de Setembro de 1999

7

FUTURE INVESTIGATION TRENDS

We think this work can be used as a basis to continue the research in Image Clustering methods using GAs. In this section we propose some research trends that can be of interest:

Gordon, A. & Henderson, J., 1977, An algorithm for euclidean sum of squared classifications. Biometric, 33, pp. 355362, 1977. Holland, J., 1975, Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbour.

• Implementation of the Local Tuning method by entering the compression percentage as a parameter. In this way we would let the number of clusters to be variable.

Ismail M. & Kamel, M., 1989, Multidimensional data clustering utilizing hybrid search strategies. Pattern recognition, Vol 22, Num 1, pp. 75-89.

• Comparison of the performance of the different clustering methods using different types of images. This research can complete or confirm the results obtained in our work.

Klein, R. & Dubes, D. 1989. Experiments in projection and clustering by simmulated annealing. Pattern Recognition, vol 22, pp. 213-220.

• Implementation of a clustering algorithm based on some of the proposed methods using as parameters the maximum and minimum error desired with regard to the original image. The algorithm must find the amount of clusters required to comply the specifications achieving the maximum possible compression. • Improvement of the Local Tuning method. in which the user must not enter the number of cycles of it. The method will detect when there are no changes in a cycle, and in this case it will stop automatically. • All the methods proposed in our work it may be possible to add a slight modification. Once you determine the amount of clusters to be used, fixed or variable, you can assign a part of them to perform the clustering of the image background and the rest of them can be used in the clustering of the image details. • Combination of various methods to perform the clustering of different parts of the image. For example you could use Local Tuning to perform the clustering of the image background and Dynamic Clustering with Variable Clusters to perform the clustering of the image details.

REFERENCES Aldenferder, M. & Blashfield, R. 1984, Cluster Analysis, Sage Publications, Beverly Hills. Bhuyan, J. 1991, Genetic Algorithm for Clustering with an Ordered Representation, Proceedings of Fourth International Conference on Genetic Algorithms, pp. 408-415. Brown, D, Huntley, R. & Spillane, C., 1989, A parallel genetic heuristic for the quadratic assignment problem. Proceedings of third international Conference on Genetic Algorithms, pp. 406-415. Everitt, B. 1980, Cluster Analysis, Heinemann Educational, London. Fisher, W., 1958, On Grouping for maximun homogeneity. Journal of American Stat. Asoc., 53, pp. 789-798. Goldberg, D. 1989, Genetic Algorithms in search optimization & machine learning. Addison-Wesley Publishing Company Inc.

Suggest Documents