¹¹institutetext: National University of Singapore ²²institutetext: Bangladesh University of Engineering & Technology ³³institutetext: Mutah University

Image Contrast Enhancement using Fuzzy Technique with Parameter Determination using Metaheuristics

Mohimenul Kabir 11 Jaiaid Mobin 22 Ahmad Hassanat 33 M. Sohel Rahman Corresponding Author: msrahman@cse.buet.ac.bd22

Abstract

In this work, we have presented a way to increase the contrast of an image. Our target is to find a transformation that will be image specific. We have used a fuzzy system as our transformation function. To tune the system according to an image, we have used Genetic Algorithm and Hill Climbing in multiple ways to evolve the fuzzy system and conducted several experiments. Different variants of the method are tested on several images and two variants that are superior to others in terms of fitness are selected. We have also conducted a survey to assess the visual improvement of the enhancements made by the two variants. The survey indicates that one of the methods can enhance the contrast of the images visually.

Keywords:

image enhancement metaheuristics fuzzy logic genetic algorithm

1 Introduction

Image enhancement is the procedure of improving an image’s quality and information content. Image enhancement aims to increase visual differences among its features and make it more suitable for applications (e.g. increasing the brightness of dark images for viewing). Some common image enhancement techniques are sharpening, smoothing, increasing contrast, noise reduction, etc . [3]. Contrast enhancement is a process that is applied to images or videos to increase their dynamic range [4].

Image enhancement has been practiced as an applicable problem in meta-heuristics for a long time [1, 2, 13]. Moreover, the blending of fuzzy systems with meta-heuristic algorithms has recently received attention in the Computational Intelligence community [5]. Many works treated the image enhancement problem as an optimization problem and concentrated on altering the image quality fitness function [1, 3, 10], combining several meta-heuristic algorithms [1, 2], optimizing parameters [14] and escaping local optima [16]. One thing that has yet to gain more attention in image enhancement problems is the adaption of a fuzzy system. Sandeep and Samrudh [12] have shown that variation in the input membership function in a fuzzy system has a positive impact on image enhancement performance.

This paper addresses the image contrast enhancement problem by using stochastic optimization on the fuzzy logic system. The paper’s main contribution is to design a meta-heuristic technique by optimizing the fuzzy logic system. The logical component of the fuzzy logic system is a set of membership functions that are used to describe an intensity transformation function. We implement genetic operators which tweak the fuzzy logic system to enhance the original image to an optimal enhanced image. The fuzzy system of our paper is based on simple contrast enhancement rules described in [6], where the authors have used some image-independent rules and fuzzy sets.

Our main idea of the fuzzy image enhancement technique is as follows: we start with a basic fuzzy rule set and a set of input membership functions. By applying a metaheuristics framework, we try to evolve the fuzzy sets. Finally, we apply the transformation function described by fuzzy sets on the value channel of HSV color space to get the final image. Thus we have converted the problem into an optimization problem. In other words, rather than generating an image, we try to generate a suitable mapping between the input and output color values. However, this is inherently challenging since, even if we consider only 8-bit color, i.e., $256$ color values, the number of possible mappings becomes huge. Thus it becomes infeasible to search through the solution space exhaustively, and there is no definite knowledge about how to improve/generate a solution. This motivates us to leverage a metaheuristics framework [7].

2 Literature Review

Some contrast manipulation techniques are gamma transformation [22], histogram equalization (HE) [23], etc. HE is very useful in contrast enhancement [8]. But HE, while increasing the contrast, fails to keep image brightness the same [9]. BiHistogram Equalization [24] solves this issue. Another problem of HE is the information loss of image [11]. Also, gamma transform, log transform [25] can be used with lower computational complexity. In the work of [20], we can find applications of gamma transformation for contrast enhancement which applies different gamma corrections on multiple parts of pixel sets automatically. Unfortunately, these techniques don’t work well in a complex illumination setting [10]. So, they can not be applied without tweaking some parameters.

Fuzzy logic and metaheuristics techniques have been applied previously in image enhancement problems. One basic method is to apply a fuzzy logic-based system from [6] that uses three rules only and trapezoidal and triangular input fuzzy sets. Joshi and Kumar [12] have proposed a similar method, albeit with a more complex rule set ( $7$ rules) and Gaussian fuzzy sets.

To evaluate the fitness of an enhanced image, Munteanu and Rosa [13] have proposed a novel objective function and applied an evolutionary algorithm to search for optimal parameters in a continuous transform function. The same objective function has also been applied in artificial bee colony optimization [3], cuckoo search algorithm [15], and also in the firefly algorithm for UAV captured image enhancement [14].

3 Preliminaries

Fuzzy Image Processing

Fuzzy image processing is a collection of all approaches that understand, represent, and process the images, their segments, and features as fuzzy sets (see Figure 1).

Refer to caption — Figure 1: Fuzzy Image Processing. The figure is taken from [18]

The coding of image data (fuzzification) and decoding of the results (defuzzification) are steps that make it possible to process images with fuzzy techniques. The main power of fuzzy image processing is in the middle step (membership modification) in Figure 1 [19].

Fuzzy Sets for Intensity Transformation

Contrast enhancement is one of the principal applications of intensity transformations. We can state the process of enhancing the contrast of a gray-scale image using the following rules [6]:

•

IF a pixel is dark, THEN make it darker
•

IF a pixel is gray, THEN make it gray
•

IF a pixel is bright, THEN make it brighter

4 Problem Statement

Our goal is to manipulate the image contrast to enhance the sharpness of the image. Thus, image features will be more differentiable visually. While this is a subjective matter, the effort has been made to quantify it using a fitness function in the literature [15, 3, 16, 17, 13]. We leverage one such fitness function in our work. This fitness function is used to measure the quality of an image. A transformation function is used to enhance the image, which is further optimized using a metaheuristics approach. So, the input of the problem is an image, and the output is another image which is (expectedly) an enhanced version of the former.

We give a more formal definition of the problem below in the context of a gray-scale image (for simplicity). Suppose a gray-scale image, $I=f(x,y)$ , where $x$ and $y$ denote the pixels’ positions of the image. Image $I$ is of $M\times N$ size. So, $0\leq x<M$ and $0\leq y<N$ . Now assume that there is a fuzzy logic-based transformation function $T(I)$ that transforms the gray value of each pixel of the image and outputs another image $I_{e}=g(x,y)$ and a quality function, $Fitness(I)$

I_{e}=T(I)=T(f(x,y))

(1)

F=Fitness(I)=log(log(E(I_{s})))*\frac{ne(I_{s})}{M*N}*H(I_{s})

(2)

So, our problem is to find a transformation function (Equation 1) by optimizing (here maximization) Equation 2. For explanation of the terms in Equation 2, see Subsection 5.2.

5 Methodology

In this section, we present our approaches. We have explored five different metaheuristic approaches. In particular, we have used Hill Climbing (three variants) and Genetic Algorithm (two variants thereof). For the Hill Climbing approach, the variants differ in the mutation functions and input membership functions, as highlighted below. We used one type of input membership function for the Genetic Algorithm, but the difference is in new generation selection strategies (see Section 5.6).

•
Hill Climbing
1. 1.
  
  neighborhood generation using simple mutation
2. 2.
  
  neighborhood generation with trapezoidal and triangular input membership set splitting mutation
3. 3.
  
  mixed neighborhood generation with Gaussian input membership set splitting mutation
•
Genetic Algorithm
1. 1.
  
  simple neighborhood generation with trapezoidal and triangular input membership set
2. 2.
  
  simple neighborhood generation with gaussian and sigmoid input membership set

All these variants have some common parts, namely, initialization, representation, and fitness assessment of an optimization session, described below.

Finally, we have presented our methodology of experiments and values of parameters used in algorithms and experiments.

5.1 Population Representation

Every meta-heuristics technique starts with some (usually random) population, and the structure of the population is problem specific. In our case, each individual (in the population) will be one of the input membership functions, as shown in Figure 2. So, our population representation holds information on a set of input membership functions. There are at least three functions in each representation. Each membership function is represented by a tuple of three values regardless of the function types (i.e., trapezoidal, triangular, Gaussian, and Sigmoidal). The first two values of the tuple determine the shape of the membership function, and the third one is used in defuzzification (as will be clear shortly).

5.2 Fitness Assessment

We will formulate the enhancement’s quality/fitness $F$ using the quality function mentioned in Equation 2. There are some terms in Equation 2 which we will explain now,

$I_{s}$: $=$ image after Applying Sobel filter on $I$
$E(I)$: $=$ sum of intensity of image $I$
$ne(I)$: $=$ number of edge pixel in $I$
$H(I)$: $=$ entropy of image $I$
$M$: $=$ width of $I$
$N$: $=$ height of $I$

HSV Conversion

If the input image is a color image, we need to convert it to the HSV format. In the case of a gray-scale image, there is no need for such a conversion.

Defuzzification

Defuzzification refers to converting the fuzzy value to a single crisp value. For any given input pixel $z_{0}$ , the output crisp value $v_{0}$ will be as follows.

v_{0}=\frac{\sum\limits_{i=1}^{n}\mu_{i}(z_{0})*v}{\sum\limits_{i=1}^{n}\mu_{i}(z_{0})};n\geq 3

(3)

Here,

$\mu_{i}(z_{0})=$: fuzzy level of the image pixel with value $z_{0}$
$v=$: constant/mutable value (depending on the algorithm variant) used in the defuzzification

5.3 Hill Climbing

Hill Climbing stochastically generates candidate solutions and stores the best-found solution. The best solution is evaluated using the fitness function shown in Equation 2. In each generation, we generate a certain number of neighbor solutions (set to $10$ ) from a fixed individual. We have tried three variants of Hill Climbing differing from each other in neighborhood generation and input membership function type.

5.3.1 Simple Neighborhood Generation

In this variant, we have initiated a solution with two trapezoidal ( $\mu\_dark,\mu\_bright$ ) and one triangular ( $\mu\_gray$ ) functions like Figure 2. When generating a new solution, only the shape of the functions (i.e., the width of the triangle and trapezoid’s oblique line’s slope) are tweaked. We have used three hyperparameters, namely, $\mathsf{ChangeProb}$ , $\mathsf{MutateMu}$ , and $\mathsf{MutateSigma}$ . Firstly, $\mathsf{ChangeProb}$ represents the threshold of the random number chosen to decide whether the shape change of a member function will be done. On the other hand, $\mathsf{MutateMu}$ and $\mathsf{MutateSigma}$ represent, respectively, the mean and variance of Gaussian distribution from which the random number is chosen when generating a new solution.

Neighborhood Generation with trapezoidal and triangular input membership set splitting mutation

This variant considers an additional tweak besides the previous one. A membership function may split into two membership functions (e.g., one triangle in Figure 2 splits into two). In addition to the previously mentioned parameters, here we have used another one, which is $\mathsf{MembershipSplitProb}$ , to control the probability of choosing function shape change or input member function splitting.

Neighborhood Generation with Gaussian input membership set splitting mutation

In this case, we have used only the Gaussian input membership function in our solution. All the hyperparameters mentioned in the previous two cases are used.

5.4 Genetic Algorithm

Unlike Hill Climbing, the Genetic Algorithm (GA) starts with a number (say, $\mathsf{PopSize}$ ) of individuals. We use the same evaluation function shown in Equation 2 for fitness evaluation in our GA approach. To introduce variations in the population, GA breeds a new population of children, selects individuals from the old population, and tweaks them to breed new individuals. To keep the footprints of both populations, it joins the parent and children populations to form a new generation of population. In our implementation, we have fixed $\mathsf{PopSize}$ equal to $30$ .

5.5 Tweaking Operations

5.5.1 Crossover

The crossover operation mixes multiple individuals (typically $2$ ) and matches to form the children. There are $3$ classical ways of doing a crossover. In our implementation, we have used uniform crossover. In the context of our representation, our crossover operation marches down all the membership functions and to combine them swaps individual functions if a coin toss comes up head with probability $p$ . To use crossover on the Genetic Algorithm, both individuals should be of the same size.

5.5.2 Mutation

Our mutation operation scans each membership function and randomly tweaks the function shape with a certain probability. To exploit both exploitation and exploration, our mutation operation uses a Gaussian distribution with a certain mean and variance. Unlike Hill Climbing, mutation operation here does not increase/decrease the number of membership functions.

5.6 Joining

The Genetic Algorithm differs in how parent and child populations are joined. In our Genetic Algorithm procedure, we have experimented with both (P, P) and (P+P) evolution strategies, where $P$ is the $\mathsf{PopSize}$ .

5.7 Experiment Process

In the experimental analysis, first, we choose the best variants among all variants of Hill Climbing and the Genetic Algorithm. Then we measure the performance of our image enhancement method with respect to common metrics of image enhancements, and finally, we compare our results with one of the simplest methods of image enhancement — histogram equalization.

5.7.1 Variant Selection

The following steps are used to choose the best variants:

1.

Run one variant on each image. The variant is applied on one image a total of $\mathsf{NumofTest}$ times. Each time (one stochastic optimization) is run for $\mathsf{PerRunTime}$ seconds. This time is kept the same for all variants. Also, mutation and population initialization-related parameters are kept the same for all variants.
2.

Measure the average rate of fitness improvement over the number of generations achieved in given $\mathsf{PerRunTime}$ . Then the average is computed for $\mathsf{NumofTest}$ times.
3.

We have compared the metric achieved in step $2$ for all five variants and then selected the best two variants. The higher the value of this metric demonstrates better performance.

The value of experiment controlling parameters is written in Table 1.

Hyperparameter name	Value
$\mathsf{NumofTest}$	$5$
$\mathsf{PerRunTime}$	$120$
$\mathsf{ChangeProb}$	$0.5$
$\mathsf{MutateMu}$	$3$
$\mathsf{MutateSigma}$	$2$
$\mathsf{MembershipSplitProb}$	$0.1$

Table 1: The value of hyperparameters

6 Experimental Results

In this section, we evaluate our proposed approach and present the results of the application of our method on different images. We have conducted our experiments on several images used in [3] and some other images (both color, gray-scale, and text images) collected from different sources from the Internet. In particular, we have experimented with a total of $18$ images.

6.1 Experimental Setup and Environment

To evaluate the efficacy of our approach, we have implemented a prototype in python. We have used the python evolutionary computation framework, deap¹¹1https://deap.readthedocs.io/en/master/. To implement fuzzy logic, we have used python fuzzy logic toolbox, skfuzzy²²2https://pythonhosted.org/scikit-fuzzy/.

All variants of our algorithm are run on a Intel(R) Core(TM) i $3$ CPU M $370@2.40$ GHz processor with $6$ GB RAM running ubuntu $18.04$ and python version $3.6.8$ .

Our experiment code can be found in this link, https://bitbucket.org/mahi045/image-enhancement/.

6.2 Results

Here, we visually present outputs for a limited number of images due to our page limitation. Figures 4, 5, 6, 7 show output images generated by our technique against the reference input images. As an outcome of our technique, the enhanced images are more clear and natural (Figures 4, 6, 7), the edges are sharper(Figures 5, 7), the histogram is expanded, and contrast between black and white portion is increased. On the contrary, some black portions (Figures 5, 6) are unnecessarily darkened.

6.3 Survey

6.3.1 Variant Selection to Select Survey Images

First we have grouped the variants according to used metaheuristic algorithms, namely Hill Climbing and Genetic algorithm. Then according to the method described in Subsection 5.7, we have ranked the variants among two groups. Based on this the selected variants from the two groups are,

1.

Hill Climbing with simple neighborhood generation (Subsection 5.3.1)
2.

Genetic Algorithm with (P, P) strategy (Subsection 5.6)

6.3.2 Survey Preparation

We have prepared an anonymous survey³³3https://forms.gle/9wuLKMYBShjyAaK59 (currently not taking any responses) to get a quantitative opinion on our enhanced images. In the survey, we have placed $36$ enhanced images (produced by our technique) side-by-side with their original image. The first $18$ images are from Hill Climbing with neighborhood generation with split mutation (see Section 5.3), and the second $18$ images are the same but from the Genetic Algorithm with (P, P) strategy (see Section 5.6). Reviewer needs to mark the enhanced image on a scale of $1-9$ . As the reference, the mark of the original image was $5$ ; therefore, a mark of $>5$ by the reviewer would mean an enhanced image, while $<5$ will mean image degradation. A mark of $5$ would mean the processed image is the same as the original visually (a subjective judgment). The prepared survey form can be found at the following link: https://forms.gle/9wuLKMYBShjyAaK59.

6.3.3 Survey Response & Observations

We have received a total of $67$ responses. From the responses, we have filtered responses with suspicious patterns like all equal marks or very high or very low marks. This filtering caused two responses to be filtered. From the remaining $65$ responses, it seems method $2$ is better (mean score $5.35$ ) than method $1$ (mean score $4.62$ ). Moreover, the method $2$ improves the original image (score $>5$ ).

From Figure 3, we can see the mean of (over images) quantitative responses of people (survey participants). Each data point in Figure 3(b) is the mean response of one participant. Figure 3(a) shows the mean (green triangle), and median (orange horizontal line) of the responses. The red points are outliers according to the $1.5$ *IQR rule [21].

From Figure 3(a), method $2$ shows a better mean score than method $1$ ’s mean score (mean over participants). From Figure 3(b), we can see that method $1$ has a higher chance of degrading the image (dense below score $5$ ), although method $2$ has achieved the image with the lowest score according to one participant.

7 Conclusion

We have applied meta-heuristics to find a good image-specific fuzzy logic-based transformation function in this work. We have employed a quality function from some previous works. As image quality is subjective, to assess our method’s success, we have conducted a survey to gain subjective opinions on the visual quality of enhancement and reported the result. From the assessment, we have seen that one variant among the five gives us a visual improvement on average.

For future work, more experiments can be done by associating more image processing operations (e.g., gamma transformation) can be added in the processing pipeline. One limitation of our approach (according to visual observation) is that it darkens the image. Adding other processing operations with automatic parameter tuning through meta-heuristics seems promising for further improvement.

References

[1] Ye, Zhiwei, Mingwei Wang, Zhengbing Hu, and Wei Liu. ”An adaptive image enhancement technique by combining cuckoo search and particle swarm optimization algorithm.” Computational intelligence and neuroscience 2015 (2015).
[2] Hoseini, Pourya, and Mahrokh G. Shayesteh. ”Efficient contrast enhancement of images using hybrid ant colony optimisation, genetic algorithm, and simulated annealing.” Digital Signal Processing 23.3 (2013): 879-893.
[3] A. Draa and A. Bouaziz. (2014). An artificial bee colony algorithm for image contrast enhancement. Swarm and Evolutionary Computation. 16. 10.1016/j.swevo.2014.01.003.
[4] Sara Hashemi, Soheila Kiani, Navid Noroozi, and Mohsen Ebrahimi Moghaddam. 2010. An image contrast enhancement method based on genetic algorithm. Pattern Recogn. Lett. 31, 13 (October 2010), 1816–1824.
[5] Fernandez, Alberto, Victoria Lopez, Maria Jose del Jesus, and Francisco Herrera. ”Revisiting evolutionary fuzzy systems: Taxonomy, applications, new trends and challenges.” Knowledge-Based Systems 80 (2015): 109-121.
[6] Rafael C. Gonzalez and Richard E. Woods. 2006. Digital Image Processing (3rd Edition). Prentice-Hall, Inc., USA.
[7] Sean Luke, 2013, Essentials of Metaheuristics, Lulu, second edition.
[8] J. A. Stark, ”Adaptive image contrast enhancement using generalizations of histogram equalization,” in IEEE Transactions on Image Processing, vol. 9, no. 5, pp. 889-896, May 2000. doi: 10.1109/83.841534
[9] P. Shanmugavadivu, K. Balasubramanian, Particle swarm optimized multi-objective histogram equalization for image enhancement, Optics & Laser Technology, Volume 57, 2014, Pages 243-251, ISSN 0030-3992, https://doi.org/10.1016/j.optlastec.2013.07.013.
[10] Oloyede, M., Hancke, G., Myburgh, H. et al. A new evaluation function for face image enhancement in unconstrained environments using metaheuristic algorithms. J Image Video Proc. 2019, 27 (2019).https://doi.org/10.1186/s13640-019-0418-7
[11] Youlian Zhu, Cheng Huang, An Adaptive Histogram Equalization Algorithm on the Image Gray Level Mapping, Physics Procedia, Volume 25, 2012, Pages 601-608, ISSN 1875-3892, https://doi.org/10.1016/j.phpro.2012.03.132.
[12] Joshi, Sandeep & Kumar, Samrudh. (2018). Image contrast enhancement using fuzzy logic.
[13] C. Munteanu and A. Rosa, ”Gray-scale image enhancement as an automatic process driven by evolution,” in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 34, no. 2, pp. 1292-1298, April 2004. doi: 10.1109/TSMCB.2003.818533
[14] Samanta, Sourav & Mukherjee, Amartya & Ashour, Amira S. & Dey, Nilanjan & Tavares, Joao & ABDESSALEMKARAA, WAHIBA & Taiar, Redha & Azar, Ahmad & Hassanien, Aboul Ella. (2018). Log Transform Based Optimal Image Enhancement Using Firefly Algorithm for Autonomous Mini Unmanned Aerial Vehicle: An Application of Aerial Photography. International Journal of Image and Graphics. 18. 1850019. 10.1142/S0219467818500195.
[15] Bhandari, A. & Maurya, Shubham. (2019). Cuckoo search algorithm-based brightness preserving histogram scheme for low-contrast image enhancement. Soft Computing. 10.1007/s00500-019-03992-7.
[16] Coelho, Leandro & Sauer, João & Rudek, Marcelo. (2009). Differential evolution optimization combined with chaotic sequences for image contrast enhancement. Chaos, Solitons & Fractals. 42. 522-529. 10.1016/j.chaos.2009.01.012.
[17] Braik, Malik & Sheta, Alaa & Ayesh, Aladdin. (2007). Particle swarm optimisation enhancement approach for improving image quality. International Journal of Innovative Computing and Applications. 1. 10.1504/IJICA.2007.016795.
[18] Joshi, Sandeep & Kumar, Samrudh. (2018). Image contrast enhancement using fuzzy logic.
[19] Haußecker, Horst & Tizhoosh, Hamid. (1999). Fuzzy Image Processing. 10.1016/B978-012379777-3/50017-0.
[20] A. S. Tarawneh, A. B. Hassanat, I. Elkhadiri, D. Chetverikov and K. Almohammadi, ”Automatic Gamma Correction Based on Root-Mean-Square-Error Maximization,” 2020 International Conference on Computing and Information Technology (ICCIT-1441), 2020, pp. 1-5, doi: 10.1109/ICCIT-144147971.2020.9213752.
[21] Rousseeuw, P.J. and Hubert, M., 2011. Robust statistics for outlier detection. Wiley interdisciplinary reviews: Data mining and knowledge discovery, 1(1), pp.73-79.
[22] Singh, G., & Mittal, A. (2014). Various image enhancement techniques-a critical review. International Journal of Innovation and Scientific Research, 10(2), 267-274.
[23] Singh, Ravindra Pal, and Manish Dixit. ”Histogram equalization: a strong technique for image enhancement.” International Journal of Signal Processing, Image Processing and Pattern Recognition 8, no. 8 (2015): 345-352.
[24] Ooi, C. H., Kong, N. S. P., & Ibrahim, H. (2009). Bi-histogram equalization with a plateau limit for digital image enhancement. IEEE transactions on consumer electronics, 55(4), 2072-2080.
[25] Singh, G., & Mittal, A. (2014). Various image enhancement techniques-a critical review. International Journal of Innovation and Scientific Research, 10(2), 267-274.