Scene Oriented Classification of Blurry and Noisy Images Using SVM with Fuzzy C Mean Clustering

Image classification is a challenging task in image processing especially in the case of blurry and noisy images. In this work, we present an extension of scene oriented hierarchical classification of blurry and noisy images using Support Vector Machine (SVM) and Fuzzy C-Mean. Generally, a system for scene-oriented classification of blurry and noisy images attempts to simulate major features of the human visual observation. These approaches are based on three strategies such as Global pathway for extracting essential signature of image, local pathway for extracting local features, and then outcome of both global and local phase are combined and define feature vector and clustered using Monte Carlo approach. Afterwards, these clustered results are fed to a SOTA Algorithm (combination of self organizing map and hierarchical clustering) for final classification. But in these approaches, combination of self organizing map and hierarchical clustering has the problem in terms of accuracy and computation time of classification, especially when used large dataset for classification. To overcome this problem, we propose a combination of Support Vector Machine (SVM) and Fuzzy C-mean. Our proposed approach provides better result in terms of accuracy, especially when used with large dataset. The proposed method is computationally efficient because fuzzy c-mean clustering is faster and less time consuming as compared to hierarchical clustering.


INTRODUCTION
Classification is an information processing task in which images are categorized into several groups. In real world thousands of images are generated daily, which implies that the requirements to classify for access them by an easy and faster way. Image classification is one of the complex areas of image processing. Several methods have been proposed by researchers in the literature to classify images and they provide good classification result but they fail to provide satisfactory classification result, especially when the image contains blurry and noisy content. Image classification means to separate an image on the basis of its content. In real world applications, the main classification difficulty is raised due to the visual ambiguity generated due to noisy content. Even though a lot of research has been directed to deal with classification of visual information, most of the techniques only address the fundamental problem of classification. That means only classify the normal images and not address the classification problems of blurry and noisy images.
Generally real world images are degraded due to noise and blur. For example, images related to natural disaster, war; crowds as well as non-professionally generated images are generally noisy. Such important yet degraded material creates additional hurdles to an already difficult classification problem. In fact, the traditional approach is applied to this type of content is less strong. Techniques used previously are saliency detection [2] or simplex global layout extraction [3] does not provide satisfactory classification results, when applied to noisy or blurred content. Eearlier used classification method [1] gives better results for blurry and noisy images but with small data set only. When used with large data set, it is quiet insufficient in terms of accuracy.
Artificial intelligence combines different fields of science such as computer science, neuroscience, psychology etc. Also new research is made by combination of more than one area. For example, advances in cognitive psychology have led us to believe that our brain processes information in a "top-down" manner rather than doing it in a "bottom-up" manner. It has also been proposed that the brain develops models and templates for information it encounters, i.e. it has a model for a building, for a game of cricket and so on for almost all things, which we can observe and feel. In case it encounters somewhat new it tries to correlate to something it already knows and at times it can form a new model as well. Furthermore, these models and conceptions of things keep evolving with time and with our interaction with environment. Capturing this huge collection of models, concepts and information in a machine is a very big challenge.
One of the main characteristics of our brain is the ability to differentiate between things, to identify and to link with our previous knowledge. This ability to recognize and interpret the environment around us is in principle the basis for any higher level of processing that we do. this concept is similar to the clustering, segmentation and classification in subject of artificial intelligence. In artificial intelligence, learning algorithm is inspired by the working of human brain. These algorithms provide us with a methodology to locate parameters which would be able to recognize and classify different objects in a given input which is helpful to solve the problem of pattern reorganization, image classification, content based retrieval system etc. The objective of this paper is to provide a significant approach to solve the classification problem of blurry and noisy images. The proposed approach has been inspired by remarkable ability of the human visual system to classify visual information under extreme conditions [11]. A key aspect of human visual perception is the ability to identify the meaning of a complex scene very quickly, even in the presence of degraded visual content such as blur or noise, and after a quick glance, solving a difficult classification problem.
In the SOTA algorithm due to hierarchical clustering, it takes more time to classify the images and self organizing map is an unsupervised learning algorithm in which sometimes maximally-separable clusters in spectral space may not match the perception of the important classes and which make more error rate. In the proposed approach, we used Fuzzy C mean clustering for grouping of images with similar feature vector which takes less time in comparison to hierarchical clustering. Finally, these images classified using Support Vector Machine (SVM).
The reaming part of this paper is organized as follows: Section II described review of literature. Proposed methodology is described in section III. Section IV elaborates the results analysis while comparison of results will be drawn in section V. Conclusion and future work has been given in last section. J a n u a r y 1 1 , 2 014

Review of Literature
The earliest work on classification of blurry and noisy images is done by Dong, Jiang Su, and Ebroul Izquierdo. They proposed SOTA algorithm for classification of blurry and noisy images. It performs feature clustering from the combined global and local features. The system learns to use feature from both the global and local pathways for classification [1].
A lot of approaches have been proposed for image classification but restricted to classification of normal images and not for classification of blurry and noisy images. These methods have not provided satisfactory results with blurry and noisy images. However, a lot of solutions are available for resolving the problem of image classification. For example, Conventional techniques as saliency detection [2] and simplex global layout extraction [3] have not provide satisfactory classification results when applied to noisy or blurry content. The main task in classification of blurry and noisy images is restoration of the image into its normal form after that classify the images through its feature vector. Fuzzy logic based filtering technique [4] is used for restoration of images corrupted by impulse noise. This can perform in two phases first the detection of noisy pixels at all locations in the image by fuzzy knowledge and recursive median filter on the corrupted pixels which can remove the impulse noise. Henry Leung [5] proposed a protocol for the classification of blurry images. It performs the tasks of blind image restoration and classification called blind image fusion. This technique uses the no negativity and support constraints recursive inverse filtering (NAS-RIF) algorithm for blind image restoration and the Markov Random Field (MRF) based fusion method for classification by Schistad Solberg etc [5]. Support vector machine is used for hierarchical classification of images [6]. In this method image database is classified with hierarchical semantics into day, night, and sunrise/sunset; close-up and non close-up; indoor and outdoor, city and landscape classes. But in the case of blurry and noisy images, its performance is not efficient. Image classification made easy, if there is a general framework for the recognition of complex visual scenes [7] [8]. But, in this type of method object detection is important. The object detection is highly depends on object locations to acquire accurate object segmentation, which is fundamentally harder than the actual classification problem itself. Global image classification approaches consider the input information as a whole and use low-dimensional representations that efficiently summarize the statistical properties of image primitives such as color, texture, and edges. These low-dimensional representations are called image descriptors. These descriptors usually capture spatial arrangements of self-similarities [9]. By analyzing whole scenes, related approaches do not have to deal with the variations of low-level noise in small isolated areas of the images. In all these cases, the challenge is to find a compact representation and integrated natural images. A new technique is proposed for the classification of indoor and outdoor images which is used edge analysis. Feature extraction is an important phase of the image classification [12]. There are a lot of techniques are available in the literature for image feature extraction and we have used Principal Component Analysis (PCA) in our proposed work.
The proposed method has been inspired by the human visual perception system tends to deal with uncertain content in a good way with little effort in the presence of blur and noise [11]. It means capture principal component in first glance especially with the blurry and noise images. Image classification methods are generally of two types: supervised and unsupervised. In our proposed work, supervised classification has been used instead of unsupervised classification technique.

Image Classification Method
Image classification is one of the important and complex processes in image processing. There are several image classification methods. The two main image classification methods are supervised classification and unsupervised classification [10].

 Supervised classification:
Supervised classification requires prior information before testing process and it must be collect by analyst. In this analysis, identifies representative training sites for each informational class and algorithm also generates decision boundaries. Generally, supervised classification approaches are parallelepiped, means least amount of distance to mean and maximum likelihood. The steps in supervised classification approach are as follows: The advantages of supervised classification are operator can detect errors and often remedy them. The disadvantage of supervised classification approach are training can be time consuming and costly and also training data selected by the analyst, may not be represent all conditions encountered throughout the image. The supervised classification is also prone to human error [10].

Select Training Fields
Extract Signature of images Classify images Final Classification J a n u a r y 1 1 , 2 014  Unsupervised classification: In unsupervised classification, prior information is not needed. It does not require human annotation, it is fully automated. This algorithm identifies clusters in data and also analyst labels clusters.

Figure 3: Unsupervised Classification
The steps in unsupervised classification are as follows: The advantages of unsupervised methods are less time consuming and minimize human errors and also no extensive or detailed a priori knowledge of the region is required. The disadvantage of this method are maximally-separable clusters in spectral space may not match our perception of the important classes on the landscape and also limited control over the "menu" of classes [10].

Fuzzy C-mean Clustering
In our proposed work, we have used (unsupervised) fuzzy C-means clustering (FCM) algorithm which is an iterative procedure. Given M input feature vector m = 1... M, the number of clusters C where 2 C < M, and the fuzzy weighting exponent z, 1 < z < 1, initialize the fuzzy membership function (0) which is an entry of a C M matrix .
The following steps are iterated for increments of b: 1) Calculate the fuzzy cluster centers with = 2) Update U with = Where and is any inner product induced norm.
3) Compare with is in a convenient matrix norm.

If
Stop, otherwise return to step 1 The value of the weighting exponent z determines the fuzziness of the clustering decision. A smaller value of z, i. e. z close to unity, will lead to a zero/one hard decision membership function, while a larger z corresponds to a fuzzier output.
In this work although the FCM based clustering assigns disconnected image segments to the same cluster, we consider disconnected segments of the same cluster as different segments. Regions near the boundary are not considered for further processing as they are often not completely available.

Run Clustering Classify Classes
Extract Feature Perform final classification J a n u a r y 1 1 , 2 014

Support Vector Machine
Support vector machine is a supervised learning model with associated learning algorithms that analyse data. SVM is commonly used to recognize patterns, used for classification. The basic SVM is a binary classifier. We are also use SVM for solve multi-class problem. The "one against one" and the "one against all" are the two most popular strategies for multiclass problem.
Multi-class problems with SVMs (where n > 2 classes) are solved by using combining many binary SVM classifier with either one-against-all or one-against-one scheme. In the 1-against-all schemes, n classifiers are placed in parallel, one for each class. The nth classifier constructs a hyper plane between class n and the n-1 other classes. A new input can be classified by choosing the maximum applied to the outputs of n SVMs. Weston and Watkins [13] propose two extensions to the SVM method of pattern recognition to solve n-class problems in one step, which do not use a combination of binary classification rules. Here, we use the one-against-all SVM-based classifiers for final classification.

Proposed Methodology
Image classification means grouping the images into semantically meaningful class. Image classification is an already a difficult problem as well noise and blur make an additional barrier into an already a difficult problem. We have proposed an extension of "scene oriented hierarchical classification of blurry and noise images" [1]. In the proposed algorithm we are replacing SOTA (combination of self organising map and fuzzy clustering) with Support Vector Machine (SVM) and fuzzy-C-Mean (FCM) clustering respectively. Figure 4 illustrates the flow diagram of proposed method.
In the SOTA algorithm self organizing map is an unsupervised learning algorithm in which sometimes maximallyseparable clusters in spectral space may not match our perception of the important classes which make more error rate. In the proposed approach, we have used Fuzzy C mean clustering for grouping of images with similar feature vector which takes less time as compare to hierarchical clustering. And after that these images classified using multi-class support vector machine.

Figure 4: Flow diagram of proposed method of image classification
The steps of the proposed algorithm are described here.
1. Import data set of blurry and noisy images.   Final Classification J a n u a r y 1 1 , 2 014 6. Perform final Classification using combination of SVM and Fuzzy C Mean Clustering.
Proposed algorithm for final classification of blurry and noisy images is provides better results in term of accuracy when we are applied proposed method in large dataset. In the proposed algorithm the task of classification is done in two steps first is Training and second is Testing.

Result Analysis
Image classification is an operation in which we are separate similar images into small group from a collection of images. Image Classification means to assign each test image to one of a number of categories. Performance of the algorithm is measured using the confusion matrix; a confusion matrix [14] holds the information about actual and expected classifications done by a classification system. Performance of such systems is generally evaluated by using the data of the matrix. The overall performance rates are measured by the average value of the diagonal entries of the confusion matrix. Here we are evaluated the performance of the system on the basis of the accuracy means how many images are correctly classified from the total no of images classified. Experimental data sets are split randomly into two separate sets of images; the some images selected as training images and remaining images use as testing images. Description of dataset of images is shown in Table I. From each data set, we have selected some images for training and perform training after that we test the performance of the system by classify the remaining images using this trained system. Table  I describe the information of data set, number of images selected for training and number of images selected for performance evaluation of the proposed system.

A. Data set description
In the proposed algorithm, we have used four data sets to validate the accuracy of proposed algorithm. These data sets are Sport dataset, outdoor dataset, residence dataset, collective dataset. The experiment has been performed by Li and Fei-Fei on the data set for sport category [8], the sport event data set covers categories of badminton, bocce, and croquet, polo, climbing, rowing, sailing, and snowboarding. The residence data set includes bedroom, kitchen, living room, house, and office. The outdoor data set contains highway, buildings, forest etc. The outdoor data set includes 2688 images classified as eight categories: coasts (360), forest (328), mountain (374), countryside (410), highway (260), city (308), buildings (356), and streets (292). The number in the bracket shows the number of images in each category. The average size of each one image is 250x250 pixels. The collective data set have 13 categories and is only in Gray scale. In this data 2688 images (eight categories) of the outdoor data set plus house (241), bedroom (174), kitchen (151), living room (289), and office (216). The average size of each image is approximately 250x300 pixels. We perform the experiment on the every data set with variety of noise and test the performance of the proposed algorithm.   In figure 5 shows some representative images from the Collective image data set. To validate the system, we have applied the proposed method on the collective data set. In collective data set 13 category of variety of classes such as bedroom, industrial, kitchen, forest, living, room etc. At the Training stage, we have randomly selected 100 images from each category and perform training phase and decide the number of classes before the classification. After that in the training phase, we perform final classification on the remaining images.

B. Result Comparison and Performance Evaluation
It is always difficult to classify blurry and noisy images into semantic classes. Generally, performance of the classification method has been evaluated on the basis of accuracy. In this section performance evaluation has been carried out on the basis of the accuracy of the method which means correctly classified images. Here we are evaluated the accuracy of the classification results using the confusion matrix. We have employed both proposed and SOTA method on the collective data set with the addition of noise and finally compared the results of proposed method with existing method. Table II shown the category wise comparative results of proposed and existing method.  J a n u a r y 1 1 , 2 014 The table II shows the comparison of the accuracy gain of the proposed algorithm and SOTA algorithm [1]. Table II shows the accuracy of the each class individually in both methods in a comparative manner. The proposed algorithm provides higher accuracy due to supervised classification in comparison to the existing methods [1]. Figure 6 depicts the results graphically. In the fig.6, it is clearly show that the accuracy of the proposed algorithm is better than the existing method.

Fig 6: Comparative graph of results in collective dataset
Our proposed method classifies the images with emphasis to classify blurry and noisy images. The proposed work shall prove its worth when classify the blurry and noisy images in efficient manner. That means the performance of the proposed should be evaluated on the image with variety of noises. We introduce various kinds of noises in the image sets and evaluate the performance of the proposed algorithm and compare the results with existing method under the same noise condition. The various noises used in the experiment are: Gaussian noise, Poisson noise, Multiplicative noise, Salt and pepper and combination of these. The results clearly depicted that our proposed method perform efficiently under Gaussian white noise, Poisson noise, and multiplicative noise.
We are applying proposed method in the outdoor data set and collective dataset under variety of noise and compare the result with existing method. Table III show the comparative result of Outdoor dataset.  Table III shows the results of outdoor data set under various noises in both the proposed and existing methods.
Table III clearly shows that system achieve better results in outdoor data set with Poisson noise with comparison to the data set with other noises. If images contains Salt and Pepper noise system gives worst results. Even in the Mixture noise it gives better results because in the mixture noise other noises reduce the complexity of Salt and Pepper noise. But our proposed algorithm gives better results with compare to the existing method [1]. Fig 7 depicts the results graphically. For testing the performance of the proposed system we are also applied proposed method in the collective dataset with various noises and calculate the accuracy of the proposed method and compare these results with the existing method [1] results. Table IV show the comparative results on the noisy collective dataset. In the Table IV we are depicts the comparative results of proposed and existing method [1] with addition to various noises in collective dataset. And it is clearly shown that proposed method gives better results with compare to the existing method [1]. Proposed system achieves better result under the Poisson noise with compare to the Gaussian noise and multiplicative noise. And and in the case of salt and peeper noise it gives worst results because salt and peeper noise is not completely removed in the Pseduo-restoration process of proposed algorithm. Fig. 8 depicts the results of Table IV graphically. The above table show that images with salt and pepper noise is hard to classify. On the other hand images with Poisson noise is classify with a good accuracy.

Figure 8: Comparison of collective dataset results with noise
In Fig 7 and fig 8 shown that proposed system is efficient in terms of accuracy with comparison to existing algorithm [1]. Even in the case of blurry and noise images proposed algorithm provide a good accuracy.