This paper presents a novel feature extraction method, Local Orientation Gradient XoR Patterns (LOGXoRP) for image indexing and retrieval. The LOGXoRP encodes the exclusive OR (XOR) operation between the center pixel and its surrounding neighbors of quantized orientation and gradient values, whereas the Local Binary Patterns (LBP) and the Local Gradient Patterns (LGP) encode the relationship between the gray values of center pixel and its neighbors. The authors shows that the LOGXoRP can extract effective texture (edge) features as compared to LBP and LGP. The performance of the proposed method is tested by conducting two experiments on Corel-5K and Corel-10K databases. The results of the proposed method after being investigated shows a significant improvement in terms of their evaluation measures as compared to LBP, LGP and other existing state-of-art techniques on respective databases.
Image retrieval is an active research topic in image processing and pattern recognition [1]. Initially, textbased image retrieval was used. To reduce the amount of labor required for image annotation and different interpretations of the images by different people, Content Based Image Retrieval (CBIR) was developed. However, some challenging problems that attract researchers' interests towards CBIR remain.
Feature extraction is crucial for CBIR whose effectiveness depends on the method adopted for extracting features from given images. Visual content descriptors are either global or local. A global descriptor represents the visual features of the whole image, whereas a local descriptor represents the visual features of regions or objects to describe an image. This descriptor is arranged as multidimensional feature vectors and constructs the feature database. Many methods such as Euclidean distance (L2), and L1 distance have been developed for similarity distance measurement. The selection of feature descriptors and similarity distance measures markedly affects the retrieval performance of an image retrieval system. The previously available literature on CBIR is presented in [1], [2], [3], [4], and [5].
If extracted in a perceptually oriented manner and kept semantically intact, the color composition of an image can turn out to be a powerful feature for CBIR. Furthermore, the color structure in a visual scenery is robust to noise, image degradation, changes in size, resolution and orientation. Most existing CBIR systems use various color descriptors [6-8] to retrieve relevant images (or visual multimedia material); however, their retrieval performance is usually limited particularly on large databases because of the lack off discrimination power of such color descriptors.
In Discrete Wavelet Transform (DWT) based features were used for texture analysis, such as texture classification [9], [10] and texture retrieval [11], [12]. However, DWT has limited directional (0º, 90º, and ±45º) selectivity. To address this directional limitation, Gabor Transform (GT) [13], rotated wavelet filters [14], Dual-Tree Complex
Wavelet Filters (DT-CWFs), DT rotated CWFs [15], and rotational invariant complex wavelet filters [16] have been proposed for texture image retrieval.
Recently, active researchers in image retrieval using a combination of color and texture features have been performed [17], [18]. Liu et al. [17] have integrated the color and texture features called Multi-Texton Histogram (MTH) for image retrieval. MTH integrates the advantages of the cooccurrence matrix and the histogram by representing the attribute of co-occurrence matrix using histogram. Further, they introduced the Micro-Structure Descriptor (MSD) [18] which is built based on the underlying colors in microstructures with similar edge orientation.
Mahmoudi et al. [19] proposed a shape predicated feature that relegates image edges predicated on: their orientation and the correlation between neighboring edges. They proved that their proposed scheme is efficacious and that it robustly abides translation, scaling, color, illumination, and viewing position variations. Qi et al. [20] proposed an efficacious shape description method which includes a contour-predicated shape descriptor and Zernike moments. In addition, a new feature-matching strategy was presented to compute the dissimilarity value between the feature vectors extracted from the images.
In this study, the authors have focused on pattern predicated features for image retrieval. Ojala et al. [21] proposed the Local Binary Patterns (LBP) which show better performance and less computational involution for texture relegation. The prosperity of LBP variants in terms of speed (turning of parameters is not required) and performance has been reported in many research areas such as texture relegation [21]–[26], face perception [27]−[29], object tracking [30], image retrieval [31]–[35], fingerprint matching [36], and interest point detection [37]. The rotational invariant LBP variance for texture relegation use feature distributions was proposed [24]. Guo et al. developed the Completed Local Binary Pattern (CLBP) scheme for texture relegation [25]. Furthermore, they [26] developed the cognitive framework that estimates the optimal pattern subset of interest by simultaneously considering robustness, discriminating power and representation capability of texture features.
They integrated these features with the subsisting LBP variants such as conventional LBP, rotation invariant patterns, local patterns with an an-isotropic structure, CLBP, and local ternary pattern, to derive incipient image features for texture relegation. Jun and Kim [27] proposed the Local Gradient Patterns (LGP) for face detection. Similar to other variations, such as LBP and modified census transform, the LGP representation is callous to ecumenical intensity variations. This representation is also insensitive to local intensity variations along the edge components. Xie et al. [28] proposed the Local Gabor XoR Patterns (LGXP) operator for face apperception.
LGP and LGXP proposed in [27] and [28] respectively, have incentivized us to propose Local Orientation Gradient XoR Patterns (LOGXoRP) for image retrieval. The main contributions of the proposed method are as follows:
(a) the quantization of Gabor replications for the LGXP operator is very arduous because the range of Gabor coefficients is different for each image, whereas the LOGXoRP uses a quantified gradient and orientation response with a constant range (orientation: 0°-360° and gradient: 0:255).
(b) LOGXoRP can extract efficacious texture (edge) features compared with LBP and LGP. The performance of the proposed method was tested by conducting two experiments on benchmark Corel-5K and Corel-10K databases
The organization of the paper as follows: In section 3, a brief review of image retrieval and cognate work is given. Section 3 presents a concise review of local patterns (LBP, LGP, and LOGXORP). Section 4, presents the feature extraction and analysis. Experimental results and discussions are given in section 5. Predicated on above work, at last conclusions have been summarized.
The LBP operator was introduced by Ojala et al. [21] for texture classification. Given a center pixel in an image, the LBP value is computed by comparing its gray scale value with its neighbors as shown in Figure 1 based on Equations (1) and (2).
where, gc is the gray value of the center pixel, gi is the gray value of its neighbors, P is the number of neighbors and R is the radius of the neighborhood.
Figure 1. Calculation of LBP
Jun and Kim [27] have proposed the LGP for face detection. Given a center pixel in an image, the LGP value is computed by comparing its gray scale value with its neighbors based on Equations (3) and (4).
More details of LGP can be found in [27].
The proposed LGP is virtually homogeneous to the consummated LBP magnitude (CLBP_M) [25]. The only distinction between these two features is that, the LGP calculates the threshold (Th) from the mean/average of Local Difference Operator (LDO) for a given pattern, whereas the CLBP_M calculates from the mean/average of the entire image LDO.
The conception of LBP [21], LGP [27] and LGXP [28] has been adopted to define the LOGXoRP. Given a center pixel in an image, the gradients (P=8) are calculated as,
where, {g1 ,g2 ,g3 ,g4 ,g5 ,g6 ,g7 ,g8 }|P=8 are the gray values of neighbors for a given center pixel, gc .
The orientation and gradient values are calculated as follows,
The Local Gradient XOR Patterns (LGXORP) and Local Orientation XOR Patterns (LOXORP) are calculated as:
where, Q(x) denotes the quantized value of x and represents the exclusive OR (XOR) operation.
Similarly, orientation and gradient patterns are calculated utilizing diagonal directions additionally.
For the local pattern with P neighborhoods, 2P coalescence of local binary patterns are possible, resulting in a feature vector length of 2P. The computational cost of this feature vector is very high. In order to reduce the computational cost, we consider the uniform patterns [24]. The uniform pattern refers to the uniform appearance pattern that has inhibited discontinuities in the circular binary representation. In this study, the patterns which have less than or identically tantamount to two discontinuities in the circular binary representation are referred to as the uniform patterns and remaining patterns are referred to as non-uniform. Thus, the distinct uniform patterns for a given query image would be p(p-1)+2. The possible uniform patterns for P=8 can be visually perceived in [24].
After identifying the local pattern, PTN (LBP or LGP or LOGXoRP (LGXoRP+LOXoRP)), the whole image is represented by building a histogram utilizing Equation (12)
where, N1xN2 represents the size of input image.
The feature extraction and analysis of the proposed method are given in the following subsections.
Figure 2 illustrates the flowchart of the proposed feature extraction method and algorithm for the same is given bellow:
Input: Image;
Output: Feature vector
1. Load the image, and convert it into gray scale.
2. Calculate the gradient and orientation values.
3. Quantize the gradient and orientation values.
4. Calculate the XoR patterns for gradients and orientations.
5. Calculate the histograms.
6. Construct the feature vector by concatenating the histograms.
Figure 2. Proposed Feature Extraction Method
Figure 3 shows the feature maps obtained by applying LBP, LGP and LOGXoRP operators on a reference image. The reference image has been culled since it provides the results which are visibly comprehensible to differentiate the efficacy of these approaches. It can be observed that, the LOGXoRP operator is able to extract a detailed-edge information as compared to LBP and LGP operators.
Figure 3. Feature Maps using Various Feature Extraction Methods
Figure 4 illustrates the comparison between the sundry features on two sample images which are culled from the different categories of the Corel-10K database. From Figure 4, it is pellucid that the proposed method (LOGXoRP) is able to differentiate the two sample images as compared to LBP and LGP.
Figure 4. Comparison of Various Features on two Sample Images from Corel-10K Database
Feature vector of query image Q is represented as fQ = (fQ1 ,fQ2 ,......fQLg) obtained after the feature extraction. Similarly each image in the database is represented with feature vector
The goal is to select n best images that resemble the query image. This involves cull of n top matched images by measuring the distance between query image and image in the database |DB|
In this study, four types of homogeneous distance metrics are used and they are shown below,
L1 or Manhattan distance measure,
L2 or Euclidean distance measure,
Canberra distance measure,
d1 distance measure,
where fDB|| is the ith feature of jth image in the database |DB|.
The performance of the proposed method is quantified in terms of average precision or Average Retrieval Precision (ARP), average recall and Average Retrieval Rate (ARR) as shown below:
For the query image Iq , the precision is defined as follows:
where, n indicates the number of retrieved images, |DB| is the size of image database, (x) is the category of 'x', Rank (Ii ,Iq ) returns the rank of image Ii (for the query image Iq) for all images of |DB| and
Recall is defined as,
where, NG is the number of relevant images in the database.
The average precision for the jth similarity category of the reference image database is computed using Equation (20).
Finally, the total Average Retrieval Precision (ARP), and Average Retrieval Rate (ARR) for the whole reference image database are computed using Equations (21) and (22) respectively.
In this study, the authors have used the Corel database for experimentation. The Corel database [38] comprises numerous images of sundry contents ranging from animals, and outdoor sports to natural images. Domain professionals classify these images into different categories of size 100. Some researchers believe that the Corel database meets all the requisites for evaluating an image retrieval system, because of its large size and heterogeneous content. For Corel-10K and Corel-5K databases, 10000 and 5000 images accumulated that consist of 100 and 50 different categories, respectively, are collected. Each category has 100 images with a size of 126×187 or 187×126.
In this experiment, the Corel-5K database was used. The performance of the proposed method was measured in terms of ARP and ARR. The results were considered satisfactory, if the average values of the precision and recall were high.
Table 1 represents the average values of precision and recall of the proposed method (LOGXoRP) and other existing methods (CS_LBP, LEPSEG, LEPINV, BLK_LBP, LBP, LGP and DLEP) on the Corel-5K and Corel-10K databases. Figures 5 (a) and (b) show the performance of various methods in terms of ARP and ARR on the Corel- 5K database. LOGXoRP evidently outperforms the other existing methods in terms of ARP and ARR on the Corel- 5K database [Table 1 and Figures 5 (a) and (b)]. The performance of the proposed method is analyzed using different quantization levels of gradient and orientation responses (Table 2). Furthermore, the four quantization levels showed a better superior performance than other quantization levels. Table 3 presents the performance of the proposed method with various distance measures and reveals that d distance 1 measure shows a better performance than other measures. Figure 6 shows the query results of the proposed method on the Corel-5K database (top left image is the query image).
Figure 5. Comparison of LOGXoRP with Other Existing Methods in terms of ARP and ARR: (a), (b) on Corel-5K and (c), (d) on Corel 10K Database
Figure 6. Query Results of Proposed Method on Corel-5K Database (Top Left Image is the Query Image)
Table 1. Performance of Various Methods in Terms of Precision and Recall on Corel-5k And Corel-10k Databases
Table 2. Performance of the Proposed Method with Various Quantization Levels in Terms of Precision and Recall on Corel-5K Database
Table 3. Performance of the Proposed Method with Various Distance Measures in Terms of Precision and Recall on Corel-5K Database
In this experiment, Corel-10K database was used. The performance of the proposed method was quantified in terms of precision, recall, Average Retrieval Precision (ARP) and Average Retrieval Rate (ARR) as shown in Equations (18)–(22).
Figures 5 (c) and (d) show the performance of sundry methods in terms of ARP and ARR on the Corel–10K database. From Table 1 and Figures 5 (c) and (d), it is pellucid that the LOGXoRP outperforms the other subsisting methods in terms of ARP and APR on the Corel-10K database. The performance of the proposed method was analyzed with different quantization levels of gradient and orientation replications as shown in Table 4. From Table 4, it was observed that the four quantization levels are exhibiting better performance as compared to other quantization levels. Table 5 illustrates the performance of the proposed method with sundry distance measures and observed that the d distance measure shows better performance as 1 compared to other measures. Figure 7 shows the query results of the proposed method on the Corel-10K database (top left image is the query image).
Figure 7. Query Results of Proposed Method on Corel- 10K Database (Top Left Image is the Query Image
Table 4. Performance of the Proposed Method with Various Quantization Levels in Terms of Precision and Recall on Corel-10K Database
Table 5. Performance of the Proposed Method With Various Distance Measures in Terms of Precision and Recall on Corel-10K Database
Table 6 shows the feature vector length for a given query image utilizing sundry methods. From the above experiments and Table 6, the observations are made as follows:
Table 6. Feature Vector Length of Query Image using Various
In this study, a new image indexing and retrieval algorithm was proposed using Local Orientation Gradient XoR Patterns (LOGXoRP). The LOGXoRP encode the images based on the basis of the XoR operation between the center pixel and its neighbors of quantized orientation and gradient replications.
The performance amendment of the proposed method was compared with that of the LBP, and LGP on grayscale images, and has been detailed below.
Due to the efficacy of the proposed method, it can be additionally opportune for other pattern apperception applications such as face apperception, dactylogram apperception, etc.
This work was supported by the Canada Research Chair Program, the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant.
Given below are the abbreviations used in the analysis of result:
LBP - Local Binary Patterns [21]
DLEP - Directional Local Extrema Patterns [33]
LEPSEG - Local Edge Patterns for Segmentation [35]
LEPINV - Rotational Invariant Local Edge Patterns [35]
BLK_LBP - Block Based LBP [34]
CS_LBP - Center Symmetric Local Binary Patterns [37]
LGP - Local gradient patterns [27]
LGXP - Local Gabor XoR patterns [28]
LGXoRP - Local Gradient XoR Patterns
LOXoRP - Local Orientation XOR Patterns
LOGXoRP - LGXoRP+ LOXoRP
LBPu2 - LBP with uniform patterns (same is applicable for other patterns.
