Leaves are one of the important parts in a tree. Extracting accurately the shape of a leaf is a crucial step in image-based systems. The partial or total absence of textures on leaf surface and the high color variability of leaves belonging to the same species make shape as the main recognition element. For such reasons, leaf segmentation plays a decisive role in the leaf extraction process. Even though many general segmentation methods have been proposed in the last decades, leaf segmentation presents specific challenges. In particular, a pixel- level precision is required in order to highlight fine scale boundary structures and discriminate similar global shapes. The authors propose a robust and accurate method for segmenting objects acquired under various controlled conditions. Then, they have improved the performance of the segmentation methods using preprocessing tools such as, color distance map and input strokes. Based on these methods, we can eliminate unwanted boundaries and localize the leaf object efficiently. They have implemented a Hybrid Guided Active Contour (GAC) method to measure geometric properties of leaf images, and have provided with a comparative study for various segmentation algorithms based on performance metrics. Based on experimental results, GAC provides improved performance in leaf datasets.
In our more urbanized and artificial world, the knowledge of plants that own to constitute our most immediate environment, has somehow been lost, except for a handful of specialists. But nowadays, with a definite resurgence of the idea that plant resources and diversity ought to be treasured, the will to reclaim some touch with the nature feels more and more tangible. And making it possible, for whoever feels the need, to recognize a plant species, to learn its history and properties, is as much a way to transmit a vanished knowledge, as to permit people to get a glance at the nature’s unfathomable richness. The identification of species is the first and necessary key to understand the plant environment. Botanists traditionally rely on the aspect and composition of fruits, flowers and leaves to spot species. But in the context of a widespread non-specialist oriented application, the prime use of leaves, which are potential to find almost all year long, simple to photograph, and easier to investigate from two-dimensional images, is the most sensible and widely used approach in image processing. Considering the shape of a leaf is then the evident choice to try to recognize the species. The system intends then to classify a photograph of a leaf, which should be approximately centered and vertically-oriented as shown in Figure 1, over around different tree species.
Figure 1. Segmentation Issues with Unconstrained Region-Based Active Contours
Radhakrishna Achanta, et.al. [1] have proposed a new method for generating super pixels, which is faster than the existing methods, more memory efficient, exhibits state-of-the-art frontier adherence, and improves the performance of segmentation algorithms. SLIC is similar to the approach used as a preprocessing step for depth estimation described, which was not fully explored in the context of super pixel generation.
P. Brigger, et.al. [2] propose a B-spline snake formulation based on the use of node points, a variable knot spacing, and the use of a multiresolution optimization strategy. The presented algorithms are characterized by fast execution speeds and the possibility for an intuitive user-interaction. The main advantage is that the user had to specify an initial contour for a specific application, it will add an advantage to perform the dedicated pre-processing, so as to obtain an automatic initialization. The disadvantage is that, the method has been demonstrated on a variety of problems, featuring different image modalities.
J. Canny [3] propose a new method for the fine- to-coarse integration of information from operators at different scales that a step edge detector performance improves considerably as the operator point spread function is extended along the edge. The disadvantage is much more complicated here than for the edge operators at different scales, because there is no clear reason to prefer one edge type over another.
D. Casanova, et. al. [4] propose a new method based on the leaf analysis for plant identification and try to explore several attributes as like shape, location and texture to make a system more accurate. The disadvantages do not have the best results among all participants made some good findings about the plant identification using leaves. The advantages have the power of texture analysis in leaf discrimination, and the texture analysis work is fine.
G. Cerutti, et.al. [5] presents a new method for tree species identification by focusing on the analysis of leaves. They developed a working process to help recognize species, starting from a picture of a leaf in a complex natural background. The main disadvantage is that, the performance of processing time, which is also concerned with the execution that lasts a few seconds on a computer will be generally 10 times longer on a smartphone.
T. Chan and L. A. Vese [6] propose a new model for active contours to detect objects whose boundaries are not necessarily defined by gradient or with very smooth boundaries, they can automatically detect interior contours starting with only one initial curve. The main disadvantage is the position of the initial curve can be anywhere in the image, and it does not necessarily surround the objects to be detected.
Y. Cheng [7] described shift, a simple iterative procedure that shifts each data point to the average of data points in its neighborhood, is generalized mean shift is a modeseeking process on a surface constructed with a “shadow” kernel Cluster analysis is treated as a deterministic problem of finding a fixed point of mean shift that characterizes the data. It is attempted to provide an appropriate generalization to the mean shift algorithm. The main advantage is to reduce this time complexity to O(n log n).
D. Comaniciu and P. Meer [8] A non parametric technique is proposed for analysis for complex multi modal feature space. The main advantage is the mean shift procedure is not computationally expensive.
C. Couprie, et.al. [9] presents a new segmentation algorithms that fixes p to produce an optimal spanning forest, but varies the power q beyond the usual watershed algorithm, which they term power watersheds. Placing the watershed algorithm in this energy minimization framework also opens new possibilities for using unary terms in traditional watershed segmentation and using watersheds to optimize more general models of use in application beyond image segmentation.
P.F. Felzenszwalb, D.P. Huttenlocher [10] developed a new method for image segmentation based on pairwise region comparison. The problems of image segmentation and grouping remain as a great challenges for computer vision. The graph-based algorithms that both help refine our understanding of the problems and provide useful computational tools. The work reported here is the normalized cuts approach.
H. Goëau et al. [11] aims to investigate the image retrieval approaches in the context of crowd sourced images of the leaves collected in a collaborative manner. Identification performances are close from mature, when using scans or photos with uniform background, but that unconstrained photos are still much more challenging. More data and evaluation are clearly required to progress on such data.
J. Horvath [12] introduced the fuzzy c-means clustering method in image segmentation. Segmentation method is based on a basic region growing method and uses membership grades' of the pixels to classify pixels into appropriate segments. Extension of feature space of fuzzy c-means clustering method brings better segmentation results. The disadvantage it concerns, demand on borders of segment.
J. Shi and J. Malik [13] propose a novel approach for solving the perceptual grouping problem in vision. An efficient computational technique based on a generalized eigen value problem can be used to optimize this criterion. A computational method has been developed and applied to the segmentation of brightness, color, and texture images. Results of experiments on real and synthetic images were found to be very encouraging.
A. Karsnas, et.al. [14] propose a new method, the vectorial Minimum Barrier Distance (MBD), for computing a gray-weighted distance transform while also incorporating information from the vectorial data. The method can be a good way of incorporating multichannel information in interactive segmentation.
M. Kass, et.al. [15] demonstrate that, a snake is an energy minimizing spline guided by the external forces and influenced by image forces that pull it toward features such as, lines and edges. Snakes are active contour models. They lock onto nearby edges, localizing them accurately. Snakes provide a number of visual problems which includes, detection of edges, lines, and subjective contours, motion tracking and stereo matching.
A. M. Khan and S. Ravi, [16] used the Semi- interactive approach algorithm, which divides an image into its constituent homogeneous regions to extract data from the attributes of the image. The disadvantage is the reduced accuracy of the segmentation results. In particular, applications can often achieve better performance.
Neeraj Kumar, et.al. [17] analyze that untrained users initially try to obtain photos of leaves in-situ, with multiple leaves present amid clutter, often with severe lighting and blur artifacts, resulting in images that we cannot hold (usually due to segmentation failures). In addition, many users also take photos of objects that are not leaves. It also greatly reduces the computational load on the server, as images that fail this classification is discarded from further processing. The system uses the distinctive shapes of leaves as the sole recognition cue.
M. Lynch, et.al. [18] propose a new approach for the automatic segmentation to extract the contours of the epicardium and endo-cardium boundary of the left ventricle of the heart. The developed segmentation scheme takes multi-slice and multi-phase Magnetic Resonance (MR) images of the heart, transversing the short-axis length from the base to the apex.
Jonathan Weber, et.al. [19] propose an efficient interactive video segmentation method, and they suggest first building a spatio-temporal over segmentation result, and then merging the different regions according to user markers. Segmentation correction is realized by editing the markers and performing again only the merging step and not the full segmentation process. If irrelevant regions are found within the initial over-segmentation result, the segmentation process is close by recomputation according to user feedback.
N. Valliammal, and S.N. Geethalakshmi [20] proposes a new hybrid approach for image segmentation that utilize K means clustering and sobel edge detector. The sobel operator performs the edge detection for plant leaf shape analysis, which should be situated as far as possible from each other to withstand against the edge distribution of a plant leaf, as equal to the number of edges amongst the data distribution. It designates the edge positions by calculating the accumulated metric between every data point and all previous line and Sharpe edge corners, and then selects data point which have the maximum value.
The aim of the study is to provide a comparative study of segmentation algorithms and highlight the performance of hybrid Guided Active contour approach.
The use of color distance map allows to enhance the image contrast and the contours. This process is based on two assumptions:
This process is characterized by five seed points respectively, one for the center and four for corners.
Geodesic Distance (GD), using the five seed points.
Coupling Global/Local Color (GLC) using only one seed point in the center.
Minimum Barrier Distance (MBD) using only one seed point in the center.
Figure 2 shows the Segmentation Algorithm which used in the Hybrid GAC.
Figure 2. Segmentation Algorithm
The simplest thresholding methods replace each pixel in an image with a black pixel, if the image intensity is less than some fixed constant T or a white pixel, if the image intensity is greater than that constant.
Mean shift is a non-parametric feature-space analysis technique for locating the maxima of a density function, a so-called mode-seeking algorithm. It is a local homogenization technique that is very useful for damping, shading or tonality differences in localized objects.
SLIC is a simple and efficient method to decompose an image in visually homogeneous regions. SLIC takes two parameters:
In the mathematical subfield of numerical analysis, a Bspline, or basis spline, is a spline function that has minimal support with respect to a given degree, smoothness, and domain partition. Any spline function of given degree can be expressed as a linear combination of B-splines of that degree. Cardinal B-splines have knots that are equidistant from each other. B-splines can be used for curve-fitting and numerical differentiation of experimental data. In the computer-aided design and computer graphics, spline functions are constructed as linear combinations of Bsplines with a set of control points.
Active contour model, also called snakes, is a framework in computer vision for delineating an object outline from a possibly noisy 2D image. The snakes model is popular in computer vision, and snakes are greatly used in applications like object tracking, shape recognition, segmentation, edge detection and stereo matching. A snake is an energy minimizing, deformable spline influenced by constraint and image forces that pull it towards object contours and internal forces that resist deformation. Snakes may be understood as a special case of the general technique of matching a deformable model to an image by means of energy minimization. In two dimensions, the active shape model represents a discrete version of this approach, taking advantage of the point distribution model to restrict the shape range to an explicit domain learned from a training set.
A watershed of a gray scale image is analogous to the notion of a catchment basin of a height map. In short, a drop of water following the gradient of an image flows along a path to finally reach a local minimum. Intuitively, the watershed of a relief corresponds to the limits of the adjacent catchment basins of the drops of water. There are different technical definitions of a watershed. In graphs, watershed lines may be defined on the nodes, the edges, or hybrid lines on both nodes and edges. Watersheds may also be defined in the continuous domain. There are also many different algorithms to compute watersheds. Watershed algorithm is used in image processing primarily for segmentation purposes.
It is based on a hierarchical multiresolution top down strategy. This algorithm starts by clustering the image content into a collection of coarse image patches sharing similiar image color characteristics. Each one of this patch is then represented as a hierarchical structure using a Binary Partition Tree (BPT) derived from the mathematical morphology theory. The image content can be considered as a forest of BPTs.
Weber's algorithm is based on two steps. The particularity of this approach is that, most of the computational effort is first made during on offline step to produce an over segmentation of the image. Then, the online interactive step involves user feedback to efficiently return objects of interest. Offline segmentation is achieved by computing quasi-flat zones from the image data. Online process is performed through a marker-based approach, where the user draw scribbles over tree leaf. User feedback introduced during this online process may even result in a refinement of the initial over segmentation.
Power Watershed is based on the principle of energy minimization of some methods such as, Graph cut. By introducing this step in a Watershed approach, a more accurate analysis of the neighbourhood of the points is obtained, which is used to refine the segmentation. The Power Watershed leads to a multilabel, scale and contrast invariant, unique global optimum obtained in practice in quasi-linear time.
Uses the resulting polygon as a shape prior to drive the evolution of an active contour. Expands with a dominant gradient term and contracts with a stronger leaf dissimilarity term.
This paper introduces a method designed to deal with the obstacles raised by such complex images, for simple and lobed tree leaves. A first segmentation step based on a light polygonal leaf model is first performed, and later used to guide the evolution of an active contour. Combining global shape descriptors given by the polygonal model with local curvature-based features, the leaves are then classified over the leaf datasets. The authors can implement the preprocessing tools for enhancing the leaf images using color distance map and input strokes. Color distance map are calculated using coupling Global Distance/Local Color (GLC), Geodesic Distance (GD), and Minimum Barrier Distance (MBD). These distance maps are used to enhance image contrast and contours. At first, they can detect if the object is center of the image and background is in the corners. These are calculated using seed points of the images. GLC method detect only one seed point in center, and GD using five seed points and MBD also use single seed points.
The input stroke allows the user to locate the leaf objects and initialize the process. And this mark has priori knowledge on the local color, and for other methods, it allows initializing the determination of the contour. Then segment the leaf image using guided active contour method with automatic descriptors.
Use the resulting polygon as a shape prior to drive the evolution of an active contour.
Energy Formulation
Instead of having an external energy term based on color consistency, or distance to a mean, they decided to reuse the dissimilarity map from the previous step, considering we already have an efficient measure of how well a pixel should fit in the leaf, in terms of color. The corresponding energy term is then based on the dissimilarity function, d detailed and can be rewritten as,
The guiding constraint term relies on a so-called stencil function, K Shape increasing with the distance to the polygonal contour, which is explicated. It guarantees that contour points that are distant from the polygon that will be pushed back towards it, when neither the color nor the gradient would be strong enough to retain it. It has then the following form:
However, this is obviously not enough and the contribution of the gradient computed on the image I is here essential, as it may allow the contour to stick to the actual boundaries of the leaf (which is ultimately the main objective), even when the color information would not be relevant enough. The gradient energy is expressed in order to penalize curve points located on pixels with low gradient magnitude and is simply formulated as:
Although, the final contour has to be precise, to capture points and teeth on the leaf margin essentially, some moderate smoothing is still necessary to prevent the contour from being too noisy. And, finally the balloon energy is here to counter balance and stabilize the other energies by adding a constant force towards the outside of the contour.
The final variation of E is determined using the calculus of variations, and the resulting evolution equation is implemented on a parametric curve.
GAC can be extended with a dominant gradient term and contraction with a stronger leaf dissimilarity term.
The performance of the system is evaluated using precision, recall, Dice index (or F-measure), The Manhattan (or Matching) index, Jaccard index, Hamming, Hausdorff distance, and MAD and SSIM.
Based on these above measurements, the proposed algorithm provides on improved rate than the existing segmentation techniques. They can compare the performance of the system using performance metrics and the result is shown in Table 1.
Table 1. Result Comparison Table
In this paper, the authors overview the algorithms that were proposed for different segmentation methods for improving the quality of segmentation. But, the result shows that segmentation algorithms have not worked properly and can't implement in large datasets rather than the proposed Hybrid GAC model. They have presented a method designed to perform the segmentation of a leaf in a natural scene, based on the optimization of a polygonal leaf model used as a shape prior for an exact active contour segmentation. It also provides a set of global geometric descriptors that, later combined with local curvature-based features extracted on the final contour, make the classification into tree species possible. The segmentation process is based on a color model that is robust to uncontrolled lighting conditions. But, a global color model for a whole image may sometimes not be enough, for leaves that are not well defined by color only. The use of an additional texture model or of an adaptive color model could lead to a good improvement.
