Optimization is ubiquitous and spontaneous process that forms an integral part of day-to-day life. In the most basic sense, it can be defined as an art of selecting the best alternative among a given set of options. Optimization plays an important role in Engineering designs, Agricultural sciences, Manufacturing systems, Economics, Physical sciences, Pattern recognition[1] and other such related fields. The objective of optimization is to seek values for a set of parameters that maximize or minimize the objective functions subject to certain constraints. A choice of values for the set of parameters that satisfy all the constraints is called a feasible solution. Feasible solutions with objective function value(s) are as good as the values of any other feasible solutions, that are called as optimal solutions. In order to use optimization successfully, they must first determine an objective through which they can measure the performance of the system under study. That objective could be time, cost, weight, potential energy or any combination of quantities that can be expressed by a single variable. The objective relies on certain characteristics of the system, called variable or unknowns. The optimization algorithms come from different areas and are inspired by different techniques. But they are sharing some common characteristics. They are iteratives that are begun with an initial guess of the optimal values of the variables and then generate a sequence of improved estimates until they converge to a solution.