A survey of basic deterministic, heuristic, and hybrid methods for single-objective optimization and response surface generation
Book Chapter
Colaço, MJ, Dulikravich, GS. (2011). A survey of basic deterministic, heuristic, and hybrid methods for single-objective optimization and response surface generation
. 355-405.
Colaço, MJ, Dulikravich, GS. (2011). A survey of basic deterministic, heuristic, and hybrid methods for single-objective optimization and response surface generation
. 355-405.
Inverse problems usually involve theminimization of some objective function as part of their formulation. Such minimization procedures require the use of an optimization technique. Thus, in this chapter, we address solution methodologies for single-objective optimization problems, based on minimization techniques. Several gradient-based and non-gradientbased (stochastic) techniques are discussed, together with their basic implementation steps and algorithms. We present some deterministic methods, such as the conjugate gradient method, the NewtonMethod, and the Davidon-Fletcher-Powell (DFP)method (Levenberg, 1944; Hestenes and Stiefel, 1952; Davidon, 1959; Fletcher and Powell, 1963;Marquardt, 1963; Fletcher and Reeves, 1964; Broyden, 1965, 1967; Daniel, 1971; Polak, 1971; Beale, 1972; Bard, 1974; Beck and Arnold, 1977; Moré, 1977; Powell, 1977; Tikhonov andArsenin, 1977; Dennis and Schnabel, 1983; Beck et al., 1985; Stoecker, 1989; Murio, 1993; Alifanov, 1994; Alifanov et al., 1995; Kurpisz and Nowak, 1995; Dulikravich and Martin, 1996; Trujillo and Busby, 1997; Jaluria, 1998; Beck, 1999; Belegundu and Chandrupatla, 1999; Colaço and Orlande, 1999, 2001a, b, 2004; Fletcher, 2000; Ozisik and Orlande, 2000; Woodbury, 2002). In addition, we present some of the stochastic approaches, such as the simulated annealing method (Corana et al., 1987; Goffe et al., 1994), the differential evolutionarymethod (Storn and Price, 1996), genetic algorithms (Goldberg, 1989; Deb, 2002), and the particle swarm method (Kennedy and Eberhart, 1995; Kennedy, 1999; Eberhart et al., 2001; Naka et al., 2001). Deterministic methods are, in general, computationally faster (they require fewer objective function evaluations in case of problemswith low number of design variables) than stochastic methods, although they can converge to a local minima or maxima, instead of the global one. On the other hand, stochastic algorithms can ideally converge to a global maxima or minima, although they are computationally slower (for problems with relatively low number of design variables) than the deterministic ones. Indeed, the stochastic algorithms can require thousands of evaluations of the objective functions and, in some cases, become nonpractical. In order to overcome these difficulties, wewill also discuss the so-called hybrid algorithms that take advantage of the robustness of the stochastic methods and the fast convergence of the deterministic methods (Dulikravich et al., 1999, 2003, 2004, 2008; Colaço and Orlande, 2001a, b; Colaço et al., 2004, 2005b, 2006, 2008; Colaço and Dulikravich, 2006, 2007; Dulikravich andColaço, 2006;Wellele et al., 2006; Silva et al., 2007; Padilha et al., 2009). Each technique provides a unique approachwith varyingdegrees of convergence, reliability, and robustness at different stages during the iterative minimization process. A set of analytically formulated rules and switching criteria can be coded into the program to automatically switch back and forth among the different algorithms as the iterative process advances (Dulikravich et al., 1999; Colaço et al., 2005b, 2008). In many optimization problems, evaluation of the objective function is extremely expen-sive and time consuming. For example, optimizing chemical concentrations of each of the alloying elements in a multicomponent alloy requires manufacturing each candidate alloy and evaluating its properties using classical experimental techniques. Even with the most efficient optimization algorithms (Dulikravich et al., 2008), this means that often thousands of alloys having different chemical concentrations of their constitutive elements would have to be manufactured and tested. This is understandably too expensive to be economically acceptable. Similar is the situation when attempting to optimize three-dimensional aerodynamic shapes. Aerodynamics of thousands of different shapes needs to be analyzed using computational fluid dynamics software, which would be unacceptably time consuming.
