A new framework with a stability theory for multipoint-type and stochastic meta-heuristic optimization algorithms

In Recent years, a paradigm of optimization algorithms referred to as "meta-heuristics" have been gaining attention as a means of obtaining approximate solutions to optimization problems quickly without any special prior knowledge of the problems. Meta-heuristics are characterized by flexibility in implementation. In practical applications, we can make use of not only existing algorithms but also revised algorithms that reflect the prior knowledge of the problems. Most meta-heuristic algorithms lack mathematical grounds, however, and therefore generally require a process of trial and error for the algorithm design and its parameter adjustment. For one of the resolution of the problem, we propose an approach to design algorithms with mathematical grounds. The approach consists of first constructing a "framework" of which dynamic characteristics can be derived theoretically and then designing concrete algorithms within the framework. In this paper, we propose such a framework that employs two following basic strategies commonly used in existing meta-heuristic algorithms, namely, (1) multipoint searching, and (2) stochastic searching with pseudo-random numbers. In the framework, the update-formula of search point positions is given by a linear combination of normally distributed random numbers and a fixed input term. We also present a stability theory of the search point distribution for the proposed framework, using the variance of the search point positions as the index of stability. This theory can be applied to any algorithm that is designed within the proposed framework, and the results can be used to obtain a control rule for the search point distribution of each algorithm. We also verify the stability theory and the optimization capability of an algorithm based on the proposed framework by numerical simulation.