This paper attempts to argue that most adaptive systems, such as evolutionary or learning systems, have inherently multiple objectives to deal with. Very often, there is no single solution that can optimize all the objectives. In this case, the concept of Pareto optimality is key to analyzing these systems.
To support this argument, we first present an example that considers the robustness and evolvability trade-off in a redundant genetic representation for simulated evolution. It is well known that robustness is critical for biological evolution, since without a sufficient degree of mutational robustness, it is impossible for evolution to create new functionalities. On the other hand, the genetic representation should also provide the chance to find new phenotypes, i.e., the ability to innovate. This example shows quantitatively that a trade-off between robustness and innovation does exist in the studied redundant representation.
Interesting results will also be given to show that new insights into learning problems can be gained when the concept of Pareto optimality is applied to machine learning. In the first example, a Pareto-based multi-objective approach is employed to alleviate catastrophic forgetting in neural network learning. We show that learning new information and memorizing learned knowledge are two conflicting objectives, and a major part of both information can be memorized when the multi-objective learning approach is adopted. In the second example, we demonstrate that a Pareto-based approach can address neural network regularizationmore elegantly. By analyzing the Pareto-optimal solutions, it is possible to identifying interesting solutions on the Pareto front.
This paper provides a short review of some of the main topics in which the current research in evolutionary multi-objective optimization is being focused. The topics discussed include new algorithms, efficiency, relaxed forms of dominance, scalability, and alternative metaheuristics. This discussion motivates some further topics which, from the author’s perspective, constitute good potential areas for future research, namely, constraint-handling techniques, incorporation of user’s preferences and parameter control. This information is expected to be useful for those interested in pursuing research in this area.
In this paper, several diversity measurements will be discussed and defined. As in other evolutionary algorithms, first the population position diversity will be discussed followed by the discussion and definition of population velocity diversity which is different from that in other evolutionary algorithms since only PSO has the velocity parameter. Furthermore, a diversity measurement called cognitive diversity is discussed and defined, which can reveal clustering information about where the current population of particles intends to move towards. The diversity of the current population of particles and the cognitive diversity together tell what the convergence/divergence stage the current population of particles is at and which stage it moves towards.
In the real-world applications, most optimization problems are subject to different types of constraints. These problems are known as constrained optimization problems (COPs). Solving COPs is a very important area in the optimization field. In this paper, a hybrid multi-swarm particle swarm optimization (HMPSO) is proposed to deal with COPs. This method adopts a parallel search operator in which the current swarm is partitioned into several subswarms and particle swarm optimization (PSO) is severed as the search engine for each sub-swarm. Moreover, in order to explore more promising regions of the search space, differential evolution (DE) is incorporated to improve the personal best of each particle. First, the method is tested on 13 benchmark test functions and compared with three stateof-the-art approaches. The simulation results indicate that the proposed HMPSO is highly competitive in solving the 13 benchmark test functions. Afterward, the effectiveness of some mechanisms proposed in this paper and the effect of the parameter setting were validated by various experiments. Finally, HMPSO is further applied to solve 24 benchmark test functions collected in the 2006 IEEE Congress on Evolutionary Computation (CEC2006) and the experimental results indicate that HMPSO is able to deal with 22 test functions.
Forecasting is an important activity in finance. Traditionally, forecasting has been done with in-depth knowledge in finance and the market. Advances in computational intelligence have created opportunities that were never there before. Computational finance techniques, machine learning in particular, can dramatically enhance our ability to forecast. They can help us to forecast ahead of our competitors and pick out scarce opportunities. This paper explains some of the opportunities offered by computational intelligence and some of the achievements so far. It also explains the underlying technologies and explores the research horizon.
Most biological systems are by nature hybrids consist of interacting discrete and continuous components, which may even operate on different time scales. Therefore, it is desirable to establish modeling frameworks that are capable of combining deterministic and stochastic, discrete and continuous, as well as multi-timescale features. In the context of molecular systems biology, an example for the need of such a combination is the investigation of integrated biological pathways that contain gene regulatory, metabolic and signaling components, which may operate on different time scales and involve on-off switches as well as stochastic effects. The implementation of integrated hybrid systems is not trivial because most software is limited to one or the other of the dichotomies above. In this study, we first review the motivation for hybrid modeling. Secondly, by using the example of a toggle switch model, we illustrate a recently developed modeling framework that is based on the combination of biochemical systems theory (BST) and hybrid functional Petri nets (HFPN). Finally, we discuss remaining challenges and future opportunities.
Evolutionary search and optimisation algorithms have been used successfully in many areas of materials science and chemistry. In recent years, these techniques have been applied to, and revolutionised the study of crystal structures from powder diffraction data. In this paper we present the application of a hybrid global optimisation technique, cultural differential evolution (CDE), to crystal structure determination from powder diffraction data. The combination of the principles of social evolution and biological evolution, through the pruning of the parameter search space shows significant improvement in the efficiency of the calculations over traditional dictates of biological evolution alone. Results are presented in which a range of algorithm control parameters, i.e., population size, mutation and recombination rates, extent of culture-based pruning are used to assess the performance of this hybrid technique. The effects of these control parameters on the speed and efficiency of the optimization calculations are discussed, and the potential advantages of the CDE approach demonstrated through an average 40% improvement in terms of speed of convergence of the calculations presented, and a maximum gain of 68% with larger population size.
In most large-scale real-world pattern classification problems, there is always some explicit information besides given training data, namely prior knowledge, with which the training data are organized. In this paper, we proposed a framework for incorporating this kind of prior knowledge into the training of min-max modular (M3) classifier to improve learning performance. In order to evaluate the proposed method, we perform experiments on a large-scale Japanese patent classification problem and consider two kinds of prior knowledge included in patent documents: patent’s publishing date and the hierarchical structure of patent classification system. In the experiments, traditional support vector machine (SVM) and M3-SVM without prior knowledge are adopted as baseline classifiers. Experimental results demonstrate that the proposed method is superior to the baseline classifiers in terms of training cost and generalization accuracy. Moreover,M3-SVM with prior knowledge is found to be much more robust than traditional support vector machine to noisy dated patent samples, which is crucial for incremental learning.
The stochastic schemata exploiter (SSE), which is one of the evolutionary algorithms based on schemata theory, was presented by Aizawa. The convergence speed of SSE is much faster than simple genetic algorithm. It sacrifices somewhat the global search performance.
This paper describes an improved algorithm of SSE, which is named as cross-generational elitist selection SSE (cSSE). In cSSE, the use of the cross-generational elitist selection enhances the diversity of the individuals in the population and therefore, the global search performance is improved.
In the numerical examples, cSSE is compared with genetic algorithm with minimum generation gap (MGG), Bayesian optimization algorithm (BOA), and SSE. The results show that cSSE has fast convergence and good global search performance.