The main bottleneck in the GP methodology is the availability of computing power.
With sufficient computing power, it is possible to use GP beneficially for solving a wide variety of realistic problems. Availability of multiple processors and parallel implementation of the GP code will be the key factors for serious GP applications.
MATLAB may not be quite suitable for this scenario (unless new MATLAB toolboxes are developed to fill this void). Compiled languages such as Fortran C, C++ may have to be employed. This is one logical step that is being pursued currently.
Solving realistic industrial size problems will require a larger population size and number of generations than we have employed in the case studies demonstrated here.
This will mean a large increase in computational time. Increase in computation power can be realized by the speed of computation or parallelizing the application. In an attempt to run the program faster, we employed distributed computing in MATLAB environment. However, the communication is limited and can only handle about up to a maximum of 5 “slave” computers. There are several research publications in parallelization of GP. Andre and Koza (1998) show the benefits of parallel implementation of genetic programming. Salhi et al., (1998) describes parallel implementation of genetic programming based tool for symbolic regression. This means that we may have to migrate from MATLAB platform to Fortran / C / C++. This migration is also needed to make the software more “user friendly” by providing an easy to use user interface.
The availability of domain specific knowledge can narrow the search space and may lead to more meaningful models. Physical insights and results of nonparametric methods should be used whenever possible. Also, when data is highly correlated, multivariate statistical tools such as principal components analysis (PCA) should be employed as preprocessing tools. The PCA model must then be integrated into the GP system. Checks for asymptotic behavior of models should be included and made use of in parameter estimation (to specify the constraints on the search space). In the calculation of fitness, we could also include a penalty if the expected asymptote nature is not met. As a means of improving the efficiency of GP, one can even consider using the “ACE transformed” variables as a terminal gene in GP.
There can be other interesting applications of the GP methodology in the control area.
Given a first principles model of an interacting multivariable system, we can use GP to find transformations of the manipulated variables and/or controlled variables so as to make the system linear (i.e. transform from a nonlinear system in original coordinates to a linear or “flat” system in altered coordinates) and/or non-interacting (i.e. transform from a interacting system in the original coordinates to a less interacting or non- interacting system in the “new” coordinates). Measures of linearity/nonlinearity and interaction need to be incorporated for such applications. These efforts can be useful in nonlinear and multivariable controller design.
REFERENCES
[Andre and Koza 1998] Andre D. and Koza J. R., “A parallel implementation of genetic programming that achieves super-linear performance”, Journal of Information Science. 106, pp. 201-218, 1998.
[Ashour et al., 2003] Ashour, A. F., L. F. Alvarez and V. V. Toropov, “Empirical Modelling of Shear Strength of RC Deep Beams by Genetic Programming”, Computers and Structures, 81, pp. 331-338, 2003
[Banzhaf et. al., 1998] Banzhaf, W., Frank D. Francone, R. E. Keller and P. Nordin,
“Genetic Programming – an introduction: on the automatic evolution of computer programs and its application”, Morgan Kaufmann Publishers, Inc. USA, 1998.
[Bard 1974] Bard, Y., “Nonlinear Parameter Estimation”, Aceademic Press, New York, NY, 1974.
[Bates and Watts 1988] D.M. Bates and D.G. Watts, “Nonlinear Regression Analysis and its Applications”, John Wiley and Sons, New York, 1988.
[Box and Jenkins 1994] G. E. P. Box and G. M. Jenkins. “Time series analysis : forecasting and control”, San Francisco, Holden-Day , 1994.
[Breiman and Friedman 1985] Breiman L. and J. H. Friedman, “Estimating Optimal Transformations for Multiple Regression and Correlation (with discussion)”, Journal of the American Statistical Association 80, 580, 1985.
[Bunday and Garside 1987] Bunday, B.D. and Garside G.R. “Optimisation Methods in Pascal”, Edward Arnold Publ., 1987
[Cao et. al., 1999] Hongqing Cao, Jingxian Yu, Lishan Kang, Yuping Chen and Yongyan Chen. “The kinetic evolutionary modeling of complex systems of chemical reactions” Computers & Chemistry, 23(2), pp. 143-152, 1999.
[Davidson et. al., 2003] Davidson, J. W., Savic, D. A. and Walters. G. A., “Symbolic and numerical regression: experiments and applications”, Information Sciences, 150 (1-2), pp. 95-117, March 2003.
[DeVeaux et. al., 1993] DeVeaux R.D., Psichogios D.C., and L.H. Ungar, “A comparison of Two Nonparametric Estimation Schemes: MARS and Neural Networks”, Comp. & Chem. Engg., 17(8), pp. 819-837, 1993.
[Englezos and Kalogerakis 2001] Englezos, P., and Kalogerakis, N., “Applied parameter estimation for chemical engineers”, Marcel Dekker, Inc. New York, 2001.
[Eskinat et. al., 1991] Eskinat, E., J., S. H. and Luyben, W. L. “Use of Hammerstein models in identification of nonlinear system”, A.I.Ch.E J. 37(2), pp. 255-268. 1991.
[Esposito et. al., 2000] Esposito, W. R. and Floudas, C. A., “Global Optimization for the Parameter Estimation of Differential-Algebraic Systems”, Ind. Eng. Chem.
Res., 39, pp. 1291-1310, 2000.
[Friedman 1991] Friedman, J. H. “Multivariate Adaptive Regression Splines (with discussion), Annals of Statistics”, 19, pp. 1-141, 1991.
[Gao and Loney 2001] Gao, L. and N. W. Loney, “Evolutionary Polymorphic Neural Network in Chemical Process Modelling”, Comp. & Chem. Engg., 25, pp. 1403- 1410, 2001.
[Goldberg 1989] Goldberg, D.E., “Genetic Algorithms in Search, Optimization and Machine Learning”, Addison-Wesley, 1989.
[Gray et. al., 1996] Gray, G. J., D. J. Murray-Smith, Y. Li and K. C. Sharman,
“Nonlinear Model Structure Identification using Genetic Programming and a Block Diagram Oriented Simulation Tool”, Electronic Letters, 32, pp. 1422-1424, 1996.
[Greeff and Aldrich 1998] Greeff, D. J. and C. Aldrich, “Empirical Modelling of Chemical Process Systems with Evolutionary Programming”, Comp. & Chem.
Engg., 22(7-8), pp. 995-1005, 1998.
[Grosman and Lewin, 2002] Grosman, B. and D. R. Lewin. “Automated nonlinear model predictive control using genetic programming”, Comp. & Chem. Engg, 26, pp.631–640. 2002.
[Hall and Seborg 1989] Hall R.C. and Seborg D.E. “Modeling and multi-loop control of a multivariable pH neutralization process: Part I: Modeling and multiloop control”, In Proceedings of the American Control Conference, Pittsburgh, 1989.
[Henson and Seborg 1994] Henson, A. M. and Seborg, E. D., “Adaptive nonlinear control of a pH neutralization process”, IEEE Trans. Control Sys. Tech. 2 (3), 1994.
[Holland 1975] Holland, J. H., “Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence”, MIT Press, Cambridge, MA. 1975.
[Hong and Rao 2003] Hong, Y-S. and B. Rao, “Evolutionary self-organising modeling of a municipal wastewater treatment plant”, Water Research, 37, pp. 1199-1212, 2003.
[Horst and Pardalos 1995] Reiner Horst and Panos M. Pardalos (editors) “Handbook of global optimization”, Dordrecht; Boston, Kluwer Academic Publishers, 1995.
[Iba 1993] Iba, H., T. Kurita and T. Sato, “System Identification using Structured Genetic Algorithms”, Proceedings of the 5th International Joint Conference on Genetic Algorithms, pp. 41-53, 1993.
[Iba 1994] Iba, H., T. Sato and Hugo de Garis, “System Identification Approach using Genetic Programming”, Proceedings of the First IEEE Conference on Evolutionary Computation, 1, pp. 401-406, Orlando, Florida, June 27 – June 29, 1994.
[Ingber 1989] Ingber, L. “Very fast simulated re-annealing”, Journal of Mathematical Computer Modelling, 12, 967-973, 1989.
[Jeong 1996] Jeong, W. and Lee, J. “Adaptive Simulated Annealing Genetic Algorithm for system identification”, Engg. Applic. Artif. Intell. 9(5), pp. 523-532, 1996.
[Kaboudan 2003] Kaboudan, M. A., ‘Forecasting with Computer-Evolved Model Specifications: a Genetic Programming Application”, Computers and Operations Research, 30, 1661-1681, 2003.
[Kirkpatrick 1983] Kirkpatrick S. Jr., C.D. Gelatt, M. Vecchi, “Optimization by Simulated Annealing”, Science, 220 (4598), pp 671–680, 1983.
[Koza 1992] John R. Koza. “Genetic programming : on the programming of computers by means of natural selection”, Cambridge, Mass., MIT Press, 1992.
[Koza 1999a] Koza, J. R., F. H. Bennet III, M. A. Kleane, “Genetic Programming III:
Darwinian Invention and Problem Solving”, Morgan Kaufmann, San Francisco, CA, 1999.
[Koza 1999b] Koza, J. R., F. H. Bennet III, W. Mydlowec, M. A. Kleane, J. Yu, O.
Stiffelman “Searching for the impossible using Genetic Programming”, GECCO- 99, Morgan Kaufmann, San Francisco, CA, pp. 1083-1091, 1999.
[Lang 1995] Lang, K. J. “Hill climbing beats genetic search on a Boolean circuit synthesis of Koza’s”, In Proceeding of the twelfth international conference on machine learning, Tahoe, CA. Morgan Kaufmann, San Francisco, CA.
[Langdon and Poli 2002] Langdon, William B., Riccardo, Poli “Foundations of genetic programming”, New York, Springer, 2002.
[Lakshminarayanan et. al., 1995] Lakshminarayanan S., Shah. L. Sirish and Nandakumar K. “Identification of Hammerstein models using multivariate statistical tools”, Chem. Engg. Sci., 50 (22), pp 3599-3613, 1995.
[Lakshminarayanan 2000] Lakshminarayanan, S., H. Fujii, B. Grosman, E. D. and D.R.
Lewin., “New product design via analysis of historical databases”, Comp. & Chem.
Engg., 24, pp. 671-676, 2000.
[Ljung 1989] Lennart Ljung. “System identification : theory for the user”, Englewood Cliffs, NJ : Prentice-Hall , 1987.
[Ljung 1999] Lennart Ljung. “System identification : theory for the user”, Upper Saddle River, N.J. : Prentice Hall, 1999. 2nd ed.
[Luus 1998] Luus, R. “Parameter Estimation of Lotka-Volterra Problem by Direct Search Optimization”, Hung. J. Ind. Chem. 26, 287. 1998.
[Marenbach 1998] Marenbach P. “Using prior knowledge and obtaining process insight in data based modeling of bioprocesses”, SAMS, 31. pp 39-59, 1998.
[McKay 1997] McKay, B., Willis, M. and Barton, G., “Steady-state modeling of chemical processing system using genetic programming”, Computer chem. Engng.
21 (9), pp 981-996, 1997.
[McPhee and Miller 1995] McPhee, N. F. and Miller, J. D. “Accurate replication in genetic programming”, In Eshelman, L., Editor, Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), pp. 303-309, 1995.
[More 1997] More´, J “The Levenberg–Marquardt Algorithm: Implementation and Theory” Numerical Analysis, in: G.A. Watson (Ed.), Lecture Notes in Math, vol.
630, Springer Verlag, Berlin, 1977.
[Salhi et. al., 1998] Salhi A., H. Glaser and D. De Roure. 1998 “Parallel implementation of a genetic-programming based tool for symbolic regression”, Information Processing Letters, 66, pp. 299-307, 1998.
[Sen and Stoffa1995] Sen, M. K. and Stoffa, P. L. “Global Optimization Methods in Geophysical Inversion”, Elsevier, Amsterdam, pp. 294, 1995.
[Srinivas and Deb1995] Srinivas, N., and Deb, K. “Multiobjective function optimization using nondominated sorting genetic algorithms”, Evolutionary Computation, 2, pp. 221–248, 1995.
[Storn and Price 1997] Storn. R and Price K. “Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous space” Journal of Global Optimization, 11, pp. 341–359, 1997.
[Swain and Morris 2003] A. K. Swain and A. S. Morris. “An evolutionary approach to the automatic generation of mathematical models”, Applied Soft Computing, 3 (1), pp. 1-21, 2003.
[Tan et al. 2003] K. C. Tan, Q. Yu, C. M. Heng and T. H. Lee. “Evolutionary computing for knowledge discovery in medical diagnosis”, Artificial Intelligence in Medicine, 27 (2), pp. 129-154, 2003.