Learning structures of conceptual models from observed dynamics using evolutionary echo state networks

• Autorzy: Abdelbari H. , Shafi K.

Identyfikatory

DOI

10.1515/jaiscr-2018-0010

• Języki publikacji: EN

Abstrakty

EN

Conceptual or explanatory models are a key element in the process of complex system modelling. They not only provide an intuitive way for modellers to comprehend and scope the complex phenomena under investigation through an abstract representation but also pave the way for the later development of detailed and higher-resolution simulation models. An evolutionary echo state network-based method for supporting the development of such models, which can help to expedite the generation of alternative models for explaining the underlying phenomena and potentially reduce the manual effort required, is proposed. It relies on a customised echo state neural network for learning sparse conceptual model representations from the observed data. In this paper, three evolutionary algorithms, a genetic algorithm, differential evolution and particle swarm optimisation are applied to optimize the network design in order to improve model learning. The proposed methodology is tested on four examples of problems that represent complex system models in the economic, ecological and physical domains. The empirical analysis shows that the proposed technique can learn models which are both sparse and effective for generating the output that matches the observed behaviour.

Słowa kluczowe

EN

complex systems modeling; conceptual models; causal loop diagrams; computational intelligence; echo state networks; evolutionary algorithms

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2018
• Tom: Vol. 8, No. 2
• Strony: 133--154
• Opis fizyczny: Bibliogr. 50 poz., rys.

Twórcy

autor

Abdelbari H.

• School of Engineering and Information Technology, University of New South Wales at Canberra, Northcott Drive, Campbell, ACT, 2600, Australia

autor

Shafi K.

• School of Engineering and Information Technology, University of New South Wales at Canberra, Northcott Drive, Campbell, ACT, 2600, Australia

Bibliografia

• [1] J. D. Sterman, Business Dynamics: Systems Thinking and Modeling for a Complex World, vol. 19.Irwin/McGraw-Hill Boston, 2000.
• [2] F. C. Billari, Agent-based computational modelling: applications in demography, social, economic and environmental sciences. Taylor & Francis, 2006.
• [3] R. A. Howard and J. E. Matheson, Influence diagrams, Decis. Anal., vol. 2, no. 3, pp. 127–143, 2005.
• [4] F.-R. Lin, M.-C. Yang, and Y.-H. Pai, A generic structure for business process modeling, Bus. Process Manag. J., vol. 8, no. 1, pp. 19–41, 2002.
• [5] L. Schruben, Simulation modeling with event graphs, Commun. ACM, vol. 26, no. 11, pp. 957–963, 1983.
• [6] S. Robinson, Simulation: the practice of model development and use. Palgrave Macmillan, 2014.
• [7] J. Ryan and C. Heavey, Requirements gathering for simulation, in Proceedings of the 3rd Operational Research Society Simulation Workshop. The Operational Research Society, Birmingham, UK, 175-184, 2006.
• [8] A. Medina-Borja and K. S. Pasupathy, Uncovering complex relationships in system dynamics modeling: Exploring the use of CART, CHAID and SEM, in Proceedings of the 25th International Conference of the System Dynamics Society, (Boston, USA), pp. 1–24, 2007.
• [9] V. Quinones-Avila and A. Medina-Borja, Universal healthcare: key behavioural factors affecting providers and recipients value propositions: a structural causal model of the puerto rico experience, Int. J. of Behav. and Hlthc. Res., vol. 3, no. 1, pp. 25–45, 2012.
• [10] M. Drobek, W. Gilani, T. Molka, and D. Soban, Automated equation formulation for causal loop diagrams, Lecture Notes in Business Information Processing, vol. 208, pp. 38–49, 2015.
• [11] E. Pruyt, S. Cunningham, J. Kwakkel, and J. De Bruijn, From data-poor to data-rich: system dynamics in the era of big data, in Proceedings of the 32nd International Conference of the System Dynamics Society, Delft, The Netherlands, 20-24 July 2014.
• [12] H. Jaeger, The ’echo state’ approach to analyzing and training recurrent neural networks-with an erratum note, Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, vol. 148, p. 34, 2001.
• [13] H. Abdelbari and K. Shafi, Learning causal loop diagram-like structures for system dynamics modeling using echo state networks, Syst. Dynam. Rev. -In Press, 2017.
• [14] D. E. Goldberg, Genetic algorithms. Pearson Education India, 2006.
• [15] R. Storn and K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces, J. Global. Optim., vol. 11, no. 4, pp. 341–359, 1997.
• [16] J. Kennedy, Particle swarm optimization, in Encyclopedia of machine learning, pp. 760–766, Springer, 2011.
• [17] Z. Wang, J. Zhang, J. Ren, and M. N. Aslam, A geometric singular perturbation approach for planar stationary shock waves, Physica D, vol. 310, pp. 19–36, 2015.
• [18] C. K. Jones, R. Marangell, P. D. Miller, and R. G. Plaza, On the stability analysis of periodic sine–gordon traveling waves, Physica D, vol. 251, pp. 63–74, 2013.
• [19] V. V. Gursky, J. Reinitz, and A. M. Samsonov, How gap genes make their domains: An analytical study based on data driven approximations, Chaos, vol. 11, no. 1, pp. 132–141, 2001.
• [20] P. Young, Data-based mechanistic modelling of environmental, ecological, economic and engineering systems, Environ. Modell. Softw., vol. 13, no. 2, pp. 105–122, 1998.
• [21] Y. Zhao, T. Weng, and M. Small, Response of the parameters of a neural network to pseudoperiodic time series, Physica D, vol. 268, pp. 79–90, 2014.
• [22] Y. Feng, Y. Liu, X. Tong, M. Liu, and S. Deng, Modeling dynamic urban growth using cellular automata and particle swarm optimization rules, Landscape Urban Plan., vol. 102, no. 3, pp. 188–196, 2011.
• [23] N. Petrov and A. Gegov, Model optimization for complex systems using fuzzy networks theory, in Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data bases, pp. 116–121, World Scientific and Engineering Academy and Society (WSEAS), 2009.
• [24] I. M. Greca and M. A. Moreira, Mental models, conceptual models, and modelling, Int. J. Sci. Educ, vol. 22, no. 1, pp. 1–11, 2000.
• [25] J. D. Sterman, Systems dynamics modeling: tools for learning in a complex world, IEEE Eng. Manag. Rev., vol. 30, no. 1, pp. 42–42, 2002.
• [26] G. Desthieux, F. Joerin, and M. Lebreton, Ulysse: a qualitative tool for eliciting mental models of complex systems, Syst. Dynam. Rev., vol. 26, no. 2, pp. 163–192, 2010.
• [27] K.-i. Funahashi and Y. Nakamura, Approximation of dynamical systems by continuous time recurrent neural networks, Neural networks, vol. 6, no. 6, pp. 801–806, 1993.
• [28] H. Jaeger, Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the” echo state network” approach, Tech. Rep. 159, Fraunhofer Institute for Autonomous Intelligent Systems (AIS), 2002b.
• [29] D. Koryakin, J. Lohmann, and M. V. Butz, Balanced echo state networks, Neural Networks, vol. 36, pp. 35–45, 2012.
• [30] I. B. Yildiz, H. Jaeger, and S. J. Kiebel, Re-visiting the echo state property, Neural networks, vol. 35, pp. 1–9, 2012.
• [31] M. Lukosevi ˇ cius, A practical guide to applying ˇecho state networks, in Neural Networks: Tricks of the Trade, pp. 659–686, Springer, 2012.
• [32] C. E. Martin and J. A. Reggia, Fusing swarm intelligence and self-assembly for optimizing echo state networks, Comput. Intell. Neurosci., vol. 2015, p. 9,2015.
• [33] A. A. Ferreira and T. B. Ludermir, Comparing evolutionary methods for reservoir computing pretraining, in Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, California, USA, pp. 283–290, July 31 - August 5 2011.
• [34] A. Deihimi and A. Solat, optimised echo state networks using a big bang–big crunch algorithm for distance protection of series-compensated transmission lines, Int. J. Elec. Power., vol. 54, pp. 408–424, 2014.
• [35] A. A. Ferreira, T. B. Ludermir, and R. R. De Aquino, An approach to reservoir computing design and training, Expert. Syst. Appl., vol. 40, no. 10, pp. 4172–4182, 2013.
• [36] D. Liu, J. Wang, and H. Wang, Short-term wind speed forecasting based on spectral clustering and optimised echo state networks, Renew. Energ., vol. 78, pp. 599–608, 2015.
• [37] J. L. Gross and J. Yellen, Handbook of graph theory. CRC press, 2004.
• [38] R. Tarjan, Depth-first search and linear graph algorithms, SIAM J. Comput., vol. 1, no. 2, pp. 146–160, 1972.
• [39] V. Petridis, S. Kazarlis, and A. Bakirtzis, Varying fitness functions in genetic algorithm constrained optimization: the cutting stock and unit commitment problems, IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 28, no. 5, pp. 629–640, 1998.
• [40] A. E. Smith and D. M. Tate, Genetic optimization using a penalty function, in Proceedings of the 5th international conference on genetic algorithms, pp. 499–505, Morgan Kaufmann Publishers Inc., 1993.
• [41] K. Langfield-Smith and A. Wirth, Measuring differences between cognitive maps, J. Oper. Res. Soc.,pp. 1135–1150, 1992.
• [42] Y.-C. Chuang, C.-T. Chen, and C. Hwang, A simple and efficient real-coded genetic algorithmfor constrained optimization, Appl. Soft. Comput.,vol. 38, pp. 87–105, 2016.
• [43] J. Lane, A. Engelbrecht, and J. Gain, Particle swarm optimization with spatially meaningful neighbours, in Swarm Intelligence Symposium, 2008. SIS 2008. IEEE, pp. 1–8, IEEE, 2008.
• [44] R. C. Eberhart and Y. Shi, Comparing inertia weights and constriction factors in particle swarm optimization, in Proceedings of the 2000 Congress on Evolutionary Computation, vol. 1, pp. 84–88, IEEE, 2000.
• [45] S. N. Grosser and M. Schaffernicht, Mental mod- ¨els of dynamic systems: taking stock and looking ahead, Syst. Dynam. Rev., vol. 28, no. 1, pp. 46–68,2012.
• [46] E. M. Aylward, P. A. Parrilo, and J.-J. E. Slotine, Stability and robustness analysis of nonlinear systems via contraction metrics and sos programming, Automatica, vol. 44, no. 8, pp. 2163–2170, 2008.
• [47] M. Rafferty, Butterflies and buffers, in Proceedings of the 27th International Conference of the System Dynamics Society, Albuquerque, Mexico, USA, July 26-30 2009.
• [48] E. Theodorsson-Norheim, Friedman and quade tests: Basic computer program to perform nonparametric two-way analysis of variance and multiple comparisons on ranks of several related samples, Comput. Biol. Med., vol. 17, no. 2, pp. 85–99, 1987.
• [49] M. R. Stoline, The status of multiple comparisons: simultaneous estimation of all pairwise comparisons in one-way anova designs, Am. Stat., vol. 35, no. 3, pp. 134–141, 1981.
• [50] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm:NSGA-II, IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182–197, 2002.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).