Information dynamics. Premises, Challenges and Results

• Autorzy: Sobczyk K.
• Języki publikacji: EN

Abstrakty

EN

In various fields of contemporary research information and dynamics are becoming the key terms. Theoretic information reasoning is well known in physics, especially in thermodynamics where the relationship between the statistical (or, informational) entropy of the system and its thermodynamical entropy has been studied since a long time. Information theory is especially relevant to data processing and statistical inference. Generally speaking, the apparatus of information theory is applicable to any probabilistic system of observations since whenever we make statistical observations (or design and conduct statistical experiments) we seek information. When the language of information theory (the concepts of enropy, mutual information between random variables and processes, information rate, maximum entropy formalism, information flow etc.) is used in connection with system dynamics we come to the notion of information dynamics. The objective of this report is to show a potential of the basic information theoretic methodology for the analysis of various problems of system dynamics. In particular, we wish to indicate some challenges and expound our recent results on the maximum information entropy approach to the analysis of stochastic dynamical systems.

Słowa kluczowe

EN

information theory; entropy; statistical thermodynamics; dynamical system

PL

teoria statystyczna; teoria informacji; entropia; układ dynamiczny; termodynamika statystyczna

• Wydawca: Instytut Podstawowych Problemów Techniki PAN
• Czasopismo: Prace Instytutu Podstawowych Problemów Techniki PAN
• Rocznik: 2000
• Tom: nr 2
• Strony: 1--32
• Opis fizyczny: Bibliogr. 38 poz.

Twórcy

autor

Sobczyk K.

• Center of Mechanics and Information Technology, Institute of Fundamental Technological Research, Polish Academy of Sciences

Bibliografia

• 1. Klir G.J., Folger T.A., Fuzzy Sets, Uncertainty and Information, Prentice Hall, Englowood, Cliffs, 1988
• 2. Sobczyk K., Stochastic Differential Equations with Application to Physics and Engineering, Kluwer Acad. Publ., Dordrecht, 1991
• 3. Waever W., Shannon C E , The Mathematical Theory of Communication, Univ. of Illinois Press, Urbana, 1949
• 4. Kullback S., Information Theory and Statistics, Chapman and Hall, N York, 1959
• 5. Ingarden R.S., Urbanik K , Information without probability, Coll. Math., Vol. 9, pp. 131-150, 1963
• 6. Kolmogorov N., Three approaches to the quantitative definition of information. Problems of Inform. Transmission, Vol 1, pp. 4-7, 1965.
• 7. Chaitin G.J., Algorithmic Information Theory, Cambrige University Press, Cambridge, 1987.
• 8. Csiszar R. I., Information-type distance measure of probability distributions and indirect observations, Sludia Scient. Mathem. Hungarica, Vol. 2, pp. 299-318, 1967.
• 9. Sugimoto S., Wada T., Spectral expressions of information measures of Gaussian time series and their relation to AIC and CAT, IEEE Trans, on Inform. Theory, Vol. 34, No. 4, 1988.
• 10. Ruelle D., Takens F., On the nature of turbulence, Comm. Math. Phys., Vol 20, pp 167- 172, 1971.
• 11. Jumarie G., Some approaches to the measures of the amount of information involved by a form, System Analysis, Modelling and Simulation, Vol. 3., pp. 479-506, 1986
• 12. Cover T.M., Thomas J.A., Elements of Information Theory, J Wiley&Sons, New York, 1991
• 13. Jaynes E.T., Information theory and statistical mechanics, Phys. Rev., Vol 106, pp. 620- 630, 1957
• 14. Ingarden R.S., Information theory and variational principles in statistical theories, Bull. Acad. Polon. Sci., Ser. Math. Astr. Phys., 11, pp 541-547, 1963 15.
• 15. Good I.J., Maximum entropy for hypothesis formulation, Ann. Math. Stat., Vol. 34, pp. 911-934, 1963
• 16 Lasota A., Mackey M.C., Chaos, Fractals and Noise; Stochastic Aspects of Dynamics, Sec Ed., Springer, New York, 1994
• 17. Mackey M.C., The dynamic origin of increasing entropy, Rev. Modern Phys., Vo! 61, pp. 763-916, Oct. 1989
• 18. Meyer M.E., Gokhale D.V., Kullback-Leibler information measure for studying convergence rates of densities and distributions, IEEE Trans. Inform. Theory, Vol. 39, pp. 1401-1403, July 1993
• 19. Share J.E., Johnson R.W., Axiomatic derivation of the principle of maximum crossentropy, IEEE Trans. Inform. Theory, Vol IT-26, pp 26-37, January 1980.
• 20. Shore J.E . Johnson R.W., Properties of cross-entropy minimization, IEEE Trans. Inform. Theory, Vol. IT-27, pp. 472-482, July 1981.
• 21. Diafari M.A , Demoment G., Maximum entropy and Bayesian approach in tomographic image reconstruction and restoration, in "Maximum Entropy and Bayesian Methods", (Ed Skilling J ), Kluwer Acad. Publ., 1989
• 22. Pristley M.B., Spectral Analysis and Time Series, Academic Press, 1981
• 23. Pinsker M.S., Information and Information Stability of Random Variables and Processes, Holden-Day, San Francisco, 1964
• 24. Ekroot L., Cover T.M., The entropy of Markov trajectories, IEEE Trans. Inform., Vol 39, pp. 1418-1421, July 1993
• 25. Cercignani C. Theory and Application of the Boltzmann Equation, Academic Press, Edinburgh-London, 1975.
• 26. Dougherty J.P., Approaches to non-equilibrium statistical mechanics, in in "Maximum Entropy and Bayesian Methods", (Ed Skilling J ), Kluwer Acad Publ, 1989
• 27. Garret A.J.M., Irreversibility, probability and entropy, in "From Statistical Physics to Statistical Inference and Back", (Eds Grassberger P., Nadal J.P.), Kluwer Acad. Publ., 1994.
• 28. Zubariev D.N., Non-equlibrium Statistical Thermodynamics, Nauka, Moscov, 1971, (in Russian).
• 29. Shaw R., Strange atrractors, chaotic behaviour and information flow, Z. Naturforsh, A36, pp 80-112, 1981.
• 30. Deco G., Schittenkopf C., Schriimann B., Determining the information flow in dynamical systems from continous probability distributions, Phys Rev., Letters, Vol. 73, No. 12, March 1997.
• 31. Sobczyk K., Trębicki J., Maximum entropy principle in stochastic dynamics, Probabilistic Eng. Mech, Vol. 5, No.3 pp. 1-10, 1990.
• 32. Sobczyk K., Trębicki J., Maximum entropy principle and non-linear stochastic oscillators, Physica A, 193, pp. 448-468, 1993.
• 33. Sobczyk K., Trębicki J., Approximate Probability Distributions for Stochastic Systems: Maximum Entropy Method, Comput. Methods Appl. Mech. Engrg., 168, pp. 91-111, 1999.
• 34. Trębicki J., Sobczyk K., Maximum entropy principle and non-stationary distributions of stochastic systems, Probab. EngngMechanics, Vol 11, pp. 169-178, 1996.
• 35. Jaynes E.T., Papers on Probability, Statistics and Statistical Physics, Reidel, 1983.
• 36. Lavenda B.H., Scherer C., Statistical inference in equilibrium and non-equilibrium thermodynamics, Riv. Nuovo Cimento, Vol. 11, No. 6, 1988.
• 37. Ingarden R.S., Kossakowski A., Ohya M., Information Dynamics and Open Systems: Classical and Quantum Approach, Kluwer Academic Publ, Dordrecht, Boston, 1997.
• 38. Haken H., Information and Self-organization: A Macroscopic Approach to Complex Systems, Springer, Berlin-Heidelberg, 1988.