Applying LCS to affective image classification in spatial - frequency domain

• Autorzy: Lee P. -M. , Hsiao T.-C.

Identyfikatory

DOI

10.1515/jaiscr-2015-0002

• Języki publikacji: EN

Abstrakty

EN

Recent studies have utilizes color, texture, and composition information of images to achieve affective image classification. However, the features related to spatial-frequency domain that were proven to be useful for traditional pattern recognition have not been tested in this field yet. Furthermore, the experiments conducted by previous studies are not internationally-comparable due to the experimental paradigm adopted. In addition, contributed by recent advances in methodology, that are, Hilbert-Huang Transform (HHT) (i.e. Empirical Mode Decomposition (EMD) and Hilbert Transform (HT)), the resolution of frequency analysis has been improved. Hence, the goal of this research is to achieve the affective image-classification task by adopting a standard experimental paradigm introduces by psychologists in order to produce international-comparable and reproducible results; and also to explore the affective hidden patterns of images in the spatial-frequency domain. To accomplish these goals, multiple human-subject experiments were conducted in laboratory. Extended Classifier Systems (XCSs) was used for model building because the XCS has been applied to a wide range of classification tasks and proved to be competitive in pattern recognition. To exploit the information in the spatial-frequency domain, the traditional EMD has been extended to a two-dimensional version. To summarize, the model built by using the XCS achieves Area Under Curve (AUC) = 0.91 and accuracy rate over 86%. The result of the XCS was compared with other traditional machine-learning algorithms (e.g., Radial-Basis Function Network (RBF Network)) that are normally used for classification tasks. Contributed by proper selection of features for model building, user-independent findings were obtained. For example, it is found that the horizontal visual stimulations contribute more to the emotion elicitation than the vertical visual stimulation. The effect of hue, saturation, and brightness; is also presented.

Słowa kluczowe

EN

image classification; pattern recognition; Hilbert-Huang transform; HHT; empirical mode decomposition; EMD; Hilbert transform; HT; Extended Classifier Systems; XCSs; Area Under Curve; AUC; Radial-Basis Function Network; RBF Network; LCS

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2014
• Tom: Vol. 4, No. 2
• Strony: 99--123
• Opis fizyczny: Bibliogr. 79 poz., rys.

Twórcy

autor

Lee P. -M.

• Institute of Computer Science and Engineering, Department of Computer Science, National Chiao Tung Universityal. 1001 University Rd., Hsinchu, Taiwan, R.O.C.

autor

Hsiao T.-C.

• Department of Computer Science, Institute of Biomedical Engineering, and Biomedical Electronics Translational Research Center and Biomimetic Systems Research Center in National Chiao Tung Universityal. 1001 University Rd., Hsinchu, Taiwan, R.O.C.

Bibliografia

• [1] Kuo, W.J., et al., Intuition and Deliberation: Two Systems for Strategizing in the Brain. Science, 2009. 324(5926): p. 519-522.
• [2] Chowdhury, R.M.M.I., G.D. Olsen, and J.W. Pracejus, Affective Responses to Images In Print Advertising: Affect Integration in a Simultaneous Presentation Context. Journal of Advertising, 2008. 37(3): p. 7-18.
• [3] Chang, C., The Impacts of Emotion Elicited By Print Political Advertising on Candidate Evaluation. Media Psychology, 2001. 3(2): p. 91-118.
• [4] Kyung-Sun, K., Effects of emotion control and task on Web searching behavior. Information Processing & Management, 2008. 44(1): p. 373-385.
• [5] Mitchell, T.M., Machine learning. 1997. Burr Ridge, IL: McGraw Hill, 1997. 45.
• [6] Orriols-Puig, A., J. Casillas, and E. Bernad-Mansilla, Genetic-based machine learning systems are competitive for pattern recognition. Evolutionary Intelligence, 2008. 1(3): p. 209-232.
• [7] Russell, S. and P. Norvig, Artificial Intelligence: A Modern Approach (2nd Edition)2002: Prentice Hall.
• [8] Wilson, S.W., Classifier fitness based on accuracy. Evol. Comput., 1995. 3(2): p. 149-175.
• [9] Picard, R.W., Affective Computing2000, Cambridge MA: The MIT Press.
• [10] Ortony, A. and T. Turner, What’s basic about basic emotions. Psychological review, 1990.
• [11] Bradley, M.M., Emotional memory: a dimensional analysis, in Emotions: Essays on emotion theory, S.H.M.v. Goozen, N.E.v.d. Poll, and J.A. Sergeant, Editors. 1994, Lawrence Erlbaum: Hillsdale, NJ. p. 97-134.
• [12] Bradley, M.M. and P.J. Lang, Emotion and motivation, in Handbook of Psychophysiology, J.T. Cacioppo, L.G. Tassinary, and G. Berntson, Editors. 2007, Cambridge University Press: New York, NY. p. 581-607.
• [13] Lang, P.J., The motivational organization of emotion: Affect-reflex connections, in Emotions: Essays on emotion theory1994, Lawrence Erlbaum: Hillsdale, NJ. p. 61-93.
• [14] Lang, P.J., The Emotion Probe - Studies of Motivation and Attention. American Psychologist, 1995. 50(5): p. 372-385.
• [15] Bolls, P.D., A. Lang, and R.F. Potter, The Effects of Message Valence and Listener Arousal on Attention, Memory, and Facial Muscular Responses to Radio Advertisements. Communication Research, 2001. 28: p. 627-651.
• [16] Antonio R, D., Emotion in the perspective of an integrated nervous system. Brain Research Reviews, 1998. 26(2–3): p. 83-86.
• [17] Bechara, A., The role of emotion in decisionmaking: Evidence from neurological patients with orbitofrontal damage. Brain and cognition, 2004. 55(1): p. 30-40.
• [18] LaBar, K.S. and R. Cabeza, Cognitive neuroscience of emotional memory. Nat Rev Neurosci, 2006. 7(1): p. 54-64.
• [19] Wu, Q., C. Zhou, and C. Wang, Content-Based Affective Image Classification and Retrieval Using Support Vector Machines, in Affective Computing and Intelligent Interaction, J. Tao, T. Tan, and R. Picard, Editors. 2005, Springer Berlin / Heidelberg. p. 239-247.
• [20] Joshi, D., et al., Aesthetics and Emotions in Images. Signal Processing Magazine, IEEE, 2011. 28(5): p. 94-115.
• [21] Liu, N., et al., Associating Textual Features with Visual Ones to Improve Affective Image Classification, in Affective Computing and Intelligent Interaction, S. D’Mello, et al., Editors. 2011, Springer Berlin / Heidelberg. p. 195-204.
• [22] Machajdik, J. and A. Hanbury, Affective image classification using features inspired by psychology and art theory, in Proceedings of the international conference on Multimedia2010, ACM: Firenze, Italy. p. 83-92.
• [23] Zhang, H., et al., Analyzing Emotional Semantics of Abstract Art Using Low-Level Image Features, in Advances in Intelligent Data Analysis X, J. Gama, E. Bradley, and J. Hollmn, Editors. 2011, Springer Berlin / Heidelberg. p. 413-423.
• [24] Lang, P.J., M.M. Bradley, and B.N. Cuthbert, International affective picture system (IAPS): Affective ratings of pictures and instruction manual, 2008: University of Florida, Gainesville, FL.
• [25] Sanchez-Navarro, J., et al., Psychophysiological, behavioral, and cognitive indices of the emotional response: A factor-analytic study. Spanish Journal of Psychology, 2008. 11(1): p. 16-25.
• [26] Kensinger, E.A., R.J. Garoff-Eaton, and D.L. Schacter, Effects of emotion on memory specificity: Memory trade-offs elicited by negative visually arousing stimuli. Journal of Memory and Language, 2007. 56(4): p. 575-591.
• [27] Lang, P.J., Behavioral treatment and biobehavioral assessment: Computer applications, in Technology in Mental Health Care Delivery Systems, J. Sidowski, J. Johnson, and T. Williams, Editors. 1980, Ablex Pub. Corp.: Norwood, NJ. p.119-137.
• [28] Morris, J.D., Observations: SAM: the Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response. Journal of advertising research, 1995. 35(6): p. 63-68.
• [29] Mehrabian, A. and J.A. Russell, An approach to environmental psychology1974, Cambridge, MA: the MIT Press.
• [30] Bradley, M.M. and P.J. Lang, The International Affective Digitized Sounds (2nd Edition; IADS-2): Affective ratings of sounds and instruction manual. University of Florida, Gainesville, FL, Tech. Rep. B-3, 2007.
• [31] Holland, J.H., Adaptation in Natural and Artificial System1992, Cambridge, MA, USA: MIT Press.
• [32] Wilson, S.W., ZCS: A Zeroth Level Classifier System. Evolutionary Computation, 1994. 2(1): p. 1- 18.
• [33] Wilson, S.W., Get Real! XCS with Continuous- Valued Inputs. Learning Classifier Systems, 2000. 1813: p. 209-219.
• [34] Stone, C. and L. Bull, For Real! XCS with Continuous-Valued Inputs. Evolutionary Computation, 2003. 11(3): p. 299-336.
• [35] Dam, H.H., H.A. Abbass, and C. Lokan, Be real! XCS with continuous-valued inputs, in Proceedings of the 2005 workshops on Genetic and evolutionary computation2005, ACM: Washington, D.C. p. 85-87.
• [36] Lanzi, P.L. Adding memory to XCS. in IEEE World Congress on Computational Intelligence. 1998.
• [37] Wilson, S.W., Compact Rulesets from XCSI, in Advances in Learning Classifier Systems, P. Lanzi, W. Stolzmann, and S. Wilson, Editors. 2002, Springer Berlin / Heidelberg. p. 65-92.
• [38] Dam, H.H., H.A. Abbass, and C. Lokan, DXCS: an XCS system for distributed data mining, in Proceedings of the 2005 conference on Genetic and evolutionary computation2005, ACM: Washington DC, USA. p. 1883-1890.
• [39] Wilson, S.W., Classifiers that approximate functions. Natural Computing, 2002. 1(2): p. 211-234.
• [40] Lanzi, P.L., et al., Generalization in the XCSF Classifier System: Analysis, Improvement, and Extension. Evol. Comput., 2007. 15(2): p. 133-168.
• [41] Bull, L., E. Bernad-Mansilla, and J. Holmes, Learning Classifier Systems in Data Mining: An Introduction, in Learning Classifier Systems in Data Mining, L. Bull, E. Bernad-Mansilla, and J. Holmes, Editors. 2008, Springer Berlin / Heidelberg. p. 1-15.
• [42] Butz, M., et al., Knowledge Extraction and Problem Structure Identification in XCS, in Parallel Problem Solving from Nature - PPSN VIII, X. Yao, et al., Editors. 2004, Springer Berlin / Heidelberg. p. 1051-1060.
• [43] Muruzbal, J., A probabilistic classifier system and its application in data mining. Evol. Comput., 2006. 14(2): p. 183-221.
• [44] Orriols-Puig, A., J. Casillas, and E. Bernad-Mansilla, First approach toward on-line evolution of association rules with learning classifier systems,in Proceedings of the 2008 GECCO conference companion on Genetic and evolutionary computation2008, ACM: Atlanta, GA, USA. p. 2031-2038.
• [45] Dam, H., C. Lokan, and H. Abbass, Evolutionary Online Data Mining: An Investigation in a Dynamic Environment, in Evolutionary Computation in Dynamic and Uncertain Environments, S. Yang, Y.-S. Ong, and Y. Jin, Editors. 2007, Springer Berlin / Heidelberg. p. 153-178.
• [46] Quirin, A., et al. Analysis and evaluation of learning classifier systems applied to hyperspectral image classification. in Intelligent Systems Design and Applications, 2005. ISDA ’05. Proceedings. 5th International Conference on. 2005.
• [47] Butz, M., et al., Effective and Reliable Online Classification Combining XCS with EDA Mechanisms, in Scalable Optimization via Probabilistic Modeling, M. Pelikan, K. Sastry, and E. CantPaz, Editors. 2006, Springer Berlin / Heidelberg. p. 249-273.
• [48] Akbar, M.A. and M. Farooq, Application of evolutionary algorithms in detection of SIP based flooding attacks, in Proceedings of the 11th Annual conference on Genetic and evolutionary computation2009, ACM: Montreal, Canada. p. 1419-1426.
• [49] Armano, G., A. Murru, and F. Roli, Stock Market Prediction by a Mixture of Genetic-Neural Experts. International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI), 2002. 16(5): p. 501-526.
• [50] Tsai, W.-C. and A.-P. Chen. Global Asset Allocation Using XCS Experts in Country-Specific ETFs. in Convergence and Hybrid Information Technology, 2008. ICCIT ’08. Third International Conference on. 2008.
• [51] Sprogar, M., M. Sprogar, and M. Colnaric, Autonomous evolutionary algorithm in medical data analysis. Computer Methods and Programs in Biomedicine, 2005. 80(Supplement 1): p. S29-S38.
• [52] Passaro, A., F. Baronti, and V. Maggini, Exploring relationships between genotype and oral cancer development through XCS, in Proceedings of the 2005 workshops on Genetic and evolutionary computation2005, ACM:Washington, D.C. p. 147-151.
• [53] Baronti, F., et al., Machine learning contribution to solve prognostic medical, in Outcome prediction in cancer, A. Taktak and A.C. Fisher, Editors. 2007, Elsevier.
• [54] Bernauer, A., et al., Combining Software and Hardware LCS for Lightweight On-chip Learning, in Organic Computing — A Paradigm Shift for Complex Systems, C. Mller-Schloer, H. Schmeck, and T. Ungerer, Editors. 2011, Springer Basel. p. 253-265.
• [55] Bernauer, A., et al., Autonomous multi-processor-SoC optimization with distributed learning classifier systems XCS, in Proceedings of the 8th ACM international conference on Autonomic computing2011, ACM: Karlsruhe, Germany. p. 213-216.
• [56] Armano, G., NXCS Experts for Financial Time Series Forecasting, in Applications of Learning Classifier Systems, L. Bull, Editor 2004, Springer. p. 68-91.
• [57] Chen, A.-P. and Y.-H. Chang. Using extended classifier system to forecast S&P futures based on contrary sentiment indicators. in Evolutionary Computation, 2005. The 2005 IEEE Congress on. 2005.
• [58] Chen, A.-P., et al., Applying the Extended Classifier System to Trade Interest Rate Futures Based on Technical Analysis, in Proceedings of the 2008 Eighth International Conference on Intelligent Systems Design and Applications - Volume 032008, IEEE Computer Society. p. 598-603.
• [59] Shankar, A. and S.J. Louis, XCS for Personalizing Desktop Interfaces. Evolutionary Computation, IEEE Transactions on, 2010. 14(4): p. 547-560.
• [60] Butz, M. and S. Wilson, An Algorithmic Description of XCS, in Advances in Learning Classifier Systems, P. Luca Lanzi,W. Stolzmann, and S.Wilson, Editors. 2001, Springer Berlin / Heidelberg. p. 267-274. Wilson, S.W., Generalization in the XCS Classifier System, 1998.
• [61] Mikels, J., et al., Emotional category data on images from the international affective picture system. Behavior Research Methods, 2005. 37(4): p. 626-630.
• [62] Bradley, M.M. and P.J. Lang, Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 1994. 25: p. 49-59.
• [63] Cohen, I., et al., Facial expression recognition from video sequences: temporal and static modeling. Computer Vision and Image Understanding, 2003.91(1–2): p. 160-187.
• [64] Kim, K.H., S.W. Bang, and S.R. Kim, Emotion recognition system using short-term monitoring of physiological signals. Medical & Biological Engineering & Computing, 2004. 42(3): p. 419-427.
• [65] Lang, P.J., M.M. Bradley, and B.N. Cuthbert, International Affective Picture System (IAPS), in Techinal Manual and Affective Ratings1999, The Center for Research in Psychophysiology, University of Florida: Gainesville, FL.
• [66] Stalph, P.O. and M.V. Butz, Documentation of JavaXCSF, 2009: Retrieved from University of Wurzburg, Cognitive Bodyspaces: Learning and Behavior website.
• [67] Butz, M.V., P.L. Lanzi, and S.W. Wilson, Function Approximation With XCS: Hyperellipsoidal Conditions, Recursive Least Squares, and Compaction. Evolutionary Computation, IEEE Transactions on, 2008. 12(3): p. 355-376.
• [68] Hall, M., et al., The WEKA data mining software: an update. SIGKDD Explor. Newsl., 2009. 11(1): p. 10-18.
• [69] Lee, P.-M., Y. Teng, and T.-C. Hsiao. XCSF for prediction on emotion induced by image based on dimensional theory of emotion. in Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference companion. 2012. ACM.
• [70] Holland, J.H. and J.S. Reitman, Cognitive systems based on adaptive algorithms. SIGART Bull., 1977(63): p. 49-49.
• [71] Li, S. and B. Yang, Multifocus image fusion using region segmentation and spatial frequency. Image and Vision Computing, 2008. 26(7): p. 971-979.
• [72] Leonard, H., et al., Brief Report: Developing Spatial Frequency Biases for Face Recognition inAutism andWilliams Syndrome. Journal of Autism and Developmental Disorders, 2011. 41(7): p. 968-973.
• [73] John G, D., Two-dimensional spectral analysis of cortical receptive field profiles. Vision Research, 1980. 20(10): p. 847-856.
• [74] Webster, M.A. and R.L. De Valois, Relationship between spatial-frequency and orientation tuning of striate-cortex cells. J. Opt. Soc. Am. A, 1985. 2(7): p. 1124-1132.
• [75] Kobayashi, K., et al., Head and body sway in response to vertical visual stimulation. Acta Otolaryngologica, 2005. 125(8): p. 858-862.
• [76] Huang, N., et al., The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. Roy. Soc. Lond. A, 1998. 454: p. 903-995.
• [77] Tay, P.C. AM-FM Image Analysis Using the Hilbert Huang Transform. in Image Analysis and Interpretation, 2008. SSIAI 2008. IEEE Southwest Symposium on. 2008.
• [78] Caseiro, P., R. Fonseca-Pinto, and A. Andrade, Screening of obstructive sleep apnea using Hilbert–Huang decomposition of oronasal airway pressure recordings. Medical Engineering & Physics, 2010. 32(6): p. 561-568.
• [79] Kaw, A. and E. Kalu, Numerical Methods with Applications2010: autarkaw.