Eye and EEG activity markers for visual comfort level of images

• Autorzy: Abromavičius V. , Serackis A.

Identyfikatory

DOI

10.1016/j.bbe.2018.08.001

• Języki publikacji: EN

Abstrakty

EN

Depth perception by binocular cues is based on the matching of image features from one retina with corresponding elements from the second retina. However, high disparities are related to the higher visual discomfort levels and may cause the eye fatigue during extended stereoscopic perception time. The goal of the investigation was to find a set of measurable features for stereoscopic image visual comfort level prediction. The investigation involved gaze, pupillometric and EEG data from 28 subjects who evaluated visual comfort level of 120 stereoscopic images. Six different time frame windows were used to analyze four measured features: the number of focus points; the dynamics of pupil size; disparity level at the focus points; the activity of EEG bands at the frontal lobe. A significant difference was found in all investigated stereoscopic image groups. 2-s and 5-s pre-DPI window showed best results for the selected feature sets. The higher disparity at the focus points, lower number of focus points are related to the lower levels of visual comfort. However, features such as the number of focus points, the pupil size and the disparity level for the images with lowest visual comfort scores showed similar results to the images scored as ‘‘comfortable’’ or ‘‘very comfortable’’.

Słowa kluczowe

EN

eye tracking; visual comfort; electroencephalogram

PL

okulografia; komfort wizualny; elektroencefalogram

• Wydawca: Nałęcz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences ; Elsevier
• Czasopismo: Biocybernetics and Biomedical Engineering
• Rocznik: 2018
• Tom: Vol. 38, no. 4
• Strony: 810--818
• Opis fizyczny: Bibliogr. 47 poz., tab., wykr.

Twórcy

autor

Abromavičius V.

• Department of Electronic Systems, Vilnius Gediminas Technical University, Naugarduko g. 41–413, Vilnius, Lithuania

autor

Serackis A.

• Department of Electronic Systems, Vilnius Gediminas Technical University, Vilnius, Lithuania

Bibliografia

• [1] Chang Y-S, Hsueh Y-H, Tung K-C, Jhou F-Y, Lin DP-C. Characteristics of visual fatigue under the effect of 3D animation. Technol Health Care 2016;24(s1):S231–5.
• [2] Curcio ID, Toukomaa H, Naik D. 360-degree video streaming and its subjective quality. SMPTE 2017 Annual Technical Conference and Exhibition, SMPTE. 2017. pp. 1–23.
• [3] Koulieris G-A, Bui B, Banks MS, Drettakis G. Accommodation and comfort in head-mounted displays. ACM Trans Graphics (TOG) 2017;36(4):87.
• [4] Terzic K, Hansard M. Causes of discomfort in stereoscopic content: a review, arXiv preprint arXiv:1703.04574 (2017) 1–22.
• [5] Wang J, Liang H, Xu Y. P-30: research on the relationship between visual fatigue and stereoscopic parallax. SID Symposium Digest of Technical Papers, Vol. 47; 2016. pp. 1234–6.
• [6] Bülthoff I, Bülthoff H, Sinha P. Top-down influences on stereoscopic depth-perception. Nat Neurosci 1998;1(3):254.
• [7] Speranza F, Tam WJ, Renaud R, Hur N. Effect of disparity and motion on visual comfort of stereoscopic images. Stereoscopic Displays and Virtual Reality Systems XIII, Vol. 6055; 2006. p. 60550B.
• [8] Terzić K, Hansard M. Methods for reducing visual discomfort in stereoscopic 3D: a review. Signal Process: Image Commun 2016;47:402–16.
• [9] Hoffman DM, Girshick AR, Akeley K, Banks MS. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. J Vision 2008;8(3):33.
• [10] Kooi FL, Toet A. Visual comfort of binocular and 3D displays. Displays 2004;25(2–3):99–108.
• [11] Wöpking M. Viewing comfort with stereoscopic pictures: an experimental study on the subjective effects of disparity magnitude and depth of focus. J Soc Inf Display 1995;3(3):101–3.
• [12] Lambooij M, Fortuin M, Heynderickx I, IJsselsteijn W. Visualdiscomfort and visual fatigue of stereoscopic displays: a review. J Imaging Sci Technol 2009;53(3). 30201–1.
• [13] Lin CJ, Widyaningrum R. The effect of parallax on eye fixation parameter in projection-based stereoscopic displays. Appl Ergon 2018;69:10–6.
• [14] Iatsun I, Larabi M-C, Fernandez-Maloigne C. Investigation and modeling of visual fatigue caused by S3D content using eye-tracking. Displays 2015;39:11–25.
• [15] Bernhard M, Dell'mour C, Hecher M, Stavrakis E, Wimmer M. The effects of fast disparity adjustment in gaze-controlled stereoscopic applications. Proceedings of the Symposium on Eye Tracking Research and Applications, ACM; 2014. p. 111–8.
• [16] Kim H, Lee S. Transition of visual attention assessment in stereoscopic images with evaluation of subjective visual quality and discomfort. IEEE Trans Multimedia 2015;17(12):2198–209.
• [17] Moon S-E, Lee J-S. Implicit analysis of perceptual multimedia experience based on physiological response: a review. IEEE Trans Multimedia 2017;19(2):340–53.
• [18] Frey J, Appriou A, Lotte F, Hachet M. Classifying EEG signals during stereoscopic visualization to estimate visual comfort. Comput Intell Neurosci 2016;2016:7.
• [19] Huang K-C, Huang T-Y, Chuang C-H, King J-T, Wang Y-K, Lin C-T, et al. An EEG-based fatigue detection and mitigation system. Int J Neural Syst 2016;26(04):1650018.
• [20] Fischmeister FPS, Bauer H. Neural correlates of monocular and binocular depth cues based on natural images: a loreta analysis. Vision Res 2006;46(20):3373–80.
• [21] Fazlyyyakhmatov M, Zwezdochkina N, Antipov V. The EEG activity during binocular depth perception of 2D images. Comput Intell Neurosci 2018.
• [22] Zou B, Liu Y, Guo M, Wang Y. EEG-based assessment of stereoscopic 3D visual fatigue caused by vergence-accommodation conflict. J Display Technol 2015;11(12):1076–83.
• [23] Kim Y-J, Lee EC. EEG based comparative measurement of visual fatigue caused by 2D and 3D displays. International Conference on Human–Computer Interaction; 2011. pp. 289–92.
• [24] Chen C, Li K, Wu Q, Wang H, Qian Z, Sudlow G. EEG-based detection and evaluation of fatigue caused by watching 3DTV. Displays 2013;34(2):81–8.
• [25] Jung YJ, Kim D, Sohn H, Lee S-I, Park HW, Ro YM. Towards a physiology-based measure of visual discomfort: brain activity measurement while viewing stereoscopic images with different screen disparities. J Display Technol 2015;11(9):730–43.
• [26] Hou C, Yue G, Shen L. Assessing the visual discomfort of compressed stereoscopic images using ERP. International Conference on Human Centered Computing; 2016. pp. 127–37.
• [27] Amin HU, Malik AS, Badruddin N, Kamel N, Hussain M. Effects of stereoscopic 3D display technology on eventrelated potentials (ERPS). Neural Engineering (NER), 2015 7th International IEEE/EMBS Conference on, IEEE. 2015. pp. 1084–7.
• [28] Mun S, Park M-C, Park S, Whang M. SSVEP and ERP measurement of cognitive fatigue caused by stereoscopic 3D. Neurosci Lett 2012;525(2):89–94.
• [29] Cho H, Kang M-K, Yoon K-J, Jun SC. Feasibility study for visual discomfort assessment on stereo images using EEG. 3D Imaging (IC3D), 2012 International Conference on, IEEE. 2012. pp. 1–6.
• [30] Lee S-I, Lee SH, Plataniotis KNK, Ro YM. Experimental investigation of facial expressions associated with visual discomfort: feasibility study toward an objective measurement of visual discomfort based on facial expression. J Display Technol 2016;12(12):1785–97.
• [31] Sohn H, Jung Y, Lee S, et al. Ivy lab stereo 3D image database for disparity remapping [internet]; 2015 [cited 15.03.18] http://ivylab.kaist.ac.kr/demo/3DVCA/3DVCA.htm.
• [32] Jung YJ, Sohn H, Lee S-I, Park HW, Ro YM. Predicting visual discomfort of stereoscopic images using human attention model. IEEE Trans Circuits Syst Video Technol 2013;23(12):2077–82.
• [33] Xu H, Jiang G, Yu M, Luo T, Peng Z, Shao F, et al. 3D visual discomfort predictor based on subjective perceived-constraint sparse representation in 3D display system. Future Gener Comput Syst 2018;83:85–94.
• [34] Jiang Q, Shao F, Lin W, Jiang G. On predicting visual comfort of stereoscopic images: a learning to rank based approach. IEEE Signal Process Lett 2016;23(2):302–6.
• [35] Jiang Q, Shao F, Jiang G, Yu M, Peng Z. Leveraging visual attention and neural activity for stereoscopic 3D visual comfort assessment. Multimedia Tools Appl 2017;76(7):9405–25.
• [36] Jiang Q, Shao F, Jiang G, Yu M, Peng Z. Visual comfort assessment for stereoscopic images based on sparse coding with multi-scale dictionaries. Neurocomputing 2017;252:77–86.
• [37] Jiang Q, Shao F, Jiang G, Yu M, Peng Z. Three-dimensional visual comfort assessment via preference learning. J Electron Imaging 2015;24(4):043002.
• [38] Union I. Subjective methods for the assessment of stereoscopic 3DTV systems, Recommendation ITU-R BT, 2021.
• [39] Pelli DG. The videotoolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision 1997;10(4):437–42.
• [40] Yoon H, Park S-W, Lee Y-K, Jang J-H. Emotion recognition of serious game players using a simple brain computer interface. ICT Convergence (ICTC), 2013 International Conference on, IEEE. 2013. pp. 783–6.
• [41] Ursuţiu D, Samoilă C, Drăgulin S, Constantin FA. Investigation of music and colours influences on the levels of emotion and concentration. Online Engineering & Internet of Things; 2018. pp. 910–8.
• [42] Chen C-M, Wang J-Y, Yu C-M. Assessing the attention levels of students by using a novel attention aware system based on brainwave signals. Br J Educ Technol 2017;48(2):348–69.
• [43] Lin F-R, Kuo C-M, Mental effort detection using EEG data in e-learning contexts, Comput Educ (2018).
• [44] Carnegie K, Rhee T. Reducing visual discomfort with HMDS using dynamic depth of field. IEEE Comput Graph Appl 2015;35(5):34–41.
• [45] Castellanos NP, Makarov VA. Recovering EEG brain signals: artifact suppression with wavelet enhanced independent component analysis. J Neurosci Methods 2006;158(2):300–12.
• [46] Oostenveld R, Fries P, Maris E, Schoffelen J-M. Fieldtrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011;2011:1.
• [47] Khaustova D, Fournier J, Wyckens E, Le Meur O. An investigation of visual selection priority of objects with texture and crossed and uncrossed disparities. Human Vision and Electronic Imaging XIX, Vol. 9014; 2014. p. 90140D.