Visual attention pooling and understanding the structural similarity index in multi-scale analysis

• Autorzy: Bondzulic B. P. , Petrovic V. S. , Mitrovic S. T. , Pavlovic B. Z. , Andric M. S.

Identyfikatory

DOI

10.5277/oa140208

• Języki publikacji: EN

Abstrakty

EN

We present a novel spatial pooling strategy and the results of an extensive multi-scale analysis of the well-known structural similarity index metric (SSIM) for objective image quality evaluation. We show, in contrast with some previous studies, that even relatively simple perceptual importance pooling strategies can significantly improve objective metric performance evaluated as the correlation with subjective quality assessment. In particular, we define an attention and quality driven pooling mechanism that focuses structural comparisons within the SSIM model to only those pixels exhibiting significant structural degradations. We show that optimal objective metric performance is achieved over very sparse spatial domains indeed that ignore most of the signal data. We also investigate an explicit breakdown of the structural models within SSIM and show that in combination with the proposed attention and quality driven pooling some of these models represent well performing metrics in their own right, when applied at appropriate scale for which there may not be a single optimal value. Our experiments demonstrate that the augmented SSIM metric using the proposed pooling model provides performance advantage on an extensive LIVE dataset covering hundreds of degraded images and 5 different distortion types compared to both conventional SSIM and state-of-the-art objective quality metrics.

Słowa kluczowe

EN

image quality assessment; structural similarity index; visual attention pooling

• Wydawca: Oficyna Wydawnicza Politechniki Wrocławskiej
• Czasopismo: Optica Applicata
• Rocznik: 2014
• Tom: Vol. 44, nr 2
• Strony: 267--283
• Opis fizyczny: Bibliogr., 26 poz., rys., tab., wykr.

Twórcy

autor

Bondzulic B. P.

• Military Academy, University of Defence in Belgrade, Generala Pavla Jurisica Sturma 33, 11000 Belgrade, Serbia

autor

Petrovic V. S.

• Imaging Science and Biomedical Engineering, University of Manchester, Oxford Rd, Manchester, M13 9PT, UK

autor

Mitrovic S. T.

• Military Academy, University of Defence in Belgrade, Generala Pavla Jurisica Sturma 33, 11000 Belgrade, Serbia

autor

Pavlovic B. Z.

• Military Academy, University of Defence in Belgrade, Generala Pavla Jurisica Sturma 33, 11000 Belgrade, Serbia

autor

Andric M. S.

• Military Academy, University of Defence in Belgrade, Generala Pavla Jurisica Sturma 33, 11000 Belgrade, Serbia

Bibliografia

• [1] WANG Z., Applications of objective image quality assessment methods, IEEE Signal Processing Magazine 28(6), 2011, pp. 137–142.
• [2] WANG Z., BOVIK A.C., Mean squared error: love it or leave it? A new look at signal fidelity measures,IEEE Signal Processing Magazine 26(1), 2009, pp. 98–117.
• [3] SHNAYDERMAN A., GUSEV A., ESKICIOGLU A.M., An SVD-based gray-scale image quality measure for local and global assessment, IEEE Transactions on Image Processing 15(2), 2006, pp. 422–429.
• [4] WANG Z., BOVIK A.C., SHEIKH H.R., SIMONCELLI E.P., Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13(4), 2004, pp. 600–612.
• [5] WANG Z., BOVIK A.C., A universal image quality index, IEEE Signal Processing Letters 9(3), 2002, pp. 81–84.
• [6] WANG Z., SIMONCELI E.P., BOVIK A.C., Multi-scale structural similarity for image quality assessment, Conference Record of the 37th IEEE Asilomar Conference on Signals, Systems and Computers, 2003, pp. 1398–1402.
• [7] CHEN G.-H., YANG C.-L., XIE S.-L., Gradient-based structural similarity for image quality assessment, Proceedings of the IEEE International Conference on Image Processing, 2006, pp. 2929–2932.
• [8] CUI L., ALLEN A.R., An image quality metric based on corner, edge and symmetry maps, Proceedings of the 19th British Machine Vision Conference, 2008, pp. 1–10.
• [9] ZHAO X., REYES M.G., PAPPAS T.N., NEUHOFF D.L., Structural texture similarity metrics for retrieval applications, Proceedings of the 15th IEEE International Conference on Image Processing, 2008, pp. 1196–1199.
• [10] WANG Z., SHANG X., Spatial pooling strategies for perceptual image quality assessment, Proceedings of the IEEE International Conference on Image Processing, 2006, pp. 2945–2948.
• [11] MOORTHY A.K., BOVIK A.C., Visual importance pooling for image quality assessment, IEEE Journal of Selected Topics in Signal Processing 3(2), 2009, pp. 193–201.
• [12] BONDZULIC B., PETROVIC V., Additive models and separable pooling, a new look at structural similarity, Signal Processing 97, 2014, pp. 110–116.
• [13] LI C., BOVIK A.C., Content-partitioned structural similarity index for image quality assessment, Signal Processing: Image Communication 25(7), 2010, pp. 517–526.
• [14] WANG Z., LI Q., Information content weighting for perceptual image quality assessment, IEEE Transactions on Image Processing 20(5), 2011, pp. 1185–1198.
• [15] NINASSI A., MEUR O.L., CALLET P.L., BARBA D., Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric, Proceedings of the IEEE International Conference on Image Processing, 2007, pp. II-169–II-172.
• [16] SHEIKH H.R., SABIR M.F., BOVIK A.C., A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Transactions on Image Processing 15(11), 2006, pp. 3441–3452.
• [17] TOURANCHEAU S., AUTRUSSEAU F., SAZZAD Z.M., HORITA Y., Impact of subjective dataset on the performance of image quality metrics, Proceedings of the 15th IEEE International Conference on Image Processing, 2008, pp. 365–368.
• [18] LARSON E.C., CHANDLER D.M., Most apparent distortion: full-reference image quality assessment and the role of strategy, Journal of Electronic Imaging 19(1), 2010, article 011006.
• [19] ROUSE D.M., HEMAMI S.S., Understanding and simplifying the structural similarity metric, Proceedings of the 15th IEEE International Conference on Image Processing, 2008, pp. 1188–1191.
• [20] PARK J., SESHADRINATHAN K., LEE S., BOVIK A.C., Video quality pooling adaptive to perceptual distortion severity, IEEE Transaction on Image Processing 22(2), 2013, pp. 610–620.
• [21] SHEIKH H.R., WANG Z., CORMACK L., BOVIK A.C., LIVE image quality assessment database, http://live.ece.utexas.edu/research/quality/subjective.htm, 08.03.2013.
• [22] Tutorial, I.T.U.T., Objective perceptual assessment of video quality – full reference television, ITUT Telecommunication Standardization Bureau ITU-T, 2004.
• [23] LARSON E.C., CHANDLER D.M., The CSIQ image database, http://vision.okstate.edu/?loc=csiq,08.03.2013.
• [24] LE CALLET P., AUTRUSSEAU F., Subjective quality assessment IRCCyN/IVC database, http://www2.irccyn.ec-nantes.fr/ivcdb/, 08.03.2013.
• [25] PARVEZ SAZZAD Z.M., KAWAYOKE Y., HORITA Y., MICT image quality evaluation database, http://mict.eng.u-toyama.ac.jp/mictdb.html, 08.03.2013.
• [26] CHANDLER D.M., HEMAMI S.S., VSNR: A wavelet-based visual signal-to-noise ratio for natural images, IEEE Transactions on Image Processing 16(9), 2007, pp. 2284–2298.