Timbra : an online tool for feature extraction, comparative analysis and visualization of timbre

• Autorzy: Szymański Filip , Łukasik Ewa , Chudy Magdalena

Identyfikatory

DOI

10.24425/ijet.2024.149551

• Języki publikacji: EN

Abstrakty

EN

Dariah.lab is a research infrastructure created for digital humanities, consisting of state-of-the-art hardware and dedicated software tools. One of the tools developed for digital musicology is Timbra, a web-based application for conducting research on sound timbre. The aim was to create an easy-to-use online tool for non-programmers. The tool can be used to calculate, visualise, and compare different timbre characteristics of uploaded audio files and to export the extracted parameters in CSV format for further processing, e.g. by classification tools. The application offers extraction and visualisation of scalar features such as zero crossing rate, fundamental frequency, spectral centroid, spectral roll-off, spectral flatness, band energy ratio, as well as feature vectors (e.g. chromagram, spectral contrast, spectrogram, and MFCCs). An interested user can compare selected sound characteristics using various types of plots and run dissimilarity analysis of timbre parameters by means of 2D or 3D multidimensional scaling (MDS). The paper showcases potential applications of the tool based on presented case studies. In terms of implementation, the calculations are performed at the backend Django server using Librosa and standard Python libraries. Dash library is used for the frontend. By offering an easy-to-use tool accessible anytime and anywhere through the Internet, we want to facilitate timbre analysis for a broader group of researchers, e.g. sound engineers, luthiers, phoneticians, or musicologists.

Słowa kluczowe

EN

timbre; audio descriptors; feature extraction; comparative analysis; visualization

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2024
• Tom: Vol. 70, No. 2
• Strony: 349--354
• Opis fizyczny: Bibliogr. 33 poz., rys., wykr.

Twórcy

autor

Szymański Filip

• Poznan University of Technology, Poznan, Poland

autor

Łukasik Ewa

• Poznan University of Technology, Poznan, Poland

autor

Chudy Magdalena

• Institute of Art, Polish Academy of Sciences, Warsaw, Poland

Bibliografia

• [1] Dariah.lab. (2023) Digital Research Infrastructure for the Arts and Humanities. [Online]. Available: https://lab.dariah.pl/en/
• [2] H. Helmholtz, On the Sensations of Tone: As a Physiological Basis for the Theory of Music, 4th ed. London: Longmans, Green & Co., 1877, (1912). Translated by Ellis, A. J. [Online]. Available: https://doi.org/10.1017/CBO9780511701801.
• [3] ANSI, American Standard Acoustical Terminology, Definition 12.9, Timbre, New York, 1960. [Online]. Available: https://asastandards.org/terms/timbre/
• [4] J. M. Grey, “Multidimensional perceptual scaling of musical timbres,” Journal of the Acoustical Society of America, vol. 61, no. 5, pp. 1270-1277, 1977. [Online]. Available: https://doi.org/10.1121/1.381428.
• [5] J. M. Grey and J. W. Gordon, “Perceptual effects of spectral modifications on musical timbres,” Journal of the Acoustical Society of America, vol. 63, no. 5, pp. 1493-1500, 1978. [Online]. Available: https://doi.org/10.1121/1.381843.
• [6] C. L. Krumhansl, “Why is musical timbre so hard to understand?” in Structure and Perception of Electroacoustic Sound and Music, S. Nielzen and O. Olsson, Eds., Elsevier, Amsterdam, 1989, pp. 43-53. [Online]. Available: http://music.psych.cornell.edu/articles/timbre/Why_Is Musical Timbre so%20hard to understand.pdf
• [7] P. Iverson and C. L. Krumhansl, “Isolating the dynamic attributes of musical timbre,” Journal of the Acoustical Society of America, vol. 94, no. 5, pp. 2595-2603, 1993. [Online]. Available: https://doi.org/10.1121/1.407371.
• [8] S. McAdams, S. Winsberg, S. Donnadieu, G. Desoete, and J. Krimphoff, “Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes,” Psychological Research, vol. 58, no. 3, pp. 177-192, 1995. [Online]. Available: https://doi.org/10.1007/BF00419633.
• [9] S. McAdams, B. L. Giordano, P. Susini, G. Peeters, and V. Rioux, “A meta-analysis of acoustic correlates of timbre dimensions (A),” Journal of the Acoustical Society of America, vol. 120, no. 5, p. 3275, 2006. [Online]. Available: https://doi.org/10.1121/1.4777215.
• [10] B. L. Giordano and S. McAdams, “Sound source mechanics and musical timbre perception: Evidence from previous studies,” Music Perception, vol. 28, no. 2, pp. 155-168, 2010. [Online]. Available: https://doi.org/10.1525/mp.2010.28.2.155.
• [11] R. A. Kendall, E. C. Carterette, and J. M. Hajda, “Perceptual and acoustical features of natural and synthetic orchestral instrument tones,” Music Perception, vol. 16, no. 3, pp. 327-363, 1999. [Online]. Available: https://doi.org/10.2307/40285796.
• [12] S. Lakatos, “A common perceptual space for harmonic and percussive timbres,” Perception & Psychophysics, vol. 62, no. 7, pp. 1426-1439, 2000. [Online]. Available: https://link.springer.com/content/pdf/10.3758/BF03212144.pdf
• [13] R. A. Fitzgerald, “Performer-dependent dimensions of timbre: identifying acoustic cues for oboe tone discrimination,” Ph.D. dissertation, School of Music, University of Leeds, 2003.
• [14] M. Barthet, P. Guillemain, R. Kronland-Martinet, and S. Ystad, “From clarinet control to timbre perception,” Acta Acustica united with Acustica, vol. 96, no. 4, pp. 678-689, 2010. [Online]. Available: https://doi.org/10.3813/AAA.918322.
• [15] M. Chudy, “Discriminating music performers by timbre: On the relation between instrumental gesture, tone quality and perception in classical cello performance,” PhD thesis, School of Electronic Engineering and Computer Science, Queen Mary University of London, 2016. [Online]. Available: https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/18378/CHUDYMagdalenaPhD120916.pdf?sequence=1
• [16] H.-G. Kim, N. Moreau, and T. Sikora, MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval. Chichester, UK: Wiley, 2005. [Online]. Available: https://doi.org/10.1002/0470093366.
• [17] G. Peeters, B. L. Giordano, P. Susini, N. Misdariis, and S. McAdams, “The Timbre Toolbox: Extracting audio descriptors from musical signals,” Journal of the Acoustical Society of America, vol. 130, no. 5, pp. 2902-2916, 2011. [Online]. Available: https://doi.org/10.1121/1.3642604.
• [18] O. Lartillot, “Miningsuite: A comprehensive Matlab framework for signal, audio and music analysis, articulating audio and symbolic approaches,” in Proceedings of the 16th Sound & Music Computing Conference, Malaga, Spain, 2019. [Online]. Available: https://doi.org/10.5281/zenodo.3249435.
• [19] O. Lartillot, P. Toiviainen, and T. Eerola, “A Matlab toolbox for Music Information Retrieval,” in Data Analysis, Machine Learning and Applications. Studies in Classification, Data Analysis, and Knowledge Organization, C. Preisach, H. Burkhardt, L. Schmidt-Thieme, and R. Decker, Eds. Springer-Verlag, 2008, pp. 261-268. [Online]. Available: https://doi.org/10.1007/978-3-540-78246-9 31.
• [20] D. Bogdanov, N. Wack, E. Gómez, S. Gulati, P. Herrera, O. Mayor, and et al., “ESSENTIA: An audio analysis library for Music Information Retrieval,” in Proceedings of the 14th International Society for Music Information Retrieval Conference (ISMIR’13), 2013, pp. 493-498, http://essentia.upf.edu/index.html. [Online]. Available: https://doi.org/10.1145/2502081.2502229.
• [21] G. Tzanetakis and P. Cook, “MARSYAS: A framework for audio analysis,” Organised Sound, vol. 4, no. 3, p. 169-175, 2000, https://github.com/marsyas/marsyas. [Online]. Available: https://doi.org/10.1017/S1355771800003071.
• [22] P. M. Brossier, “The aubio library at MIREX 2006,” in Music Information Retrieval Evaluation eXchange (MIREX), 2006, https://aubio.org/. [Online]. Available: https://www.music-ir.org/mirex/abstracts/2006/MIREX2006Abstracts.pdf.
• [23] J. Bullock, “Libxtract: A lightweight library for audio feature extraction,” in Proceedings of the 2007 International Computer Music Conference, Copenhagen, Denmark, 2007, pp. 25-28, https://www.jamiebullock.com/LibXtract/documentation/. [Online]. Available: https://quod.lib.umich.edu/i/icmc/bbp2372.2007.116/1
• [24] B. Mathieu, S. Essid, T. Fillon, J. Prado, and G. Richard, “YAAFE, an easy to use and efficient audio feature extraction software,” in Proceedings of the 11th International Conference on Music Information Retrieval (ISMIR’10), Utrecht, Netherlands, 2010, p. 441-446, http://yaafe.github.io/Yaafe/. [Online]. Available: https://ismir2010.ismir.net/proceedings/ismir2010-75.pdf
• [25] C. McKay, I. Fujinaga, and P. Depalle, “jAudio: A feature extraction library,” in Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR’05), London, UK, 2005, p. 600-603, https://jmir.sourceforge.net/jAudio.html. [Online]. Available: https://ismir2005.ismir.net/proceedings/2103.pdf
• [26] B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Battenberg, and O. Nieto, “Librosa: Audio and music signal analysis in Python,” in Proceedings of the 14th Python in Science Conference, 2015, https://librosa.org/. [Online]. Available: http://conference.scipy.org.s3-website-us-east-1.amazonaws.com/proceedings/scipy2015/brian mcfee.html
• [27] H. Rawlinson, N. Segal, and J. Fiala, “Meyda: An audio feature extraction library for the Web Audio API,” in Proceedings of the 1st Web Audio Conference, Paris, France, 2015, https://meyda.js.org/. [Online]. Available: https://webaudioconf.com/posts/2015 17/
• [28] D. Moffat, D. Ronan, and J. D. Reiss, “An evaluation of audio feature extraction toolboxes,” in Proceedings of the 18th International Conference on Digital Audio Effects (DAFx-15), Trondheim, Norway, 2015. [Online]. Available: https://www.dafx.de/paper-archive/2015/DAFx-15 submission 43 v2.pdf
• [29] Django. (2023) The web framework for perfectionists with deadlines. [Online]. Available: https://www.djangoproject.com/
• [30] Dash. (2023) The low-code framework for rapidly building data apps in Python. [Online]. Available: https://dash.plotly.com/
• [31] Plotly. (2023) Low-code data apps. [Online]. Available: https://plotly.com/
• [32] M. Mrozik, “VST software synthesizer imitating sounds of recorded musical instruments,” Master’s thesis, Poznan University of Technology, 2023, E. Łukasik, supervisor.
• [33] X. Serra and J. Smith III, “Spectral modeling synthesis: A sound analysis/synthesis system based on a deterministic plus stochastic decomposition,” Computer Music Journal, vol. 14, pp. 12-14, winter 1990. [Online]. Available: https://doi.org/10.2307/3680788.