A toolkit for storage QoS provisioning for data-intensive applications

• Autorzy: Słota R. , Król D. , Skałkowski K. , Orzechowski M. , Nikolow D. , Kryza B. , Wrzeszcz M. , Kitowski J.
• Języki publikacji: EN

Abstrakty

EN

This paper describes a programming toolkit developed in the PL-Grid project, named QStorMan, which supports storage QoS provisioning for data-intensive applications in distributed environments. QStorMan exploits knowledgeoriented methods for matching storage resources to non-functional requirements, which are defined for a data-intensive application. In order to support various usage scenarios, QStorMan provides two interfaces, such as programming libraries or a web portal. The interfaces allow to define the requirements either directly in an application source code or by using an intuitive graphical interface. The first way provides finer granularity, e.g., each portion of data processed by an application can define a different set of requirements. The second method is aimed at legacy applications support, which source code can not be modified. The toolkit has been evaluated using synthetic benchmarks and the production infrastructure of PL-Grid, in particular its storage infrastructure, which utilizes the Lustre file system.

Słowa kluczowe

EN

data-intensive application; storage management; QoS; grid

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2012
• Tom: Vol. 13 (1)
• Strony: 63--73
• Opis fizyczny: Bibliogr. 18 poz., rys., tab., wykr.

Twórcy

autor

Słota R.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland
• AGH University of Science and Technology, ACC Cyfronet AGH, Kraków, Poland

autor

Król D.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland

autor

Skałkowski K.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland
• AGH University of Science and Technology, ACC Cyfronet AGH, Kraków, Poland

autor

Orzechowski M.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland

autor

Nikolow D.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland
• AGH University of Science and Technology, ACC Cyfronet AGH, Kraków, Poland

autor

Kryza B.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland

autor

Wrzeszcz M.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland

autor

Kitowski J.

• AGH University of Science and Technology, Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer Science, Krakow, Poland
• AGH University of Science and Technology, ACC Cyfronet AGH, Kraków, Poland

Bibliografia

• [1] Kryza B., Dutka L., Slota R., Kitowski J.: Dynamic VO Establishment in Distributed Heterogeneous Business Environment. [in:] G. Allen, J. Nabrzyski, E. Seidel, G.D. van Albada, J. Dongarra, and P. M. A. Sloot (Eds.), LNCS, vol 5545(2009) pp. 709–718.
• [2] The PL-Grid project website. [on-line: www.plgrid.pl, as of December 16, 2011]
• [3] Google Processing 20 Petabytes a Day, [on-line: www.datacenterknowledge. com/archives/2008/01/09/google-processing-20-petabytes-a-day/, as of December 16, 2011].
• [4] German Climate Computing Center (DKRZ) website, [on-line: http://www.dkrz.de/daten-en, as of December 16, 2011].
• [5] Deelman E., Chervenak A.: Data Management Challenges of Data-Intensive Scientific Workflows. Proc. of the 8th IEEE/ACMInternational Symposium on Cluster, Cloud and Grid Computing, 2008, pp. 687–692.
• [6] Ekanayake J., Li H., Zhang B., Gunarathne T., Bae S., Qiu J., Fox G.: Twister: a runtime for iterative MapReduce. Proc. of the International Symposium on High Performance Distributed Computing, 2010, pp. 810–818.
• [7] Raicu I., Foster I., Zhao Y., Little P., Moretti C., Chaudhary A., Thain D.: The quest for scalable support of data-intensive workloads in distributed systems. Proc. of the International Symposium on High Performance Distributed Computing, 2009, pp. 207–216. A toolkit for storage QoS provisioning for data-intensive applications 71
• [8] Paszyński M., Pardo D., Torres-Verd´ın C., Demkowicz L., Calo A.: A parallel direct solver for the self-adaptive hp Finite Element Method. Journal of Parallel and Distributed Computing, vol. 70(3) (2010) pp. 270–281.
• [9] Cettei M., Ligon W., Ross R.: Support for parallel out of core applications on Beowulf workstations. Proc. of Aerospace Conference, vol. 4(1998), pp. 355–365.
• [10] Kimura H., Tatebe O.: MPI-IO/Gfarm: An Optimized Implementation of MPIIO for the Gfarm File System. Cluster, Cloud and Grid Computing (CCGrid), May 2011, doi: 10.1109/CCGrid.2011.82, pp. 610–611.
• [11] Dickens P., Logan J.: Y-Lib: A User Level Library to Increase the Performance of MPI-IO in a Lustre File System Environment. Proc. of the 18th ACM international symposium on High performance distributed computing, 2009, Garching, Germany, ACM, pp. 31–38.
• [12] The Lustre file system website: [on-line: wiki.lustre.org, as of June 20, 2011].
• [13] Roy A. J.: End-to-end quality of service for high-end applications. PhD thesis. Adviser – Ian Foster, 2001.
• [14] Krol D., Kryza B., Skalkowski K., Nikolow D., Slota R., Kitowski J.: QoS Provisioning for Data-Oriented Applications in PL-GRID. [in:] M. Bubak, M. Turala, K. Wiatr (Eds.), Proc. of Cracow Grid Workshop - CGW’10, October 2010, ACC Cyfronet AGH, 2011, Krakow, pp. 142–150.
• [15] Slota R., Krol D., Skalkowski K., Kryza B., Nikolow D., Kitowski J.: FiVO /QStorMan: toolkit for supporting data-oriented applications in PL-Grid. Proc.of KU KDM 2011, March, 2011, Zakopane, ACC Cyfronet AGH, ISBN 978-83- 61-433-04-0, pp. 68.
• [16] Slota R., Krol D., Funika W., Kryza B., Nikolow D., Kitowski J.: FiVO/QStor- Man Semantic Toolkit for Supporting Data-Intensive Applications in Distributed Environments. Computing and Informatics, 2011, in press.
• [17] Slota R., Skital L., Nikolow D., Kitowski J.: Algorithms for Automatic Data Replication in Grid Environment. 6th international conference, PPAM 2005. Pozna´n, Poland, September 11–14, 2005, eds. Roman Wyrzykowski et al., Berlin, Heidelberg, Springer-Verlag, 2006, LNCS, vol. 3911, pp. 707–714.
• [18] Nikolow D., Slota R., Dziewierz M., Kitowski J.: Access time estimation for tertiary storage systems, Euro-Par 2002, Paderborn, Germany, August 27–30, 2002, Proc., eds. Burkhard Monien, Rainer Feldmann, Berlin, Springer, 2002, LNCS, vol. 2400, pp. 873–880.