Policy-based sla storagemanagement model fordistributeddata storage services

• Autorzy: Nikolow D. , Słota R. , Polak S. , Pogoda M. , Kitowski J.

Identyfikatory

DOI

10.7494/csci.2018.19.4.2878

• Języki publikacji: EN

Abstrakty

EN

There is high demand for storage related services supporting scientists in their research activities. Those services are expected to provide not only capacity but also features allowing for more exible and cost ecient usage. Such features include easy multiplatform data access, long term data retention, support for performance and cost dierentiating of SLA restricted data access. The paper presents a policy-based SLA storage management model for distributed data storage services. The model allows for automated management of distributed data aimed at QoS provisioning with no strict resource reservation. The problem of providing users with the required QoS requirements is complex, and therefore the model implements heuristic approach for solving it. The corresponding system architecture, metrics and methods for SLA focused storage management are developed and tested in a real, nationwide environment. Keywords distributed storage systems, quality of service, service level agreement, management model, heuristics.

Słowa kluczowe

EN

distributed storage systems; quality of service; service level agreement; management model; heuristics

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2018
• Tom: Vol. 19 (4)
• Strony: 403--429
• Opis fizyczny: Bibliogr. 57 poz., rys., tab.

Twórcy

autor

Nikolow D.

• AGH University of Science and Technology, ACC Cyfronet AGH, ul. Nawojki 11, 30-950 Krakow, Poland
• School of Banking and Management, al. Kijowska 14, 30-079 Krakow, Poland

autor

Słota R.

• AGH University of Science and Technology, ACC Cyfronet AGH, ul. Nawojki 11, 30-950 Krakow, Poland
• AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications, Department of Computer Science, al. A. Mickiewicza 30, 30-059 Krakow, Poland

autor

Polak S.

• AGH University of Science and Technology, Faculty of Computer Science, Electronics and Telecommunications, Department of Computer Science, al. A. Mickiewicza 30, 30-059 Krakow,Poland

autor

Pogoda M.

• AGH University of Science and Technology, ACC Cyfronet AGH, ul. Nawojki 11, 30-950 Krakow, Poland and AGH University of Science

autor

Kitowski J.

• AGH University of Science and Technology, ACC Cyfronet AGH, ul. Nawojki 11, 30-950 Krakow, Poland
• School of Banking and Management, al. Kijowska 14, 30-079 Krakow, Poland

Bibliografia

• [1] Andrieux A., Czajkowski K., Dan A., Keahey K., Ludwig H., Nakata T., Pruyne J., Rofrano J., Tuecke S., Xu M.: Web Services Agreement Specication (WS-Agreement), OGF proposed recommendation (GFD.107), 2007.
• [2] Andronikou V., Mamouras K., Tserpes K., Kyriazis D., Varvarigou T.: Dynamic QoS-aware data replication in grid environments based on data \importance", Future Generation Computer Systems, vol. 28(3), pp. 544-553, 2012. https: //doi.org/10.1016/j.future.2011.02.003.
• [3] Baru C., Moore R.W., Rajasekar A., Wan M.: The SDSC storage resource broker. In: CASCON First Decade High Impact Papers, CASCON'10, pp. 189-200. IBM Corp., Riverton, NJ, 2010. https://doi.org/10.1145/1925805.1925816.
• [4] Billaud J.P., Gulati A.: hClock: hierarchical QoS for packet scheduling in a hypervisor. In: Hanzalek Z., Hartig H., Castro M., Kaashoek M.F. (eds.), Proceedings of Eighth Eurosys Conference 2013, EuroSys '13, Prague, Czech Republic, April 14-17, 2013, pp. 309-322, 2013. https://doi.org/10.1145/2465351.2465382.
• [5] Brzeźniak M., Jankowski G., Jankowski M., Jankowski S., Jankowski T., Meyer N., Miko lajczak R., Zawada A., Zdanowski S. (PSNC): National Data Storage 2: secure storage cloud with efficient and easy data access. In: Foster D. (ed.), Proceedings of 29th TERENA Networking Conference (TNC), 2013. http://www.terena.org/publications/tnc2013-proceedings/.
• [6] Carlson M., Yoder A., Schoeb L., Deel D., Pratt C., Lionetti Ch., Voigt D.: Software Dened Storage, white paper, Storage Networking Industry Association (SNIA), 2015. http://www.snia.org/sites/default/files/SNIA Software D efined Storage %20White Paper v1.pdf.
• [7] Chuang J.C., Sirbu M.A.: Distributed network storage service with quality-of- -service guarantees, Journal of Network and Computer Applications, vol. 23(3), pp. 163-185, 2000. https://doi.org/10.1006/jnca.2000.0109.
• [8] Elnably A., Wang H., Gulati A., Varman P.J.: Efficient QoS for Multi-Tiered Storage Systems. In: Rangaswami R., (ed.) 4th USENIX Workshop on Hot Topics in Storage and File Systems, HotStorage'12, Boston, MA, USA, June 13-14, 2012. USENIX Association, 2012. https://www.usenix.org/conference/hots torage12/workshop-program/presentation/elnably.
• [9] EOSC-hub { Services for the European Open Science Cloud. http://www.eosc -hub.eu/.
• [10] eScience on Distributed Computing Infrastructure - Achievements of PLGrid Plus Domain-Specific Services and Tools, Lecture Notes in Computer Science, vol. 8500, Bubak M., Kitowski J., Wiatr K. (eds.), Springer, 2014. https://doi.or g/10.1007/978-3-319-10894-0.
• [11] European Grid Infrastructure. http://www.egi.eu/.
• [12] GlusterFS. http://www.gluster.org/.
• [13] GPFS - General Parallel File System. https://www.ibm.com/support/knowle dgecenter/en/SSFKCN/gpfs welcome.html/.
• [14] Gulati A., Ahmad I., Waldspurger C.A.: PARDA: Proportional Allocation of Resources for Distributed Storage Access. In: Seltzer M., Wheeler R., (eds.) Proccedings of the 7th Conference on File and Storage Technologies, FAST'09, pp. 85-98. USENIX Association, Berkeley, CA, USA, 2009. http://dl.acm.org /citation.cfm?id=1525908.1525915.
• [15] Helix-Nebula { The Science Cloud. http://www.helix-nebula.eu/.
• [16] Hu C., Deng Y.: QoS-Oriented Capacity Provisioning in Storage Clusters by Modeling Workload Patterns. In: 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC), pp. 108-115, 2017. https://doi.org/10.1109/ISPA/IUCC.2017.00025.
• [17] Hu C., Deng Y., Yang L.T.: On-Demand Capacity Provisioning in Storage Clusters Through Workload Pattern Modeling, IEEE Access, vol. 5, pp. 24830-24841, 2017. https://doi.org/10.1109/ACCESS.2017.2767703.
• [18] Huang H.H., Grimshaw A.S.: Design, implementation and evaluation of a virtual storage system, Concurrency and Computation: Practice and Experience, vol. 23(4), pp. 311-331, 2011. http://dblp.uni-trier.de/db/journals/conc urrency/concurrency23.html#HuangG11.
• [19] iRODS { Open Source Data Management Software. http://www.irods.org.
• [20] Kapanowski M., S lota R., Kitowski J.: Resource Storage Management Model for Ensuring Quality of Service in the Cloud Archive Systems, Computer Science, vol. 15(1), pp. 3-18, 2014. http://journals.agh.edu.pl/csci/article/view/ 577.
• [21] Karlsson M., Karamanolis C., Zhu X.: Triage: Performance dierentiation for storage systems using adaptive control, ACM Transactions on Storage (TOS), vol. 1(4), pp. 457-480, 2005. https://doi.org/10.1145/1111609.1111612.
• [22] Keller E., Ludwig H.: The WSLA Framework: Specifying and Monitoring Service Level Agreements for Web Services, Journal of Network and Systems Management, vol. 11, pp. 57-81, 2003.
• [23] Kryza B., Dutka L., Slota R., Kitowski J.: Dynamic VO Establishment in Distributed Heterogeneous Business Environments. In: Allen G., Nabrzyski J., Seidel E., van Albada G.D., Dongarra J., Sloot P.M.A. (eds.), Computational Science - ICCS 2009, Lecture Notes in Computer Science, vol. 5545, pp. 709-718. Springer, Berlin - Heidelberg, 2009. http://dx.doi.org/10.1007/978-3-642-01973 -9 79.
• [24] Lin J.W., Chen C.H., Chang J.M.: QoS-Aware Data Replication for Data- -Intensive Applications in Cloud Computing Systems, IEEE Transactions on Cloud Computing, vol. 1(1), pp. 101{115, 2013. http://dx.doi.org/http: //doi.ieeecomputersociety.org/10.1109/TCC.2013.1.
• [25] Lumb C.R., Merchant A., Alvarez G.A.: Facade: Virtual Storage Devices with Performance Guarantees. In: Proceedings of the 2Nd USENIX Conference on File and Storage Technologies, FAST '03, pp. 131-144. USENIX Association, Berkeley, CA, USA, 2003. http://dl.acm.org/citation.cfm?id=1090694.1090710.
• [26] National Data Storage project. http://nds.psnc.pl.
• [27] Nikolow D.: Semantic-Based Storage QoS Management Methodology - Case Study for Distributed Environments, Computing and Informatics, vol. 31(6), pp. 1345-1366, 2012. http://www.cai.sk/ojs/index.php/cai/article/view/ 1311.
• [28] Nikolow D., Słota R., Lakovic D., Winiarczyk P., Pogoda M., Kitowski J.: Management Methods in SLA-aware Distributed Storage Systems, Computer Science, vol. 13(3), pp. 35-44, 2012.
• [29] Onedata project. http://onedata.org.
• [30] Orzechowski M., Kapanowski M., Słota R., Kitowski J.: Storage System Control with Decision Trees for Storage QoS Provisioning. In: Proceedings of KU KDM 2014: seventh ACC Cyfronet AGH users' conference, pp. 91-92. ACK Cyfronet AGH, 2014.
• [31] Polish Optical Internet PIONIER. http://www.pionier.net.pl/.
• [32] PL-Grid project. http://projekt.plgrid.pl/en.
• [33] PL-Grid Core project. http://www.plgrid.pl/en/projects/core.
• [34] PL-Grid NG project. http://www.plgrid.pl/en/projects/ng.
• [35] Scality RING. http://www.scality.com/ring/.
• [36] Shue D., Freedman M.J., Shaikh A.: Performance Isolation and Fairness for Multi-Tenant Cloud Storage. In: Thekkath C., Vahdat A. (eds.), 10th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2012, Hollywood, CA, USA, October 8{10, 2012, pp. 349-362. USENIX Association, 2012. https://www.usenix.org/conference/osdi12/technical-sessions/pr esentation/shue.
• [37] Shue D., Freedman M.J., Shaikh A.: Fairness and isolation in multi-tenant storage as optimization decomposition, Operating Systems Review, vol. 47(1), pp. 16-21, 2013. https://doi.org/10.1145/2433140.2433145.
• [38] Skene J., Lamanna D.D., Emmerich W.: Precise Service Level Agreements. In: Proceedings of the 26th International Conference on Software Engineering, ICSE'04, pp. 179-188. IEEE Computer Society, Washington, DC, USA, 2004. http://dl.acm.org/citation.cfm?id=998675.999422.
• [39] Skita l L., Janusz M., Słota R., Kitowski J.: Service Level Agreement Metrics for Real-Time Application on the Grid. In: Wyrzykowski R., Dongarra J., Karczewski K., Wasniewski J. (eds.), Parallel Processing and Applied Mathematics, 7th International Conference, PPAM, Gdansk, Poland, September 9{12, 2007, Revised Selected Papers, Lecture Notes in Computer Science, vol. 4967, pp. 798-806. Springer, 2008.
• [40] Słota R., Dutka L., Wrzeszcz M., Kryza B., Nikolow D., Król D., Kitowski J.: Storage Management Systems for Organizationally Distributed Environments PLGrid PLUS Case Study. In: Wyrzykowski R., Dongarra J., Karczewski K., Wasniewski J. (eds.), Parallel Processing and Applied Mathematics, 10th International Conference, PPAM 2013, Warsaw, Poland, September 8{11, 2013, Revised Selected Papers, Part I, Lecture Notes in Computer Science, vol. 8384, pp. 724-733, Springer, 2014.
• [41] Słota R., Nikolow D., Kitowski J., Król D., Kryza B.: FiVO/QStorMan Semantic Toolkit for Supporting Data-Intensive Applications in Distributed Environments, Computing and Informatics, vol. 31(5), pp. 1003-1024, 2012.
• [42] Słota R., Nikolow D., Skalkowski K., Kitowski J.: Management of Data Access with Quality of Service in PL-Grid Environment, Computing and Informatics, vol. 31(2), pp. 463-479, 2012.
• [43] Słota R., Nikolow D., Skita l L., Kitowski J.: Implementation of Replication Methods in the Grid Environment. In: Sloot P.M.A., Hoekstra A.G., Priol T., Reinefeld A., Bubak M. (eds.), Advances in Grid Computing { EGC 2005, European Grid Conference, Amsterdam, The Netherlands, February 14-6, 2005, Revised Selected Papers, Lecture Notes in Computer Science, vol. 3470, pp. 474-84, Springer, 2005. https://doi.org/10.1007/11508380 49.
• [44] Śnieżyński B.: Agent-based Adaptation System for Service-oriented Architectures Using Supervised Learning. In: Abramson D., Lees M., Krzhizhanovskaya V.V., Dongarra J., Sloot P.M.A. (eds.), Proceedings of the International Conference on Computational Science, ICCS 2014, Cairns, Queensland, Australia, 10-12 June, 2014, Procedia Computer Science, vol. 29, pp. 1057-1067, Elsevier, 2014. https: //doi.org/10.1016/j.procs.2014.05.095.
• [45] Stoica I., Morris R., Liben-Nowell D., Karger D.R., Kaashoek M.F., Dabek F., Balakrishnan H.: Chord: A scalable peer-to-peer lookup protocol for Internet applications, IEEE/ACM Transactions on Networking, vol. 11(1), pp. 17-32, 2003. https://doi.org/10.1109/TNET.2002.808407.
• [46] Tanimura Y., Hidetaka K., Kudoh T., Kojima I., Tanaka Y.: A distributed storage system allowing application users to reserve I/O performance in advance for achieving SLA. In: Grid Computing (GRID), 2010 11th IEEE/ACM International Conference on, pp. 193-200. 2010. https://doi.org/10.1109/GRID.2 010.5697948.
• [47] Tivoli Storage Manager for Space Management. http://ibm-tivoli-storage -manager.software.informer.com/wiki/.
• [48] Uttamchandani S., Alvarez G., Agha G.: DecisionQoS: an adaptive, self-evolving QoS arbitration module for storage systems. In: Policies for Distributed Systems and Networks, 2004, POLICY 2004, Proceedings, Fifth IEEE International Workshop on, pp. 67-76, 2004. https://doi.org/10.1109/POLICY.2004.1309 151.
• [49] Voulodimos A., Gogouvitis S.V., Mavrogeorgi N., Talyansky R., Kyriazis D., Koutsoutos S., Alexandrou V., Kolodner E.K., Brand P., Varvarigou T.A.: A United Management Model for Data Intensive Storage Clouds. In: NCCA, pp. 69-72, 2011. https://doi.org/10.1109/NCCA.2011.18.
• [50] Wang C.M., Yeh T.C., Tseng G.F.: Provision of Storage QoS in Distributed File Systems for Clouds. In: 2012 41st International Conference on Parallel Processing, Pittsburgh, PA, pp. 189-198, 2012. https://doi.org/10.1109/ICPP.201 2.52.
• [51] Weil S.A., Brandt S.A., Miller E.L., Long D.D.E., Maltzahn C.: Ceph: A Scalable, High-Performance Distributed File System. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI'06), pp. 307-320, 2006.
• [52] Weil S.A., Brandt S.A., Miller E.L., Maltzahn C.: CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC '06. ACM, New York, NY, USA, 2006. http://dx.doi.org/10.1145/1188455.1188582.
• [53] Wrzeszcz M., Nikolow D., Lichoń T., Słota R., Dutka L., Słota R., Kitowski J.: Consistency Models for Global Scalable Data Access Services. In: Wyrzykowski R., Dongarra J.J., Deelman E., Karczewski K. (eds.), Parallel Processing and Applied Mathematics - 12th International Conference, PPAM 2017, Lublin, Poland, September 10-13, 2017, RevisedSelected Papers, Part I, Lecture Notes in Computer Science, vol. 10777, pp. 471-480, Springer, 2017. https: //doi.org/10.1007/978-3-319-78024-5 41.
• [54] Wrzeszcz M., Opioła L., Zemek K., Kryza B., Dutka L., Słota R., Kitowski J.: Effective and Scalable Data Access Control in Onedata Large Scale Distributed Virtual File System. In: Koumoutsakos P., Lees M., Krzhizhanovskaya V.V., Dongarra J.J., Sloot P.M.A. (eds.), International Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland, Procedia Computer Science, vol. 108, pp. 445-454, Elsevier, 2017. https://doi.org/10.1016/j.pr ocs.2017.05.054.
• [55] Wrzeszcz M., Słota R., Kitowski J.: Towards Transparent Data Access with Context Awareness, Computer Science, vol. 19(2), pp. 201-221, 2018. https: //doi.org/10.7494/csci.2018.19.2.2844.
• [56] XDC - Extreme Data Cloud. http://www.extreme-datacloud.eu/.
• [57] Xu H., Russell T., Coposky J., Rajasekar A., Moore R., de Torcy A., Wan M., Shroeder W., Chen S.Y.: iRODS Primer 2: Integrated Rule-Oriented Data System. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers, 2017. https://doi.org/10.2200/S00760ED1V01Y2 01702ICR057.