Data and Task Scheduling in Distributed Computing Environments

• Autorzy: Szmajduch M.
• Języki publikacji: EN

Abstrakty

EN

Data-aware scheduling in today’s large-scale heterogeneous environments has become a major research and engineering issue. Data Grids (DGs), Data Clouds (DCs) and Data Centers are designed for supporting the processing and analysis of massive data, which can be generated by distributed users, devices and computing centers. Data scheduling must be considered jointly with the application scheduling process. It generates a wide family of global optimization problems with the new scheduling criteria including data transmission time, data access and processing times, reliability of the data servers, security in the data processing and data access processes. In this paper, a new version of the Expected Time to Compute Matrix (ETC Matrix) model is defined for independent batch scheduling in physical network in DG and DC environments. In this model, the completion times of the computing nodes are estimated based on the standard ETC Matrix and data transmission times. The proposed model has been empirically evaluated on the static grid scheduling benchmark by using the simple genetic-based schedulers. A simple comparison of the achieved results for two basic scheduling metrics, namely makespan and average flowtime, with the results generated in the case of ignoring the data scheduling phase show the significant impact of the data processing model on the schedule execution times.

Słowa kluczowe

EN

data cloud; data grid; data processing; data scheduling; ETC Matrix

• Wydawca: Instytut Łączności - Państwowy Instytut Badawczy
• Czasopismo: Journal of Telecommunications and Information Technology
• Rocznik: 2014
• Tom: nr 4
• Strony: 71--78
• Opis fizyczny: Bibliogr. 23 poz., rys., tab.

Twórcy

autor

Szmajduch M.

• Department of Computer Science, Cracow University of Technology, Cracow, Poland

Bibliografia

• [1] J. Kołodziej, Evolutionary Hierarchical Multi-Criteria Metaheuristics for Scheduling in Large-Scale Grid Systems. Studies in Computational Intelligence Serie, vol. 419. Berlin-Heidelberg: Springer, 2012.
• [2] H. Casanova, G. Obertelli, F. Berman, and R. Wolski, “The AppLeS parameter sweep template: user-level middleware for the grid”, in Proc. 2000 ACM/IEEE Conf. on Supercomputing SC 2000), Dallas, TX, USA, 2000.
• [3] R. Buyya, M. Murshed, D. Abramson, and S. Venugopal, “Scheduling parameter sweep applications on global Grids: a deadline and budget constrained cost-time optimization algorithm”, Softw. Pract. Exper., vol. 35, no. 5, pp. 491–512, 2005.
• [4] T. Kosar and M. Balman, “A new paradigm: Data-aware scheduling in grid computing”, Future Gener. Comp. Syst., vol. 25, no. 4, pp. 406–413, 2009.
• [5] J. Kołodziej, S. U. Khan, and F. Xhafa, “Genetic algorithms for energy-aware scheduling in computational grids”, in Proc. 6th IEEE Int. Conf. P2P, Parallel, Grid, Cloud, and Internet Comput. 3PGCIC, Barcelona, Spain, 2011, pp. 17–24.
• [6] G. L. Valentini et al., “An overview of energy efficiency techniques in cluster computing systems”, Cluster Comput., vol. 16, no. 1, pp. 3–15, 2011.
• [7] H. Liu and D. Orban, “GridBatch: Cloud Computing for Large-Scale Data-Intensive Batch Applications”, in Proc. 8th IEEE Int. Symp. Cluster Comput. and the Grid CCGRID 2008, Lyon, France, 2008, pp. 295–305.
• [8] J. Kołodziej and F. Xhafa, “A game-theoretic and hybrid genetic meta-heuristic model for security-assured scheduling of independent jobs in computational grids”, in Proc. Int. Conf. Complex, Intell. Softw. Inten. Syst. CISIS 2010, Krakow, Poland, 2010, pp. 93–100.
• [9] L. Wang and S. U. Khan, “Review of performance metrics for green data centers: a taxonomy study”, J. Supercomput., vol. 63, no. 3, pp. 639–656, 2013.
• [10] S. Zeadally, S. U. Khan, and N. Chilamkurti, “Energy-efficient networking: past, present, and future”, J. Supercomput., vol. 62, no. 3, pp. 1093–1118, 2012.
• [11] S. Ali, H. J. Siegel, M. Maheswaran, and D. Hensgen, “Task execution time modeling for heterogeneous computing systems”, in Proc. 9th Heterogen. Comput. Worksh. HCW 2000, Cancun, Mexico, 2000, pp. 185–199.
• [12] J. Kołodziej and F. Xhafa, “Meeting security and user behaviour requirements in grid scheduling”, Simul. Model. Pract. Theory, vol. 19, no. 1, pp. 213–226, 2011.
• [13] J. Kołodziej and F. Xhafa, “Integration of task abortion and security requirements in GA-based meta-heuristics for independent batch grid scheduling”, Comp. Mathem. Appl., vol. 63, no. 2, pp. 350–364, 2011.
• [14] L. L. Lapin, Probability and Statistics for Modern Engineering, 2nd ed. Long Grove, USA: Waveland Pr. Inc., 1998.
• [15] A. Deshpande, Z. G. Ives, and V. Raman, “Adaptive query processing”, Foundation and Trends in Databases, vol. 1, no. 1, pp. 1–140, 2007.
• [16] S. Ali, H. J. Siegel, M. Maheswaran, and D. Hensgen, “Representing task and machine heterogeneities for heterogeneous computing systems”, Tamkang J. Sci. Engin., vol. 3, no. 3, pp. 195–207, 2000.
• [17] S. Venugopal and R. Buyya, “An SCP-based heuristic approach for scheduling distributed data-intensive applications on global grids”, J. Parallel Distrib. Comp., vol. 68, pp. 471–487, 2008.
• [18] F. Xhafa, L. Barolli, and A. Durresi, “Batch mode schedulers for grid systems”, Int. J. Web and Grid Serv., vol. 3, no. 1, pp. 19–37, 2007.
• [19] F. Pinel, J. E. Pecero, P. Bouvry, and S. U. Khan, “A two-phase heuristic for the scheduling of independent tasks on computational grids”, in Proc. of ACM/IEEE/IFIP Int. Conf. High Perform. Comput. Simul. HPCS 2011, Istanbul, Turkey, 2011, pp. 471–477.
• [20] J. Kołodziej and F. Xhafa, “Enhancing the genetic-based scheduling in computational grids by a structured hierarchical population”, Future Gener. Comp. Syst., vol. 27, pp. 1035–1046, 2011.
• [21] F. Xhafa and A. Abraham, “Computational models and heuristic methods for grid scheduling problems”, Future Gener. Comp. Syst., vol. 26, pp. 608–621, 2010.
• [22] J. Kołodziej and S. U. Khan, “Multi-level hierarchic genetic-based scheduling of independent jobs in dynamic heterogeneous grid environment”, Inform. Sci., vol. 214, pp. 1–19, 2012.
• [23] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. Berlin: Springer, 1992.