An agent-based hierarchical approach for executing bag-of-tasks in clouds

• Autorzy: Funika W. , Mazurek K. , Kruczkowski W.

Identyfikatory

DOI

10.7494/csci.2014.15.1.35

• Języki publikacji: EN

Abstrakty

EN

Unlike message-passing applications, “bag-of-tasks” applications (BoTs), whose tasks are unrelated and independent (no inter-task communication), can be highly parallelized and executed in any acceptable order. A common practice when executing bag-of-tasks applications (BoT) is to exploit the master-slave topology. Cloud environments offer some features that facilitate the execution of BoT applications. One of the approaches to control-cloud resources is to use agents that are flexible actors in a dynamic environment. Given these assumptions, we have designed a combination of approaches which can be classified as distributed, hierarchical solutions to the issue of scalable execution of bag-of-tasks. The concept of our system relates to a project that is focused on processing huge quantities of incoming data from a network of sensors through the Internet. Our aim is to create a mechanism for processing such data as a system that executes jobs while exploiting load balancing for cloud resources which use applications such as Eucalyptus. The idea is to create a hybrid architecture which takes advantage of some centralized parts of the system and full distributedness in other parts. On the other hand, we balance dependencies between system components using a hierarchical master-slave structure.

Słowa kluczowe

EN

cloud computing; bag-of-tasks; load balancing; master-slave paradigm; scalability

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2014
• Tom: Vol. 15 (1)
• Strony: 35--60
• Opis fizyczny: Bibliogr. 35 poz., rys., wykr.

Twórcy

autor

Funika W.

• AGH University of Science and Technology, ACC CYFRONET AGH, Krakow, Poland

autor

Mazurek K.

• AGH University of Science and Technology, Krakow, Poland

autor

Kruczkowski W.

• AGH University of Science and Technology, Krakow, Poland

Bibliografia

• [1] Amazon EC2 website. http://aws.amazon.com/ec2. Access: 12.09.2013.
• [2] AWS SDK for Java. http://aws.amazon.com/sdkforjava. Access: 12.09.2013.
• [3]Cloud-Based Support for Distributed Multiscale Applications. https://portal.futuregrid.org/node/937/project-details. Access: 17.09.2013.
• [4] CXF website. http://cxf.apache.org. Access: 23.08.2013.
• [5] Euca2ools. http://www.eucalyptus.com/docs#euca2ools. Access: 17.09.2013.
• [6] Eucalyptus. http://www.eucalyptus.com. Access: 17.09.2013.
• [7] Eucalyptus and AWS. http://www.eucalyptus.com/aws-compatibility. Access: 17.09.2013.
• [8] FutureGrid website. https://portal.futuregrid.org. Access: 17.09.2013.
• [9] JSON website. http://www.json.org. Access: 23.08.2013.
• [10] Quartz Scheduler website. http://quartz-scheduler.org. Access: 23.08.2013.
• [11] Ambroszkiewicz S., Cetnarowicz K., Kozlak J., Nowak T., Penczek W.: Modeling Agent Organizations. In: Intelligent Information Systems, Advances in Soft Computing, vol. 4, pp. 135–144. Physica-Verlag HD, 2000. ISBN 978-3-7908-1309-8. http://dx.doi.org/10.1007/978-3-7908-1846-8_13.
• [12] Anglano C., Canonico M.: Scheduling algorithms for multiple Bag-of-Task applications on Desktop Grids: A knowledge-free approach. In: IPDPS, pp. 1–8. IEEE, 2008.
• [13] Antonis K., Garofalakis J., Mourtos I., Spirakis P.: A hierarchical adaptive dis-tributed algorithm for load balancing. J. Parallel Distrib. Comput., 64(1): 151–162, 2004. ISSN 0743-7315. http://dx.doi.org/10.1016/j.jpdc.2003.07.002.
• [14] Balis B., Kasztelnik M., Bubak M., Bartynski T., Gubaa T., Nowakowski P., Broekhuijsen J.: The UrbanFlood Common Information Space for Early Warning Systems.Procedia Computer Science, vol. 4, pp. 96–105, 2011. ISSN 1877-0509. http://dx.doi.org/http://dx.doi.org/10.1016/j.procs.2011.04.011. In: Proceedings of the International Conference on Computational Science, ICCS 2011.
• [15] Beaumont O., Carter L., Ferrante J., Legrand A., Marchal L., Robert Y.: Centralized versus distributed schedulers for multiple bag-of-task applications. In: Proceedings of the 20th international conference on Parallel and distributed pro-cessing, IPDPS’06, pp. 24–24. IEEE Computer Society, Washington, DC, USA, 2006. ISBN 1-4244-0054-6.
• [16] Bruno J., Coffman Jr. E.G., Sethi R.: Scheduling independent tasks to reduce mean finishing time.Commun. ACM, 17(7): 382–387, 1974. ISSN 0001-0782. http://dx.doi.org/10.1145/361011.361064.
• [17] Cetnarowicz K.: From Algorithm to Agent. In: Proceedings of the 9th Interna-tional Conference on Computational Science, ICCS 2009, pp. 825–834. Springer-Verlag, Berlin, Heidelberg, 2009. ISBN 978-3-642-01972-2. http://dx.doi.org/10.1007/978-3-642-01973-9_92.
• [18] da Silva F. A. B., Senger H.: Scalability limits of Bag-of-Tasks applications running on hierarchical platforms. J. Parallel Distrib. Comput., 71(6): 788–801, 2011. ISSN 0743-7315. http://dx.doi.org/10.1016/j.jpdc.2011.01.002.
• [19] Gamma E., Helm R., Johnson R., Vlissides J. M.: Design patterns: elements of reusable object-oriented software. Addison-Wesley, Reading, Mass. [u.a.], 1994. ISBN 0-201-63361-2.
• [20] Geelan J.: Twenty one experts define cloud computing. Cloud Computing Journal, http://virtualization.sys-con.com/node/612375 , 2009.
• [21] Gutierrez-Garcia J., Sim K. M.: Agent-Based Service Composition in Cloud Computing. In: Grid and Distributed Computing, Control and Automation, T. H. Kim, S. Yau, O. Gervasi, B. H. Kang, A. Stoica, D. lzak (Eds.), Communications in Computer and Information Science, vol. 121, pp. 1–10. Springer Berlin Heidel-berg, 2010. ISBN 978-3-642-17624-1. http://dx.doi.org/10.1007/978-3-642-17625-8_1.
• [22] Gutierrez-Garcia J. O., Sim K. M.: A Family of Heuristics for Agent-Based Cloud Bag-of-Tasks Scheduling. In: Proceedings of the 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CYBERC’11, pp. 416–423. IEEE Computer Society, Washington, DC, USA, 2011. ISBN 978-0-7695-4557-8. http://dx.doi.org/10.1109/CyberC.2011.74.
• [23] Ibarra O. H., Kim C. E.: Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors. J. ACM, 24(2): 280–289, 1977. ISSN 0004-5411. http://dx.doi.org/10.1145/322003.322011.
• [24] Kopacz T.: Programming in Windows Azure. Presentation from the 4th MIC conference ,,Nowoczesne technologie bli ̇zej nas”, Poznan, Poland http://mic.psnc.pl/download/conf10/MIC_NTBN10_Azure.pdf, 2010.
• [25] Kr ́ol D., Wrzeszcz M., Kryza B., Dutka L., Kitowski J.: Massively Scalable Platform for Data Farming Supporting Heterogeneous Infrastructure. In: The Fourth International Conference on Cloud Computing, GRIDs, and Virtualization, IARIA Cloud Computing 2013, pp. 144–149. Valencia, Spain, 2013.
• [26] Marco J., Campos I., Coterillo I.e.a.: The interactive European grid: Project objectives and achievements. Computing and Informatics, 27(2): 161–171, 2008. ISSN 1335-9150.
• [27] Mell P., Grance T.: The NIST Definition of Cloud Computing - Recommen-dations of National Institute of Standards and Technology. Special Publication 800-145, NIST US DOC, Sept. 2011 http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf.
• [28] Netto M. A., Buyya R.: Coordinated rescheduling of Bag-of-Tasks for executions on multiple resource providers. Concurr. Comput. : Pract. Exper. , 24(12): 1362–1376, 2012. ISSN 1532-0626. http://dx.doi.org/10.1002/cpe.1841.
• [29] Oprescu A., Kielmann T.: Bag-of-Tasks Scheduling under Budget Constraints. In: Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, pp. 351–359. 2010. http://dx.doi.org/10.1109/CloudCom.2010.32.
• [30] Pandey S., Karunamoorthy D., Buyya R.: Workflow Engine for Clouds. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 321–344. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch12.
• [31] Schubert L., Assel M., Kipp A., Wesner S.: Resource Cloud Mashups. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 533–548. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch21.
• [32] Senger H., da Silva F. A. B., Miranda Filho L. J.: Influence of Communication Models on the Scalability of Master-Slave Platforms Running Bag-of-Tasks Ap-plications. In: Proceedings of the 2010 11th Symposium on Computing Systems, WSCAD-SCC ’10, pp. 25–32. IEEE Computer Society, Washington, DC, USA, 2010. ISBN 978-0-7695-4274-4. http://dx.doi.org/10.1109/WSCAD-SCC.2010.27. 2014/04/07; 22:30 str. 25/26 An agent-based hierarchical approach for executing bag-of-tasks in clouds 59
• [33] Suchta A., Marsza lek K., Smoktunowicz U.: Little rain from great cloud (Z du zej chmury ma ly deszcz). Article on CRN website: http://www.crn.pl/artykuly/biznes/2011/09/z-duzej-chmury-maly-deszcz, 2011.
• [34] Voorsluys W., Broberg J., Buyya R.: Introduction to Cloud Computing. In: Cloud Computing: Principles and Paradigms, R. Buyya, J. Broberg, A. Goscinski (Eds.), pp. 1–41. John Wiley & Sons, Inc., New York, USA, 2011. ISBN 9780470940105. http://dx.doi.org/10.1002/9780470940105.ch1.
• [35] Wism ̈uller R., Bubak M., Funika W.e.a.: Support for User-Defined Metrics in the On-line Performance Analysis Tool G-PM. In: Grid Computing – Second European AcrossGrids Conference AxGrids2004, M.D. Dikaiakos, ed., Lecture Notes in Computer Science, vol. 3165, pp. 159–168. Springer-Verlag, Nicosia, Cyprus, 2004,