Analysis of series of measurements from job-centric monitoring by statistical functions

• Autorzy: Hilbrich M. , Frank M.

Identyfikatory

DOI

10.7494/csci.2017.18.1.3

• Języki publikacji: EN

Abstrakty

EN

The rising number of executed programs (jobs) enabled by the growing amount of available resources from Clouds, Grids, and HPC (for example) has resulted in an enormous number of jobs. Nowadays, most of the executed jobs are mainly unobserved, so unusual behavior, non-optimal resource usage, and silent faults are not systematically searched and analyzed. Job-centric monitoring enables permanent job observation and, thus, enables the analysis of monitoring data. In this paper, we show how statistic functions can be used to analyze job-centric monitoring data and how the methods compare to more-complex analysis methods. Additionally, we present the usefulness of job-centric monitoring based on practical experiences.

Słowa kluczowe

EN

job-centric monitoring; monitoring; similarity; series of measurements; statistical functions; grid; cloud; analysis

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2017
• Tom: Vol. 18 (1)
• Strony: 3--19
• Opis fizyczny: Bibliogr. 29 poz., rys., wykr., tab.

Twórcy

autor

Hilbrich M.

• s-lab – Software Quality Lab, Universitat Paderborn, Zukunftsmeile 1, 33102 Paderborn, German

autor

Frank M.

• Technische Universitat Chemnitz, Straße der Nationen 62, 09107 Chemnitz, Germany

Bibliografia

• [1] Bellman R.: Dynamic Programming. Princeton University Press, Princeton, NJ, USA, 2010.
• [2] Chan P., Stolfo S.J.: Toward Parallel and Distributed Learning by Meta- Learning. In: AAAI Workshop in Knowledge Discovery in Databases , pp. 227– 240, 1993.
• [3] Charles J., Hocker A., Lacker H., Laplace S., Diberder F., Malcl ́es J., Ocariz J., Pivk M., Roos L.: CP violation and the CKM matrix: assessing the impact of the asymmetric B factories. The European Physical Journal C Particles and Fields , vol. 41(1), pp. 1–131, 2005, http://dx.doi.org/10.1140/epjc/ s2005-02169-1 .
• [4] Denning D.E.: An intrusion-detection model. IEEE Transactions on Software Engineering , vol. 13(2), pp. 222–232, 1987.
• [5] Dickerson J.E., Dickerson J.A.: Fuzzy network profiling for intrusion detection. In: Proceedings of NAFIPS 19th International Conference of the North American Fuzzy Information Processing Society, Atlanta , pp. 301–306, 2000.
• [6] Dobai R., Balaz M.: Genetic method for compressed skewed-load delay test generation. In: Design and Diagnostics of Electronic Circuits Systems (DDECS), 2012 IEEE 15th International Symposium on , pp. 242–247, 2012.
• [7] Grefenstette J.: Optimization of Control Parameters for Genetic Algorithms. Systems, Man and Cybernetics, IEEE Transactions on , vol. 16(1), pp. 122–128, 1986.
• [8] von Grunigen D.: Digitale Signalverarbeitung: Mit einer Einfuhrung in die kontinuierlichen Signale und Systeme. Fachbuchverlag, Leipzig, 2008.
• [9] Gusfield D.: Algorithms on Stings, Trees, and Sequences. Computer Science and Computational Biology , 1997.
• [10] Hilbrich M.: Jobzentrisches Monitoring in Verteilten Heterogenen Umgebungen mit Hilfe Innovativer Skalierbarer Methoden . Dissertation, Fakultat Informatik der Technischen Universitat Dresden, Germany, 2014. 2017/03/13; 18:16 str. 15/17 Analysis of series of measurements from job-centric monitoring (...) 17
• [11] Hilbrich M., Muller-Pfefferkorn R.: A Scalable Infrastructure for Job-Centric Monitoring Data from Distributed Systems. In: M. Bubak, M. Turala, K. Wiatr, eds., Proceedings Cracow Grid Workshop ’09 , pp. 120–125, 2010.
• [12] Hilbrich M., Muller-Pfefferkorn R.: Achieving scalability for job centric monitoring in a distributed infrastructure. In: G. Muhl, J. Richling, A. Herkersdorf, eds., ARCS Workshops , LNI , vol. 200, pp. 481–492, GI, 2012.
• [13] Hilbrich M., Muller-Pfefferkorn R.: Cross-Correlation as Tool to Determine the Similarity of Series of Measurements for Big-Data Analysis Tasks. In: 2015 International Conference on Cloud Computing and Big Data (CloudCom-Asia) , 2015.
• [14] Hilbrich M., Weber M., Tschuter R.: Automatic Analysis of Large Data Sets: A Walk-Through on Methods from Different Perspectives. In: Cloud Computing and Big Data (CloudCom-Asia) , pp. 373–380, 2013.
• [15] Hocker A., Lacker H., Laplace S., Le Diberder F.: A new approach to a global fit of the CKM matrix. The European Physical Journal C – Particles and Fields , vol. 21(2), pp. 225–259, 2001, http://dx.doi.org/10.1007/s100520100729 .
• [16] Hoefler T., Schneider T., Lumsdaine A.: Characterizing the Influence of System Noise on Large-Scale Applications by Simulation. In: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis , SC ’10, pp. 1–11, IEEE Computer Society, Washington, DC, USA, 2010, http://dx.doi.org/10.1109/SC.2010.12 .
• [17] Holland J.: Genetic Algorithms. Scientific American, vol. 267(1), pp. 66–72, 1992.
• [18] Lazarevic A., Ertoz L., Kumar V., Ozgur A., Srivastava J.: A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection. In: D. Barbar ́a, C. Kamath, eds., Proceedings of SIAM Conference on Data Mining , 2003.
• [19] Lee W., Stolfo S., Mok K.: A data mining framework for building intrusion detection models. In: Security and Privacy, 1999. Proceedings of the 1999 IEEE Symposium on , pp. 120–132, 1999.
• [20] Lee W., Stolfo S.J.: Data Mining Approaches for Intrusion Detection. In: Proceedings of the 7th conference on USENIX Security Symposium – Volume 7 , SSYM’98, pp. 6–6, USENIX Association, Berkeley, CA, USA, 1998, http: //dl.acm.org/citation.cfm?id=1267549.1267555 .
• [21] Lee W., Stolfo S.J.: A framework for constructing features and models for intrusion detection systems. ACM Transactions on Information and System Security , vol. 3(4), pp. 227–261, 2000, http://doi.acm.org/10.1145/382912.382914 .
• [22] Lorenz D., Borovac S., Buchholz P., Eichenhardt H., Harenberg T., Mattig P., Mechtel M., Muller-Pfefferkorn R., Neumann R., Reeves K., Uebing C., Walkowiak W., William T., Wismuller R.: Job monitoring and steering in D-Grid’s High Energy Physics Community Grid. Future Generation Computer Systems , vol. 25, pp. 308–314, 2009, http://dx.doi.org/10.1016/j.future. 2008.05.009 . 2017/03/13; 18:16 str. 16/17 18 Marcus Hilbrich, Markus Frank
• [23] Muller-Pfefferkorn R., Neumann R., William T.: AMon – a User-Friendly Job Monitoring for the Grid. In: T. Priol, M. Vanneschi, eds., CoreGRID , pp. 185– 192, Springer, 2007.
• [24] Myers E.W.: An O(ND) difference algorithm and its variations. Algorithmica , vol. 1, pp. 251–266, 1986.
• [25] Needleman S.B., Wunsch C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology , vol. 48(3), pp. 443–453, 1970.
• [26] Paxson V.: Bro: a system for detecting network intruders in real-time. Computer Networks , vol. 31(23–24), pp. 2435–2463, 1999, http://www.sciencedirect. com/science/article/pii/S1389128699001127 .
• [27] Roesch M., Telecommunications S.: Snort – Lightweight Intrusion Detection for Networks. pp. 229–238, 1999.
• [28] Smaha S.E.: Haystack: An intrusion detection system. In: Proceedings of the IEEE 4th Aerospace Computer Security Applications Conference , 1988.
• [29] Tang K., Man K., Kwong S., He Q.: Genetic algorithms and their applications. Signal Processing Magazine, IEEE , vol. 13(6), pp. 22–37, 1996

Uwagi

PL

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).