Consolidating logs in varying formats into a structured repository

• Autorzy: Opaliński A.

Warianty tytułu

PL

Konsolidacja logów w zmiennych formatach do strukturalnego repozytorium

• Języki publikacji: EN

Abstrakty

EN

During multi-agent systems development, testing and research, possibly voluminous logs are created at distributed agent locations. These logs describe individual agents’ behavior and state changes. To get the complete picture of the multi-agent system working, it is necessary to consolidate these logs and to store them in a format appropriate for diverse analyzes. Specific issues arise during consolidation if the log formats are not uniform. This paper presents an approach to consolidate logs in various formats into a single repository suitable for further analyzes.

PL

Podczas rozwijania, testowania i badania systemów wieloagentowych tworzone są obszerne logi w rozproszonych lokalizacjach, w których działają agenty. Logi opisują zachowania poszczególnych agentów oraz zmiany ich stanów. Żeby uzyskać kompletny obraz działania systemu wieloagentowego, konieczna jest konsolidacja tych logów do postaci przydatnej dla różnorodnych analiz. Niejednolity format logów różnych agentów tworzy specyficzne wyzwania. Artykuł opisuje podejście do konsolidacji logów w zmiennych formatach do postaci jednego repozytorium, przydatnego do prowadzenia dalszych analiz.

Słowa kluczowe

EN

data consolidation; data extraction; metadata management

PL

konsolidacja danych; ekstrakcja danych; zarządzanie meta-danymi

• Wydawca: Wydawnictwo Politechniki Śląskiej
• Czasopismo: Studia Informatica
• Rocznik: 2014
• Tom: Vol. 35, nr 2
• Strony: 55--68
• Opis fizyczny: Bibliogr. 26 poz.

Twórcy

autor

Opaliński A.

• Politechnika Gdańska, Wydział Elektrotechniki i Automatyki

Bibliografia

• 1. Sheth B., Maes P.: Evolving Agents for Personalized Information Filtering. Proc. of the 9th Conference on Artificial Intelligence for Applications, IEEE Computer Society Press, 1993.
• 2. Maes P.: Agents that reduce work and information overload. Communications of the ACM, Vol. 37, 1994, p. 31÷40.
• 3. Gutiérrez C., García-Magariño I.: Extraction of Execution Patterns In Multi-Agent Sys-tems. IEEE LA Transactions, Vol. 8, No. 3, 2010.
• 4. Ilie S., Scafes M., Badica C., Neidhart T., Pinchuk R.: Semantic logging in a distributed multi-agent system. Proc. of International Joint Conference on Computational Cyber-netics and Technical Informatics (ICCC-CONTI), 2010, p. 265÷270.
• 5. Pavlin G., Kamermans M., Scafej M.: Dynamic process integration framework: Toward efficient information processing in complex distributed systems. Proc. of 3rd Interna-tional Symposium on Intelligent Distributed Computing (IDC2009), Springer Verlag, Studies in Computational Intelligence, Vol. 237, 2009, p. 161÷174.
• 6. Unruh A., Bailey J., Ramamohanarao K.: Building More Robust Multi-Agent Systems Using a Log-based Approach. Web Intelligence and Agent Systems, Vol. 7, Issue 1, IOS Press Amsterdam, 2009, p. 65÷87.
• 7. Varakantham P. R., Gangwani S. K., Karlapalem K.: On Handling Component and Transaction Failures in Multi Agent Systems. ACM SIGecom Exchanges-Chains of commitment, Vol. 3, Issue 1, 2002, p. 32÷43.
• 8. Figueiredo J., Lau N., Pereira A.: Multi-Agent Debugging and monitoring framework. Proc. Of First IFAC Workshop on Multivehicle Systems (MVS), 2006.
• 9. Rozinat A., Zickler S., Veloso M., van der Aalst W. M. P., McMillen C.: Analyzing multi-agent activity logs using process mining techniques. [in:] Asama H., Kurokawa H., Ota J., Sekiyama K. (eds.): Distributed Autonomous Robotic Systems 8 (9th Inter-national Symposium, Tsukuba, Ibaraki, Japan, November 17-19, 2008), part IV, Springer, 2009, p. 251÷260.
• 10. Reijers H. A., Song M., Jung B.: Analysis of a Collaborative Workflow Process with Distributed Actors. Information Systems Frontiers, 2008.
• 11. Process Mining tool home page, http://processmining.org/ (DOA: 15.12.2013).
• 12. Ogston E., Brazier F.: AgentScope: Multi-Agent Systems Development in Focus. Proc. of 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), 2011, p. 389÷396.
• 13. Bhat S., Wahid A.: Log Agent for Knowledge Management Based on Multi-Agent Sys-tem. International Journal of Emerging Technology and Advanced Engineering, Vol. 2, Issue 7, 2012.
• 14. Ilarri S., Serrano J. L., Mena E., Trillo R.: 3D Monitoring Of Distributed Multiagent Systems. International Conference on Web Information Systems and Technologies (WEBIST), available online: http://sid.cps.unizar.es/PUBLICATIONS/POSTSCRIPTS/ webist07-agents.pdf, 2007 (DOA: 15.12.2013).
• 15. Product documentation: WebSphere Application Server (IBM i), Version 8.5, Chapter: Storing logs in a relational database for high availability, available online: http://pic.dhe.ibm.com (DOA: 15.12.2013).
• 16. Marcus A.: The NoSQL Ecosystem. [in:] Brown E., Wilson G. (eds.): The Architecture of Open Source Applications. Available online: http://www.aosabook.org/en/ nosql.html, (DOA:15.12.2013).
• 17. Kovacs K.: A technical comparison of NoSQL products. Available online: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis, (DOA: 15.12.2013).
• 18. White T.: Hadoop: The Definitive Guide. 2009.
• 19. Capriolo E.: Cassandra High Performance Cookbook. 2011.
• 20. Facebook White Paper, Facebook's Petabyte Scale Data Warehouse using Hive and Ha-doop, 2010, available online: http://www.sfbayacm.org/wp/wp-content/uploads/2010/ 01/sig_2010_v21.pdf, (DOA: 15.12.2013).
• 21. Dimiduk N., Khurana A.: HBase in Action. 2012.
• 22. Dean J., Ghemawat S.: MapReduce: Simplified Data Processing on Large Clusters. Proc. of Sixth Symposium on Operating System Design and Implementation (OSDI), 2004.
• 23. MongoDB Ecosystem, part of MongoDB Manual, Chapter Storing Log Data. Available online: http://docs.mongodb.org/ecosystem/use-cases/storing-log-data/ (DOA: 15.12.2013).
• 24. SQLite library project home page. Available online: http://www.sqlite.org/ (DOA: 15.12.2013).
• 25. The proof-of-concept program, as set of Code::Blocks projects in C. Available online: https://sites.google.com/site/flecabinet/downloads/BDAS2014.zip (DOA:15.12.2013).
• 26. Nilsson J.: Implement a Continuously Updating, High-Resolution Time Provider for Windows. MSDN Magazine 2004, available online: http://msdn.microsoft.com/en-us/ magazine/cc163996.aspx (DOA: 15.12.2013).