Pre-Processing and Modeling Tools for Big Data

• Autorzy: Hashem H. , Ranc D.

Identyfikatory

DOI

10.1515/fcds-2016-0009

• Języki publikacji: EN

Abstrakty

EN

Modeling tools and operators help the user / developer to identify the processing field on the top of the sequence and to send into the computing module only the data related to the requested result. The remaining data is not relevant and it will slow down the processing. The biggest challenge nowadays is to get high quality processing results with a reduced computing time and costs. To do so, we must review the processing sequence, by adding several modeling tools. The existing processing models do not take in consideration this aspect and focus on getting high calculation performances which will increase the computing time and costs. In this paper we provide a study of the main modeling tools for Big Data and a new model based on pre-processing.

Słowa kluczowe

EN

Data Modeling; big data; NoSQL; MapReduce; Pre-processing

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2016
• Tom: Vol. 41, No. 3
• Strony: 151--162
• Opis fizyczny: Bibliogr. 24 poz., fig., tab.

Twórcy

autor

Hashem H.

• Télécom SudParis, France

autor

Ranc D.

• Télécom SudParis, France

Bibliografia

• [1] Afrati F.N., Ullman J.D., Optimizing joins in a map-reduce environment, International Conference on Extending Database Technology, 2010, Print ISBN 978-1-60558-945-9.
• [2] Chalkiopoulos A., Programming MapReduce with Scalding, Packt Publishing, 2014, Print ISBN 978-1783287017.
• [3] DeCandia G., Hastorun D., Jampani M., Kakulapati G., Lakshman A., Pilchin A., Sivasubramanian S., Vosshall P., Vogels W., Dynamo: Amazon’s highly available key-value store, ACM Symposium on Operating Systems Principles, 2007, DOI 10.1145/1323293.1294281.
• [4] Ghemawat S., Gobioff H., Leung S.K., The Google File System, SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles, 2003, Print ISBN 1-58113-757-5.
• [5] Hashem H., Ranc D., An integrative Modeling of Big Data Processing, International Journal of Computer Science and Applications, 2014, Print ISSN 0972-9038.
• [6] Hashem H., Ranc D., Predicate-based Cloud Computing, 8th International Conference on Next Generation Mobile Apps, Services and Technologies, 2014, Print ISBN 978-1-4799-5072-0.
• [7] Kaur K., Rani R., Modeling and querying data in NoSQL databases, Big Data, 2013 IEEE International Conference, INSPEC Accession Number 13999217.
• [8] Kim S., Jung W., Kim H.S., A location inference algorithm based-on smart phone user data modelling, International Conference on Advanced Communication Technology, 2014, Print ISBN 978-89-968650-2-5.
• [9] Li Y., Manoharan S., A performance comparison of SQL and NoSQL databases, Communications, Computers and Signal Processing, 2013 IEEE Pacific Rim Conference, Print ISSN 1555-5798.
• [10] Lin J., Bahety A., Konda S., Mahindrakar S., Lowlatency, high-throughput access to static global resources within the Hadoop framework, Technical Report - University of Maryland, 2009, HCIL-2009-01.
• [11] Lin J., Dyer C., Data-Intensive Text Processing with MapReduce (Synthesis Lectures on Human Language Technologies), Morgan and Claypool Publishers, 2010, Print ISBN 978-1608453429.
• [12] Lin J., Schatz M., Design Patterns for Efficient Graph Algorithms in MapReduce, University of Maryland, College Park, 2010, Print ISBN 978-1-4503-0214-2.
• [13] Malewicz G., Austern M.H., Bik A.J.C., Dehnert J.C., Horn I., Leiser N., Czajkowski G., Pregel: A system for largescale graph processing, ACM Conference on Management of Data, 2010, Print ISBN 978-1-4503-0032-2.
• [14] Mesiti M., Valtolina S., Towards a User-Friendly Loading System for the Analysis of Big Data in the Internet of Things, Computer Software and Applications Conference Workshops, 2014 IEEE 38th International Conference, Print ISBN 978-1-4799-3578-9.
• [15] Miner D., Shook A., MapReduce Design Patterns: Building Effective Algorithms and Analytics for Hadoop and Other Systems, O'Reilly Media, 2012, Print ISBN 978-1449327170.
• [16] Mohanty S., Jagadeesh M., Srivatsa H., Big Data Imperatives: Enterprise Big Data Warehouse, BI Implementations and Analytics, APress, 2013, Print ISBN 978-1430248729.
• [17] Petrovic S., Osborne M., Lavrenko V., Streaming first story detection with application to Twitter, International Conference of the North American Chapter of the Association for Computational Linguistics, 2010, Print ISBN 1-932432-65-5.
• [18] Schroeder B., Pinheiro E., Weber W.D., DRAM errors in the wild: A large-scale field study, International Joint Conference on Measurement and Modeling of Computer Systems, 2009, Print ISBN 978-1-60558-511-6.
• [19] Shaikh M.A., Jiaxin W., Investigative Data Mining: Identifying Key Nodes in Terrorist Networks, IEEE International Conference on Multitopic Conference, 2006, Print ISBN 1-4244-0795-8.
• [20] Suraworachet W., Premsiri S., Cooharojananone N., The Study on the Effect of Facebook's Social Network Features toward Intention to Buy on F-commerce in Thailand, IEEE/IPSJ International Symposium on Applications and the Internet, 2012, Print ISBN 978-1-4673-2001-6.
• [21] Tudorica B.G., Bucur C., A comparison between several NoSQL databases with comments and notes, Roedunet International Conference (RoEduNet), 2011, Print ISBN 978-1-4577-1233-3.
• [22] Wang G., Tang J., The NoSQL Principles and Basic Application of Cassandra Model, Computer Science & Service System, 2012 International Conference, Print ISBN 978-1-4673-0721-5.
• [23] Yang H.C., Dasdan A., Hasiao R.L., Parker D.S., MapReduce-Merge: Simplified relational data processing on large clusters, ACM Conference on Management of Data, 2007, Print ISBN 978-1-59593-686-8.
• [24] Zheng Z., Zhu J., Lyu M.R., Service-Generated Big Data and Big Data-as-a-Service: An Overview, IEEE International Congress on Big Data, 2013, Print ISBN 978-0-7695-5006-0.

Uwagi

This is an extended version of the paper presented at the International Conference on Big Data Intelligence and Computing (DataCom 2015), Chengdu, China, December 19-21, 2015.

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę.