Exploring complex and big data

Stefanowski, J.; Krawiec, K.; Wrembel, R.

doi:10.1515/amcs-2017-0046

Artykuł - szczegóły

Tytuł artykułu

Exploring complex and big data

Autorzy

Stefanowski J. , Krawiec K. , Wrembel R.

Treść / Zawartość

Pełne teksty:

01_stefanowski_krawiec_exploring_complex_2017_4.pdf

Pobierz

Identyfikatory

DOI

10.1515/amcs-2017-0046

Warianty tytułu

Języki publikacji

Abstrakty

This paper shows how big data analysis opens a range of research and technological problems and calls for new approaches. We start with defining the essential properties of big data and discussing the main types of data involved. We then survey the dedicated solutions for storing and processing big data, including a data lake, virtual integration, and a polystore architecture. Difficulties in managing data quality and provenance are also highlighted. The characteristics of big data imply also specific requirements and challenges for data mining algorithms, which we address as well. The links with related areas, including data streams and deep learning, are discussed. The common theme that naturally emerges from this characterization is complexity. All in all, we consider it to be the truly defining feature of big data (posing particular research and technological challenges), which ultimately seems to be of greater importance than the sheer data volume.

Słowa kluczowe

big data complex data data integration data provenance data streams deep learning

big data dane złożone integracja danych pochodzenie danych strumień danych uczenie głębokie

Wydawca

Oficyna Wydawnicza Uniwersytetu Zielonogórskiego

Czasopismo

International Journal of Applied Mathematics and Computer Science

Rocznik

2017

Tom

Vol. 27, no. 4

Strony

669--679

Opis fizyczny

Bibliogr. 51 poz.

Twórcy

autor

Stefanowski J.

jerzy.stefanowski@cs.put.poznan.pl

Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

autor

Krawiec K.

krawiec@cs.put.poznan.pl

Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

autor

Wrembel R.

robert.wrembel@cs.put.poznan.pl

Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

Bibliografia

[1] Ahmadov, A., Thiele, M., Eberius, J., Lehner, W. and Wrembel, R. (2015). Towards a hybrid imputation approach using web tables, IEEE/ACM International Symposium on Big Data Computing (BDC), Limassol, Cyprus, pp. 21–30.
[2] Bekkerman, R., Bilenko, M. and Langford, J. (2011). Scaling Up Machine Learning: Parallel and Distributed Approaches, Cambridge University Press, New York, NY.
[3] Benjelloun, O., Garcia-Molina, H., Menestrina, D., Su, Q., Whang, S.E. and Widom, J. (2009). Swoosh: A generic approach to entity resolution, The VLDB Journal 18(1): 255–276.
[4] Bayer, M.A. and Edjlali, R. (2014). Magic Quadrant for Data Warehouse Database Management Systems, Gartner Publications, Stamford, CT, https://www.gartner.com/doc/2678018/magic-quadrant-data-warehouse-database.
[5] Beyer, M. and Laney, D. (2012). The Importance of “Big Data”: A Definition, Gartner Publications, Stamford, CT.
[6] Boyd, D. and Crawford, K. (2012). Critical questions for big data, Information, Communication and Society 15(5): 662–679.
[7] Brzezinski, D. and Stefanowski, J. (2014). Combining block-based and online methods in learning ensembles from concept drifting data streams, Information Sciences 265: 50–67.
[8] Che, D., Safran, M. and Peng, Z. (2013). From big data to big data mining: Challenges, issues, and opportunities, in B. Hong et al. (Eds.), International Conference on Database Systems for Advanced Applications, Lecture Notes in Computer Science, Vol. 7827, Springer, Berlin/Heidelberg, pp. 1–15.
[9] Chen, C.L.P. and Zhang, C. (2014). Data-intensive applications, challenges, techniques and technologies: A survey on big data, Information Sciences 275(10): 314–347.
[10] Custers, B., Calders, T., Schermer, B. and Zarsky, T.Z. (Eds.) (2013). Discrimination and Privacy in the Information Society—Data Mining and Profiling in Large Databases, Studies in Applied Philosophy, Epistemology and Rational Ethics, Vol. 3, Springer, Berlin/Heidelberg.
[11] Ditzler, G., Roveri, M., Alippi, C. and Polikar, R. (2015). Learning in nonstationary environments: A survey, IEEE Computational Intelligence Magazine 10(4): 12–25.
[12] Domingos, P. and Hulten, G. (2000). Mining high-speed data streams, ACM SIGKDD International Conference on Knowledge Discovery Data Mining, Boston, MA, USA, pp. 71–80.
[13] Duggan, J., Elmore, A.J., Stonebraker, M., Balazinska, M., Howe, B., Kepner, J., Madden, S., Maier, D., Mattson, T. and Zdonik, S. (2015). The BigDAWG polystore system, SIGMOD Record 44(2): 11–16.
[14] Elmagarmid, A., Rusinkiewicz, M. and Sheth, A. (Eds.) (1999). Management of Heterogeneous and Autonomous Database Systems, Morgan Kaufmann, San Francisco, CA.
[15] Fernández, A., del Río, S., Chawla, N.V. and Herrera, F. (2017). An insight into imbalanced big data classification: Outcomes and challenges, Complex & Intelligent Systems 3(2): 105–120.
[16] Francisco, P. (2012). Oracle Exadata and IBM Netezza data warehouse appliance compared, IBM White Paper, www.ibmbigdatahub.com/pdf/Oracle_Exadata_IBMNetezza_Compared_WP_EN.pdf.
[17] Gama, J. (2010). Knowledge Discovery from Data Streams, Chapman and Hall, Boca Raton, FL.
[18] Gama, J., Zliobaite, I., Bifet, A., Pechenizkiy, M. and Bouchachia, A. (2014). A survey on concept drift adaptation, ACM Computing Surveys 46(4): 44:1–44:37.
[19] Gens, F. (2011). IDC predictions 2012: Competing for 2020. IDC analyze the future, https://www.virtustream.com/sites/default/files/IDCTOP10Predictions2012.pdf.
[20] Gessert, F., Schaarschmidt, M., Wingerath, W., Witt, E., Yoneki, E. and Ritter, N. (2017). Quaestor: Query web caching for database-as-a-service providers, PVLDB 10(12): 1670–1681.
[21] Glavic, B. (2014). Big data provenance: Challenges and implications for benchmarking, in T. Rabl et al. (Eds.), Specifying Big Data Benchmarks, Springer, New York, NY, pp. 72–80.
[22] Gupta, A. (2009). Data provenance, in L. Liu and M.T. Özsu (Eds.), Encyclopedia of Database Systems, Springer, Berlin, pp. 608–608.
[23] Han, J. and Kamber, M. (Eds.) (2011). Data Mining. Concepts and Techniques, Morgan Kaufmann, San Francisco, CA.
[24] Hashem, H. and Ranc, D. (2016). Pre-processing and modeling tools for bigdata, Foundations of Computing and Decision Sciences 41(3): 151–162.
[25] Japkowicz, N. and Stefanowski, J. (2016a). A machine learning perspective on big data analysis, in N. Japkowicz and J. Stefanowski (Eds.), Big Data Analysis: New Algorithms for a New Society, Springer, Cham, pp. 1–31.
[26] Japkowicz, N. and Stefanowski, J. (Eds.) (2016b). Big Data Analysis: New Algorithms for a New Society, Studies in Big Data, Vol. 16, Springer, Cham.
[27] Kingma, D.P. and Welling, M. (2013). Auto-encoding variational Bayes, ArXiv e-prints, 1312.6114a.
[28] Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J. and Wozniak, M. (2017). Ensemble learning for data stream analysis: A survey, Information Fusion 37: 132–156.
[29] Krempl, G., Zliobaite, I., Brzezinski, D., H¨ullermeier, E., Last, M., Lemaire, V., Noack, T., Shaker, A., Sievi, S., Spiliopoulou, M. and Stefanowski, J. (2014). Open challenges for data stream mining research, SIGKDD Explorations 16(1): 1–10.
[30] Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks, in F. Pereira et al. (Eds.), Advances in Neural Information Processing Systems 25, Curran Associates, Inc., Red Hook, NY, pp. 1097–1105.
[31] Krawiec, K. (2016). Evolutionary feature selection and construction, in S. Claude and G. Webb (Eds.), Encyclopedia of Machine Learning and Data Mining, Springer, Boston, MA.
[32] Langegger, A., Wöß, W. and Blöchl, M. (2008). A semantic web middleware for virtual data integration on the web, European Semantic Web Conference on the Semantic Web: Research and Applications (ESWC), Tenerife, Canary Islands, Spain, pp. 493–507.
[33] LeCun, Y., Bengio, Y. and Hinton, G. (2015). Deep learning, Nature 521(7553): 436–444.
[34] Liu, M. and Wang, Q. (2016). Rogas: A declarative framework for network analytics, International Conference on Very Large Data Bases (VLDB), New Delhi, India, pp. 1561–1564.
[35] Matwin, S. (2013). Privacy-preserving data mining techniques: Survey and challenges, in B. Custers et al. (Eds.), Discrimination and Privacy in the Information Society, Vol 3. Springer, Berlin/Heidelberg, pp. 209–221.
[36] Mauro, A.D., Greco, M. and Grimaldi, M. (2015). What is big data? A consensual definition and a review of key research topics, International Conference on Integrated Information, Madrid, Spain, pp. 97–104.
[37] Miao, X., Gao, Y., Guo, S. and Liu, W. (2017). Incomplete data management: A survey, Frontiers of Computer Science, DOI: 10.1007/s11704-016-6195-x.
[38] Moreau, L., Clifford, B., Freire, J., Futrelle, J., Gil, Y., Groth, P., Kwasnikowska, N., Miles, S., Missier, P., Myers, J., Plale, B., Simmhan, Y., Stephan, E. and den Bussche, J.V. (2011). The open provenance model core specification (v1.1), Future Generation Computer Systems 27(6): 743–756.
[39] Napierala, K. and Stefanowski, J. (2016). Types of minority class examples and their influence on learning classifiers from imbalanced data, Journal of Intelligent Information Systems 46(3): 563–597.
[40] Naumann, F. (2014). Data profiling revisited, SIGMOD Record 42(4): 40–49.
[41] Rudin, C. (2014). Algorithms for interpretable machine learning, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, pp. 1519–1519.
[42] Russom, P. (2017). Data lakes: Purposes, practices, patterns, and platforms. TDWI White Paper, https://info.talend.com/rs/talend/images/WP_EN_BD_TDWI_DataLakes.pdf.
[43] Schmidhuber, J. (2015). Deep learning in neural networks: An overview, Neural Networks 61(C): 85–117.
[44] Shaker, A. and Hüllermeier, E. (2014). Survival analysis on data streams: Analyzing temporal events in dynamically changing environments, International Journal of Applied Mathematics and Computer Science 24(1): 199–212, DOI: 10.2478/amcs-2014-0015.
[45] Soltanpoor, R. and Sellis, T. (2016). Prescriptive analytics for big data, Australasian Database Conference on Databases Theory and Applications (ADC), Sydney, Australia, pp. 245–256.
[46] Sun, Y., Tang, K., Minku, L.L., Wang, S. and Yao, X. (2016). Online ensemble learning of data streams with gradually evolved classes, IEEE Transactions on Knowledge and Data Engineering 28(6): 1532–1545.
[47] Terrizzano, I., Schwarz, P., Roth, M. and Colino, J.E. (2015). Data wrangling: The challenging journey from the wild to the lake, Conference on Innovative Data Systems Research (CIDR), Asiloma, CA, USA.
[48] Wang, J., Crawl, D., Purawat, S., Nguyen, M.H. and Altintas, I. (2015). Big data provenance: Challenges, state of the art and opportunities, IEEE International Conference on Big Data, Santa Clara, CA, USA, pp. 2509–2516.
[49] Wiederhold, G. (1992). Mediators in the architecture of future information systems, IEEE Computer 25(3): 38–49.
[50] Wylot, M., Cudré-Mauroux, P., Hauswirth, M. and Groth, P.T. (2017). Storing, tracking, and querying provenance in linked data, IEEE Transactions on Knowledge and Data Engineering 29(8): 1751–1764.
[51] Zakhary, V., Agrawa, D. and El Abbadi, A. (2017). Caching at the web scale, International Conference on World Wide Web Companion, Perth, Australia, pp. 909–912.

Uwagi

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).

Typ dokumentu

Bibliografia

Identyfikator YADDA

bwmeta1.element.baztech-ee0c5973-960f-4ba3-950e-1b7ae554f422