Integration of data from heterogeneous sources using ETL technology

• Autorzy: Macura M.

Identyfikatory

DOI

10.7494/csci.2014.15.2.109

• Języki publikacji: EN

Abstrakty

EN

Data integration is a crucial issue in the environments of heterogeneous data sources. At present, the afore-mentioned heterogeneity is becoming widespread. Based on various data sources, if we want to gain useful information and knowledge, we must solve data integration problems in order to apply appropriate analytical methods to comprehensive and uniform data. Such activity is known as knowledge discovery from the data process. Therefore, approaches to the data integration problem are very interesting and bring us closer to the “age of information”. This paper presents an architecture which implements knowledge discovery from the data process. The solution combines ETL technology and a wrapper layer known from mediated systems. It also provides semantic integration through connection mechanism between data elements. The solution allows for integration of any data sources and implementation of analytical methods in one environment. The proposed environment is verified by applying it to data sources in the foundry industry.

Słowa kluczowe

EN

data integration; integration approaches; ETL technology; knowledge discovery from data; business intelligence

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2014
• Tom: Vol. 15 (2)
• Strony: 109--132
• Opis fizyczny: Bibliogr. 24 poz., rys., wykr.

Twórcy

autor

Macura M.

• AGH University of Science and Technology, Krakow, Poland

Bibliografia

• [1] Calvanese D., Giacomo G.D., Lenzerini M., Nardi D., Rosati R.: Source Integration in Data Warehousing.DEXA Workshop, 1998.
• [2] Calvanese D., Giacomo G.D., Lenzerini M., Nardi D., Rosati R.: A Principle Approach to Data Integration and Reconciliation in Data Warehousing. In: Proceedings of the International Workshop on Design and Management of Data Warehouses, 1999.
• [3] Calvanese D., Giacomo G. D., Lenzerini M., Nardi D., Rosati R.: Data Integration in Data Warehousing. Int. J. Cooperative Inf. Syst., 2001.
• [4] Doan A., Halevy A., Ives Z.: Principles of Data Integration. Morgan Kaufmann, 2012.
• [5] Halevy A. Y., Rajaraman A., Ordille J. J.: Data Integration: The Teenage Years. VLDB , 2006.
• [6] Han J., Kamber M.:Data Mining: Concepts and Techniques. Morgan Kaufmann, 2012.
• [7] Hull R., Zhou G.: A Framework for Supporting Data Integration Using the Materialized and Virtual Approaches. 1996.
• [8] Inmon W. H.: Building the Data Warehouse. Wiley Publishing, Inc., 2005.
• [9] Ives Z. G.: Efficient Query Processing for Data Integration. A dissertation for the degree of Doctor of Philosophy, 2002.
• [10] Kermanshahani S.: Semi-materialized framework: a hybrid approach to data integration. ACM, 2008.
• [11] Kermanshahani S.:IXIA (IndeX-based Integration Approach) A Hybrid Approach to Data Integration. A dissertation for the degree of Doctor of Philosophy, 2009.
• [12] Kimball R., Caserta J.:The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data. Wiley Publishing, Inc., 2004.
• [13] Kimball R., Ross M.:The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling. Wiley Publishing, Inc., 2002.
• [14] Koch C.: Data Integration against Multiple Evolving Autonomous Schemata CERN-THESIS-2001-036, 2001.
• [15] Kurgan L. A., Musilek P.: A survey of Knowledge Discovery and Data Mining process models.The Knowledge Engineering Review, 2006.
• [16] Lenzerini M.: Data Integration: A Theoretical Perspective. PODS ’02 Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 2002.
• [17] Levy A. Y.: The Information Manifold Approach to Data Integration. IEEE Intelligent Systems, 1998.
• [18] Negash S.: Business Intelligence. AMCIS, 2003.
• [19] Tatbul N., Karpenko O., Convey C., Yan J.: Data Integration Services. Brown University, Computer Science, 2001.
• [20] Vassiliadis P.: A Survey of Extract-Transform-Load Technology. Integrations of Data Warehousing, Data Mining and Database Technologies, 2011.
• [21] Vassiliadis P., Simitsis A.: Extraction, Transformation, and Loading. Encyclopedia of Database Systems, 2009.
• [22] Vercellis C.: Business Intelligence: Data Mining and Optimization for Decision Making. A John Wiley and Sons, Ltd., 2009.
• [23] Widom J.: Research Problems in Data Warehousing. In: Proceedings of International Conference on Information and Knowledge Management, 1995.
• [24] Wiederhold G.: Mediators in the architecture of future information systems. IEEE COMPUTER, 1992