E-ETL: framework for managing evolving ETL workflows

• Autorzy: Wojciechowski A.
• Języki publikacji: EN

Abstrakty

EN

Data warehouses integrate external data sources (EDSs), which very often change their data structures (schemas). In many cases, such changes cause an erroneous execution of an already deployed ETL workflow. Structural changes of EDSs are frequent, therefore an automatic reparation of an ETL workflow, after such changes, is of a high importance. This paper presents a framework, called E-ETL, for handling the evolution of an ETL layer. Detection of changes in EDSs causes a reparation of the fragment of ETL workflow which interacts with the changed EDSs. The proposed framework was developed as a module external to a standard commercial or open-source ETL engine, accessing the engine by means of API. The innovation of this framework consists in: (1) the algorithms for semi-automatic reparation of an ETL workflow and (2) its ability to interact with various ETL engines that provide API.

Słowa kluczowe

EN

data warehouse; data integration; ETL; data source evolution; ETL evolution; E-ETL

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2013
• Tom: Vol. 38, No. 2
• Strony: 131--142
• Opis fizyczny: Bibliogr. 10 poz., tab., fig.

Twórcy

autor

Wojciechowski A.

• Poznań University of Technology, Institute of Computing Science, Poznań, Poland

Bibliografia

• [1] G. Papastefanatos, P. Vassiliadis, A. Simitsis, T. Sellis, and Y. Vassiliou, "Rule- based Management of Schema Changes at ETL sources," in Proc. of Conf. Advances in Databases and Information Systems Workshops (ADBIS), pp. 55{62, Springer, LNCS 5968, 2010.
• [2] E. A. Rundensteiner, A. Koeller, X. Zhang, A. J. Lee, A. Nica, A. Van Wyk, and Y. Lee, "Evolvable View Environment (EVE): Non-Equivalent View Maintenance under Schema Changes," in Proc. of ACM Int. Conf. on Management of Data (SIGMOD), pp. 553-555, ACM Press, 1999.
• [3] A. Wojciechowski, "E-ETL: Framework For Managing Evolving ETL Processes," in Proc. of Conf. Advances in Databases and Information Systems Workshops (ADBIS), vol. 185 of Advances in Intelligent Systems and Computing, pp. 441- 449, Springer, 2013.
• [4] R. Wrembel and B. Bebel, "The Framework for Detecting and Propagating Changes from Data Sources Structure into a Data Warehouse," Foundations of Computing & Decision Sciences, vol. 30, no. 4, pp. 361-372, 2005.
• [5] A. Wojciechowski and R. Wrembel, "Research Problems of the ETL Technology.," Foundations of Computing and Decision Sciences, vol. 35, no. 5, pp. 283-306, 2010.
• [6] R. Wrembel, "On handling the evolution of external data sources in a data ware- house architecture," in Data Mining and Database Technologies: Innovative Approaches (D. Taniar and L. Chen, eds.), IGI Group, 2011.
• [7] G. Papastefanatos, P. Vassiliadis, A. Simitsis, and Y. Vassiliou, "Policy- Regulated Management of ETL Evolution.," J. Data Semantics, pp. 147-177, 2009.
• [8] A.Wojciechowski, "E-ETL: Framework For Managing Evolving ETL Processes.," in Proc. of Ph.D. Students in Information and Knowledge Management Work- shop (PIKM), pp. 59-66, ACM Press, 2011.
• [9] J. Eder, C. Koncilia, and T. Morzy, "The COMET Metamodel for Temporal Data Warehouses," in Proc. of Int. Conf. on Advanced Information Systems Engineering (CAiSE), pp. 83-99, Springer-Verlag, 2002.
• [10] E. A. Rundensteiner, A. Koeller, and X. Zhang, "Maintaining data warehouses over changing information sources," Communications of the ACM, vol. 43, no. 6, pp. 57-62, 2000.