Data cleaning of medical data sets

• Autorzy: Widera A. , Widera M. , Feige D. , Horoba K. , Stankiewicz A.
• Języki publikacji: EN

Abstrakty

EN

Each database system evolves during the time. If the primary database schema was designed only to store the limited scope of abstraction classes then the database system improvement process is performed in traditional way (using alter table, update table and create table commands). Anyhow it could be impossible, from the engineering point of view, or too expensive from the economic point of view. Transferring the data from one database schema to another database schema one has to perform an additional step called Data Cleaning. This paper present a basic sketch for the data cleaning theory based on the materialised views idea and corresponding data cleaning environment. The proposed methodology is suitable not only for the data verification but also for the reengineering of the broken references between data fields, recreation of missing rows and data types conversion.

Słowa kluczowe

EN

data cleaning; materialised views; knowledge discovery in medical databases

PL

czyszczenie danych; zmaterializowane widoki; odkrywanie wiedzy w bazach danych medycznych

• Wydawca: University of Silesia, Institute of Informatics, Computer Systems Department
• Czasopismo: Journal of Medical Informatics & Technologies
• Rocznik: 2004
• Tom: Vol. 8
• Strony: MM129--140
• Opis fizyczny: Bibliogr. 15 poz., rys., tab.

Twórcy

autor

Widera A.

• Institute of Medical Technology and Equipment, 118 Roosevelt St, 41-800 Zabrze, POLAND

autor

Widera M.

• Institute of Medical Technology and Equipment, 118 Roosevelt St, 41-800 Zabrze, POLAND

autor

Feige D.

• Institute of Medical Technology and Equipment, 118 Roosevelt St, 41-800 Zabrze, POLAND

autor

Horoba K.

• Institute of Medical Technology and Equipment, 118 Roosevelt St, 41-800 Zabrze, POLAND

autor

Stankiewicz A.

• Institute of Medical Technology and Equipment, 118 Roosevelt St, 41-800 Zabrze, POLAND

Bibliografia

• [1] MONGE A. E.: Matching Algorithms within a Duplicate Detection System. IEEE Bulletin of the Technical Committee on Data Engineering, 2000, t. 23, nr 4, s. 14-20.
• [2] NAVARRO G., BAEZA-YATES R., SUTINEN E., TARHIO J.: Indexing Methods for Approximate String Matching. IEEE Bulletin of the Technical Committee on Data Engineering, 2001, t. 24 nr 4, s. 19-27.
• [3] RAHM E., DO H.: Data Cleaning: Problems and Current Approaches. IEEE Bulletin of the Technical Committee on Data Engineering, 2000, t. 23 nr 4, s. 3-13.
• [4] GALHARDAS H., FLORESCU D., SHASHA D.: Declarative Data Cleaning: Language, Model, and Algorithms. INRIA Technical Report RR-4149, 2001.
• [5] GAŁECKA, J., JAROCKI B., ŻMUDZIŃSKI J., KOSIŃSKI W., BADURA G., FIGA D.: Measurement station with database IMPULS supporting the management of patients with implanted pacemaker. Prace Naukowe Instytutu Górnictwa Politechniki Wrocławskiej Seria: Konferencje. 1999, Nr 24, s. 133-137.
• [6] GALHARDAS H., FLORESCU D., SHASHA D., SIMON E.: AJAX: An Extensible Data Cleaning Tool. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, Texas, USA, 2000, s. 590.
• [7] VASSILIADIS P., VAGENA Z., SKIADOPOULOS S., KARAYANNIDIS N., SELLIS T.: ARKTOS: A Tool For Data Cleaning and Transformation in Data Warehouse Environments. IEEE Bulletin of the Technical Committee on Data Engineering, 2000, t. 23 nr 4, s. 42- 47.
• [8] RAMAN V., HELLERSTEIN J. M.: Potter's Wheel: An Interactive Data Cleaning System. Proceedings of 27th International Conference on Very Large Data Bases, September 11-14, 2001, Roma, Italy. 2001, s. 381-390.
• [9] CUI Y., WIDOM J.: Lineage tracing for general data warehouse transformations. The VLDB Journal - The International Journal on Very Large Data Bases, 2003, t. 12, z. 1, s. 41-58.
• [10] SUNG S. Y., LI Z., SUN P.: A Fast Filtering Scheme for Large Database Cleansing. Proceedings of the eleventh international conference on Information and knowledge management 2002, McLean, Virginia, USA , November 04-09 2002, 2002, s. 76-83.
• [11] HERBERT K. G., GEHANI N. H., PIEL W.H., WANG J. T. L., WU C. H.: BIO-AJAX: An Extensible Framework for Biological Data Cleaning. SIGMOD Record , 2004, t. 33, nr 2.
• [12] VASSILIADIS P., SIMITSIS A., SKIADOPOULOS S.: Conceptual Modeling for ETL Processes. Proceedings of the 5th ACM international workshop on Data Warehousing and OLAP 2002, McLean, Virginia, USA, November 08008 2002, 2002, s. 14-21.
• [13] MALETIC J. I., MARCUS A.: Data Cleansing: Beyond Integrity Analysis. Proceedings of the 5th conference on Information Quality IQ2000, Boston, MA, USA, October 20-23, 2000, s. 200-209.
• [14] HERNANDEZ M. A., STOLFO S. J.: Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem. Data Mining and Knowledge Discovery, 1998, t. 2 nr 1, s. 2-37.
• [15] WIDERA A., WIDERA M., OWCZAREK A., MOMOT. M., SIKORA M., Metoda oczyszczania danych w oparciu o mechanism perspektyw materializowanych. Współczesne problemy sieci komputerowych - Nowe technologie. red. S. Węgrzyn, B. Pochopień, T. Czachórski, Wydawnictwa Naukowo-Techniczne, s. 401, Warszawa, 2004.