doi: 10.7763/IJET.2012.V4.477
Removing Fully and Partially Duplicated Records through K-Means Clustering
Bilal Khan
,
,
,
- Department of Computer Science, University of Peshawar, Pakistan
Abstract
Records duplication is one of the prominent problems in data warehouse. This problem arises when various databases are integrated. This research focuses on the identification of fully as well as partially duplicated records. In this paper we propose a de-duplicator algorithm which is based on numeric conversion of entire data. For efficiency, data mining technique k-mean clustering is applied on the numeric value that reduces the number of comparisons among records. To identify and remove the duplicated records, divide and conquer technique is used to match records within a cluster which further improves the efficiency of the algorithm.
Keywords
- Data cleansing
- De-Duplicator
- partial duplication
- K-Mean clustering
How to Cite
Bilal Khan, Azhar Rauf, Huma Javed, and Shah Khusro, "Removing Fully and Partially Duplicated Records through K-Means Clustering," International Journal of Engineering and Technology, vol. 4, no. 6, pp. 750-754, 2012. https://doi.org/ 10.7763/IJET.2012.V4.477
Copyright & License
Copyright © 2012 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).