文档介绍：计算机研究与发展
ISSN 1000- 11-1777?TP
Journal puter Research and Development 50(Suppl.):170-179,2013
基于的大数据实体识别算法
Map-Reduce
霍然王宏志朱鎔李建中高宏
哈尔滨工业大学计算机科学与技术学院哈尔滨
( 150001)
(******@hit.)
Map-Reduce Based Entity Identification in Big Data
Huo Ran,Wang Hongzhi,Zhu Rong,Li Jianzhong,and Gao Hong
(School puter Science and Technology,Harbin Institute of Technology,Harbin150001)
Abstract With the development of information technology,problems caused by“big data”and“dirty
data”have aroused widespread concern, which results an extensively research focus in data
management of quality and identification technology is one of the key problems for
quality-quantity management in big data and plays a decisive role in improving the quality of data,
which is to identify different records that describe the same object and the same record forms which
represents different we propose an entity identification algorithm (EIBM)in big data
based on map-reduce under the background of big data information algorithm firstly
computes an attribute-value based similarity between record pairs using map- then output
entity identification results by graph ,we have performed extensive experiments
in the Hadoop platform using the real dataset and artificial experiment results evaluate
the degree of p