MapReduce多数据源信息提取系统的开发与实现

MapReduce, a famous distributed parallel computing model ,is widely used in large-scale parallel processing . However, applications often need to extract lots of information from numbers of heterogeneous, distributed, multiple data sources . Therefore, for MapReuce, shielding the underlying complex data source connection , mapping heterogeneous data sources to a unified interface, providing effectively access to heterogeneous data for upper layer applications, has become imperative for data analysising at this stage. This article will fouce on MapReduce to improve existing methods, improve existing methods, design unified data model for HDFS、HBase and Mysql ,integrated data manipulation code and reduce duplication of code in order to improve development efficiency.