HDFS数据节点本地缓存的设计与实现

s the increase of the Internet applications and network data, the process and storage of massive data has become one of the most important problems. Hadoop distributed file system is a file system developed by Apache Hadoop project which is designed to run on commodity hardware. HDFS is high reliable ,tolerable and able to provide high throughput access to large data sets. It is proper for storage and distributed computing for massive data sets. However, the frequent access of small data segments will bring frequent disk I/O operations and causes the disk bottlenecks and resource overloads of servers. To solve this problem, this paper proposes a solution that appending a local cache function to DataNodes of HDFS. The source code of reading data part of HDFS has been analyzed and modified to implement the DataNode local cache of HDFS. The experiments results validate that the data access rate are improved. It also prove that CPU and disk utilization of servers are decreased.