基于GlusterFS的Hadoop系统的架构设计与实现

he Apache Hadoop is the most popular solution in the industry to store and process extremely large data sets from terabyte to petabyte based on Hadoop Distributed File System(HDFS) on commodity hardware. Since HDFS is a metadata-based distributed file system to store and retrieve any data in it, the metadata server can have much bigger pressure as the data size grows, which obviously can lead to the single point of failure problem. GlusterFS is a scale-out distributed file system that does not depend on a metadata server to locate the data in it, but uses the Elastic Hash algorithm, which makes GlusterFS extremely suitable for linear scalability and have no performance bottlenecks related to metadata.In this paper, we present two different architectures for Hadoop MapReduce enablement on GlusterFS, and evaluate each architecture's performance, then make a comparison with the original architecture based on HDFS+MapReduce.