基于Storm的流计算框架

Information technology has entered a big-data era, big-data processing becomes even more important.Every industry especially in the Internet industry,they need more hardware resources to handle their increasing data in TB level.MapReduce framework and its open-source implement --Hadoop provide a efficient method to process big data.But people gradually found that Hadoop can't deal with every problems,real-time processing big data is its shortcoming.Storm,a free and open source distributed real-time computation system developed by Twitter can deal this problem.Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing.This paper first give a description of Storm's architecture and principle of operation;then introduce a universal method for real-time processing of data stream;last, author designs a hybrid system that combines real-time processing and batch processing.