Storm下基于最佳并行度的贪心调度算法

Open-source distributed real-time computing framework Storm in the Internet, finance, e-commerce and other fields has been widely used. By default, Storm uses the polling scheduling policy and relies on the user's configuration of Topology tasks in parallel. When the configuration is unreasonable, Storm still causes delays in Topology processing and decreases throughput. To solve this problem, this paper proposes a Greedy Scheduling Algorithm based on best parallelism in Storm. When scheduling, the best parallelism of each component in Topology task is solved first, and then greedy policy is adopted to minimize the network communication between nodes. Compared with the default scheduling algorithm, the online scheduling algorithm and the hot-side scheduling algorithm, the results show that the algorithm can effectively reduce the Storm processing delay and improve the system throughput and resource utilization.