Abstract:
For the issue that Hadoop parameter sets helpful for achieving good performance are difficult to be confirmed in cloud environment, a MapReduce simulator was designed in this paper. Firstly, various parameters of Hadoop were modeled. Then, the parameters were readed from cluster by cluster read element so that simulative Hadoop cluster environment was established. Finally, the simulative job is tracked by using job tracker, and task tracker was used to run a single task. Performance of Hadoop application with multi-angles was researched by the designed simulator, and it concentrated at simulating the Hadoop map and reduction behavior so that it made up the deficiency of the MRPerf designing. The effectiveness of designed simulator was verified by the baseline testing results and MapReduce application of customized definition