Abstract: For the ensembles with the same training error, the one with higher margin distribution on training examples often has better generalization performance. Based on this results, a new metric to evaluate the importance of an classifier was designed, and then a new ensemble selection method was proposed. This method initializes the ensemble to be full (or empty), and then iteratively adds into (or remove from) the ensemble the classifier with largest (or smallest) MM to reduce ensemble size and improves its accuracy. The experimental results on 24 data sets showed that, compared with other state-of-the-art greedy ensemble pruning methods, the method performed better generalization ability.