This is the supplementary material web page related to the GECCO-2015 paper by Brockhoff, Tran, and Hansen [1]. Below, you will find the source code to reproduce the results of the paper as well as many plots which had to be omitted from the paper due to space limitations.

Note that in the meantime, the code has evolved into the new version of the COCO platform.

Download Source Code (19MB)

Empirical Cumulative Distribution Function (ECDF) plots for all 300 bi-objective problems with search space dimension 2, 3, 5, 10, and 20 can be found in the following pdfs:

single problem ECDFs in 2D (6.9MB)
single problem ECDFs in 3D (6.6MB)
single problem ECDFs in 5D (6.4MB)
single problem ECDFs in 10D (6MB)
single problem ECDFs in 20D (6MB)

The paper only shows the combination of f_2 (separable ellipsoid) with f_3 (separable Rastrigin).

ECDF plots, aggregating over function groups, can be found here (1MB).

The scaling plots of the Empirical Running Time (ERT) of NSGA-II, MOEA/D, and MO-CMA-ES over the search space dimension for all 300 bi-objective problems and a target precision of 1e-3 can be found in the following pdf:

scaling plots of NSGA-II, MOEA/D, and MO-CMA-ES on all 300 problems and f_target = 1e-3 (6MB)

The paper only shows the combination of f_2 (separable ellipsoid) with f_3 (separable Rastrigin), f_1 (sphere) with f_9 (rotated Rosenbrock), f_1 with f_12 (bent cigar), and for the combination of f_2 (separable ellipsoid) with f_16 (Weierstrass).

All ECDF plots, showing the impact of NSGA-II's population size can be found in the following pdf:

ECDFs of NSGA-II with different population sizes in 5D (9MB)

[1] Dimo Brockhoff, Thanh-Do Tran, and Nikolaus Hansen. Benchmarking Numerical Multiobjective Optimizers Revisited. In Genetic and Evolutionary Computation Conference (GECCO 2015), pages 639–646. ACM, 2015. (pdf | suppl. material)