Advanced Search

Query:

Fill in the form and click »Search«...

Format:

Display: entries per page entries

Zbl 1222.68184
Demšar, Janez
Statistical comparisons of classifiers over multiple data sets. (English)
[J] J. Mach. Learn. Res. 7, 1-30 (2006). ISSN 1532-4435; ISSN 1533-7928/e

Summary: While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.

MSC 2000:: *68T05 Learning and adaptive systems
62G30 Order statistics, etc.

Keywords: comparative studies; statistical methods; Wilcoxon signed ranks test; Friedman test; multiple comparisons tests

Cited in: Zbl 1225.68178

Comment on this Item PDF MathML XML ASCII DVI PS BibTeX Online Ordering Article Journal

	Scientific prize winners of the ICM 2010
	Overhang
	Lie groups, physics and geometry. An introduction for physicists, engineers and chemists.

Advanced Search

Zentralblatt MATH Berlin [Germany]