id: 05949687
dt: a
an: 05949687
au: Wang, Tao; Neville, Jennifer; Gallagher, Brian; Eliassi-Rad, Tina
ti: Correcting bias in statistical tests for network classifier evaluation.
so: Gunopulos, Dimitrios (ed.) et al., Machine learning and knowledge discovery
    in databases. European conference, ECML PKDD 2011, Athens, Greece,
    September 5‒9, 2011. Proceedings, Part III. Berlin: Springer (ISBN
    978-3-642-23807-9/pbk). Lecture Notes in Computer Science 6913. Lecture
    Notes in Artificial Intelligence, 506-521 (2011).
py: 2011
pu: Berlin: Springer
la: EN
cc: 
ut: Social network analysis; Network classification
ci: 
li: doi:10.1007/978-3-642-23808-6_33
ab: Summary: It is difficult to directly apply conventional significance tests
    to compare the performance of network classification models because
    network data instances are not independent and identically distributed.
    Recent work [6] has shown that paired $t$-tests applied to overlapping
    network samples will result in unacceptably high levels (e.g., up to
    50\%) of Type I error (i.e., the tests lead to incorrect conclusions
    that models are different, when they are not). Thus, we need new
    strategies to accurately evaluate network classifiers. In this paper,
    we analyze the sources of bias (e.g. dependencies among network data
    instances) theoretically and propose analytical corrections to standard
    significance tests to reduce the Type I error rate to more acceptable
    levels, while maintaining reasonable levels of statistical power to
    detect true performance differences. We validate the effectiveness of
    the proposed corrections empirically on both synthetic and real
    networks.
rv: