×

Testing observer uncertainty in a nominal-scale agreement analysis. (Tests de l’incertitude des observateurs dans une analyse de concordance sur une échelle nominale.) (French) Zbl 0972.62540

Summary: The concept of uncertainty of multiple raters is defined in a context of agreement analysis on a nominal scale. Goodness-of-fit tests are used to test a hypothesis concerning uncertainty. These tests are based on an agreement measure. An aggregation of subjects technique is applied in order to relax the conditions of utilisation of the tests.

MSC:

62P15 Applications of statistics to psychology
PDFBibTeX XMLCite
Full Text: Numdam EuDML

References:

[1] Bloch , D.A. and Kraemer , H.C. ( 1989 ), 2 x 2 Kappa coefficients : measures of agreement or association , Biometrics 45 , 269 - 287 . Zbl 0715.62113 · Zbl 0715.62113 · doi:10.2307/2532052
[2] Cohen , J. ( 1960 ), A coefficient of agreement for nominal scales, Educ. and psychol . Measurement 20 , 37 - 46 .
[3] Cohen , J. ( 1968 ), Weighted kappa : nominal scale agreement with provision for scaled disagreement or partial credit , Psychological Bulletin 70 , 213 - 220 .
[4] Fleiss , J.L. ( 1971 ), Measuring nominal scale agreement among many raters , Psychological Bulletin 76 , N^\circ 5 , 378 - 382 .
[5] Fleiss , J.L. , Nee , J.C.M. and Landis , J.R. ( 1979 ), Large sample variance of kappa in the case of different sets of raters , Psychological Bulletin 86 , 974 - 977 .
[6] Huber , L. ( 1977 ), Kappa revisited , Psychological Bulletin 84 , 289 - 297 .
[7] James , I.R. ( 1983 ), Analysis of nonagreements among multiple raters , Biometrics 39 , 651 - 657 . MR 719930 | Zbl 0523.62091 · Zbl 0523.62091 · doi:10.2307/2531092
[8] Jolayemi , E.T. ( 1990 ), On the measure of agreement between two raters , Biometrical Journal 32 , 87 - 93 .
[9] Kendall , J.R. and Stuart , A. ( 1979 ), The Advanced Theory of Statistics , 4 th ed., Griffin , London . Zbl 0353.62013 · Zbl 0353.62013
[10] Kraemer , H.C. ( 1980 ), Extension of the kappa coefficient , Biometrics 36 , 207 - 216 . Zbl 0463.62103 · Zbl 0463.62103 · doi:10.2307/2529972
[11] Landis , J.R. and Koch , G.G. ( 1975 a), A review of statistical methods in the analysis of data arising from observer reliability studies (Part I , Statistica Neerlandica 29 , 101 - 123 . MR 375711 | Zbl 0414.62051 · Zbl 0414.62051 · doi:10.1111/j.1467-9574.1975.tb00254.x
[12] Landis , J.R. and Koch , G.G. ( 1975 b), A review of statistical methods in the analysis of data arising from observer reliability studies (Part II , Statistica Neerlandica 29 , 151 - 161 . MR 375712 | Zbl 0334.62045 · Zbl 0334.62045 · doi:10.1111/j.1467-9574.1975.tb00259.x
[13] Landis , J.R. and Koch , G.G. ( 1977 ), A one-way components of variance model for categorical data , Biometrics 33 , 671 - 679 . MR 483213
[14] Marshall , A.W. and Olkin , I. ( 1979 ), Inequalities : Theory of Majorization and Its Applications, Mathematics in Science and Engineering , Vol 143 , Academic Press Inc ., New York . MR 552278 | Zbl 0437.26007 · Zbl 0437.26007
[15] Scheffé , H. ( 1959 ), The Analysis of Variance , John Wiley , New York . MR 116429 | Zbl 0086.34603 · Zbl 0086.34603
[16] Schouten , H. J.A. ( 1982 ), Measuring pairwise agreement among many observers. II. Some improvements and additions , Biometrical Journal 24 , N^\circ 5 , 497 - 504 . MR 690481 · Zbl 0491.62093 · doi:10.1002/bimj.4710240502
[17] Schouten , H.J.A. ( 1986 ), Nominal scale agreement among observers , Psychometrika 51 , 453 - 466 . MR 903415
[18] Tassi , P. ( 1989 ), Méthodes Statistiques , 2 e ed., Economica , Paris . MR 1030997
[19] Titterington , D. , Smith , A. and Makov , U. ( 1985 ), Statistical Analysis of Finite Mixture Distributions , John Wiley , New York . MR 838090 | Zbl 0646.62013 · Zbl 0646.62013
[20] Tricot , J.M. ( 1991 ), Un modèle d’accord entre observateurs sur une échelle nominale , Comptes Rendus Mathématiques de l’Académie des Sciences du Canada , Vol XIII , 4 , 146 - 150 . MR 1124728 | Zbl 0745.62059 · Zbl 0745.62059
[21] Yaglom , A.M. et Yaglom , I.M. ( 1969 ), Probabilité et Information , Dunod , Paris .
[22] Zwick , R. ( 1988 ), Another look at interrater agreement , Psychological Bulletin 103 , 374 - 378 .
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.