Display options
Cite
Morris R, MacNeela P, Scott A, et al. Ambiguities and conflicting results: the limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: a discussion paper. Int J Nurs Stud. 2007;45(4):645-7doi: 10.1016/j.ijnurstu.2007.07.005.
Morris, R., MacNeela, P., Scott, A., Treacy, P., Hyde, A., O'Brien, J., Lehwaldt, D., Byrne, A., & Drennan, J. (2008). Ambiguities and conflicting results: the limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: a discussion paper. International journal of nursing studies, 45(4), 645-7. https://doi.org/10.1016/j.ijnurstu.2007.07.005
Morris, Roisin, et al. "Ambiguities and conflicting results: the limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: a discussion paper." International journal of nursing studies vol. 45,4 (2008): 645-7. doi: https://doi.org/10.1016/j.ijnurstu.2007.07.005
Morris R, MacNeela P, Scott A, Treacy P, Hyde A, O'Brien J, Lehwaldt D, Byrne A, Drennan J. Ambiguities and conflicting results: the limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: a discussion paper. Int J Nurs Stud. 2008 Apr;45(4):645-7. doi: 10.1016/j.ijnurstu.2007.07.005. Epub 2007 Oct 23. PMID: 17959177.
Copy Download .nbib Format: NLM
Share it on

Int J Nurs Stud. 2008 Apr;45(4):645-7. doi: 10.1016/j.ijnurstu.2007.07.005. Epub 2007 Oct 23.

Ambiguities and conflicting results: the limitations of the kappa statistic in establishing the interrater reliability of the Irish nursing minimum data set for mental health: a discussion paper.

International journal of nursing studies

Roisin Morris, Padraig MacNeela, Anne Scott, Pearl Treacy, Abbey Hyde, Julian O'Brien, Daniella Lehwaldt, Anne Byrne, Jonathan Drennan

Affiliations

  1. Dublin City University, Ollscoil Chathair Bhaile Atha Cliath 9, Eire, Dublin, Ireland. [email protected]

PMID: 17959177 DOI: 10.1016/j.ijnurstu.2007.07.005

Abstract

BACKGROUND: In a study to establish the interrater reliability of the Irish Nursing Minimum Data Set (I-NMDS) for mental health difficulties relating to the choice of reliability test statistic were encountered.

OBJECTIVES: The objective of this paper is to highlight the difficulties associated with testing interrater reliability for an ordinal scale using a relatively homogenous sample and the recommended kw statistic.

METHOD: One pair of mental health nurses completed the I-NMDS for mental health for a total of 30 clients attending a mental health day centre over a two-week period. Data was analysed using the kw and percentage agreement statistics.

RESULTS: A total of 34 of the 38 I-NMDS for mental health variables with lower than acceptable levels of kw reliability scores achieved acceptable levels of reliability according to their percentage agreement scores.

CONCLUSION: The study findings implied that, due to the homogeneity of the sample, low variability within the data resulted in the 'base rate problem' associated with the use of kw statistic. Conclusions point to the interpretation of kw in tandem with percentage agreement scores. Suggestions that kw scores were low due to chance agreement and that one should strive to use a study sample with known variability are queried.

MeSH terms

Publication Types