Correcting biased Cohen’s Kappa in NVivo

This study demonstrates that NVivo, a popular research tool for content analysis, produces biased Cohen’s kappa as it calculates the reliability statistic at the character level, which is unsuitable for most content-analytic studies adopting a higher unit of analysis (e.g., sentence, paragraph). Based on empirical data and a statistical simulation, this study proposes a method of bias correction applicable to situations where two independent raters conduct binary coding.