Fleiss' Kappa Calculator
Measure agreement between three or more raters, corrected for chance. Paste one row per subject with the count of raters who chose each category.
Built into PaperSurvey.io
Skip the copy-paste. Scan your paper or web surveys and PaperSurvey computes these metrics automatically on your real data, ready to export to Excel, SPSS and R. See how
When to use Fleiss' kappa
Cohen's kappa handles exactly two raters. Fleiss' kappa extends the idea to any fixed number of raters, and the raters do not have to be the same people for every subject. You supply how many raters assigned each category to each subject rather than individual rating pairs.
Interpreting the result
Kappa ranges up to 1, where 1 is perfect agreement and 0 is agreement no better than chance. Following Landis and Koch, values below 0.20 are slight, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial and above 0.80 almost perfect. Negative values mean less agreement than chance alone would produce.
Frequently asked questions
How should I lay out my data?
One row per subject, one column per rating category. Each cell is the number of raters who picked that category for that subject, and every row must add up to the same total number of raters.
Get Started with PaperSurvey.io Software
Start your 14-day free trial now, no credit card required.