Kappa Calculation - 12-28-2010 by James Lani - Statistics Solutions - http://www.statisticssolutions.com
Kappa Calculation by James Lani http://www.statisticssolutions.com/academic-solutions/resources/dissertation-resources/survey-researchand-administration/kappa-calculation/ Click here for to get help with your Thesis or Dissertation. Click here for FREE Thesis and Dissertation resources (templates, samples, calculators).
There are three steps to calculate a Kappa coefficient: Step one, rater sheets should be filled out for each rater. In the example rater sheet below, there are three excerpts and four themes. Enter 1 in the corresponding cell if the rater thought the theme was present in that excerpt; enter 0 if the rater thought the theme was absent. Rater Sheet: RATER 1 Excerpt # Excerpt 1 Excerpt 2 Excerpt 3 Excerpt 4 Excerpt 5
Theme W 1 0 0 1 1
Theme(1=Present Theme X 1 1 1 0 1
0=Absent) Theme Y 1 0 1 0 1
Theme Z 1 1 1 1 1
Theme(1=Present Theme X 1 0 1 0 1
0=Absent) Theme Y 0 1 1 0 1
Theme Z 0 1 1 1 1
Rater Sheet: RATER 2 Excerpt # Excerpt 1 Excerpt 2 Excerpt 3 Excerpt 4 Excerpt 5
Theme W 1 0 1 1 1
Step two, you will need to fill out a matrix for each theme. For theme W for example, Rater 1 and Rater 2 agreed 3 times that theme W was present in the excerpts, so enter 3 in box A of the matrix. One time Rater 1 thought theme W was absent and Rater 2 thought theme W was present, so enter 1 in box B. Rater 1 never thought a theme was present that Rater 2 thought was absent, so enter 0 in box C. One time Rater 1 and Rater 2 agreed that theme W was absent, so enter 1 in box D. Key
1/3
Kappa Calculation - 12-28-2010 by James Lani - Statistics Solutions - http://www.statisticssolutions.com
Rater 2
Present Absent
Rater 1 Present A C
Absent B D
Present Absent
Rater 1 Present 3 0
Absent 1 1
Present Absent
Rater 1 Present 3 1
Absent 0 1
Present Absent
Rater 1 Present 2 1
Absent 1 1
Present Absent
Rater 1 Present 4 1
Absent 0 0
Theme W
Rater 2
Theme X
Rater 2
Theme Y
Rater 2
Theme Z
Rater 2
Enter your theme matrix values in the calculator to determine your kappa values. Theme W X Y Z
A 3 3 2 4
B 1 0 1 0
C 0 1 1 1
D 1 1 1 0
Observed 0.8 0.8 0.6 0.8
Expected 0.56 0.56 0.52 0.8
Kappa 0.00 0.00 0.00 0.00
2/3
Kappa Calculation - 12-28-2010 by James Lani - Statistics Solutions - http://www.statisticssolutions.com
Click here to calculate your Kappa Coefficients
_______________________________________________
3/3 Powered by TCPDF (www.tcpdf.org)
