What is the expected total discrepancy score in a R choice task?
Suppose we have R possible choices and each of these is equally likely to be the true one.
If we consider a discrepancy as the difference between the true choice and the one given by a subject then
The expected total discrepancy of the absolute value of discrepancies equals
$$\sum_{k=1}^R (k=1)k $$, $$1 \leq k \leq R $$
with the average sum of the absolute values of discrepancies per rating equal to
$$\frac{\sum_{k=1}^R (k=1)k}{R}$$.
For example the table below gives all the abs(discrepancies) for the case where R = 4.
k=1 |
k=2 |
k=3 |
k=4 |
True Rank |
|||
0 |
1 |
2 |
3 |
1 |
|||
1 |
0 |
1 |
2 |
2 |
|||
2 |
1 |
0 |
1 |
3 |
|||
3 |
2 |
1 |
0 |
4 |
|||
Expected total score assuming random guesses at true rank
= 2(1+2+3)+2(1+1+2)= 20
= 1x2 + 2x3 + 3x4
= $$\sum_{k=1}^4 (k=1)k $$.
The average sum of abs(discrepancies) per rating = 20/4 = 5.