= How to test for ordering of an unknown nature (ie not necessarily monotonic) in a set of subjects =
Let us suppose we have five subjects which are plotted in 2D space each with three observations ranked from 1 to 3 as shown [[attachment:plot.tif|here.]] We want to see if there is a relationship between the rank orderings and both the x-axis and y-axis variables.
__Comparing to each axis separately__
Now looking at the plot all the five subjects have the same ordering 2,3,1 (going from left to right on each subject line) ie with respect to the x-axis. For the y-axis we have 3 subjects with a 123 and 2 subjects with a 213 (going from bottom to top). So we are breaking down the plot into its basis vectors for the x-axis and y-axis.
Now there are six possible orderings of the three points (6=3!=3x2x1) namely,
123, 132, 213, 231, 321, 312 which we would expect to happen equally often by chance (ie one sixth of the subjects would exhibit each pattern) . We have all five of our subjects showing the same ordering of 231 with respect to the x-axis. The probability of this happening by chance would be 1/6^5 = 0.00013 = P(all five have a particular ordering).
For the y-axis it’s a bit more tricky. We again wish to evaluate the p-value which represents the chance of seeing an equal or more extreme result under H0 (equally likely orderings) than that observed. In this particular case the p-value represents the chance of one or two categories of the six possible orderings occurring. To do this we can use the multinomial distribution (The pmf is given [[http://en.wikipedia.org/wiki/Multinomial_distribution|here]]) which in our case is the probability for n=5 subjects of certain numbers of these falling into one of 6 (=k) categories with the xi the i-th frequencies in each of the six categories. The 1/6^5 result earlier is a special case (as is the binomial distribution).
Now in our example there are five as extreme or more extreme cases of orderings: namely:
1/6^5 All five subjects are in 1 particular category only
5! / (4! 1! 0! 0! 0! 0!) 1/6^5 two particular categories with 4 participants in one of the two
5! / (4! 1! 0! 0! 0! 0!) 1/6^5 two particular categories with 4 participants in the other one
5! / (3! 2! 0! 0! 0! 0!) 1/6^5 two particular categories with 3 subjects in one of the two
5! / (3! 2! 0! 0! 0! 0!) 1/6^5 two particular categories with 3 subjects in the other one
Summing these probabilities (with 0!=1) I get 0.00013 + 0.00064 + 0.00064 + 0.00128 + 0.00128 = 0.00397 = probability of the observed pattern or a more extreme one occurring by chance.
One other thought: we have been talking about one particular category (ordering occurring) e.g. 231 but we could broaden this to talk about any particular ordering occurring in all subjects which would be 6 x 1/6^5 = 0.00077 as the five subjects could all have each of the six orderings.
Similarly for the y-axis variable we could talk about the five subjects all having the same ordering (no specific ordering) and having any two orderings out of a possible six This gives:
6 x 1/6^5
15 x 5! / (4! 1! 0! 0! 0! 0!) 1/6^5 any two categories with 4 participants in one of the two
15 x 5! / (4! 1! 0! 0! 0! 0!) 1/6^5 any two categories with 4 participants in the other
15 x 5! / (3! 2! 0! 0! 0! 0!) 1/6^5 any two categories with 3 subjects in one of the two
15 x 5! / (3! 2! 0! 0! 0! 0!) 1/6^5 any two categories with 3 subjects in the other
= 6 x 0.00013 + 15 x (0.00064+0.00064) + 15 x(0.00128+0.00128) = 0.058. (15 = 6!/2!4! possibilities of just 2 of six orderings occurring)
__Comparing the ranks to the x and y-axis together using their spatial positions__
We could extend the multinomial approach to the characterisation of the 3rd point falling in a specific quadrant with respect to the other two (ie comparing to chance probability of ¼), which takes into account both axes simultaneously. The probability of this (assuming a single a priori order of the first two points) is:
0.25^5 * ( 5!/5! + 2 * 5!/4! + 2 * 5!/(3!2!) ) = 0.25^5 * (1 + 2*5 + 2*10) = .0303
But if there could be any a priori quadrant with respect to the first two points, then this becomes:
0.25^5 * ( 4* 5!/5! + 6 * 2 * 5!/4! + 6 * 2* 5!/(3!2!) ) = 0.25^5 * (4 + 6*2*5 + 6*2*10) = .1797
(6 = 4!/2!2! possibilities of 2 of 4 quadrants occurring).