How to test for ordering of an unknown nature (ie not necessarily monotonic) in a set of subjects
Let us suppose we have five subjects which are plotted in 2D space each with three observations ranked from 1 to 3 as shown here. We want to see if there is a relationship between the rank orderings and both the x-axis and y-axis variables.
Comparing to each axis separately
Now looking at the plot all the five subjects have the same ordering 2,3,1 (going from left to right on each subject line) ie with respect to the x-axis. For the y-axis we have 3 subjects with a 123 and 2 subjects with a 213 (going from bottom to top). So we are breaking down the plot into its basis vectors for the x-axis and y-axis.
Now there are six possible orderings of the three points (6=3!=3x2x1) namely,
123, 132, 213, 231, 321, 312 which we would expect to happen equally often by chance (ie one sixth of the subjects would exhibit each pattern) . We have all five of our subjects showing the same ordering of 231 with respect to the x-axis. The probability of this happening by chance would be 1/6^5 = 0.00013 = P(all five have a particular ordering).
For the y-axis it’s a bit more tricky. We again wish to evaluate the p-value which represents the chance of seeing an equal or more extreme result under H0 (equally likely orderings) than that observed. In this particular case the p-value represents the chance of one or two categories of the six possible orderings occurring. To do this we can use the multinomial distribution (The pmf is given here) which in our case is the probability for n=5 subjects of certain numbers of these falling into one of 6 (=k) categories with the xi the i-th frequencies in each of the six categories. The 1/6^5 result earlier is a special case (as is the binomial distribution).
Now in our example there are five as extreme or more extreme cases of orderings: namely:
1/6^5 All five subjects are in 1 particular category only
5! / (4! 1! 0! 0! 0! 0!) 1/6^5 two particular categories with 4 participants in one of the two 5! / (4! 1! 0! 0! 0! 0!) 1/6^5 two particular categories with 4 participants in the other one
5! / (3! 2! 0! 0! 0! 0!) 1/6^5 two particular categories with 3 subjects in one of the two 5! / (3! 2! 0! 0! 0! 0!) 1/6^5 two particular categories with 3 subjects in the other one
Summing these probabilities (with 0!=1) I get 0.00013 + 0.00064 + 0.00064 + 0.00128 + 0.00128 = 0.00397 = probability of the observed pattern or a more extreme one occurring by chance.
One other thought: we have been talking about one particular category (ordering occurring) e.g. 231 but we could broaden this to talk about any particular ordering occurring in all subjects which would be 6 x 1/6^5 = 0.00077 as the five subjects could all have each of the six orderings.
Similarly for the y-axis variable we could talk about the five subjects all having the same ordering (no specific ordering) and having any two orderings out of a possible six This gives:
6 x 1/6^5
15 x 5! / (4! 1! 0! 0! 0! 0!) 1/6^5 any two categories with 4 participants in one of the two 15 x 5! / (4! 1! 0! 0! 0! 0!) 1/6^5 any two categories with 4 participants in the other
15 x 5! / (3! 2! 0! 0! 0! 0!) 1/6^5 any two categories with 3 subjects in one of the two 15 x 5! / (3! 2! 0! 0! 0! 0!) 1/6^5 any two categories with 3 subjects in the other
= 6 x 0.00013 + 15 x (0.00064+0.00064) + 15 x(0.00128+0.00128) = 0.058. (15 = 6!/2!4! possibilities of just 2 of six orderings occurring)
Comparing the ranks to the x and y-axis together using their spatial positions
We could extend the multinomial approach to the characterisation of the 3rd point falling in a specific quadrant with respect to the other two (ie comparing to chance probability of ¼), which takes into account both axes simultaneously. The probability of this (assuming a single a priori order of the first two points) is:
0.255 * ( 5!/5! + 2 * 5!/4! + 2 * 5!/(3!2!) ) = 0.255 * (1 + 2*5 + 2*10) = .0303
But if there could be any a priori quadrant with respect to the first two points, then this becomes:
0.255 * ( 4* 5!/5! + 6 * 2 * 5!/4! + 6 * 2* 5!/(3!2!) ) = 0.255 * (4 + 6*2*5 + 6*2*10) = .1797
(6 = 4!/2!2! possibilities of 2 of 4 quadrants occurring).