A note about using ranked outcomes in t-tests and ANOVAs and ANCOVA using Quade's test (see reference at end)
This note aims to show the state-of-play on the use of ranked responses in analysis of variance particularly with regard to testing for interactions.
One way of handling nonparametric interactions is to regress the main effects of the factors on the response to remove main effects and saving the unstandardized residuals and then comparing the combinations of factor levels (formed into a single categorical variable) on the ranked residuals. The use, however, of ranks to assess interactions is controversial. This method (Quade's test) is available as an extension in SPSS V24 for a between subjects ANCOVA. There is also a quade.test procedure in R which is used as an alternative to a Friedman test for a one-way repeated measures ANOVA. A discussion about the use of Quade's test for both repeated measures and between subjects designs is given here.
Conover and Iman (1981) showed that the Mann-Whitney test is equivalent to a t-test on ranked responses. Huitema (1980) in Chapter 12 and Shirley (1981) illustrate the use and interpretation of an analysis of covariance on ranked outcomes with worked examples. Shirley (1987) extends Conover and Iman's work to investigating interactions by presenting a worked example involving a full factorial between subjects analysis of variance. Blair, Sawilowsky and Higgins (1987) and Wilcox (2003, p.604) do not advocate using ranked outcomes to test for interactions 'in more complex designs'. Thompson (1991) takes a conservative stance, stating that ANOVAs on ranked data can be used to test for interactions only "when there are exactly two levels of both main effects or when there is only one main effect" (p. 697). Akritas (1991) shows that rank transformed responses are not applicable to repeated measures ANOVA.
Leys and Schumann (2010) present a step-by-step worked example which tests for an interaction in a pair of between subjects factor ANOVA (see here) using an adjusted rank transform with a working input spreadsheet. This spreadsheet will work out a nonparametric two-way interaction for 2 between subjects factors (BB) each having two levels. The response and group raw data and number of cases are entered into Sheet1 and the results displayed in Sheet2. The calculations follow the method illustrated by the simple example in Leys and Schumann (2010). This approach involves testing the two-way interaction tests on ranked data when the main effects have been removed and is known as the adjusted rank transformation test (ART). A .exe file for use on windows to perform more general ART tests for nonparametric interactions for factorial ANOVAs is downloadable from here. There is also an ART procedure in R which extends to a greater range of ANOVA models.
Mansouri (1998) also presents some examples using SAS for looking at interactions between two between subjects factors (see here). Mansouri and Chang (1995) note down problems using rank tranforms when responses follow a Cauchy distribution and advocate using Normal scores to downweight the inflated Type I error.
More recently Zimmerman (2011) has shown that using t-tests on ranked outcomes, even when there are a large proportion of equal scores (having tied ranks), have greater power and smaller type I error than the conventional t-test on raw scores when the outcome is not normally distributed. He also claims using ranked outcomes works well with one-way ANOVAs.
Beasley TM, Zumbo BD (2009) give SAS code for fitting interactions for a two-way interaction involving one between and one within subjects factor using ranked aligned responses adjusted for main effects. They do warn that a limitation of using aligned ranks to test for the interaction is the need for the distributions and variances in the groups being compared (which can be skewed) to be the same, otherwise the presence of interactions may be wrongly inferred. Beasley (2002) finds such an approach can control for type I error in non-Normal data. A freeware tool (Wobbrock et al. (2011)) which also aids in performing aligned ranks analysis by producing aligned ranks prior to analysis by ANOVA is available.
Wilcox (2020) suggests using trimmed means in analysis of variance with the WRS2 procedure in R. A more thorough treatment is in Wilcox (2017). Haan et al. (2009) found trimmed means performed well in an analysis of variance on gene data.
An alternative strategy ignoring ranks altogether is to fit a generalized linear mixed model (GLMM) assuming responses are counts and follow a poisson distribution. In this way interactions for repeated measures data may be fitted e.g. to 20 people in one of four diagnostic groups each being assessed in three positions (lying, sitting and standing). One inputs data in 'long' format consisting of a single response with columns indicating subject, response, group and position. The GLMM can then be fitted in R reading a SPSS data file and using the syntax below. An ordinal or multinomial logistic regression could similarly be fitted to a grouped response to assess interactions.
References
Akritas, MG (1991) Limitations of the rank transform procedure: a study of repeated measures designs, part 1. Journal of the Americal Statistical Association 86(414) 457-460.
Beasley, TM (2002) Multivariate aligned rank test for interactions in multiple group repeated measures designs. Multivariate Behavioral research 37(2), 197-226.
Beasley, TM, Zumbo, BD (2009) Aligned Rank Tests for Interactions in Split-Plot Designs:Distributional Assumptions and Stochastic Heterogeneity. Journal of Modern Applied Statistical Methods 8(1), 16-50
Blair, RC, Sawilowsky, SS and Higgins, JJ (1987). Limitations of the rank transform statistic in tests for interactions. Communications in Statistics - Simulation and Computation 16, 1133-1145.
Conover, WJ and Iman, RL (1981). Rank transformations as a bridge between parametric and nonparametric statistics. American Statistician 35 124-129. doi:10.2307/2683975
de Haan JR, Bauerschmidt S, van Schaik RC, Piek E, Buydens LMC and Wehrens R (2009). Robust ANOVA for microarray data. Chemometrics and Intelligent Laboratory Systems 98 38-44.
Edgington, ES (1995). Randomization tests. Third Edition. Marcel Dekker Inc:New York.
Huitema, BE (1980). The analysis of covariance and alternatives. 2nd Edition. Wiley: New York.
Leys, C. and Schumann, S (2010). A nonparametric method to analysze interactions: the adjusted rank transform test. Journal of Experimental Psychology 46 684-688.
Mansouri, H. (1998). Multifactor analysis of variance based on the aligned rank transform technique. Computational Statistics & Data Analysis 29(2) 177-189.
Mansouri, H. and Chang, G-H. (1995). A comparative study of some rank tests for interaction. Computational Statistics & Data Analysis 19(1), 85-96.
Quade, D (1967). Rank analysis of covariance. Journal of the American Statistical Association 62 1187-1200.
Shirley, EAC (1981). A distribution-free method for analysis of covariance based on ranked data. Applied Statistics 30 158-162.
Shirley, EAC (1987). Applications of ranking methods to multiple comparison procedures and factorial experiments. Applied Statistics 36(2) 205-213.
Thompson, GL (1991). A note on the rank transform for interactions. Biometrika 78 697–701. This is quite a mathematical paper but it suggests under certain conditions the sums of squares for a two-way interaction from a ANOVA on a ranked response can be compared to a chi-square distribution.
Wilcox, RR (2003, p.604). Applying contemporary statistical techniques. Academic Press:New York.
Wilcox, R (2017). Introduction to Robust Estimation and Hypothesis Testing, chapter 8.6 Between-by-Within and Within-by-Within Designs. Academic Press: Cambridge, MA.
Wilcox, R. (2020). Robust statistical methods in R using the WRS2 package Behavior Research Methods 52 464–488
Zimmerman, DW (2011). Power comparisons of significance tests of location using scores, ranks, and modular ranks. British Journal of Mathematical and Statistical Psychology 64 233-243.