Random effect models are particularly useful for handling groups of either nested (where they are also called multilevel models) or crossed data. An example of the former is classes in schools in regions in countries or patients visiting different clinics. Bickel (2007, p.282) suggests having at least 20 groups (e.g. clinics) each containing 30 observations (e.g. patients) to enable the fitting of multilevel models. An example of crossed data is repeated measurements e.g. exam scores over time where different correlation structures can be fitted to repeated measures. They are useful if you wish to generalise from an observed set of factors, such as a particular set of words (items) having the same stem, to the general population (e.g. of all words having this particular stem) in that they take account of sampling variation measured on a set of subjects. Some authors such as Krystal (2004) recommend using random effects models (of factors and covariates) over repeated measures ANOVA. The repeated measures is actually a special case of a subject OR item random effect which can be modelled using random intercepts (see e.g. Liu, Rovine and Molenarr (2012), p.19). More general random effect models are also able to use incomplete data e.g. over time points where not every participant has completed responses at each time point (Hedeker and Gibbons (1997)). Galbraith, Daniel and Vissel (2010) illustrate uses of multilevel models in neuroimaging studies to account for cluster effects and their paper is presented here.
Advantages from comparing the use of random effect models compared to repeated measures ANOVA is given here. This is reproduced here (if the link is broken). Thom Baguley also makes some comments about multilevel and RM ANCOVA approaches to analysing baseline and follow-up repeated measures data here.
Nan Laird mentions that the EM algorithm, based upon summary measures from incomplete data, may be used to estimate mixed model parameters (Laird and Hirschland, 2021).
Examples of the fitting of these models using MIXED and GLM in SPSS are given in pdf format here. Peugh and Enders (2005) also illustrate the fitting of random effect models using SPSS. An example using SPSS to fit random effects models with repeated measures (e.g. time) nested within subjects are given here.
Baguley (2012) shows repeated measures one-way ANOVA can be expressed as a random effects model with subject, repeated measures factor, time, (within subject) for a given outcome in SPSS.
MIXED outcome BY time
/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0,
ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)
/FIXED=time | SSTYPE(3)
/METHOD=REML
/REPEATED=time | SUBJECT(subject) COVTYPE(CS).Examples of the fitting of these models using the SAS MIXED procedure are here.
Information and downloadable multilevel and nlme libraries in R is available from here where this pdf file about fitting multilevel models in R was taken. Links to pdf files giving an excellent comprehensive illustration using lmer and glmer procedures in R by Bates and Rahway are given here. R does not like to give p-values for terms in random effect models but there is a function called pvals.fnc which can be used (details are given here.) or, more recently, the R function lmertest will add on a p-value for fixed effects when used with lmer. In particular if one uses pvals.fnc in later versions of R one obtains the message below.
Error in pvals.fnc(model_both) : MCMC sampling is no longer supported by lme4. For p-values, use the lmerTest package, which provides functions summary() and anova() which give p-values of various kinds.
(Thanks to Tanya Wen for alerting me to this).
A MS Word file with Basic course material on using R for beginners is here. An example using R to perform mixed ANOVA with code for computing and plotting confidence intervals for predictions for groups at each time point is given here. (with thanks to Adam Wagner).
An overview (with applications) comparing usage in SPSS, SAS and R is given in this pdf file.
You can, if from an academic institution, download for free MLwiN software which was developed by researchers at the Institute of Education in London. There are, however, no free manuals available but there is a comprehensive multilevel modeling bibliography. MLwiN can also be run from inside R using a R software interface (see here).
- Formulating as a structural equation model see Curran and Bauer (2007) and Dunn et al(2002).
- Baird and Maxwell (2016) discuss choosing between treating the time variable in a model as fixed or random and suggest, due to convergence difficulties, that time should be treated as a fixed factor/variable unless the samples are very large.
References
Baayen RH (2008) Analyzing linguistic data. A practical introduction to statistics using R. Cambridge University Press. A PDF copy of this book is available from here. This book is also in the CBSU library.
Baguley T (2012) Serious Stats. A guide to advanced statistics for the behavoral sciences. Palgrave MacMillan:new York. Chapter 18 gives comprehensive coverage and illustrations of random effect models.
Baird R and Maxwell SE (2016) Performance of time-varying predictors in multilevel models under an assumption of fixed or random effects. Psychological Methods 21(2) 175-188.
Bickel R (2007) Multilevel Analysis for Applied Research: It's Just Regression! (Methodology in the Social Sciences). The Guilford Press:New York. How to do multilevel models using SPSS. [Only £30 in Paperback and in CBU library]. Andy Field's (2009) Discovering Statistics using SPSS. Third Edition. Sage:London covers multilevel models in SPSS in chapter 19.
Curran PJ and Bauer DJ (2007) Building path diagrams for multilevel models. Psychological Methods 12(3) 283-297. (Copy free to download and print out for CBSU users)
Dunn G, Everitt B. and Pickles A. (2002) Modelling covariances and latent variables using EQS. Chapman and Hall/CRC Press:London. pp.124-131.
Galbraith S, Daniel JA and Vissel B (2010) A Study of Clustered Data and Approaches to Its Analysis. Journal of Neuroscience 30(32) 10601-10608.
Gelman A and Hill J (2007) Data analysis using regression and multilevel/hierarchical models. Cambridge University Press. (R and BUGS code used in case studies).
Heck RH, Thomas SL and Tabata LN (2010) Multilevel and Longitudinal Modeling with IBM SPSS (Quantitative Methodology Series) (Paperback) Routledge Academic:New York. Further details about this book are given here. Type 'Heck' in the search window (top right). There is also now a second edition (2014).
Heck RH, Thomas SL and Tabata LN (2012) Multilevel Modeling of Categorical Outcomes using IBM SPSS (Quantitative Methodology Series) (Paperback) Routledge Academic:New York.
Hedeker D and Gibbons RD (1997). Application of Random-effects Pattern-Mixture Models for Missing Data in Longitudinal Studies. Psychologcal Methods 2(1) 64-78. On-line pdf file of this paper is is here.
Krystal GR (2004) Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Archives of General Psychiatry 61(3) 310-317. A copy of the abstract is given here.
Laird N and Hirschland E (2021) From the Apollo programme to the EM algorithm and beyond. Significance 18(4) 34-37.
Liu S, Rovine MJ and Molenaar PCM (2012) Selecting a linear mixed model for longitudinal data: repeated measures analysis of variance, covariance pattern model, and growth curve approaches. Psychological Methods 17(1) 15-30. (A free copy is available on PSYCNET for CBSU users).
Luke DA (2004) Multilevel Modeling Sage:London. (This is a good primer).
McNeish, DM and Stapleton LM (2016) The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration. Educational Psychology Review 28(2), 295–314. (mentions that one needs an adequately large number of clusters otherwise some model parameters will be estimated with bias).
Peugh JL and Enders CK (2005) Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models. Educational and Psychological Measurement 65 714-741.
Wright DB and London K (2009) Multilevel modelling: beyond the basic applications. British Journal of Mathematical and Statistical Psychology 62 439-456. Tutorial featuring examples of using R syntax to fit models. A PDF copy of this article is available from here.