How do I compute Akaike's information criterion (AIC) to compare regression models?
Akaike's information criterion is used to compare both the efficiency of multivariate models looking at the same data combining the degree of fit with the number of terms in the model. Better fitting simpler models are preferred with smaller AICs. AIC can be used as an alternative to the F ratio in stepwise regressions investigating the effectiveness of adding or subtracting one or more predictors from a model (see an example in the Regression Grad talk).
AIC = n ln(RSS/n) + 2 df(model)
where RSS is the Residual Sum of Squares which is routinely outputted from the regression analysis, n is the total sample size and df(model) is the degrees of freedom of the regression model which is the number of parameters equal to the number of predictors + 1 (for the intercept). The above formula for AIC is also given on page 63 of Burnham and Anderson (2002).
There is also a Bayesian Information Criterion (BIC) or Schwarz's criterion
BIC = n ln(RSS/n) + [(k+1) ln(n)]/n
where n is the total sample size and there are k parameters (including the intercept).
Nagin (1999) suggests using bij=exp(BIC(1)-BIC(2)) as a means of deciding on whether one BIC is meaningfully lower than another BIC (page 147 and Table 2 on page 148 gives some rules of thumb). It is also mentioned in Chapter Four of Nagin(2005).
bij |
Interpretation |
|
bij < 1/10 |
Strong evidence for model j |
|
1/10 < bij < 1/3 |
Moderate evidence for model j |
|
1/3 < bij < 1 |
Weak evidence for model j |
|
1 < bij < 3 |
Weak evidence for model |
|
3 <bij< 10 |
Moderate evidence for model |
|
bij > 10 |
Strong evidence for model |
|
References
(A pdf copy of the above book may also be downloaded for free from here.)
Nagin DS (1999) Analyzing Developmental Trajectories: A Semiparametric, Group-Based Approach. Psychological Methods 4(2) 139-157.
Nagin DS (2005) Group-based Modeling of Development. Harvard University Press: Massachusetts.