How do I compute Akaike's and Bayesian information criteria (AIC, BIC) to compare regression models and how do I interpret them?
Akaike's information criterion is used to compare both the efficiency of multivariate models looking at the same data combining the degree of fit with the number of terms in the model. Better fitting simpler models are preferred with smaller AICs. AIC can be used as an alternative to the F ratio in stepwise regressions investigating the effectiveness of adding or subtracting one or more predictors from a model (see an example in the Regression Grad talk). Information criteria can also be used to compare logistic regression models with overdispersion (Agresti, 1996).
AIC = n ln(RSS/n) + 2 df(model)
where RSS is the Residual Sum of Squares which is routinely outputted from the regression analysis, n is the total sample size and df(model) is the degrees of freedom of the regression model which is the number of parameters equal to the number of predictors + 1 (for the intercept). The above formula for AIC is also given on page 63 of Burnham and Anderson (2002).
There is also a Bayesian Information Criterion (BIC) or Schwarz's criterion
BIC = n ln(RSS/n) + [(k+1) ln(n)]/n
where n is the total sample size and there are k parameters (including the intercept). The BIC may be used as a form of Bayes Factor (see for example here) and also here comparing logistic regression models.
Nagin (1999) suggests using bij=exp(BIC(1)BIC(2)) as a means of deciding on whether one BIC is meaningfully lower than another BIC (page 147 and Table 2 on page 148 gives some rules of thumb). It is also mentioned in Chapter Four of Nagin(2005).
bij 
Interpretation 

bij < 1/10 
Strong evidence for model j 

1/10 < bij < 1/3 
Moderate evidence for model j 

1/3 < bij < 1 
Weak evidence for model j 

1 < bij < 3 
Weak evidence for model 

3 <bij< 10 
Moderate evidence for model 

bij > 10 
Strong evidence for model 
From here a raw difference between a pair of BICs of more than 10 is regarded as a difference in model fit for BICs obtained in structural equation models (see Raftery (1995)).
Jones, Nagin and Roeder (2001) alternatively suggest using twice the raw difference in BICs to compare models.
2(Diff in BICs) 
Interpretation 

0 to 2 
Not worth mentioning 

2 to 6 
Positive 

6 to 10 
Strong 

> 10 
Very Strong 
On a related note Shafer and, also, Jeffreys (1961) give rules of thumb for sizes of Bayes Factors (which compare an alternative model to a null model) suggesting Bayes Factors under 3 are weak (Shafer) and anecdotal (Jeffreys).
Some rules of thumb for using Bayes factors (Jeffreys 1961)
1 < Bayes factor <= 3 
weak evidence for M1 

3 < Bayes factor <= 10 
substantial evidence for M1 

10 < Bayes factor <= 100 
strong evidence for M1 

100 < Bayes factor 
decisive evidence for M1 
Kass and Raftery (1995) came up with slightly different rules of thumb to evaluate sizes of Bayes Factors: 1 to 3 (Not worth more than a bare mention), 3 to 20 (Positive), 20 to 150 (Strong) and > 150 (Very Strong) and use R code to compare the equality of group variances. There are also some rules of thumb for Bayes Factors on page 10 of the presentation here.
You can combine Bayes Factors, for example, if we have three conditions A, B and C and we wish to test if the mean of B is nearer to the mean of A or to the mean of C we could perform two one sample ttests on the differences BA and CB, obtain Bayes Factors, BFs, for these two tests and take the ratio BF(BA)/BF(CB) with a high value > 1 suggesting the mean of B is closer to the mean of C and a value < 1 suggesting the mean of B is closer to the mean of A.
Free Bayesian analysis software (JASP) is available from here which acts as a front end to the R BayesFactor procedure.
 Jeon M and De Boeck P (2017) compare translational approaches finding that a pvalue of 0.01 is roughly equivalent to a Bayes Factor of 3 and refute earlier work linking this Bayes Factor to a pvalue of 0.05 (see below and Dienes Z. (2014)).
Simulations with R code for a Bayesian power analysis with details here if the link is broken. A ttest Bayesian power simulation is here reproduced here if the link is broken.
A web calculator converting a Bayes Factor into a Cohen's d is here. Note however that there are no established procedures for computing power for the Bayes factor analysis. Indeed, Dienes (2014) has argued that Bayes factors obviate the need to perform power analyses.
A note about choice of priors for a one sample binomial test
There is also a pdf guide to computing and interpreting Bayes Factors from JASP (software) in factorial ANOVAs here. In particular pages 2831 of this guide show how to compare pairs of Bayes Factors representing different models fitted to the same data e.g. with and without a main effect to assess the importance of the extra terms (e.g. the main effect) in the fuller model using the Bayes Factors. Our experiences fitting these models suggests that there is close agreement in inference between pairwise comparisons of these Bayes Factors and classical maximum likelihood approaches such as the F test. In other words if group means are found to differ using the F test they will also be seen to do when comparing pairs of Bayes Factors and viceversa and similarly if there is no evidence of differences in group means using the F test there will be a similar lack of evidence of group means differences comparing the Bayes Factors.
The journal issue of Psychological Methods 2017, Volume 22, No. 2 is entirely devoted to describing and illustrating applications of Bayesian methods including the evaluation of Bayes factors in hierarchical analysis of variance using the BayesFactor procedure in R, comparing Bayes Factors equal to 3 with pvalues (Jeon M and De Boeck P (2017) find these seem to approximately correspond to a pvalue of 0.01) and the comparison of model selection criteria in Factor Analysis with the BIC performing well. Jeon M and De Boeck P (2017) further suggest the traditional view that a Bayes Factor of 3 can correspond to a pvalue of 0.05 ( as suggested, for example, here or or here if the link is broken) may be incorrect but that using a Bayes Factor of 3 may still be a good cutoff to use for suggesting a rejection of the null hypothesis.
An online web calculator (Rouder et al., 2009) which also (like JASP above) uses the BayesFactor procedure in R converts t values to Bayes Factors. This calculator is available to use here. There is also a companion calculator here for obtaining Bayes Factors from regression coefficients (Liang et al., 2008).
Note that there is also a second order AIC (Sugiura 1978, Hurvich and Tsai 1991) which is recommended for small sample sizes.
References
Agresti A (1996) An introduction to categorical data analysis. Wiley:New York.
(A pdf copy of the above book may also be downloaded for free from here.)
Hurvich CM and Tsai CL (1991) Bias of the corrected AIC criterion for underfitted regression and time series models. Biometrika 78 499–509.
Jeffreys H (1961) Theory of Probability, 3rd ed. Oxford Classic Texts in the Physical Sciences. Oxford Univ. Press: Oxford.
Jeon M and De Boeck P (2017) Decision qualities of Bayes factor and p valuebased hypothesis testing. Psychological Methods 22(2) 340360.
Jones B, Nagin D & Roeder KA (2001) SAS Procedure Based on Mixture Models for Estimating Developmental Trajectories. Sociological Methods & Research 29 374393.
Kass RE & Raftery AE (1995) Bayes factors. Journal of the American Statistical Association 90 773795.
Liang F, Paulo R, Molina G, Clyde MA and Berger JO (2008), Mixtures of g Priors for Bayesian Variable Selection. Journal of the American Statistical Association 103, 410423.
Nagin DS (1999) Analyzing Developmental Trajectories: A Semiparametric, GroupBased Approach. Psychological Methods 4(2) 139157.
Nagin DS (2005) Groupbased Modeling of Development. Harvard University Press: Massachusetts.
Raftery AE (1995). Bayesian Model Selection in Social Research. Sociological Methodology, 25, 111163.
Rouder JN, Speckman PL, Sun D, Morey RD, & Iverson G (2009) Bayesian tTests for Accepting and Rejecting the Null Hypothesis. Journal Psychonomic Bulletin & Review 16 225237.
Sugiura N (1978) Further analysis of the data by Akaike’s information criterion and the finite corrections. Communications in Statistics: Theory and Methods A7 13–2