Why an overall sum of squares does not tell the whole story

When there are 3 or more groups the overall F ratio from the ANOVA can miss specific relationships. For example it can miss trends between the means. The sums of squares term comparing k groups in an ANOVA represents the sum of k-1 orthogonal contrasts between the group means. It is possible for one of these contrasts to be present and for the overall F ratio to be non-significant especially when k is large. Note that when k=2 the test for the difference in group means is equivalent to a test of linear trend. In general the relationships between k group means can be exactly explained by contrasts representing trends upto order k-1. So three group means can be exactly explained by the sum of linear and quadratic contrasts and four means by the sum of linear, quadratic and cubic contrasts.

For example suppose we wish to compare three group means. The linear contrast between the group means is represented by the coefficients -1,0,1 and the quadratic contrast by the coefficients 1,-2,1. If we perform a linear regression on the response with an intercept and each person's linear and quadratic contrast coefficient based on which group they are in and add together the sums of squares for the two contrasts we will obtain the sum of squares from the usual ANOVA comparing the difference between the three group means. The reason for this additivity stems from the orthogonality of the contrasts.

So knowing the total sum of squares for the k-1 contrasts does not tell us anything about its component k-1 contrast sums of squares when k>2. It follows that, for example, a linear trend could be statistically significant even if the F ratio based on k-1 contrasts is non-significant and vice-versa. A SPSS example data set of the former is given [attachment:trendeg.sav here.]

See the post-hoc grad talk for further examples of fitting orthogonal contrasts including a test of a trend.