FAQ/os2 - CBU statistics Wiki

Revision 69 as of 2013-08-13 14:26:02

Clear message
location: FAQ / os2

Partial omega-squared as an effect size in analysis of variance

Partial $$\omega2 $$ has been suggested by Field (2013, p.473-4), Keppel (1991,pp 222-224) and Olejnik and Algina (2003) as an unbiased alternative to partial $$\eta^text{2}$$ when comparing the size of sources of variation across studies from analysis of variance where MSE is the mean square of the error term.

(Partial) $$\omega2 = $$ [SS(effect)-df(effect) MSE(effect)] / [SS(effect)+(N-df(effect))MSE]

In the special case of a between subjects ANOVA with b groups and a total of N subjects the denominator in the above may be rewritten as below.

SS(effect)+(N-df(effect))MSE = SS(group) + (N-df(group))MSE

= SS(group) + (N-(b-1))MSE = SS(group) + (N-b+1)MSE = SS(group) + (N-b)MSE + MSE

= Total SS + MSE = SS(group) + SSE + MSE = SS(group) +((df(error term)+1) x MSE)

since in a one-way ANOVA between subjects design the total SS = SS(group) + (N-b)MSE. The latter denominator term (SS(group) +((df(error term)+1) x MSE)) will also be correct for any main effect in a factorial ANOVA since the df(effect) takes the same form as above ie equal to the number of groups - 1 provided the remaining factors are non-random ie chosen by the experimenter. Baguley (2012, p.483) and Field (2013, p.473) give the above formula for $$\omega2 $$ with Total SS + MSE in the denominator.

Olejnik and Algina further present and illustrate an alternative to the above for ANOVAs (both between subjects only or including a single repeated measures factor) for effects involving upto three factors using a more general formula for partial $$\omega2 $$ to take into account any non-manipulated factors in the ANOVA.

(Partial) $$\omega2 $$ = [ SS(Effect) - df(effect)MSE ] / [d(SS(Effect) - df(effect)MSE) + \sum_M_ [SS(M) - df(M)(MSE of M)] + N MSE ]

where d=1 if the effect of interest is non-random and 0 otherwise, M are any sources of variation which include at least one random factor and N is the total number of observations (scores).

$$\omega2 $$ may take values between $$\pm$$ 1 with a value of zero indicating no effect. A negative value will result if the observed F is less than one.

The numerator in $$\omega2 $$ compares the mean square of the effect (=SS(effect)/df(effect)) to its mean square error which theoretically should be equal (giving a difference, as measured in the numerator, of zero) because the ratio of the mean squares may be approximated by a F distribution which takes a minimum value of one indicating no statistical evidence of the effect.

Field (2013, p.537-8) illustrates computing $$\omega2 $$ for between subjects ANOVAs with more than one factor which is somewhat more involved and uses variance components.

He also (pages 566-567) mentions that a different form of $$\omega2 $$ to that discussed above is needed for a factor in a repeated measures analysis of variance since the estimate used for a between subjects factor overestimates effect size if used in a repeated measures.

In particular for a factor in a one-way repeated measures ANOVA with k levels and n subjects

$$ \omega2 = $$ [ [(k-1)/(nk)] (MS(effect)-MSE)] / [MSE + (MSB - MSE)/k + [(k-1)/(nk)] (MS(effect)-MSE) ]

where MS(B) = (total variance(N-1)-SS(effect) - SSE)/(N-1) where SSE is the sums of square error for the repeated measures factor (ie the error sum of squares for the effect). Field also suggests only using effect sizes based upon pairwise group comparisons for repeated measures with more than one factor (including for mixed between-within factor ANOVAs) using a simpler effect size equal to sqrt[F(1,dfe)/(F(1,dfe)+dfe)] which is interpreted as a correlation.

$$\omega2 $$ is unaffected by small sample sizes unlike $$\eta2 $$ which tends to overestimate the effect in smaller samples (Larson-Hall (2010), p.120; Field (2013), p.473) and, since it is a population based measure unlike $$\eta2$$ , will consequently take a smaller value.

References

Baguley T (2012). Serious Stats. A guide to advanced statistics for the behavioral sciences. Palgrave Macmillan:New York.

Field AP (2005, p.417-419). Discovering statistics using SPSS. Sage:London. Formulae using ANOVA output to compute omega-squared for factorial designs.

Field A (2013) Discovering statistics using IBM SPSS statistics. Fourth Edition. Sage:London. On page 474 it is suggested using values for $$\omega^text{2}$$ of 0.01, 0.06, 0.14 to indicate small, medium and large effects respectively.

Olejnik S and Algina J (2003). Generalized Eta and Omega Squared Statistics: Measures of effect size for some common research designs. Psychological Methods 8(4) 434-447. A pdf of this is available for free to CBUers using PsychNet and also here.

Keppel G (1991). Design and analysis:A researcher's handbook. Prentice-Hall:Englewood Cliffs, NJ

Larson-Hall J (2010). A Guide to Doing Statistics in Second Language Research Using SPSS. Routledge:New York.