Comparing pairs of correlations

A variable in common (overlap) e.g. of form r(W,X) = r(W,Z).

A test for this comparison goes under various names the Williams test, Williams-Hotelling or Hotelling test.

Some SPSS syntax is available (from Bruce Weaver). Just cut and paste into a SPSS syntax window to use. In the syntax window choose select all and click the run arrow. Edit the data in the spreadsheet as required. You can also use the Williams-Hotelling test by typing equalcor at a UNIX prompt on a CBU machine.

The R procedure cocor performs a large number of tests to compare pairs of different correlations including tests described in this article.

More SPSS syntax

• Comparing two correlations from two independent samples

• Comparing two correlations from the same sample with no variables in common eg of form r(A,B)=r(X,Y)

• A z-test alternative to Williams-Hotelling for correlation comparisons of form r(W,X)=r(W,Z)

• Comparing 2 or more correlation coefficients (in attached Alan Pickering notes)

Some of the above tests may be performed using the free downloadable program FZT.EXE but note FZT.EXE does not compute p-values. Williams test may also be performed using this spreadsheet.

SPSS (and SAS) programs by Bruce Weaver and Karl Wuensch for comparing correlations on the same participants with or without a variable in common as well as two independent correlations are available from here.

Steiger (1980) recommends a z-test in preference to Williams test which can be computed using this spreadsheet. The spreadsheet also can be used to test if the differences of r(W,X) - r(W,Z) observed in two groups differ ie if (r1(W,X) - r1(W,Z)) - (r2(W,X) - r2(W,Z)) = 0. See also Howell (2002).

Since the two groups are independent a z-test to test whether the differences in correlations is the same in the two groups may be constructed of form

[ (r1'(W,X) - r1'(W,Z)) - (r2'(W,X) - r2'(W,Z)) ] / [Sqrt{(s.e. r1 difference)² + (s.e. r2 difference)² )} ]

where r1' and r2' are the Fisher transformed correlations in groups 1 and 2 respectively with standard errors as given in Steiger (1980).

Also note Algina J and Keselman HJ (1999) give formulae for comparing squared multiple correlations from regressions on different data. Baguley (2012) has R code available for computing differences in pairs of correlations and confidence intervals for these differences using Zou(2007) (see here for further details).

If one is correlating time series data it may be worth removing serial dependencies prior to correlation by differencing (see here and here).

Shrout and Yip-Bannicq (2017) quote the paired t-test formula from Cohen, Cohen, West and Aiken (2003, p.641) to compare regression coefficients to assess the difference between a pair of correlations and also quote, and use a z-test, comparing Fisher z-transformed correlations to do the same thing which gives a similar result.

References

Algina, J. and Keselman, H.J. (1999) Comparing squared multiple correlation coefficients: examination of a confidence interval and a test of significance. Psychological methods 4(1) 76-83.

Baguley, T. (2012) Serious stats: A guide to advanced statistics for the behavioral sciences. Basingstoke: Palgrave.

Cohen, J., Cohen P., West, S.G. and Aiken, L.S. (2003) Applied multiple regression/correlation analysis for the behavioral sciences. Routledge:New York.

Howell, D.C. (2002) Statistical Methods for Psychology. Fifth Edition. Duxbury Press:Belmont,CA.

Shrout, P.E. and Yip-Bannicq M. (2017) Inferences about competing measures based on patterns of binary significance tests are questionable. Psychological Methods 22(1) 84-93. Gives formulae for comparing associations between a pair of variables using Fisher z-transformed correlations and also doing the same test using a t-test comparing regression estimates.

Steiger JH (1980) Tests for comparing elements of a correlation matrix. Psychological Bulletin 87(2) 245-251. (Available free via ScienceDirect to CBSU users).

Zou GY (2007) Toward using confidence intervals to compare correlations. Psychological Methods 12: 399-413.

[COPY AND PASTE SPSS SYNTAX IN THE BOX BELOW TO PERFORM WILLIAMS TEST]

* Dependent Correlation Comparison Program. 
* Compares correlation coefficients from the same sample. 
* See Cohen & Cohen (1983), p. 57. 
* Sam Field, sfield@mail.la.utexas.edu, March 1, 2000. 

* Three pairs of correlations to compare*****

set format f10.5.
DATA LIST free
 /r12 r13  r23  nsize.

BEGIN DATA
.50 .32 .65 50
.59 .31 .71 30
.80 .72 .89 26
END DATA.

define williams (rxy = !tokens(1)
                      /rvy = !tokens(1)
                      /rxv = !tokens(1)
                      /n = !tokens(1)).

COMPUTE #diffr = !rxy - !rvy.
COMPUTE #detR = (1 - !rxy **2 - !rvy**2 - !rxv**2)+ (2*!rxy*!rxv*!rvy).

COMPUTE #rbar = (!rxy + !rvy)/2.

COMPUTE #tnum = (#diffr) * (sqrt((!n-1)*(1 + !rxv))).

COMPUTE #tden = sqrt(2*((!n-1)/(!n-3))*#detR + ((#rbar**2) * ((1-!rxv)**3))).

COMPUTE t= (#tnum/#tden).

COMPUTE df = !n - 3.

COMPUTE p_1_tail = 1 - CDF.T(abs(t),df).

COMPUTE p_2_tail = (1 - CDF.T(abs(t),df))*2.

LIST t  df    p_1_tail    p_2_tail.

exe.

!enddefine.

*********************

williams rxy=r12 rvy=r13 rxv=r23 n=nsize.

• Multiple invocations of the macro in the same interactive session will produce the following note:

>The macro name specified on the DEFINE command duplicates the name of a previously defined macro. This instance will take precedence.

This warning does not indicate a problem and may be ignored.