Analysing a binary vs. ordinal variable
2b: Effect size (Rosenthal correlation)
To determine how big the difference is between two groups we can report a so-called effect size measure. There are a few methods to do this with a Mann-Whitney U test. The Common Language Effect Size (or variations on it), the (Glass) Rank Biserial Correlation, and the Rosenthal correlation. The Rosenthal correlation is mentioned as the effect size to report by some authors (Fritz, Morris, & Richler, 2012; Tomczak & Tomczak, 2014), so will also be the one I’ll use. See the end notes at the bottom of the page for more information about the others.
The Rosenthal correlation is a generic one that simply divides the standardized test statistics, by the square root of the sample size (Rosenthal, 1991, p. 19), which I will refer to as the Rosenthal correlation coefficient (as to differentiate it with other correlation coefficients). In the example this would be r = -2.84534 / v(45) = -0.42.
As for the interpretation various rules of thumb exist. One such rule of thumb is from Bartz (1999, p. 184) shown in Table 1.
| Rosenthal Correlation | Interpretation |
|---|---|
0.00 < 0.20 |
very low |
0.20 < 0.40 |
low |
0.40 < 0.60 |
moderate |
0.60 < 0.80 |
strong |
0.80 < 1.00 |
very strong |
The -0.42 from the example would then indicate a moderate effect size. We can add this now to the other test results from the previous section:
An exact Mann-Whitney U test indicated that the mean ranks for male and female were moderately but significantly different, U(n1 = 11, n2 = 34) = 285.5, p = .008, rRosenthal = .42.
Click here to see how you can obtain this effect size...
with Python
with an Online calculator
Enter the standardized test statistic, and the sample size:
Manually (Formula and Example)
Formula
The formula to determine the Rosenthal Correlation Coefficient is:
\(r_{Rosenthal}=\frac{Z}{\sqrt{n}}\)
In this formula Z is the Z-statistic, which in the case of a Wilcoxon one-sample test is the adjusted W statistic, and n the sample size.
Example
Note: different example then the one used in the rest of this section.
We are given the scores of one group of people:
\(X_1=(1,2,5,2,2)\)
And another group:
\(X_2=(4,3,5,5)\)
Note the total number of scores is nine, so:
\(n_1=5, n_2=4, N=5+4=9\)
In the test section we already calculated the Z value for this example. That was:
\(Z=\frac{3\sqrt{35}}{10}\approx1.775\)
Now we can determine the Rosenthal correlation:
\(r_{Rosenthal}=\frac{Z}{\sqrt{n}} =\frac{\frac{3\sqrt{35}}{10}}{\sqrt{9}} =\frac{\frac{3\sqrt{35}}{10}}{3} =\frac{3\sqrt{35}}{3\times10} ;=\frac{\sqrt{35}}{10}\approx0.5916\)
The last step is to combine all our previous report bits into one. This will be done in the next section.
Appendix: Other effect size measures
Common language effect size (Vargha-Delaney A) (click for more info)
The common language effect size (McGraw & Wong, 1992) was further developed by Vargha and Delaney (2000) who called this the measure of stochastic superiority. It gives the probability that a score taken at random from one distribution will be greater than a score taken from another. This can quickly be calculated taking the U statistic and divide it by the product of the sample sizes of each group. In the example the U statistic is 88.5 and the two sample sizes 11 and 34. The common language effect size is therefore: 88.5 / (11 x 34) = 0.24.
Alternatively you can also take the sum of the ranks from one group divided by the sample size of that group, then subtract from this the sample size plus one, divided by two. Then divide all this by the sample size of the other group. In the example this gives: (154.5 / 11 – (11 + 1) / 2) / 34 = 0.24.
(Glass) Rank-biserial correlation / Cliff's Delta (click for more info)
The rank-biserial correlation (Cureton, 1956) shows the percentage of the difference of pairs that are favourable to the hypothesis and the percentage that is unfavourable to the hypothesis. It can either be calculated directly by multiplying the difference in mean ranks twice, and divide it by the total sample size. In the example this gives: 2 x (14.04 – 25.90) / 45 = -0.53.
You can also calculate it from the common language effect size. Simply multiply the common language effect size by two, and then subtract one. In the example 2 x 0.24 – 1 = -0.52 (differences due to rounding).
The measure is often attributed to Glass and although different in calculation Cliff's delta will produce the same result.
Note SPSS mentions this is the same as Spearman’s Rho (see here), but that is not the case as mentioned here.
Besides these, the effect sizes used for more than two categories and an ordinal variable can also be used. See Unpaired -> Nominal vs. Ordinal -> Effect size.
Google adds