Scale vs Scale paired
2b: Effect size
From the example on the previous pages we got the impression that a commercial had an effect on the opinion of a brand, since overall the difference was positive between the before and after score.
The last part would be to determine how big the difference can be classified as. For this we can make use of an effect size, but there are many options and different authors suggest different ones. Cohen's is probably the most frequently used one, but there are several variations on Cohen's d. I'll use the one that is based on the differences (sometimes denoted as dz), but with a correction proposed by Hedges for small sample sizes.
In the example from that we started with dz* = .48. Unfortunately there is no formal way to determine if 0.48 (you can ignore the negative sign) is high or low (although almost everyone would agree this is high), and the rules of thumb floating around on the internet vary quite a lot, often depending on the field (e.g. biology, medicine, business, etc.). I would the following rule of thumb (combining Cohen (1988) and Sawilowsky (2009)):
0.00 < 0.20 - Very weak (Sawilowsky)
0.20 < 0.50 - Weak (Cohen)
0.50 < 0.80 - Moderate (Cohen)
0.80 < 1.20 - Strong (Cohen)
1.20 < 2.00 - Very strong (Sawilowsky)
2 or more - extremely strong (Sawilowsky)
These rule of thumb are however for Cohen's d, not dz. Luckily though, you can simply multiply a Cohen dz with the square root of 2, to convert it to a Cohen d (Cohen, 1988, p. 48). In this example we could therefor speak of a medium effect. The results could be reported as:
A paired samples t-test showed that there was a significant but weak difference in scores on opinion about the brand before and after seeing the commercial, t(23) = 2.40, p = .025, Cohen's d (based on differences) with Hedges correction = 0.48.
Click here to see how to determine Cohen's d, or Hedges g, with SPSS, Excel (using SPSS output), R (studio), Python, or Manually
New since SPSS 27 a direct method. For older versions of SPSS, see the 'using (SPSS output and) Excel'
using (SPSS output and) Excel
with R (Studio)
Manually (formulas and example)
See the manually part in the test section for the formulas for
Note: continuation of the example in the manually part of the test section.
We are given the following pairs:
Since the second pair is missing a score, it will not be used and removed, so:
There are five pairs remaining, so n = 5.
In the test section we already calculated the following:
Cohen's d is therefor:
If we had used the alternative formula, we would have gotten the same result:
For the Hedges correction we can use the approximation. That would yield:
Which gives an approximation for Hedges g of:
We can now complete the report by combining all the results, which will be done on the next page.
Scale vs Scale paired