Analysing a nominal and ordinal variable
Part 3b: Post-hoc test (Dunn's test)
On the previous page, we saw there was a significant influence from the nominal variable on the ordinal variable. If the nominal variable consist out of more than two categories we need to further test to see which categories are then significantly different from each other. This is called a post-hoc test and for the Kruskal-Wallis H test what can be done is perform a so-called Mann-Whitney U test (Mann & Whitney, 1947) for each possible pair. In the example this means a Mann-Whitney U test for Diemen vs. Haarlem, Diemen vs. Rotterdam, and Haarlem vs. Rotterdam. A specific test that does this similarly is known as a Dunn's test* (Dunn, 1967).
The ‘Adj. Sig.’ is short for ‘adjusted significance’. Since we are doing multiple tests, we have a danger of making a wrong decision of 5% each time. Although this seems low, it can quickly compound to making at least one wrong decision. Therefor the regular significance for each pairwise test gets adjusted. There are various methods to do this adjustment, but SPSS uses in this case the Bonferroni adjustment.
The .000 at Rotterdam-Diemen indicates that the chance of having a sample with a test statistic of 21.467 or even higher, if there would be no difference in the population, is almost 0. This is below the usual 0.050, so we can say that indeed the opinion of the students on the teacher motivation, in Rotterdam is significantly different from the students in Diemen. The same goes for Haarlem vs Diemen, but Rotterdam vs Haarlem is not significantly different.
Almost there, we know that Diemen scored significantly different from Haarlem and Rotterdam, but did it score better or lower? From the bar-chart we can probably tell, but more formally we can also see this from the mean ranks. For Diemen it is 41.53, for Haarlem 21.81 and for Rotterdam 20.06. These numbers are the so-called average ranks (or mean rank). The number in itself is not very important, but if it is higher or lower. In this example we see that Diemen scores the highest mean rank.
A big WARNING though. A high mean rank is not necessarily indicating ‘better’. It depends on how the ordinal variable is coded. If 1 = fully disagree to 5 = fully agree, then yes, a high mean rank indicates students tended more towards (fully) agree, but if it is coded as 1 = fully agree to 5 = fully disagree then a high mean rank indicates students tended more towards (fully) disagree.
In this example it was 1 = fully disagree to 5 = fully agree, so the higher mean rank for Diemen indicated that students agreed more to the statement that the Teacher was able to motivate them.
We can add this to our report:
A Kruskal-Wallis test showed that Location had a significant effect on how motivated students were by the teacher, χ2(2, N = 54) = 21.33, p < .001. A post-hoc test using Dunn's test with Bonferroni correction showed the significant differences between Diemen and Haarlem, p < .05, and between Diemen and Rotterdam, p < .001.
Click here to see how you can perform a pairwise post-hoc Dunn's test, with SPSS, R, Excel, or Python
The video below shows how to perform the Kruskal-Wallis H test with SPSS, and also already the post-hoc test
click here if the pairwise comparison will not show
with SPSS 22 or 23 you might get an error for the pairwise comparison, this could be resolved following the steps in the video below
with R (Studio)
The last step in our analysis is to also indicate how strong the effect is, so we should add a so called effect size. This will be the topic on the next page.
*Alternatives for the Dunn test include: Conover-Iman test and the Dwass-Steel-Critchlow-Fligner test.
Nominal vs. Ordinal