# Analysing a single nominal variable

## Part 3c: Effect size (Cramér's V and Relative Risks)

In the previous part we saw that all percentages in the example were different from each other. However if a sample size is large enough any minor difference will still result in a significant result. For this it is recommended to also add a so-called **effect size**.

We can determine an effect size for the overall test (the Pearson chi-square), and also for the pairwise comparisons (the binomial tests).

### Effect size for the overall test

A possible effect size for the very first test we did (the omnibus test known as Pearson Chi-square test) is Cramér's V (Cramér, 1946). This measure is actually designed for the chi-square test for independence but can be adjusted for the goodness-of-fit test (Kelley & Preacher, 2012, p. 145; Mangiafico, 2016, p. 474). It gives an estimate of how well the data then fits the expected values, where 0 would indicate that they are exactly equal. If you use the equal distributed expected values (as we did in the example) the maximum value would be 1, otherwise it could actually also exceed 1.

As for the interpretation for Cramér's V various rules of thumb exist. In table 1 the one from Rea & Parker (1992) is shown:

Cramér's V | Interpretation |
---|---|

0.00 < 0.10 | Negligible |

0.10 < 0.20 | Weak |

0.20 < 0.40 | Moderate |

0.40 < 0.60 | Relatively strong |

0.60 < 0.80 | Strong |

0.80 < 1.00 | Very strong |

In the example Cramér's V is 0.401 (see videos below on how to determine this) which would indicate a relatively strong effect. Another table is from Cohen (1988) who uses: 0 < .10 => negligible; .10 < .30 = > small; .30 < .50 => medium; .50+ => large.

We could add this to our report:

A chi-square test of goodness-of-fit was performed to determine whether the marital status were equally chosen. The marital status was not equally distributed in the population, *χ*^{2}(4, *N* = 1941) = 1249.13, *p* < .001, with a relatively strong (Cramér's *V* = .40) effect size according to conventions for Cramér's V (Rea & Parker, 1992).

**Click here to see how to obtain Cramér's V, with SPSS, R (studio), Excel, Python, an Online calculator, or Manually.**

**with SPSS**

Unfortunately SPSS does not have a method to determine Cramér's V directly from the GUI, however the calculation is not very difficult once you have the output from the previous part.

The video below shows how this could be done with a bit of help from Excel

**with R (Studio)**

Download R script from video here.

**with Excel**

Download Excel file from video here.

**with Python**

Download Jupyter Notebook from video here.

**Online calculator**

Enter the requested information below:

**Manually (formula and example)**

**Formula**

The formula for Cramér's V is:

In the above formula *χ ^{2}* is the chi-square test value,

*n*is the total sample size, and

*df*is the degrees of freedom, determined by

*df*=

*k*- 1.

*k*is the number of categories.

**Example**.

If we have a chi-square value of 1249.13, a total sample size of 1941, and had five categories, we can first determine the degrees of freedom (df):

*df* = *k* - 1 = 5 - 1 = 4

Then we can fill out all values in the formula for Cramér's V:

Alternatives for Cramér's V as an effect size measure can also be **Cohen's w**, or **Johnston-Berry-Mielke E**.

A correction can be applied using the procedure proposed by Bergsma (2013)). This is actually for a Cramér’s V with a chi-square test of independence, but adapting it for a goodness-of-fit is possible.

### Effect size for the pairwise tests

For the pairwise comparisons we did with the binomial test an often reported effect size is the Relative Risk (JonB, 2015), although **Cohen's h** could also be used (Cohen, 1988). Relative Risk tells how many times more likely than was expected a category was chosen compared to the other.

In the example the **Relative Risks** ranged from 1.11 (divorced vs. never married) to 1.85 (married vs. seperated). Which we can also add to our report:

The binomial test pairwise comparison with Bonferroni correction of marital status showed that all proportions were significantly different from each other (*p* < .003). The Relative Risks ranged from 1.11 (divorced vs. never married) to 1.85 (married vs. seperated).

**Click here to see how to obtain Relative Risks, with R, or Excel (not possible with SPSS).**

**with Excel**

**with R**

**not possible with SPSS**

Unfortunately the GUI of SPSS does not have a way to determine Relative Risks from a binomial test.

Now we have fully analyzed our nominal variable. In the next part we combine all the reports section to create a final example of how you could report all the findings.

## Click here if you want to see the formula's for Cramér's V, Cohen's w, Johnston-Berry-Mielke E, and Cohen's h.

**Cramér's V**:

In the above formula *χ ^{2}* is the chi-square test value,

*n*is the total sample size, and

*df*is the degrees of freedom, determined by

*df*=

*k*- 1.

*k*is the number of categories.

**Cohen's w**:

In the above formula *χ ^{2}* is the chi-square test value,

*n*is the total sample size, and

*k*is the number of categories.

Given Cramer's V (*V*) and the number of categories (*k*), the formula for Cohen's w will be:

**Johnston-Berry-Mielke E** (for Pearson chi-square, and for likelihood ratio):

(from Johnston, Berry, and Mielke (2006)).

In the above formula's *p _{i} *is the observed proportion in category

*i*,

*q*is the expected proportion in category

_{i }*i*,

*q*the minimum of all

*q*'s, and

_{i}*k*the number of categories.

**Cohen's h**

**Single nominal variable**

Google adds