Analysing two scale variables

Part 3: Test and effect size

On the previous page we got a first impression from the sample data, and noticed there might be a relation between the current and the beginning salary. To formally test if the two variables have an association, we can perform what is known as a regression analysis, and since we keep things simple we will limit ourselves to predicting based on a straight line, so linear regression, and since we only have two variables the full name would be bivariate linear regression. The most commonly used measure to test if this relation also exists in the population, is the Pearson Correlation Coefficient (or in it's full name the Pearson product-moment correlation coefficient) (Pearson, 1896).

Pearson Correlation varies between -1 and +1. If it is -1 there is a perfect negative lineair relationship, if it is 0 there is no lineair relationship and at +1 there is a perfect positive lineair relationship.

A positive relation means that if one variable goes up, the other also goes up (for example number of ice cream sold versus temperature), a negative relation indicates if one goes down, the other goes up (for example number of winter jackets sold versus temperature).

We can test if Pearson Correlation might be significantly different from 0 in the population. In the example the significance of this test is .000. This is the chance of finding a correlation coefficient of .880 or even higher in a sample, if in the population it would be 0 (no association). This is such a low chance, that we can say that in the population the correlation coefficient will be indeed different from zero, and conclude that there is a significant linear association between the two variables.

To determine the strength we only look at the absolute value (which means to ignore any minus sign, so the absolute value of for example -0.4 is simply 0.4).

Unfortunately there is no formal way to determine if 0.880 is high or low (although almost everyone would agree this is pretty high), and the rules of thumb floating around on the internet vary quite a lot, often depending on the field (e.g. biology, medicine, business, etc.). For example the same rule of thumb sizes from Rea and Parker (1992):

0.00 < 0.10 - Negligible
0.10 < 0.20 - Weak
0.20 < 0.40 - Moderate
0.40 < 0.60 - Relatively strong
0.60 < 0.80 - Strong
0.80 < 1.00 - Very strong

In this example we can therefor speak of a very strong effect size. A nice website where you can see and practice this can be found here. Click here to see a table with various other rule of thumbs for the interpretation.

Often you will find instead of the correlation the determination coefficient reported. The determination coefficient is the square of the correlation coefficient and indicates the proportion of the variance in the dependent variable that is predictable from the independent variable. In the example 0.880² = 0.775, which indicates that 77.5% of the variance in the current salary can be predicted based on the beginning salary.

We can now write up the results of this regression analysis in our report:

The result of the regression indicated that the two variables have a significant very strong correlation, F(1, 472) = 1622.12, p < .001, R² = .775.

Click here to see how you can perform the analysis with SPSS, R (studio), Excel, Python, or Manually.

with SPSS

the Correlatation coefficient and significance only

This method will show immediately if the coefficient is negative, and you can perform multiple tests in one time. Unfortunately though, it will not show the determination coefficient, or any other values needed to report the results.

the full Regression analysis

Be careful that the correlation coefficient with this method will always appear as positive. This method will however show the determination coefficient, and the other values needed for the report.

Click on the thumbnail below to see where to look in the output.

with R (Studio)

the regression analysis

Click on the thumbnail below to see where to look in the output.

the correlation coefficient

with Excel

with Python

Manually (formulas and example)

Formulas

There are a few different notations for the formula, but all produce the same result. Two of them for example:

$r=\frac{s_{x,y}}{s_x\times{s_y}}=\frac{\sum_{i=1}^n(x_i-\bar{x}_i)\times(y_i-\bar{y}_i)}{SS_x\times{SS_y}}$

Where:

$s_{x,y}=\frac{\sum_{i=1}^n(x_i-\bar{x}_i)\times(y_i-\bar{y}_i)}{n-1}$

$s_{x}=\sqrt{\frac{SS_x}{n-1}}$

$s_{y}=\sqrt{\frac{SS_y}{n-1}}$

With:

$SS_{x}=\sum_{i=1}^n(x_i-\bar{x}_i)^2$

$SS_{y}=\sum_{i=1}^n(y_i-\bar{y}_i)^2$

In these formulas $x_i, y_i$ are the i-th score of variable x and variable y. The $\bar{x}_i, \bar{y}_i$ are the mean of x and the mean of y.

$\bar{x}_i=\frac{\sum_{i=1}^nx_i}{n}$

$\bar{y}_i=\frac{\sum_{i=1}^ny_i}{n}$

Note that only the cases for which a score is known on both variables is used.

The test-statistic is a t-value, determined by:

$t=r_s\times\sqrt{\frac{n-2}{1-r_s^2}}$

And with degrees of freedom of:

$df=n-2$

Example

Note: Different than in the rest of this page.

Given are the scores on two variables:

$X=(4,5,8,3,5),Y=(3,5,9,7,6)$

So the first case scored a 4 on X, and a 3 on Y. Note that there are 5 scores, so n = 5.

First we determine the means (average):

$\bar{x}_i=\frac{\sum_{i=1}^nx_i}{n} =\frac{4+5+8+3+5}{5} =\frac{25}{5} =5$

$\bar{y}_i=\frac{\sum_{i=1}^ny_i}{n} =\frac{3+5+9+7+6}{5} =\frac{30}{5} =6$

Second the sum of squares:

$SS_{x}=\sum_{i=1}^n(x_i-\bar{x}_i)^2 =\sum_{i=1}^5(x_i-5)^2$

$=(4-5)^2+(5-5)^2+(8-5)^2+(3-5)^2+(5-5)^2$

$=(-1)^2+(0)^2+(3)^2+(-2)^2+(0)^2 =1+0+9+4+0 =14$

$SS_{y}=\sum_{i=1}^n(y_i-\bar{y}_i)^2 =\sum_{i=1}^5(y_i-6)^2$

$=(3-6)^2+(5-6)^2+(9-6)^2+(7-6)^2+(6-6)^2$

$=(-3)^2+(-1)^2+(3)^2+(1)^2+(0)^2 =9+1+9+1+0 =20$

Third is the covariance:

$\sum_{i=1}^n(x_i-\bar{x}_i)\times(y_i-\bar{y}_i) =\sum_{i=1}^5(x_i-5)\times(y_i-6)$

$=(4-5)\times(3-6)+(5-5)\times(5-6)+(8-5)\times(9-6)+(3-5)\times(7-6)+(5-5)\times(6-6)$

$=(-1)\times(-3)+(0)\times(-1)+(3)\times(3)+(-2)\times(1)+(0)\times(0)$

$=3+0+9+-2+0 =10$

Fourth the Pearson correlation coefficient:

$r=\frac{\sum_{i=1}^n(x_i-\bar{x}_i)\times(y_i-\bar{y}_i)}{\sqrt{SS_x\times{SS_y}}} =\frac{10}{\sqrt{14\times20}} =\frac{10}{\sqrt{14\times2^2\times5}}$

$=\frac{10}{2\sqrt{14\times5}} =\frac{5}{\sqrt{70}} =\frac{5}{\sqrt{70}}\times\frac{\sqrt{70}}{\sqrt{70}} =\frac{5\times\sqrt{70}}{\sqrt{70}\times\sqrt{70}}$

$=\frac{5\times\sqrt{70}}{70} =\frac{\sqrt{70}}{14} \approx0.5976$

Fifth we can determine the t-statistic:

$t=r\times\sqrt{\frac{n-2}{1-r^2}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{5-2}{1-\left(\frac{\sqrt{70}}{14}\right)^2}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{1-\frac{\left(\sqrt{70}\right)^2}{14^2}}}$

$=\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{1-\frac{70}{14^2}}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{1-\frac{5\times14}{14^2}}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{\frac{14}{14}-\frac{5}{14}}}$

$=\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{\frac{14-5}{14}}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{3}{\frac{9}{14}}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{3\times14}{9}} =\frac{\sqrt{70}}{14}\times\sqrt{\frac{14}{3}}$

$=\frac{\sqrt{70}}{14}\times\frac{\sqrt{14}}{\sqrt{3}} =\frac{\sqrt{70}\times\sqrt{14}}{14\times\sqrt{3}} =\frac{\sqrt{5\times14\times14}}{14\times\sqrt{3}}$

$=\frac{14\sqrt{5}}{14\times\sqrt{3}} =\frac{\sqrt{5}}{\sqrt{3}} =\frac{\sqrt{5}}{\sqrt{3}}\times\frac{\sqrt{3}}{\sqrt{3}} =\frac{\sqrt{5}\times\sqrt{3}}{\sqrt{3}\times\sqrt{3}} =\frac{\sqrt{15}}{3} \approx1.2910$

And sixth the degrees of freedom:

$df=n-2=5-2=3$

We can now complete the report by combining all the parts on the next page.

Scale vs Scale

Google adds