Scale vs Scale paired

2a: Test (paired samples t-test)

From the example on the previous pages we got the impression that a commercial had an effect on the opinion of a brand, since overall the difference was positive between the before and after score.

A paired samples t-test (a.k.a. correlated paires t-test, dependent samples t-test, or paired t-test) can be used to check if the difference in the population will also be different from zero (i.e. the two means are not the same). Note that unlike a two samples t-test we do not need to check if the variances are equal, since the data is paired (McDonald, 2014, p. 182). For the interested reader, there is a nice discussion on this on researchgate.

In the example we find a t-value of 2.399 with a significance of 0.025. This means that there is a 0.025 (2.5%) chance of a t-value of less than -2.399 or a t-value of more than 2.399, if this t-value would be 0 in the population (i.e. no difference = equal means). With a usual 0.05 significance level we consider this chance to be so low, that most likely there is actually a difference in the population as well (and not only in the sample).

We could report the results as:

A paired samples t-test showed that there was a significant difference in scores on opinion about the brand before and after seeing the commercial, t(23) = 2.40, p = .025.

Click here to see how you can perform a paired samples t-test, with SPSS, R (studio), Excel, Python, or Manually.

with SPSS

Click on the thumbnail below to see where to look in the output.

with R (Studio)

Click on the thumbnail below to see where to look in the output.

with Excel

Three different methods. The first avoids having to determine the difference for each pair, but this makes some other calculations a bit trickier, the second simply does use the difference for each pair in the calculation, and the third method uses the data analysis add-in. I personally prefer method 2.

method 1: avoiding differences per pair

method 2: with differences per pair

method 3: using add-in

with Python

Manually (formulas and example)

Formulas

$t=\frac{\bar{d}-\mu_{h_0}}{SE_d}$

$\bar{d}=\frac{\sum_{i=1}^nd_i}{n}=\bar{x}-\bar{y}$

$d_i=x_i-y_i$

$\bar{x}=\frac{\sum_{i}^n{x_i}}{n}$

$SE_d=\frac{s_d}{\sqrt{n}}$

$s_d=\sqrt{\frac{\sum_{i=1}^n{(d_i-\bar{d})^2}}{n-1}}=\sqrt{s_x^2+s_y^2-2\times{s_{x,y}}}$

$s_x^2=\frac{\sum_{i=1}^n{(x_i-\bar{x})^2}}{n-1},s_y^2=\frac{\sum_{i=1}^n{(y_i-\bar{y})^2}}{n-1}$

$s_{x,y}=\frac{\sum_{i=1}^n(x_i-\bar{x}_i)\times(y_i-\bar{y}_i)}{n-1}$

$df=n-1$

Symbols used:

x_i and y_i are the i-th scores from all the pairs (note that if one of the two is missing the case is removed and not used).

n is the total number of pairs.

μ_h0 is the difference that is expected in the population. Often this is set to zero, i.e. that there is no difference between the two variables in means.

Example

Note different example than in the others.

We are given the following pairs:

$P=\left\{(9,8),(2,),(5,3),(4,6),(8,4),(8,3)\right\}$

Since the second pair is missing a score, it will not be used and removed, so:

$X=\left\{(9,8),(5,3),(4,6),(8,4),(8,3)\right\}$

There are five pairs remaining, so n = 5.

Lets calculate the differences:

$D=\left\{9-8,5-3,4-6,8-4,8-3\right\}=\left\{1,2,-2,4,5\right\}$

Then we determine the average (mean) of these differences:

$\bar{d}=\frac{\sum_{i=1}^nd_i}{n} =\frac{\sum_{i=1}^5d_i}{5} =\frac{1+2+-2+4+5}{5} =\frac{10}{5}=2$

Then the sample standard deviation of the differences:

$s_d=\sqrt{\frac{\sum_{i=1}^n{(d_i-\bar{d})^2}}{n-1}} =\sqrt{\frac{\sum_{i=1}^5{(d_i-\bar{d})^2}}{5-1}}$

$=\sqrt{\frac{(1-2)^2+(2-2)^2+(-2-2)^2+(4-2)^2+(5-2)^2}{4}}$

$=\sqrt{\frac{(-1)^2+(0)^2+(-4)^2+(2)^2+(3)^2}{4}} =\sqrt{\frac{1+0+16+4+9}{4}} =\sqrt{\frac{30}{4}}$

$=\frac{\sqrt{30}}{\sqrt{4}} =\frac{\sqrt{30}}{2} \approx2.7386$

The Standard Error can now be determined:

$SE_d=\frac{s_d}{\sqrt{n}} =\frac{\frac{\sqrt{30}}{2}}{\sqrt{5}} =\frac{\sqrt{30}}{2\times\sqrt{5}} =\frac{1}{2}\times\frac{\sqrt{30}}{\sqrt{5}}$

$=\frac{1}{2}\times\sqrt{\frac{30}{5}} =\frac{\sqrt{6}}{2} \approx1.2247$

Assuming that we want to test that the difference would be 0 in the population, we can then determine the t-value:

$t=\frac{\bar{d}-\mu_{h_0}}{SE_d} =\frac{2-0}{\frac{\sqrt{6}}{2}} =\frac{2}{\frac{\sqrt{6}}{2}} =\frac{2\times2}{\sqrt{6}} =\frac{4}{\sqrt{6}} =\frac{4}{\sqrt{6}}\times\frac{\sqrt{6}}{\sqrt{6}}$

$=\frac{4\times\sqrt{6}}{\sqrt{6}\times\sqrt{6}} =\frac{4\times\sqrt{6}}{6} =\frac{2\times\sqrt{6}}{3} =\frac{2}{3}\sqrt{6} \approx1.6330$

The degrees of freedom is:

$df=n-1=5-1=4$

The last part would be to determine how big the difference can be classified as, which will be done on the next page.

Scale vs Scale paired

Google adds