Hypothesis

Testing
Learning Objectives
Understand . . .
• The differences between parametric and nonparametric tests and
when to use each.
• The factors that influence the selection of an appropriate test of
statistical significance.
• How to interpret the various test statistics

17-2
When Data Present a Clear Picture

Researchers use hypothesis

testing to hunt for truth. As
Abacus states in this ad,
when researchers ‘sift
through the chaos’ and ‘find
what matters’ they experi-
ence the “ah ha!” moment.

17-3
Statistical Testing Procedures

State
Statenull
nullhypoth-
hypoth-
esis
esis

Interpret
Interpretthe
the Choose
Choosestatisti-
statisti-
test
test cal
caltest
test
Stages
Stages

Obtain
Obtaincritical Select
critical Selectlevel
levelofof
test
testvalue
value significance
significance
Compute
Computedif-
dif-
ference
ferencevalue
value
17-4
Tests of Significance

Parametric Nonparametric

17-5
Test Conducted
Non-parametric Tests?
- While most common statistical analyses (e.g., t-tests, ANOVA) are parametric, they
need to fulfil a number of criteria before we use them
- These criteria include satisfying the assumptions of outliers, linearity, normality, ho-
moscedasticity, to name a few
- If the data do not fulfil the criteria to conduct the parametric tests, we can opt for
non-parametric tests, which do not require those assumptions
- Do note that non-parametric tests make less assumptions, not no assumptions!
- The trade-off is that non-parametric tests are generally lower in power
Types of Non-parametric Tests
- In this set of slides, the focus is on 4 non-parametric tests
- Each of these 4 tests is a non-parametric version of t-tests and ANOVAs

Parametric Test Non-parametric Test

Between Subjects t-test Mann-Whitney U Test

Within Subjects t-test Wilcoxon Signed Ranked Test

One-way Between
Kruskal-Wallis One-way ANOVA
Subjects ANOVA
One-way Within
Friedman’s ANOVA
Subjects ANOVA
The Mann-Whitney U test
• The Mann-Whitney U test is used to compare differences between two
independent groups when the dependent variable is either ordinal or
continuous, but not normally distributed. For example, you could use the
Mann-Whitney U test to understand whether attitudes towards pay
discrimination, where attitudes are measured on an ordinal scale, differ
based on gender (i.e., your dependent variable would be "attitudes towards
pay discrimination" and your independent variable would be "gender", which
has two groups: "male" and "female").
• Alternately, you could use the Mann-Whitney U test to understand whether
salaries, measured on a continuous scale, differed based on educational level
(i.e., your dependent variable would be "salary" and your independent
variable would be "educational level", which has two groups: "high school"
and "university"). The Mann-Whitney U test is often considered the
nonparametric alternative to the independent t-test although this is not
always the case.
Mann-Whitney U Test

“A researcher is interested in finding out if there are

differences in cholesterol concentration for two
independent groups
Group 1 follows a strict diet and Group 2 does
exercise
 Mann-Whitney U Test - SPSS

Assume that the data has multiple out-

liers, which is why the researcher opted to
conduct a Mann-Whitney U test, rather
than a t-test.

Analyze -> Nonparametrics Tests -> Leg-

acy Dialogs -> 2 Independent Samples…
Mann-Whitney U Test - SPSS

1. Move choles to the right under Test Vari-

able List
2. Move groups as our Grouping Variable
3. Then define groups by clicking on Define
Groups
4. Input ‘1’ and ‘2’ as groups 1 and 2 respect-
ively
5. Continue and OK!
"U" reflects the difference
between the two rank totals.
Types of Non-parametric Tests

Parametric Test Non-parametric Version

Between Subjects t-test Mann-Whitney U Test

Within Subjects t-test Wilcoxon Signed Ranked Test

One-way Between
Kruskal-Wallis One-way ANOVA
Subjects ANOVA

One-way Within
Friedman’s ANOVA
Subjects ANOVA
Wilcoxon signed-rank test
• The Wilcoxon signed-rank test is the nonparametric test
equivalent to the dependent t-test. It is used to
compare two sets of scores that come from the same
participants. This can occur when we wish to
investigate any change in scores from one time point to
another, or when individuals are subjected to more
than one condition
Wilcoxon Signed-Ranks Test- The Wilcoxon signed-rank test is the
nonparametric test equivalent to the dependent t-test.

A researcher wants to find out if implementing a

reading program will help improve reading
speed. The researcher recruited 50 participants to
enrol in the reading program, and recorded
their reading speed (in seconds) at 2 time
periods: before and after the reading program.
Wilcoxon Signed-Ranks Test - SPSS

Assume that the researcher only

managed to get 40 participants, and
opted to conduct a Wilcoxon signed
ranked test, rather than a within sub-
jects t-test.

Analyze -> Nonparametrics Tests ->

Legacy Dialogs -> 2 Related
Samples….
Wilcoxon Signed-Ranks Test - SPSS

1. Move Pretest and Posttest as Pair

1
2. Tick Wilcoxon in Test type
3. OK!
The rank mean of one group is com-
pared
to the overall rank mean to determine a
test statistic called a z-score. If the
groups
are evenly distributed, then the z-score
will be closer to 0
Types of Non-parametric Tests

Parametric Test Non-parametric Version

Between Subjects t-test Mann-Whitney U Test

Within Subjects t-test Wilcoxon Signed Ranked Test

One-way Between
Kruskal-Wallis One-way ANOVA
Subjects ANOVA

One-way Within
Friedman’s ANOVA
Subjects ANOVA
Kruskal-Wallis One-Way ANOVA

• The Kruskal-Wallis H test (sometimes also called the

"one-way ANOVA on ranks") is a rank-based
nonparametric test that can be used to determine if
there are statistically significant differences between
two or more groups of an independent variable on a
continuous or ordinal dependent variable. It is
considered the nonparametric alternative to the
one-way ANOVA, and an extension of the
Mann-Whitney U test to allow the comparison of more
than two independent groups.
Kruskal-Wallis One-Way ANOVA

A researcher is interested in finding out if

there is a difference in physical
well-being (rated 1-100) among
teenagers, young adults, and
seniors. He recruited 10 teenagers, 10
adults, and 10 seniors for the
experiment.

In this case, the IV is age group,

and DV is physical well-being
 Kruskal-Wallis One-Way ANOVA

Assume that the data did not meet the

criteria of parametric tests, thus the re-
searcher opted to conduct a Kruskal-Wal-
lis test.

Analyze -> Nonparametrics Tests -> Leg-

acy Dialogs -> K Independent Samples….
Kruskal-Wallis One-Way ANOVA

1. Move PhysicalWellBeing into the test

variable list box, and AgeGroup into
the grouping variable box
2. Tick Kruskal-Wallis H under Test type
3. Then define the grouping variable
(Define Range)
4. Go to Options and select Descriptives
Kruskal-Wallis One-Way ANOVA

To define groups:
5. In our dataset, Teenagers were
coded as ‘1’, Adults as ‘2’, and Seni-
ors as ‘3’
6. Hence, the range for our grouping
variable is 1-3; with a minimum of 1
and maximum of 3
7. Click Continue, and OK
 Kruskal-Wallis One-Way ANOVA

Kruskal-Wallis H score =
Similar to Mann-Whitney 7.50, p = .024
U tests, SPSS ranks the
data. Given an alpha value
The value here displays of .05, there is a signific-
the average of the rank- ant difference between
ings teenagers’, adults’, and
seniors’ self reported
physical wellbeing
However

• Although we now know that there is a significant difference between the

3 groups, we do not know exactly where the difference(s) lie

• It could lie between teenagers and adults, adults and seniors, teenagers
and seniors, or even all of the above
Types of Non-parametric Tests

Parametric Test Non-parametric Version

Between Subjects t-test Mann-Whitney U Test

Within Subjects t-test Wilcoxon Signed Ranked Test

One-way Between
Kruskal-Wallis One-way ANOVA
Subjects ANOVA

One-way Within
Friedman’s ANOVA
Subjects ANOVA
Friedman’s ANOVA

• The Friedman test is the non-parametric alternative to

the one-way ANOVA with repeated measures. It is used
to test for differences between groups when the
dependent variable being measured is ordinal
Friedman’s ANOVA

A researcher wants to find out if implementing a

reading program will help improve reading
speed. The researcher recruited 50 participants to
enrol in the reading program, and recorded
their reading speed (in seconds) at 3 time
periods: before and after the reading program,
and at one month follow-up.
Friedman’s ANOVA - SPSS

Assume that the data did not meet the

criteria of parametric tests, thus the re-
searcher opted to conduct a Friedman’s
ANOVA.

Analyze -> Nonparametrics Tests ->

Legacy Dialogs -> K Related Samples….
Friedman’s ANOVA - SPSS

1. Move Pretest, Posttest, and

OneMonthFollowup inot the test
variables box
2. Tick Friedman in Test type
3. Go to Statistics and select Descript-
ives
4. OK!
Friedman’s ANOVA - SPSS

Chi-square statistic = 12.2, p = .002

Given an alpha value of .05, there is

a significant difference between
pre-test, postttest, and the one
month follow up
However

• Just like the Kruskal-Wallis test, although we now know that there is a sig-
nificant difference between the three groups, we do not know exactly
where the difference(s) lie
• Simply by eyeballing the mean ranks, we can probably guess that the dif-
ference comes from the improvement from pre-test to post-test (2.9 vs
1.6), but not so much from the post-test to one month follow-up (1.6 vs
1.5)
Thank you