Compare three or more groups: ANOVA and Kruskal-Wallis

When you compare more than two groups, running a t-test for every pair is tempting but wrong. With four faculties there are six pairs, and at α = .05 for each, the chance of at least one false positive rises to about 26%. Analysis of variance (ANOVA) tests all groups at once with a single F-test, and post-hoc tests then compare pairs while controlling the overall error rate.

RQ2: does stress differ between faculties?

The descriptive table shows mean stress of 3.16 in Arts, 3.23 in Business, 3.48 in Science and 3.55 in Engineering. Levene’s test, F(3, 236) = 0.41, p = .746, supports equal variances. The ANOVA table gives F(3, 236) = 2.87, p = .037: stress differs significantly between faculties.

How big is the effect?

Eta squared (η²) is the share of the variance in stress explained by faculty: η² = .035, so faculty accounts for 3.5% of the differences in stress between students. Omega squared (ω² = .023) is a less biased estimate of the same quantity in the population. Benchmarks for η² are .01 small, .06 medium and .14 large, so this is a small effect. Welch’s F, which does not assume equal variances, gives the same conclusion: F(3, 127.90) = 2.95, p = .035.

The post-hoc puzzle

Now read the Tukey HSD table. Surprisingly, no pair of faculties differs significantly; the closest is Arts versus Engineering, a difference of 0.38 points with an adjusted p of .063. How can the overall test be significant when no single pair is?

This happens regularly and is not a mistake. The F-test asks whether the means are spread out more than chance would predict, and it pools information from all groups. Tukey’s test protects against false positives across six comparisons, which makes each comparison more conservative. Here the pattern is that the two “technical” faculties (Engineering and Science) are higher than Arts and Business, but no single pair is different enough on its own. Report it honestly: “The omnibus test was significant, but no pairwise difference reached significance after Tukey adjustment; descriptively, stress was highest in Engineering (M = 3.55) and lowest in Arts (M = 3.16).” If you had a specific prediction before collecting data (for example, Engineering versus Arts), a planned contrast would have been more powerful than all-pairs post-hoc testing.

Choosing a post-hoc test

• Tukey HSD (Tukey-Kramer for unequal group sizes): the standard choice when variances are equal.
• Games-Howell: when variances differ between groups.
• Bonferroni: simple and conservative; good for a small number of planned comparisons.

Kruskal-Wallis: when the outcome is skewed

Study hours are skewed, so for comparing them between faculties use the Kruskal-Wallis H test, the rank-based alternative to one-way ANOVA.

Median weekly study hours are 10 in Arts and Business, 13 in Science and 16 in Engineering. The difference is significant and fairly large: H(3) = 28.42, p < .001, ε² = .12. Dunn’s pairwise tests with Bonferroni correction show that Engineering students study more than Arts (p_Bonf < .001) and Business students (p_Bonf < .001), and Science students more than Business students (p_Bonf = .043). Always use the adjusted column when you report pairwise results.

Writing it up

A one-way ANOVA showed that stress differed between faculties, F(3, 236) = 2.87, p = .037, η² = .035. Tukey-Kramer post-hoc tests did not identify significant pairwise differences; stress was descriptively highest among Engineering students (M = 3.55, SD = 0.84) and lowest among Arts students (M = 3.16, SD = 0.81). Weekly study hours also differed between faculties, Kruskal-Wallis H(3) = 28.42, p < .001, ε² = .12. Bonferroni-corrected Dunn tests showed that Engineering students (Mdn = 16) studied more than Arts and Business students (both Mdn = 10), and Science students (Mdn = 13) more than Business students.

Beyond one factor

If you have two factors, such as faculty and gender, use a two-way ANOVA to test both main effects and their interaction. For the same people measured three or more times, use repeated measures ANOVA or the Friedman test. To adjust for a covariate such as age, use ANCOVA. All of these tools follow the same logic you used here.