Choosing a statistical test feels hard because there are dozens of them. In practice, almost every choice comes down to three questions: what you want to find out, what kind of variables you have, and how your data were collected. Answer those, and the list shrinks to one or two candidates.
This guide walks through the three questions, gives you a quick lookup table, and links each test to a free tool on FreeStatisticalTools.com that runs in your browser.
- Goal: are you comparing groups, looking for a relationship, predicting an outcome, or checking a measure?
- Variables: is your outcome continuous (numbers), ordinal (ranks) or categorical (groups)?
- Design: are the groups independent (different people) or related (the same people measured twice)?
- Check assumptions before you trust a parametric test, and have a nonparametric backup ready.
In this guide
Question 1: What do you want to find out?
Most research questions fall into one of five families:
- Compare groups. "Do men and women differ in stress scores?" "Did scores change after training?" These call for t-tests, ANOVA or their nonparametric versions.
- Look for a relationship. "Is study time related to exam marks?" This is a correlation question.
- Predict an outcome. "Which factors predict purchase intention?" This is regression.
- Test an association between categories. "Is smoking status related to gender?" This is a chi-square-type question.
- Check a measurement. "Is my questionnaire reliable?" "Do my items measure the constructs I think they measure?" This is reliability analysis or factor analysis.
If your question involves why or when an effect happens (an indirect path through a third variable, or an effect that depends on another variable), you are in mediation and moderation territory, covered at the end of this guide.
Question 2: What type are your variables?
The outcome (dependent variable) matters most. Statisticians usually group variables into three types:
| Type | What it looks like | Examples |
|---|---|---|
| Continuous (interval/ratio) | Numbers where differences are meaningful | Age, income, reaction time, a scale score averaged over items |
| Ordinal | Ordered categories, uneven gaps | Single Likert item (1–5), education level, rankings |
| Categorical (nominal) | Labels with no order | Gender, country, treatment group, yes/no |
A scale score made by averaging several Likert items is normally treated as continuous. A single Likert item is safer to treat as ordinal, which points you to nonparametric tests.
Question 3: How was the data collected?
- Independent groups: each person appears in only one group (for example, a treatment group and a control group).
- Related (paired or repeated) measures: the same people are measured more than once (before and after), or people are matched in pairs.
Using an independent-groups test on paired data (or the reverse) is one of the most common mistakes in student theses, so be clear about this before you choose.
The quick lookup table
Find your goal and data type, then follow the link to run the test.
| Goal | Outcome | Groups / predictors | Parametric test | Nonparametric alternative |
|---|---|---|---|---|
| Compare a mean to a known value | Continuous | One sample | One-sample t-test | Wilcoxon signed-rank test |
| Compare 2 independent groups | Continuous | 2 groups | Independent samples t-test | Mann-Whitney U test |
| Compare 2 related measurements | Continuous | Same people twice | Paired t-test | Wilcoxon signed-rank test |
| Compare 3+ independent groups | Continuous | 3+ groups | One-way ANOVA | Kruskal-Wallis test |
| Compare 3+ related measurements | Continuous | Same people 3+ times | Repeated measures ANOVA | Friedman test |
| Two grouping factors | Continuous | 2 factors | Two-way ANOVA | — |
| Relationship between two variables | Continuous / ordinal | — | Pearson correlation | Spearman correlation (same tool) |
| Predict a continuous outcome | Continuous | 1+ predictors | Multiple regression | Regression with robust SEs |
| Predict a yes/no outcome | Binary | 1+ predictors | Logistic regression | — |
| Association between two categorical variables | Categorical | — | Chi-square test | Fisher's exact test (small samples) |
| Change in a yes/no outcome, same people | Binary | Paired | McNemar test | — |
| Questionnaire reliability | Items | — | Cronbach's alpha | McDonald's omega |
| Find the structure behind items | Items | — | Exploratory factor analysis | — |
| Test a theoretical model with latent variables | Items | Constructs and paths | CB-SEM Studio | PLS-SEM Studio |
Before any test, run descriptive statistics and look at a histogram or box plot of each variable. Skewed distributions, outliers or tiny groups will change which test you should use. The descriptive statistics calculator gives you means, medians, SDs, skewness and charts in one go.
Parametric or nonparametric?
Parametric tests (t-tests, ANOVA, Pearson correlation, regression) compare means and assume, roughly, that:
- observations are independent;
- the outcome (or the model's residuals) is approximately normally distributed;
- groups have similar variances (for t-tests and ANOVA).
With moderate samples (say, 30 or more per group) these tests are fairly robust to non-normality, thanks to the central limit theorem. They are less forgiving of heavy outliers and very unequal variances. Two practical rules:
- Unequal variances? Report Welch's version. The independent t-test tool shows Student and Welch results side by side, and the one-way ANOVA tool reports Welch's F.
- Small, skewed samples or ordinal outcomes? Use the nonparametric alternative from the table above. You can check normality with the Shapiro-Wilk test, but always look at a plot too: with large samples, the test flags trivial deviations.
Worked examples
Example 1. A nurse researcher compares anxiety scores (a 20-item scale) between patients who received a pre-surgery briefing and patients who did not. Outcome: continuous. Groups: two, independent. → Independent samples t-test (Welch version if variances differ).
Example 2. A teacher measures the same 25 students' reading scores in September, January and June. Outcome: continuous. Design: same people, three time points. → Repeated measures ANOVA, or the Friedman test if the scores are badly skewed.
Example 3. A marketer asks 400 customers whether they would recommend the brand (yes/no) and records their age group. Two categorical variables. → Chi-square test of independence, reporting Cramér's V as the effect size.
Example 4. A PhD student surveys 350 employees and wants to know whether job autonomy predicts engagement through intrinsic motivation. That is an indirect effect. → Mediation analysis with bootstrapped confidence intervals, or a full structural equation model if the constructs are measured with several items each.
Beyond the basics: mediation, moderation and SEM
- Mediation asks whether X affects Y through M. Use the mediation analysis calculator and read our step-by-step mediation guide.
- Moderation asks whether the effect of X on Y depends on W. Use the moderation analysis calculator, which gives simple slopes and the Johnson-Neyman region; see our moderation guide.
- Both together (moderated mediation) are covered by the PROCESS macro online tool.
- Latent variables measured by several questionnaire items are best handled with structural equation modeling. Our comparison of CB-SEM vs PLS-SEM helps you choose between the two approaches.
Common mistakes to avoid
- Running many t-tests instead of one ANOVA. Each extra test inflates the chance of a false positive.
- Treating paired data as independent, which throws away power and gives wrong p-values.
- Using a chi-square test when more than 20% of expected counts are below 5. Switch to Fisher's exact test for 2×2 tables.
- Reporting only p-values. Always add an effect size (Cohen's d, eta squared, r, Cramér's V or an odds ratio) and a confidence interval.
- Choosing the test after seeing which one gives p < .05. Decide on the analysis before you look at the results.
Frequently asked questions
How do I know which statistical test to use?
Answer three questions: what is your goal (compare groups, find a relationship, predict, or check a measure), what type is your outcome variable (continuous, ordinal or categorical), and are your groups independent or related? The lookup table in this guide maps each combination to a test.
Should I use a t-test or ANOVA?
Use a t-test to compare exactly two group means and a one-way ANOVA for three or more groups. With two groups, ANOVA and the t-test give the same p-value (F equals t squared).
When should I use a nonparametric test?
Use one when your outcome is ordinal (for example a single Likert item), when samples are small and clearly skewed, or when there are extreme outliers you cannot justify removing. Nonparametric tests compare ranks rather than means.
Can I treat Likert scale data as continuous?
A scale score that averages several Likert items is usually treated as continuous and analysed with parametric tests. A single Likert item is better treated as ordinal, so use tests such as the Mann-Whitney U or Kruskal-Wallis test.
Is there free software that runs these tests?
Yes. Every test linked in this guide runs free in your browser on FreeStatisticalTools.com, with no sign-up, and your data stays on your device. Results come out as APA-style tables.