What Is ANOVA?
Analysis of Variance (ANOVA) is a statistical technique that tests whether the means of three or more groups differ significantly. While the name says "variance," it is fundamentally a test about means. The core idea: ANOVA partitions the total variability in your data into between-group variability and within-group variability, then asks whether the between-group variability is large enough to be unlikely by chance.
ANOVA was developed by Ronald A. Fisher in the 1920s and remains one of the most commonly used methods in experimental research. Any time you have a categorical independent variable (e.g., treatment group A, B, C) and a continuous dependent variable (e.g., tumor volume, test score), one-way ANOVA is likely your starting point.
When to use ANOVA (and when not)
A t-test compares exactly two groups. ANOVA extends this to three or more. Running multiple pairwise t-tests instead causes multiple comparisons inflation: 6 pairwise t-tests across 4 groups at alpha = 0.05 push the overall Type I error rate to roughly 26%. ANOVA controls this with a single omnibus test, then defers pairwise questions to post-hoc tests.
Three assumptions matter: independence of observations within and between groups; normality of the dependent variable within each group (robust to moderate violation when n > 15 per group); and homogeneity of variances (Levene's test checks this; if violated, use Welch's ANOVA or Kruskal-Wallis).
| Scenario | Recommended Test |
|---|
| 2 groups | T-test (or Welch's t-test) |
| 3+ groups, 1 factor | One-way ANOVA |
| 3+ groups, 2+ factors | Two-way / Factorial ANOVA |
| Repeated measures on same subjects | Repeated-measures ANOVA |
| Non-normal data, 3+ groups | Kruskal-Wallis test |
How ANOVA works: F-statistic and effect size
ANOVA calculates the F-statistic, the ratio of between-group variance to within-group variance:
F = MSbetween / MSwithin
MSbetween captures how much group means differ from the grand mean; MSwithin captures average variability within groups. A large F means the groups differ much more than random variation would predict. The F-statistic follows an F-distribution with (k−1, N−k) degrees of freedom, where k is the number of groups and N the total sample size.
Always report effect size alongside p-value. Eta-squared (η²) is the proportion of total variance explained by the grouping variable:
| η² | Interpretation |
|---|
| 0.01 | Small effect |
| 0.06 | Medium effect |
| 0.14 | Large effect |
Post-hoc tests: finding which groups differ
A significant ANOVA tells you that at least one group mean differs, but not which. Post-hoc tests perform pairwise comparisons while controlling for multiple testing. If ANOVA is not significant (p > 0.05), do not run post-hoc tests — the omnibus result has already shown insufficient evidence.
Tukey's HSD (Honestly Significant Difference) is the standard choice: compares every pair of group means while controlling the family-wise error rate. Best when sample sizes are equal or nearly so. Bonferroni correction is simpler but more conservative — divide alpha by the number of comparisons (3 groups → alpha = 0.05/3 = 0.0167); easy to apply but can be overly strict with many groups.
Non-parametric alternative: Kruskal-Wallis
When normality is violated or data is ordinal, the Kruskal-Wallis test is the non-parametric equivalent of one-way ANOVA. It ranks all observations and tests whether mean ranks differ between groups. It assumes neither normality nor equal variances, but is less powerful than ANOVA when assumptions hold.
Frequently asked questions
Can I use ANOVA with only 2 groups?
Yes, and the result will be mathematically identical to an independent t-test (F = t²). However, for 2 groups a t-test is simpler and more conventional.
What if my group sizes are very unequal?
Unequal group sizes reduce the robustness of ANOVA to violations of homogeneity of variances. Use Welch's ANOVA or the Kruskal-Wallis test when variances differ and sample sizes are unbalanced.
How do I report ANOVA results?
APA style: "A one-way ANOVA revealed a significant effect of treatment on tumor volume, F(2, 45) = 8.73, p < .001, η² = .28. Tukey's HSD indicated that Group A (M = 12.3, SD = 3.1) differed significantly from Group C (M = 18.7, SD = 4.2), p = .002."
My data is ordinal (e.g., Likert scale ratings). Can I use ANOVA?
This is debated. Many researchers use ANOVA on Likert data with 5+ points, arguing it is robust enough. The conservative approach is to use the Kruskal-Wallis test for ordinal data. Either way, report your rationale.