What Is a T-Test?
The t-test is one of the most widely used statistical procedures in science. Developed by William Sealy Gosset in 1908 (published under the pseudonym "Student"), it determines whether the means of two groups are significantly different from each other. The core logic is straightforward: is the observed difference between group means large enough to be unlikely due to random chance alone?
T-tests are appropriate when you are comparing means from continuous, approximately normally distributed data. They form the basis for more complex analyses like ANOVA and are a staple of biomedical research, psychology, education, and quality control.
Three types of t-tests
1. Independent two-sample t-test. Use when comparing means from two separate groups of subjects (treatment vs. control, male vs. female, drug A vs. drug B). The key assumption is that observations in one group are independent of those in the other. The classic version assumes equal variances; when violated (common in practice), use the Welch t-test instead, which adjusts degrees of freedom for unequal variances.
2. Paired (dependent) t-test. Use when the same subjects are measured twice — before and after treatment, left eye vs. right eye, or two matched conditions. The paired test works on the differences within each pair, removing inter-subject variability and increasing power. Example: measuring blood glucose in 20 patients before and after a drug, then testing whether the mean change differs from zero.
3. Welch's t-test. A modification of the independent t-test that does not assume equal variances. It uses the Welch-Satterthwaite equation to estimate degrees of freedom. For groups close to normal in shape, Welch is a useful default because it performs well even when variances happen to be equal.
Understanding p-Values
The p-value answers a specific question: If the null hypothesis were true (i.e., no real difference exists), how likely would we be to observe a difference at least as extreme as what we found? A small p-value (conventionally p < 0.05) suggests the observed difference is unlikely under the null hypothesis.
However, p-values have important limitations. They do not tell you the probability that the null hypothesis is true. They do not tell you the size or importance of the effect. A very large sample can produce a tiny p-value for a trivially small difference. This is why effect sizes are essential.
Effect Size: Hedges' g
This calculator reports Hedges' g, the small-sample-corrected version of Cohen's d. Cohen's d quantifies the magnitude of the difference in standard deviation units. It answers: "How big is the difference, practically speaking?"
| Cohen's d | Interpretation | Example |
|---|
| 0.2 | Small | Barely noticeable in practice |
| 0.5 | Medium | Visible to careful observers |
| 0.8 | Large | Obvious to anyone |
| > 1.2 | Very large | Dramatic, often clinically meaningful |
Always report both the p-value and the effect size. A statistically significant result with a tiny effect size may not be practically meaningful. Conversely, a non-significant result with a large effect size may indicate insufficient sample size rather than absence of an effect.
Assumptions of the T-Test
- Continuous data. The dependent variable must be measured on an interval or ratio scale.
- Independence. For the independent t-test, observations within and between groups must be independent.
- Normality. The data (or differences, for paired tests) should be approximately normally distributed. With n > 30 per group, the Central Limit Theorem makes the t-test robust to moderate non-normality.
- Equal variances (for classic independent t-test only). Use Levene's test to check, or simply default to Welch's test.
When the T-Test Is Not Appropriate
When the fixed shape rule flags skewness, Auto selects the Mann–Whitney U test for independent samples or the Wilcoxon signed-rank test for paired differences. Otherwise, it selects Welch or paired t. If you are comparing more than two groups, use ANOVA instead.
Step-by-Step: How to Use This Calculator
- Enter or paste data for both groups. Each group needs at least two valid observations; paired groups must have the same length. Commas, spaces, tabs, and newlines are accepted.
- Choose the design: Independent or Paired.
- Choose Auto or an available method for that design: Student's t, Welch, or Mann-Whitney U for independent groups; paired t or Wilcoxon signed-rank for paired groups.
- Results update automatically as you enter data or change options. Review the test statistic, degrees of freedom when applicable, p-value, confidence interval, and Hedges' g (or rank-biserial r for rank-based methods), then interpret the effect size alongside the p-value.
Frequently Asked Questions
Should I use a one-tailed or two-tailed test?
Use a two-tailed test unless you have a strong, pre-specified directional hypothesis before collecting data. In practice, most journals and reviewers expect two-tailed tests. One-tailed tests cut your p-value in half but are only justified when you can genuinely rule out an effect in the opposite direction.
My sample sizes are unequal. Is that a problem?
Unequal sample sizes are common and acceptable. However, they can make the classic t-test sensitive to variance differences. For groups close to normal in shape, Auto uses Welch to handle unequal n's and variances; if the fixed shape rule flags skewness, Auto uses Mann–Whitney U.
What if my data is not normally distributed?
Auto uses a fixed shape rule, not sample size or the Shapiro–Wilk p-value. For independent groups, it selects Mann–Whitney U if either group has |adjusted Fisher–Pearson skewness| > 1 or a mean–median gap > 0.5 SD. For paired data, it applies the same rule to the differences and selects Wilcoxon signed-rank; otherwise it selects Welch or paired t.
How do I report a t-test result in a paper?
Follow APA style: "An independent-samples t-test revealed a significant difference in blood pressure between the treatment (M = 120.3, SD = 8.2) and control groups (M = 132.1, SD = 9.7), t(38) = 4.12, p < .001, Hedges' g = 1.31." Always include means, SDs, t-statistic, df, p-value, and effect size.