📈 Linear Regression

What Is Linear Regression?

Linear regression is a method for modeling the relationship between a dependent variable (Y) and one or more independent variables (X) by fitting a straight line through the data. The simplest form — simple linear regression — uses one predictor and fits the equation:

Y = β₀ + β₁X + ε

Where β₀ is the intercept (value of Y when X = 0), β₁ is the slope (change in Y for a one-unit increase in X), and ε represents the error term (the part of Y not explained by X).

Linear regression is arguably the most widely used statistical method in science. It underpins calibration curves in analytical chemistry, dose-response relationships in pharmacology, growth curves in biology, and predictive models in machine learning.

Understanding R² (Coefficient of Determination)

R² tells you what proportion of the variance in Y is explained by X. It ranges from 0 to 1:

• R² = 0: The model explains none of the variability. The regression line is no better than simply using the mean of Y as a prediction.
• R² = 1: The model explains all variability. Every data point falls exactly on the regression line.
• R² = 0.75: 75% of the variability in Y is accounted for by the linear relationship with X.

Context matters enormously when evaluating R². In controlled laboratory experiments, R² > 0.95 is common and expected (e.g., standard curves). In behavioral or social science research, R² = 0.3 might be excellent. Never judge R² in isolation — consider the field, the noise level of the measurements, and the complexity of the phenomenon.

The Slope and Its Interpretation

The slope (β₁) is the heart of regression analysis. It tells you: for every one-unit increase in X, Y changes by β₁ units, on average. The sign indicates direction (positive = both increase together; negative = one increases as the other decreases). The magnitude indicates the rate of change.

The p-value for the slope tests the null hypothesis that β₁ = 0 (i.e., X has no linear effect on Y). A significant p-value means the slope is distinguishable from zero. The 95% confidence interval for the slope gives you a range of plausible values.

Why Residual Analysis Matters

Residuals are the differences between observed Y values and the values predicted by the regression line. Examining residuals is not optional — it is essential. The regression equation can always be computed, but it is only meaningful if the underlying assumptions hold.

What to Look for in Residual Plots

• Random scatter (good): Residuals should appear randomly distributed around zero with no discernible pattern. This suggests the linear model is appropriate.
• Funnel shape (bad): Residuals fan out as X increases, indicating heteroscedasticity (unequal variance). Consider transforming Y (e.g., log transform) or using weighted least squares.
• Curved pattern (bad): Suggests the true relationship is non-linear. A polynomial term or non-linear model may be needed.
• Clusters or outliers (investigate): Isolated extreme residuals may represent data entry errors, distinct subpopulations, or genuinely unusual observations.

Assumptions of Linear Regression

1. Linearity. The relationship between X and Y is linear. Check with a scatter plot and residual plot.
2. Independence. Observations are independent of each other. Violated in time-series data or clustered designs.
3. Normality of residuals. Residuals should be approximately normally distributed. Check with a Q-Q plot or histogram of residuals. Moderate violations are tolerable with large samples.
4. Homoscedasticity. The variance of residuals should be constant across all levels of X. Check by plotting residuals vs. predicted values.

Confidence Bands vs. Prediction Bands

The confidence band shows the uncertainty around the regression line itself (the mean of Y at each X). It is narrower and answers: "Where might the true regression line be?"

The prediction band shows the uncertainty for a new individual observation. It is always wider than the confidence band because it accounts for both the uncertainty in the line and the natural scatter of individual data points.

Beyond Simple Linear Regression

This calculator performs simple linear regression (one predictor). For more complex scenarios:

• Multiple regression: Two or more predictors (Y = β₀ + β₁X₁ + β₂X₂ + ...)
• Polynomial regression: Non-linear relationships modeled with X², X³ terms
• Logistic regression: When Y is binary (yes/no, survive/die)

Frequently Asked Questions