Simple Linear Regression Model Study Pack
Kibin's free study pack on Simple Linear Regression Model includes a 7-section study guide, 25 quiz questions, 30 flashcards, and 5 open-ended Explain review questions. Sign up free to track your progress toward mastery, plus upload your own notes and recordings to create personalized study packs organized by course.
Last updated May 28, 2026
Simple Linear Regression Model Study Guide
Break down simple linear regression from the equation ŷ = b₀ + b₁x to least squares estimation, residual analysis, r², and hypothesis testing on the slope — everything you need to model, validate, and interpret a linear relationship between two variables.
Key Takeaways
- •Simple linear regression models the relationship between one explanatory variable and one continuous response variable using the equation ŷ = b₀ + b₁x, where b₀ is the y-intercept and b₁ is the slope.
- •The least squares method determines b₀ and b₁ by minimizing the sum of squared residuals — the vertical distances between observed data points and the fitted regression line.
- •The slope b₁ quantifies the average change in the response variable for each one-unit increase in the explanatory variable, while b₀ gives the predicted value of y when x equals zero.
- •The coefficient of determination (r²) measures the proportion of total variability in the response variable that is explained by the linear relationship with the explanatory variable.
- •Residual analysis — checking that residuals are roughly normally distributed, have constant variance, and show no pattern — is essential for validating regression assumptions before interpreting results.
- •A hypothesis test on the slope (H₀: β₁ = 0) determines whether the linear relationship between x and y is statistically significant, using a t-statistic with n − 2 degrees of freedom.
- •Predictions made within the observed range of x (interpolation) are generally reliable, while extrapolation beyond that range can produce misleading or invalid estimates.
What Simple Linear Regression Does and When to Use It
Simple linear regression is a statistical technique for describing and quantifying a straight-line relationship between one numerical explanatory variable and one numerical response variable, and for using that relationship to make predictions.
Response and Explanatory Variables
- •The response variable (also called the dependent variable, labeled y) is the outcome being predicted or explained.
- •The explanatory variable (also called the independent or predictor variable, labeled x) is the variable used to explain or predict changes in y.
- •Simple linear regression involves exactly one explanatory variable; adding more predictors moves into multiple regression.
Conditions That Justify a Linear Model
- •A scatterplot of y versus x should show a roughly linear trend before fitting a regression line — applying a linear model to a curved relationship produces systematically wrong predictions.
- •Both variables must be measured on a numerical scale; regression does not apply directly to categorical response variables.
- •Observations should be independent of one another, meaning the value of y for one case should not influence the value for another.
The Regression Equation: Slope, Intercept, and Predicted Values
The regression model expresses the predicted value of the response variable as a linear function of the explanatory variable, and each coefficient in that equation carries a specific, interpretable meaning.
Structure of the Regression Equation
- •The fitted regression equation is written ŷ = b₀ + b₁x, where ŷ (pronounced 'y-hat') is the predicted value of y for a given value of x.
- •The population version of the model is written μy = β₀ + β₁x, where β₀ and β₁ are true (unknown) parameters that b₀ and b₁ estimate from sample data.
Interpreting the Slope (b₁)
- •The slope b₁ represents the average change in ŷ for each one-unit increase in x, holding all else constant.
- •A positive slope indicates that y tends to increase as x increases; a negative slope indicates the opposite.
- •The slope is the most substantively important coefficient because it directly describes the direction and rate of the linear relationship.
Interpreting the Y-Intercept (b₀)
- •The intercept b₀ is the predicted value of y when x = 0.
- •In many real-world contexts the intercept has no practical meaning because x = 0 falls outside the observed data range or is physically impossible — in those cases the intercept is a mathematical anchor for the line, not a meaningful prediction.
Making Predictions with the Equation
- •Substituting any specific value of x into ŷ = b₀ + b₁x yields a point estimate of the mean response at that x-value.
- •Interpolation — predicting within the range of x values used to build the model — is valid; extrapolation beyond that range is unreliable because the linear pattern may not continue.
Estimating Coefficients: The Least Squares Method
The regression line is not placed arbitrarily — a specific mathematical criterion called least squares uniquely determines the best-fitting line from the data.
Residuals: Measuring Prediction Error
- •A residual is the difference between an observed value and the model's predicted value: eᵢ = yᵢ − ŷᵢ.
- •Positive residuals indicate the model underestimated the observed value; negative residuals indicate overestimation.
- •Residuals are the raw material for assessing both model fit and assumption validity.
The Least Squares Criterion
- •The least squares method selects b₀ and b₁ to minimize the sum of squared residuals (SSE = Σeᵢ²), rather than the simple sum, so that positive and negative errors do not cancel each other out.
- •Squaring the residuals also penalizes large errors more heavily than small ones, making the line sensitive to outliers.
Formulas for the Least Squares Coefficients
- •The slope is computed as b₁ = Σ[(xᵢ − x̄)(yᵢ − ȳ)] / Σ[(xᵢ − x̄)²], which is the ratio of the covariance of x and y to the variance of x.
- •Once the slope is known, the intercept follows from b₀ = ȳ − b₁x̄, which guarantees that the fitted line always passes through the point (x̄, ȳ).
- •These formulas produce the unique line that minimizes SSE — no other straight line will yield a smaller total squared error for the same data set.
Measuring Model Fit: Variance Partitioning and r²
After fitting a regression line, it is important to quantify how well the model actually explains the variability in the response variable, rather than simply accepting the equation at face value.
Partitioning Total Variability
- •The total variability in y is measured by SST (total sum of squares) = Σ(yᵢ − ȳ)², which captures how spread out the y values are around their mean.
- •SST splits into two parts: SSR (regression sum of squares), the variability explained by the linear relationship with x, and SSE (error sum of squares), the variability left unexplained.
- •The relationship SST = SSR + SSE holds exactly for every simple linear regression model.
Coefficient of Determination (r²)
- •The coefficient of determination r² = SSR / SST expresses the proportion of total variability in y that is accounted for by the linear regression on x.
- •r² ranges from 0 to 1; a value of 0.85 means 85% of the variation in y is explained by the linear relationship with x, and 15% remains unexplained.
- •r² is the square of the Pearson correlation coefficient r, so the sign of r (positive or negative) tells you the direction of the relationship that r² alone cannot convey.
Relationship Between r² and Model Usefulness
- •A high r² does not automatically mean the model is correctly specified — a curved relationship can still produce a moderately high r² when fit with a straight line while violating linearity assumptions.
- •r² should always be interpreted alongside residual plots rather than in isolation.
Unlock the rest of this study guide
- Access the full study pack
- Track your mastery and be test-day ready
- Upload your own notes to build personalized study guides, quizzes, flashcards, and more
About this Study Pack
Created by Kibin to help students review key concepts, prepare for exams, and study more effectively. This Study Pack was checked for accuracy and curriculum alignment using authoritative educational sources. See sources below.
Sources
Question 1 of 25
Your progress is saved after each question and counts toward mastery.
What does the slope b₁ represent in a simple linear regression equation?
Card 1 of 30
Your progress is saved after each card and counts toward mastery.
Concept 1 of 5
Your progress is saved after each concept and counts toward mastery.
The Regression Equation and Its Coefficients
Explain what the regression equation ŷ = b₀ + b₁x means. What does each part of the equation represent, and how would you interpret the slope and y-intercept in a real-world example?
More in Statistics
See all topics →ANOVA Foundations
Break down ANOVA from the ground up — covering between- and within-groups variance, the F-statistic ratio, degrees of freedom, and why a significant result still requires post-hoc tests to pinpoint which group means differ.
Binomial Distributions
Master the binomial distribution from the ground up — covering the four conditions for a binomial setting, the probability formula P(X = k) = C(n, k) · pᵏ · (1 - p)ⁿ⁻ᵏ, and how to find mean and standard deviation using np and √(np(1 - p)).
Central Limit Theorem
Unpack the Central Limit Theorem and see why sample means form a normal distribution as n grows — even when the population isn't normal. Master key mechanics like standard error (σ/√n), the n ≥ 30 rule, and applying z-scores to sample mean problems.
Confidence Level and Margin of Error
Unpack the mechanics of confidence intervals and margin of error, from how critical values (z* and t*) are chosen to how sample size affects interval width. Master the distinction between z- and t-distributions and what confidence level actually means across repeated samples.
Data Visualization and Distribution Shapes
Visualize how raw data takes shape through histograms, dot plots, and box plots while mastering symmetric, skewed, and uniform distributions — and learn why skewness shifts the mean toward the tail but leaves the median largely unaffected.
Experimental Design and Bias
Unpack the core principles of experimental design — from random assignment and confounding variables to single- and double-blind procedures, placebo controls, and bias types — so you can confidently distinguish true experiments from flawed ones.
Hypothesis Testing Logic
Unpack the logic behind hypothesis testing, from null and alternative hypotheses to p-values, significance levels, and Type I and II errors. This pack clarifies how test statistics and tail direction shape your reject-or-fail-to-reject decision.
Measures of the Center of the Data
Master the three measures of center — mean, median, and mode — and learn how each responds to outliers, skewness, and weighted values. Understand when to use the median over the mean for skewed data like income, and how symmetric vs. skewed distributions shift these measures apart.
Measures of Variability
Unpack the core measures of spread — range, standard deviation, variance, and IQR — and learn when to use each based on data shape and outliers. Covers Chebyshev's theorem and the n − 1 correction for unbiased sample estimates.
Percentiles and Z-Scores
Unpack the relationship between percentiles, z-scores, and the standard normal distribution — covering the z = (x − μ) / σ formula, IQR-based outlier detection, and how standardized scores translate raw data into comparable percentile ranks.