Simple Linear Regression Model Study Pack

Kibin's free study pack on Simple Linear Regression Model includes a 3-section study guide, 8 quiz questions, 10 flashcards, and 1 open-ended Explain review question. Sign up free to track your progress toward mastery, plus upload your own notes and recordings to create personalized study packs organized by course.

Last updated May 21, 2026

Topic mastery0%
Save progress →

Simple Linear Regression Model Study Guide

Build a solid foundation in simple linear regression by mastering the equation ŷ = a + bx, the least squares method, and how to interpret slope and y-intercept. This pack covers residual analysis, correlation, r², hypothesis testing on the slope, and the risks of extrapolation — everything you need to apply and evaluate linear regression models with confidence.

Key Takeaways

  • •Simple linear regression models the relationship between one explanatory variable (x) and one response variable (y) using the equation ŷ = a + bx, where a is the y-intercept and b is the slope.
  • •The slope b quantifies how much ŷ changes for each one-unit increase in x, while the y-intercept a gives the predicted value of y when x equals zero.
  • •The least squares method determines the best-fit line by minimizing the sum of squared residuals — the vertical distances between observed data points and the fitted line.
  • •The correlation coefficient r measures the strength and direction of the linear relationship, and r² (the coefficient of determination) represents the proportion of variability in y explained by x.
  • •Residuals (observed y minus predicted ŷ) must be examined to verify that regression assumptions — linearity, constant variance, independence, and normality of errors — are reasonably satisfied.
  • •A hypothesis test on the slope (H₀: β = 0) determines whether x has a statistically significant linear relationship with y in the population, using a t-statistic with n − 2 degrees of freedom.
  • •Predictions made within the range of observed x-values (interpolation) are considered reliable, while predictions outside that range (extrapolation) carry substantial uncertainty and should be made cautiously.

Purpose and Structure of the Simple Linear Regression Model

Simple linear regression provides a mathematical framework for describing how one quantitative variable changes in relation to another, and for using that relationship to make predictions.

Response and Explanatory Variables

  • •The explanatory variable (also called the predictor or independent variable, denoted x) is the variable used to explain or predict outcomes.
  • •The response variable (also called the dependent variable, denoted y) is the outcome being predicted or explained.
  • •Regression does not require x to cause y — it only models the association — but researchers typically choose x because they believe it has predictive or causal relevance.

The Population Regression Model

  • •The theoretical population model is written as y = α + βx + ε, where α (alpha) is the true y-intercept, β (beta) is the true slope, and ε (epsilon) is the random error term.
  • •Because α and β are unknown population parameters, they are estimated from sample data as a (the estimated intercept) and b (the estimated slope).
  • •The random error term ε accounts for the natural scatter of individual observations around the true line — no real dataset falls perfectly on a straight line.

The Fitted Regression Equation

  • •The sample regression equation is written ŷ = a + bx, where ŷ (y-hat) is the predicted value of y for a given x.
  • •This equation represents the best estimate of the true population line based on available data.

Estimating the Regression Line: Least Squares Method

The least squares method is the standard technique for calculating the values of a and b that produce the best-fitting line through a set of data points.

Defining Residuals

  • •A residual for any data point is the difference between its observed y-value and the value predicted by the regression line: e = y − ŷ.
  • •Positive residuals indicate points above the line; negative residuals indicate points below the line.
  • •The sum of all residuals always equals zero for a least squares line, meaning the line passes through the center of the data.

The Least Squares Criterion

  • •The least squares method selects the values of a and b that minimize the sum of squared residuals (SSE = Σ(y − ŷ)²), giving greater penalty to large deviations.
  • •Squaring the residuals prevents positive and negative errors from canceling each other out and magnifies the influence of extreme deviations.

Formulas for Slope and Intercept

  • •The slope is calculated as b = r × (sy / sx), where r is the correlation coefficient between x and y, sy is the standard deviation of y, and sx is the standard deviation of x.
  • •The y-intercept is calculated as a = ȳ − b × x̄, which ensures the regression line always passes through the point (x̄, ȳ) — the means of both variables.
  • •Because a depends on b, the slope must be computed first.

About this Study Pack

Created by Kibin to help students review key concepts, prepare for exams, and study more effectively. This Study Pack was checked for accuracy and curriculum alignment using authoritative educational sources. See sources below.

Sources

More in Statistics

See all topics →

Browse other courses

See all courses →