In this article
Linear regression answers one question: if two things move together, what straight line describes that movement best? The line has the form y = m × x + b. The slope m says how much y changes for each step in x, and the intercept b says where the line starts when x is zero.
The example: hours studied and test score
Eight students recorded how many hours they studied and what they scored out of 100. Plotting them shows a clear upward drift, but no single straight line touches every point. Regression finds the best compromise.
| Hours (x) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| Score (y) | 52 | 55 | 61 | 60 | 68 | 72 | 75 | 83 |
Average hours = 4.5, average score = 65.75. The best-fit line always passes through this point of averages.
The two lines stay close together. The biggest miss is at 4 hours, where the student scored 3.6 marks below the line.
Building the line in four steps
- 1Find the averagesx̄ = 4.5 hours, ȳ = 65.75 marks.
- 2Measure the pullAdd up (x − x̄)(y − ȳ) for every point: 179. Add up (x − x̄)²: 42.
- 3Slopem = 179 ÷ 42 = 4.2619. Each extra hour adds about 4.26 marks.
- 4Interceptb = ȳ − m × x̄ = 65.75 − 4.2619 × 4.5 = 46.57.
These are the standard least squares formulas. The calculator does the same arithmetic for any number of points.
Reading slope and intercept
The intercept is a mathematical starting point, not always a real-life fact. Nobody studies for zero hours and scores exactly 46.57; it only anchors the line.
Residuals: how far each point is from the line
A residual is the actual value minus the predicted one. Positive means the student did better than the line expected; negative means worse. Least squares makes the residuals add up to zero and keeps their squares as small as possible.
| Hours | Actual | Predicted | Residual |
|---|---|---|---|
| 1 | 52 | 50.8 | +1.2 |
| 2 | 55 | 55.1 | −0.1 |
| 3 | 61 | 59.4 | +1.6 |
| 4 | 60 | 63.6 | −3.6 |
| 5 | 68 | 67.9 | +0.1 |
| 6 | 72 | 72.1 | −0.1 |
| 7 | 75 | 76.4 | −1.4 |
| 8 | 83 | 80.7 | +2.3 |
Squared residuals add up to about 24.6, against about 787.5 for the total variation around the average score.
R²: how much does the line explain?
R² compares two amounts: the leftover error around the line (24.6) and the total spread of scores around their average (787.5). R² = 1 − 24.6 ÷ 787.5 = 0.969. About 97% of the differences between students' scores line up with study hours in this tiny sample.
- Explained by the line96.9%
- Left over (residuals)3.1%
A high R² does not prove that studying causes higher scores. Motivation, sleep or prior knowledge could drive both.
These bands are a rough guide. What counts as good depends on the field: 0.5 can be excellent for human behaviour and poor for a physics experiment.
Making predictions
Put a new x into the equation. For 9 hours the line says 4.2619 × 9 + 46.57 = 84.9. That is just past our data (the maximum was 8), so it is a mild stretch. For 20 hours the line gives 131.8 marks, which is impossible on a test out of 100.
- 6 hours gives about 72 marks
- The line was built from points around it
- Still carries an error of a few marks
- The line predicts 131.8 out of 100
- Real relationships bend and flatten
- Nothing in the data supports it
A straight line is an approximation over the range you measured. Always ask whether the relationship could plausibly continue.
One strange point can tilt the whole line
Regression gives every point a vote, and points far from the rest have the loudest voice. Change only the last student's score from 83 to 40 (say, they fell ill) and the slope collapses from 4.26 to about 0.68 marks per hour. Always plot your data before trusting the line.
One altered point out of eight cut the slope by 84%. Check unusual points: are they errors, or real cases the line should explain?
Checklist before you trust a regression
- Plot first. If the points curve, a straight line is the wrong shape.
- Look for outliers and decide whether they are errors or real.
- Keep predictions inside the data range wherever possible.
- Use enough points. Eight points are fine for learning, but too few for serious conclusions.
- Do not claim cause from fit alone.
Questions people ask
What is linear regression?
It is a method that finds the straight line y = mx + b that best fits a set of points, so you can describe the relationship between two variables and predict new values.
What is the least squares method?
It chooses the line that minimises the sum of the squared vertical gaps between the points and the line. Squaring stops positive and negative gaps from cancelling and punishes large misses.
What does R-squared mean?
R² is the share of the variation in y that the line explains. 0 means the line explains nothing and 1 means it explains everything. In the example, 0.97 means about 97 percent.
What is a residual?
A residual is the actual value minus the value the line predicts. Small, random-looking residuals suggest a good fit; patterns in them suggest a straight line is the wrong model.
Can I predict values outside my data?
You can, but be careful. Relationships often bend or flatten outside the measured range, so predictions far beyond your data can be badly wrong.
Is regression the same as correlation?
They are related. Correlation r measures how tightly points follow a line, and for one variable R² equals r squared. Regression goes further and gives the actual equation of the line.