Implementing a Linear Regression from Scratch - MSE & Gradient Descent - 2025

Tutorial ~ GDSimpson3

Resources

Dataset: https://raw.githubusercontent.com/GDSimpson3/ML-datasets/refs/heads/main/Salary_dataset.csv

Notebook: https://github.com/GDSimpson3/LinearRegression-From-Scratch/blob/main/LinReg002.ipynb

Board

Video (Youtube) (click to visit)

URL (Youtube): https://www.youtube.com/watch?v=y5PBAsUJsTo

Content Covered

• Introduction
• Getting our Dataset
• Visualise the Data
• Regression Line Concept
• Loss Functions
• Residual
• Mean Squared Error
• The Gradient Descent on a Simple Quadratic
• The Gradient Descent on our Bi Variate Function
• Partial Derivatives Explanation
• Differentiating a Summation
• Partial Derivatives of our MSE
• Gradient Descent with Partial Derivatives
• Implement in Code
• Drawing a line
• Implementing the MSE
• Partial Derivatives implementation
• Gradient Descent Implementation
• Visualise the MSE
• Pitfalls of the Gradient Descent
• Extra

Tools used

• Google Colab
• Microsoft Whiteboard
• OBS