---
title: "Learn ML Algorithms from Scratch"
url: "https://ridwaanhall.com/projects/learn-ml-algorithm-from-scratch"
markdown: "https://ridwaanhall.com/projects/learn-ml-algorithm-from-scratch.md"
summary: "The building blocks of machine learning written in plain NumPy, so each one can be read and tested on its own."
---

# Learn ML Algorithms from Scratch

> The building blocks of machine learning written in plain NumPy, so each one can be read and tested on its own.

- Category: Machine Learning
- Status: Completed
- Started: June 23, 2025
- Built with: Python, NumPy
- Source: https://github.com/ridwaanhall/Learn-ML-Algorithm-from-Scratch

## About

### The problem

Libraries make it easy to train a model without seeing what an optimizer, a loss or an activation actually computes. That leaves beginners unsure which piece to change when a model does not learn.

### The approach

I wrote each component in NumPy alone, as small modules with one job each. Linear regression is built from them end to end, with worked examples.

- Gradient descent, Momentum, RMSProp and Adam optimizers.
- MSE, MAE, Huber and cross-entropy losses, and sigmoid, ReLU, tanh and softmax activations.
- Accuracy, precision, recall, F1 and R² metrics, with scalers, encoders and a train/test splitter.

### The result

An open-source study project with a test suite for every module, which pairs with Neural Network from Scratch and LSTM Gold from Scratch.

## Features

- **Optimizers**: Gradient descent, Momentum, RMSProp and Adam.
- **Losses and Activations**: Four of each, written from their formulas.
- **Metrics and Preprocessing**: Classification and regression metrics, scaling and splitting.
- **Tested**: A unit test for every module.

## Tags

from scratch, Machine Learning, numpy, Education, Python

## Elsewhere

- This page for people: https://ridwaanhall.com/projects/learn-ml-algorithm-from-scratch
- Index of every page as Markdown: [llms.txt](https://ridwaanhall.com/llms.txt)