Before transformers, before backprop, there was one neuron — Frank Rosenblatt's 1958 Perceptron. Build it and you understand the atom that every deep network is made of.
This is Day 1 of DeepLearningFromZero: neural nets built from a single neuron up, no framework magic.
Take inputs, multiply each by a weight, add a bias, then apply an activation. The original used a step: output +1 if the sum is ≥ 0, else −1.
let w = [Math.random(), Math.random()], b = 0;
const sum = x => w[0]*x[0] + w[1]*x[1] + b;
const predict = x => sum(x) >= 0 ? 1 : -1;
w₁·x₁ + w₂·x₂ + b = 0
is the equation of a straight line — the decision boundary. One side is class +1, the other is −1. So a neuron's entire "knowledge" is the tilt and position of one line.
Predict each point. If correct, do nothing. If wrong, nudge the weights toward the right answer:
for (const { x, y } of data) {
if (predict(x) !== y) { // y is the true label, +1 or -1
w[0] += lr * y * x[0];
w[1] += lr * y * x[1];
b += lr * y;
}
}
Geometrically, that rotates and shifts the line so the misclassified point ends up on the correct side. Repeat for a few epochs:
for (let epoch = 0; epoch < 50; epoch++) trainStep(0.1);
Rosenblatt proved it: if the two classes can be separated by a straight line, the perceptron will find one in a finite number of steps. Watch the accuracy climb to 100% and training stop.
A single neuron can only draw one straight line. The famous failure is XOR — no single line separates it. That limitation nearly killed neural nets in the 1970s.
The fix: stack neurons into layers, swap the step for a smooth activation, and train with gradient descent + backprop. That arc — from this one neuron to a transformer — is the rest of the series.
🌐 Train the neuron live (watch the boundary rotate): https://dev48v.infy.uk/dl/day1-perceptron.html
Day 1 of DeepLearningFromZero. From one neuron to transformers, built from scratch.