AI Fundamentals: What is a Neuron?

An artificial neuron is a simple math function with four components: inputs, weights, a bias, and an activation function. Weights are initialized to small random numbers to break symmetry, and the bias shifts the neuron's output. Activation functions introduce non-linearity, enabling multi-layer networks to learn complex patterns beyond linear relationships.

An artificial neural network is built by combining simple computational units called neurons. Each neuron takes in some numbers, performs a few calculations, and outputs a single number. That’s it. So where does the intelligence come from? Not from any one neuron. It comes from connecting thousands, or even millions, of these small building blocks together. As information flows through the network, each neuron contributes one small piece to the final prediction. In this article, we’ll break down exactly how a neuron works, the math behind it, and then build it from scratch. A neuron is a small math function with four moving parts: Inputs x₁, x₂, x3, ..., xₙ carry numerical values into the neuron. A neuron never sees raw text, pixels, or audio directly; it only ever sees numbers, organized as a vector. Before anything reaches the first layer of a network, the raw input has to be converted into that numerical form, and exactly how depends on the type of data: From there, every layer after the first builds on what came before it: each neuron’s inputs are simply the outputs produced by the neurons in the previous layer. Weights w₁, w₂, ..., wₙ determine how much influence each input has, one weight per input. They're initialized to small random numbers rather than a fixed value like zero. This matters due to a problem called symmetry breaking : if every neuron in a layer started with identical weights, they'd all compute the exact same output from the exact same inputs, and during training they'd all get adjusted in exactly the same way, leaving every neuron in that layer stuck learning the same thing forever. Randomizing the starting weights gives each neuron a different starting point, so as training progresses, gradient descent, for example, can push each one to specialize and pick up on different patterns in the data. The neuron sums the weighted inputs and adds a bias b , producing a single number z. Summation multiplies each input xᵢ by its weight wᵢ , then add all of those products together into one number. That's the z = w₁x₁ + w₂x₂ + ... + wₙxₙpart of the formula below. The bias is then added on top as one extra constant term. Tip:Think of the bias as an extension of the weights. It behaves like a weight attached to an input that’s always fixed at 1, so its value never depends on the actual data coming in. Without it, the neuron's weighted sum would pass through zero if every input is zero, which needlessly limits what the neuron can represent. The bias gives the neuron the freedom to shift its output up or down, similar to how b shifts the line up or down in y = mx + b. The result of the previous steps, z, could be any number, from -1,000 to 1,000. The activation function f takes this intermediate result and reshapes it into the neuron's actual output, f z , usually squeezing it into a smaller, more useful range. Its real job is introducing non-linearity : without it, stacking layers would be pointless, because a chain of purely linear operations collapses down into just one bigger linear operation, no matter how many layers you stack. Non-linear activation functions are what let a multi-layer network learn curved, complex patterns instead of only straight lines and flat planes. There are several common choices, including: Before jumping into the full neuron equation, it helps to start somewhere familiar: the equation for a straight line. y = mx + b A single artificial neuron uses this exact idea, but instead of one x and one m, a neuron typically receives several inputs, each with its own weight. The same idea extended into higher dimensions: instead of drawing a line, the neuron is defining a flat plane or hyperplane that separates or weighs its inputs. In this higher-dimensional picture, the weights still control the tilt of that plane, and the bias still slides it up or down, just like m and b do for a simple line. The output from the neuron represents how strongly the neuron “activated” based on the inputs it received. The activation function, f, produces that final output: output = f x = f w₁x₁ + w₂x₂ + ... + wₙxₙ + b The activation function introduces non-linearity, which is what allows networks of neurons to learn complex patterns rather than just straight lines and flat planes. A simple choice is the sigmoid function, which squashes any input into a value between 0 and 1: f z = 1 / 1 + e⁻ᶻ Let’s look at a simplified version of how a language model actually picks the next word. Every word in its vocabulary has a fixed coordinate in a massive, multi-dimensional space, its embedding. Words with related meanings tend to cluster near each other: “apple,” “banana,” and “fruit” sit close together in one region, while “car,” “engine,” and “highway” cluster somewhere else entirely. Because deep networks commonly use ReLU as their activation function, and ReLU zeroes out every negative value, a large share of the network’s neurons stay inactive for any given input; only the neurons relevant to that input end up contributing to the result. At the model’s output layer, there’s roughly one neuron per word in the vocabulary, and each one calculates a raw score for how well its word matches what the model currently expects to come next. If the model’s current internal representation lands close to the “fruit” cluster, the neuron for “banana” produces a high raw score for example, 9.4 , while the neuron for “car” produces a low, even negative, score for example, -4.2 , since “car” has nothing to do with the current context. These raw scores, often called logits , aren’t probabilities on their own, since they can be any real number. A function called softmax converts the entire set of scores across the model’s full vocabulary, often 100,000 or more words into probabilities that add up to 1. Softmax pushes the highest score close to 1 for example, 94% while pushing low scores close to 0%. Rather than picking the single highest-probability word, which tends to produce repetitive, robotic text, many language models sample from this probability distribution instead. A word with a 94% probability will almost always get picked, but a word sitting at 5% still has a small chance of being chosen, which is part of what gives generated text some natural variation. Below is a minimal Python implementation of a single artificial neuron, using the weighted-sum-plus-bias formula and a sigmoid activation function. python import math Note: this example skips training entirely. There's no dataset here, and no code to learn from. The weights and bias below are just hardcoded, standing in for values a real network would learn by training on data.class Neuron: def init self, weights, bias : """ weights: list of floats, one weight per input bias: float, the neuron's bias term """ self.weights = weights self.bias = bias def sigmoid self, z : """Activation function: squashes z into a range between 0 and 1.""" return 1 / 1 + math.exp -z def forward self, inputs : """ Computes the neuron's output for a given set of inputs. inputs: list of floats, same length as self.weights """ Weighted sum: w1 x1 + w2 x2 + ... + wn xn + b z = sum w x for w, x in zip self.weights, inputs + self.bias Apply the activation function return self.sigmoid z Example usageif name == " main ": A neuron with 3 inputs. In a real network, these weights and the bias would be learned from a training dataset for example, billions of sentences , rather than set by hand like this. neuron = Neuron weights= 0.5, -0.6, 0.1 , bias=0.2 For this example, treat these three numbers as a simplified stand-in for an embedding representing one input. inputs = 1.0, 0.5, -1.5 output = neuron.forward inputs print f"Neuron output: {output:.4f}" Running this code passes the inputs 1.0, 0.5, -1.5 through the neuron's weighted sum and sigmoid activation, printing a single output value between 0 and 1, the neuron's decision. Because every neuron starts with slightly different random weights, they naturally drift toward capturing different patterns as training progresses. One neuron’s line might end up separating verbs from nouns. Another’s might end up separating happy words from sad ones. By the time training is finished, the network has a massive grid of billions of intersecting lines, carving the embedding space into precise conceptual territories: the foundation that everything else in a neural network, from recognizing images to generating language, is built on. AI Fundamentals: What is a Neuron? https://pub.towardsai.net/ai-fundamentals-what-is-a-neuron-62e76a6f3500 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.