{"slug": "neural-network-basics-for-graphics-people", "title": "Neural network basics, for graphics people", "summary": "Ben Houston published a reference guide explaining neural-network fundamentals for graphics developers, covering MLPs, weights, biases, ReLU, forward pass, loss, gradient descent, backpropagation, Adam, latent vectors, feature grids, and decoder networks. The guide is intended to support related posts on neural textures, materials, and appearance, and underpins his neural-texture compression blog post for three.js.", "body_md": "# Neural network basics, for graphics people\n\nA short reference on the neural-network vocabulary used across the neural texture, neural material and neural appearance posts — MLP, weights and biases, ReLU, forward pass, loss, gradient descent, backpropagation, Adam, latent vectors, feature grids and decoder networks — explained once, concretely, so those posts can link here instead of re-deriving it.\n\n[Ben Houston](/about) • • 11 min read\n\nThis is a reference, not a story. It defines the neural-network vocabulary —\nMLPs, training, gradient descent, Adam, latent vectors, feature grids and\ndecoder networks — once, carefully, and in plain terms. Nothing here is\nspecific to graphics. If you already know what a multilayer perceptron, Adam\nand a latent vector are, you don't need this page. (It also underpins the [neural-texture compression blog post](/blog/neural-texture-compression-in-threejs).)\n\n## A model is a function with dials on it[#](#a-model-is-a-function-with-dials-on-it)\n\nA **model**, in this context, is nothing more than a\n[function](https://en.wikipedia.org/wiki/Function_(mathematics)) with\nadjustable numbers built into it: give it an input, it produces an output,\nand the exact output for a given input depends on the current values of those\nnumbers. The numbers are the\n[ parameters](https://en.wikipedia.org/wiki/Parameter) (also called\n\n**weights** when they multiply something, and\n\n**biases** when they just add a constant).\n\n**Training** is the process of adjusting the parameters so the function's output matches some reference you already have. Nothing here requires the function to be a\n\n[\"neural network\"](https://en.wikipedia.org/wiki/Neural_network_(machine_learning))specifically — the same words describe fitting a line, a\n\n[polynomial](https://en.wikipedia.org/wiki/Polynomial), or a GGX lobe to measured data. A neural network is just one particular shape of function, chosen because it can represent things a line or a polynomial can't.\n\n## The multilayer perceptron (MLP)[#](#the-multilayer-perceptron-mlp)\n\nAn ** MLP** is a stack\nof layers, where each layer does the same two things to its input: a\nweighted sum, then a fixed nonlinear clamp.\n\nis a matrix of weights, is a vector of biases, is the input vector, and is a fixed, non-learned function applied to every element of the result. Stack a few of these and you have an MLP:\n\nThree layers. `x`\n\nis the **input**, `y`\n\nis the **output**, `h1`\n\nand `h2`\n\nare\n**hidden layers** — \"hidden\" meaning neither the input nor the output, just\nintermediate values nobody outside the function ever looks at directly. The\nsize of a hidden layer (how many numbers `h1`\n\nholds) is called its **width**,\nchosen before training and fixed afterward, because it determines the shape of\n`W0`\n\nand `W1`\n\n. Running this function once, input to output, is a **forward\npass**.\n\nWidth and depth (number of layers) are two of the network's\n**hyperparameters**, covered more fully below.\n\nThat's the entire architecture. An MLP is not more mysterious than this: a short, fixed sequence of matrix-multiply-then-clamp steps.\n\n### ReLU, and why the clamp matters[#](#relu-and-why-the-clamp-matters)\n\nis the most common\n[ activation function](https://en.wikipedia.org/wiki/Activation_function):\nidentity for positive inputs, zero for negative ones. It's called the\n\n[, hence ReLU.](https://en.wikipedia.org/wiki/Rectifier_(neural_networks))\n\n**rectified linear unit** The clamp isn't decoration. Delete it and , , which is — a single matrix, no matter how many layers you stack. Every \"deep\" network without a nonlinearity between its layers collapses algebraically into one linear transform, and a linear function can't represent anything with a bend or a peak in it. ReLU is the cheapest nonlinearity that avoids that collapse, which is why it's the default choice unless something specific rules it out.\n\nEach ReLU unit behaves like a hinge: it contributes nothing until its input crosses zero, then grows linearly. A layer of many hinges is a piecewise-linear function with that many possible folds, and stacking layers composes those folds into shapes a single layer couldn't reach. More width, more folds, finer approximation — at the cost of more parameters to fit and more work per evaluation.\n\n## Loss, gradient descent, and training as fitting[#](#loss-gradient-descent-and-training-as-fitting)\n\nYou've done this before, informally: fit a polynomial through some points,\nproject a function onto\n[spherical harmonics](https://en.wikipedia.org/wiki/Spherical_harmonics), tune\na GGX roughness parameter until a render looks right. All of those are the\nsame activity — a reference you trust, an approximation with free parameters,\nand an adjustment process that reduces the gap between them.\n\nConcretely, for an MLP:\n\n- Run a forward pass on some inputs you have reference outputs for.\n- Compare the network's output to the reference. That comparison, reduced to\na single number, is the\n— a plain measure of how wrong the current parameters are. Squared error is the simplest common choice: , where is the network's output and is the reference value.**loss** - Compute how the loss would change if each parameter moved a little, in\nwhich direction. That's the\n, , of the loss with respect to the parameters (the collected weights and biases).**gradient** - Move every parameter a small step opposite its gradient — downhill on the\nloss: . The step\nsize is the\n.**learning rate** - Repeat, on a new batch of inputs each time.\n\nOne repetition of that loop is a training **step** or **iteration**. A group\nof inputs processed together in one step is a **batch**, and its size is the\n**batch size**. Unlike fitting a polynomial, there's no closed-form solution\nfor an MLP's parameters — you can't just solve for them directly — so training\nnudges iteratively instead. This iterative process is\n[ gradient descent](https://en.wikipedia.org/wiki/Gradient_descent).\n\n### Backpropagation, briefly[#](#backpropagation-briefly)\n\nStep 3 above — computing the gradient — is done with the\n[chain rule](https://en.wikipedia.org/wiki/Chain_rule), applied from the\noutput backward through the layers. That algorithm has a name,\n[ backpropagation](https://en.wikipedia.org/wiki/Backpropagation), but the\nmechanics are ordinary calculus: each layer\nreceives an error signal from the layer after it, uses that signal to compute\nits own weights' gradients, and passes a modified error signal further back.\nTwo consequences of this matter later. First, a ReLU's derivative is either 1\nor 0 (it's linear where it's active, flat where it's clamped), so gradient\neither passes straight through a unit or stops there completely — a unit whose\noutput is always negative can go permanently silent. Second, computing the\nbackward pass requires the values the forward pass produced at every layer, so\nrunning backpropagation costs memory proportional to how many inputs you're\nprocessing at once, not just to how many parameters the network has.\n\n### Adam[#](#adam)\n\nPlain gradient descent takes the same size step for every parameter, which\nworks poorly when some parameters consistently have large gradients and others\nhave small ones.\n[ Adam](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam) is\nan optimizer that fixes this by keeping two\nrunning averages per parameter: the mean of recent gradients (so it keeps\nmoving through flat regions instead of stalling) and the mean of recent\n\n*squared*gradients (so it can shrink the step for parameters whose gradient has been consistently large, and grow it for ones that have been small):\n\nDividing the first by the square root of the second gives each parameter its own effective learning rate without hand-tuning one per parameter ( is just a small constant that keeps the division from blowing up when is near zero). The practical cost: Adam needs two extra buffers the same size as your parameter set, one for each running average — a real memory cost when the parameter set includes a multi-megabyte grid of trained values, not just an MLP's weights.\n\n## Latent vectors, feature grids, and decoders[#](#latent-vectors-feature-grids-and-decoders)\n\nThe three graphics posts this page supports all use one more pattern beyond a plain MLP, worth naming precisely because \"latent\" gets used loosely elsewhere.\n\nA [ latent vector](https://en.wikipedia.org/wiki/Latent_variable) (or\n\n**latent code**) is a set of numbers that started as free parameters — usually small random values — and were given meaning purely by training, rather than by any predefined encoding a person wrote down. They aren't RGB, they aren't roughness, they aren't anything you could name before training started; whatever structure they end up representing is whatever the optimizer found useful for minimizing the loss. This is different from an ordinary texture channel, where a person decided in advance that this number means roughness.\n\nA **feature grid** is a 2D (or 3D) grid of latent vectors — one small latent\nvector stored at every grid cell, addressed by a coordinate the same way a\ntexture is addressed by UV. Query it at an arbitrary coordinate and you\nbilinearly interpolate the latent vectors at the surrounding cells, exactly\nlike sampling a texture, except what you get back isn't a color — it's a\nlatent vector that some other function still has to interpret. A feature grid\nis itself a set of trained parameters, not a fixed input; it's optimized\nalongside everything else during training.\n\nA [ decoder network](https://en.wikipedia.org/wiki/Autoencoder) is the MLP\nthat turns a latent vector (or several,\nconcatenated) into something with an actual meaning again — RGB, a normal, a\nBRDF response, whatever the task calls for. The pattern across all three\nposts is: coordinate in, feature grid lookup, MLP decode, meaningful output\nout. The grid supplies the \"what's stored at this location,\" the decoder\nsupplies the \"how do these stored numbers become a real answer,\" and both are\ntrained together so neither one has to be designed by hand.\n\n### Trained parameters vs. everything else[#](#trained-parameters-vs-everything-else)\n\nWorth being precise about one distinction that recurs: a **trained parameter**\nis a number training is allowed to change — every weight and bias in the\ndecoder, every latent value in the grid. A\n[ hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning))\nis a number a person chooses before training starts and training never\ntouches — layer\nwidth, learning rate, batch size, grid resolution, number of training steps.\nGetting this backward is a common source of confusion: changing a\nhyperparameter (like grid resolution) changes how many trained parameters\nexist, but the hyperparameter itself is never something gradient descent\nadjusts.\n\n### Precision: fp32 during training, fp16 often on export[#](#precision-fp32-during-training-fp16-often-on-export)\n\nOne practical detail shows up in all three posts and is worth defining once.\nTraining typically keeps every parameter in\n[32-bit floating point](https://en.wikipedia.org/wiki/Single-precision_floating-point_format)\n(fp32), because the small updates gradient descent applies at each step need\nthat much precision to accumulate correctly over thousands of iterations.\nOnce training is done, the trained values are often exported at\n[half precision](https://en.wikipedia.org/wiki/Half-precision_floating-point_format)\n(fp16) — enough precision to reconstruct the result well, at half the storage\nand bandwidth cost, and (in a WebGPU/WebGL context specifically) because\n16-bit float textures are guaranteed hardware-filterable while 32-bit float\ntextures generally aren't without an explicit device feature. That's a\nstatement about what these systems actually do, not a universal law — whether\nfp16 is \"enough\" depends on how sensitive the specific output is to rounding,\nand each post that makes this trade explains why it holds for its own case.\n\n## A short glossary[#](#a-short-glossary)\n\n| Term | Meaning |\n|---|---|\n| Model | A function with adjustable parameters |\n| Parameter / weight / bias | A number training is allowed to change |\n| Hyperparameter | A number chosen before training and left fixed |\n| MLP | A short stack of weighted-sum-then-clamp layers |\n| Hidden layer | A layer that's neither the input nor the output |\n| Activation function | The fixed nonlinearity applied after each layer's weighted sum |\n| ReLU | , the most common activation function |\n| Forward pass | Running the network once, input to output |\n| Loss | A single number measuring how wrong the current parameters are |\n| Gradient | How the loss would change if each parameter moved slightly |\n| Gradient descent | Repeatedly stepping parameters opposite their gradient |\n| Learning rate | The size of each gradient-descent step |\n| Batch / batch size | A group of inputs processed together in one training step |\n| Backpropagation | The chain-rule algorithm that computes gradients layer by layer |\n| Adam | An optimizer that gives every parameter its own adaptive step size |\n| Latent vector / latent code | Free parameters whose meaning is defined entirely by training |\n| Feature grid | A grid of latent vectors, addressed and interpolated like a texture |\n| Decoder network | The MLP that turns a latent vector into a meaningful output |\n\nKeep these in hand and the rest of the series should read as engineering, not mathematics you have to take on faith.", "url": "https://wpnews.pro/news/neural-network-basics-for-graphics-people", "canonical_source": "https://ben3d.ca/blog/neural-network-basics", "published_at": "2026-08-18 00:00:00+00:00", "updated_at": "2026-08-28 21:48:40.130874+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence"], "entities": ["Ben Houston", "three.js"], "alternates": {"html": "https://wpnews.pro/news/neural-network-basics-for-graphics-people", "markdown": "https://wpnews.pro/news/neural-network-basics-for-graphics-people.md", "text": "https://wpnews.pro/news/neural-network-basics-for-graphics-people.txt", "jsonld": "https://wpnews.pro/news/neural-network-basics-for-graphics-people.jsonld"}}