A neural net in pure SQL is a teaching tool

Priya Nair reports on a neural network implemented entirely in SQL using xarray-sql, demonstrating that matrix operations like multiplication and backpropagation can be expressed as joins and aggregations. The Fashion-MNIST trainer, while not production-ready, serves as a teaching tool that reveals the relational algebra underlying deep learning frameworks. The implementation achieves a 1.8× speedup on sparse inputs by filtering zero pixels before joins.

AI https://sourcefeed.dev/c/ai Article A neural net in pure SQL is a teaching tool Backprop as joins and group-bys shows what matrix math really is under the hood. Priya Nair https://sourcefeed.dev/u/priya nair Every few years someone reimplements a neural network in SQL. The latest is a Fashion-MNIST https://github.com/zalandoresearch/fashion-mnist trainer built on xarray-sql https://github.com/xqlsystems/xarray-sql : weights and biases live as tables, forward and backward passes are queries, and training is a loop of set operations. It is not a production stack. It is one of the clearer maps you will find from textbook backprop to the relational model. The interesting claim is not "SQL can do deep learning." It is that once you force every matmul, activation, and gradient step into joins and aggregations, the algebra stops being abstract. That is useful for anyone who has only ever called loss.backward and wants to see what the framework is hiding. Layers as tables, matmul as a join The network is a three-layer MLP with widths 784, 196, 32, 10 : flattened 28×28 pixels, a 196-unit hidden layer, a 32-unit tanh layer, then a 10-way softmax. In the demo each weight matrix is a 2D array with dimensions like inp i, out i , and each bias is a 1D vector on out i . Those arrays become named tables layer0 , bias0 , … . Matrix multiplication then looks like the classic relational rewrite. You join activations to weights on the contracting dimension, multiply, and GROUP BY the free dimensions with SUM . Older MySQL demos did the same thing with explicit CROSS JOIN … ON column index = row index and a SUM weight cell value . The xarray-sql version is the same idea with better coordinate handling and lazy chunked inputs, but the relational skeleton has not changed since people were hand-rolling this in SQL Server for weather forecasts and bike-buyer scores. That continuity is the point. The math was always join-and-aggregate. Frameworks paper over it with BLAS kernels; SQL makes the paper thin. Backprop without a tape Training uses a fixed learning rate of 0.5 , 60 steps, and a subsample of 700 examples 70% train . Gradients are not free. You re-express the usual chain rule as more joins: error at the output, weighted contributions back through each layer, outer products for weight updates, then an UPDATE or equivalent rewrite of the parameter tables. Historically this is where pure-SQL nets got ugly. A 2016 MySQL prototype split training into product, nonlinearity, loss, gradient, and update steps, each as a separate query, on a toy dataset with three parameters. A 2009 SQL Server design modeled the whole graph as an ERD and iterated until error hit ~5%. Neither was flexible or fast. The modern demo is cleaner because the array-to-table mapping is systematic and the engine can stream chunked zarr from object storage, but the algorithmic shape is the same: no autograd tape, just explicit reverse-mode algebra written by hand. If you have ever debugged a vanishing gradient by printing intermediate tensors, this is that discipline with worse ergonomics and better forced clarity. Zero pixels are free rows you can drop Fashion-MNIST images are roughly half zeros dark background . A zero times a weight is zero, so those input rows contribute nothing to the first layer. The demo sets SKIP ZERO PIXELS = True and filters them out of the layer-0 contraction. On real Fashion-MNIST that cut step time from about 2.56s to 1.45s ~1.8× . On dense inputs it is a no-op. In a dense framework you might rely on sparse kernels or accept the multiply. In SQL the optimization is just a predicate before the join. The join shrinks exactly; the result is identical. That is a nice illustration of when relational operators win: sparsity that is natural in the data model becomes a free filter, not a special-case kernel. Chunk size 250 and the lazy open of the zarr store matter for the same reason. Nothing materializes the full training set in memory up front; the SQL engine samples and streams. That is closer to how analytical warehouses already work than to a single-GPU training loop. What this is actually good for Nobody should train production models this way. Sixty steps on 700 samples is a smoke test, not a run. You will not beat PyTorch, JAX, or even a decent DuckDB UDF calling a small ONNX runtime. Latency, optimizer variety, mixed precision, and distributed data parallel are not the point. The practical uses are narrower and real: Pedagogy. If you teach or interview people on backprop, forcing them to write the weight update as a join is ruthless and effective. They cannot hide behind nn.Linear . Engine stress tests. Dense matmul and reduction patterns are a hard, regular workload for SQL engines that claim strong analytical performance. The skip-zero trick is a good check that predicate pushdown and join reordering still work under load. In-database inference adjacent work. Shipping trained weights into a warehouse and scoring with SQL or a UDF is common. Understanding the forward pass as joins makes those scoring queries less mysterious. Training in pure SQL is the extreme version of the same idea. Historical continuity. Microsoft’s Analysis Services Neural Network, DMX, and various "ANN in T-SQL" articles solved a different problem black-box mining models next to the data . The pure-query approach is the opposite: full transparency, no mining server, terrible throughput. Knowing both ends of that spectrum helps when someone asks "can we keep the model in the database?" If you want to try it, the path is: open the Fashion-MNIST zarr or the synthetic offline fallback , build the weight/bias dataset with the table-name map, run the training loop with SKIP ZERO PIXELS on, and inspect intermediate tables after a few steps. Watch which joins dominate. Compare a step with and without the zero filter. That is the whole exercise. Not a new backend, a better whiteboard SQL neural nets keep coming back because the mapping is honest. A weight matrix is a relation. A matmul is a join plus a sum. Backprop is more of the same with error signals. The xarray-sql Fashion-MNIST trainer is a polished instance of that mapping, with modern lazy I/O and a sparsity trick that actually measures well on real images. Treat it as a whiteboard you can execute, not as a competitor to PyTorch https://pytorch.org or as a reason to move training into your warehouse. The developers who get value from it are the ones who need to see the gradients as data, or who need to know exactly how far set-based engines can be pushed before you should stop. For everyone else, keep training in a framework and score where the data lives. The SQL version has already done its job if it made the algebra legible. Sources & further reading - Show HN: I implemented a neural network in SQL https://github.com/xqlsystems/xarray-sql/blob/claude/xarray-sql-mnist-demo/benchmarks/nn.py — github.com - Neural Networks in MySQL - Yurii Shevchuk https://blog.itdxer.com/2016/07/01/neural-networs-in-mysql.html — blog.itdxer.com - A Neural Network in SQL Server – SQLServerCentral https://www.sqlservercentral.com/articles/a-neural-network-in-sql-server — sqlservercentral.com - Implement Artificial Neural Networks ANNs in SQL Server https://www.sqlshack.com/implement-artificial-neural-networks-anns-in-sql-server/ — sqlshack.com Priya Nair https://sourcefeed.dev/u/priya nair · AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. Discussion 0 No comments yet Be the first to weigh in.