{"slug": "show-hn-i-implemented-a-neural-network-in-sql", "title": "Show HN: I implemented a neural network in SQL", "summary": "A developer implemented a neural network in SQL using the xarray_sql library, demonstrating that machine learning models can be trained and executed entirely within a database environment. The project, shared on Hacker News, uses Fashion-MNIST data and achieves a speedup by skipping zero-valued pixels during matrix operations.", "body_md": "-\n[Notifications](/login?return_to=%2Fxqlsystems%2Fxarray-sql)You must be signed in to change notification settings -\n[Fork 16](/login?return_to=%2Fxqlsystems%2Fxarray-sql)\n\n## Expand file tree\n\n/\n\nCopy path# nn.py\n\nMore file actions\n\n488 lines (446 loc) · 18.8 KB\n\n/\n\nCopy path# nn.py\n\n## File metadata and controls\n\n488 lines (446 loc) · 18.8 KB\n\n1\n\n2\n\n3\n\n4\n\n5\n\n6\n\n7\n\n8\n\n9\n\n10\n\n11\n\n12\n\n13\n\n14\n\n15\n\n16\n\n17\n\n18\n\n19\n\n20\n\n21\n\n22\n\n23\n\n24\n\n25\n\n26\n\n27\n\n28\n\n29\n\n30\n\n31\n\n32\n\n33\n\n34\n\n35\n\n36\n\n37\n\n38\n\n39\n\n40\n\n41\n\n42\n\n43\n\n44\n\n45\n\n46\n\n47\n\n48\n\n49\n\n50\n\n51\n\n52\n\n53\n\n54\n\n55\n\n56\n\n57\n\n58\n\n59\n\n60\n\n61\n\n62\n\n63\n\n64\n\n65\n\n66\n\n67\n\n68\n\n69\n\n70\n\n71\n\n72\n\n73\n\n74\n\n75\n\n76\n\n77\n\n78\n\n79\n\n80\n\n81\n\n82\n\n83\n\n84\n\n85\n\n86\n\n87\n\n88\n\n89\n\n90\n\n91\n\n92\n\n93\n\n94\n\n95\n\n96\n\n97\n\n98\n\n99\n\n100\n\n101\n\n102\n\n103\n\n104\n\n105\n\n106\n\n107\n\n108\n\n109\n\n110\n\n111\n\n112\n\n113\n\n114\n\n115\n\n116\n\n117\n\n118\n\n119\n\n120\n\n121\n\n122\n\n123\n\n124\n\n125\n\n126\n\n127\n\n128\n\n129\n\n130\n\n131\n\n132\n\n133\n\n134\n\n135\n\n136\n\n137\n\n138\n\n139\n\n140\n\n141\n\n142\n\n143\n\n144\n\n145\n\n146\n\n147\n\n148\n\n149\n\n150\n\n151\n\n152\n\n153\n\n154\n\n155\n\n156\n\n157\n\n158\n\n159\n\n160\n\n161\n\n162\n\n163\n\n164\n\n165\n\n166\n\n167\n\n168\n\n169\n\n170\n\n171\n\n172\n\n173\n\n174\n\n175\n\n176\n\n177\n\n178\n\n179\n\n180\n\n181\n\n182\n\n183\n\n184\n\n185\n\n186\n\n187\n\n188\n\n189\n\n190\n\n191\n\n192\n\n193\n\n194\n\n195\n\n196\n\n197\n\n198\n\n199\n\n200\n\n201\n\n202\n\n203\n\n204\n\n205\n\n206\n\n207\n\n208\n\n209\n\n210\n\n211\n\n212\n\n213\n\n214\n\n215\n\n216\n\n217\n\n218\n\n219\n\n220\n\n221\n\n222\n\n223\n\n224\n\n225\n\n226\n\n227\n\n228\n\n229\n\n230\n\n231\n\n232\n\n233\n\n234\n\n235\n\n236\n\n237\n\n238\n\n239\n\n240\n\n241\n\n242\n\n243\n\n244\n\n245\n\n246\n\n247\n\n248\n\n249\n\n250\n\n251\n\n252\n\n253\n\n254\n\n255\n\n256\n\n257\n\n258\n\n259\n\n260\n\n261\n\n262\n\n263\n\n264\n\n265\n\n266\n\n267\n\n268\n\n269\n\n270\n\n271\n\n272\n\n273\n\n274\n\n275\n\n276\n\n277\n\n278\n\n279\n\n280\n\n281\n\n282\n\n283\n\n284\n\n285\n\n286\n\n287\n\n288\n\n289\n\n290\n\n291\n\n292\n\n293\n\n294\n\n295\n\n296\n\n297\n\n298\n\n299\n\n300\n\n301\n\n302\n\n303\n\n304\n\n305\n\n306\n\n307\n\n308\n\n309\n\n310\n\n311\n\n312\n\n313\n\n314\n\n315\n\n316\n\n317\n\n318\n\n319\n\n320\n\n321\n\n322\n\n323\n\n324\n\n325\n\n326\n\n327\n\n328\n\n329\n\n330\n\n331\n\n332\n\n333\n\n334\n\n335\n\n336\n\n337\n\n338\n\n339\n\n340\n\n341\n\n342\n\n343\n\n344\n\n345\n\n346\n\n347\n\n348\n\n349\n\n350\n\n351\n\n352\n\n353\n\n354\n\n355\n\n356\n\n357\n\n358\n\n359\n\n360\n\n361\n\n362\n\n363\n\n364\n\n365\n\n366\n\n367\n\n368\n\n369\n\n370\n\n371\n\n372\n\n373\n\n374\n\n375\n\n376\n\n377\n\n378\n\n379\n\n380\n\n381\n\n382\n\n383\n\n384\n\n385\n\n386\n\n387\n\n388\n\n389\n\n390\n\n391\n\n392\n\n393\n\n394\n\n395\n\n396\n\n397\n\n398\n\n399\n\n400\n\n401\n\n402\n\n403\n\n404\n\n405\n\n406\n\n407\n\n408\n\n409\n\n410\n\n411\n\n412\n\n413\n\n414\n\n415\n\n416\n\n417\n\n418\n\n419\n\n420\n\n421\n\n422\n\n423\n\n424\n\n425\n\n426\n\n427\n\n428\n\n429\n\n430\n\n431\n\n432\n\n433\n\n434\n\n435\n\n436\n\n437\n\n438\n\n439\n\n440\n\n441\n\n442\n\n443\n\n444\n\n445\n\n446\n\n447\n\n448\n\n449\n\n450\n\n451\n\n452\n\n453\n\n454\n\n455\n\n456\n\n457\n\n458\n\n459\n\n460\n\n461\n\n462\n\n463\n\n464\n\n465\n\n466\n\n467\n\n468\n\n469\n\n470\n\n471\n\n472\n\n473\n\n474\n\n475\n\n476\n\n477\n\n478\n\n479\n\n480\n\n481\n\n482\n\n483\n\n484\n\n485\n\n486\n\n487\n\n488\n\n# /// script\n\n# requires-python = \">=3.12\"\n\n# dependencies = [\n\n# \"xarray_sql\",\n\n# \"xarray\",\n\n# \"numpy\",\n\n# \"s3fs\",\n\n# \"zarr<3\",\n\n# ]\n\n#\n\n# [tool.uv.sources]\n\n# xarray_sql = { path = \"..\", editable = true }\n\n# ///\n\nfrom __future__ import annotations\n\nfrom typing import Callable\n\nimport numpy as np\n\nimport xarray as xr\n\nimport datetime\n\nimport xarray_sql as xql\n\nSIDE = 28 # images are 28x28; flatten index is height * SIDE + width\n\nWIDTHS = (\n\nSIDE * SIDE,\n\n196,\n\n32,\n\n10,\n\n) # 784 pixels -> 196 -> 32 tanh -> 10 softmax\n\nN_SAMPLES, TRAIN_FRAC = 700, 0.7 # total samples; fraction used for training\n\nLR, STEPS, CHUNK = 0.5, 60, 250\n\n# Drop zero-valued pixels from the (dominant) layer-0 contraction. A background\n\n# pixel contributes 0 * weight = 0, so skipping those rows shrinks the join\n\n# *exactly* — the result is identical, and the speedup scales with the fraction\n\n# of zeros (a dark background). On dense inputs it is a no-op.\n\n#\n\n# Measured ~1.8x on real Fashion-MNIST (~50% zero pixels): 2.56 -> 1.45 s/step.\n\nSKIP_ZERO_PIXELS = True\n\ndef fashion_mnist():\n\n\"\"\"The whole training set, left lazy so SQL streams and samples it.\n\nThe real path returns a dask-backed (chunked) Dataset — nothing is pulled\n\ninto memory here; ``from_dataset`` reads it chunk by chunk on demand, and\n\nthe random subsample happens later in SQL. The offline fallback is a small\n\nsynthetic set built in memory.\n\n\"\"\"\n\ntry:\n\nds = xr.open_dataset(\n\n\"s3://carbonplan-share/xbatcher/fashion-mnist-train.zarr\",\n\nengine=\"zarr\",\n\nchunks=None,\n\nbackend_kwargs={\"storage_options\": {\"anon\": True}},\n\n)\n\nif \"channel\" in ds.dims:\n\nds = ds.isel(channel=0, drop=True)\n\n# To float64, lazily (no full read). This zarr already stores images\n\n# as float in [0, 1]; only integer-encoded sources ([0, 255]) rescale.\n\nimages = ds[\"images\"].astype(\"float64\")\n\nif not np.issubdtype(ds[\"images\"].dtype, np.floating):\n\nimages = images / 255.0\n\nds = ds.assign(images=images, labels=ds[\"labels\"].astype(\"int64\"))\n\nexcept Exception:\n\n# Offline fallback: a separable synthetic set (per-class template +\n\n# noise), so the same pipeline still learns without the network. A pool\n\n# larger than N_SAMPLES so the SQL subsample still has something to pick.\n\nrng = np.random.default_rng(0)\n\nn = 3 * N_SAMPLES\n\ntemplates = rng.standard_normal((10, SIDE, SIDE))\n\nlabels = rng.integers(0, 10, n).astype(\"int64\")\n\nimages = templates[labels] + 0.6 * rng.standard_normal((n, SIDE, SIDE))\n\nds = xr.Dataset(\n\n{\n\n\"images\": ((\"sample\", \"height\", \"width\"), images),\n\n\"labels\": ((\"sample\",), labels),\n\n}\n\n)\n\n# Integer index coords are the SQL join keys (sample, height, width).\n\nreturn ds[[\"images\", \"labels\"]].assign_coords(\n\nsample=np.arange(ds.sizes[\"sample\"]),\n\nheight=np.arange(ds.sizes[\"height\"]),\n\nwidth=np.arange(ds.sizes[\"width\"]),\n\n)\n\ndef build_model_with_table_names(\n\ninit_weight: Callable[[int, int], np.ndarray],\n\ninit_bias: Callable[[int], np.ndarray],\n\nwidths=WIDTHS,\n\n) -> tuple[xr.Dataset, dict[tuple[str, ...], str]]:\n\n\"\"\"The network as one Dataset that splits into tables per layer.\n\nLayer ``i`` is a weight matrix `` layer_i (inp_i, out_i)`` and a separate\n\nbias vector ``bias_i (out_i,)``.\n\n\"\"\"\n\nweights = {\n\nf\"layer_{i}\": ((f\"inp_{i}\", f\"out_{i}\"), init_weight(inp, out))\n\nfor i, (inp, out) in enumerate(zip(widths[:-1], widths[1:]))\n\n}\n\nbiases = {\n\nf\"bias_{i}\": ((f\"out_{i}\",), init_bias(out))\n\nfor i, out in enumerate(widths[1:])\n\n}\n\ncoords = {}\n\ncoords.update(\n\n{f\"inp_{i}\": np.arange(inp) for i, inp in enumerate(widths[:-1])}\n\n)\n\ncoords.update(\n\n{f\"out_{i}\": np.arange(out) for i, out in enumerate(widths[1:])}\n\n)\n\nds = xr.Dataset({**weights, **biases}, coords=coords)\n\nnames: dict[tuple[str, ...], str] = {}\n\nfor i in range(len(weights)):\n\nnames[(f\"inp_{i}\", f\"out_{i}\")] = f\"layer{i}\"\n\nnames[(f\"out_{i}\",)] = f\"bias{i}\"\n\nreturn ds, names\n\ndef main():\n\nrng = np.random.default_rng(1)\n\nmnist = fashion_mnist()\n\nctx = xql.XarrayContext()\n\n# One Dataset splits into two tables: pixels (sample, height, width) and\n\n# labels (sample). The dim names are the join keys.\n\nctx.from_dataset(\n\n\"mnist\",\n\nmnist,\n\nchunks=dict(sample=CHUNK),\n\ntable_names={\n\n(\"sample\", \"height\", \"width\"): \"pixels\",\n\n(\"sample\",): \"labels\",\n\n},\n\n)\n\n# Draw a random N_SAMPLES subset in SQL (ORDER BY random() LIMIT), carrying\n\n# each sample's label and a train/test tag. `data` is the working label\n\n# table: cache() pins the chosen subset so every downstream query sees the\n\n# same split without rescanning the source. `ORDER BY random()` shuffles the\n\n# whole label column, so the subset is order-independent even if the on-disk\n\n# samples are class-sorted.\n\ndata = ctx.sql(f\"\"\"\n\nSELECT sample, labels,\n\nCASE WHEN random() < {TRAIN_FRAC} THEN 'train' ELSE 'test' END AS split\n\nFROM mnist.labels\n\nORDER BY random()\n\nLIMIT {N_SAMPLES}\n\n\"\"\").cache()\n\nctx.register_table(\"data\", data)\n\n# Materialise just the sampled images once: a single lazy scan of the full\n\n# dataset extracts the ~N_SAMPLES subset into `pixels`, which the per-step\n\n# forward joins instead of rescanning the source 60x. Only the subset lives\n\n# in memory; the full set stays lazy.\n\npixels = ctx.sql(\"\"\"\n\nSELECT p.sample, p.height, p.width, p.images\n\nFROM mnist.pixels p JOIN data d ON p.sample = d.sample\n\n\"\"\").cache()\n\nctx.register_table(\"pixels\", pixels)\n\n# The gradient averages over the actual train count (random, ~frac * N),\n\n# read once from the materialized split.\n\nn_train = ctx.sql(\n\n\"SELECT COUNT(*) AS n FROM data WHERE split = 'train'\"\n\n).to_pandas()[\"n\"][0]\n\ndef init_weight(inp: int, out: int):\n\n\"\"\"Small random weights.\"\"\"\n\nreturn rng.standard_normal((inp, out)) * 0.1\n\ndef init_bias(out: int):\n\n\"\"\"Biases start at zero.\"\"\"\n\nreturn np.zeros(out)\n\nmodel, table_names = build_model_with_table_names(init_weight, init_bias)\n\nctx.from_dataset(\n\n\"model\",\n\nmodel,\n\ntable_names=table_names,\n\n# Each layer table is one chunk: weights on (inp_i, out_i) and the bias\n\n# vector on (out_i,), so every dim needs a size here.\n\nchunks={\n\n**{\n\nf\"inp_{i}\": model.sizes[f\"inp_{i}\"]\n\nfor i in range(len(WIDTHS) - 1)\n\n},\n\n**{\n\nf\"out_{i}\": model.sizes[f\"out_{i}\"]\n\nfor i in range(len(WIDTHS) - 1)\n\n},\n\n},\n\n)\n\n# Unify the per-layer weight tables into one working weight(layer, inp, out,\n\n# val) relation the loop rewrites in place, tagging each layer with its\n\n# index.\n\nseed = \" UNION ALL \".join(\n\nf\"SELECT {i} AS layer, inp_{i} AS inp, out_{i} AS out, layer_{i} AS val \"\n\nf\"FROM model.layer{i}\"\n\nfor i in range(len(WIDTHS) - 1)\n\n)\n\nctx.register_table(\"weight\", ctx.sql(seed).cache())\n\n# The biases live in their own bias(layer, out, val) relation, summed into\n\n# each layer's pre-activation as a separate term (z = W @ a + b).\n\nbias_seed = \" UNION ALL \".join(\n\nf\"SELECT {i} AS layer, out_{i} AS out, bias_{i} AS val FROM model.bias{i}\"\n\nfor i in range(len(WIDTHS) - 1)\n\n)\n\nctx.register_table(\"bias\", ctx.sql(bias_seed).cache())\n\n# The zero-pixel skip. fwd0 has no WHERE (it forwards all samples), so it\n\n# needs a fresh `WHERE`; g0 already filters to the train split, so it\n\n# appends an `AND`. Empty strings when the flag is off.\n\nzero_where = \"WHERE images <> 0\" if SKIP_ZERO_PIXELS else \"\"\n\nzero_and = \"AND images <> 0\" if SKIP_ZERO_PIXELS else \"\"\n\nfor step in range(STEPS):\n\n#\n\n# --- forward pass -----------------------------------------------------\n\n#\n\n# Each layer contracts its activation with the weight table (JOIN on the\n\n# shared index + grouped SUM), then adds the layer's bias as a separate\n\n# term (JOIN the bias table on `out`), and keeps the pre-activation z\n\n# (tanh(z) for hidden, linear output). .cache() materialises each stage\n\n# so the per-step plan stays flat.\n\n#\n\n# The forward runs over ALL samples: train rows drive learning, test\n\n# rows ride along so we can score them from the same logits. Only delta2\n\n# is restricted to train, so the gradients (and the trained weights) are\n\n# identical to a train-only forward — test is never backpropagated.\n\nfwd0 = ctx.sql(f\"\"\"\n\nWITH c AS (\n\n-- z = x @ W: matmul of the input and first weight matrix\n\nSELECT a.sample, w.out AS out, SUM(a.val * w.val) AS z\n\nFROM (\n\nSELECT sample, height * {SIDE} + width AS inp, images AS val\n\nFROM pixels\n\n{zero_where}\n\n) a\n\nJOIN weight w ON a.inp = w.inp AND w.layer = 0\n\nGROUP BY a.sample, w.out\n\n)\n\n-- activation(z + b): Add in the bias term, then perform activation\n\nSELECT c.sample, c.out AS out, c.z + b.val AS z,\n\ntanh(c.z + b.val) AS val\n\nFROM c JOIN bias b ON c.out = b.out AND b.layer = 0\n\n\"\"\").cache()\n\nctx.deregister_table(\"fwd0\")\n\nctx.register_table(\"fwd0\", fwd0)\n\nfwd1 = ctx.sql(f\"\"\"\n\nWITH c AS (\n\nSELECT a.sample, w.out AS out, SUM(a.val * w.val) AS z\n\nFROM (SELECT sample, out AS inp, val FROM fwd0) a\n\nJOIN weight w ON a.inp = w.inp AND w.layer = 1\n\nGROUP BY a.sample, w.out\n\n)\n\nSELECT c.sample, c.out AS out, c.z + b.val AS z,\n\ntanh(c.z + b.val) AS val\n\nFROM c JOIN bias b ON c.out = b.out AND b.layer = 1\n\n\"\"\").cache()\n\nctx.deregister_table(\"fwd1\")\n\nctx.register_table(\"fwd1\", fwd1)\n\n# Output layer is linear (softmax lives in the loss / output error),\n\n# but still gets its bias summed in.\n\nlogits = ctx.sql(f\"\"\"\n\nWITH c AS (\n\nSELECT a.sample, w.out AS out, SUM(a.val * w.val) AS z\n\nFROM (SELECT sample, out AS inp, val FROM fwd1) a\n\nJOIN weight w ON a.inp = w.inp AND w.layer = 2\n\nGROUP BY a.sample, w.out\n\n)\n\nSELECT c.sample, c.out AS out, c.z + b.val AS z\n\nFROM c JOIN bias b ON c.out = b.out AND b.layer = 2\n\n\"\"\").cache()\n\nctx.deregister_table(\"logits\")\n\nctx.register_table(\"logits\", logits)\n\n#\n\n# --- backward pass ----------------------------------------------------\n\n#\n\n# Output error delta2 = softmax(logits) - onehot(label). The one\n\n# hand-derived rule: softmax couples classes through a per-sample\n\n# normaliser.\n\ndelta2 = ctx.sql(f\"\"\"\n\nWITH m AS (SELECT sample, MAX(z) AS m FROM logits GROUP BY sample),\n\ne AS (SELECT logits.sample, logits.out, exp(logits.z - m.m) AS e\n\nFROM logits JOIN m ON logits.sample = m.sample),\n\ns AS (SELECT sample, SUM(e) AS s FROM e GROUP BY sample)\n\nSELECT e.sample, e.out,\n\ne.e / s.s - CASE WHEN e.out = y.labels THEN 1.0 ELSE 0.0 END AS val\n\nFROM e JOIN s ON e.sample = s.sample\n\nJOIN data y ON y.sample = e.sample\n\n-- restrict the error to train, so every downstream gradient is train-only\n\nWHERE e.sample IN (SELECT sample FROM data WHERE split = 'train')\n\n\"\"\").cache()\n\nctx.deregister_table(\"delta2\")\n\nctx.register_table(\"delta2\", delta2)\n\n# Weight gradient of layer 2: fwd1.T @ delta2 / N.\n\ng2 = ctx.sql(f\"\"\"\n\nSELECT a.inp AS inp, d.out AS out, SUM(a.val * d.val) / {n_train} AS val\n\nFROM (SELECT sample, out AS inp, val FROM fwd1) a\n\nJOIN delta2 d ON a.sample = d.sample\n\nGROUP BY a.inp, d.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"g2\")\n\nctx.register_table(\"g2\", g2)\n\n# Bias gradient of layer 2: the mean output error per unit.\n\ngb2 = ctx.sql(f\"\"\"\n\nSELECT out, SUM(val) / {n_train} AS val FROM delta2 GROUP BY out\n\n\"\"\").cache()\n\nctx.deregister_table(\"gb2\")\n\nctx.register_table(\"gb2\", gb2)\n\n# Propagate to layer 1: delta1 = (delta2 @ W2.T) * tanh'(z1). The local\n\n# derivative is grad(tanh(z), z) at fwd1's pre-activation.\n\ndelta1 = ctx.sql(f\"\"\"\n\nWITH dc AS (\n\nSELECT d.sample, w.inp AS out, SUM(d.val * w.val) AS val\n\nFROM delta2 d JOIN weight w ON d.out = w.out AND w.layer = 2\n\nGROUP BY d.sample, w.inp\n\n)\n\nSELECT dc.sample, dc.out,\n\ndc.val * grad(tanh(fwd1.z), fwd1.z) AS val\n\nFROM dc JOIN fwd1 ON dc.sample = fwd1.sample AND dc.out = fwd1.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"delta1\")\n\nctx.register_table(\"delta1\", delta1)\n\ng1 = ctx.sql(f\"\"\"\n\nSELECT a.inp AS inp, d.out AS out, SUM(a.val * d.val) / {n_train} AS val\n\nFROM (SELECT sample, out AS inp, val FROM fwd0) a\n\nJOIN delta1 d ON a.sample = d.sample\n\nGROUP BY a.inp, d.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"g1\")\n\nctx.register_table(\"g1\", g1)\n\ngb1 = ctx.sql(f\"\"\"\n\nSELECT out, SUM(val) / {n_train} AS val FROM delta1 GROUP BY out\n\n\"\"\").cache()\n\nctx.deregister_table(\"gb1\")\n\nctx.register_table(\"gb1\", gb1)\n\n# Propagate to layer 0: delta0 = (delta1 @ W1.T) * tanh'(z0).\n\ndelta0 = ctx.sql(f\"\"\"\n\nWITH dc AS (\n\nSELECT d.sample, w.inp AS out, SUM(d.val * w.val) AS val\n\nFROM delta1 d JOIN weight w ON d.out = w.out AND w.layer = 1\n\nGROUP BY d.sample, w.inp\n\n)\n\nSELECT dc.sample, dc.out,\n\ndc.val * grad(tanh(fwd0.z), fwd0.z) AS val\n\nFROM dc JOIN fwd0 ON dc.sample = fwd0.sample AND dc.out = fwd0.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"delta0\")\n\nctx.register_table(\"delta0\", delta0)\n\ng0 = ctx.sql(f\"\"\"\n\nWITH a AS (\n\nSELECT sample, height * {SIDE} + width AS inp, images AS val\n\nFROM pixels\n\nWHERE sample IN (SELECT sample FROM data WHERE split = 'train')\n\n{zero_and}\n\n)\n\nSELECT a.inp AS inp, d.out AS out, SUM(a.val * d.val) / {n_train} AS val\n\nFROM a JOIN delta0 d ON a.sample = d.sample\n\nGROUP BY a.inp, d.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"g0\")\n\nctx.register_table(\"g0\", g0)\n\ngb0 = ctx.sql(f\"\"\"\n\nSELECT out, SUM(val) / {n_train} AS val FROM delta0 GROUP BY out\n\n\"\"\").cache()\n\nctx.deregister_table(\"gb0\")\n\nctx.register_table(\"gb0\", gb0)\n\n#\n\n# --- SGD update: one query per relation -------------------------------\n\n#\n\n# weight <- weight - lr * gradient and bias <- bias - lr * gradient,\n\n# joining every layer at once against the per-layer gradients tagged\n\n# with their layer index.\n\nw = ctx.sql(f\"\"\"\n\nWITH grad AS (\n\nSELECT 0 AS layer, inp, out, val FROM g0\n\nUNION ALL SELECT 1 AS layer, inp, out, val FROM g1\n\nUNION ALL SELECT 2 AS layer, inp, out, val FROM g2\n\n)\n\nSELECT w.layer, w.inp, w.out,\n\nw.val - {LR} * COALESCE(g.val, 0) AS val\n\nFROM weight w LEFT JOIN grad g\n\nON w.layer = g.layer AND w.inp = g.inp AND w.out = g.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"weight\")\n\nctx.register_table(\"weight\", w)\n\nb = ctx.sql(f\"\"\"\n\nWITH gb AS (\n\nSELECT 0 AS layer, out, val FROM gb0\n\nUNION ALL SELECT 1 AS layer, out, val FROM gb1\n\nUNION ALL SELECT 2 AS layer, out, val FROM gb2\n\n)\n\nSELECT b.layer, b.out,\n\nb.val - {LR} * COALESCE(g.val, 0) AS val\n\nFROM bias b LEFT JOIN gb g\n\nON b.layer = g.layer AND b.out = g.out\n\n\"\"\").cache()\n\nctx.deregister_table(\"bias\")\n\nctx.register_table(\"bias\", b)\n\nif step % 5 == 0 or step == STEPS - 1:\n\n# Train cross-entropy (logits span all samples, so filter to train).\n\nloss = ctx.sql(f\"\"\"\n\nWITH m AS (SELECT sample, MAX(z) AS m FROM logits GROUP BY sample),\n\ne AS (SELECT logits.sample, logits.out, exp(logits.z - m.m) AS e\n\nFROM logits JOIN m ON logits.sample = m.sample),\n\ns AS (SELECT sample, SUM(e) AS s FROM e GROUP BY sample)\n\nSELECT -AVG(ln(e.e / s.s)) AS loss\n\nFROM e JOIN s ON e.sample = s.sample\n\nJOIN data y ON y.sample = e.sample\n\nWHERE e.out = y.labels\n\nAND e.sample IN (SELECT sample FROM data WHERE split = 'train')\n\n\"\"\").to_pandas()[\"loss\"][0]\n\n# Accuracy per split: argmax the shared logits, join the split label.\n\n# Both come from the one all-samples forward — no second pass.\n\nacc = (\n\nctx.sql(f\"\"\"\n\nWITH pred AS (\n\nSELECT sample, out,\n\nROW_NUMBER() OVER (PARTITION BY sample ORDER BY z DESC) AS rk\n\nFROM logits)\n\nSELECT d.split,\n\nAVG(CASE WHEN p.out = d.labels THEN 1.0 ELSE 0.0 END) AS acc\n\nFROM pred p JOIN data d ON d.sample = p.sample\n\nWHERE p.rk = 1\n\nGROUP BY d.split\n\n\"\"\")\n\n.to_pandas()\n\n.set_index(\"split\")[\"acc\"]\n\n)\n\nprint(\n\nf\"step {step:2d}: loss {loss:.3f} \"\n\nf\"train_acc {acc['train']:.3f} test_acc {acc['test']:.3f}\"\n\n)\n\n# The trained parameters come back out as xarray in the *same shape as the\n\n# input model*: one weight variable per layer with its own (inp_i, out_i)\n\n# dims, plus one bias variable per layer on (out_i,). Each is read from its\n\n# relation by the `layer` column, so the result is a ragged set of per-layer\n\n# matrices and vectors — no dense array padded with NaN.\n\ntrained = xr.Dataset(\n\n{\n\n**{\n\nf\"layer_{i}\": ctx.sql(\n\nf\"SELECT inp AS inp_{i}, out AS out_{i}, val AS layer_{i} \"\n\nf\"FROM weight WHERE layer = {i}\"\n\n).to_dataset(dims=[f\"inp_{i}\", f\"out_{i}\"])[f\"layer_{i}\"]\n\nfor i in range(len(WIDTHS) - 1)\n\n},\n\n**{\n\nf\"bias_{i}\": ctx.sql(\n\nf\"SELECT out AS out_{i}, val AS bias_{i} \"\n\nf\"FROM bias WHERE layer = {i}\"\n\n).to_dataset(dims=[f\"out_{i}\"])[f\"bias_{i}\"]\n\nfor i in range(len(WIDTHS) - 1)\n\n},\n\n}\n\n)\n\nprint(f\"trained {WIDTHS} MLP; weights -> xarray {dict(trained.sizes)}.\")\n\nprint(trained)\n\ntrained.to_zarr(\n\nf\"fashion_mnist_mlp_\"\n\nf\"{datetime.datetime.now().isoformat(timespec='seconds')}.zarr\"\n\n)\n\nif __name__ == \"__main__\":\n\nmain()", "url": "https://wpnews.pro/news/show-hn-i-implemented-a-neural-network-in-sql", "canonical_source": "https://github.com/xqlsystems/xarray-sql/blob/claude/xarray-sql-mnist-demo/benchmarks/nn.py", "published_at": "2026-07-13 20:00:10+00:00", "updated_at": "2026-07-13 20:47:59.689209+00:00", "lang": "en", "topics": ["neural-networks", "machine-learning", "developer-tools"], "entities": ["xarray_sql", "Fashion-MNIST", "Hacker News"], "alternates": {"html": "https://wpnews.pro/news/show-hn-i-implemented-a-neural-network-in-sql", "markdown": "https://wpnews.pro/news/show-hn-i-implemented-a-neural-network-in-sql.md", "text": "https://wpnews.pro/news/show-hn-i-implemented-a-neural-network-in-sql.txt", "jsonld": "https://wpnews.pro/news/show-hn-i-implemented-a-neural-network-in-sql.jsonld"}}