{"slug": "pytorch-written-in-c", "title": "PyTorch Written in C", "summary": "Developer thevoxium released bare-lm, a minimal autograd tensor library written in C, installable via a git clone and make build or prebuilt Linux and macOS tarballs. The library provides a Memory object with 256 MB permanent and temporary arenas, a ParameterList for trainable tensors, and operations including linear_t, relu_t, sigmoid_t, mseloss_t, backward, zero_grad, and sgd_step, as shown in a 500-epoch XOR training example. bare-lm requires a C compiler, make, and OpenBLAS, and installs to /usr/local/include/bare.h and /usr/local/lib/libbare.{so,dylib}.", "body_md": "A minimal autograd tensor library in C.\n\nYou can install `bare-lm` in two ways.\n\nRequirements:\n\n- A C compiler (`cc` ,`gcc` , or`clang` )\n- `make`\n- OpenBLAS (headers + library)\n\nInstall OpenBLAS first:\n\n```\n# macOS (Homebrew)\nbrew install openblas\n\n# Ubuntu / Debian\nsudo apt update && sudo apt install -y libopenblas-dev\n```\n\nThen build and install:\n\n```\ngit clone https://github.com/thevoxium/bare-lm.git\ncd bare-lm\n\n# macOS\nmake OPENBLAS_PREFIX=/opt/homebrew/opt/openblas\n\n# Linux\nmake\n\nsudo make install\n```\n\nBy default this installs to `/usr/local/include/bare.h` and `/usr/local/lib/libbare.{so,dylib}`.\n\nEach tagged release publishes prebuilt archives for Linux and macOS:\n\n- `bare-<version>-linux.tar.gz`\n- `bare-<version>-macos.tar.gz`\n\nAfter downloading and extracting a release tarball, install headers and library:\n\n```\ntar -xzf bare-<version>-<artifact>.tar.gz\ncd bare-<version>-<artifact>\nsudo cp include/bare.h /usr/local/include/\nsudo cp lib/libbare.* /usr/local/lib/\ncc file.c -lbare\n```\n\nIf your system does not search `/usr/local/lib` by default, add `-L/usr/local/lib` and `-I/usr/local/include`.\n\n```\n#include \"bare.h\"\n\nint main() {\n  Memory *mem = create_global_mem(1 << 28);\n  ParameterList *pl = create_param_list(mem);\n\n  int x_shape[] = {4, 2};\n  int y_shape[] = {4, 1};\n\n  Tensor *x = tensor_init(mem, x_shape, 2, PERM);\n  Tensor *y = tensor_init(mem, y_shape, 2, PERM);\n\n  float x_data[] = {0, 0, 0, 1, 1, 0, 1, 1};\n  float y_data[] = {0, 1, 1, 0};\n  for (int i = 0; i < 8; i++) x->data[i] = x_data[i];\n  for (int i = 0; i < 4; i++) y->data[i] = y_data[i];\n\n  Linear *l1 = create_linear(mem, pl, 2, 8);\n  Linear *l2 = create_linear(mem, pl, 8, 1);\n\n  load_checkpoint(pl, \"xor.btw\");\n\n  for (int epoch = 0; epoch < 500; epoch++) {\n    zero_grad(pl);\n\n    Tensor *h = linear_t(mem, l1, x);\n    h = relu_t(mem, h);\n    Tensor *o = linear_t(mem, l2, h);\n    o = sigmoid_t(mem, o);\n\n    Tensor *loss = mseloss_t(mem, o, y);\n    backward(mem, loss);\n\n    if (epoch % 100 == 0)\n      printf(\"epoch %3d  loss=%.4f\\n\", epoch, loss->data[0]);\n\n    sgd_step(pl, 0.1f);\n    reset_temp_mem(mem);\n  }\n\n  save_checkpoint(pl, \"xor.btw\");\n\n  free_global_mem(mem);\n}\n```\n\n**Step 1: Create memory and a parameter list.** A single `Memory` object holds two arenas (permanent and temporary). `1 << 28` = 256 MB per arena. A `ParameterList` tracks all trainable tensors.\n\n```\nMemory *mem = create_global_mem(1 << 28);\nParameterList *pl = create_param_list(mem);\n```\n\n**Step 2: Allocate persistent tensors.** Inputs `x` and `y` are plain PERM tensors (not trainable). Pass `PERM` so they survive `reset_temp_mem`.\n\n```\nTensor *x = tensor_init(mem, x_shape, 2, PERM);\nTensor *y = tensor_init(mem, y_shape, 2, PERM);\n```\n\n**Step 3: Create layers with auto-registration.** `create_linear` allocates weights and bias as PERM tensors and automatically adds them to the parameter list.\n\n```\nLinear *l1 = create_linear(mem, pl, 2, 8);\nLinear *l2 = create_linear(mem, pl, 8, 1);\n// pl now contains l1->weights, l1->bias, l2->weights, l2->bias\n```\n\n**Step 4: Forward pass.** Every operation (`linear_t`, `relu_t`, `sigmoid_t`, `mseloss_t`) allocates its result from the temp arena. No `malloc` calls, no cleanup code.\n\n```\nTensor *h = linear_t(mem, l1, x);\nh = relu_t(mem, h);\nTensor *o = linear_t(mem, l2, h);\no = sigmoid_t(mem, o);\nTensor *loss = mseloss_t(mem, o, y);\n```\n\n**Step 5: Backward pass.** `backward` builds a topological sort of the computation graph (also temp-allocated) and propagates gradients. Intermediate tensor data is still valid at this point.\n\n```\nbackward(mem, loss);\n```\n\n**Step 6: Zero grads, update, reset.** `zero_grad` clears all parameter gradients. `sgd_step` applies SGD to all parameters in one call. After gradients are consumed, `reset_temp_mem` zeroes the temp arena in O(1). All intermediate tensors from the forward pass are gone.\n\n```\nzero_grad(pl);\nsgd_step(pl, 0.1f);\nreset_temp_mem(mem);\n```\n\n**Step 7: Cleanup.** `free_global_mem` releases both arenas and the `Memory` struct.\n\n```\nfree_global_mem(mem);\n```\n\nAll allocations go through `allocate_mem(mem, size, perm)`. There are no individual `free` calls.\n\n| Flag | Lifetime | Used for | \n|---|---|---|\n| `PERM` | Until `free_global_mem` | Weights, biases, persistent inputs/targets | \n| `TEMP` | Until `reset_temp_mem` | Operation results, graph traversal arrays, loss | \n\n`reset_temp_mem` resets the temp arena pointer to 0. It is called **after** both forward and backward complete, because backward reads intermediate tensor data.\n\n```\nepoch N:  forward (temp fills) → backward (reads temp) → update → reset_temp_mem\nepoch N+1: forward (temp fills from 0) → ...\n```\n\n`ParameterList` is a dynamic array of trainable tensors. It is a `typedef` of `Dt_array` allocated in the PERM arena.\n\n```\nParameterList *pl = create_param_list(mem);\n\n// Manual registration\nparam_list_add(mem, pl, my_tensor);\n\n// Or automatic — create_linear registers weights and bias for you\nLinear *l = create_linear(mem, pl, 128, 64);\n```\n\nOnce built, the parameter list drives the training loop:\n\n```\nzero_grad(pl);       // zero all parameter gradients\nbackward(mem, loss); // backprop\nsgd_step(pl, 0.01f); // update all parameters\n```\n\n| Function | Signature | Description | \n|---|---|---|\n| `create_global_mem` | `(size_t size) → Memory*` | Allocate perm + temp arenas (e.g., 1<<28 = 256MB each) | \n| `reset_temp_mem` | `(Memory *mem)` | Reset temp arena to empty | \n| `allocate_mem` | `(Memory *mem, size_t size, uint8_t perm) → void*` | Arena allocation | \n| `free_global_mem` | `(Memory *mem)` | Free both arenas and Memory | \n| `save_checkpoint` | `(ParameterList *pl, const char *path)` | Save all parameters to binary file | \n| `load_checkpoint` | `(ParameterList *pl, const char *path)` | Load parameters from binary file | \n\n| Function | Signature | Description | \n|---|---|---|\n| `create_param_list` | `(Memory *mem) → ParameterList*` | Create an empty PERM parameter list | \n| `param_list_add` | `(Memory *mem, ParameterList *pl, Tensor *t)` | Add a tensor to the list | \n| `zero_grad` | `(ParameterList *pl)` | Zero gradients for all parameters | \n| `sgd_step` | `(ParameterList *pl, float lr)` | SGD update: `data -= lr * grad` | \n| `clip_gradients` | `(ParameterList *pl, float threshold)` | Clip gradients to [-threshold, threshold] | \n| `adam_init` | `(Memory*, ParameterList*, lr, beta1, beta2, eps, t) → Adam*` | Initialize Adam optimizer state | \n| `adam_step` | `(Adam*, ParameterList*)` | Adam update step | \n| `adamw_init` | `(Memory*, ParameterList*, lr, beta1, beta2, eps, weight_decay, t) → AdamW*` | Initialize AdamW optimizer state | \n| `adamw_step` | `(AdamW*, ParameterList*)` | AdamW update step | \n\n| Function | Signature | Description | \n|---|---|---|\n| `tensor_init` | `(Memory*, int *shape, int ndim, uint8_t perm) → Tensor*` | Zero-initialized tensor | \n| `tensor_zeros` | `(Memory*, int *shape, int ndim, uint8_t perm) → Tensor*` | Same as tensor_init | \n| `tensor_ones` | `(Memory*, int *shape, int ndim, uint8_t perm) → Tensor*` | All ones | \n| `tensor_randn` | `(Memory*, int *shape, int ndim, uint8_t perm) → Tensor*` | Random normal (Box-Muller) | \n| `tensor_xavier` | `(Memory*, int *shape, int ndim, uint8_t perm) → Tensor*` | Xavier initialization | \n| `tensor_eye` | `(Memory*, int n, int m, uint8_t perm) → Tensor*` | Identity matrix (n×n or n×m) | \n| `tensor_full_like` | `(Memory*, Tensor *a, float fill_value, uint8_t perm) → Tensor*` | Same shape as a, filled with fill_value | \n| `tensor_zeros_like` | `(Memory*, Tensor *a, uint8_t perm) → Tensor*` | Same shape as a, filled with zeros | \n| `tensor_ones_like` | `(Memory*, Tensor *a, uint8_t perm) → Tensor*` | Same shape as a, filled with ones | \n| `print_t` | `(Tensor*, uint8_t grad)` | Print tensor (grad=1 to include gradients) | \n\n| Function | Signature | Description | \n|---|---|---|\n| `add_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Element-wise a + b | \n| `sub_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Element-wise a - b | \n| `mul_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Element-wise a * b | \n| `divide_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Element-wise a / b | \n| `neg_t` | `(Memory*, Tensor *a) → Tensor*` | Element-wise -a | \n| `pow_t` | `(Memory*, Tensor *a, float exp) → Tensor*` | Element-wise pow(a, exp) | \n| `scale_t` | `(Memory*, Tensor *a, float v) → Tensor*` | Element-wise a * scalar | \n\n| Function | Signature | Description | \n|---|---|---|\n| `exp_t` | `(Memory*, Tensor *a) → Tensor*` | Element-wise exp | \n| `log_t` | `(Memory*, Tensor *a) → Tensor*` | Element-wise log | \n\n| Function | Signature | Description | \n|---|---|---|\n| `sum_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Sum along dimension | \n| `mean_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Mean along dimension | \n| `max_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Max along dimension | \n\n| Function | Signature | Description | \n|---|---|---|\n| `relu_t` | `(Memory*, Tensor *a) → Tensor*` | ReLU | \n| `gelu_t` | `(Memory*, Tensor *a) → Tensor*` | GELU (tanh approximation) | \n| `sigmoid_t` | `(Memory*, Tensor *a) → Tensor*` | Sigmoid | \n| `tanh_t` | `(Memory*, Tensor *a) → Tensor*` | Tanh | \n| `softmax_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Softmax along dimension | \n\n| Function | Signature | Description | \n|---|---|---|\n| `mask_t` | `(Memory*, Tensor *a, Tensor *b, float val) → Tensor*` | Where b==1, set to val; else keep a | \n\n| Function | Signature | Description | \n|---|---|---|\n| `matmul_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Matrix multiply (2D) | \n| `bmm_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Batch matrix multiply (3D: [B,T,D] x [B,D,T] → [B,T,T]) | \n| `transpose_t` | `(Memory*, Tensor *a) → Tensor*` | Transpose 2D | \n| `dot_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Dot product (1D) | \n\n| Function | Signature | Description | \n|---|---|---|\n| `embedding_t` | `(Memory*, Tensor *vocab, Tensor *indices) → Tensor*` | Look up embeddings (vocab: [V,D], indices: [B,T]) | \n\n| Function | Signature | Description | \n|---|---|---|\n| `reshape_t` | `(Memory*, Tensor *a, int *shape, int ndim) → Tensor*` | Reshape (shares data) | \n| `squeeze_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Remove dim of size 1 | \n| `unsqueeze_t` | `(Memory*, Tensor *a, int dim) → Tensor*` | Insert dim of size 1 | \n| `broadcast_t` | `(Memory*, Tensor *a, int *shape, int tar_dim) → Tensor*` | Broadcast to target shape | \n| `permute_t` | `(Memory*, Tensor *a, int *dims, int total_dim) → Tensor*` | Permute dimensions | \n| `concat_t` | `(Memory*, Tensor *a, Tensor *b, int dim) → Tensor*` | Concatenate along dimension | \n| `slice_t` | `(Memory*, Tensor *a, int dim, int split_size) → Pair_T*` | Split tensor along dim, returns pair {F, S} | \n\n| Function | Signature | Description | \n|---|---|---|\n| `mseloss_t` | `(Memory*, Tensor *a, Tensor *b) → Tensor*` | Mean squared error | \n| `crossentropyloss_t` | `(Memory*, Tensor *logits, Tensor *targets) → Tensor*` | Cross-entropy (logits: [N,C], targets: [N]) | \n\n| Function | Signature | Description | \n|---|---|---|\n| `create_linear` | `(Memory*, ParameterList *pl, int d_in, int d_out) → Linear*` | Linear layer, auto-registers weights + bias | \n| `linear_t` | `(Memory*, Linear*, Tensor *x) → Tensor*` | Forward: x @ W^T + b | \n| `create_layernorm` | `(Memory*, ParameterList *pl, int normalized_shape, float eps) → LayerNorm*` | LayerNorm, auto-registers weight + bias | \n| `layernorm_t` | `(Memory*, LayerNorm*, Tensor *x) → Tensor*` | Layer normalization forward | \n\n| Function | Signature | Description | \n|---|---|---|\n| `backward` | `(Memory*, Tensor *root)` | Backpropagate from root | \n\nAll operations register backward passes and track parent tensors automatically.", "url": "https://wpnews.pro/news/pytorch-written-in-c", "canonical_source": "https://github.com/thevoxium/bare-lm", "published_at": "2026-09-22 06:02:47+00:00", "updated_at": "2026-09-22 06:23:54.784233+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "developer-tools", "ai-research"], "entities": ["bare-lm", "thevoxium", "OpenBLAS", "GitHub", "Linux", "macOS"], "alternates": {"html": "https://wpnews.pro/news/pytorch-written-in-c", "markdown": "https://wpnews.pro/news/pytorch-written-in-c.md", "text": "https://wpnews.pro/news/pytorch-written-in-c.txt", "jsonld": "https://wpnews.pro/news/pytorch-written-in-c.jsonld"}}