PyTorch Written in C

Developer thevoxium released bare-lm, a minimal autograd tensor library written in C, installable via a git clone and make build or prebuilt Linux and macOS tarballs. The library provides a Memory object with 256 MB permanent and temporary arenas, a ParameterList for trainable tensors, and operations including linear_t, relu_t, sigmoid_t, mseloss_t, backward, zero_grad, and sgd_step, as shown in a 500-epoch XOR training example. bare-lm requires a C compiler, make, and OpenBLAS, and installs to /usr/local/include/bare.h and /usr/local/lib/libbare.{so,dylib}.

A minimal autograd tensor library in C. You can install bare-lm in two ways. Requirements: - A C compiler cc , gcc , or clang - make - OpenBLAS headers + library Install OpenBLAS first: macOS Homebrew brew install openblas Ubuntu / Debian sudo apt update && sudo apt install -y libopenblas-dev Then build and install: git clone https://github.com/thevoxium/bare-lm.git cd bare-lm macOS make OPENBLAS PREFIX=/opt/homebrew/opt/openblas Linux make sudo make install By default this installs to /usr/local/include/bare.h and /usr/local/lib/libbare.{so,dylib} . Each tagged release publishes prebuilt archives for Linux and macOS: - bare-<version -linux.tar.gz - bare-<version -macos.tar.gz After downloading and extracting a release tarball, install headers and library: tar -xzf bare-<version -<artifact .tar.gz cd bare-<version -<artifact sudo cp include/bare.h /usr/local/include/ sudo cp lib/libbare. /usr/local/lib/ cc file.c -lbare If your system does not search /usr/local/lib by default, add -L/usr/local/lib and -I/usr/local/include . include "bare.h" int main { Memory mem = create global mem 1 << 28 ; ParameterList pl = create param list mem ; int x shape = {4, 2}; int y shape = {4, 1}; Tensor x = tensor init mem, x shape, 2, PERM ; Tensor y = tensor init mem, y shape, 2, PERM ; float x data = {0, 0, 0, 1, 1, 0, 1, 1}; float y data = {0, 1, 1, 0}; for int i = 0; i < 8; i++ x- data i = x data i ; for int i = 0; i < 4; i++ y- data i = y data i ; Linear l1 = create linear mem, pl, 2, 8 ; Linear l2 = create linear mem, pl, 8, 1 ; load checkpoint pl, "xor.btw" ; for int epoch = 0; epoch < 500; epoch++ { zero grad pl ; Tensor h = linear t mem, l1, x ; h = relu t mem, h ; Tensor o = linear t mem, l2, h ; o = sigmoid t mem, o ; Tensor loss = mseloss t mem, o, y ; backward mem, loss ; if epoch % 100 == 0 printf "epoch %3d loss=%.4f\n", epoch, loss- data 0 ; sgd step pl, 0.1f ; reset temp mem mem ; } save checkpoint pl, "xor.btw" ; free global mem mem ; } Step 1: Create memory and a parameter list. A single Memory object holds two arenas permanent and temporary . 1 << 28 = 256 MB per arena. A ParameterList tracks all trainable tensors. Memory mem = create global mem 1 << 28 ; ParameterList pl = create param list mem ; Step 2: Allocate persistent tensors. Inputs x and y are plain PERM tensors not trainable . Pass PERM so they survive reset temp mem . Tensor x = tensor init mem, x shape, 2, PERM ; Tensor y = tensor init mem, y shape, 2, PERM ; Step 3: Create layers with auto-registration. create linear allocates weights and bias as PERM tensors and automatically adds them to the parameter list. Linear l1 = create linear mem, pl, 2, 8 ; Linear l2 = create linear mem, pl, 8, 1 ; // pl now contains l1- weights, l1- bias, l2- weights, l2- bias Step 4: Forward pass. Every operation linear t , relu t , sigmoid t , mseloss t allocates its result from the temp arena. No malloc calls, no cleanup code. Tensor h = linear t mem, l1, x ; h = relu t mem, h ; Tensor o = linear t mem, l2, h ; o = sigmoid t mem, o ; Tensor loss = mseloss t mem, o, y ; Step 5: Backward pass. backward builds a topological sort of the computation graph also temp-allocated and propagates gradients. Intermediate tensor data is still valid at this point. backward mem, loss ; Step 6: Zero grads, update, reset. zero grad clears all parameter gradients. sgd step applies SGD to all parameters in one call. After gradients are consumed, reset temp mem zeroes the temp arena in O 1 . All intermediate tensors from the forward pass are gone. zero grad pl ; sgd step pl, 0.1f ; reset temp mem mem ; Step 7: Cleanup. free global mem releases both arenas and the Memory struct. free global mem mem ; All allocations go through allocate mem mem, size, perm . There are no individual free calls. | Flag | Lifetime | Used for | |---|---|---| | PERM | Until free global mem | Weights, biases, persistent inputs/targets | | TEMP | Until reset temp mem | Operation results, graph traversal arrays, loss | reset temp mem resets the temp arena pointer to 0. It is called after both forward and backward complete, because backward reads intermediate tensor data. epoch N: forward temp fills → backward reads temp → update → reset temp mem epoch N+1: forward temp fills from 0 → ... ParameterList is a dynamic array of trainable tensors. It is a typedef of Dt array allocated in the PERM arena. ParameterList pl = create param list mem ; // Manual registration param list add mem, pl, my tensor ; // Or automatic — create linear registers weights and bias for you Linear l = create linear mem, pl, 128, 64 ; Once built, the parameter list drives the training loop: zero grad pl ; // zero all parameter gradients backward mem, loss ; // backprop sgd step pl, 0.01f ; // update all parameters | Function | Signature | Description | |---|---|---| | create global mem | size t size → Memory | Allocate perm + temp arenas e.g., 1<<28 = 256MB each | | reset temp mem | Memory mem | Reset temp arena to empty | | allocate mem | Memory mem, size t size, uint8 t perm → void | Arena allocation | | free global mem | Memory mem | Free both arenas and Memory | | save checkpoint | ParameterList pl, const char path | Save all parameters to binary file | | load checkpoint | ParameterList pl, const char path | Load parameters from binary file | | Function | Signature | Description | |---|---|---| | create param list | Memory mem → ParameterList | Create an empty PERM parameter list | | param list add | Memory mem, ParameterList pl, Tensor t | Add a tensor to the list | | zero grad | ParameterList pl | Zero gradients for all parameters | | sgd step | ParameterList pl, float lr | SGD update: data -= lr grad | | clip gradients | ParameterList pl, float threshold | Clip gradients to -threshold, threshold | | adam init | Memory , ParameterList , lr, beta1, beta2, eps, t → Adam | Initialize Adam optimizer state | | adam step | Adam , ParameterList | Adam update step | | adamw init | Memory , ParameterList , lr, beta1, beta2, eps, weight decay, t → AdamW | Initialize AdamW optimizer state | | adamw step | AdamW , ParameterList | AdamW update step | | Function | Signature | Description | |---|---|---| | tensor init | Memory , int shape, int ndim, uint8 t perm → Tensor | Zero-initialized tensor | | tensor zeros | Memory , int shape, int ndim, uint8 t perm → Tensor | Same as tensor init | | tensor ones | Memory , int shape, int ndim, uint8 t perm → Tensor | All ones | | tensor randn | Memory , int shape, int ndim, uint8 t perm → Tensor | Random normal Box-Muller | | tensor xavier | Memory , int shape, int ndim, uint8 t perm → Tensor | Xavier initialization | | tensor eye | Memory , int n, int m, uint8 t perm → Tensor | Identity matrix n×n or n×m | | tensor full like | Memory , Tensor a, float fill value, uint8 t perm → Tensor | Same shape as a, filled with fill value | | tensor zeros like | Memory , Tensor a, uint8 t perm → Tensor | Same shape as a, filled with zeros | | tensor ones like | Memory , Tensor a, uint8 t perm → Tensor | Same shape as a, filled with ones | | print t | Tensor , uint8 t grad | Print tensor grad=1 to include gradients | | Function | Signature | Description | |---|---|---| | add t | Memory , Tensor a, Tensor b → Tensor | Element-wise a + b | | sub t | Memory , Tensor a, Tensor b → Tensor | Element-wise a - b | | mul t | Memory , Tensor a, Tensor b → Tensor | Element-wise a b | | divide t | Memory , Tensor a, Tensor b → Tensor | Element-wise a / b | | neg t | Memory , Tensor a → Tensor | Element-wise -a | | pow t | Memory , Tensor a, float exp → Tensor | Element-wise pow a, exp | | scale t | Memory , Tensor a, float v → Tensor | Element-wise a scalar | | Function | Signature | Description | |---|---|---| | exp t | Memory , Tensor a → Tensor | Element-wise exp | | log t | Memory , Tensor a → Tensor | Element-wise log | | Function | Signature | Description | |---|---|---| | sum t | Memory , Tensor a, int dim → Tensor | Sum along dimension | | mean t | Memory , Tensor a, int dim → Tensor | Mean along dimension | | max t | Memory , Tensor a, int dim → Tensor | Max along dimension | | Function | Signature | Description | |---|---|---| | relu t | Memory , Tensor a → Tensor | ReLU | | gelu t | Memory , Tensor a → Tensor | GELU tanh approximation | | sigmoid t | Memory , Tensor a → Tensor | Sigmoid | | tanh t | Memory , Tensor a → Tensor | Tanh | | softmax t | Memory , Tensor a, int dim → Tensor | Softmax along dimension | | Function | Signature | Description | |---|---|---| | mask t | Memory , Tensor a, Tensor b, float val → Tensor | Where b==1, set to val; else keep a | | Function | Signature | Description | |---|---|---| | matmul t | Memory , Tensor a, Tensor b → Tensor | Matrix multiply 2D | | bmm t | Memory , Tensor a, Tensor b → Tensor | Batch matrix multiply 3D: B,T,D x B,D,T → B,T,T | | transpose t | Memory , Tensor a → Tensor | Transpose 2D | | dot t | Memory , Tensor a, Tensor b → Tensor | Dot product 1D | | Function | Signature | Description | |---|---|---| | embedding t | Memory , Tensor vocab, Tensor indices → Tensor | Look up embeddings vocab: V,D , indices: B,T | | Function | Signature | Description | |---|---|---| | reshape t | Memory , Tensor a, int shape, int ndim → Tensor | Reshape shares data | | squeeze t | Memory , Tensor a, int dim → Tensor | Remove dim of size 1 | | unsqueeze t | Memory , Tensor a, int dim → Tensor | Insert dim of size 1 | | broadcast t | Memory , Tensor a, int shape, int tar dim → Tensor | Broadcast to target shape | | permute t | Memory , Tensor a, int dims, int total dim → Tensor | Permute dimensions | | concat t | Memory , Tensor a, Tensor b, int dim → Tensor | Concatenate along dimension | | slice t | Memory , Tensor a, int dim, int split size → Pair T | Split tensor along dim, returns pair {F, S} | | Function | Signature | Description | |---|---|---| | mseloss t | Memory , Tensor a, Tensor b → Tensor | Mean squared error | | crossentropyloss t | Memory , Tensor logits, Tensor targets → Tensor | Cross-entropy logits: N,C , targets: N | | Function | Signature | Description | |---|---|---| | create linear | Memory , ParameterList pl, int d in, int d out → Linear | Linear layer, auto-registers weights + bias | | linear t | Memory , Linear , Tensor x → Tensor | Forward: x @ W^T + b | | create layernorm | Memory , ParameterList pl, int normalized shape, float eps → LayerNorm | LayerNorm, auto-registers weight + bias | | layernorm t | Memory , LayerNorm , Tensor x → Tensor | Layer normalization forward | | Function | Signature | Description | |---|---|---| | backward | Memory , Tensor root | Backpropagate from root | All operations register backward passes and track parent tensors automatically.