A developer trained a 10.77M-parameter GPT on 4.45M characters of personal Google data from 15 years, achieving a loss of 1.60 after pretraining and supervised fine-tuning on 8,520 email reply pairs, …
Patrick O'Driscoll, a former nanotech physicist and current AI architect, introduces Competitive AI Safety as a paradigm to focus the field on measurable, tractable goals, drawing inspiration from Ope…
An AI developer tested Graphify, an open-source tool with 86k+ GitHub stars that converts codebases into knowledge graphs. On a 4-file project, it extracted 15 nodes, 3 communities, and 17 relationshi…
A guide explains how to build a tiny large language model from scratch, covering tokenization, pretraining, supervised fine-tuning, and alignment. It aims to demystify the LLM pipeline for developers …
A new character-level language model called EntropyBeam, using gradient-free count tables and a Dirichlet prior, achieved a validation loss of 1.596 nats on the Shakespeare character benchmark, outper…
A developer implemented GPTQ quantization from scratch on a nanoGPT model, achieving only 1.1% perplexity degradation across 61 quantized layers. The implementation uses second-order optimization to r…
A new open-source simulator called Cosmicgpt models how space radiation, including cosmic-ray bit flips and other faults, affects GPT inference on satellites. The tool injects single-event effects int…
A developer built a character-level language model from scratch using Shakespeare's complete works, training a nanoGPT model on a consumer-grade MacBook Pro in about 15 minutes. The project demonstrat…
Pyrefly, a Python type checker, has introduced an experimental feature that brings tensor shapes into Python's type system, allowing shape annotations to become inferred type hints instead of comments…