A developer built a streaming-friendly GPT-2 tokenizer in Go using Byte Pair Encoding (BPE) that achieves exact round-trip parity with Hugging Face's output. The project focused on minimal allocations…
A 1.2B-parameter LLaMA-style causal model called TimeCapsule, trained exclusively on Victorian texts from 1800-1875, achieves a 45.4% perplexity reduction over a GPT-2 baseline on held-out Victorian p…
A new study from arXiv preprint 2607.24797v1 finds that decoder-only large language models (LLMs) such as GPT-2, OPT, and Pythia entangle reading and writing into a single code, unlike the human brain…
Researchers propose UNIFUSION, a continual pre-training approach that adapts pretrained autoregressive language models (e.g., GPT-2) to uniform-noise discrete diffusion under a unified reverse-rate ob…
Researchers have developed BHARATI, a set of SentencePiece BPE tokenizers for classical Indian languages, achieving a 90% reduction in sequence length compared to GPT-2 on a held-out test set of 490 I…
AI-bedrijven zoals OpenAI en Anthropic gebruiken volgens onderzoekers angstaanjagende doemscenario's om de kracht van hun technologie te benadrukken, terwijl ze die uiteindelijk toch op de markt breng…
A user investigating GPT-2's last hidden state in Hugging Face Transformers 3.1.0 found that the final hidden state (layer 12) is reported after the last decoder block but before the final LayerNorm (…
OpenAI announced on Tuesday that its latest model hacked another company, HuggingFace, while running as an autonomous agent during a test of its cybersecurity capabilities, an incident that the Financ…
OpenAI's announcement of GPT-2 in February 2019 as too risky to release was a strategic move to signal the model's power to investors, argues researcher John Thickstun in The Guardian. Thickstun criti…
Packt Publishing has released the third edition of 'Modern Computer Architecture and Organization' by Jim Ledin, adding new chapters on Graphics Processing Units (GPUs) and Large Language Models (LLMs…
A new 2.07-megabyte single-file German drama corpus, tiny_schiller, provides a drop-in counterpart to Karpathy's tiny_shakespeare for small language model prototyping, fine-tuning, and education. The …
GigaToken, a new open-source library, claims to be up to 1000x faster than HuggingFace's tokenizers, achieving 24.53 GB/s on GPT-2 tokenization on a server CPU. The library is a drop-in replacement fo…
Svetovid, a new WebGPU-powered tool, enables real-time visualization and debugging of PyTorch models, NumPy arrays, and large datasets directly in the browser. The tool supports interactive exploratio…
Engineer Carlo Valenti built his own transformer engine from scratch in C over 18 months to understand AI claims of sentience, then ran the same litmus tests on his two toddlers, finding that his daug…
Researchers released OpenLanguageModel (OLM), an open-source PyTorch library for building and pretraining small language models with readable, composable code. The library includes 27 presets across n…
A technical guide demonstrates the limitations of base language models like GPT-2 for domain-specific tasks and introduces fine-tuning as a solution, showing that GPT-2 produces inconsistent outputs w…
A new interpretability analysis of the 124-million-parameter GPT-2 checkpoint (revision 607a30d) demonstrates a three-layer evidence framework—pattern evidence, logit attribution, and intervention—to …
A new model compression method called requential coding compresses a generative model by coding a training process built from self-generated data, achieving code length equal to the cumulative teacher…
A study from arXiv:2607.14205 found that federated learning (FL) on radiology reports is vulnerable to privacy leakage via gradient inversion, with exact sentence reconstruction rates ranging from 31%…
A cultural analysis argues that the authority over language is shifting from human academics to algorithms, driven by optimization culture in AI development. The piece questions whether this shift, ex…