Machine learning news — deep learning, reinforcement learning, neural architecture search, diffusion models, and new ML frameworks and libraries.
CurioSFT, a new fine-tuning method that incorporates intrinsic curiosity, outperformed traditional supervised fine-tuning by 2.5 points on in-distribution mathematical reasoning tasks and 2.9 points on out-of-distributio…
ECHO, a novel framework for traceable context reconstruction in agentic reinforcement learning, achieves 43.4% held-out accuracy on BrowseComp-Plus, outperforming GRPO's 28.9% and SUPO's 36.1% with fewer turns and lower …
MedDiffuseMix, a novel saliency-guided diffusion mixing tool, improves medical image classification by selectively enhancing less important regions while preserving critical diagnostic details. Tested on four public benc…
A novel convolution-inspired network structure for classifying ancient cuneiform tablets outperforms the state-of-the-art transformer-based network Point-BERT, according to researchers who developed the architecture. The…
A user requests an INT8 ConvRot version for FireRed Image Edit v1.1, citing text artifacts and facial drift with standard FP8 variants, and suggests using ComfyUI's native ConvRot support for a quantization pass to impro…
A new pipeline called ARGUS aims to fix blind spots in AI retrieval systems by preemptively addressing weak spots in data retrieval. Developed using a large-scale dataset from Wikidata and Wikipedia, ARGUS improves retri…
A new AI architecture called Dendritron Transformer, designed to never forget, is explained in a video. The system aims to address catastrophic forgetting in neural networks by incorporating dendritic computation. This a…
WordPiece, Unigram, and SentencePiece are three subword tokenization algorithms that differ in how they split text, with WordPiece using a likelihood-maximizing merge criterion and a ## continuation marker, while Sentenc…
Byte-Pair Encoding (BPE), originally a data compression algorithm published by Philip Gage in 1994, was adapted in 2016 by Sennrich, Haddow, and Birch for subword tokenization in neural machine translation, solving the v…
The encoder-decoder (seq2seq) architecture, proposed by Cho et al. and Sutskever et al. in 2014, compresses the entire source sentence into a single fixed-length vector, causing performance to degrade sharply on longer s…
Recurrent neural networks (RNNs) process sequences by maintaining a hidden state that compresses the entire history into a fixed-size vector, as formalized in Elman's 1990 paper "Finding Structure in Time." The hidden st…
GloVe, FastText, and word2vec are all word embedding methods that produce dense vector representations of words, but they differ in their underlying approaches: GloVe uses a weighted least-squares regression on global co…
Google researchers led by Tomas Mikolov published word2vec in 2013, a method that made high-quality dense word embeddings cheap to train at billion-token scale on a single machine, transforming them from a research curio…
A technical blog post explains that AdamW with warmup and cosine decay is the optimizer almost everyone uses today to train large language models, tracing the evolution from vanilla SGD through momentum and Adam. The pos…
Gradient descent is the core algorithm behind deep learning, enabling neural networks with millions of parameters to minimize loss by iteratively adjusting weights in the direction opposite to the gradient. The gradient,…
Cross-entropy loss is the standard loss function for training language models like GPT, LLaMA, and BERT because it is differentiable and captures probabilistic confidence, unlike accuracy which is a step function and ign…
Language models generate text by constructing a categorical probability distribution over their vocabulary at each decoding step, converting raw neural network outputs called logits into valid probabilities using the sof…
A blog post explains that the L2 norm measures a vector's magnitude as its Euclidean distance from the origin, while the dot product quantifies similarity between vectors and is the core operation behind attention mechan…
The first post in the 'Holding the LLM Stack in Your Head' series rebuilds intuition for linear algebra, explaining that vectors and matrices are the foundation of neural network operations. The author defines a 'space' …
The AIMO Interpretability Challenge, organized by the AI Mathematical Olympiad (AIMO) and the Fields Model Initiative, aims to uncover the reasoning processes behind AI models' decision-making by testing them on olympiad…