Natural Language Processing news and analysis on Web Pulse: 5573 curated articles tracking the latest Natural Language Processing developments, tools, and research, updated continuously from vetted sources.
Researchers have introduced a novel Audio-Visual Speech Enhancement (AVSE) framework that uses a Large Language Model (LLM) to generate narrative descriptions of enhanced speech, which are then analyzed by a sentiment an…
Large language models like OLMo-2 7B exhibit left-right brain asymmetry in language processing that mirrors the human brain's left-hemisphere dominance, according to research using fMRI. As the models train, their intern…
WordPiece, Unigram, and SentencePiece are three subword tokenization algorithms that differ in how they split text, with WordPiece using a likelihood-maximizing merge criterion and a ## continuation marker, while Sentenc…
Byte-Pair Encoding (BPE), originally a data compression algorithm published by Philip Gage in 1994, was adapted in 2016 by Sennrich, Haddow, and Birch for subword tokenization in neural machine translation, solving the v…
A language model does not see letters or pixels; it sees a sequence of integers from a vocabulary table, and the raw input is just bytes. Unicode assigns every character a unique integer code point, but encoding these co…
The encoder-decoder (seq2seq) architecture, proposed by Cho et al. and Sutskever et al. in 2014, compresses the entire source sentence into a single fixed-length vector, causing performance to degrade sharply on longer s…
Recurrent neural networks (RNNs) process sequences by maintaining a hidden state that compresses the entire history into a fixed-size vector, as formalized in Elman's 1990 paper "Finding Structure in Time." The hidden st…
GloVe, FastText, and word2vec are all word embedding methods that produce dense vector representations of words, but they differ in their underlying approaches: GloVe uses a weighted least-squares regression on global co…
Google researchers led by Tomas Mikolov published word2vec in 2013, a method that made high-quality dense word embeddings cheap to train at billion-token scale on a single machine, transforming them from a research curio…
N-gram language models suffer from the curse of sparsity: as context length increases, the number of possible n-grams grows exponentially (Vⁿ), far exceeding any training corpus, leading to most unseen n-grams receiving …
A language model assigns a probability to the next token given all previous tokens, a task that the chain rule shows is sufficient to capture any pattern in language. Claude Shannon demonstrated in 1951 that humans can p…
Language models generate text by constructing a categorical probability distribution over their vocabulary at each decoding step, converting raw neural network outputs called logits into valid probabilities using the sof…
A blog post explains that the L2 norm measures a vector's magnitude as its Euclidean distance from the origin, while the dot product quantifies similarity between vectors and is the core operation behind attention mechan…
Dukaan Digital has built a prototype that lets kirana store owners digitize their inventory by taking a photo of a wholesale invoice or shelf, using a vision model to extract items and generate an ONDC-compliant catalog.…
The NHS is rolling out Heidi, an AI-powered speech-to-text transcription software, across 15 trusts in the West Midlands to reduce administrative burdens and speed up consultations. The tool, which listens to doctor-pati…
A new benchmark called the Unified Embodied Seeking and Following Benchmark (UESF-Bench) challenges AI to find and follow humans in dynamic settings using only language descriptions, moving beyond current benchmarks that…
Researchers introduce Self-Correcting Coupled Markov Jump Processes (SC-CMJP), a framework enabling concurrent image understanding and generation through cross-modal coupling, and its training-free sampler CO₂Jump achiev…
Researchers propose Samba, a hybrid Mamba architecture for audio-visual navigation that replaces conventional GRUs with an adaptive selection-enabled Mamba State Encoder (M-SE) and introduces an Audio Mamba Encoder (AME)…
Researchers propose LAPO (Leave-One-Turn Attribution), a self-generated process-supervision method for multi-turn search reasoning that uses backward leave-one-turn attribution to estimate each turn's contribution via An…
A team of researchers achieved an average precision of 0.875 and a macro-F1 of 0.72 on the Ambivalence/Hesitancy Video Recognition Challenge of the 11th ABAW Competition using an audio-text system that excludes visual fr…