Researchers propose SeFoRA, a sketch-aggregated federated low-rank adaptation algorithm that enables direct aggregation of client updates with heterogeneous LoRA ranks, alleviating bilinear mismatch a…
A new study from arXiv (arXiv:2607.28658v1) finds that downstream fine-tuning on GLUE does not reliably preserve the reference ranking of federated pre-trained models, whereas direct next-token predic…
Liquid AI released LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, general-purpose bidirectional encoders that match or beat larger models on GLUE, SuperGLUE, and multilingual tasks while running about 3…
A new framework called LAARA (Layer-Aware Adaptive Rank Allocation) dynamically assigns adapter ranks per transformer layer using lightweight diagonal Fisher estimates, outperforming LoRA, AdaLoRA, Dy…
Researchers introduced ReCoLoRA, a spectrum-aware recursive consolidation method for continual fine-tuning of large language models, which prevents task interference by re-decomposing the effective we…
A new study finds that averaging benchmark scores produces misleading rankings when evaluation data is sparse and item difficulty varies widely, with Spearman rank correlation dropping from 1.000 to 0…
A developer built a generative simulation benchmarking framework for heritage language revitalization, using embodied agent feedback loops to evaluate cultural resonance beyond standard metrics like B…
Researchers introduced the Integral Transform Network (ITNet), a unified architecture that subsumes convolution, attention, and recurrence as special cases of a learnable integral transform. ITNet mat…
Researchers at arXiv propose a hybrid pre-training objective for language models that combines a Joint Embedding Predictive Architecture (JEPA) latent-space prediction loss with masked language modell…