04:00
2026-06-05
arxiv.org
machine-learning
Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning
Researchers at arXiv propose a hybrid pre-training objective for language models that combines a Joint Embedding Predictive Architecture (JEPA) latent-space prediction loss with masked language modell…