Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling
Researchers introduced LPES, a layer-specific positional embedding scaling method that mitigates the 'lost-in-the-middle' problem in LLMs by assigning distinct scaling factors to each layer, achieving up to 11.2% accurac…