Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels

Researchers introduced Semidirect Fourier Delta Attention (SFDA), a phase-controlled generalization of Kimi Delta Attention that replaces real diagonal decay with block-rotational Fourier control. The method achieves exact affine chunk transfer and formal stability bounds, and in toy state-tracking experiments, SFDA learned cyclic memory where the phase-disabled baseline remained near chance.

arXiv:2607.11897v1 Announce Type: new Abstract: Linear attention replaces softmax attention's growing KV cache with a fixed recurrent state, but this compression limits exact state tracking and long-context memory. We introduce \emph{Semidirect Fourier Delta Attention} SFDA , a phase-controlled generalization of Kimi Delta Attention that replaces real diagonal decay with block-rotational Fourier control: \ S t= I-\beta t k tk t^ \Lambda tS {t-1}+\beta tk tv t^ , \qquad \Lambda t=\diag \alpha t\odot e^{i\theta t} . \ Our main result is a constructive chunk-WY factorization for products \ A t=\Lambda t-u tr t^ \ , giving \ A t\cdots A 1=\Gamma t-Y tM tW t^ \ with rank growth bounded inside fixed chunks. This yields an exact affine chunk transfer, formal stability and complexity bounds, and a compact characterization of phase-plus-low-rank memory. We verify the algebra numerically and show in toy state-tracking experiments that SFDA learns cyclic memory where the phase-disabled KDA baseline remains near chance. Fused kernels and large-scale language-model comparisons are left to future work.