arXiv:2611.21907v1 Announce Type: new Abstract: Visual brain decoding from functional magnetic resonance imaging (fMRI) data has made remarkable progress in recent years, driven by increasingly powerful artificial intelligence (AI) models and the growing availability of large-scale public datasets. However, despite the impressive advances achieved in static image decoding, the reconstruction of videos from fMRI signals remains challenging due to the temporal mismatch between rapidly changing video stimuli and the slow dynamics of the blood-oxygen-level-dependent (BOLD) response. We evaluate a Bayesian-inspired proposal-and-rerank strategy on CC2017 and the BOLD Moments Dataset (BMD), using a linear semantic decoder, a pretrained video generator, and three neural encoding streams with ridge readouts and learned pooling in two streams. We refer to this framework as REEL (Reranking Encoder Embeddings with Linear decoding). Compared with a single generated candidate, reranking increases mean Video 50-way accuracy from 0.2359 to 0.2434 across three CC2017 participants and from 0.3653 to 0.3920 across ten BMD participants. These findings suggest an alternative and conceptually simpler strategy for improving video brain decoding, highlighting the potential of our approach as a complementary direction to increasingly complex pipelines.
REEL: Brain-Guided Video Generation and Selection by Reranking Encoder Embeddings with Linear Decoding from fMRI data
A Bayesian-inspired proposal-and-rerank framework called REEL raised mean Video 50-way accuracy in fMRI-based video brain decoding from 0.2359 to 0.2434 across three CC2017 participants and from 0.3653 to 0.3920 across ten BMD participants, according to the arXiv paper 2611.21907v1. REEL combines a linear semantic decoder, a pretrained video generator, and three neural encoding streams with ridge readouts and learned pooling in two streams, and reranks multiple generated video candidates instead of relying on a single one. The authors present reranking as a conceptually simpler complementary direction to increasingly complex decoding pipelines for reconstructing videos from fMRI signals, which remains difficult because of the temporal mismatch between fast video stimuli and slow BOLD response dynamics.
When decoding video from fMRI, generate several candidate videos and rerank them with a linear semantic decoder rather than accepting a single sample; the source measured 50-way accuracy gains of 0.0075 on CC2017 and 0.0267 on BMD.
more in playbook: Evals and testing for LLM appsRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.