AI-assisted: this article was drafted with AI assistance and checked against BCI Report’s published results.
Look at a light flickering ten times a second, and your visual cortex starts to oscillate at ten hertz, and at twenty and thirty. This steady-state visual evoked potential (SSVEP) is strong enough to read from a few scalp electrodes, which is why SSVEP spellers work: each key flickers at its own rate, and a decoder names the rate you are looking at.
In 2006, Lin and colleagues showed how to do that with no training at all. Canonical correlation analysis (CCA) writes down sine waves at each candidate frequency and its harmonics, and asks which set the EEG window lines up with best.
Twenty years later, BCI Report trained small neural networks on 32 of a benchmark’s 40 flicker frequencies, then asked them about the 8 they had never seen. They reached at most 41.46% among those eight. CCA, untrained, reached 80.90% on the same windows.
No large model was trained along the way. The learned parts are heads of a few hundred thousand parameters, ridge readouts and one 64-unit MLP; the only foundation-model features, from a frozen CBraMod encoder, appear in experiment 1.
A reader’s comments on the Hugging Face post about that result raised the questions that became fourteen experiments. Was it the network, the input, or the way each frequency was asked? And can learning improve on CCA? The experiments were published on one result page in October 2026. Here is what they found.
TL;DR
The data. BETA (Liu et al., 2020) is a public SSVEP benchmark: 70 people and 40 targets flickering from 8.0 to 15.8 Hz in 0.2 Hz steps, recorded outside a shielded room. The experiments use 11,200 two-second windows from eight electrodes over the back of the head.
The split. People are divided into seven folds, so every test person is new to the decoder. Frequencies are divided into five rotations, each with 32 used for training and 8 held out. The main score is accuracy at choosing among the 8 held-out frequencies, where chance is 12.5%. A second score, balanced accuracy among the 32 trained frequencies (chance 3.13%), is a different task. “Unseen” means unseen by the trained part of the decoder.
The second dataset. Five experiments run on Wearable-102 (Zhu et al., 2021), one of them (experiment 7) only there. It has 102 people and 12 targets from 9.25 to 14.75 Hz, recorded with a dry and a wet headband. Models trained on BETA run there unchanged.
Two pieces of jargon.
Neither needs training.
How to read the numbers. Each experiment fixed its comparisons before it ran. Intervals are 95% intervals from resampling people and, unless noted, are not adjusted for multiple comparisons. When an interval includes zero, the direction is unresolved, which is not the same as “no effect”.
Each dot is a mean over 70 people with the 95% interval from its own experiment. The numeric-code row is the original study’s control without text; the dashed line marks chance.
Experiments 1 to 4 on the result page.
Asked through a text embedding, the original head reached 28.69% on new frequencies. Asked with a plain numeric code, a control without text, the same FiLM head reached 41.46%. Experiment 1 gave it something more physical: the spectrum of CCA’s own reference signals, sine and cosine at the frequency and its second and third harmonics. It reached 39.00% (36.07% to 41.78%), a paired difference of −2.46 pp from the numeric code (−3.47 to −1.52).
Swapping the head did not clear the bar either. A matching head in place of the FiLM head changed balanced accuracy on trained frequencies by −3.60 pp (−4.37 to −2.73) or +2.54 pp (+1.74 to +3.37), depending on the question, and neither met the protocol’s bar for a meaningful gain. The harmonics did matter: dropping the second and third from the question changed accuracy on new frequencies by −13.27 pp (−15.04 to −11.45).
Phase cut both ways. A complex spectrum keeps phase, and also changes the normalization and the head’s size. Asked with question numbers, it raised balanced accuracy on the 32 trained frequencies by +18.03 pp (+14.13 to +21.77), to 50.08%, still below CCA’s 65.46%. Asked with the reference spectrum, the same head scored 65.56% on eight frequencies in the rotations where they were trained. When the same eight were held out, it scored 8.67%, below chance. BETA gives each target its own stimulus phase, so phase carries label information a head can learn for its training targets. Telling the head each held-out target’s phase lifted it only to 24.97%.
Two last tries scrambled phase during training:
The bet in the comment thread had been that it would clear 39%; it did not.
What round 1 showed: the heads learned something about the frequencies they trained on. Less of it carried to held-out frequencies: at most 41.46%, above the 12.5% chance level but far below CCA’s 80.90%.
Experiment 5.
On the same splits, CCA on the raw two-second window reached 80.90% (76.80% to 85.13%). Tapering the window with a Hann function first, a standard step before taking a spectrum, changed it by −7.95 pp (−9.07 to −6.78).
A training-free match on the spectral coordinates that the complex-spectrum heads read, taken before the heads’ fitted scaling, reached 73.20%. That is +0.24 pp from Hann-tapered CCA (−0.12 to +0.64), unresolved. Simpler scorers that read only each channel’s energy reached 55.21% untrained and 57.45% with a learned ridge.
What round 2 showed: with no training, the coordinates the complex-spectrum heads read supported 73.20%. The learned heads on that kind of input reached at most 30.56%. The page quotes these side by side, with no paired contrast.
Experiments 6 and 7.
The next idea was humbler: keep CCA, and let a learned part only nudge its scores. A bounded learned correction on top of CCA reached 86.15% on new frequencies (82.84% to 89.35%), against 80.90% for CCA alone: +5.24 pp (+3.91 to +6.62). On the trained-frequency task its balanced accuracy was 73.19%, against 65.46% for CCA.
The correction is bounded by design. It can reorder only candidates whose CCA correlations lie within 0.2 of each other, and decisions with a wider margin never changed. It turned 7.39% of held-out decisions from wrong to right, 2.14% from right to wrong and 2.59% from one wrong answer to another: 12.12% changed in all.
Then the same 105 trained models (7 folds × 5 rotations × 3 seeds) ran unchanged on Wearable-102. Paired with CCA on the same windows, using the same three harmonics, they changed accuracy by +0.07 pp on dry electrodes (−0.71 to +0.91) and −0.61 pp on wet (−1.48 to +0.23). Both intervals include zero, and no equivalence margin was set. Wearable-102’s stored amplitude unit is undocumented and the correction reads amplitude spectra, so these are as-stored figures.
Hardware, reference, people, frequency grid and window onset all change between the datasets, so no single cause can be named. The dataset’s documentation also cannot rule out that some people appear in both.
Experiments 8 to 10.
Could a learned scorer do better with richer statistics than channel energy? Experiment 8 gave one shared set of learned weights the cross-spectral matrices between channels, at the fundamental and two harmonics:
These scorers read Hann-tapered, cropped spectral coordinates while CCA reads the raw window, so each contrast against CCA changes the representation as well.
Experiment 9 is the one worth remembering. It summarized each harmonic’s matrix by its eigenvalues, which reproduce CCA’s squared correlation for that harmonic, and trained a ridge head on them. The head reached 82.97% (79.09% to 86.82%), +2.07 pp above CCA (+1.09 to +3.12). That looks like learning on top of CCA working, but the head is handed statistics derived from CCA, so that contrast does not isolate what learning adds.
The same experiment also ran the untrained control: simply summing the same per-harmonic statistics. The sum reached 82.90% (78.90% to 86.93%). The head minus the sum was +0.07 pp (−0.45 to +0.62), unresolved. No equivalence margin was set, so it is not shown to be zero either.
Experiment 10 tried other ways of pooling those statistics:
Each row is a paired difference in percentage points with its 95% interval. The Wearable-102 rows run the BETA-trained models unchanged.
Experiments 11 to 14.
A filter bank. Filter-bank CCA (Chen et al., 2015) runs CCA on several band-pass-filtered copies of the window, called sub-bands, and adds the results with fixed, decaying weights. BCI Report’s SciPy realization of the published recipe, with seven sub-bands and five harmonics, reached 90.21% (87.65% to 92.62%) with two-second windows and eight electrodes. That is +7.31 pp over the harmonic sum (+5.15 to +9.47), the experiment’s primary contrast, and +9.31 pp over CCA (+6.84 to +11.71). It was above CCA in every window and electrode setting tested.
Shorter windows and fewer electrodes.
Shaded bands are 95% intervals.
Where the harmonic sum’s gain arrives. The untrained harmonic sum was +2.00 pp above joint CCA (+1.32 to +2.74), though both read the same three harmonic references. Experiment 12 inserted an intermediate score: the harmonics are scored separately and added, but must share one combination of electrodes.
Both are 99.44% intervals from a Bonferroni family of nine comparisons. So nearly all of the gain arrives at the step that lets each harmonic have its own spatial filter. The page treats this accounting as descriptive, not as a cause. On Wearable-102 the same step added +1.17 pp on wet electrodes (99.44% interval +0.35 to +2.01) and +0.66 pp on dry, unresolved (−0.25 to +1.58).
Weighting the harmonics. Filter-bank CCA weights its sub-bands with a fixed decaying rule, n^−1.25 + 0.25 for the n-th sub-band. Experiment 13 applied the same rule in two places:
Experiment 14 tried the opposite: rising weights of 1, 1.5 and 1.9 on the three harmonics. It is a coverage heuristic: filter-bank CCA’s sub-bands share one upper edge, so a higher harmonic falls inside more of them.
No weight was learned or searched.
Each row is a paired difference against flat weights in the same construction, with its 95% interval. The decay rule on the harmonics changed Wearable-102 accuracy by −1.28 pp dry (−2.05 to −0.54) and −1.36 pp wet (−1.91 to −0.79).
1. New frequencies were the hard part for the original kind of head. The heads of experiments 1 to 4 learned something about the trained frequencies and paid a large cost on held-out ones. CCA builds in a model of the response at any frequency. The heads had to learn it from 32 trained frequencies, and none reached CCA’s level.
2. On BETA, a bounded learned correction added about five points over CCA; on Wearable-102 every primary learned-versus-untrained contrast was unresolved. The correction changed about one decision in eight, in both directions. The eigenvalue head came out nearly level with an untrained sum of the same statistics.
3. Untrained choices moved accuracy too. Filter-bank CCA is a complete pipeline in which sub-bands, five harmonics and weights change together, and it was +9.31 pp above CCA. Letting each harmonic have its own spatial filter came with about two points on BETA. One fixed weighting rule changed accuracy by +5.21 pp in one place and −2.99 pp in another.
None of this says learning cannot help SSVEP decoding. Supervised spatial templates learned from a person’s own labeled trials were not compared. Each learned part was one fixed recipe. The only foundation-model features tested were frozen CBraMod features, in experiment 1.
What it does say: for frequencies and people the trained part never saw, the learned heads tested here did not replace CCA’s analytic structure. Against CCA alone, the bounded learned correction added +5.24 pp on BETA (+3.91 to +6.62).
On the result page the experiments appear as cards 1 to 14. The page also lists, card by card, what was not run, its checks and any engineering disclosures.
Standard CCA (Lin et al., 2006) and its many variants, most of them calibrated, have been compared systematically (Nakanishi et al., 2015). A survey of the field framed the choice as “to train or not to train” (Zerafa et al., 2018), and Chen et al. (2015) introduced filter-bank CCA with its comparison to standard CCA.
Some methods learn each person’s responses from labeled trials of every target, such as task-related component analysis (Nakanishi et al., 2018). Others transfer what a person’s calibrated targets teach to stimuli they have not calibrated (Wong et al., 2021; Wang et al., 2023; Mai et al., 2025). Here both the people and the frequencies are new to the trained part, and none of those methods was run.
The first article in this thread, Can an EEG Decoder Answer a Question it Was Never Trained on?, covers the head that started it.
Every number above comes from two public JSON files, pinned here to the commit of the release; no EEG recordings or weights are needed:
import jsonfrom urllib.request import urlopencommit = "39ebb1e30585cd6f4711e66f2c69913c140bfb28"base = f"https://raw.githubusercontent.com/twu3202/bci-report/{commit}/site/public/data/"def load(name): with urlopen(base + name, timeout=30) as response: return json.load(response)["studies"]early, late = load("new-ssvep-frequencies-update.json"), load("ssvep-followups-update.json")def pct(x): return f"{100 * x:.2f}"def level(study, key): row = study["levels"][key] lo, hi = row["interval"] or (None, None) return pct(row["estimate"]) + (f"% ({pct(lo)}% to {pct(hi)}%)" if lo is not None else "%")def diff(study, key): row = study["contrasts"][key] lo, hi = row["interval"] return f'{100 * row["estimate"]:+.2f} pp ({100 * lo:+.2f} to {100 * hi:+.2f})'print("CCA, held-out:", level(early["cca_diagnostics"], "CCA-RAW/u8"))print("CCA + learned correction:", diff(early["cca_adapter"], "RAW-RESIDUAL minus CCA-RAW/u8"))for sensor in ("dry", "wet"): print(" Wearable-102", sensor, diff(early["wearable_transfer"], f"RAW-RESIDUAL_minus_CCA-RAW/{sensor}"))print("Eigenvalue head - harmonic sum:", diff(late["coherence_eigen"], "EIG-RAW minus RAW-HARM-SUM/u8"))print("Filter-bank CCA:", level(late["practical_settings"], "w500-c8/FBCCA-M3-N7-H5/u8"))print(" minus CCA:", diff(late["practical_settings"], "w500-c8/FBCCA-M3-N7-H5 minus w500-c8/CCA-RAW-H3/u8"))print("Separate spatial filters:", diff(late["harmonic_gap"], "S minus B/u8"))print("Rising weights, BETA:", diff(late["rising_weights"], "literal_rise minus flat/u8"))print("Rising weights, dry:", diff(late["rising_weights"], "literal_rise minus flat/dry"))
The card 12 contrast prints its 99.44% family interval. The result page, the data API and the Hugging Face dataset carry every level, contrast and check count.
BETA was created by Bingchuan Liu and colleagues (Figshare, CC BY 4.0); its participant consent described use for non-commercial scientific research. The stimulus phases in cards 2 and 3 come from the BETA authors’ corrigendum. Wearable-102 is by Zhu F, Jiang L, Dong G, Gao X and Wang Y (Figshare version 4, CC BY 4.0); its paper describes consent for noncommercial scientific research. CBraMod’s code is MIT and its weights Apache-2.0. Filter-bank CCA follows Chen et al. (2015); no code is distributed. BCI Report’s aggregate measurements are CC BY 4.0. The dataset and model authors do not endorse these evaluations. All figures were rendered from BCI Report’s published aggregates; no EEG recordings, individual scores or trained weights accompany this article.
To cite these results: BCI Report (2026), doi:10.5281/zenodo.23123296, together with the upstream datasets. The release used here is open-register-update-20261028 (doi:10.5281/zenodo.23735101).
BCI Report, which produced these evaluations, is a personal, noncommercial project; this article uses only its published aggregate results.
Can AI Improve a 20-Year-Old Brain Decoder? A Little, and Not Everywhere was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.