Train Once, Reuse Later: Three Tests of Skipping BCI Calibration

BCI Report's later-sessions release, published October 8, 2026, tests whether a brain-computer interface decoder trained on an earlier session still works in a later session without any labels from that later session, across three public datasets. The release reports four results, two of them on the WBCIC-SHU motor-imagery dataset, and does not pool them; WBCIC-SHU's two-class cohort of 51 people contributes 10,195 test trials and its three-class cohort of 11 people contributes 3,300, with each person's session 1 used for training and session 3 for testing. The work frames between-session reuse as dataset shift over time, distinct from the cross-subject generalization most EEG benchmarks score.

AI-assisted: this article was drafted with AI assistance and checked against BCI Report’s published results. Key results Many brain–computer interface BCI sessions begin with calibration: the user performs a few minutes of cued trials so the decoder can be refitted to today’s signals. That cost is easy to overlook in a benchmark table and hard to ignore in daily use. The obvious way to avoid it is to train a decoder once and reuse it the next time the same person puts on the cap. That is a different generalization problem from the one most EEG benchmarks score. A decoder for someone it has never seen has to cope with a new head, a new brain and a new way of performing the task. A decoder reused by the same person keeps all of that, and still meets a different recording. In machine-learning terms this is dataset shift over time Quiñonero-Candela et al., 2009 https://doi.org/10.7551/mitpress/9780262170055.001.0001 , and an unusually clean case of it: same person, same task, a later session’s data. BCI Report’s later-sessions release https://bci.report/topics/later-sessions/ , published on October 8, 2026, asks the question directly on three public datasets: does a decoder trained on an earlier session still work later, with no labels from the later session? It reports four results, two of them on WBCIC-SHU motor imagery, and does not pool them. This article walks through each one, what it does and does not show, and what it suggests for anyone building or evaluating a BCI. EEG measures microvolt potentials through electrodes resting on the scalp. Between sessions the cap sits slightly differently, electrodes shift, the hardware calibration differs, and the person is more or less alert, drowsy or fatigued Chevallier et al., 2024 https://arxiv.org/abs/2404.15319 . A review of sensorimotor BCIs frames this between-session variability as covariate shift, a change in the distribution of EEG features that stops a model’s fitted parameters from carrying over, and attributes it to changing cognitive state, motivation and the variability that comes with learning the task itself Saha and Baumert, 2020 https://doi.org/10.3389/fncom.2019.00087 . The three datasets cover three kinds of decoding: Every result follows the same rules. Each decoder is fitted on the earlier session only: the scaler, the model and, where there is one, the probability calibration. The later session contributes no labels at all. Every prediction was saved and locked before the later session’s labels were scored, and every cohort mean gives each person or, for cursor tracking, each record equal weight. The session labels need care. WBCIC-SHU’s session numbers give the recording order, not a verified gap in days. The RSVP visits are the publisher’s nominal labels Day 1, 7, 80 and 200 , not verified participant-specific intervals, so Figure 1 spaces them evenly. The three datasets, the two motor-imagery cohorts, and the two cursor-tracking cohorts and outcomes are each reported on their own: they differ in task, metric and chance level, so there is no single “later-session score.” WBCIC-SHU https://doi.org/10.1038/s41597-025-04826-y Yang et al., 2025 recorded each participant in three sessions. A two-class cohort of 51 people imagined grasping with the left or the right hand; a separate three-class cohort of 11 people also imagined a foot movement. Each person’s complete session 1 trains the decoder and session 3 is the test: 10,195 test trials in the two-class cohort and 3,300 in the three-class one. Session 2 is not used. The dataset’s own paper reports decoding only within each session, and attributes accuracy that rose from session to session to participants getting better at motor imagery. Three decoders were compared, each fitted per person on session 1 only, on the publisher’s processed data 58 channels at 250 Hz, four-second trials : None of the three uses a hyperparameter search, normalization with test-session statistics a form of test-time adaptation or calibration. Bars show participant-mean balanced accuracy the mean recall over classes for each person, averaged over people ; whiskers are 95% intervals from 10,000 participant-bootstrap draws within each cohort; dashed lines mark chance. The prior has no interval because it sits at chance for everyone. The spectral ridge carried a little information across sessions: +3.9 percentage points pp above the prior in the two-class cohort, 95% interval +2.7, +5.1 , and +4.3 pp +2.1, +6.6 in the three-class cohort. Frozen CBraMod features carried much more: +13.8 pp +11.3, +16.4 above the spectral ridge in the two-class cohort and +14.0 pp +7.5, +20.8 in the three-class cohort. These differences come from paired resampling of the same people and trials, so read them from the intervals rather than from whether the whiskers in Figure 2 overlap. The result compares two complete pipelines. It does not show that pretraining caused the gap, for three reasons the release states. There is no matched control with the same architecture and random weights. The two feature spaces differ in size and preprocessing, and the same ridge penalty does not regularize 232 and 11,600 features equally. And whether this CBraMod checkpoint saw WBCIC-SHU during pretraining is not established: the model paper describes pretraining on the Temple University Hospital EEG corpus, and its repository lists a downstream task named SHU, but neither gives a checkpoint-specific list of pretraining files. On two other BCI Report protocols, a random-weight control with a fixed frozen-ridge readout put pretrained CBraMod above the mean of three random initializations by +6.39 pp +3.06, +9.50 on motor imagery versus rest 10 people and +5.90 pp +2.31, +9.57 on mental workload 36 people does pretraining help? https://bci.report/topics/does-pretraining-help/ . Those are different datasets and splits, so they argue for running the control here, not for its outcome. Two more conditions travel with the numbers. The test set was not untouched: the CBraMod follow-up was specified after spectral-ridge results on these same trials were known. And although the session-3 labels were opened only after every prediction was locked, earlier quality checks had looked at which label values occurred and how often. The longitudinal RSVP dataset https://doi.org/10.1038/s41597-025-05378-x Zhang et al., 2025 recorded the same people at visits the publisher labels Day 1, 7, 80 and 200. BCI Report uses the 15 people of its Group A. For each person, a decoder is trained and calibrated on the first visit only, then scored unchanged on 72,000 events per later visit across the cohort, of which about 2.5% are targets. The dataset’s authors describe each participant’s average ERP as stable across the four visits, and report a baseline decoder without carrying one across days. The decoder is deliberately plain: EEG from 57 channels around each image, averaged into six 100 ms windows from 100 to 700 ms after the image 342 features , standardized, then an L2-regularized logistic regression fitted on three first-visit blocks. A fourth first-visit block fits Platt scaling Platt, 2000 https://doi.org/10.7551/mitpress/1113.003.0008 , a sigmoid that maps scores to probabilities. Dots are participant means of AUROC with 95% participant-bootstrap intervals; bars are participant means of average precision, for which no interval is published. The visits are labels, evenly spaced, with no curve fitted between them. AUROC asks how often a random target is scored above a random non-target. It stayed well above chance 0.5 at every visit and was lower at the last: Day 200 minus Day 7 was −0.035, paired 95% interval −0.065, −0.006 , and 6 of the 15 people lost 0.05 or more. Average precision summarizes the precision–recall curve, and for rare targets it is the more demanding view: a random ranking scores about the target share, 0.025 here, a reference level rather than a scored comparator Saito and Rehmsmeier, 2015 https://doi.org/10.1371/journal.pone.0118432 . It fell from 0.359 at Day 7 to 0.310 and 0.240. AUROC moved by a few hundredths while average precision lost about a third, the pattern you would expect when a few more non-targets start to outrank the targets in a stream where non-targets outnumber targets about 40 to 1. Neither measure is a precision at a chosen threshold or an online selection success rate. These probability measures are descriptive: no intervals or paired differences are published for them. The expected calibration error ECE , the gap between predicted probability and observed frequency averaged over 10 bins Pakdaman Naeini et al., 2015 https://doi.org/10.1609/aaai.v29i1.9602 , was 0.023 at Day 7 and 0.039 at Day 200, and the Brier score moved from 0.029 to 0.038. The log loss was lowest at Day 80. That mixed pattern is a reason to report calibration at every later session rather than assume the first visit’s calibration still holds; general machine-learning work has found that calibration that holds on in-distribution data often degrades under shift Ovadia et al., 2019 https://arxiv.org/abs/1906.02530 . None of this ties the decline to elapsed time. The cohort is 15 people with one fixed offline decoder, and the result describes this cohort and this decoder. As with WBCIC-SHU, the labels were not wholly unseen: RSVP event roles and target labels were inspected during quality checks, though none entered normalization, fitting, calibration or prediction. One more detail is disclosed on the result page: the dataset’s paper and sample script code 1 as non-target and 2 as target, its trigger file says the reverse, and the release followed the paper, a choice made before modeling. The Forenzo & He continuous-tracking dataset https://doi.org/10.1184/R1/25360300.v1 recorded people steering a cursor after a moving target with motor imagery, in two cohorts the publisher calls Main and Transfer Learning. That is the publisher’s name for how the cohort’s data were collected; no transfer-learning model was trained here. Results are counted per record, one participant’s data within one cohort, because BCI Report has not verified that the cohorts are disjoint. The original study retrained its deep-learning decoders on a participant’s earlier sessions as the study went on; this test deliberately does not. For each record, a spectral ridge regression is trained on the earliest complete recorded session and tested on the latest. Its features are band power in five bands from 4 to 30 Hz on 62 channels over the preceding second, 310 per row, with a new prediction every 40 ms. Two outcomes are predicted, separately: the velocity output of the decoder that was in use at recording time, as the publisher stored it, so the ridge learns to imitate the old online decoder a distillation-style target, not intended movement ; and a constructed proxy, target position minus cursor position. The comparator is a constant: the training session’s mean response, repeated for every row. Errors are an RMSE normalized by the training session’s response spread, so lower is better and 1 is roughly the size of the response’s own variation. Means are record means with 95% record-bootstrap intervals beneath; no interval is published for the ridge’s median, and no median for the constant. The ridge had a higher overall error than the constant for every record, in both cohorts and both outcomes: 9 of 9 in Main and 14 of 14 in Transfer Learning. The ridge’s median error 0.99 to 1.32 was close to the constant’s mean error 0.86 to 1.27 , but a few records carried very large errors, so its mean error is 45 to 570 times its median. No scored record was removed, clipped or winsorized. The secondary metrics are descriptive, and the every-record statement does not extend to them; no paired differences are published for them. The ridge’s record-mean R² values are all below −5,000, and its record-mean Pearson correlations with the response are small, between 0.02 and 0.07. What caused the extreme errors feature drift, sensitivity to scaling or something else is not established. This is the result where a simple baseline earns its place. Without the constant, a reader would see a decoder with a moderate median error and might call it “degraded.” With it, the decoder is worse, in overall error, than predicting the training session’s average for every record, and the mean shows how badly it can fail. Forecasting research makes the same argument for scaling errors against a naive benchmark Hyndman and Koehler, 2006 https://doi.org/10.1016/j.ijforecast.2006.03.001 . The design limits what this can say. Session, practice, tracking speed and the decoder in use at recording time are confounded, and the recorded decoder labels are observational, not randomized; the every-record result does not extend to each recorded-decoder context. The Main cohort includes 9 of its 14 candidate records, after 5 were set aside for metadata problems before any scoring; Transfer Learning includes 14 of 14. This is offline evidence about one fixed decoder, and it says nothing about the original study’s online control. None of these experiments had a calibration condition, so none says how much a few labeled trials from the later session would help. A separate BCI Report result does, on a different dataset: OpenBMI https://doi.org/10.1093/gigascience/giz002 motor imagery Lee et al., 2019 , with two classical decoders trained on each person’s session 1 and tested on the final 60 trials of their session 2 how much calibration? https://bci.report/topics/calibration-budget/ next-session . For a log-covariance decoder with shrinkage LDA, adding the first 40 labeled session-2 trials raised balanced accuracy from 67.9% to 72.4% across 51 people, +4.6 pp +2.3, +6.9 . Ten or 20 trials gave smaller gains whose intervals include zero, so those gains are not established. Even with 40 trials, 13 of the 51 people scored lower than without them, 6 by five points or more, though each person’s change rests on only 60 test trials. For a relative-band-power ridge, the 40-trial gain was +1.7 pp −0.1, +3.5 , not established. These numbers come from another dataset and protocol and are not comparable with the WBCIC-SHU figures, but they show why a calibration budget deserves its own condition: on average it helped the log-covariance decoder, and for some people it did not. Each result passed a numerical audit by a separate checker, which replayed its saved predictions, metrics and intervals. None of the audits repeated the full feature extraction from raw EEG, and for CBraMod the encoder was not rerun on all the recordings, although the checker did refit the session-1 readouts from the saved embeddings. Every interval is a pointwise 95% bootstrap interval over people or records, for cursor tracking within a cohort, conditional on the fixed trained models and predictions. Refitting, preprocessing choices, model selection and, for CBraMod, pretraining overlap are not included, and there is no adjustment for multiple comparisons. The data carry their own conditions. In WBCIC-SHU, 6 of 186 session records contain fewer trials than the nominal count 40,490 delivered against 40,500 , 3 of them in the sessions used; they are admitted under a reviewed variable-count rule, with nothing filled, truncated or excluded on performance, and why the trials are missing is unknown. WBCIC-SHU’s physical amplitude unit is unresolved, which is one reason CBraMod receives a dimensionless input rather than its native amplitude, so upstream benchmark numbers are not reproduced. Everything is offline: the publishers’ preprocessing and normalization over each whole trial window establish no real-time, closed-loop, clinical or deployment performance. Finally, every result is the same person or record . Each decoder needs that person’s labeled earlier session. Nothing here measures a decoder on someone it has never seen. Non-stationarity has been a central BCI problem for two decades. Early work measured systematic shifts between calibration and feedback data and showed, on data recorded during online sessions, that simple adaptive classifiers could significantly raise performance Shenoy et al., 2006 https://doi.org/10.1088/1741-2560/3/1/R02 . Reusing a person’s earlier sessions to skip calibration is also an old idea: “zero-training” classifiers built from previous sessions matched same-day calibration online, although they still adjusted a bias term from a short cued block recorded that day Krauledat et al., 2008 https://doi.org/10.1371/journal.pone.0002967 . A family of alignment methods attacks the shift directly, without labels from the new session. Riemannian re-centering transforms each session’s covariance matrices so that the session’s reference state sits at the identity Zanini et al., 2018 https://doi.org/10.1109/TBME.2017.2742541 , and Euclidean alignment whitens every trial with the mean covariance of its block of data He and Wu, 2020 https://doi.org/10.1109/TBME.2019.2913914 : roughly the EEG analogue of re-estimating normalization statistics for each new domain. Both are natural next conditions for a same-person, zero-label protocol like this one; neither was tested here. Benchmarks have been slower to score this setting than to describe it. The MOABB framework defines a cross-session evaluation that holds out one session per person Jayaram and Barachant, 2018 https://doi.org/10.1088/1741-2552/aadea0 , though its largest reproducibility study reports within-session results Chevallier et al., 2024 https://arxiv.org/abs/2404.15319 . A Shanghai University dataset built to study cross-session variability, 25 people recorded in five sessions, reported that a decoder trained on session 1 and tested on later sessions was not significantly better than chance in the authors’ analysis, against up to 68.8% within a session Ma et al., 2022 https://doi.org/10.1038/s41597-022-01647-1 . That is a different dataset, decoder and protocol from WBCIC-SHU, so the numbers are context, not a comparison. Outside EEG, the WILDS benchmark made the case for evaluating on held-out domains, including shifts over time, because in-distribution scores overstate performance under shift Koh et al., 2021 https://arxiv.org/abs/2012.07421 . EEG sessions give that argument an unusually clean test bed. All the later-sessions numbers in Results 1–3 come from one public JSON file the random-weight figures are on the does-pretraining-help page ; no EEG recordings or weights are needed: python import jsonfrom urllib.request import urlopencommit = "461ebfc1c8ad2a5e316c69dc74bbd23f48ee72d8"url = f"https://raw.githubusercontent.com/twu3202/bci-report/" f"{commit}/site/public/data/later-sessions-update.json" with urlopen url, timeout=30 as response: results = json.load response "results" Motor imagery: frozen CBraMod minus the spectral ridgemi = results "wbcic-frozen-cbramod" "cohorts" for cohort, row in mi.items : acc = row "frozen cbramod" "balanced accuracy" "mean" d = row "paired" "balanced accuracy difference" lo, hi = round 100 x, 1 for x in d "interval 95" print f'{cohort}, {row "people" } people: {100 acc:.1f}%,', f'difference {100 d "mean" :+.1f} pp {lo}, {hi} ' Target detection: AUROC and average precision per visitfor v in results "rsvp-later-visits" "visits" : print v "visit" , f'AUROC {v "auroc" "mean" :.3f},', f'AP {v "average precision" "mean" :.3f}' Cursor tracking: ridge mean and median against the constantct = results "forenzo-continuous-control" "cohorts" for cohort, c in ct.items : for arm, a in c "arms" .items : r = a "ridge" "primary" k = a "source mean" "primary" worse = a "paired" "records with higher ridge error" print cohort, arm print f' ridge mean {r "mean" :.1f},', f'median {r "median" :.2f};', f'constant {k "mean" :.2f};', f'worse on {worse} of {a "records" } records' The result page https://bci.report/topics/later-sessions/ , data API https://bci.report/api/ and Hugging Face dataset https://huggingface.co/datasets/Twu31/bci-report carry the secondary metrics and audit summaries. WBCIC-SHU was described by Banghua Yang, Fenqi Rong, Yunlong Xie, Du Li, Jiayang Zhang, Fu Li, Guangming Shi and Xiaorong Gao; the data are the WBCIC-SHU Motor Imagery Dataset, version 5 https://doi.org/10.25452/figshare.plus.22671172.v5 by Banghua Yang and Fenqi Rong on Figshare+, CC BY 4.0. The longitudinal ERP dataset https://doi.org/10.6084/m9.figshare.27201003.v1 is by Chen Yang, Yufeng Zhang, Hongxin Zhang, Yixuan Li, Yijun Wang and Xiaorong Gao on figshare, CC0. The continuous-tracking dataset https://doi.org/10.1184/R1/25360300.v1 is by Dylan Forenzo and Bin He on KiltHub, CC BY 4.0. OpenBMI is by Min-Ho Lee and colleagues on GigaDB https://doi.org/10.5524/100542 , CC0. CBraMod’s code is MIT-licensed and its weights Apache-2.0. BCI Report’s aggregate measurements are CC BY 4.0. The dataset and model authors do not endorse these evaluations, and this analysis is not a reproduction of any original paper’s results. All figures and tables were rendered from BCI Report’s published aggregates; no EEG recordings, individual scores or trained weights accompany this article. To cite these results: BCI Report 2026 , doi:10.5281/zenodo.23123296 https://doi.org/10.5281/zenodo.23123296 , together with the upstream datasets. BCI Report, which produced these evaluations, is a personal, noncommercial project; this article uses only its published aggregate results. Train Once, Reuse Later: Three Tests of Skipping BCI Calibration https://pub.towardsai.net/train-once-reuse-later-three-tests-of-skipping-bci-calibration-59641e38b4ae was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.