{"slug": "epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara", "title": "EpiGauge: A benchmark for T-cell immunogenicity prediction from Norvara Therapeutics", "summary": "Norvara Therapeutics released EpiGauge, a benchmark for comparing T-cell immunogenicity predictors, to test whether such models can prioritize AAV capsid peptides for experimental follow-up. In discussion of the release, Norvara said EpiGauge's contamination census flags exact (allele, peptide) deposit containment as Tier 2 and treats everything outside documented pools as Tier 0 with an explicit caveat that it does not certify independence. Norvara reported that stripping deposit overlap left 441 independent rows across four small arms, with minimum detectable effects of roughly 0.11–0.34 that were too small to separate models.", "body_md": "We’re Norvara Therapeutics, and we’ve released EpiGauge to help researchers compare T-cell immunogenicity predictors. We’re interested in whether these models can help prioritize AAV capsid peptides for experimental follow-up.\n\nIf you’ve benchmarked peptide predictors, how do you check for overlap between a model’s training data and your test peptides when the training set is only partly documented? We’d welcome feedback on the evaluation and suggestions for baselines.\n\n[GitHub repository](https://github.com/Norvara-Therapeutics/epigauge)  · [Norvara Therapeutics](https://norvaratx.com/)\n\n \n \n[nootxlm](https://discuss.huggingface.co/u/nootxlm)\n2\n \nWith a partly documented training set you cannot prove zero overlap, so bound it instead: search every test peptide against all documented training sequences for exact matches and near neighbours, then report each model’s score by distance to its nearest documented neighbour — the far band is the only estimate clean of documented overlap, and the near band shows how much of the headline could be leakage. Whatever portion of training stays undocumented remains an explicit unknown in the result, not a silent zero — if the far-band ranking matches the headline ranking, the comparison holds; if it flips, overlap was carrying it. If the far band is too thin to rank on, report that as a result of its own: it means the documented portion sits too close to the test set to clear the comparison either way.\n\n \n \nThanks—that framing fits directly into how we’re thinking about EpiGauge’s contamination census. Right now we flag exact `(allele, peptide)` deposit containment (`Tier 2`) and treat everything outside documented pools as `Tier 0` with an explicit caveat that it doesn’t certify independence, so stratifying `Tier 0` by minimum sequence distance to documented training sets (and keeping undocumented training volume explicitly uncertified rather than assumed zero) is a clean way to separate verbatim/homology leakage from genuine generalization. For AAV capsids in particular, we expect conserved viral motifs to push most peptides into the near band and leave a thin far band—which matches what we saw when stripping deposit overlap down to 441 independent rows across four small arms (MDEs of ~0.11–0.34, too small to separate models)—so reporting an underpowered far band as a null/inconclusive result rather than over-interpreting noisy rankings is a great guard to build in.\n\n \n \nThat’s a useful way of looking at it. Stratifying by sequence similarity rather than treating all non-overlapping peptides equally could provide a more realistic picture of generalisation, particularly when the training data is incompletely documented.\n\nThe point about an underpowered far band is especially important — a model ranking that changes depending on the subset could easily lead to misleading conclusions if the sample size isn’t considered.\n\nWe’ll take this into account as we continue developing EpiGauge. Thanks for the thoughtful feedback.", "url": "https://wpnews.pro/news/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara", "canonical_source": "https://discuss.huggingface.co/t/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara-therapeutics/191824#post_4", "published_at": "2026-11-22 22:13:30+00:00", "updated_at": "2026-11-22 22:46:20.457334+00:00", "lang": "en", "topics": ["ai-research", "machine-learning", "ai-safety"], "entities": ["Norvara Therapeutics", "EpiGauge", "nootxlm", "GitHub", "Hugging Face"], "also_reported_by": [], "takeaway": "Stratify test peptides by minimum sequence distance to documented training sets and report an underpowered far band as inconclusive; after removing deposit overlap, EpiGauge retained 441 independent rows across four arms with minimum detectable effects of ~0.11–0.34, too small to separate models.", "recommendation": {"advice": "When training data is only partly documented, bound overlap by stratifying test peptides by distance to nearest documented training sequence and report thin far bands as inconclusive rather than ranking models on them.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara", "markdown": "https://wpnews.pro/news/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara.md", "text": "https://wpnews.pro/news/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara.txt", "jsonld": "https://wpnews.pro/news/epigauge-a-benchmark-for-t-cell-immunogenicity-prediction-from-norvara.jsonld"}}