{"slug": "on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for", "title": "On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series", "summary": "Researchers at arXiv introduced PathoFM, an encoder-centric transformer for clinical time-series learning that uses three complementary pretraining objectives—Local Completion, Temporal Continuity, and Unsupervised In-Context Dynamics—to learn generalizable representations from multivariate gait windows in spinal cord injury patients. The study found that dynamics-centric mixtures of objectives produce the most balanced transfer across classification and regression tasks, outperforming grouping or reconstruction-only approaches. This work addresses the challenge of learning from small, heterogeneous clinical cohorts by identifying inductive biases that enable representations to generalize across task types and subjects.", "body_md": "arXiv:2605.26194v1 Announce Type: new\nAbstract: Clinical time-series learning is routinely constrained by small, heterogeneous cohorts and protocol drift, while its downstream use spans both classification (e.g., pathology diagnosis) and regression (e.g., temporal forecasting). These constraints make foundation-model pretraining appealing, but raises an important question of which inductive biases should the pretraining objective impose so that representations transfer across task types and subjects. We study this question in pathological gait analysis for spinal cord injury (SCI) via PathoFM, an encoder-centric transformer pretrained on multivariate gait windows with three complementary objectives: Local Completion (reconstruct contiguous masked spans to enforce local structure), Temporal Continuity (predict a masked mid-horizon continuation from an observed prefix to enforce smoothness and causal consistency), and Unsupervised In-Context Dynamics (support-query reconstruction conditioned on subject exemplar windows via attention). Empirically comparing objective families (grouping/contrastive, dynamics-based, and generative reconstruction), we find that dynamics-centric mixtures produce the most balanced transfer: grouping objectives favor discriminative margins but can degrade magnitude fidelity needed for continuous targets, whereas reconstruction-only objectives preserve waveform structure but may underperform on classification. Overall, combining local reconstruction with temporal continuity, and adding in-context conditioning when exemplar access is realistic, yields robust subject-generalizing representations.", "url": "https://wpnews.pro/news/on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for", "canonical_source": "https://arxiv.org/abs/2605.26194", "published_at": "2026-05-27 04:00:00+00:00", "updated_at": "2026-05-27 04:29:42.503533+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence", "ai-research"], "entities": ["PathoFM", "SCI"], "alternates": {"html": "https://wpnews.pro/news/on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for", "markdown": "https://wpnews.pro/news/on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for.md", "text": "https://wpnews.pro/news/on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for.txt", "jsonld": "https://wpnews.pro/news/on-the-role-of-inductive-bias-in-time-series-pretraining-a-case-study-in-for.jsonld"}}