{"slug": "google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero", "title": "Google AI Introduces TabFM: A Hybrid-Attention Tabular Foundation Model for Zero-Shot Classification and Regression", "summary": "Google Research introduced TabFM, a foundation model for tabular data that performs zero-shot classification and regression without dataset-specific training. The model uses a hybrid-attention architecture combining TabPFN and TabICL, trained on hundreds of millions of synthetic datasets from structural causal models. TabFM is available on Hugging Face and GitHub, and Google BigQuery will soon expose it through an AI.PREDICT SQL command.", "body_md": "Google Research introduced [TabFM](https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/), a foundation model built for tabular data. TabFM performs classification and regression without dataset-specific training. Every prediction comes from a single forward pass. The model reframes tabular prediction as an in-context learning problem. It is available now on Hugging Face and GitHub.\n\n**TL;DR**\n\n- TabFM predicts on unseen tables with no training, tuning, or feature engineering.\n- It reads the full dataset as one prompt, then predicts via in-context learning.\n- The architecture combines TabPFN-style row/column attention with TabICL-style in-context learning.\n- Training used hundreds of millions of synthetic datasets from structural causal models.\n- Google BigQuery will expose TabFM through an AI.PREDICT SQL command soon.\n\n**What is TabFM?**\n\nTabular data forms the backbone of enterprise data infrastructure. Tasks like customer churn and financial fraud detection live in tables. For years, tree-based methods dominated this space. XGBoost, AdaBoost, and random forests offered robust results on structured data. Google frames TabFM as the tabular counterpart to TimesFM, its zero-shot time-series model.\n\nThat reliability carried a cost. Fitting XGBoost to a new dataset is rarely one `.fit()`\n\ncall. Data scientists spend hours on hyperparameter optimization and feature engineering. They do this just to extract a reliable signal from raw data. TabFM targets exactly that bottleneck.\n\nTabFM applies the zero-shot logic that large language models made familiar. LLMs learn new tasks from in-context examples, without updating any weights. This technique is called in-context learning (ICL). TabFM brings the same idea to tables. It generates predictions on previously unseen tables in one pass.\n\n**How It Works**\n\nTraditional models update parameters for each dataset’s distribution. TabFM skips that step entirely. It takes the whole dataset as a single unified prompt. That prompt holds both training examples and target testing rows. The model reads column and row relationships at inference time.\n\nTables are not text. They are two-dimensional and inherently orderless. Swapping two rows or two columns does not change their meaning. Standard language models process one-dimensional, ordered sequences instead. To bridge that gap, TabFM synthesizes TabPFN and TabICL into a hybrid design.\n\n**It relies on three mechanisms**:\n\n**Alternating row and column attention:** The raw table passes through a multilayer attention module. Following TabPFN, attention alternates across columns (features) and rows (examples). This deep contextualization captures feature interactions and dependencies. It performs work that would otherwise need manual feature crafting.**Row compression:** Each row’s cross-attended information compresses into a single dense vector.**In-context learning:** A dedicated Transformer runs over these compressed embeddings. Following TabICL, attending to compressed rows cuts computation cost sharply. Prediction stays efficient even on much larger datasets.\n\n**Training On Synthetic Data at Scale**\n\nFoundation models need vast, diverse data. High-quality tabular datasets are scarce in the open-source space. Industrial tables carry proprietary schemas and sensitive information. That makes them inaccessible for broad pre-training.\n\nSynthetic tables can be generated to be arbitrarily large. Google’s research team calls them effectively the only viable option at this scale. So TabFM trains entirely on hundreds of millions of synthetic datasets. These are generated dynamically using structural causal models (SCMs). Each incorporates a wide variety of random functions. The approach captures distributions and complex feature relationships found in real tables. The research team reports the model generalizes well to unseen real-world data.\n\n**Performance and Benchmarking**\n\nThe research team evaluated TabFM on TabArena. TabArena is a living benchmark that computes Elo scores from head-to-head win rates. The evaluation spans 38 classification datasets and 13 regression datasets. Sample sizes range from 700 to 150,000.\n\nTwo configurations were tested. Plain TabFM runs out-of-the-box in a single forward pass. It needs no tuning or cross-validation. TabFM-Ensemble adds cross features and SVD (Singular Value Decomposition) features. It computes optimal weights for a 32-way ensemble using a non-negative least squares solver. For classification, it also adds Platt scaling as a calibration step.\n\nThe research team reports TabFM consistently outperforms heavily tuned, industry-standard supervised algorithms. Full per-fold metrics and head-to-head win rates sit on the GitHub page.\n\n| Aspect | Traditional GBDT (XGBoost) | TabFM | TabFM-Ensemble |\n|---|---|---|---|\n| Per-dataset training | Required | None (in-context learning) | None |\n| Hyperparameter tuning | Extensive, manual | None | Ensemble weights via NNLS |\n| Feature engineering | Manual, domain-specific | Learned by attention | Adds cross + SVD features |\n| Prediction | After full training | Single forward pass | 32-way ensemble |\n| Calibration | Manual (optional) | — | Platt scaling (classification) |\n\n**Getting Started: Installation and Code**\n\nInstallation clones the repository and installs it locally. The base install uses CPU-only JAX. A `cuda`\n\nextra pulls the CUDA 12 plugin and NVIDIA libraries for GPU runs.\n\nCore requirements are specific. You need Python 3.11 or later. It pins `jax==0.10.1`\n\nand `flax==0.12.7`\n\n, using the modern `flax.nnx`\n\nAPI. Hugging Face Hub downloads the pre-trained weights automatically.\n\n``` python\nimport numpy as np\nimport pandas as pd\nfrom tabfm import tabfm_v1_0_0\nfrom tabfm import TabFMClassifier\n\n# Load pre-trained TabFM v1.0.0 (downloads from Hugging Face)\nmodel = tabfm_v1_0_0.load()\n\n# scikit-learn compatible classifier\nclf = TabFMClassifier(model=model)\n\nX_train = pd.DataFrame({\n    \"age\": [25.0, 45.0, 35.0, 50.0],\n    \"job\": [\"engineer\", \"manager\", \"engineer\", \"manager\"],\n    \"income\": [80000, 120000, 90000, 130000]\n})\ny_train = np.array([\"low_risk\", \"high_risk\", \"low_risk\", \"high_risk\"])\n\nX_test = pd.DataFrame({\n    \"age\": [30.0, 48.0],\n    \"job\": [\"engineer\", \"manager\"],\n    \"income\": [85000, 125000]\n})\n\nclf.fit(X_train, y_train)\npredictions = clf.predict(X_test)\nprobabilities = clf.predict_proba(X_test)\n\nprint(\"Predictions:\", predictions)\nprint(\"Class Probabilities:\\n\", probabilities)\n```\n\nHere `fit()`\n\nprepares ordinal encoders and numerical scalers. It does not train model weights on your data. The regressor mirrors this pattern with `TabFMRegressor`\n\nand `reg.predict()`\n\n.\n\n**Use Cases With Examples**\n\nThe API fits common predictive tasks directly. For customer churn, the context holds past customers labeled churned or retained. TabFM scores churn risk for new customers in one pass.\n\nFor credit risk, rows carry age, job, and income features. Labels mark `low_risk`\n\nor `high_risk`\n\n, as in the sample code. New applicants get scored without a training cycle.\n\nFor regression, house price prediction is a natural fit. Context rows carry square footage and neighborhood. TabFM returns a predicted price for unseen listings.\n\n**Interactive Explainer**\n\nCheck out the** Repo **and\n\n**.**\n\n[Technical details](https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/)**Also, feel free to follow us on**\n\n**and don’t forget to join our**[Twitter](https://x.com/intent/follow?screen_name=marktechpost)\n\n**and Subscribe to**\n\n[150k+ML SubReddit](https://www.reddit.com/r/machinelearningnews/)**. Wait! are you on telegram?**\n\n[our Newsletter](https://www.aidevsignals.com/)\n\n[now you can join us on telegram as well.](https://t.me/machinelearningresearchnews)Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? [Connect with us](https://forms.gle/wbash1wF6efRj8G58)", "url": "https://wpnews.pro/news/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero", "canonical_source": "https://www.marktechpost.com/2026/07/01/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero-shot-classification-and-regression/", "published_at": "2026-07-01 07:48:59+00:00", "updated_at": "2026-07-01 08:04:11.786327+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "ai-products", "ai-infrastructure"], "entities": ["Google Research", "TabFM", "Hugging Face", "GitHub", "Google BigQuery", "TabPFN", "TabICL", "TimesFM"], "alternates": {"html": "https://wpnews.pro/news/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero", "markdown": "https://wpnews.pro/news/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero.md", "text": "https://wpnews.pro/news/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero.txt", "jsonld": "https://wpnews.pro/news/google-ai-introduces-tabfm-a-hybrid-attention-tabular-foundation-model-for-zero.jsonld"}}