Google AI Introduces TabFM: A Hybrid-Attention Tabular Foundation Model for Zero-Shot Classification and Regression

Google Research introduced TabFM, a foundation model for tabular data that performs zero-shot classification and regression without dataset-specific training. The model uses a hybrid-attention architecture combining TabPFN and TabICL, trained on hundreds of millions of synthetic datasets from structural causal models. TabFM is available on Hugging Face and GitHub, and Google BigQuery will soon expose it through an AI.PREDICT SQL command.

Google Research introduced TabFM https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/ , a foundation model built for tabular data. TabFM performs classification and regression without dataset-specific training. Every prediction comes from a single forward pass. The model reframes tabular prediction as an in-context learning problem. It is available now on Hugging Face and GitHub. TL;DR - TabFM predicts on unseen tables with no training, tuning, or feature engineering. - It reads the full dataset as one prompt, then predicts via in-context learning. - The architecture combines TabPFN-style row/column attention with TabICL-style in-context learning. - Training used hundreds of millions of synthetic datasets from structural causal models. - Google BigQuery will expose TabFM through an AI.PREDICT SQL command soon. What is TabFM? Tabular data forms the backbone of enterprise data infrastructure. Tasks like customer churn and financial fraud detection live in tables. For years, tree-based methods dominated this space. XGBoost, AdaBoost, and random forests offered robust results on structured data. Google frames TabFM as the tabular counterpart to TimesFM, its zero-shot time-series model. That reliability carried a cost. Fitting XGBoost to a new dataset is rarely one .fit call. Data scientists spend hours on hyperparameter optimization and feature engineering. They do this just to extract a reliable signal from raw data. TabFM targets exactly that bottleneck. TabFM applies the zero-shot logic that large language models made familiar. LLMs learn new tasks from in-context examples, without updating any weights. This technique is called in-context learning ICL . TabFM brings the same idea to tables. It generates predictions on previously unseen tables in one pass. How It Works Traditional models update parameters for each dataset’s distribution. TabFM skips that step entirely. It takes the whole dataset as a single unified prompt. That prompt holds both training examples and target testing rows. The model reads column and row relationships at inference time. Tables are not text. They are two-dimensional and inherently orderless. Swapping two rows or two columns does not change their meaning. Standard language models process one-dimensional, ordered sequences instead. To bridge that gap, TabFM synthesizes TabPFN and TabICL into a hybrid design. It relies on three mechanisms : Alternating row and column attention: The raw table passes through a multilayer attention module. Following TabPFN, attention alternates across columns features and rows examples . This deep contextualization captures feature interactions and dependencies. It performs work that would otherwise need manual feature crafting. Row compression: Each row’s cross-attended information compresses into a single dense vector. In-context learning: A dedicated Transformer runs over these compressed embeddings. Following TabICL, attending to compressed rows cuts computation cost sharply. Prediction stays efficient even on much larger datasets. Training On Synthetic Data at Scale Foundation models need vast, diverse data. High-quality tabular datasets are scarce in the open-source space. Industrial tables carry proprietary schemas and sensitive information. That makes them inaccessible for broad pre-training. Synthetic tables can be generated to be arbitrarily large. Google’s research team calls them effectively the only viable option at this scale. So TabFM trains entirely on hundreds of millions of synthetic datasets. These are generated dynamically using structural causal models SCMs . Each incorporates a wide variety of random functions. The approach captures distributions and complex feature relationships found in real tables. The research team reports the model generalizes well to unseen real-world data. Performance and Benchmarking The research team evaluated TabFM on TabArena. TabArena is a living benchmark that computes Elo scores from head-to-head win rates. The evaluation spans 38 classification datasets and 13 regression datasets. Sample sizes range from 700 to 150,000. Two configurations were tested. Plain TabFM runs out-of-the-box in a single forward pass. It needs no tuning or cross-validation. TabFM-Ensemble adds cross features and SVD Singular Value Decomposition features. It computes optimal weights for a 32-way ensemble using a non-negative least squares solver. For classification, it also adds Platt scaling as a calibration step. The research team reports TabFM consistently outperforms heavily tuned, industry-standard supervised algorithms. Full per-fold metrics and head-to-head win rates sit on the GitHub page. | Aspect | Traditional GBDT XGBoost | TabFM | TabFM-Ensemble | |---|---|---|---| | Per-dataset training | Required | None in-context learning | None | | Hyperparameter tuning | Extensive, manual | None | Ensemble weights via NNLS | | Feature engineering | Manual, domain-specific | Learned by attention | Adds cross + SVD features | | Prediction | After full training | Single forward pass | 32-way ensemble | | Calibration | Manual optional | — | Platt scaling classification | Getting Started: Installation and Code Installation clones the repository and installs it locally. The base install uses CPU-only JAX. A cuda extra pulls the CUDA 12 plugin and NVIDIA libraries for GPU runs. Core requirements are specific. You need Python 3.11 or later. It pins jax==0.10.1 and flax==0.12.7 , using the modern flax.nnx API. Hugging Face Hub downloads the pre-trained weights automatically. python import numpy as np import pandas as pd from tabfm import tabfm v1 0 0 from tabfm import TabFMClassifier Load pre-trained TabFM v1.0.0 downloads from Hugging Face model = tabfm v1 0 0.load scikit-learn compatible classifier clf = TabFMClassifier model=model X train = pd.DataFrame { "age": 25.0, 45.0, 35.0, 50.0 , "job": "engineer", "manager", "engineer", "manager" , "income": 80000, 120000, 90000, 130000 } y train = np.array "low risk", "high risk", "low risk", "high risk" X test = pd.DataFrame { "age": 30.0, 48.0 , "job": "engineer", "manager" , "income": 85000, 125000 } clf.fit X train, y train predictions = clf.predict X test probabilities = clf.predict proba X test print "Predictions:", predictions print "Class Probabilities:\n", probabilities Here fit prepares ordinal encoders and numerical scalers. It does not train model weights on your data. The regressor mirrors this pattern with TabFMRegressor and reg.predict . Use Cases With Examples The API fits common predictive tasks directly. For customer churn, the context holds past customers labeled churned or retained. TabFM scores churn risk for new customers in one pass. For credit risk, rows carry age, job, and income features. Labels mark low risk or high risk , as in the sample code. New applicants get scored without a training cycle. For regression, house price prediction is a natural fit. Context rows carry square footage and neighborhood. TabFM returns a predicted price for unseen listings. Interactive Explainer Check out the Repo and . Technical details https://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/ Also, feel free to follow us on and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58