TL;DR
Yes, foundation models really do top the tabular leaderboard now, and on TabArena’s benchmark they beat a well-tuned tree-based algorithm more than nine times in ten. My own runs agreed, until I gave all training records to the tree-based algorithms. Then the edge disappeared: four datasets, four ties, one of them level to the last decimal. The real upside is development time, with nothing to tune and no pipeline to maintain, but you pay for it twice over: serving costs jump by orders of magnitude, and the best model I tested is research-only, so shipping it is off the table. My rule of thumb (for now anyway): under a thousand rows with no team, reach for the foundation model, but once you have forty thousand rows, just use traditional ML.
Every tabular project I have worked on has the same thing underneath. A feature pipeline only the people who built it understood, a hyperparameter search that ran overnight, a retraining schedule someone set once and never looked at again, or no retraining at all, and a model registry with four versions in it so someone could say the team was doing MLOps.
So when someone tells me a pretrained model can classify a table it has never seen, with nothing to fit and nothing to retrain, that pitch lands hard. It is a pitch about deleting most of the project, and only incidentally about accuracy. I wanted to know whether it survives real data. It half does, and the half that fails is the half nobody mentions.
TabArena's maintainers integrated TabFM themselves in July and re-ran it on their own infrastructure. It sits top of their individual-model leaderboard. Their published head-to-head win rates put it ahead of tuned and ensembled XGBoost 92.6% of the time, LightGBM 90.8%, CatBoost 90.3%. Their baselines are properly tuned, which defuses the usual objection before anyone raises it.
So at leaderboard scale the accuracy question is closed. What happens once tree-based algorithms gets all its training records is the separate question. That part is still open.
Four real datasets from OpenML, all four appearing by name in TabArena's suite. German credit risk, FICO home equity lines, Portuguese bank telemarketing, and Orange telecom customer records. Credit, lending, marketing, and churn data: the four areas most data scientist team is usually asked to predict.
I kind of threw out my first dataset. It was a synthetic Kaggle table where one column scored 0.9589 on its own, and every model landed between 0.98 and 0.99. A dataset with no headroom cannot rank anything, so it became the runnability check instead.
Seven models on identical rows. TabFM, TabICL, TabDPT, then XGBoost, LightGBM, CatBoost and a logistic regression. Every model on a dataset sees exactly the same split, which is what makes the comparison paired rather than seven unrelated experiments.
Two models could not take part, and both reasons are worth stating. TabPFN refuses to download its weights without an interactive licence acceptance, so it cannot be installed unattended and cannot go into CI. Nori is regression only: its API defines Task = Literal["regression", "reg"] and the package contains no reference to classification at all.
Cost logged as a first-class column, not a footnote. Prediction seconds per thousand rows, peak memory, on the same hardware for every model.
Paired bootstrap, 5k shared resamples, Holm correction within each dataset, control set to the strongest classical model on that dataset. The bootstrap measures test-row noise on this one split, not variation across partitions or training seeds.
A two-point drop in AUC represent substantial financial impact on credit book, remaining far below the performance suggested by the leaderboard.
On German credit dataset, TabDPT posts the biggest lead in the whole study, so it looks like a clear winner, however, it is actually the weakest. German credit has only about a thousand rows, and on that little data a lead can be luck as easily as skill. There is not enough there to say the model is really better. And German credit is the dataset people always reach for to argue that foundation models win on small data. It turns out to be too small to prove even that.
The telecom dataset tells the same story. TabFM has the top score there, but the thing it is trying to predict almost never happens, so the test set holds only a few dozen real cases. With that little cases to check against, no score means much, and the top one proves nothing.
Every model above was trained on 700 or 1,000 rows. That cap exists for a reason: TabFM's cost grows with your training set multiplied by your query set, and larger runs were not affordable.
Tree-based algorithms have no such constraint. So I ran the same LightGBM on every training records outside the held-out split, scored on identical test rows.
Read the bank marketing row. LightGBM climbs to match with TabFM exactly. That looked at first like a hair’s-breadth win for the tree. It is not. It is the same score, printed as 0.9421 in one table and 0.9420 in the other, so I ran the paired bootstrap to be sure.
The bootstrap confirms it. On all four datasets, once the tree has its full data (hereby I mean training dataset), no model wins by a margin. The closest anything came to a real gap was German credit, and even that fell just short.
And the tree matches it using data the business already owns, so getting there costs nothing extra. Running it costs almost nothing either. The tree makes a thousand predictions in about a hundredth of a second. The foundation model takes over an hour to make the same thousand.
Telecom is the same story, with the tree now the one in front. Give it all its data and it pulls more than five points ahead of the foundation model, and it still cannot prove the lead is real. The test set holds only 53 examples of the rare event it is trying to predict, and 53 is not enough to settle anything.
None of this makes the earlier wins fake. At a thousand rows the foundation model really did win, and some real problems only ever hand you a thousand rows. But the win lives in one narrow place, small data. And the data was small here only because the foundation model could not afford more, not because the problem was small. Give the tree the rest, and the win is gone.
Most model selection and leaderboard focus on quality and stops. There are three axes: time, cost and quality, and anyone who has worked on data science or any project knows you can rarely win all three. Pretending the other two do not exist is how a proof of concept becomes a production costs nobody forecast, and sometimes a failure.
Quality. Foundation models take it, but only narrowly: two points of AUC, significant on two of the four datasets. Honest, and smaller than advertised.
Time, and it splits in two. On development time the foundation model wins outright, and this is the part I am not going to downplay. There is no hyperparameter search and no feature engineering loop. Nothing retrains on a schedule, and there is no model registry to keep straight. fit returns in under half a second because it does not train, it label-encodes your target, checks you against a ten-class ceiling, fits an encoder, and builds 32 shuffled copies of your data to use as a prompt. When your data changes you swap the prompt. For a team of one with a deadline, that is a genuine saving, and it is the only axis the technology takes.
On serving time it loses badly. The same three thousand customers a tree scores instantly take TabFM more than seven hours.
Cost. On a CPU, a single prediction from the foundation model costs far more than one from a tree. The tree is lighter on memory too, under a gigabyte against ten to fifteen for the foundation model.
Then there is the cost with no price on it. TabFM's weights are non-commercial. The best model in my study is the one a business cannot use, and TabArena's leaderboard does not have a column for that.
Two more things matter once you have to ship it, and neither shows up in an accuracy score.
Risk. A few things about TabFM would worry me in production. When you load it, it pulls the latest weights rather than a pinned version, so the model can quietly change between one run and the next. It can only handle up to ten classes. And version 1.0.1 carries a guard against a data-type trap that would otherwise mishandle some columns with no error at all. You find these in an incident review, never on a leaderboard.
Cost credibility. Google describes TabFM as making a prediction in one pass. The version you install actually runs thirty-two copies and averages them. I tried both. The default costs about thirty-two times as much and gains almost nothing, roughly 0.002 in accuracy, which is lost in the noise. So any speed figure quoted for a single pass is off by thirty-two times from what really ships.
The most useful thing I found was not about foundation models against trees. It was about foundation models against each other.
On the telecom dataset:
TabFM and TabICL produced that gap on identical rows. Calling them one category tells you nothing.
The reason is how they are built. TabFM feeds your entire training table into the model on every single prediction. TabDPT instead looks up a handful of the most similar rows and uses only those. Same idea, opposite cost. That is why TabDPT scores those three thousand telecom customers in about two-thirds of a second, while TabFM takes over seven hours.
TabDPT is also the easy one to actually use. Its licence lets a business ship it, it never needed more than about a gigabyte of memory, and it quietly posted the top score on German credit. That lead is too small to confirm on a thousand rows, but it is still the highest number there. It is the model I would put in front of a client, and it is not the one topping the leaderboard.
Under a thousand rows, no team, nothing already built. Use one. Start with TabDPT for its licence and low cost, or TabICL if you need the highest accuracy. The tuning you skip is worth more than the two points of accuracy you gain.
Ten thousand rows or more. Put the data into a tree first. On three of my four datasets, more data beat a better model, and it cost nothing to run. Do that before you even test a foundation model, not after.
Rare events, wide tables, lots of categories. Test before you believe anything. On the telecom data, two of the three foundation models scored below plain logistic regression, the simplest baseline there is. The reason is starvation: only 1.78% of those customers convert, so a thousand-row sample carries only about eighteen real conversions, and that is all a foundation model has to learn from on each call. Give a tree the full forty-seven thousand rows and it learns from about 837 instead. Which foundation models sink and which stays afloat comes down to how each is built, not how many of those conversions it keeps.
Serving at scale. Accuracy stops being the question. Nothing here justifies four thousand seconds to score a thousand rows.
Anything commercial. Check the licence before you benchmark, not after. TabFM and TabPFN both come with weights you are not allowed to use commercially, and installing TabPFN now gives you one of those by default. A team can end up breaking the licence without ever deciding to.
My trees are untuned. I gave each one sensible fixed settings and ran no search for better ones, while TabArena tries 201 settings per model and blends the best. That is the first objection a knowledgeable reader will raise, and it is a fair one.
But it cuts the other way. Even untuned, the tree already caught TabFM once it had the full dataset. Tuning the tree would only widen that gap, not close it, so my setup is biased in the foundation models’ favour, on the two small-data wins as much as on the ties.
The rest of the honest list. One random seed, one split of the data, four datasets, one measure of quality. That measure is ROC AUC, which only asks whether the model ranks customers in the right order, so I never checked whether its probabilities are calibrated or where to set the cut-off. On a rare-event problem like telecom, a commercial team would watch precision and recall instead. Everything ran on a CPU, and these models are much faster on a GPU. That would not change the order of the cost column, since trees stay far cheaper, but it would shrink the size of the gap. Four datasets is enough to show the answer changes from one to the next. It is not enough to say which kind of dataset gives which answer.
Foundation models did to time series what they are now trying to do to tables, and there the verdict came in faster and cleaner. When I ran that comparison, they won 30 of 38 contests. Here they take the top score on all four small-data tests. But once the tree gets its full data they tie all four, and on cost they lose by a huge margin.
The difference is how much each model has to read. A time-series model reads a fixed window of history, say two thousand past points, no matter how big your company is, so the cost of each forecast stays about the same. A tabular model reads your whole training set, so its cost climbs as your data grows. That is the entire gap between the two stories, and it comes from how the models are built, not from where the field happens to be today.
Something will fix this, and TabDPT’s lookup approach already shows how. But the model at the top of the leaderboard today is not the one you can actually deploy, and the lead it wins by disappears for free the moment you hand a tree the data you already have.
One note on timing. I ran this on 29 August 2026, and this field moves fast, so what I measured today may not hold next month, or even next week. The point stands, though: plenty of leaders will assume a foundation model can do everything. You, the practitioner reading this, know better.
Choosing the model to fit the business use case is what matters.
Code, datasets, every individual number and the script that verifies them: tabular-fm-scorecard. The time-series companion is tsfm-bakeoff.
If you got something out of this, the best way to catch the next one is to follow me here on Medium or ask me any questions on LinkedIn. The Leaderboard Model You Cannot Actually Deploy was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.