Member-only story
A practical comparison covering architecture, TabArena benchmarks, latency, licensing, and runnable code. #
Gradient boosted trees have been the default choice for tabular prediction for roughly a decade. XGBoost, CatBoost and LightGBM, being the best Python libraries for these models, have dominated almost every tabular Kaggle competition and production ML system since 2016.
That status quo is now being challenged by tabular foundation models (TFMs). These are pretrained transformers, such as TabPFN (PriorLabs) and TabFM (Google), that make predictions without running gradient descent on your data at all.
On TabArena, the closest thing the field has to a standardized benchmark, the highest-performing single models by accuracy are now TFMs rather than gradient-boosted trees.
In this post, we’ll cover what’s structurally different between these two model families, and we’ll walk through both architectures in enough detail to help you reason about the trade-offs for yourself, and decide which model is best suited…