150,000 rows of data, where will I store it all?!
You don't need to. Just sample 1% of data, let the model do the feature engineering, ask model model to replicate everything in Pandas (or R, or whatever else), so you can run it for a full dataset.
TabFM: A zero-shot foundation model for tabular data
11–16 of 16 posts
Re: TabFM: A zero-shot foundation model for tabular data
#12[deleted]
Re: TabFM: A zero-shot foundation model for tabular data
#13Earlier quoted context omitted.
GitHub Repo: Please see the results folder Results folder: Here's some undocumented parquet files Definitely feels like they're hiding the ball lol. If they had good benchmarks they'd talk about them. Not comparing to tuned xgboost is also a warning sign.
wouldn't xgboost be covered under autogluon? not perfect, but not missing either
Honestly, I don't really know AutoGluon, if this does xgboost tuning that's good.
I do still think ELO scores are still a way to obscure results though. For all we know this is like 0.1% better than a "normal" approach on like 70% of tasks and a tire fire on others.
Re: TabFM: A zero-shot foundation model for tabular data
#14Earlier quoted context omitted.
wouldn't xgboost be covered under autogluon? not perfect, but not missing either
Honestly, I don't really know AutoGluon, if this does xgboost tuning that's good. I do still think ELO scores are still a way to obscure results though. For all we know this is like 0.1% better than a "normal" approach on like 70% of tasks and a tire fire on others.
AutoGluon does their own benchmarking, the default is Elo, but you can switch to other metrics:
https://huggingface.co/spaces/TabArena/leaderboard
xgboost is listed, they say "tuned" but who knows what that means. and its below CatBoost and LightGBM
as an aggregator of models (trees, neural networks, clustering, ...) AutoGluon doesn't really have a dog in this fight
Re: TabFM: A zero-shot foundation model for tabular data
#15In hindsight, the Prior Labs exit to SAP couldn't have been timed better.
They don't show the strongest TabPFN variants in the plot unfortunately, i.e. thinking and ensembled. Not really apples to apples.
Re: TabFM: A zero-shot foundation model for tabular data
#16Earlier quoted context omitted.
Honestly, I don't really know AutoGluon, if this does xgboost tuning that's good. I do still think ELO scores are still a way to obscure results though. For all we know this is like 0.1% better than a "normal" approach on like 70% of tasks and a tire fire on others.
AutoGluon does their own benchmarking, the default is Elo, but you can switch to other metrics: https://huggingface.co/spaces/TabArena/leaderboard xgboost is listed, they say "tuned" but who knows what that means. and its below CatBoost and LightGBM as an aggregator of models (trees, neural networks, clustering, ...) AutoGluon doesn't really have a dog in this fight
Thanks for the pointer, though I can't seem to find TabFM and AutoGluon at that link?