LLM limits on structured data are getting documented clearly
The thread on 'Why Large Language Models Fail at Tabular Prediction' got a sharp comment that reframes the whole question: it might not be that LLMs fail at tabular data, it might be that gradient-boosted trees are just extraordinarily good at it. That is a more useful frame for builders than 'LLMs are bad at tables.'
This connects to the broader 'LLMs reward expertise' thread, which argues that the people getting the most out of LLMs are the ones who already understand the domain well enough to know when the model is wrong. For tabular prediction, that expertise means knowing that XGBoost or LightGBM will almost certainly outperform an LLM on structured data tasks.
The pattern: LLM capability is real but domain-specific. The mistake is treating them as general-purpose replacements rather than tools with specific strengths. The people winning with LLMs are the ones who know the difference.
So what?
If you are building a product that uses LLMs for data analysis, classification, or prediction on structured data, benchmark against a gradient-boosted tree before you ship. You will likely lose that comparison, and your users will notice. Match the tool to the task rather than defaulting to whatever model is easiest to call.