Feature engineering didn't die. Engineers just stopped doing it

Somewhere around when LLMs started eating every roadmap, a quiet belief took over a lot of teams. If the model is big enough, it'll figure out the patterns on its own. Why bother hand crafting features when you can just throw raw data at something with a few hundred billion parameters.

That belief is wrong for most of what people are actually shipping. I've seen it cost real accuracy in production, not theory.

Here's the thing nobody likes admitting out loud. The algorithm you pick matters far less than the features you feed it. A plain logistic regression with smart features will often beat a fancy neural network choking on raw, messy data.

Let's take churn prediction in e-commerce, since most of us can picture it immediately.

Say you want to predict whether a customer is about to leave. The tempting move is to dump the raw table straight into the model:

df = pd.DataFrame({
    'user_id':       [101, 102, 103],
    'last_purchase': ['2024-03-15', '2025-11-20', '2025-12-28'],
    'total_orders':  [12, 1, 8],
    'signup_date':   ['2022-01-10', '2025-10-01', '2023-06-15'],
})

On paper, that looks fine. In practice, it isn't enough. If you leave last_purchase

as a naive numeric encoding, the model reads 2024-03-15

as 20,240,315

and has no idea March 2024 sits close to January 2024. It also can't tell that total_orders = 12

from someone who's been a customer for three years means something completely different from the same number on a three-month-old account. One is steady. The other is on fire.

Raw columns like this aren't features. They're just data that happens to be numeric.

Now compare it to features someone actually thought about. This follows the RFM pattern, one of the oldest tricks in e-commerce and retail ML, Recency, Frequency, Monetary:

from datetime import datetime
today = datetime.now()

df['days_since_last_purchase'] = (
    today - pd.to_datetime(df['last_purchase'])
).dt.days

df['account_age_days'] = (
    today - pd.to_datetime(df['signup_date'])
).dt.days

df['order_frequency_per_month'] = (
    df['total_orders'] / (df['account_age_days'] / 30)
)

Now the model has something to actually reason with. days_since_last_purchase

of 2 reads as active, 200 reads as gone, in one training pass. order_frequency_per_month

separates a loyal customer from a one-off buyer even when their raw order count looks identical. Add monetary_value

and a baseline-adjusted cart value, and a useless table turns into a working churn predictor.

In my experience, swapping raw columns for features like these moves accuracy more than swapping algorithms ever has. It's not even close.

LLMs aren't a replacement for any of this. They're a new way to do part of it.

An LLM can take messy text and turn it into structured signal. A review like "shipping took forever but the product was great" becomes delivery_sentiment_negative

and product_sentiment_positive

. That's still feature engineering. The LLM is the tool. It's not the thinking.

And the same rule applies in reverse. Feed an LLM a raw dump of every order, timestamp, and click, and you're making the same mistake people made with older models, assuming the model will do thinking your pipeline should have done first. Hand it days_since_last_purchase: 2, order_frequency_per_month: 4.2, recent_complaint: shipping delay

instead, and it reasons better, for the same reason a tree model does.

Most business data is still structured. Customer tables, transaction logs, inventory, support tickets. Not everything is a text blob waiting for an LLM to solve it.

On many tabular problems, models like XGBoost and LightGBM are still hard to beat. They split on real feature values, so a well-engineered column becomes a clean decision boundary instead of noise. They don't need huge datasets to find patterns. They handle mixed numeric and categorical data without much preprocessing. Feed them garbage, and they fail fast and obviously. No architecture hides a bad feature.

So next time the plan is "let the model figure it out," ask one question first. Have you actually given it anything worth figuring out?

Most accuracy problems aren't model problems. They're feature problems hiding behind a model's reputation.