{"slug": "decision-trees-aren-t-trained-they-re-grown", "title": "Decision Trees Aren't Trained. They're Grown.", "summary": "An SRE engineer's exploration of decision trees reveals that unlike logistic regression, decision trees are grown through local optimization rather than global gradient descent, making them intuitive for operational systems but unstable to data changes. The engineer advises versioning trees like configuration and diffing their structure between retrains to detect shifts in explanations.", "body_md": "**Classic Machine Learning Through the Eyes of an SRE — Part 2**\n\nThe second algorithm I studied broke everything I'd just learned from the first.\n\nLogistic regression taught me that training means gradient descent: guess, measure error, adjust the weights, repeat until convergence. So when I opened decision trees, I went looking for the optimizer.\n\nThere wasn't one.\n\nA decision tree isn't optimized the way I expected. It's grown.\n\nAt each step it finds the locally best split, commits to it, and recursively repeats the process. No backtracking. No second chances. There is optimization happening — each split minimizes impurity — but only locally, one step at a time. Finding the globally optimal tree is NP-hard, so the algorithm doesn't even try.\n\nThat felt surprisingly familiar.\n\nIn incident response or capacity planning, we rarely know the perfect answer. We make the best decision with the information we have, knowing a different first choice might have led somewhere else. Decision trees simply turn that idea into an algorithm.\n\n**The bet a tree makes**\n\nEvery machine learning algorithm makes a different bet about the world.\n\nLogistic regression assumes relationships are smooth. Risk gradually increases as signals change.\n\nDecision trees make the opposite assumption. They assume the world is made of boxes.\n\nA project isn't slightly riskier because velocity drops. It's risky when several conditions happen together: a fixed-price contract, a new account manager, and a month-end delivery. Inside that box, projects fail. Outside it, they're usually fine.\n\nThis is exactly how many operational systems work. Severity matrices, routing rules, escalation policies, approval workflows — they're all collections of decision boxes.\n\nThat's why trees immediately felt intuitive to me.\n\n**The hidden cost of flexibility**\n\nTrees make very few assumptions about the data. That sounds like an advantage.\n\nThe price is instability.\n\nChange a small part of the training data and the first split can change. Since every later split depends on that first decision, the entire tree can be completely different after retraining.\n\nSame data. Different explanation.\n\nI actually made this mistake while learning. My first notes said that because trees make fewer assumptions, they must be more stable.\n\nExactly backwards.\n\nFewer assumptions mean more freedom to fit whatever the sample contains. More freedom means higher variance. I had confused flexibility with reliability.\n\n**What this looks like in production**\n\nImagine retraining a churn model every month.\n\nLast month the first split said ticket volume is the biggest predictor. This month it says response time is.\n\nThe model might perform equally well. But if people treat the tree as an explanation rather than just a prediction, you've just changed the organization's understanding of reality.\n\nThat's why I would version decision trees the same way we version configuration.\n\nDon't just monitor accuracy. Diff the structure. If the explanation changes dramatically between retrains, someone should know why.\n\n**Strategy transfers. Mechanics don't.**\n\nThis algorithm taught me something more useful than decision trees.\n\nSome things transferred directly from logistic regression: frame the business problem first, understand the cost of false positives and false negatives, watch for data leakage, and treat turning predictions into actions as a business decision. Those are strategies.\n\nOther things didn't transfer at all — gradient descent, differentiable loss, model coefficients. Those are mechanics.\n\nSince then, every time I learn a new algorithm, my first question is: what assumptions is this algorithm making about the world?\n\nThe answer usually predicts how it learns — and how it eventually fails.\n\n**What I'd tell my SRE team**\n\nUse a single decision tree when the model is the runbook.\n\nIf someone needs to explain every decision to a customer, auditor, or compliance reviewer, it's hard to beat a tree.\n\nJust remember that the explanation itself is unstable. Version it. Diff it. Treat changes like configuration changes.\n\n**Production takeaway**\n\nA decision tree is the model that is the runbook. Use it when humans need to read, audit, and defend every decision path. Monitor the structure — not just the accuracy.\n\n**Common interview mistake**\n\nTwo answers I now know are wrong: \"decision trees are trained using gradient descent,\" and \"decision trees are more stable because they make fewer assumptions.\" Neither is true. Trees are grown greedily, and fewer assumptions usually mean higher variance, not greater stability.\n\n**Where I'd use it**\n\nTicket triage · escalation routing · customer churn explanations · compliance workflows · any system where the business needs to understand why the model made a decision.\n\n*Part 2 of \"Classic Machine Learning Through the Eyes of an SRE\" — a decade in production, now moving into AI platform engineering, documenting it in public. Next: random forests, the first algorithm that felt like distributed systems disguised as machine learning.*", "url": "https://wpnews.pro/news/decision-trees-aren-t-trained-they-re-grown", "canonical_source": "https://dev.to/nishant_banginwar_80b7dc5/decision-trees-arent-trained-theyre-grown-fhe", "published_at": "2026-08-04 03:16:57+00:00", "updated_at": "2026-08-04 03:52:07.943141+00:00", "lang": "en", "topics": ["machine-learning", "mlops", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/decision-trees-aren-t-trained-they-re-grown", "markdown": "https://wpnews.pro/news/decision-trees-aren-t-trained-they-re-grown.md", "text": "https://wpnews.pro/news/decision-trees-aren-t-trained-they-re-grown.txt", "jsonld": "https://wpnews.pro/news/decision-trees-aren-t-trained-they-re-grown.jsonld"}}