# Decision Trees Aren't Trained. They're Grown. > Source: > Published: 2026-08-04 03:16:57+00:00 **Classic Machine Learning Through the Eyes of an SRE — Part 2** The second algorithm I studied broke everything I'd just learned from the first. Logistic regression taught me that training means gradient descent: guess, measure error, adjust the weights, repeat until convergence. So when I opened decision trees, I went looking for the optimizer. There wasn't one. A decision tree isn't optimized the way I expected. It's grown. At each step it finds the locally best split, commits to it, and recursively repeats the process. No backtracking. No second chances. There is optimization happening — each split minimizes impurity — but only locally, one step at a time. Finding the globally optimal tree is NP-hard, so the algorithm doesn't even try. That felt surprisingly familiar. In incident response or capacity planning, we rarely know the perfect answer. We make the best decision with the information we have, knowing a different first choice might have led somewhere else. Decision trees simply turn that idea into an algorithm. **The bet a tree makes** Every machine learning algorithm makes a different bet about the world. Logistic regression assumes relationships are smooth. Risk gradually increases as signals change. Decision trees make the opposite assumption. They assume the world is made of boxes. A project isn't slightly riskier because velocity drops. It's risky when several conditions happen together: a fixed-price contract, a new account manager, and a month-end delivery. Inside that box, projects fail. Outside it, they're usually fine. This is exactly how many operational systems work. Severity matrices, routing rules, escalation policies, approval workflows — they're all collections of decision boxes. That's why trees immediately felt intuitive to me. **The hidden cost of flexibility** Trees make very few assumptions about the data. That sounds like an advantage. The price is instability. Change a small part of the training data and the first split can change. Since every later split depends on that first decision, the entire tree can be completely different after retraining. Same data. Different explanation. I actually made this mistake while learning. My first notes said that because trees make fewer assumptions, they must be more stable. Exactly backwards. Fewer assumptions mean more freedom to fit whatever the sample contains. More freedom means higher variance. I had confused flexibility with reliability. **What this looks like in production** Imagine retraining a churn model every month. Last month the first split said ticket volume is the biggest predictor. This month it says response time is. The model might perform equally well. But if people treat the tree as an explanation rather than just a prediction, you've just changed the organization's understanding of reality. That's why I would version decision trees the same way we version configuration. Don't just monitor accuracy. Diff the structure. If the explanation changes dramatically between retrains, someone should know why. **Strategy transfers. Mechanics don't.** This algorithm taught me something more useful than decision trees. Some things transferred directly from logistic regression: frame the business problem first, understand the cost of false positives and false negatives, watch for data leakage, and treat turning predictions into actions as a business decision. Those are strategies. Other things didn't transfer at all — gradient descent, differentiable loss, model coefficients. Those are mechanics. Since then, every time I learn a new algorithm, my first question is: what assumptions is this algorithm making about the world? The answer usually predicts how it learns — and how it eventually fails. **What I'd tell my SRE team** Use a single decision tree when the model is the runbook. If someone needs to explain every decision to a customer, auditor, or compliance reviewer, it's hard to beat a tree. Just remember that the explanation itself is unstable. Version it. Diff it. Treat changes like configuration changes. **Production takeaway** A decision tree is the model that is the runbook. Use it when humans need to read, audit, and defend every decision path. Monitor the structure — not just the accuracy. **Common interview mistake** Two answers I now know are wrong: "decision trees are trained using gradient descent," and "decision trees are more stable because they make fewer assumptions." Neither is true. Trees are grown greedily, and fewer assumptions usually mean higher variance, not greater stability. **Where I'd use it** Ticket triage · escalation routing · customer churn explanations · compliance workflows · any system where the business needs to understand why the model made a decision. *Part 2 of "Classic Machine Learning Through the Eyes of an SRE" — a decade in production, now moving into AI platform engineering, documenting it in public. Next: random forests, the first algorithm that felt like distributed systems disguised as machine learning.*