# Nine Algorithms Later, I Stopped Learning Algorithms > Source: > Published: 2026-09-02 05:29:21+00:00 *Classic Machine Learning Through the Eyes of an SRE, Part 10: The Finale What nine algorithms taught me to ask about the tenth.* **Ten parts later, I stopped trying to learn algorithms.** That sounds like a strange conclusion for a series about learning classic machine learning, but somewhere between the first supervised model and the last unsupervised one, the goal changed. I was no longer trying to remember how nine algorithms worked. I was trying to build a way of looking at an unfamiliar one. That distinction matters to me because I came to ML from SRE and DevOps, where I am used to asking a different set of questions. What does the system assume? Where can it fail? What happens when the input changes? Can I reproduce the result? What does the system do when it meets the real world at 3 AM? I wanted to see whether the same thinking could work for machine learning. It did. **Three slots I kept mixing together** Across the series, I kept coming back to three questions: what shape does the algorithm assume the world has, what does it consider wrong, and how does it go looking for a less-wrong answer? In ML terms, those are roughly the hypothesis, the loss, and the optimization. They sound separate when you write them down. I still managed to mix them up repeatedly. On SVM, I initially described gradient descent as the loss. It is the optimizer. The loss is hinge loss with L2 regularization. In the same pass, I treated convexity like a tunable knob, when it is actually a property of the optimization problem. The knobs are things like C and gamma. Then I made a similar mistake with logistic regression by reaching for squared error, which belongs to linear regression rather than the usual log-loss objective for logistic classification. Three articles, three versions of essentially the same confusion. I did not see the pattern while I was writing them individually. I saw it only when I put the algorithms next to each other. That changed how I approached the rest of the series. Instead of starting with the algorithm name and trying to memorize its mechanics, I started filling those three slots one at a time. Once they were clear, the unfamiliar algorithm became much easier to reason about. **Five ways an algorithm goes looking for an answer** The optimization slot eventually produced another pattern. These labels are my own shorthand, not standard ML terminology, but they became useful because they describe something I kept seeing. **K-Means searches.** Lloyd's algorithm repeatedly assigns points to centroids and moves the centroids, so where it starts can influence where it ends. **DBSCAN defines.** It does not optimize a conventional objective. It defines a core point through eps and minPts, then builds density-connected clusters from that definition. **PCA solves.** It can use eigendecomposition to find principal directions directly, rather than relying on an iterative local search. **Hierarchical clustering builds.** It starts with individual points and repeatedly makes greedy, irreversible merges. There is no grand global search that goes back and reconsiders everything. **Isolation Forest builds through randomness.** Its trees use random partitions, and the model uses how quickly observations become isolated as its signal for anomaly detection. Those are five very different ways of getting to an answer. This was roughly where I stopped memorizing the algorithms individually. I started thinking about the kind of machinery each one uses to arrive at its result. **Loss has shapes too** The loss column developed a similar structure. Some algorithms explicitly minimize something. Logistic regression minimizes log loss. SVM works with hinge loss plus regularization. K-Means minimizes within-cluster squared distances, commonly expressed as inertia. DBSCAN has no loss function in the conventional sense because it is applying a density definition rather than optimizing an objective. PCA is about preservation. It looks for directions that retain as much variance as possible rather than minimizing prediction error against a target. Hierarchical clustering is more conditional. There is no single loss function governing the entire hierarchy. The merges are greedy and local. Under Ward linkage, however, each merge is chosen to minimize the increase in within-cluster variance, specifically under Euclidean distance. Isolation Forest is different again. It uses randomized partitioning and path length to measure how easily a point can be isolated. The distinction became more interesting to me as I went through the series because of what it says about human judgment. The less an algorithm optimizes, the more the human setup becomes the intelligence. With DBSCAN, someone has to decide what counts as dense enough. With PCA, someone is making the assumption that the variance being preserved represents something useful. With Isolation Forest, contamination represents an assumption about how much of the population should be anomalous. There is no universal business rule hidden inside the algorithm that can choose it for you. That is where machine learning started feeling surprisingly familiar to me. In production systems, the configuration around the software is often just as important as the software itself. **The properties that look like bugs** The parts of the last few articles that stayed with me were often not the equations. They were the properties that looked wrong until I understood why they weren't. PCA components, for example, are defined only up to sign. A refit can return a component with its direction flipped while explaining exactly the same amount of variance. Nothing crashes. The metrics can look fine. Yet if someone is comparing component directions across model versions, the change suddenly matters. Hierarchical clustering has a similar trap. The left-to-right order of leaves in a dendrogram is not itself a similarity measure. Either subtree can be placed on either side without changing the underlying hierarchy, so the same clustering can have multiple valid drawings. Merge height carries the meaningful distance information, but humans naturally look at the diagram and start interpreting its horizontal arrangement. Isolation Forest gave me another example. Its trees commonly use relatively small subsamples, around 256 observations per tree in implementations such as scikit-learn, rather than growing each tree from the entire dataset. That can look like an obvious compromise until you understand that isolation is relative. A small subsample is part of the method, not simply a shortcut for faster training. These examples changed what I look for. If a property feels like a bug but isn't, I want to understand it before I try to "fix" it. Those are also the properties most likely to cause trouble in production, because they are the ones people don't think to monitor or explain. **The tenth algorithm** Gradient boosting wasn't part of this series. But after nine algorithms, I can look at it without immediately reaching for a tutorial. I can make a first-pass prediction about how I expect it to work and where I would investigate first. The hypothesis is familiar because the base learner is commonly a decision tree. So I would expect the model to represent relationships through regions and splits rather than smooth global functions. That puts it in territory I already explored with decision trees and related models. The optimization is where it starts to separate from random forests. Random forests grow trees independently and combine their results. Boosting builds trees sequentially, with later trees responding to what earlier trees got wrong. That immediately gives me a production question: what does that sequential dependency mean for training time, parallelism, and scaling? The loss is another place I would look closely. Boosting typically builds an additive model by minimizing a differentiable loss through successive corrections. So although the base learner may be a tree, the way the overall model is trained is fundamentally different from simply growing many independent trees and averaging them. Then there is the failure hypothesis I would test first. A random forest can reduce variance by averaging many independently trained trees. With boosting, adding more stages is not automatically harmless because the later stages continue trying to correct what came before. I would therefore investigate early stopping, learning rate, tree depth, and the model's behavior as the number of boosting stages increases. I have not verified that last part as a claim about a specific implementation. It is a hypothesis I would test. And that distinction is exactly why the previous nine mattered. I don't need to pretend I already know the tenth. I need to know where to start looking. **Production takeaway** The most useful lesson from these nine algorithms was not an algorithm at all. It was that mathematical correctness and production usefulness are different things. DBSCAN can fail deterministically. That doesn't make the system reliable. It just makes the failure reproducible. A flat CSAT score of 2 out of 10 has zero variance, so a variance-based technique can treat it as uninteresting while the business sees a serious problem. The model isn't wrong about the statistic. The statistic simply isn't the same thing as the business signal. **Common interview mistake** The common mistake, given how I now think about these questions, is treating "Do you know X?" as a yes-or-no question about whether you have completed a tutorial. I would rather say, "I haven't studied it deeply yet, but I know the questions I need to answer." For an unfamiliar algorithm, I can start with the hypothesis, loss, and optimization. Then I can ask what assumptions it makes, what can make those assumptions fail, and what its behavior means for a production system. That doesn't replace actually learning the algorithm. It gives me a much better way to learn it. **Where I'd use this in a real production system** I would use this framework in model selection and design reviews. When someone proposes an algorithm, a short discussion around its assumptions, objective, optimization strategy, failure modes, and operational behavior can expose a mismatch before the model becomes part of a pipeline. That is especially useful when the conversation is otherwise dominated by algorithm names. "Why gradient boosting?" "Why PCA?" "Why clustering?" Those questions are less useful than asking what problem shape we believe we have, what we consider a useful result, and what happens when reality disagrees with our assumptions. The algorithm comes after that. **What nine algorithms actually taught me** I said at the end of Part 4 that I did not want to leave this series knowing four algorithms. I wanted to leave knowing what to ask when I met a fifth. I think nine was enough to prove the idea. I started this series trying to learn classic ML from the perspective of someone who had spent most of his career thinking about production systems, reliability, incidents, automation, and failure. I expected the biggest challenge to be the mathematics. It wasn't. The harder part was learning to stop looking at each algorithm as an isolated thing to memorize and start looking for the decisions underneath it: what it assumes, what it optimizes, what it ignores, and what happens when those choices meet messy real-world data. That is also why I am comfortable ending the series without pretending I now know every important algorithm. Gradient boosting is still the tenth algorithm. There will always be an eleventh. The useful thing I have now is not a finished list. It is a way to approach the next one. Classic ML through the eyes of an SRE ends here. The learning doesn't.