{"slug": "i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers", "title": "I Added ML Anamoly Detection to My Cybersecurity Tool - Here's What the Numbers Actually Showed", "summary": "A developer added an Isolation Forest model to their Log Sentinel cybersecurity tool, which detects attacks in Apache server logs, and found that while rule-based detectors achieved perfect precision and recall on a synthetic dataset, the ML model caught all attackers but flagged five false positives. The developer argues that ML's ability to flag unusual behavior is valuable in production, where rules only catch anticipated patterns, and suggests running both detectors together.", "body_md": "A few weeks ago, I built Log Sentinel, a log analysis dashboard that detects brute force attacks, directory scanning, and error spikes in Apache server logs using rule-based detectors. The rules worked well, but I kept wondering: what would a machine learning model catch that my rules missed? And more importantly, would ML actually be better?\n\nI spent two weeks adding an Isolation Forest model to find out. The results were more interesting than I expected.\n\n**What I built**\n\nThe ML upgrade adds a second detection layer on top of the existing rule-based system. Instead of checking specific conditions like \"did this IP make 10+ failed logins?\", the Isolation Forest learns what normal traffic looks like from the data itself and flags anything statistically unusual.\n\nBefore the model can run, I transform each IP address into a 5-feature behavioural vector:\n\nrequest_count — total requests made\n\nerror_rate — fraction of requests returning 4xx or 5xx errors\n\nunique_paths — how many different URLs they visited\n\nnight_traffic_ratio — fraction of requests made between midnight and 6 am\n\navg_bytes — average response size\n\nAll five features get normalized to mean=0, std=1 using StandardScaler. Without normalization, an IP with 1000 requests would dominate an IP with 10 requests purely because of scale.\n\nThe model is configured with contamination=0.15; I tuned this by testing values from 0.05 to 0.50 and picking the one with the best F1 score on a labelled evaluation dataset.\n\n**The evaluation setup**\n\nTo measure performance properly, I generated a labelled dataset of 230 IPs: 200 normal and 30 attackers (10 brute force, 10 directory scanners, 10 error spikers). Each IP has a ground truth label so I can compute precision, recall, and F1 for both detectors.\n\nPrinting accuracy on training data proves nothing. You need labelled ground truth to know if your model actually works.\n\n**The results**\n\nMetric Rule-Based Isolation Forest\n\nPrecision 1.0000 0.8571\n\nRecall 1.0000 1.0000\n\nF1 Score 1.0000 0.9231\n\nFalse Positives 0 5\n\nFalse Negatives 0 0\n\nThe rule-based detectors achieved perfect scores. The Isolation Forest caught every single attacker (recall=1.0) but flagged 5 innocent IPs along the way (precision=0.857).\n\n**What this actually means**\n\nMy first instinct was to be disappointed that ML didn't win. But the more I thought about it, the more interesting the result became.\n\nThe rule-based detectors achieved perfect scores because the evaluation dataset contains well-defined, structured attack patterns, exactly the patterns the rules were written to catch. Of course they're perfect: I designed both the rules and the data.\n\nThe Isolation Forest's false positives tell a different story. It flagged 5 normal IPs that happened to have slightly unusual behaviour; maybe they made a burst of requests, or visited an uncommon path. The model found something statistically interesting about them even though they weren't attackers.\n\nIn a real production environment, this is actually the more valuable behaviour. Rules only catch what you anticipated. ML catches things you didn't think to write rules for.\n\nThe real insight is that both detectors belong in the same system. Rules handle known attack signatures with zero false alarms. ML handles the unknown. My dashboard now runs both and shows you exactly where they agree and where they disagree, which is where the interesting cases live.\n\n**What I'd do differently**\n\nThe biggest limitation of my evaluation is that I generated the labelled dataset myself. Real attack patterns are messier and more diverse than synthetic data. A more honest evaluation would use real labelled log data from a production environment.\n\nI'd also experiment with other unsupervised algorithms; Local Outlier Factor and One-Class SVM are worth comparing against Isolation Forest on the same dataset.\n\n**The live dashboard**\n\nBoth detectors run on every analysis. The dashboard has three views: rule-based results, ML results, and a comparison showing which IPs were caught by both, rules only, or ML only. There's also a Model Performance section showing the confusion matrices and F1 scores side by side.\n\nLive demo: atenahfr.github.io/log-sentinel/frontend/index.html\n\nSource code: github.com/atenahfr/log-sentinel", "url": "https://wpnews.pro/news/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers", "canonical_source": "https://dev.to/atenahfr/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-heres-what-the-numbers-actually-showed-4jpp", "published_at": "2026-08-15 19:25:45+00:00", "updated_at": "2026-08-15 19:41:51.692574+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["Log Sentinel", "Isolation Forest", "Apache"], "alternates": {"html": "https://wpnews.pro/news/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers", "markdown": "https://wpnews.pro/news/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers.md", "text": "https://wpnews.pro/news/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers.txt", "jsonld": "https://wpnews.pro/news/i-added-ml-anamoly-detection-to-my-cybersecurity-tool-here-s-what-the-numbers.jsonld"}}