{"slug": "i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack", "title": "I let a local 27B LLM audit and fix my Splunk + Sysmon stack", "summary": "A security analyst studying for CySA+ demonstrated that a local 27B-parameter LLM running entirely on their own GPU could audit and remediate a Splunk and Sysmon home SOC stack without any data leaving the machine. Across five audits the agent corrected the analyst's assumptions three times, diagnosed issues such as double log ingestion via a Universal Forwarder and three dead data inputs, and flagged that Sysmon was not detecting LSASS credential access. The analyst noted the model hit 97.4% of its 128K context window chasing an irrelevant detail during remediation, requiring a system-prompt rule to stop once a root cause was evidenced.", "body_md": "The question was not \"can an LLM do SOC work\". The question I actually wanted answered was narrower and harder: **can a 27B model running on my own GPU, with zero bytes leaving the machine, audit my Splunk install, find what is broken and fix it — without me telling it how?**\n\nAfter an afternoon of back and forth, the answer is yes, with caveats worth writing down. Including the mistakes, because those are the interesting part.\n\nContext first, since it shapes how you should read this: I am a security analyst **without prior SOC experience** — I am studying for CySA+, and building a home SOC is the only way I have to practise on something that behaves like a real environment. Everything here ran on my own hardware, was reviewed by the model and verified against real evidence. No third-party data, no borrowed infrastructure.\n\nThree moving parts:\n\n`llama-server` and driven by an agent.\nThe goal was not a mentor that explains things to me. It was an agent that audits the real install, finds what is wrong, and remediates it: edits configs, proposes commands, removes what is redundant.\n\nI did not hand my Splunk over on day one. I ran five real audits against it: config inspection, data pipeline review, logging hardening, 24-hour log analysis, and IoC detection.\n\nThe verdict: it reasons like a senior analyst. It corrected wrong assumptions of mine three times, debugged a dozen issues on its own, and — this matters — said \"I cannot confirm this\" instead of inventing an answer.\n\nThoroughness is gold in an audit and poison in a remediation.\n\nWhen I handed it the first finding to fix, it went full forensic: tried performance-counter tools to verify counters, read the vendor spec, ran the failing binary by hand to reproduce it — and ran out of context. It hit 97.4% of its 128K window chasing the source name of a Windows Update log that had nothing to do with the fix.\n\nThe model knew the answer. It did not know when to stop. The fix was a system-prompt rule: *stop as soon as you have the root cause with evidence*.\n\n**Double ingestion.** My indexer was already collecting logs locally, and a Universal Forwarder on the same box was sending them again. Double licence usage, double noise. The valuable part was not the diagnosis: it read the two product GUIDs out of the MSI registry entries and explicitly warned me which one to remove and which one **never to touch**, because that second one was Splunk itself with all my data in it. A careless `msiexec /x` there and the lab is gone. It marked that one as excluded.\n\nAlso worth knowing: the straightforward `msiexec /x ... /qn` returned a phantom `-1`, because `msiexec` is asynchronous and PowerShell does not capture its exit code reliably. `Start-Process msiexec -Wait -PassThru` returned a clean `ExitCode: 0` and removed service, directory and registry entry.\n\n**Three data inputs dead since install day.** Zero events since July, three different root causes:\n\n`% Processor Time` but `% de tiempo de procesador`. Even the object name is localised. The fix was to force the tool to use the English API.\nAnd a detail I appreciate more than the fix itself: the config change was correct, but the binary still died when sampling. It documented that as \"needs binary debugging\" instead of claiming success.\n\n**Sysmon was blind to credential theft.** My config did not detect access to LSASS — the process that holds credentials in memory, and the thing Mimikatz goes after. An attacker could have dumped credentials and I would have learned about it later. Enabling `ProcessAccess` with a filter on `lsass.exe` is a three-line change, but along the way we discovered that **both the model and I were carrying a wrong Sysmon event-ID mapping**. The numbers I had been using were simply incorrect. The real config file disproved them. Verify technical numbers against reality; do not trust recall, including your own.\n\n**One command, three traps.** To log process creation with the command line:\n\n```\nauditpol /set /subcategory:\"{0CCE922B-69AE-11D9-BED3-505054503030}\" /success:enable /failure:enable\n```\n\nIt failed for three reasons at once: the privilege was missing, the category name is localised (so the English string does not resolve), and the braces of the GUID break PowerShell syntax unless quoted. Elevation, localisation, syntax.\n\nTwo earlier findings turned out to be **false positives, and the model is what caught them**, correcting previous audits (one of them a note of mine):\n\nAn analyst that corrects its own assumptions with evidence beats one that gets lucky.\n\nAfter watching it blow through the context window, I fixed the cause rather than the symptom: temperature down from 0.8 to 0.3, plus a working-rules system prompt (answer only what is asked; in remediation stop at root cause with evidence; in audit be exhaustive; use scripts for mechanical operations; verify IDs, ports and GUIDs before asserting; never invent; be concise).\n\n|  | Before | After | \n|---|---|---|\n| Output | Huge, rambling | Short and direct | \n| Context used | 97.4% (overflowed) | 49.7% | \n| Assumptions | Taken for granted | Verified against reality | \n| Result | 1 finding in 30 min | 4 findings in a row | \n\nOut of ten findings: double ingestion resolved, two inputs fixed and one pending binary debugging, credential access now detected, process logging enabled, two false positives corrected, one of my own notes fixed. Not a perfect lab result — which is exactly why I trust the report.\n\n**What I take away:**\n\nAn LLM does not replace your judgement or the urge to understand what is happening. But as a mentor that never sleeps and never leaves your machine, for someone starting out like me, it is a genuine advantage — and **not a single log leaves my computer**.\n\nThe full technical breakdown of this audit — screenshots, exact commands and config files — is on my blog: **[A 27B SOC Analyst: Auditing and Remediating Splunk + Sysmon](https://sammideblas.com/posts/0x73-a-27b-soc-analyst-auditing-and-remediating-splunk-sysmon)**.\n\nRelated, if you are building the same kind of thing: **[Building Your Own Log Analyzer](https://sammideblas.com/posts/0x74-building-your-own-log-analyzer)** and **[Gravity SOC, my home security operations center](https://sammideblas.com/posts/0x46-gravity-soc-my-security-operations-center)**.\n\nI write up what I learn, mistakes included, at **[sammideblas.com](https://sammideblas.com)**.", "url": "https://wpnews.pro/news/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack", "canonical_source": "https://dev.to/analista_83/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack-c47", "published_at": "2026-09-17 20:15:24+00:00", "updated_at": "2026-09-17 20:22:48.063111+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools", "mlops"], "entities": ["Splunk", "Sysmon", "Mimikatz", "llama-server", "CySA+"], "alternates": {"html": "https://wpnews.pro/news/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack", "markdown": "https://wpnews.pro/news/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack.md", "text": "https://wpnews.pro/news/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack.txt", "jsonld": "https://wpnews.pro/news/i-let-a-local-27b-llm-audit-and-fix-my-splunk-sysmon-stack.jsonld"}}