# How to Encode Tacit Knowledge into SOC Agents: Five Methods That Actually Work > Source: > Published: 2026-08-19 04:47:18+00:00 *The hardest problem in security AI is not building an LLM. It is capturing the sixth sense of a senior analyst who can spot a breach just by looking at a log line wrong.* There is a moment every SOC analyst recognizes. An alert fires. On the surface, it is routine: a single failed SSH login from an external IP. Nothing in the rules flags it as critical. But something in the pattern — the timing, the source region, the precise rhythm of the attempts — makes a senior analyst pause. That does not feel right. They dig deeper. Three hours later, they have uncovered a staged intrusion by a threat actor using novel lateral movement techniques never seen in any signature database. No rule caught it. The AI did not catch it. A human caught it. Because the human had something no system had ever been taught: **tacit knowledge**. This is the central challenge of AI-powered security operations. Explicit knowledge — rule signatures, CVE databases, playbooks — is relatively easy to encode. Tacit knowledge — the gut feeling, the pattern recognition, the "I cannot explain it but I know it when I see it" — is what makes a senior analyst irreplaceable. And it is exactly what makes encoding intelligence into a SOC agent so brutally hard. This article is not a product pitch. It is a practical map of the approaches that actually work, ranked by feasibility, with enough honesty to be useful. Polanyi is famous phrase — "we know more than we can tell" — was written in 1966, long before SIEM tools existed. But it describes the SOC problem perfectly. Senior analysts accumulate years of pattern recognition that they cannot easily articulate: Explicit knowledge is codified: documented procedures, signature rules, threat intelligence feeds. Tacit knowledge lives in the analyst head, shaped by thousands of hours of experience, near-misses, and hard-won lessons. When you deploy an AI agent in a SOC, it gets access to all the explicit knowledge. What it lacks is the tacit layer. Closing that gap is what the methods below are trying to solve. **Feasibility: High. Start today.** Tacit knowledge often surfaces naturally when experts work through problems out loud. The Think Aloud Protocol — a technique from cognitive psychology — exploits this: ask an expert to narrate their thinking while they work, without editing or organizing their thoughts. What comes out is raw tacit knowledge. **Step 1 — Record the session.** Give your senior analysts a microphone and screen capture tool. Ask them to process real alerts while talking through their reasoning. No preparation needed. No formal language required. A real transcript might sound like this: Okay, this IP is from Eastern Europe, 185.220.101.x. That is a Tor exit node. I have seen a lot of commodity scanning from this range — script kiddies, mostly. But the timing is weird. It is 3 AM UTC. And they are hitting port 445 specifically. That is a SMB exploit probe. The threshold rules did not fire because there were only 12 attempts in 3 minutes. But the specificity of the target port combined with the Tor exit node plus the time of day — that is a pattern I have seen before. This feels like a red team probe, not random scanning. That last sentence — "this feels like" — is gold. That is tacit knowledge escaping. **Step 2 — Transcribe and annotate.** Use Whisper or any ASR tool to transcribe the recordings. Then have a junior analyst or knowledge engineer mark the gut feeling moments: where the expert made a judgment call that was not rule-driven. **Step 3 — Inject into the agent.** Use these annotated snippets as few-shot examples in the agent system prompt, training samples for fine-tuning, or validation cases to test whether the agent reproduces the intuition. | Aspect | Detail | |---|---| Time to first results | 1-2 weeks | Data quality | High (real incidents) | Scale | Labor-intensive per hour of recording | What it captures well | Contextual judgment calls, pattern descriptions | What it misses | Subtle perceptual cues, fast intuition that does not get verbalized | Not every analyst is comfortable narrating their thinking. Some find it awkward. This method works best when the organizational culture already values knowledge sharing. **Feasibility: Medium. Requires setup but produces high-quality data.** Give your senior analyst a red team role — attacking a simulated environment — while the AI agent plays blue. As the expert attempts to evade detection, they will naturally deploy the same techniques and reasoning that have helped them bypass real SOC tools. That evasion reasoning is tacit knowledge in its purest form. **Step 1 — Set up a simulation environment.** Use Caldera (MITRE ATT&CK-aligned), Atomic Red Team, or similar tools to simulate realistic attack chains. **Step 2 — Red team with explanations.** The senior analyst operates the red team, but verbalizes their reasoning at each step. I am slowing my beacon callback to exactly 10 minutes because I know the SOC is threshold-based and will not flag anything under 15 minutes. I am using port 443 because 443 is whitelisted in most proxy logs. I am avoiding PowerShell and using LOLBin instead because the rules are tuned for PowerShell. **Step 3 — Record and distill.** The red team attempts, the agent responses, and a human mentor corrections all become training data. The key output is the red team mental model: why they believe a technique will evade detection. **Step 4 — Encode the evasion logic.** Extract the evasion reasoning and encode it as rules, constraints, or prompt injections for the agent. This method produces data that is nearly impossible to get any other way: the adversarial reasoning behind real-world evasion techniques. You are teaching the agent how sophisticated attackers think — which is exactly what the senior analyst has internalized. **Feasibility: Medium-High. Best with structured expert involvement.** Capture not just the expert conclusion, but their full reasoning chain — including the leaps that feel intuitive. Then use these reasoning chains as supervised training signals to teach a smaller model to replicate the same thinking pattern. **The raw alert:** SSH brute force attempt from 185.220.101.x. 50 attempts in 3 minutes. **The expert reasoning chain:** - Source IP check: 185.220.101.x is a known Tor exit node range. Any traffic from this range carries elevated suspicion context. - Timing analysis: 50 attempts in 3 minutes = ~3.6 seconds per attempt. Too fast for a real brute force. This is automated scanning, not manual credential attack. - Port specificity: Port 22 being targeted in this pattern suggests credential stuffing probe. - Contextual cross-check: Are there other alerts from this IP range in the last 24 hours? If not, this is likely a first probe. - Intuition layer: Tor exit nodes are commonly used by script kiddies and red teamers, not APTs. Threat level is lower than the alert might suggest, but the IP should be marked and monitored. - Decision: Low priority. Log as IOC. Do not block yet. Monitor for follow-up activity. **The agent response after training:** Source IP 185.220.101.x is a Tor exit node — elevated context. Timing pattern indicates automated scanning rather than manual brute force. Port 22 targeting suggests credential stuffing probe. Assessment: script kiddie or red team activity (low APT probability). Recommended action: Log IOC, monitor for follow-up, do not block immediately. Collect 500-1000 such reasoning chains across diverse alert types. Use them to fine-tune a smaller model (Llama 3.1 8B, Qwen 2.5 7B), inject as structured few-shot examples in prompts, or build a reasoning template library. **Feasibility: Medium. Requires deployment infrastructure and ongoing human involvement.** Deploy the agent in a low-stakes mode. Let it make recommendations. Have senior analysts review its decisions daily and provide feedback. Use that feedback as a reward signal to continuously update the agent behavior. **Phase 1 — Shadow mode deployment.** Run the agent in parallel with the human SOC. It sees all the same alerts and produces its own recommendations, but takes no action. **Phase 2 — Feedback collection.** After the analyst resolves each alert, reveal the agent recommendation. The analyst rates it: correct, incorrect, or partially correct with modifications. **Phase 3 — Reward signal.** Convert feedback into a reward signal. Correct = positive reward. Incorrect = negative reward. Modified = partial reward with adjustment direction. **Phase 4 — Policy update.** Use PPO (Proximal Policy Optimization) or DPO (Direct Preference Optimization) to update the agent policy model based on accumulated feedback. **Phase 5 — Graduated autonomy.** Once the agent reaches a consistent accuracy threshold in shadow mode, gradually give it authority over low-risk alert categories. Google Security AI Workbench uses a version of this approach: analyst adoption rate of AI recommendations is used as an implicit feedback signal to continuously refine the model. **Feasibility: Low. Academic frontier, high complexity.** Explicitly model tacit knowledge using cognitive science frameworks: Cognitive architectures like ACT-R and Soar were designed for exactly this. In practice, a more tractable version is a hybrid expert system + ML approach: encode known patterns as explicit rules, use ML to detect deviations, and flag surprises for human review. This is the most intellectually ambitious approach and the least production-ready. For a real SOC trying to improve its AI agent this quarter, start elsewhere. Enroll 5-10 senior analysts in a Think Aloud recording program. Target 2 hours of real incident processing per analyst. Transcribe and annotate the recordings. Target 200+ reasoning chain samples. Inject the best ones into your agent prompts as few-shot examples. Run a small validation set. Set up a simulation environment. Run monthly adversarial exercises. Capture the evasion reasoning and encode it back into the agent. Deploy the agent in shadow mode on real traffic. Collect analyst feedback. Begin RLHF fine-tuning pipeline. If feedback data is rich enough, fine-tune a dedicated model on your accumulated reasoning chains and feedback data. Tacit knowledge is not static. It evolves as the threat landscape changes. Your senior analyst is continuously recalibrating their intuition against new data. Your agent must do the same. Build a **continuous learning loop** from day one: If you build an agent that captures tacit knowledge but has no mechanism for updating it, you have not built an AI SOC analyst. You have built a frozen snapshot of a senior analyst from six months ago — and in security, that is worse than nothing, because it creates false confidence. Tacit knowledge is the last mile of SOC automation. It is also the hardest. But it is not unsolvable. It just requires the right combination of techniques, the patience to collect good data, and the organizational commitment to keep the system learning. Start recording. Start asking your experts to explain what they are thinking. The rest follows.