Are We Sending Too Much Data to LLMs? Agentic Production Support (APS)

A developer working on agentic AI for production support is advocating for treating LLMs as a data egress boundary, proposing a sanitization layer before data reaches RAG or embedding models. The approach emphasizes redacting sensitive information such as hostnames, emails, and authorization tokens, and suggests that if sensitive data is still detected after sanitization, the model should not be called. The developer argues that AI governance should be enforced through technical controls rather than policy documents alone.

While working on Agentic AI for production support, one question came to my mind: Do we really know what data we are sending to the LLM? Let's take a simple production incident. Host: ip-10–0–21–145 Memory: 1024 MB Contact: user@example.com mailto:user@example.com Authorization: Bearer abc.def.ghi ERROR: Service failed due to disk space issue For RCA, the LLM mainly needs to understand: "Service failed because of a disk space issue." It doesn't really need the actual hostname, email, AWS resource details, file paths, capacity values or authorization token. This is where I started looking at the LLM as a Data Egress Boundary. Instead of: Production Data → RAG → LLM I am looking at: Production Data → Clean/Sanitization Layer → RAG → LLM Before anything goes to an LLM or embedding model, sensitive information should be cleaned or redacted. But there is another important point. RAG itself is not the security layer. We normally think that our data is safe because we are using RAG. But before storing a document in a vector database, we generate embeddings. So the actual flow can be: Raw Data → Embedding Model → Vector DB → Retrieval → LLM This means sanitization should happen before embedding as well, not only before the final LLM call. The same principle should apply to retrieval queries and AI observability logs. For high-risk information like API keys, passwords, JWTs or bearer tokens, I prefer a simple rule: If sensitive data is still detected after sanitization → don't call the model. In my view, AI governance should not be only a policy document saying: "Don't send sensitive data to AI." Raw Context → Sanitize → Validate → RAG / LLM The objective is not to remove useful context. The objective is: Give AI enough context to solve the problem, but don't give it information it doesn't need. AgenticAI AIGovernance LLM RAG GenAI AIArchitecture ProductionSupport