# Show HN: Intelligent Log Sanitization and logs to metrics

> Source: <https://github.com/sanitizeai/logguardai>
> Published: 2026-07-21 06:25:22+00:00

**Intelligent Log Sanitization for Java** — Automatically detect and mask sensitive data in logs using rule-based analysis and AI.

LogGuardAI is a Log4j2 plugin that intercepts log events and intelligently sanitizes them to **prevent accidental exposure of sensitive data** like passwords, API keys, tokens, and personal information.
It also supports logs-to-metrics extraction for Log4j2, converting log lines into Prometheus-style metrics using custom regex patterns.

```
❌ Logs with secrets:
  [LOG] User auth: userId=12345 apiKey=sk-abcdef123456 password=secret
        ↑ Oops! Now this secret is in logs, monitoring dashboards, and backups!
✅ Logs with LogGuardAI:
  [LOG] User auth: userId=***** apiKey=***** password=*****
        ↑ Secrets automatically masked, structure preserved!
```

- 🔐
**20+ Sensitive Keyword Detection**— Automatically detects passwords, tokens, API keys, etc. - 📊
**Smart Risk Scoring**— Multi-factor analysis (keywords, patterns, entropy) - ⚡
**Lightning Fast**— <5ms latency, non-blocking design - 🛡️
**Fail-Safe**— Logging never breaks, even if something fails - 📝
**Structured Data Support**— Handles key=value, JSON, query strings

- 🧠
**AI-Powered Sanitization**— OpenAI GPT-based intelligent masking with context - 💰
**LRU Caching**— 80-95% cache hit rate =** 80% cost savings** - 🏆
**Enhanced Exceptions**— AI explains what went wrong - 🔍
**Data Classification**— Automatically identify PII vs. sensitive data - 📊
**Configurable Sampling**— Control AI costs (5%-100%)

-
📦

**Batch AI Processing**— Process multiple sensitive values together (5x efficiency) -
⚡

**Async/Non-Blocking**— AI calls never block log threads (<5ms guaranteed) -
🔄

**Smart Fallbacks**— Rule-based masking if AI fails or times out -
🐛

**Spring Boot Compatibility**— Fixed Log4j2 plugin registration issues

- 🤖
**Multi-Provider AI**— OpenAI, Anthropic Claude, or Azure OpenAI - 🏗️
**Anthropic Claude Integration**— Claude 3 models with Messages API - ☁️
**Azure OpenAI Integration**— Custom endpoints and deployments - 🔧
**Provider-Specific Configuration**— Azure endpoints, deployments, API versions - 🏷️
**Enhanced Model Support**— GPT-4, Claude 3, Azure deployments - 📊
**Configurable Batch Size**— Tune performance vs. API efficiency - 🤖
**Multi-Provider Support**— OpenAI, Anthropic Claude, Azure OpenAI - 📈
**Metrics Extraction**— Convert legacy unstructured logs into rich JSON metrics. - 🏗️
**Backward Compatible**— Existing configurations work unchanged

**Released:** May 27, 2026

- 📊
**Pattern-Based Metrics**— Define custom regex patterns to extract metrics directly from logs - 🎯
**User-Defined Patterns**— Create metrics for HTTP requests, FTP transfers, database queries, etc. - 💾
**Append-Only File**— Metrics stored in Prometheus text format with cumulative history - 📈
**Logs-to-Metrics Conversion**— Convert Log4j2 log entries into counters and labels for monitoring and alerting - ⚙️
**Periodic Flushing**— Configurable flush interval (default: 60 seconds) - 🔐
**Cardinality Protection**— Configurable limits prevent unbounded metric combinations (default: 10,000) - 🔌
**Zero Dependencies**— No external monitoring infrastructure required

```
<LogGuardLayout 
  extractMetrics="true"
  metricsFilePath="logs/metrics.txt"
  metricsFlushIntervalMs="60000"
  metricsMaxCardinality="10000"
  metricsPatterns="http_requests|(GET|POST|DELETE) ([/\\w/]+) (\\d{3})|http_requests_total|method,endpoint,status"/>
```

See [Metrics Configuration](/sanitizeai/logguardai/blob/main/docs/guides/metrics-configuration.md) for full configuration details and examples.

```
<repositories>
    <repository>
        <id>jitpack.io</id>
        <url>https://jitpack.io</url>
    </repository>
</repositories>

<dependency>
    <groupId>com.logguardai</groupId>
    <artifactId>logguardai</artifactId>
    <version>0.5.0</version>  <!-- Latest: Logs to metrics -->
</dependency>
<Configuration>
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <LogGuardLayout/>
    </Console>
  </Appenders>

  <Loggers>
    <Root level="info">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>
```

Tip: To enable pattern-based metrics, set `extractMetrics="true"`

and configure `metricsFilePath`

, `metricsFlushIntervalMs`

, `metricsMaxCardinality`

, and `metricsPatterns`

as shown in the v0.5 section above.

LogGuardAI v0.5 makes it easy to transform Log4j2 log messages into Prometheus-compatible metrics. Use `metricsPatterns`

to extract method, endpoint, status, latency, and other useful labels from structured log lines.

Example: convert request logs into a counter metric for `http_requests_total`

with labels like `method`

, `endpoint`

, and `status`

.

This is ideal for:

- Log4j2 metrics extraction
- Logs-to-metrics pipelines
- Prometheus text metric generation from application logs
- Regex-based log analytics without external agents

``` php
<!-- OpenAI (default) -->
<LogGuardLayout
    aiEnabled="true"
    aiProvider="openai"
    aiApiKey="${OPENAI_API_KEY}"
    aiModel="gpt-3.5-turbo"
    samplingRate="0.1"
    extractMetrics="true"
    batchSize="5"/>

<!-- Anthropic Claude -->
<LogGuardLayout
    aiEnabled="true"
    aiProvider="anthropic"
    aiApiKey="${ANTHROPIC_API_KEY}"
    aiModel="claude-3-sonnet-20240229"
    samplingRate="0.1"
    batchSize="5"/>

<!-- Azure OpenAI -->
<LogGuardLayout
    aiEnabled="true"
    aiProvider="azure-openai"
    aiApiKey="${AZURE_OPENAI_API_KEY}"
    aiModel="gpt-35-turbo"
    azureEndpoint="https://your-resource.openai.azure.com"
    azureDeployment="your-deployment-name"
    azureApiVersion="2023-12-01"
    samplingRate="0.1"
    batchSize="5"/>
```

| Feature | v0.1 | v0.2 | v0.3 | v0.4 | v0.5 |
|---|---|---|---|---|---|
Rule-Based Masking |
✅ | ✅ | ✅ | ✅ | ✅ |
AI Sanitization |
❌ | ✅ | ✅ | ✅ | ✅ |
LRU Caching |
❌ | ✅ | ✅ | ✅ | ✅ |
Batch Processing |
❌ | ❌ | ✅ | ✅ | ✅ |
Async/Non-Blocking |
❌ | ❌ | ✅ | ✅ | ✅ |
Multi-Provider AI |
❌ | ❌ | ❌ | ✅ | ✅ |
Metrics Extraction |
❌ | ❌ | ❌ | ❌ | ✅ (pattern-based) |
Latency |
<5ms | <5ms (rule-based) ~1500ms (API) <1ms (cached) |
<5ms guaranteed | <5ms guaranteed | <5ms guaranteed |
Cost/10M logs |
$0 | $95 (no cache) $19 (with cache) |
$15-20 (with batching) | $15-20 (with batching) | $15-25 (depends on metrics config) |
Dependencies |
0 | 0 new | 0 new | 0 new | 0 new |
Backward Compatible |
N/A | ✅ 100% | ✅ 100% | ✅ 100% | ✅ 100% |

```
Logger logger = LoggerFactory.getLogger("MyApp");

// This will be automatically sanitized by LogGuardAI:
logger.info("User login: userId=12345 token=sk-abcdef123 password=secret");

// Output:
// User login: userId=***** token=***** password=*****
```

**Development (AI Testing)**

```
<LogGuardLayout aiEnabled="true" aiProvider="openai" samplingRate="0.05" aiTimeoutMs="1000" batchSize="3"/>
<!-- Or with Anthropic: aiProvider="anthropic" aiModel="claude-3-haiku-20240307" -->
```

**Production (Balanced)**

```
<LogGuardLayout aiEnabled="true" aiProvider="openai" samplingRate="0.1" aiTimeoutMs="2000" batchSize="5"/>
<!-- Or with Azure: aiProvider="azure-openai" azureEndpoint="..." azureDeployment="..." -->
```

**High-Security (Premium)**

```
<LogGuardLayout aiEnabled="true" aiProvider="openai" aiModel="gpt-4" samplingRate="1.0" batchSize="10"/>
<!-- Or with Claude: aiProvider="anthropic" aiModel="claude-3-sonnet-20240229" -->
```

**Fast (No AI)**

``` php
<LogGuardLayout aiEnabled="false"/>  <!-- v0.1 behavior -->
```

— 5-minute setup (with AI examples)[Quick Start Guide](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/QUICK_START.md)— Deep dive into AI features (3000+ words)[AI Integration Guide](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/AI_GUIDE.md)— v0.1 → v0.2 upgrade path[Migration Guide](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/MIGRATION.md)

— All configuration options[Configuration Reference](/sanitizeai/logguardai/blob/main/docs/guides/configuration.md)— Common issues & solutions[Troubleshooting](/sanitizeai/logguardai/blob/main/docs/guides/troubleshooting.md)— Code organization[Project Structure](/sanitizeai/logguardai/blob/main/docs/guides/project-structure.md)

— How LogGuardAI works[System Overview](/sanitizeai/logguardai/blob/main/docs/architecture/overview.md)— Detailed component descriptions[Components](/sanitizeai/logguardai/blob/main/docs/architecture/components.md)— Interface reference[AIService API](/sanitizeai/logguardai/blob/main/docs/api/air-service-interface.md)

— Complete version history[CHANGELOG](/sanitizeai/logguardai/blob/main/CHANGELOG.md)— v0.2 features[v0.2.0 Release Notes](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/RELEASE_NOTES.md)— Previous version[v0.1.0 Docs](/sanitizeai/logguardai/blob/main/docs/versions/v0.1)

— How to publish[JitPack Publishing](/sanitizeai/logguardai/blob/main/docs/guides/jitpack-publishing.md)

```
// Transaction logs with credit card & billing info
logger.info("Transaction: user_id={} amount={} card={} crypto_key={}",
    userId, amount, creditCard, encryptionKey);
// Automatically masked! ✅
// Auth logs with tokens & credentials
logger.info("Auth attempt: username={} password={} jwt_token={}",
    username, password, jwtToken);
// All masked automatically! ✅
// Headers & API keys
logger.info("Incoming request: Authorization={} X-API-Key={} User-Agent={}",
    bearerToken, apiKey, userAgent);
// Headers sanitized intelligently! ✅
```

Also: use `metricsPatterns`

to extract request counts, paths and status codes for lightweight in-app metrics collection.

```
Rule-Based Path:  <5ms    (fast, no external calls)
AI Hit (cached):  <1ms    (in-memory cache)
AI Miss (API):    ~1.5s   (to OpenAI, happens ~20% with 80% cache hit rate)
Without AI:              $0/month   (rule-based only)
With AI (no cache):      $95/month  (5% sampling)
With AI + caching:       $19/month  (5% sampling, 80% cache hits)
Savings:                 80%
Rule-Based Only:        >50,000 logs/sec
With AI Sampling:       >45,000 logs/sec (minimal impact)
With Caching:           >100,000 logs/sec (cached dominate)
```

✅ **Secrets Never Exposed**

- API keys, passwords, tokens, PII automatically masked
- Fail-safe: if LogGuard fails, logging still works (never breaks!)
- Timeout protection: no hanging requests

✅ **API Key Protection**

- API key stored in environment variable, never hardcoded
- No logs sent to external services (caching keeps results locally)

✅ **Non-Blocking Design**

- Independent executor for AI calls
- No thread blocking
- Application never slows down

- Java 8+
- Maven 3.6+

```
mvn clean package
mvn test
ls target/logguardai-0.5.0.jar
```

| Version | Release Date | Status | What's New |
|---|---|---|---|
v0.5.0 |
2026-05-27 | ✅ Latest | Pattern-Based Metrics (user-defined regex metrics) |
v0.4.0 |
2026-05-03 | ✅ Stable | Multi-Provider AI Support |
v0.3.0 |
2026-05-01 | ✅ Stable | Plugin Registration Fix |
v0.2.0 |
2026-04-23 | ✅ Stable | AI + Caching |
v0.1.0 |
2026-04-22 | ✅ Stable | Rule-based masking |

→ See [CHANGELOG](/sanitizeai/logguardai/blob/main/CHANGELOG.md) for complete history

Contributions welcome! Please:

- Fork the repository
- Create a feature branch
- Add tests for new features
- Submit a pull request

**Questions?**[GitHub Discussions](https://github.com/sanitizeai/logguardai/discussions)** Report Issues:**[GitHub Issues](https://github.com/sanitizeai/logguardai/issues)** Full Documentation:**[docs/README.md](/sanitizeai/logguardai/blob/main/docs/README.md)

MIT License - See [LICENSE](/sanitizeai/logguardai/blob/main/LICENSE) for details

**Get Started**→[Quick Start Guide](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/QUICK_START.md)** Enable AI**→[AI Integration Guide](/sanitizeai/logguardai/blob/main/docs/versions/v0.2/AI_GUIDE.md)** Configure**→[Configuration Reference](/sanitizeai/logguardai/blob/main/docs/guides/configuration.md)** Troubleshoot**→[Troubleshooting Guide](/sanitizeai/logguardai/blob/main/docs/guides/troubleshooting.md)

**LogGuardAI: Secure your logs. Automatically.**

*Built with ❤️ by sanitizeai*