# How I Debugged and Fixed Memory & Goroutine Leaks in ProjectDiscovery Nuclei Engine 🚀

> Source: <https://dev.to/thrylox/how-i-debugged-and-fixed-memory-goroutine-leaks-in-projectdiscovery-nuclei-engine-4794>
> Published: 2026-06-28 01:20:58+00:00

If you work in cloud security or vulnerability scanning, chances are high that you rely on **ProjectDiscovery Nuclei**—the gold standard open-source vulnerability scanner powered by YAML templates.

While Nuclei performs exceptionally well as a standalone CLI tool, embedding it as an underlying SDK engine inside long-running microservices or continuous scanning workers introduces unique architectural challenges: **memory bloat** and **goroutine leaks** over extended execution loops.

Recently, I investigated and resolved these exact engine lifecycle leaks in ** Nuclei Issue #7503** and submitted

When embedding `NucleiEngine`

into a long-running application loop (where engines are instantiated and closed dynamically per scan target), I noticed that memory consumption climbed steadily over time, and orphaned goroutines remained active long after calling `engine.Close()`

.

Upon profiling the engine lifecycle in Go, I identified three primary memory leaks:

`sync.Map`

in HTTP-to-HTTPS Port Tracker:`HTTPToHTTPSPortTracker`

stored host port mapping states in an unbounded `sync.Map`

. Over thousands of target scans, this map grew infinitely without eviction.`PerHostRateLimitPool`

). When an engine execution finished, worker background routines were not cleanly shut down or purged.`parsedTemplatesCache`

and `compiledTemplatesCache`

) retained parsed representations in memory between engine instances without an explicit cache purging mechanism during engine teardown.Instead of holding unbounded host entries in a `sync.Map`

, I replaced the storage structure with an expirable LRU (Least Recently Used) cache configured with a strict capacity bound (4,096 entries) and a 24-hour TTL:

```
// Replacing unbounded sync.Map with bounded expirable LRU cache
type HTTPToHTTPSPortTracker struct {
    cache *expirable.LRU[string, struct{}]
}

func NewHTTPToHTTPSPortTracker() *HTTPToHTTPSPortTracker {
    return &HTTPToHTTPSPortTracker{
        cache: expirable.NewLRU[string, struct{}](4096, nil, 24*time.Hour),
    }
}
```

This guarantees that host mappings automatically expire and memory remains strictly bounded regardless of how many millions of URLs are scanned.

`protocolstate.Close()`

I updated the global protocol state tear-down procedure in `pkg/protocols/common/protocolstate/state.go`

to release rate limiter worker routines and purge trackers upon `Close()`

:

```
func Close(executionID string) {
    stateLock.Lock()
    defer stateLock.Unlock()

    if state, ok := globalStateMap[executionID]; ok {
        // Release per-host rate limiters and background goroutines
        if state.PerHostRateLimitPool != nil {
            state.PerHostRateLimitPool.Close()
        }
        // Purge HTTP to HTTPS tracker entries
        if state.HTTPToHTTPSPortTracker != nil {
            state.HTTPToHTTPSPortTracker.Purge()
        }
        delete(globalStateMap, executionID)
    }
}
```

Finally, I added a thread-safe `Purge()`

method to the template parser struct and invoked interface type assertions during `NucleiEngine.Close()`

:

```
// Safely purge compiled template caches on engine close
func (e *NucleiEngine) closeInternal() error {
    if e.parser != nil {
        e.parser.Purge()
    }
    if purger, ok := e.executerOpts.Parser.(interface{ Purge() }); ok {
        purger.Purge()
    }
    return nil
}
```

When designing solutions for large open-source codebases, evaluating architectural trade-offs is essential:

`Options.HTTPToHTTPSCacheSize`

) would be a clean future addition.`interface{ Purge() }`

) keeps the codebase decoupled and preserves backward compatibility for third-party SDK consumers using custom parsers without breaking their implementations.I validated these fixes across Nuclei unit test packages (`httpclientpool`

, `protocolstate`

, `templates`

, and `lib`

), verifying `100%`

success with zero memory accumulation between consecutive engine shutdowns.

Fixes #7503 by implementing the required leak-prevention cleanup mechanisms outlined in #7502 for long-running embedded engines.

`sync.Map`

in `HTTPToHTTPSPortTracker`

(`pkg/protocols/http/httpclientpool/http_to_https_tracker.go`

) with a size-bounded expirable LRU cache (4096 entries max, 24h TTL) and added `Purge()`

.`protocolstate.Close()`

(`pkg/protocols/common/protocolstate/state.go`

) to release per-host rate-limit pool goroutines and purge the HTTP-to-HTTPS tracker on shutdown.`NucleiEngine.Close()`

/ `closeInternal()`

(`lib/sdk.go`

) and `Parser`

(`pkg/templates/parser.go`

) to purge parsed and compiled template caches on engine close.The embedded engine can leak memory and goroutines over time during long-running usage.

Implement the leak-prevention work described in #7502:

`HTTPToHTTPS`

tracker with an LRUWithout explicit cleanup and bounded caching, long-running embedders can accumulate memory usage and leave background goroutines running indefinitely.

`HTTPToHTTPS`

tracking / redirect bookkeeping`HTTPToHTTPS`

tracker is size-bounded and evicts old entries.PR title: fix leaks

`sync.Map`

in Long-Running Apps:`sync.Map`

is convenient, it lacks eviction policies. Use LRU caches with TTLs for dynamic lookup tables.`Close()`

/ `Purge()`

methods to release background channels and goroutines.`if purger, ok := obj.(interface{ Purge() }); ok`

enable clean resource cleanup without introducing rigid package dependencies.*Written by @Thrylox. Connect with me on GitHub!*