Observed behavior
When server.Options.NoSublistCache: true is set, the gateway/leaf subscription sublist (gwLeafSubs) is still created with NewSublistWithCache() at server initialization in server.go:762. On leaf node deployments under sustained message traffic, this sublist's cache fills past the 1024-entry limit and continuously spawns reduceCacheCount goroutines to sweep it — consuming the majority of CPU despite NoSublistCache being explicitly set.
CPU profile from a leaf node running ~480 msg/s across 5 fixed subjects with NoSublistCache: true:
flat flat% sum% cum cum%
0.25s 3.34% 6.98s 93.32% server.(*Sublist).reduceCacheCount
0.26s 3.48% 5.61s 75.00% runtime.mapdelete_faststr
0.66s 8.82% 5.32s 71.12% internal/runtime/maps.(*Map).Delete
1.72s 22.99% 3.86s 51.60% internal/runtime/maps.(*table).Delete
1.85s 24.73% 1.85s 24.73% internal/runtime/maps.probeSeq.next (inline)
The offending line:
server.go:762: gwLeafSubs: NewSublistWithCache() hardcoded, ignores NoSublistCache
This is pretty much the same as issue #8070 but for leaf nodes instead of mqtt subsystem. I disabled the mqtt subsystem to avoid issue #8070 but ran into the same problem for leaf nodes.
Expected behavior
When NoSublistCache: true is set, gwLeafSubs should be initialized with NewSublistNoCache(), consistent with the server options. No reduceCacheCount goroutines should be spawned.
Server and client version
nats-server v2.12.7
Host environment
macOS host, Linux container (ARM64). Embedded server via the Go server package (not standalone binary), configured as a leaf node with a remote connection.
Steps to reproduce
- Embed nats-server with
NoSublistCache: true and leaf node remotes configured
- Publish ~480 msg/s across 4–5 fixed subjects via connected client
- Collect a CPU profile after several minutes (go tool pprof)
- Observe
reduceCacheCount accumulating >90% of cumulative CPU samples despite NoSublistCache: true
- Confirm via -focus filter that the hot path originates from the server goroutine launcher, not account sublists or MQTT.
Suggested fix:
// server.go ~line 762
gwLeafSubs: func() *Sublist {
if opts.NoSublistCache {
return NewSublistNoCache()
}
return NewSublistWithCache()
}(),
Or a helper similar to setAccountSublist() line 1932.
Observed behavior
When
server.Options.NoSublistCache: trueis set, the gateway/leaf subscription sublist (gwLeafSubs) is still created withNewSublistWithCache()at server initialization in server.go:762. On leaf node deployments under sustained message traffic, this sublist's cache fills past the 1024-entry limit and continuously spawnsreduceCacheCountgoroutines to sweep it — consuming the majority of CPU despiteNoSublistCachebeing explicitly set.CPU profile from a leaf node running ~480 msg/s across 5 fixed subjects with
NoSublistCache: true:The offending line:
server.go:762:
gwLeafSubs: NewSublistWithCache()hardcoded, ignoresNoSublistCacheThis is pretty much the same as issue #8070 but for leaf nodes instead of mqtt subsystem. I disabled the mqtt subsystem to avoid issue #8070 but ran into the same problem for leaf nodes.
Expected behavior
When
NoSublistCache: trueis set, gwLeafSubs should be initialized withNewSublistNoCache(), consistent with the server options. NoreduceCacheCountgoroutines should be spawned.Server and client version
nats-server v2.12.7
Host environment
macOS host, Linux container (ARM64). Embedded server via the Go server package (not standalone binary), configured as a leaf node with a remote connection.
Steps to reproduce
NoSublistCache: trueand leaf node remotes configuredreduceCacheCountaccumulating >90% of cumulative CPU samples despiteNoSublistCache: trueSuggested fix:
Or a helper similar to
setAccountSublist()line 1932.