GitHub:[https://github.com/phero20/concurrent-resource-scheduler]
Go documentation:[https://pkg.go.dev/github.com/phero20/concurrent-resource-scheduler]
A resource scheduler sounds simple until many goroutines start competing for a small number of resources.
At that point, the problem stops being:
"How do I pick the next item?"
and becomes:
"How do I pick the right item while thousands of goroutines are acquiring, releasing, updating, and observing resources concurrently?"
That is the problem I built Concurrent Resource Scheduler (CRS) to solve.
CRS is a domain-agnostic Go library for managing reusable resources with:
The current release is v1.2.3.
This post is a release-focused technical deep dive into the architecture, the reasoning behind it, and the actual measurements from the current release.
Imagine a service with a pool of reusable resources.
Those resources could be:
Now imagine:
10,000 concurrent requests
│
▼
┌─────────────────┐
│ Scheduler │
└────────┬────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
Resource A Resource B Resource C
The scheduler needs to answer several questions at once:
A simple slice and mutex can answer some of these questions.
The difficult part is doing all of them concurrently and predictably.
The first design most people naturally reach for is something like:
type Scheduler struct {
mu sync.Mutex
resources []*Resource
}
Then acquisition becomes:
func (s *Scheduler) Acquire() (*Resource, error) {
s.mu.Lock()
defer s.mu.Unlock()
// Scan resources.
// Find the best available resource.
// Mark it acquired.
// Return it.
return resource, nil
}
For a small pool, this is completely reasonable.
The problem appears when concurrency and resource count grow.
Suppose:
Resources: 10,000
Concurrent goroutines: 10,000
Every operation now enters the same synchronization boundary:
GLOBAL MUTEX
│
┌──────────────┼──────────────┐
▼ ▼ ▼
goroutine goroutine goroutine
│ │ │
└──────────────┼──────────────┘
▼
WAIT
Even if the actual operation only concerns a small part of the resource pool, the lock protects everything.
There is another problem.
If resources are stored in a slice and the scheduler has to find the best candidate, acquisition can require a linear scan:
O(N)
Now the same global lock is protecting an operation whose amount of work grows with the number of resources.
That is the combination I wanted to avoid.
The core idea behind CRS is:
Partition the active resource pool into independently synchronized shards.
Instead of:
ONE LOCK
│
ONE BIG POOL
CRS uses:
Scheduler
│
┌──────────────┼──────────────┐
▼ ▼ ▼
Shard 0 Shard 1 Shard N
│ │ │
Heap Heap Heap
│ │ │
Mutex Mutex Mutex
Each shard owns a priority heap.
Each heap has its own synchronization boundary.
That gives us a much more useful concurrency model:
goroutine A ──► shard 0 ──► heap 0
goroutine B ──► shard 1 ──► heap 1
goroutine C ──► shard 2 ──► heap 2
The goal is not to claim that sharding magically eliminates contention.
It doesn't.
The goal is to reduce the amount of unrelated work competing for the same lock.
That distinction matters.
Sharding solves one problem:
How do we reduce contention?
We still need to solve another:
How do we efficiently maintain resource priority?
A slice makes this straightforward but potentially expensive:
resources:
[ A, B, C, D, E, F, G, ... ]
Find minimum priority:
scan everything
A priority heap gives us an ordered structure where the highest-priority candidate can be accessed from the heap root.
Conceptually:
best resource
│
▼
[ 10 ]
/ \
[ 20 ] [ 30 ]
/ \
[40] [50]
So CRS combines the two ideas:
RESOURCE POOL
│
┌─────────┴─────────┐
▼ ▼
Sharding Priority
│ │
▼ ▼
Lower lock Heap ordering
contention / fast candidate
This is the central design tradeoff of the project.
One thing I learned while building CRS is that the heap is actually only one part of the problem.
The scheduler has several independent concerns:
CRS
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
Acquisition State Lookup
Strategy Lifecycle Map
│ │ │
▼ ▼ ▼
Adaptive Active/Inactive O(1)
Weighted Acquire/Release lookup
Round Robin Include/Exclude
Affinity
Then there are extensions:
Event Dispatcher
│
┌─────────┴─────────┐
▼ ▼
Cooldown Telemetry
│
▼
Prometheus
Keeping these responsibilities separated makes the implementation easier to reason about and lets the core remain domain-agnostic.
Priority ordering tells us:
"Which resource should be considered first?"
But APIs also need direct resource lookup.
For example:
resource, err := sched.Get("worker-42")
We don't want to search every heap for that.
CRS therefore maintains a lookup structure alongside the active heap shards.
Conceptually:
resource ID
│
▼
┌─────────────┐
│ Lookup Map │
└──────┬──────┘
│
▼
node
│
┌───────┴───────┐
▼ ▼
resource shard
This gives the scheduler a useful separation:
These are different access patterns, so they shouldn't be forced into the same data structure.
Another design decision was not to hard-code one definition of "best resource."
Different systems want different behavior.
For example:
Choose resources based on observed scheduling state.
Some resources should receive more traffic than others.
Distribute acquisitions sequentially.
Prefer a resource or shard associated with some identifier.
That means the scheduler's core doesn't need to know why an application considers one resource better than another.
The application provides the policy.
The scheduler provides the concurrency-safe machinery around that policy.
This distinction is especially important.
A resource might support:
Shared
where multiple callers can use it simultaneously.
Or:
Exclusive
where only one caller can own it at a time.
These are not just two API names.
They produce different synchronization behavior.
For example, under shared acquisition, the scheduler can inspect the best candidate without removing it from the heap.
Conceptually:
Shared:
heap
│
▼
peek candidate
│
▼
use resource
Whereas exclusive acquisition may need to remove the candidate from the active scheduling position:
Exclusive:
heap
│
▼
pop candidate
│
▼
exclusive owner
This distinction is also reflected in the documentation and complexity model.
Resources don't simply exist or disappear.
They move through states.
A simplified lifecycle looks like:
┌──────────────┐
│ ACTIVE │
└──────┬───────┘
│
acquire / exclude
│
▼
┌──────────────┐
│ INACTIVE │
└──────┬───────┘
│
include / release
│
▼
┌──────────────┐
│ ACTIVE │
└──────────────┘
There are also terminal operations such as removal and shutdown.
The important part is that the scheduler must maintain consistent state while these operations happen concurrently.
That's why lifecycle management is part of the scheduler design rather than an afterthought.
Cooldown is a good example of why the architecture is modular.
Suppose a resource fails or needs temporary exclusion:
Resource
│
▼
Cooldown
│
├── removed from active scheduling
│
└── restored after duration
The core scheduler shouldn't need to understand every possible reason a resource temporarily leaves the pool.
The cooldown extension coordinates with the scheduler's lifecycle controller.
This was also one of the areas tightened in v1.2.3: the cooldown documentation and example now use the correct LifecycleController
wrapper pattern instead of trying to initialize the cooldown manager before the scheduler exists.
That ordering matters because the scheduler is the object that ultimately owns the resource lifecycle.
v1.2.3 is not a giant feature release.
It is a correctness, consistency, and release-hardening release.
The main fixes include:
The cooldown extension's initialization example was corrected to use the proper lifecycle-controller wrapper pattern.
This avoids the circular initialization problem of trying to construct a component that needs the scheduler before the scheduler has been created.
The cooldown example was adjusted so it does not immediately race an asynchronous exclusion event.
The example now demonstrates the intended lifecycle deterministically.
The scheduler benchmark file was moved to the black-box package boundary:
package scheduler_test
because it only relies on exported APIs.
That makes the benchmark follow the same testing convention as the rest of the package.
README and API documentation were updated to match the implementation, including error documentation and complexity behavior.
The optional Prometheus module is also aligned to:
v1.2.3
while remaining a separate nested Go module.
Architecture diagrams are useful.
Numbers are better.
I ran the current benchmark suite with:
go test -run="^$" -bench="." -benchmem ./...
The measurements below were collected locally on:
OS: Windows
Arch: amd64
CPU: AMD Ryzen 5 6600H with Radeon Graphics
These are local measurements, not universal performance guarantees.
The current release produced the following verified measurements:
| Strategy | Select/GetShard Cost |
|---|---|
ConsistentHashRing.GetShard |
|
| 7.2 ns | |
WeightedStrategy.Select |
|
| 19.1 ns | |
AdaptiveStrategy.Select |
|
| 27.0 ns |
The strategy selection costs are all in the low-nanosecond range on this machine. The scheduler benchmarks below separately report allocation counts for the larger operations.
The scheduler benchmarks are more representative of actual scheduler operations.
| Operation | HeapCount=1 | HeapCount=8 | HeapCount=32 | Allocs/op |
|---|---|---|---|---|
Add |
||||
| 591.9 ns | 836.2 ns | 718.5 ns | 3 | |
Update |
||||
| 280.6 ns | 228.9 ns | 200.4 ns | 1 | |
BatchAdd (1,000 resources) |
||||
| 308.1 µs | 351.3 µs | 331.1 µs | ~1,100 | |
Acquire (Shared, Sequential) |
||||
| 12.36 ns | 11.52 ns | 11.23 ns | 0 | |
Acquire (Shared, Parallel) |
||||
| 63.14 ns | 16.80 ns | 17.99 ns | 0 | |
Acquire + Release (Exclusive) |
||||
| 241.2 ns | 249.2 ns | 208.8 ns | 0 |
There is also a parallel acquisition benchmark:
| Benchmark | Heap count | ns/op | B/op | allocs/op |
|---|---|---|---|---|
| AcquireSharedParallel | 1 | 63.14 | 0 | 0 |
| AcquireSharedParallel | 8 | 16.80 | 0 | 0 |
| AcquireSharedParallel | 32 | 17.99 | 0 | 0 |
The parallel benchmark is particularly useful when thinking about the sharding architecture.
On this machine and workload, moving from a single heap to multiple heaps reduced the measured AcquireSharedParallel
time substantially:
HeapCount=1 63.14 ns/op
HeapCount=8 16.80 ns/op
HeapCount=32 17.99 ns/op
That is one workload on one machine, not a universal scaling law.
But it is exactly the kind of behavior the architecture is intended to make measurable.
It would be easy to benchmark only:
HeapCount = 1
and stop there.
That wouldn't tell us much about the reason for sharding.
Instead, the scheduler benchmarks compare:
HeapCount = 1
HeapCount = 8
HeapCount = 32
This gives us a way to observe how the synchronization structure behaves as the number of shards changes.
The expected tradeoff is not:
"More shards are always faster."
It is:
"The right number of shards depends on workload, resource count, contention, and scheduling strategy."
More shards also mean more structures to manage.
So sharding is a tuning dimension, not a magic constant.
The benchmark suite also measures batch insertion:
| Benchmark | Heap count | ns/op | B/op | allocs/op |
|---|---|---|---|---|
| BatchAdd | 1 | 308.1 µs | 283,715 | 1,042 |
| BatchAdd | 8 | 351.3 µs | 283,530 | 1,095 |
| BatchAdd | 32 | 331.1 µs | 282,314 | 1,223 |
This is important because it prevents cherry-picking only the fastest numbers.
AcquireShared
is extremely cheap in the measured workload.
BatchAdd
is much more expensive.
That's expected.
Adding a large batch involves substantially more work than peeking at an already-populated scheduling structure.
A useful benchmark suite should expose those differences instead of presenting one "magic" performance number.
A concurrency library can produce beautiful microbenchmarks and still be broken.
That's why v1.2.3 was validated with multiple layers.
go test ./...
Result:
PASS
All tested packages completed successfully.
go test -race ./...
Result:
PASS
No data races were reported.
go vet ./...
Result:
PASS
gofmt -l .
Result:
(no output)
The working tree is therefore clean according to gofmt
.
For a concurrency-heavy library, this command is not optional validation theater:
go test -race ./...
A normal test can pass while a race exists.
The race detector instruments memory accesses and can expose unsafe concurrent access that ordinary functional assertions don't catch.
It does not prove that a concurrent system is mathematically bug-free.
But it is one of the most important tools available for catching a class of real concurrency bugs.
For v1.2.3, the race-enabled test suite completed without reported races.
Microbenchmarks answer:
"How quickly does this small operation execute?"
A load test asks a different question:
"What happens when the whole scheduler is placed inside a realistic concurrent workload?"
For CRS, the dedicated load-test harness models things such as:
The distinction is important.
A benchmark like:
25 ns/op
does not mean a real request takes 25 ns.
A real request also includes:
scheduler
+
application work
+
network
+
backend
+
serialization
+
other system costs
So I treat microbenchmarks and load tests as different measurements.
One of the release validation workloads uses:
10,000 concurrent workers
This is useful because it puts the scheduler under a very different kind of pressure from a small benchmark.
The important question isn't:
"Can Go start 10,000 goroutines?"
Of course it can.
The question is:
"Can the resource-management layer maintain correct lifecycle and acquisition behavior while thousands of workers continuously compete for a small pool?"
That is where the sharded architecture, lookup synchronization, acquisition strategies, and lifecycle rules interact.
For the load-test results themselves, I treat the numbers as workload-specific measurements rather than claiming they represent every deployment.
The load-test harness also separates scheduler-side failures from simulated backend failures, because those are operationally different failure modes.
This distinction is easy to lose in a load test.
Imagine:
10,000 workers
│
▼
Scheduler
│
▼
Resource
│
▼
Backend request
│
▼
FAILURE
The backend failed.
That doesn't mean:
scheduler failed
Similarly:
scheduler acquire failure
is not the same thing as:
backend failure
A serious load test should keep these categories separate.
That's why the harness tracks acquisition behavior, backend behavior, release behavior, and timeout/cancellation behavior independently.
Consider four backend resources:
backend-1
backend-2
backend-3
backend-4
and:
10,000 concurrent workers
You cannot turn four resources into 10,000 concurrent backend operations just because there are 10,000 goroutines.
If the acquisition policy is exclusive, the resource pool remains the bottleneck:
10,000 workers
│
▼
┌───────────┐
│ Scheduler │
└─────┬─────┘
│
▼
4 resources
That is not a failure.
That's exactly what resource scheduling is supposed to enforce.
The scheduler doesn't know whether a resource is:
type APIKey struct {
ID string
}
or:
type GPUWorker struct {
ID string
}
or:
type DatabaseReplica struct {
ID string
}
The application provides the resource type and the functions needed to identify and compare it.
For example:
compare := func(a, b *Worker) int {
if a.Priority < b.Priority {
return -1
}
if a.Priority > b.Priority {
return 1
}
return 0
}
keyFunc := func(w *Worker) string {
return w.ID
}
The scheduler doesn't need to understand what Priority
means.
It only needs a consistent comparison function.
That's what makes the library reusable across domains.
The basic flow remains intentionally small:
package main
import (
"fmt"
"log"
"github.com/phero20/concurrent-resource-scheduler/config"
"github.com/phero20/concurrent-resource-scheduler/scheduler"
)
type Worker struct {
ID string
Priority int
}
func main() {
compare := func(a, b *Worker) int {
if a.Priority < b.Priority {
return -1
}
if a.Priority > b.Priority {
return 1
}
return 0
}
keyFunc := func(w *Worker) string {
return w.ID
}
cfg := config.Config[*Worker, string]{
HeapCount: 8,
Comparator: compare,
KeyFunc: keyFunc,
}
sched, err := scheduler.New(cfg)
if err != nil {
log.Fatal(err)
}
defer sched.Shutdown()
_ = sched.Add(&Worker{
ID: "worker-1",
Priority: 10,
})
_ = sched.Add(&Worker{
ID: "worker-2",
Priority: 20,
})
resource, err := sched.Acquire()
if err != nil {
log.Fatal(err)
}
fmt.Println("Acquired:", resource.ID)
}
The important thing is how little domain logic is inside the scheduler.
The application defines:
Resource type
Key function
Comparator
Configuration
CRS manages the rest.
The release isn't about replacing the architecture.
It is about tightening the implementation and making the public surface accurately represent what the implementation already does.
The release includes:
The cooldown manager now has documentation and examples that correctly reflect its dependency on a lifecycle controller.
The example accounts for asynchronous event dispatch so the demonstration is deterministic instead of racing an event-driven state transition.
The scheduler benchmark uses:
package scheduler_test
rather than relying on internal package access.
The README and API documentation were aligned with the actual implementation, including exported errors and shared/exclusive complexity behavior.
The optional Prometheus extension is tagged:
extensions/prometheus/v1.2.3
and references the corresponding core release.
This is probably the most important disclaimer in the entire post.
I don't want to say:
"CRS is 10x faster than every mutex-based scheduler."
I haven't proven that.
I don't want to say:
"CRS handles 37,000 requests/sec in production."
A workload-specific test is not a production guarantee.
I don't want to say:
"32 shards is always optimal."
It isn't.
The benchmarks show what happened under a particular machine and workload.
The architecture explains why those measurements are interesting.
That's a much more useful claim.
Sharding is most interesting when:
many concurrent operations
+
multiple independent resource groups
+
shared scheduling state
It can reduce contention by allowing unrelated operations to work against different synchronization boundaries.
But it introduces its own tradeoffs:
So the design isn't:
"Sharding is always better."
It's:
"Sharding is a useful way to control contention when the workload benefits from partitioning."
The design can be summarized as four decisions:
1. Heap
↓
efficient priority ordering
2. Sharding
↓
reduce shared lock contention
3. Lookup map
↓
fast direct resource access
4. Pluggable strategies
↓
separate scheduling policy from
resource-management mechanics
Then the lifecycle/event system sits around those primitives:
Scheduler Core
│
┌──────────────┼──────────────┐
▼ ▼ ▼
Heap Lookup Policy
│ │ │
└──────────────┼──────────────┘
│
▼
Lifecycle
│
┌───────┴───────┐
▼ ▼
Cooldown Events
│
▼
Telemetry
That separation is the part I'm most interested in continuing to improve.
Before publishing this release, I wanted the project to pass more than just a version bump.
The current validation includes:
go test ./...
go test -race ./...
go vet ./...
gofmt -l .
The benchmark suite was also run with:
go test -run="^$" -bench="." -benchmem ./...
And the release artifacts are available through the Go module ecosystem:
github.com/phero20/concurrent-resource-scheduler@v1.2.3
github.com/phero20/concurrent-resource-scheduler/extensions/prometheus@v1.2.3
The core module targets Go 1.22+.
The optional Prometheus extension is a separate module targeting the newer Go toolchain used by that extension.
v1.2.3 is a good point to stop and evaluate the architecture under more workloads.
The areas I'm particularly interested in are:
The goal isn't simply to produce a bigger benchmark number.
The goal is to understand where the scheduler's architecture helps, where it doesn't, and what tradeoffs become visible at scale.
Building a concurrent scheduler taught me that the difficult part isn't implementing:
Acquire()
The difficult part is everything around it.
You need to coordinate:
RESOURCE SCHEDULING
Priority
│
▼
Concurrency ───────► Scheduler ◄────── Acquisition Policy
│
┌───────────┼───────────┐
▼ ▼ ▼
Lookup Lifecycle Events
│ │ │
▼ ▼ ▼
O(1) map Active/ Telemetry
Inactive │
▼
Prometheus
That is what CRS is trying to provide.
Not just a priority queue.
Not just a resource pool.
But a reusable concurrency layer where:
can exist as separate pieces without forcing the application to rebuild the entire system.
v1.2.3 is the current release.
If you're building an LLM gateway, proxy pool, database router, GPU worker pool, API-key manager, or another system where many concurrent requests compete for reusable resources, I'd genuinely like to hear how you're solving it.
Concurrent Resource Scheduler
GitHub: https://github.com/phero20/concurrent-resource-scheduler
Go documentation: https://pkg.go.dev/github.com/phero20/concurrent-resource-scheduler
Current release:
v1.2.3
Optional Prometheus extension:
extensions/prometheus/v1.2.3
If you find the project useful, a GitHub star is appreciated.
If you find a concurrency bug, even better: open an issue.
The most useful validation for a concurrency library isn't another diagram.
It's putting it under a workload the author didn't design for.