bunqueue v2.8.12
Fixes 2 bugs. Re-check the gate before starting.
To install bunqueue v2.8.12
Section titled “To install bunqueue v2.8.12”bun add bunqueue@2.8.12npm install bunqueue@2.8.12To upgrade bunqueue
Section titled “To upgrade bunqueue”bun add bunqueue@latestBugfixes
Section titled “Bugfixes”The concurrency gate lived only in poll() (activeJobs >= concurrency), but the
counter is incremented later in startJob(), with await doPullBatch() (a TCP
round-trip) in between. Nothing serialized concurrent tryProcess() runs, so a burst
of fast-completing jobs (e.g. a DLQ retry that finds nothing to do) could fire several
finally → poll → tryProcess calls that all passed the gate while activeJobs was
still low, all suspended at the pull await, and each then called startJob(), driving
activeJobs past the configured limit. A second path made it worse: startJob()
schedules tryProcess() via setImmediate, which bypasses poll()’s gate entirely.
Reported over TCP with a slow network: up to 10 jobs in flight against a concurrency
of 3.
- Re-check the gate before starting.
tryProcess()now re-testsactiveJobs >= concurrencyimmediately beforestartJob(). There is noawaitbetween the check andstartJob()’sactiveJobs++, so the check is atomic with the increment and cannot overshoot. This single guard closes both the pull-await path and thesetImmediatebypass. - No job loss. When the gate is closed the already-pulled job is requeued to the front of the worker’s local buffer (it stays owned via the pull lock) and is started as soon as a slot frees.
Reproduced with a deterministic test that models the slow pull
(test/issue96-concurrency-race.test.ts): observed concurrency now stays at the limit
(was 4 with concurrency: 3).
Thanks to 1 contributor!
Section titled “Thanks to 1 contributor!”← bunqueue v2.8.11 · Release on GitHub · Commits since v2.8.11 · Full changelog · All posts