guide · stall-detection

Stuck jobs come back.

If a worker crashes or hangs mid-job, the job is not lost. bunqueue notices the silence, retries the job, and parks repeat offenders in the dead letter queue.

30s without a heartbeat marks a job stalled3 stalls before a job moves to the DLQ5s grace period after a job starts

While a worker processes a job it sends periodic heartbeats, small “I’m still alive” signals. If heartbeats stop (crashed process, hung code, dead network), the job is stalled: bunqueue re-queues it for another worker, and after too many stalls moves it to the dead letter queue (DLQ), the holding area for jobs that keep failing.

Stall detection is on by default with sensible defaults. A job may run for hours without stalling as long as automatic heartbeats continue. Tune these thresholds when one uninterrupted work segment can block heartbeats for more than 30 seconds, or when you need a different recovery budget.

Configuration

Bun

import { Queue } from 'bunqueue/client';

const queue = new Queue('my-queue', { embedded: true });

queue.setStallConfig({
  enabled: true,         // on by default
  stallInterval: 30000,  // stalled after 30s without a heartbeat
  maxStalls: 3,          // move to DLQ after 3 stalls
  gracePeriod: 5000,     // no stall checks in the first 5s of a job
});

Node.js / Deno

import { Queue } from 'bunqueue-client';

const queue = new Queue('my-queue');

await queue.setStallConfig({
  enabled: true,         // on by default
  stallInterval: 30000,  // stalled after 30s without a heartbeat
  maxStalls: 3,          // move to DLQ after 3 stalls
  gracePeriod: 5000,     // no stall checks in the first 5s of a job
});

Python

from bunqueue import Queue

queue = Queue("my-queue")

queue.set_stall_config({
    "enabled": True,         # on by default
    "stallInterval": 30000,  # stalled after 30s without a heartbeat
    "maxStalls": 3,          # move to DLQ after 3 stalls
    "gracePeriod": 5000,     # no stall checks in the first 5s of a job
})

PHP

The PHP SDK does not expose this broker command yet. Set the same queue policy through the HTTP API:

curl -X PUT http://localhost:6790/queues/my-queue/stall-config \
  -H 'content-type: application/json' \
  -d '{"config":{"enabled":true,"stallInterval":30000,"maxStalls":3,"gracePeriod":5000}}'

Go

The Go SDK does not expose this broker command yet. Set the same queue policy through the HTTP API:

curl -X PUT http://localhost:6790/queues/my-queue/stall-config \
  -H 'content-type: application/json' \
  -d '{"config":{"enabled":true,"stallInterval":30000,"maxStalls":3,"gracePeriod":5000}}'

Rust

The Rust SDK does not expose this broker command yet. Set the same queue policy through the HTTP API:

curl -X PUT http://localhost:6790/queues/my-queue/stall-config \
  -H 'content-type: application/json' \
  -d '{"config":{"enabled":true,"stallInterval":30000,"maxStalls":3,"gracePeriod":5000}}'

Elixir

The Elixir SDK does not expose this broker command yet. Set the same queue policy through the HTTP API:

curl -X PUT http://localhost:6790/queues/my-queue/stall-config \
  -H 'content-type: application/json' \
  -d '{"config":{"enabled":true,"stallInterval":30000,"maxStalls":3,"gracePeriod":5000}}'

The stall policy is server-side state per queue: a policy set from any client (or the HTTP API) governs jobs processed by workers in every language. The stall-config helper ships in the Bun, TypeScript, and Python clients; the PHP, Go, Rust, and Elixir SDKs do not expose one yet.

Option	Default	Description
enabled	true	Enable/disable stall detection
stallInterval	30000	Time (ms) without a heartbeat before a job is stalled
maxStalls	3	Max stalls before moving to DLQ
gracePeriod	5000	Initial grace period (ms) after a job starts

On the worker side, heartbeats are automatic:

Bun

const worker = new Worker('queue', processor, {
  embedded: true,
  heartbeatInterval: 10000, // heartbeat every 10 seconds (default)
});

Node.js / Deno

const worker = new Worker('queue', processor, {
  heartbeatIntervalS: 10, // heartbeat every 10 seconds (default, 0 = disabled)
});

Python

worker = Worker("queue", process, heartbeat_interval_s=10.0)  # default; 0 disables

PHP

$worker = new Worker('queue', $processor, [
    'heartbeatIntervalS' => 10.0, // Fires between jobs (sequential worker)
]);

Go

worker := bunqueue.NewWorker("queue", processor, bunqueue.WorkerOptions{
    HeartbeatIntervalS: 10, // Heartbeats are disabled by default in Go
})

Rust

let worker = Worker::new("queue", processor, WorkerOptions {
    heartbeat_interval: Some(Duration::from_secs(10)), // None disables
    ..Default::default()
});

Elixir

worker = Bunqueue.Worker.new("queue", handler, heartbeat_interval: 10_000)  # ms

Keep heartbeatInterval well below stallInterval, otherwise healthy jobs get flagged as stalled.

With SQLite persistence enabled, a custom stall policy and every job’s cumulative stall count survive process restarts. A crash consumes one attempts slot and one stallCount slot; reaching either maxAttempts or maxStalls is terminal and moves the job to the DLQ. Repeated crashes therefore cannot reset either retry budget.

Long-running jobs

A long total runtime is safe with heartbeats. Use a wider stall window when a single processing segment can block the runtime or network long enough to miss the default 30-second window:

Bun

// Video processing may take hours
const videoQueue = new Queue('video-processing', { embedded: true });

videoQueue.setStallConfig({
  stallInterval: 300000,  // 5 minutes
  maxStalls: 2,
  gracePeriod: 60000,
});

const worker = new Worker('video-processing', async (job) => {
  for (const chunk of video.chunks) {
    await processChunk(chunk);
    await job.updateProgress(chunk.progress); // also counts as a heartbeat
  }
}, { embedded: true, heartbeatInterval: 30000 });

Node.js / Deno

// Video processing may take hours
const videoQueue = new Queue('video-processing');

await videoQueue.setStallConfig({
  stallInterval: 300000,  // 5 minutes
  maxStalls: 2,
  gracePeriod: 60000,
});

const worker = new Worker('video-processing', async (job) => {
  for (const chunk of video.chunks) {
    await processChunk(chunk);
    await job.updateProgress(chunk.progress); // also counts as a heartbeat
  }
}, { heartbeatIntervalS: 30 });

Python

# Video processing may take hours
video_queue = Queue("video-processing")

video_queue.set_stall_config({
    "stallInterval": 300000,  # 5 minutes
    "maxStalls": 2,
    "gracePeriod": 60000,
})

def process(job):
    for chunk in video.chunks:
        process_chunk(chunk)
        job.update_progress(chunk.progress)  # also counts as a heartbeat

worker = Worker("video-processing", process, heartbeat_interval_s=30.0)

PHP

Configure the five-minute broker policy through the HTTP API shown above, then report progress from the sequential handler so it also refreshes the stall timer:

$worker = new Worker('video-processing', function (Bunqueue\Job $job) use ($video) {
    foreach ($video->chunks as $chunk) {
        processChunk($chunk);
        $job->updateProgress($chunk->progress);
    }
}, ['lockTtlMs' => 300000]);

Go

Configure the five-minute broker policy through the HTTP API shown above. The worker heartbeat and progress updates both refresh liveness:

worker := bunqueue.NewWorker("video-processing", func(job *bunqueue.Job) (any, error) {
    for _, chunk := range video.Chunks {
        processChunk(chunk)
        if err := job.UpdateProgress(chunk.Progress, ""); err != nil { return nil, err }
    }
    return nil, nil
}, bunqueue.WorkerOptions{HeartbeatIntervalS: 30, LockTtlMs: 300_000})

Rust

Configure the five-minute broker policy through the HTTP API shown above. The worker heartbeat and progress updates both refresh liveness:

let worker = Worker::new("video-processing", move |job| {
    for chunk in &video.chunks {
        process_chunk(chunk);
        job.update_progress(chunk.progress, None)
            .map_err(|error| ProcessError::retryable(error.to_string()))?;
    }
    Ok(Value::Nil)
}, WorkerOptions {
    heartbeat_interval: Some(Duration::from_secs(30)),
    lock_ttl_ms: 300_000,
    ..Default::default()
});

Elixir

Configure the five-minute broker policy through the HTTP API shown above. The worker heartbeat and progress updates both refresh liveness:

handler = fn job ->
  Enum.each(video.chunks, fn chunk ->
    process_chunk(chunk)
    {:ok, _} = Bunqueue.Job.update_progress(job, chunk.progress)
  end)
  {:ok, nil}
end

worker =
  Bunqueue.Worker.new("video-processing", handler,
    heartbeat_interval: 30_000, lock_ttl: 300_000
  )

Two things reset the stall timer: the worker’s automatic heartbeat (every heartbeatInterval ms) and any job.updateProgress() call. For long jobs without natural progress points, the automatic heartbeat is enough.

What happens when a job stalls

1. Retry: the job goes back to waiting with its stall count incremented, and waiting workers are notified immediately, so it is picked up without delay.
2. DLQ: the job becomes terminal when either its cumulative stall count reaches maxStalls or the interrupted delivery consumes its final normal attempts slot. A stall-budget terminal is classified as stalled; an attempts-budget terminal is classified as max_attempts_exceeded.

Listening for stalls

The Bun package can listen through embedded or TCP QueueEvents:

Bun

import { QueueEvents } from 'bunqueue/client';

const events = new QueueEvents('my-queue', {
  embedded: false,
  connection: { host: '127.0.0.1', port: 6789 },
});
await events.waitUntilReady();
events.on('stalled', ({ jobId }) => {
  console.log(`Job ${jobId} stalled`);
});

Node.js / Deno

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

Python

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

PHP

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

Go

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

Rust

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

Elixir

The network SDK does not receive broker-side stall events. Subscribe to the queue’s SSE stream and handle frames whose eventType is stalled:

curl -N http://localhost:6790/events/queues/my-queue

The Bun Worker also emits stalled in both embedded and TCP modes. Its TCP path uses the same dedicated authenticated broker subscription as QueueEvents and re-subscribes after reconnect. External network SDKs should use SSE or WebSocket for this broker-side event. Stall webhooks are not emitted, so do not register job.stalled as a webhook event. This notification is preserved when an expired lease consumes the final maxStalls or maxAttempts slot: the broker publishes stalled before the terminal failed queue event and moves the job to the DLQ.

Monitoring

Bun

const stats = queue.getDlqStats();
console.log('Stalled jobs in DLQ:', stats.byReason.stalled);

const stalledJobs = queue.getDlq({ reason: 'stalled' });

Node.js / Deno

const entries = await queue.getDlq();
const stalledJobs = entries.filter((entry) => entry.reason === 'stalled');

Python

entries = queue.get_dlq()
stalled_jobs = [entry for entry in entries if entry.get("reason") == "stalled"]

PHP

$entries = $queue->getDlq();
$stalledJobs = array_filter(
    $entries,
    fn ($entry) => ($entry['reason'] ?? null) === 'stalled'
);

Go

entries, err := queue.GetDlq(0) // 0 means no explicit count bound
stalledJobs := make([]map[string]any, 0)
for _, entry := range entries {
    if entry["reason"] == "stalled" { stalledJobs = append(stalledJobs, entry) }
}

Rust

// Entries are MessagePack maps; inspect each map's `reason` field.
let entries = queue.get_dlq(None)?;

Elixir

{:ok, entries} = Bunqueue.Queue.dlq(queue)
stalled_jobs = Enum.filter(entries, &(&1["reason"] == "stalled"))

The aggregate getDlqStats() and the reason filter are Bun-client conveniences. The external SDKs list DLQ entries with getDlq(count?) / get_dlq() / GetDlq(count) and each entry carries a reason field ('stalled' for stall evictions) you can filter on client-side.

SandboxedWorker

Experimental

SandboxedWorker depends on experimental Bun Workers. For production, use the standard Worker. See Worker vs SandboxedWorker.

SandboxedWorker also sends heartbeats automatically in both modes; in embedded mode heartbeatInterval defaults to 5000 ms. If its jobs run longer than stallInterval, either raise stallInterval, call progress() periodically, or disable stall detection with queue.setStallConfig({ enabled: false }).

Related Guides

• Dead Letter Queue - Where stalled jobs end up after max stalls
• Worker API - Configure heartbeat intervals
• CPU-Intensive Workers - Prevent stalls in CPU-heavy workloads