guide · worker

More at once, fewer round-trips.

Concurrency decides how many jobs a single worker runs in parallel. Batch pulling decides how many it fetches per request. Together they set the ceiling on what one process can do.

Process jobs in parallel

Bun

const worker = new Worker('my-queue', processor, {
  embedded: true,
  concurrency: 5,  // Up to 5 jobs at once (default: 1)
});

Node.js / Deno

const worker = new Worker('my-queue', processor, {
  concurrency: 5,  // Up to 5 jobs at once (default: 4)
});

Python

worker = Worker("my-queue", process, concurrency=5)  # default: 4

PHP

// The PHP worker is sequential by design: one job at a time.
// Scale out by running more worker processes instead:
//   php worker.php & php worker.php &
$worker = new Worker('my-queue', $processor);

Go

worker := bunqueue.NewWorker("my-queue", processor, bunqueue.WorkerOptions{
    Concurrency: 5, // Up to 5 jobs at once (default: 4)
})

Rust

let worker = Worker::new("my-queue", processor, WorkerOptions {
    concurrency: 5, // Up to 5 jobs at once (default: 4)
    ..Default::default()
});

Elixir

worker = Bunqueue.Worker.new("my-queue", handler, concurrency: 5)  # default: 1

You can change concurrency at runtime, without restarting:

Bun

worker.concurrency = 10;  // Scale up under load
worker.concurrency = 2;   // Scale back down (minimum: 1)

Node.js / Deno

The network SDK fixes concurrency at construction. Drain and replace the worker to change it safely:

await worker.close();
worker = new Worker('my-queue', processor, { concurrency: 10 });

Python

The Python SDK fixes concurrency at construction. Drain and replace the worker to change it safely:

worker.close()
worker = Worker("my-queue", process, concurrency=10)

PHP

PHP workers are sequential. Change process-level parallelism by starting or stopping worker processes under your process supervisor.

php worker.php &
php worker.php &

Go

The Go SDK fixes concurrency at construction. Stop and replace the worker:

worker.Stop()
worker.Close()
worker = bunqueue.NewWorker("my-queue", processor,
    bunqueue.WorkerOptions{Concurrency: 10})

Rust

The Rust SDK fixes concurrency at construction. Stop and replace the worker:

worker.stop();
worker.close();
let worker = Worker::new("my-queue", processor, WorkerOptions {
    concurrency: 10,
    ..Default::default()
});

Elixir

The Elixir SDK fixes concurrency at construction. Stop and replace the worker:

:ok = Bunqueue.Worker.stop(worker)
worker = Bunqueue.Worker.new("my-queue", handler, concurrency: 10)

Runtime concurrency changes are a Bun client feature; the external SDKs fix concurrency at construction.

Batch pulling

For high-volume queues, pull many jobs per round-trip and long-poll while idle:

Bun

const worker = new Worker('queue', processor, {
  embedded: true,
  batchSize: 100,     // Request up to 100; free concurrency slots cap the pull
  pollTimeout: 5000,  // Wait up to 5s for jobs (long polling)
});

Node.js / Deno

const worker = new Worker('queue', processor, {
  batchSize: 100,       // Request up to 100; free concurrency slots cap the pull
  pollTimeoutMs: 5000,  // Wait up to 5s for jobs (long polling)
});

Python

worker = Worker(
    "queue",
    process,
    batch_size=100,        # Request up to 100; free concurrency slots cap the pull
    poll_timeout_ms=5000,  # Wait up to 5s for jobs (long polling)
)

PHP

$worker = new Worker('queue', $processor, [
    'batchSize' => 100,       // Request up to 100; free slots cap the pull
    'pollTimeoutMs' => 5000,  // Wait up to 5s for jobs (long polling)
]);

Go

worker := bunqueue.NewWorker("queue", processor, bunqueue.WorkerOptions{
    BatchSize:     100,  // Request up to 100; free slots cap the pull
    PollTimeoutMs: 5000, // Wait up to 5s for jobs (long polling)
})

Rust

let worker = Worker::new("queue", processor, WorkerOptions {
    batch_size: 100,        // Request up to 100; free slots cap the pull
    poll_timeout_ms: 5_000, // Wait up to 5s for jobs (long polling)
    ..Default::default()
});

Elixir

worker =
  Bunqueue.Worker.new("queue", handler,
    batch_size: 100,     # Request up to 100; free slots cap the pull
    poll_timeout: 5_000  # Wait up to 5s for jobs (long polling)
  )

Bulk pushes wake idle long-polling workers immediately, they never wait out the timeout.

Where to go next

Worker	Create a worker and process your first job
The Job Object Inside a Worker Processor	Everything the processor receives and can do
Worker Events	completed, failed, stalled and the rest
Worker Error Handling, Retries and Backoff	Retries, backoff, timeouts and giving up
Worker Lifecycle	Pause, resume and shut down without losing work
Heartbeats, Stall Detection and Lock Ownership	Heartbeats, stall recovery and lock ownership
SandboxedWorker	Experimental isolation for CPU-heavy handlers
WorkerOptions Reference	Every WorkerOptions field, with defaults