reference · faq

Asked, answered.

One-paragraph answers on storage, modes, performance, retries, scaling and migration. If your question is not here, GitHub Discussions is the next stop.

Short answers to the questions people actually ask. Each answer links to the page that owns the topic.

Basics

What is bunqueue?

bunqueue is a job queue for Bun: you push jobs (units of work, like “send this email”) onto a named queue, and workers pull and process them with retries, priorities, and scheduling. It uses memory/SQLite by default and offers an optional PostgreSQL 15–18 server backend for multiple brokers; 18.6 is recommended. Its API is compatible with BullMQ, so migration is mostly an import change. Start with the quickstart.

Why SQLite instead of Redis?

For the default deployment it means one less server. There is nothing to install or monitor, and backup is one safe SQLite snapshot (or the built-in S3 backup). Bun’s native SQLite bindings keep the hot path synchronous; see the comparison. When brokers themselves must scale horizontally, select the separate PostgreSQL backend instead.

Does it run on Node.js?

The server is Bun-only (bun:sqlite, Bun.serve, Bun.listen do not exist in Node), but your producers and workers run anywhere: official SDKs exist for TypeScript (Node.js, Deno, Bun, Cloudflare Workers), Python, PHP, Go, Rust and Elixir, all speaking the same TCP protocol.

bunqueue-client builds its default API from the same TypeScript sources as bunqueue/client, including Queue, Worker, QueueEvents, QueueGroup, FlowProducer, and Simple Mode. On Node.js and Deno, use embedded: false, nested connection options, and the Async queue methods for remote reads and mutations. The historical SDK API remains available explicitly from bunqueue-client/legacy.

What are the requirements?

Bun 1.4.0 or newer (enforced via the package engines field) on macOS, Linux, or Windows via WSL. An SSD helps write throughput. Install steps are on the installation page.

How heavy is the install?

One runtime dependency: msgpackr for the MessagePack wire and persistence formats. Cron parsing uses Bun’s native Bun.cron.parse(). The MCP SDK is an optional peer dependency: only install @modelcontextprotocol/sdk if you use the MCP server, the launcher tells you if it is missing. Queue and Worker users never need it.

Modes and storage

What is the difference between embedded and server mode?

Embedded mode (new Queue('q', { embedded: true })) runs the queue inside your process with memory/SQLite. Server mode runs bunqueue start and clients connect over TCP. A server may use memory/SQLite with one broker or PostgreSQL 15–18 with multiple brokers. Pick one backend per deployment; the same SQLite file must never be opened by two processes at once.

Where is my data stored?

Nowhere, unless you say so. Without a data path or PostgreSQL URL, jobs are held in memory and lost on restart. Set dataPath in embedded mode, a SQLite data-path variable/flag for one server, or BUNQUEUE_POSTGRES_URL plus the PostgreSQL driver for a multi-broker server:

BUNQUEUE_DATA_PATH=./data/production.db bunqueue start

# or, in standalone multi-broker mode
BUNQUEUE_STORAGE_DRIVER=postgres \
BUNQUEUE_POSTGRES_URL='postgres://bunqueue:secret@postgres:5432/bunqueue' \
bunqueue start

How does persistence work?

Jobs are written to SQLite in WAL mode (write-ahead logging, a journal that lets reads and writes overlap). By default writes are buffered for up to 10 ms and flushed in batches. If that process-crash window is unacceptable for a job, add it with { durable: true } to commit before add() returns. This closes bunqueue’s application buffer; power-loss durability still depends on SQLite’s host, filesystem, and storage. Current native evidence measures public on-disk addBulk and sequential durable adds as separate workloads; see benchmarks rather than comparing one rounded headline.

In PostgreSQL mode, PostgreSQL is authoritative and each mutation is transactional. Competing brokers claim with row locks and SKIP LOCKED; opaque database-clock leases fence stale ACK/FAIL attempts. SQLite buffering and its published performance figures do not apply to that backend.

Can I use PostgreSQL or MySQL instead?

PostgreSQL 15–18 is supported in standalone server mode and allows multiple active bunqueue brokers to share one database/namespace; 18.6 is pinned and recommended. SQLite remains the default and the only persistent embedded backend. MySQL is not supported. See storage backends for configuration and boundaries.

How do I back up and restore?

For SQLite, enable the built-in S3 backup (S3_BACKUP_ENABLED=1 plus bucket and credentials) or take a safe SQLite snapshot. Restore with bunqueue backup list and bunqueue backup restore <key> --force. For PostgreSQL, use your database provider’s backup and point-in-time-recovery tooling; bunqueue’s S3 command does not snapshot PostgreSQL. Full SQLite guide: backup.

Performance

How fast is it?

The current native campaign measured a 186,384 jobs/s median for public on-disk Embedded addBulk, 158,779 jobs/s for TCP PUSHB, and 60,835 / 27,191 ops/s for sequential durable Embedded/TCP adds. These are different workloads, not one interchangeable throughput number. Dated distributions, integrity totals, and methodology live on the benchmarks page.

How do I get more throughput?

Three knobs, in order of impact: raise worker concurrency (parallel jobs per worker), batch your inserts with queue.addBulk(jobs) (one round-trip for many jobs), and raise the worker’s batchSize so it pulls and acknowledges jobs in batches. In TCP mode, concurrent queue.add() calls are also auto-batched for you by default. See worker options.

Jobs and retries

How does deduplication work?

Pass a jobId. Adding the same jobId twice returns the existing job instead of creating a duplicate, same behavior as BullMQ. This makes webhook handlers and restart-recovery code safe to re-run:

Bun

await queue.add('charge', data, { jobId: `order-${orderId}` }); // idempotent

Node.js / Deno

await queue.add('charge', data, { jobId: `order-${orderId}` }); // idempotent

Python

queue.add("charge", data, job_id=f"order-{order_id}")  # idempotent

PHP

$queue->add('charge', $data, ['jobId' => "order-{$orderId}"]); // idempotent

Go

queue.Add("charge", data, bunqueue.JobOptions{"jobId": "order-" + orderID}) // idempotent

Rust

// idempotent
queue.add("charge", data, JobOptions {
    job_id: Some(format!("order-{order_id}")),
    ..Default::default()
})?;

Elixir

# idempotent
{:ok, _job} = Bunqueue.Queue.add(queue, "charge", data, jobId: "order-#{order_id}")

How do retries and backoff work?

attempts sets the maximum tries, backoff sets the wait between them. A plain number means exponential backoff (waits roughly double each retry: ~2s, ~4s, ~8s with a 1000ms base); the object form { type: 'fixed' | 'exponential', delay } matches BullMQ. All delays get automatic jitter, a small random spread so thousands of failed jobs do not retry in the same instant, and are capped at 1 hour by default.

Bun

await queue.add('task', data, { attempts: 5, backoff: 1000 });

Node.js / Deno

await queue.add('task', data, { attempts: 5, backoff: 1000 });

Python

queue.add("task", data, attempts=5, backoff=1000)

PHP

$queue->add('task', $data, ['attempts' => 5, 'backoff' => 1000]);

Go

queue.Add("task", data, bunqueue.JobOptions{"attempts": 5, "backoff": 1000})

Rust

use bunqueue_client::{Backoff, JobOptions};

queue.add("task", data, JobOptions {
    attempts: Some(5),
    backoff: Some(Backoff::Milliseconds(1000)),
    ..Default::default()
})?;

Elixir

{:ok, _job} = Bunqueue.Queue.add(queue, "task", data, attempts: 5, backoff: 1000)

What happens when a worker crashes mid-job?

Workers send heartbeats (periodic “still alive” pings). If one goes silent, stall detection marks its active jobs as stalled and requeues them. A job that stalls too many times goes to the dead letter queue instead of looping forever. See stall detection.

What is the dead letter queue?

The DLQ is the holding area for jobs that exhausted their retries, threw an unrecoverable error, or stalled too many times. Nothing is silently dropped: you can inspect entries, retry them (queue.retryDlq()), or purge them. It also supports auto-retry and expiration policies. See DLQ.

Can I control processing order?

FIFO (first in, first out) is the default for jobs of equal priority, so ordered processing needs no options. Use { priority: 10 } to jump the line (higher runs sooner), or { lifo: true } for the newest-first LIFO partition. At equal numeric priority, LIFO jobs run ahead of FIFO jobs; FIFO entries retain oldest-first order within their partition.

Scaling and production

Can I run multiple workers?

Yes. Any number of worker processes can connect over TCP and share queues. With SQLite the queue broker stays single while workers multiply. With PostgreSQL 15–18, both workers and bunqueue broker processes can multiply; 18.6 is recommended.

Does bunqueue support multiple brokers or high availability?

Yes, when server mode uses PostgreSQL 15–18; 18.6 is recommended. Brokers share authoritative rows, leases, limits, cron/worker state, and durable events; a stale broker cannot finalize a lease recovered elsewhere. Database availability, routing, backup, and failover remain operator responsibilities. SQLite mode deliberately stays single-broker. This is not a multi-region consensus layer; see storage backends and deployment.

Is bunqueue production-ready?

Yes. Retries with backoff, stall detection, a dead letter queue, rate limiting, native TLS, and Prometheus metrics are built in. SQLite mode also includes S3 snapshots; PostgreSQL deployments use their database’s backup and PITR tooling. The production guide covers deployment, sizing, and operations.

Migration

Can I migrate from BullMQ?

Yes, and it is usually small: change the import to bunqueue/client, delete the Redis connection, add embedded: true (or a TCP connection). Queue, Worker, QueueEvents, FlowProducer, job options, and events keep the same shapes. The real differences (backoff shorthand, repeat.cron renamed to pattern, boolean-only removeOnComplete) are listed in the migration guide.

Can I migrate from other queues?

There is no importer, but the job format is plain JSON. Export your jobs and bulk-insert them:

await queue.addBulk(
  oldJobs.map((j) => ({
    name: j.type,
    data: j.payload,
    opts: { priority: j.priority },
  }))
);

Workflows

What is the Workflow Engine, and when do I use it over FlowProducer?

FlowProducer builds parent-child job trees: fan out children, run the parent when they finish. The Workflow Engine is a step-by-step orchestrator for business processes: ordered steps with per-step retry, conditional branching, parallel blocks, loops, saga compensation (automatic rollback of completed steps when a later one fails), and waitFor signals for human approval. Use FlowProducer for job dependency graphs, Workflow for processes with rollback, branching, or human decisions. See workflow and flows.

Do workflows survive restarts?

Yes, when the Engine has a persistent dataPath. Its execution state (current step, step results, received signals) lives in that local SQLite file, so recover() can resume it after a restart. Without dataPath, the workflow store is in memory and does not survive process exit. The Engine can enqueue work through embedded, TCP, or PostgreSQL-backed bunqueue servers, but PostgreSQL does not replace this separate local workflow store.

Troubleshooting

”SQLITE_BUSY: database is locked”

Two processes are writing to the same SQLite file. Run exactly one embedded instance per file, or switch to server mode so all processes go through one server. More cases in troubleshooting.

”Job not found”

The job was already removed: it completed with removeOnComplete: true, failed with removeOnFail: true, or was deleted manually. Completed-job records are also bounded in memory, so very old results eventually age out.

High memory usage

Usually accumulation: completed jobs kept around, a growing DLQ, or oversized job payloads (keep payloads small, store big blobs elsewhere and pass a reference). Add jobs with removeOnComplete: true, purge the DLQ periodically (queue.purgeDlq()), and clean old jobs with queue.clean(3600000, 1000) (grace period in ms, max jobs to remove).

Contributing

How can I contribute?

Report bugs on GitHub Issues, propose features in Discussions, or send a PR. To develop locally: clone the repo, bun install, bun test.

Related

• Troubleshooting, debug common issues
• Quickstart, first queue in two minutes
• bunqueue vs BullMQ, features and honest benchmarks