blog · design
A job queue without Redis.
BullMQ, Bee-Queue, and friends all assume a running Redis server. bunqueue can keep a durable queue in-process with SQLite instead: simpler deployments and zero external services, with a separate PostgreSQL multi-broker path when you scale. Unconfigured queues use memory storage.
## The Redis Problem
Redis is excellent software, but it introduces real operational complexity for job queues:
- **Another service to manage** - deploy, monitor, backup, scale
- **Network latency** - every job operation crosses the network
- **Memory limits** - Redis stores everything in RAM
- **Connection management** - pool sizing, reconnection, timeouts
For many applications, especially single-server deployments and small teams, this overhead isn't justified.
## Why SQLite?
SQLite runs in-process. There's no network hop, no connection pool, no separate service to manage.
Zero Dependencies
No Redis, no external services. Just your app and a file on disk, with msgpackr{' '} as the only runtime dependency.
In-Process Speed
Direct memory access instead of TCP round-trips to Redis.
Persistence When Selected
Set a SQLite data path for WAL-backed recovery with concurrent reads; without one, the queue is intentionally in memory.
Simple Backups
Copy one file, or use built-in S3 backup.
## Bun Makes It Possible
Bun's native SQLite driver (`bun:sqlite`) is what makes this architecture viable. It's not a Node.js addon compiled from C - it's built directly into the runtime with zero-copy optimizations.
```typescript
import { Database } from 'bun:sqlite';
const db = new Database('./queue.db', { create: true });
// WAL mode for concurrent reads + writes
db.exec('PRAGMA journal_mode = WAL');
db.exec('PRAGMA synchronous = NORMAL');
```
This gives us prepared statements, transactions, and WAL mode with performance that rivals in-memory datastores for our workload patterns.
## The Write Buffer Strategy
Raw SQLite writes are fast, but bunqueue goes further with a **write buffer** that batches disk operations:
```typescript
// Jobs are added to an in-memory priority queue immediately
// The write buffer flushes to SQLite every 10ms
const queue = new Queue('emails', {
embedded: true,
dataPath: './data/bunqueue.db',
});
// This returns instantly - job is in memory
await queue.add('send', { to: 'user@example.com' });
// For critical jobs, bypass the buffer:
await queue.add('payment', data, { durable: true });
```
The `durable: true` option bypasses bunqueue's write buffer and commits to
SQLite before `add()` returns, trading throughput for a smaller process-crash
window. Host, filesystem, and physical-media durability still depend on the
deployment.