Client SDK Architecture: Pooling & Worker Modes

architecture · client sdk

Thin client, smart server.

The client layer provides the interface for applications to interact with bunqueue. It supports both embedded (in-process) and TCP (server) modes.

Module Structure

src/client/
├── queue/          # Job submission (Queue class)
├── worker/         # Job processing (Worker class)
├── tcp/            # Network communication
├── flow.ts         # Job dependencies (FlowProducer)
└── queueGroup.ts   # Namespace isolation

Dual-Mode Architecture

Dual-mode architectureone API, two transports

Application Queue.add(), Worker.process()

↓

Embedded mode direct calls to QueueManager

TCP mode TcpPool → Server, msgpack protocol

Mode	Throughput	Use Case
Embedded	~100k jobs/sec	Single process
TCP	Network limited	Distributed workers

Job Submission Flow

Job submissionQueue.add(name, data, options)

Merge options with defaults

↓

Mode check

↓

Embedded direct manager.push()

TCP tcpPool.send({ cmd: 'PUSH', queue, data, priority, delay, ...options })

↓

Return Job with methods

Auto-batching (TCP mode, on by default): concurrent add() calls are transparently coalesced into a single PUSHB round-trip (defaults: maxSize 50, maxDelayMs 5). Sequential awaits send immediately with no penalty; durable jobs bypass the batcher and go out as individual PUSH.

Worker Processing Flow

Worker processing loopWorker(queue, processor, options)

Start heartbeat timer default: 10s interval

↓

Poll loop respects concurrency limit

↓

Pull batch from server PULLB command

↓

for each job

1. Mark active

2. Execute processor(job)

3. On success: ACK batch

4. On failure: FAIL

After each batch the worker returns to the poll loop.

Connection Pool Architecture

TcpConnectionPool4 connections per pool, default

Client 1 socket, parser, health

Client 2 socket, parser, health

Client 3 socket, parser, health

Client 4 socket, parser, health

Round-robin selection, health tracking, auto-reconnect, shared pool management

Key Features:

4 connections per pool (default)
Load-aware client selection
Automatic reconnection with exponential backoff
Shared pools across Queue/Worker instances

Heartbeat & Stall Detection

Heartbeat and stall detectionworker to server

worker

heartbeatTimer every 10s sends JobHeartbeatB { ids, tokens } to the server

pulledJobIds all pulled jobs get heartbeat

activeJobIds jobs being processed

jobTokens lock tokens for verification

↓

server

No heartbeat for stallInterval (30s): 1. mark job as stalled, 2. increment stallCount, 3. after maxStalls (3) move to DLQ

ACK Batching Flow

ACK batchingjob completes

AckBatcher.queue(id, result)

↓

Buffer pending ACKs max 10 or 50ms timeout

↓

Batch full

Timer fires

↓

Send ACKB { ids, results, tokens }

Benefits:

Reduces network round-trips
Batches lock verification
Handles retry on failure

FlowProducer (Dependencies)

The Bun client plans the complete graph and commits it through one PUSHF operation in both embedded and TCP modes. Validation occurs before mutation; all affected shard locks are held through publication, and configured SQLite commits every node before workers are notified. All six current external SDKs use the same atomic command. PUSH plus UpdateParent remains compatible for previously published clients; a predeclared late edge is a child-only durable back-patch and never rewrites an active or terminal parent.

Dependency chainaddChain([A, B, C])

A no dependencies, queued

→

B dependsOn: [A]

→

C dependsOn: [B]

↓

Server tracks in waitingDeps until dependencies complete

Graceful Shutdown

Graceful shutdownworker.close()

1. Stop poll loop

↓

2. Stop heartbeat

↓

3. Wait active jobs finish

↓

4. Flush pending ACKs

↓

5. Wait in-flight flushes

↓

6. Close TCP connections