PGMQ: When Postgres Is All the Message Queue You Need

PGMQ: When Postgres Is All the Message Queue You Need

If Postgres is already running, your first message queue should probably live inside it too. PGMQ turns Postgres tables into a real queue — visibility timeouts, transactional enqueue, dead-letter handling — with no new broker and no second system to maintain.

At some point in every backend's life, a job needs to happen later or elsewhere: send the invoice email, recompute the report, fire the webhook. The reflex answer is "add a message queue," and the reflex implementation is Kafka or RabbitMQ. On a large share of the systems I build and review, that reflex bolts a distributed system onto a product whose database could already do the job.

A queue that lives inside your database

PGMQ is a Postgres extension that implements a message queue as tables plus a SQL API: pgmq.send(), pgmq.read(), pgmq.delete(), pgmq.archive(). There is no broker process, no new port, no new deployment artifact.

Client libraries exist where you want the ergonomics — in Node, pgmq-js wraps queue creation, send, and read in a few calls. But everything is ultimately SQL against the database you already operate.

That framing matters more than any feature list: the queue inherits everything your Postgres already has. Backups, replication, monitoring, access control, point-in-time recovery — none of it has to be built a second time for a second system.

Visibility timeout, not locks

Think of a restaurant marking a table "being served" the moment a waiter takes the order. Other waiters can't grab that table — but if the first waiter vanishes without finishing, the table opens back up and someone else can take it. Visibility timeout works the same way.

PGMQ's delivery model is the one SQS made famous. A consumer reads a message with a visibility timeout — say 30 seconds. The message doesn't leave the queue; it becomes invisible to other consumers for that window. Finish the work and delete or archive the message, and it's gone for good. Crash mid-job and do nothing, and the message simply reappears when the timeout expires, ready for the next consumer.

That gives you at-least-once delivery with crash recovery and no stuck locks. It also puts one obligation on you: consumers must be idempotent, because "at least once" occasionally means twice. Design every handler so a repeat delivery is harmless, and the semantics take care of themselves.

The killer feature: transactional enqueue

Here is what Kafka and RabbitMQ cannot give you at any price: enqueueing a message in the same commit as your business data.

With an external broker there are two systems, so there are two failure modes. Commit the order but fail to publish the event, and downstream never hears about it. Publish the event but roll back the order, and downstream processes something that doesn't exist.

The industry's answer is the outbox pattern — write to an outbox table transactionally, relay from it later — which is a polite way of saying "build half a queue inside Postgres anyway." With PGMQ the queue is a table in the same database:

BEGIN;
INSERT INTO orders (...) VALUES (...);
SELECT pgmq.send('notifications', '{"order_id": 118}');
COMMIT;

One commit. Either the order and its message both exist, or neither does. In an operations backend I built for a logistics company, this single property eliminated a reconciliation job whose only purpose had been catching dual-write drift between the database and the broker.

The operational weight you skip

Kafka is brokers, a controller quorum, partitions, consumer groups, retention tuning, and a dashboard you check when rebalancing goes sideways. RabbitMQ is milder, but it's still another stateful cluster with its own upgrade path and failure modes. Both are worth that cost when you genuinely need them. Neither is ever free.

A queue in Postgres is one more schema. It's backed up by the backup you already take, monitored by the monitoring you already run, and upgraded when the database is. For a small team, that difference decides whether the on-call rotation is survivable.

Throughput ceilings, and when to graduate

Honest numbers: on modest hardware I've benchmarked PGMQ at single-digit millisecond read latencies and thousands of messages per second — several orders of magnitude above what a typical line-of-business backend produces.

The ceiling is real, though. Consumers poll rather than receive pushes, hot queue tables need vacuum attention, and everything shares I/O with your application.

Graduate when the workload changes shape, not just size: sustained tens of thousands of messages per second, replay of history, or many independent consumer groups reading the same stream. Those are event-log semantics, and Kafka is an event log. PGMQ is a work queue. Different tools — most backends only ever need the second one.

Dead letters

Poison messages — the ones that crash their consumer every time — must not circulate forever. PGMQ tracks a read count per message, so dead-letter handling is a few lines of SQL: when read_ct passes your threshold, move the message to a dead-letter queue and alert on its depth. The archive table completes the picture: every processed message stays queryable, an audit trail you'd otherwise have to build.

The takeaway

You get visibility-timeout semantics, transactional enqueue that brokers structurally cannot offer, and an operational story identical to the database you already know. Move to Kafka when you need an event log, not because a diagram looked more impressive with one.

This is the same philosophy as grouping across time zones in PostgreSQL: push the work into the database that's already there. Many of the IoT backends I build run their job queues exactly this way.