Message Queue

SNS/SQS-style durable messaging: topics fan out to queues through subscriptions with attribute filters, consumers pull messages under a visibility timeout and acknowledge by deleting, FIFO queues preserve per-group order, and poison messages move to a dead letter queue you can inspect and redrive. Every example on this page was executed against a live server.

When to reach for it: order pipelines, work distribution across competing consumers, fan-out where each consumer group needs its own durable copy, retry-until-acked processing with a DLQ safety net.

When not to: scheduled or delayed background jobs with worker retry policies belong to Job Queue; ephemeral fire-and-forget broadcasts to connected clients belong to PubSub.

Concepts

• Topics are the write side. Publishers send to a topic, never to a queue directly.
• Queues are the read side. Every queue subscribed to a topic receives its own durable copy of each message, so consumer groups process independently.
• Subscriptions connect the two, optionally with a filter policy that routes only matching messages into the queue.
• At-least-once delivery - receiving a message hides it for the visibility timeout instead of removing it. Deleting it with the receipt handle is the acknowledgment; an unacked message reappears.
• FIFO is a queue property (there is no FIFO topic): a --fifo queue delivers in order, and group_id labels a message's ordering group.
• Dead letter queue - a message received more than max_receive_count times without being acked moves to the queue's DLQ (any ordinary queue named at creation time) instead of being delivered again.

The pipeline

Topic, DLQ, work queue, subscription - then publish, receive, ack:

snug mq topics create -n order-events -r 1440 -t env=prod  # -r is retention in MINUTES
snug mq queues create -n failed-orders
snug mq queues create -n order-processing -v 30 --max-receives 5 --dlq failed-orders
snug mq subscriptions create -t order-events -q order-processing -f '{"type":["created"]}'

snug mq publish -t order-events -b '{"order_id":123,"total":49.99}' -a type=created
snug mq receive -q order-processing -m 10 -w 20   # long-poll up to 20s
snug mq delete -q order-processing -r rh_PnsQcyJTLYSs   # ack, receipt handle from receive

Over HTTP, publishing is a POST to the topic's messages collection:

curl -X POST -H "Authorization: Bearer $SNUG_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"body":{"order_id":125,"total":12.50},"attributes":{"type":"created"}}' \
  http://localhost:4000/api/v1/mq/topics/order-events/messages

{
  "status": 200,
  "msg": "OK",
  "data": {
    "message_id": "msg_tsJbTyjKHSUB",
    "sequence_number": "00000000000000000003",
    "published_at": "2026-08-28T05:01:36.631190Z",
    "delivered_to_queues": 1
  }
}

Watch delivered_to_queues: publishing succeeds even when no queue gets the message - a topic with no subscriptions, or a filter that matched nothing, returns delivered_to_queues: 0, not an error.

Filtering

A filter policy maps attribute names to arrays of allowed values (not glob patterns) and matches against the -a key=value attributes on publish. Verified: with the policy {"type":["created"]} above, a message published with -a type=cancelled returned delivered_to_queues: 0 and never appeared in the queue; -a type=created was delivered. A message missing the attribute entirely does not match either. -a is repeatable.

Deduplication

--dedup-id works on any topic (not FIFO-only). Republishing with the same ID inside the 5-minute window fails:

snug mq publish -t order-events -b '{"n":1}' --dedup-id order-123-created
# again within 5 minutes: 400 duplicate_message

Batch publishing

publish-batch takes a JSON file - a bare array of at most 10 messages, each with the same fields as a single publish - and splits the response into successful (with message_id and sequence_number) and failed arrays. delete-batch on the consumer side has the same 10-entry cap.

snug mq publish-batch -t order-events -f messages.json

[
  { "body": {"order_id": 301}, "attributes": {"type": "created"} },
  { "body": {"order_id": 302}, "attributes": {"type": "created"} }
]

FIFO queues

snug mq queues create -n job-runner --fifo -v 60
snug mq subscriptions create -t jobs -q job-runner
snug mq publish -t jobs -b '{"step":1}' --group-id build-42

Verified: three messages published with the same --group-id came back from a single receive in publish order, each carrying message_group_id and an ascending sequence_number in its system attributes.

Inspecting a queue

queues get returns the configuration plus live statistics and the subscriptions feeding it:

snug --output json mq queues get order-processing

{
  "name": "order-processing",
  "visibility_timeout_seconds": 30,
  "max_receive_count": 5,
  "dead_letter_queue": "failed-orders",
  "statistics": {
    "approximate_message_count": 0,
    "approximate_not_visible_count": 0,
    "...": "..."
  },
  "subscriptions": [
    { "topic_name": "order-events", "filter_policy": { "type": ["created"] } }
  ]
}

topics list and queues list enumerate your resources with the standard pagination object. Topics and queues are per-user: another user's receive on your queue fails with a plain 404 queue_not_found (verified with a second token).

Behaviors and gotchas

• Retention flags are minutes, not seconds: -r 1440 keeps messages one day; the default is 10080 (7 days).
• Each queue copy is its own message. Publish returns the topic-side message_id; the copy a consumer receives has a different one (the topic sequence_number is preserved).
• Teardown: subscriptions delete -t <topic> -s <queue> detaches a queue; queues delete <name> and topics delete <name> take a positional name with no confirmation flag; queues purge <name> drops all messages and requires --confirm.
• Receiving, visibility timeouts, receipt-handle lifetimes, and the DLQ loop have their own page: Receiving, visibility, and the DLQ.

Limits and configuration

Message bodies up to 256 KB, 10 messages per receive or batch, 20-second maximum long-poll, 12-hour maximum visibility timeout, and 1000 topics/queues per user by default - all tunable via the MQ_* variables in the Message Queue CONFIG reference.

Reference

• API reference: Topics, Queues, Subscriptions, Publishing, Receiving, DLQ
• Related: Job Queue for scheduled background jobs, PubSub for ephemeral broadcast, Webhooks for push delivery to external URLs