Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/azure-service-bus-transport.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
'@node-ts/bus-azure-service-bus': minor
---

Add `@node-ts/bus-azure-service-bus`, an Azure Service Bus transport (#346).

- Each message is sent to a topic of its own, and each service receives from its own queue, which a subscription on every topic it handles forwards into. `resolveTopicName` maps message names to topics.
- Connect with a `connectionString`, or a `fullyQualifiedNamespace` and a `TokenCredential`, or pass your own `ServiceBusClient` and `ServiceBusAdministrationClient`.
- Messages are received `concurrency` at a time in peek-lock mode, with their locks renewed while they're handled (`lockDuration`, `maxAutoLockRenewalDurationInMs`).
- Retries schedule a copy of the message after the recoverability policy's delay, with its own message id. Dead-lettered messages use Service Bus' dead-lettering, with the failure metadata in a `bus-failure` application property, and are forwarded to the shared dead letter queue.
- `bus provision` creates the topics, queues and subscriptions with the Manage right, and plans the Azure role assignments the service needs at runtime (`azure-rbac`). At startup the transport checks its queue with a peek, which only needs Listen, and its subscriptions too with `verifySubscriptions`.
- Replies go straight to the requester's queue in the same namespace. `AzureServiceBusMessageTooLarge`, `AzureServiceBusTierNotSupported` and `AzureServiceBusConnectionNotConfigured` name the problem and the fix.
45 changes: 44 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ jobs:
curl -sf http://127.0.0.1:4566/_localstack/health
- run:
name: Integration Test
command: pnpm run test:integration
# bus-azure-service-bus' tests need the Service Bus emulator, which
# the azure-service-bus job runs, so they're left out here
command: pnpm exec dotenv -e test.env -- jest --runInBand '^(?!(.*/)?packages/bus-azure-service-bus/).*(src/.+\.|/)integration\.ts$'
- run:
name: Integration Test (Redis transport on Valkey)
command: REDIS_URL=redis://127.0.0.1:6381 pnpm exec dotenv -e test.env -- jest --runInBand --coverage=false packages/bus-redis/src/redis-transport.integration.ts
Expand All @@ -96,6 +98,45 @@ jobs:
paths:
- .

azure-service-bus:
# The Service Bus emulator needs a mounted config file, SQL Server and
# about 2 GB of RAM, so it runs with docker compose on a machine executor
# rather than as secondary containers. Starting it accepts the licences of
# the emulator and SQL Server (ACCEPT_EULA in docker-compose.yml).
machine:
image: ubuntu-2404:current
resource_class: medium
working_directory: ~/repo
steps:
- checkout
- run:
name: Start the Service Bus emulator
command: docker compose --profile azure-service-bus up -d servicebus servicebus-sql
# Each step is a new shell, which starts on the image's own Node.js, so
# the steps that need ours switch to it with nvm first
- run:
name: Install Node.js and pnpm
command: |
nvm install "$(cat .nvmrc)"
npm install --global pnpm@12.4.1
- run:
name: Install dependencies
command: nvm use "$(cat .nvmrc)" && pnpm i
- run:
name: Build
command: nvm use "$(cat .nvmrc)" && pnpm --filter '@node-ts/bus-azure-service-bus...' run build
- run:
name: Wait for the Service Bus emulator
command: |
for i in $(seq 1 90); do
curl -sf http://localhost:5300/health | grep -q '"healthy"' && break
sleep 2
done
curl -sf http://localhost:5300/health || (docker compose --profile azure-service-bus logs servicebus && exit 1)
- run:
name: Integration Test
command: nvm use "$(cat .nvmrc)" && pnpm exec dotenv -e test.env -- jest --runInBand 'packages/bus-azure-service-bus/.+\.integration\.ts$'

docs:
<<: *defaults
steps:
Expand Down Expand Up @@ -177,12 +218,14 @@ workflows:
build-deploy:
jobs:
- build
- azure-service-bus
- docs:
requires:
- build
- deploy:
requires:
- build
- azure-service-bus
filters:
branches:
only:
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ pnpm exec dotenv -e test.env -- jest packages/bus-sqs/src/sqs-transport.spec.ts
- Jest is configured once at the root (`jest.config.ts`, ts-jest using `tsconfig.test.json`; a `moduleNameMapper` lets relative `.js` imports, as in generated message types, resolve to `.ts` source); `test/setup.ts` drops the default logger's `@node-ts/…` console warnings and errors (set `BUS_TEST_LOGS=true` to see them). Other console output still shows, so don't leave `console.log` in tests.
- Tests: `*.spec.ts` = unit, `*.integration.ts` = integration. `packages/bus-test/src/in-memory.integration.ts` runs bus-test's own round trip suites over the in-memory queue and persistence.
- **Build before testing across packages**: every package has `main: ./dist/index.js`, so e.g. bus-sqs tests import the built bus-core and bus-test, not their source. Changes in bus-core need `pnpm build` before they're visible to other packages.
- Integration tests for adapters need local infra; `docker compose up -d --wait` starts it all from the root `docker-compose.yml`, and waits until it's healthy (Mongo's healthcheck initiates its replica set). Endpoints default to the compose ports and can be overridden with env vars (listed in `test.env`): `LOCALSTACK_ENDPOINT` (default `http://localhost:4566`, dummy AWS creds in `test.env`), `RABBITMQ_URL` (`amqp://guest:guest@0.0.0.0`), `POSTGRES_URL` (`postgres://postgres:password@localhost:6432/postgres`), `MONGODB_URL` (`mongodb://localhost:27017/workflows?directConnection=true`, a single-node replica set, see `packages/bus-mongodb/CLAUDE.md`), `REDIS_URL` (`redis://localhost:6380`; compose also has Valkey on 6381). CI runs all integration tests against CircleCI secondary containers (`.circleci/config.yml`), and bus-redis' again against Valkey.
- Integration tests for adapters need local infra; `docker compose up -d --wait` starts it all from the root `docker-compose.yml`, and waits until it's healthy (Mongo's healthcheck initiates its replica set). Endpoints default to the compose ports and can be overridden with env vars (listed in `test.env`): `LOCALSTACK_ENDPOINT` (default `http://localhost:4566`, dummy AWS creds in `test.env`), `RABBITMQ_URL` (`amqp://guest:guest@0.0.0.0`), `POSTGRES_URL` (`postgres://postgres:password@localhost:6432/postgres`), `MONGODB_URL` (`mongodb://localhost:27017/workflows?directConnection=true`, a single-node replica set, see `packages/bus-mongodb/CLAUDE.md`), `REDIS_URL` (`redis://localhost:6380`; compose also has Valkey on 6381). The Service Bus emulator (with SQL Server, about 2 GB) only starts with its profile, `docker compose --profile azure-service-bus up -d`, on AMQP port 5673 and management port 5300 (`SERVICE_BUS_EMULATOR_HOST`, `SERVICE_BUS_EMULATOR_AMQP_PORT`, `SERVICE_BUS_EMULATOR_MANAGEMENT_PORT`). CI runs all other integration tests against CircleCI secondary containers (`.circleci/config.yml`), bus-redis' again against Valkey, and bus-azure-service-bus' in the `azure-service-bus` job, on a machine executor that runs the emulator with `docker compose`.
- Formatting is automatic: a `.claude/settings.json` hook runs prettier on every file Claude edits, and husky + lint-staged format on commit.
- Linting: `eslint.config.mjs` is `recommended` from ESLint and typescript-eslint, plus the type-aware `no-floating-promises`, `no-misused-promises` and `await-thenable`, with `eslint-config-prettier` so it never fights prettier. Type information comes from `tsconfig.eslint.json` (specs and `test/` fixtures included), and cross-package types come from each package's `dist`, so build first. lint-staged also runs ESLint on staged TS/JS files. Await or `.catch` every promise; use `void` only for deliberate fire-and-forget, with a comment saying why. Unused parameters are allowed when prefixed with `_`.
- Dependency updates are grouped by Renovate (`renovate.json`); the maintainer installs the Renovate GitHub app.
Expand All @@ -46,7 +46,7 @@ pnpm exec dotenv -e test.env -- jest packages/bus-sqs/src/sqs-transport.spec.ts

- `bus-messages` — `Message`/`Command`/`Event` base types and `MessageAttributes`. Messages identify themselves by `$name` (usually a static `NAME`) and `$version`.
- `bus-core` — the bus itself, the `Transport`/`Persistence`/`Serializer`/`Receiver` interfaces (`Transport.endpointName` is the service queue's name), and in-memory defaults (`InMemoryQueue`, `InMemoryPersistence`, `JsonSerializer`).
- Transports: `bus-sqs` (SNS topics fanned out to SQS queues, with policy generation), `bus-rabbitmq`, `PostgresTransport` in `bus-postgres` (queues in shared Postgres tables, no broker; see `packages/bus-postgres/CLAUDE.md`), and `bus-redis` (a Redis Stream per queue, read by a consumer group, on Redis 7+/Valkey 7.2+; see `packages/bus-redis/CLAUDE.md`).
- Transports: `bus-sqs` (SNS topics fanned out to SQS queues, with policy generation), `bus-rabbitmq`, `bus-azure-service-bus` (a topic per message forwarded into each service's queue; see `packages/bus-azure-service-bus/CLAUDE.md`), `PostgresTransport` in `bus-postgres` (queues in shared Postgres tables, no broker; see `packages/bus-postgres/CLAUDE.md`), and `bus-redis` (a Redis Stream per queue, read by a consumer group, on Redis 7+/Valkey 7.2+; see `packages/bus-redis/CLAUDE.md`).
- Persistence (workflow state and delayed messages): `bus-postgres`, `bus-mongodb`.
- `bus-sqs-lambda` — a `Receiver` that feeds Lambda SQS events into the bus instead of the bus polling the transport.
- `bus-opentelemetry` — `openTelemetry()` middleware for spans, W3C trace context (carried in message attributes) and messaging metrics, with only `@opentelemetry/api` as a peer. See `packages/bus-opentelemetry/CLAUDE.md`.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Requires Node.js 24 or later.
- [@node-ts/bus-messages](https://github.com/node-ts/bus/tree/master/packages/bus-messages) - The base types of commands, events and their attributes, used to declare your own messages
- [@node-ts/bus-cli](https://github.com/node-ts/bus/tree/master/packages/bus-cli) - Command line tools, including `bus generate-message-types`, which lets the bus restore Dates and class instances in messages
- [@node-ts/bus-rabbitmq](https://github.com/node-ts/bus/tree/master/packages/bus-rabbitmq) - A RabbitMQ transport
- [@node-ts/bus-azure-service-bus](https://github.com/node-ts/bus/tree/master/packages/bus-azure-service-bus) - An Azure Service Bus transport
- [@node-ts/bus-sqs](https://github.com/node-ts/bus/tree/master/packages/bus-sqs) - An Amazon SQS transport
- [@node-ts/bus-sqs-lambda](https://github.com/node-ts/bus/tree/master/packages/bus-sqs-lambda) - A receiver that handles SQS messages in AWS Lambda
- [@node-ts/bus-postgres](https://github.com/node-ts/bus/tree/master/packages/bus-postgres) - A Postgres persistence for workflow state, and a Postgres transport that needs no broker
Expand Down
29 changes: 29 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,32 @@ services:
interval: 2s
timeout: 10s
retries: 30

# Microsoft's Service Bus emulator, for bus-azure-service-bus. It needs SQL
# Server and about 2 GB of RAM, so it only starts with its profile:
# docker compose --profile azure-service-bus up -d
# Starting it accepts the licences of both images (ACCEPT_EULA):
# https://github.com/Azure/azure-service-bus-emulator-installer/blob/main/EMULATOR_EULA.txt
# and https://go.microsoft.com/fwlink/?LinkId=746388. AMQP is on 5673, since
# RabbitMQ has 5672, and the management API and /health are on 5300.
servicebus:
profiles: ['azure-service-bus']
image: mcr.microsoft.com/azure-messaging/servicebus-emulator:2.0.1
volumes:
- ./packages/bus-azure-service-bus/emulator/config.json:/ServiceBus_Emulator/ConfigFiles/Config.json
environment:
SQL_SERVER: servicebus-sql
MSSQL_SA_PASSWORD: 'Emulator_Passw0rd'
ACCEPT_EULA: 'Y'
ports:
- '5673:5672'
- '5300:5300'
depends_on:
- servicebus-sql

servicebus-sql:
profiles: ['azure-service-bus']
image: mcr.microsoft.com/mssql/server:2022-CU27-ubuntu-22.04
environment:
MSSQL_SA_PASSWORD: 'Emulator_Passw0rd'
ACCEPT_EULA: 'Y'
1 change: 1 addition & 0 deletions docs/.vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ const infrastructure: DefaultTheme.SidebarItem[] = [
items: [
{ text: 'RabbitMQ', link: '/transports/rabbitmq' },
{ text: 'Amazon SQS', link: '/transports/amazon-sqs' },
{ text: 'Azure Service Bus', link: '/transports/azure-service-bus' },
{ text: 'SQS and Lambda', link: '/transports/sqs-lambda' },
{ text: 'Postgres', link: '/transports/postgres' },
{ text: 'Redis', link: '/transports/redis' },
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,4 @@ The generator needs TypeScript types. In a JavaScript project, write the message

- [Handling messages](/getting-started/handling-messages), to receive your first message
- [Serializers](/guide/serializers), for what the message types restore
- [Transports](/transports), to run on RabbitMQ or Amazon SQS
- [Transports](/transports), to run on RabbitMQ, Amazon SQS or Azure Service Bus
11 changes: 6 additions & 5 deletions docs/guide/middleware.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,12 @@ Outgoing middleware can also set native headers for the transport in `context.he

<<< @/snippets/middleware.ts#headers

| Transport | Writes each header as | Reserved names |
| ------------------------------------ | --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| [RabbitMQ](/transports/rabbitmq) | an AMQP header, kept when the message retries | `attributes`, `stickyAttributes`, `sentAt`, `failedAttempts`, `bus-failure`, `x-death`, and names starting `x-first-death-` or `x-last-death-` |
| [Amazon SQS](/transports/amazon-sqs) | an SNS message attribute under its own name | `correlationId`, `messageId`, `sentAt`, `replyTo`, `bus-failure`, and names starting `attributes.` or `stickyAttributes.` |
| In-memory queue | an entry in the raw message's `headers` | `bus-failure` |
| Transport | Writes each header as | Reserved names |
| -------------------------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [RabbitMQ](/transports/rabbitmq) | an AMQP header, kept when the message retries | `attributes`, `stickyAttributes`, `sentAt`, `failedAttempts`, `bus-failure`, `x-death`, and names starting `x-first-death-` or `x-last-death-` |
| [Amazon SQS](/transports/amazon-sqs) | an SNS message attribute under its own name | `correlationId`, `messageId`, `sentAt`, `replyTo`, `bus-failure`, and names starting `attributes.` or `stickyAttributes.` |
| [Azure Service Bus](/transports/azure-service-bus) | an application property under its own name, kept when the message retries | `messageId`, `sentAt`, `failedAttempts`, `bus-failure`, `DeadLetterReason`, `DeadLetterErrorDescription`, and names starting `attributes.` or `stickyAttributes.` |
| In-memory queue | an entry in the raw message's `headers` | `bus-failure` |

Setting a reserved name makes the `send()`, `publish()` or `reply()` throw `TransportHeaderReserved`, before the message is buffered or sent. AWS's limit of 10 message attributes on SQS only applies with SNS raw message delivery, which the SQS transport doesn't use.

Expand Down
Loading
Loading