Skip to content

Core message id, sent time and endpoint name #322

Closed

Description

@adenhertog

Context

Core gives outgoing messages no identity and no timestamp. TransportMessage.id is whatever the transport uses natively, and it isn't stable:

  • SQS: the MessageId, which changes when fail copies the message to the DLQ
  • RabbitMQ: a new randomUUID() per publish, which the retry copy keeps
  • in-memory: undefined

Core also doesn't know which endpoint it runs as. The transports each have a queue name in their config, but the Transport interface doesn't expose it.

Several Phase 3 issues need these:

This issue adds them once, so those PRs don't each add them in parallel. It doesn't depend on #263 and can run alongside it.

Decisions

Answered by the maintainer:

  1. Core stamps messageId and sentAt on every outgoing message, every transport carries them, and they survive retries and the DLQ.
  2. Transport gets an endpointName.
  3. This runs in parallel with Incoming and outgoing pipeline behaviours (middleware) #263. It blocks Delayed / deferred message delivery #258, Inbox / idempotent consumer (message deduplication) #261, Configurable recoverability policy and error queue with failure metadata #262 and OpenTelemetry tracing and metrics #264.

Decided from the code:

  1. messageId and sentAt live in MessageAttributes, as optional messageId?: string and sentAt?: string (ISO 8601), next to correlationId.
    • send/publish already take Partial<MessageAttributes>, so a caller can pass its own messageId for an idempotent send (e.g. derived from an HTTP idempotency key) with no new parameter.
    • Handlers already receive the attributes, so they can read the id without a change to HandlerContext.
    • Every transport already serializes MessageAttributes, and Durable transactional outbox #260's outbox rows store them as they are.
    • They're optional because messages from outside the bus (custom handlers, system messages) may not have them.
    • TransportMessage.id stays the native id: bus-sqs-lambda reports batch item failures by the SQS messageId.
  2. Unlike correlationId, neither is inherited. prepareTransportOptions generates a new messageId (randomUUID()) and sentAt for each outgoing message unless the caller passes them. It never copies them from the message being handled.
  3. Transport.endpointName: string is a required readonly property (a breaking change for custom transports). It returns the queue name:
    • bus-sqs: queueName, or the name from queueArn
    • bus-rabbitmq: queueName
    • InMemoryQueue: a new endpointName option, defaulting to 'in-memory'

Non-goals

Approach

  • bus-messages: add the two optional fields to MessageAttributes, with JSDoc. Accept them in messageAttributes().
  • bus-core: stamp them in BusInstance.prepareTransportOptions. Add endpointName to Transport and InMemoryQueue. Carry them through toTransportMessage.
  • bus-sqs: add top-level SNS attributes messageId and sentAt in toMessageAttributeMap, and read them in fromMessageAttributeMap. bus-sqs-lambda gets this for free. fail and the redrive policy copy the envelope, so both survive.
  • bus-rabbitmq:
    • Set the AMQP messageId property from the attribute, in place of randomUUID(), and read it back from there.
    • Carry sentAt in a header, because the AMQP timestamp property only has second precision.
    • The retry and DLQ copies already keep the properties and headers.
  • bus-test: extend transportTests. A sent message arrives with a messageId and sentAt. A messageId the caller passes is kept. Both are the same after returnMessage and in the dead letter queue (readAllFromDeadLetterQueue).
  • Add a MIGRATING entry for Transport.endpointName and a changeset.

Acceptance criteria

  • Every outgoing message has a unique messageId and a sentAt, unless the caller passes them. A message sent from a handler doesn't inherit the handled message's values.
  • A messageId the caller passes reaches the handler unchanged.
  • messageId and sentAt survive retry and dead-lettering on in-memory, SQS (including through bus-sqs-lambda) and RabbitMQ, checked by transportTests.
  • Transport.endpointName is implemented by every transport and documented in /transports/custom.
  • Docs: /guide/message-attributes describes both fields. MIGRATING entry and changeset added.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestroadmapPart of the project roadmap

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions