Repository navigation
Closed
Labels
enhancementNew feature or requestNew feature or requestroadmapPart of the project roadmapPart of the project roadmap
Milestone
Description
Context
Core gives outgoing messages no identity and no timestamp. TransportMessage.id is whatever the transport uses natively, and it isn't stable:
- SQS: the
MessageId, which changes whenfailcopies the message to the DLQ - RabbitMQ: a new
randomUUID()per publish, which the retry copy keeps - in-memory:
undefined
Core also doesn't know which endpoint it runs as. The transports each have a queue name in their config, but the Transport interface doesn't expose it.
Several Phase 3 issues need these:
- Inbox / idempotent consumer (message deduplication) #261 deduplicates by
(endpoint, messageId) - Configurable recoverability policy and error queue with failure metadata #262 puts the endpoint and the attempt history in failure metadata
- OpenTelemetry tracing and metrics #264 sets
messaging.message.idand measures critical time fromsentAt - Delayed / deferred message delivery #258 stores scheduled messages by id
This issue adds them once, so those PRs don't each add them in parallel. It doesn't depend on #263 and can run alongside it.
Decisions
Answered by the maintainer:
- Core stamps
messageIdandsentAton every outgoing message, every transport carries them, and they survive retries and the DLQ. Transportgets anendpointName.- This runs in parallel with Incoming and outgoing pipeline behaviours (middleware) #263. It blocks Delayed / deferred message delivery #258, Inbox / idempotent consumer (message deduplication) #261, Configurable recoverability policy and error queue with failure metadata #262 and OpenTelemetry tracing and metrics #264.
Decided from the code:
messageIdandsentAtlive inMessageAttributes, as optionalmessageId?: stringandsentAt?: string(ISO 8601), next tocorrelationId.send/publishalready takePartial<MessageAttributes>, so a caller can pass its ownmessageIdfor an idempotent send (e.g. derived from an HTTP idempotency key) with no new parameter.- Handlers already receive the attributes, so they can read the id without a change to
HandlerContext. - Every transport already serializes
MessageAttributes, and Durable transactional outbox #260's outbox rows store them as they are. - They're optional because messages from outside the bus (custom handlers, system messages) may not have them.
TransportMessage.idstays the native id: bus-sqs-lambda reports batch item failures by the SQSmessageId.
- Unlike
correlationId, neither is inherited.prepareTransportOptionsgenerates a newmessageId(randomUUID()) andsentAtfor each outgoing message unless the caller passes them. It never copies them from the message being handled. Transport.endpointName: stringis a required readonly property (a breaking change for custom transports). It returns the queue name:- bus-sqs:
queueName, or the name fromqueueArn - bus-rabbitmq:
queueName InMemoryQueue: a newendpointNameoption, defaulting to'in-memory'
- bus-sqs:
Non-goals
- Deduplication, failure metadata, tracing and scheduling. Those are Inbox / idempotent consumer (message deduplication) #261, Configurable recoverability policy and error queue with failure metadata #262, OpenTelemetry tracing and metrics #264 and Delayed / deferred message delivery #258.
- A
messageIdonHandlerContext. The attributes already have it. - Changing
TransportMessage.id.
Approach
- bus-messages: add the two optional fields to
MessageAttributes, with JSDoc. Accept them inmessageAttributes(). - bus-core: stamp them in
BusInstance.prepareTransportOptions. AddendpointNametoTransportandInMemoryQueue. Carry them throughtoTransportMessage. - bus-sqs: add top-level SNS attributes
messageIdandsentAtintoMessageAttributeMap, and read them infromMessageAttributeMap. bus-sqs-lambda gets this for free.failand the redrive policy copy the envelope, so both survive. - bus-rabbitmq:
- Set the AMQP
messageIdproperty from the attribute, in place ofrandomUUID(), and read it back from there. - Carry
sentAtin a header, because the AMQPtimestampproperty only has second precision. - The retry and DLQ copies already keep the properties and headers.
- Set the AMQP
- bus-test: extend
transportTests. A sent message arrives with amessageIdandsentAt. AmessageIdthe caller passes is kept. Both are the same afterreturnMessageand in the dead letter queue (readAllFromDeadLetterQueue). - Add a MIGRATING entry for
Transport.endpointNameand a changeset.
Acceptance criteria
- Every outgoing message has a unique
messageIdand asentAt, unless the caller passes them. A message sent from a handler doesn't inherit the handled message's values. - A
messageIdthe caller passes reaches the handler unchanged. -
messageIdandsentAtsurvive retry and dead-lettering on in-memory, SQS (including through bus-sqs-lambda) and RabbitMQ, checked bytransportTests. -
Transport.endpointNameis implemented by every transport and documented in/transports/custom. - Docs:
/guide/message-attributesdescribes both fields. MIGRATING entry and changeset added.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestroadmapPart of the project roadmapPart of the project roadmap
Activity