Problem
RedisStreamClusterBroker.listen() only ever calls XREADGROUP ... >. Unlike RedisStreamBroker.listen(), it never reads the consumer group's pending-entries list, so there is no XAUTOCLAIM pass.
If a worker dies after a message is delivered but before it is acked, that message stays in the PEL indefinitely: it is never redelivered and never retried, so the task silently never completes. SmartRetryMiddleware doesn't help here, since nothing runs in a process that has already died.
The cluster variant is also missing the corresponding knobs: idle_timeout, unacknowledged_batch_size, unacknowledged_lock_timeout, and xread_count (so XREADGROUP runs uncapped).
Version: taskiq-redis 1.2.2, redis 7.4.1.
Impact
AWS ElastiCache Serverless runs cluster-mode-enabled only, so users on it are forced onto RedisStreamClusterBroker and lose crash recovery as a side effect of their infrastructure choice, with no signal that it happened. The same applies to other managed cluster-mode-only Redis/Valkey offerings.
There's no cluster-specific blocker
XAUTOCLAIM is a single-key command, and the reclaim lock key (autoclaim:{group}:{stream}) is a single key, so both hash to one slot and
work unchanged under cluster mode. RedisCluster exposes equivalent .lock() and .xautoclaim() methods.
Suggestion
Bring RedisStreamClusterBroker.listen() to parity with RedisStreamBroker.listen(), including the accompanying constructor
parameters. Ideally by factoring the reclaim logic into a shared mixin or helper used by both brokers, so the two don't drift again as the non-cluster path evolves.
Problem
RedisStreamClusterBroker.listen()only ever callsXREADGROUP ... >. UnlikeRedisStreamBroker.listen(), it never reads the consumer group's pending-entries list, so there is noXAUTOCLAIMpass.If a worker dies after a message is delivered but before it is acked, that message stays in the PEL indefinitely: it is never redelivered and never retried, so the task silently never completes.
SmartRetryMiddlewaredoesn't help here, since nothing runs in a process that has already died.The cluster variant is also missing the corresponding knobs:
idle_timeout,unacknowledged_batch_size,unacknowledged_lock_timeout, andxread_count(soXREADGROUPruns uncapped).Version:
taskiq-redis1.2.2,redis7.4.1.Impact
AWS ElastiCache Serverless runs cluster-mode-enabled only, so users on it are forced onto
RedisStreamClusterBrokerand lose crash recovery as a side effect of their infrastructure choice, with no signal that it happened. The same applies to other managed cluster-mode-only Redis/Valkey offerings.There's no cluster-specific blocker
XAUTOCLAIMis a single-key command, and the reclaim lock key (autoclaim:{group}:{stream}) is a single key, so both hash to one slot andwork unchanged under cluster mode.
RedisClusterexposes equivalent.lock()and.xautoclaim()methods.Suggestion
Bring
RedisStreamClusterBroker.listen()to parity withRedisStreamBroker.listen(), including the accompanying constructorparameters. Ideally by factoring the reclaim logic into a shared mixin or helper used by both brokers, so the two don't drift again as the non-cluster path evolves.