feat(glm5_next): serve GLM-5.3-Flash with tensor parallelism

[calvarado2004]

What

Serve GLM-5.3-Flash (glm5_next) with tensor parallelism. Today the model loads at TP=1 only.

• MLA/DSA: attention heads are split per rank (q_b/kv_b column-parallel, o_proj row-parallel). The latent projections (q_a, kv_a) and the DSA indexer stay replicated, so every rank selects the same blocks and reads the same latent KV.
• KDA: each rank owns whole heads: q|k|v|b, the f_b/g_b up-projections, the conv channels, A_log and dt_bias. The rank-128 f_a/g_a bottleneck moves to its own replicated projection (in_proj_fg). The linear state pool is sized per rank.
• Dense and shared-expert MLPs: column/row-parallel.
• NVFP4 routed experts: each rank keeps its block of the intermediate dimension (gate/up rows, down columns, and their block scales). The Triton NVFP4 kernel accepts TP.
• Weight reader: cuts every tensor per rank, including each KDA part before the parts are fused.

Tests

• tests/models/test_glm5_next_tp.py: the per-rank shape contract, that the reader's shards tile back to the full tensors, and NVFP4 shard packing.
• tests/models/test_glm5_next_tp_equivalence.py (multi-GPU, slow): TP=2 and TP=4 prefill and decode logits match TP=1 on a tiny checkpoint, with real NCCL collectives.

Run on the branch head (4x RTX A4000; production held GPUs 0-1, so two GPUs were visible):

CUDA_VISIBLE_DEVICES=2,3 pytest tests/models -k glm5_next
30 passed, 434 deselected

With two GPUs the equivalence test runs its TP=2 case and skips TP=4.

Hardware

4x RTX A4000 (16 GB, Ampere), 2x Xeon, 502 GB RAM, RedHatAI/GLM-5.3-Flash-NVFP4, TP=4 with hybrid CPU/GPU experts:

ft serve --model RedHatAI/GLM-5.3-Flash-NVFP4 --tensor-parallel-size 4 --memory-ratio 0.80 \
  --max-running-requests 1 --max-prefill-length 2048 --expert-load serial --kv-reserve-tokens 131072

Plain decode on this box runs at 14-15 tok/s (single stream, temperature 1.0).