Skip to content

feat(models): add GLM-5 Next quantization support - #3045

Open
ZX-ModelCloud wants to merge 3 commits into
mainfrom
zx/glm5-next-support
Open

feat(models): add GLM-5 Next quantization support#3045
ZX-ModelCloud wants to merge 3 commits into
mainfrom
zx/glm5-next-support

Conversation

@ZX-ModelCloud

Copy link
Copy Markdown
Collaborator

Summary

  • make dequantized safetensors output resumable and enforce a compatible contiguous layout
  • register glm5_next with hybrid KDA/DSA, dense and routed-expert quantization coverage
  • preserve the auxiliary MTP checkpoint tensors while quantizing all loaded decoder layers
  • preserve DSA replay indices and GLM expert clamp behavior during calibration
  • add model-quality expectations and update the supported-model documentation
  • require defuser>=0.0.27 from feat: support GLM-5 Next expert defusion Defuser#66

Evaluation

BF16 ARC Challenge baseline (1,172 samples):

  • acc: 0.49146757679180886
  • acc_norm: 0.5273037542662116

Tests

  • GLM-5 Next support tests: 7 passed
  • dequant tests: 10 passed; the optional bitsandbytes case was not run because that package is unavailable in the test environment
  • GLM-5 Next model eval test collection: 1 test collected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant