From 28a401414ecbe8872bcd6e7592f692d759cd8555 Mon Sep 17 00:00:00 2001 From: "Qiaoyu (Joey) Deng" Date: Thu, 5 Nov 2026 09:45:31 -0800 Subject: [PATCH] chore(release): update changelog Build the 0.4.0 section of CHANGELOG.md from the changelog.d fragments with towncrier, delete the consumed fragments, and add the 0.4.0 compare link. The 154 fragment no longer says ExportBackend._TORCH is deprecated, because the 168 fragment removes it in the same release. --- CHANGELOG.md | 30 +++++++++++++++++++ .../+auto-detect-palettized-weights.added | 1 - changelog.d/+preserve-lossy-float-casts.fixed | 1 - ...ip-single-element-activation-quant.changed | 1 - changelog.d/+vector-kmeans.added | 1 - changelog.d/+vector-kmeans.changed | 1 - changelog.d/133.added | 1 - changelog.d/135.added | 1 - .../139.joint-sparsity-palettization.added | 1 - changelog.d/154.public-torch-backend.added | 1 - changelog.d/157.prepare-state-dict.added | 1 - changelog.d/157.prepare-state-dict.changed | 1 - changelog.d/157.prepare-state-dict.fixed | 1 - changelog.d/157.prepare-state-dict.removed | 1 - .../167.int6-uint6-torch-backend.added | 1 - .../168.remove-private-torch-backend.removed | 1 - 16 files changed, 30 insertions(+), 15 deletions(-) delete mode 100644 changelog.d/+auto-detect-palettized-weights.added delete mode 100644 changelog.d/+preserve-lossy-float-casts.fixed delete mode 100644 changelog.d/+skip-single-element-activation-quant.changed delete mode 100644 changelog.d/+vector-kmeans.added delete mode 100644 changelog.d/+vector-kmeans.changed delete mode 100644 changelog.d/133.added delete mode 100644 changelog.d/135.added delete mode 100644 changelog.d/139.joint-sparsity-palettization.added delete mode 100644 changelog.d/154.public-torch-backend.added delete mode 100644 changelog.d/157.prepare-state-dict.added delete mode 100644 changelog.d/157.prepare-state-dict.changed delete mode 100644 changelog.d/157.prepare-state-dict.fixed delete mode 100644 changelog.d/157.prepare-state-dict.removed delete mode 100644 changelog.d/167.int6-uint6-torch-backend.added delete mode 100644 changelog.d/168.remove-private-torch-backend.removed diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b7cb8b3..3dec3adf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 +## [0.4.0] - 2026-11-05 + +### Added + +- Added support for Python 3.14. +- Added the public `ExportBackend.Torch` backend for fast torch inference, e.g. running evals. `finalize(backend=ExportBackend.Torch)` folds weight compression (palettization, weight quantization, pruning) into dense weights and keeps activation fake quantization. +- Added a `state_dict` argument to `prepare()` for `Quantizer` (eager and graph mode) and `MagnitudePruner`, matching `KMeansPalettizer`. It reloads a previously prepared model's compression state (calibrated qparams, pruning masks and step count, centroids), validates it against the config, and leaves the model in the same state as a regular `prepare()` call. +- Added MXINT8 quantization (`int8` with `float8_e8m0fnu` scales) in `QuantizationSpec`, supported in eager and graph mode. It is only supported for `ExportBackend.Torch` for now. +- Added the `torch.int6` and `torch.uint6` quantization dtypes. They are only supported for `ExportBackend.Torch` for now: `finalize` raises `NotImplementedError` for the Core AI and Core ML backends. +- Added `VectorKMeansConfig` (`vector_kmeans_config` on `ModuleKMeansPalettizerConfig`) to set the k-means++ candidates per seeding step (`n_local_trials`), restarts, iteration cap and convergence tolerance of vector palettization (`cluster_dim > 1`). Vector k-means now clusters on a CUDA GPU, with flash-kmeans when it is installed and a batched PyTorch loop otherwise or for sensitivity-weighted clustering, and on CPU with scikit-learn, falling back to the PyTorch loop with a warning when scikit-learn is not installed. +- K-means palettization detects weights that are already palettized for their spec (at most `2 ** n_bits` distinct values or vectors per block, e.g. a fake-palettized checkpoint) and uses those unique values as the centroids instead of running k-means. Other weights fall back to k-means after one small probe. +- Added the `sparsity_detection_threshold` option to `PalettizationSpec`. A weight with at least that fraction of exact zeros is treated as already sparse and palettized jointly with its sparsity: only its nonzero values are palettized, the zeros stay exactly zero, and the exported model stores the weight in a sparse representation together with the palettized ops for the nonzero values. Supports per-tensor and per-group-channel granularity, per-channel scales and, on Core AI, quantized LUTs and vector palettization. + +### Changed + +- Graph-mode `Quantizer` now skips activations and states with at most one element (e.g. the 0-d output of `x.sum()`), matching eager mode. This also fixes `prepare` failing on 0-d parameters under a per-channel weight spec. +- Vector palettization (`cluster_dim > 1`) now seeds with greedy k-means++ instead of farthest-point initialization, stops each block once its clustering error stops improving instead of running a fixed number of iterations, and is reproducible without manual seeding, including with `num_workers > 1`. It is much faster on GPU. With `num_workers > 1`, tensors that cluster on the GPU run in the main process while the worker processes handle the others. +- Pruning modules now store their step count in a `step_count` buffer, so a `MagnitudePruner` state dict saved before this change cannot be reloaded with `prepare(state_dict=...)`. + +### Removed + +- Removed the deprecated `ExportBackend._TORCH` alias. Use `ExportBackend.Torch` instead. +- Removed the `prev_sparsity` argument from `SparsityScheduleBase.compute_sparsity` and its subclasses. Schedules are now pure functions of the step count; `PolynomialDecaySchedule` returns the last update's sparsity for steps between updates instead of passing through `prev_sparsity` or raising `ValueError` when it is omitted. + +### Fixed + +- Fixed `KMeansPalettizer.prepare(state_dict=...)` failing for configs with a quantized LUT, and `MagnitudePruner` restarting its sparsity schedule from step 0 after a model saved mid-schedule was reloaded. +- Preserve original floating-point casts that lose precision or range when cleaning up cast chains. + ## [0.3.0] - 2026-09-25 ### Added @@ -59,3 +88,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [0.2.0]: https://github.com/apple/coreai-optimization/commits/v0.2.0/ [0.2.1]: https://github.com/apple/coreai-optimization/compare/v0.2.0...v0.2.1/ [0.3.0]: https://github.com/apple/coreai-optimization/compare/v0.2.1...v0.3.0/ +[0.4.0]: https://github.com/apple/coreai-optimization/compare/v0.3.0...v0.4.0/ diff --git a/changelog.d/+auto-detect-palettized-weights.added b/changelog.d/+auto-detect-palettized-weights.added deleted file mode 100644 index ab543ed0..00000000 --- a/changelog.d/+auto-detect-palettized-weights.added +++ /dev/null @@ -1 +0,0 @@ -K-means palettization detects weights that are already palettized for their spec (at most `2 ** n_bits` distinct values or vectors per block, e.g. a fake-palettized checkpoint) and uses those unique values as the centroids instead of running k-means. Other weights fall back to k-means after one small probe. diff --git a/changelog.d/+preserve-lossy-float-casts.fixed b/changelog.d/+preserve-lossy-float-casts.fixed deleted file mode 100644 index 6c2172a4..00000000 --- a/changelog.d/+preserve-lossy-float-casts.fixed +++ /dev/null @@ -1 +0,0 @@ -Preserve original floating-point casts that lose precision or range when cleaning up cast chains. diff --git a/changelog.d/+skip-single-element-activation-quant.changed b/changelog.d/+skip-single-element-activation-quant.changed deleted file mode 100644 index 312121ba..00000000 --- a/changelog.d/+skip-single-element-activation-quant.changed +++ /dev/null @@ -1 +0,0 @@ -Graph-mode `Quantizer` now skips activations and states with at most one element (e.g. the 0-d output of `x.sum()`), matching eager mode. This also fixes `prepare` failing on 0-d parameters under a per-channel weight spec. diff --git a/changelog.d/+vector-kmeans.added b/changelog.d/+vector-kmeans.added deleted file mode 100644 index 061d0965..00000000 --- a/changelog.d/+vector-kmeans.added +++ /dev/null @@ -1 +0,0 @@ -Add `VectorKMeansConfig` (`vector_kmeans_config` on `ModuleKMeansPalettizerConfig`) to set the k-means++ candidates per seeding step (`n_local_trials`), restarts, iteration cap and convergence tolerance of vector palettization (`cluster_dim > 1`). Vector k-means now clusters on a CUDA GPU, with flash-kmeans when it is installed and a batched PyTorch loop otherwise or for sensitivity-weighted clustering, and on CPU with scikit-learn, falling back to the PyTorch loop with a warning when scikit-learn is not installed. diff --git a/changelog.d/+vector-kmeans.changed b/changelog.d/+vector-kmeans.changed deleted file mode 100644 index aae90391..00000000 --- a/changelog.d/+vector-kmeans.changed +++ /dev/null @@ -1 +0,0 @@ -Vector palettization (`cluster_dim > 1`) now seeds with greedy k-means++ instead of farthest-point initialization, stops each block once its clustering error stops improving instead of running a fixed number of iterations, and is reproducible without manual seeding, including with `num_workers > 1`. It is much faster on GPU. With `num_workers > 1`, tensors that cluster on the GPU run in the main process while the worker processes handle the others. diff --git a/changelog.d/133.added b/changelog.d/133.added deleted file mode 100644 index ac9be5a2..00000000 --- a/changelog.d/133.added +++ /dev/null @@ -1 +0,0 @@ -Add MXINT8 support. diff --git a/changelog.d/135.added b/changelog.d/135.added deleted file mode 100644 index 6c075498..00000000 --- a/changelog.d/135.added +++ /dev/null @@ -1 +0,0 @@ -Added support for Python 3.14. diff --git a/changelog.d/139.joint-sparsity-palettization.added b/changelog.d/139.joint-sparsity-palettization.added deleted file mode 100644 index 4c082374..00000000 --- a/changelog.d/139.joint-sparsity-palettization.added +++ /dev/null @@ -1 +0,0 @@ -Added the `sparsity_detection_threshold` option to `PalettizationSpec`. A weight with at least that fraction of exact zeros is treated as already sparse and palettized jointly with its sparsity: only its nonzero values are palettized, the zeros stay exactly zero, and the exported model stores the weight in a sparse representation together with the palettized ops for the nonzero values. Supports per-tensor and per-group-channel granularity, per-channel scales and, on Core AI, quantized LUTs and vector palettization. diff --git a/changelog.d/154.public-torch-backend.added b/changelog.d/154.public-torch-backend.added deleted file mode 100644 index d842a834..00000000 --- a/changelog.d/154.public-torch-backend.added +++ /dev/null @@ -1 +0,0 @@ -Added the public `ExportBackend.Torch` backend for fast torch inference, e.g. running evals. `finalize(backend=ExportBackend.Torch)` folds weight compression (palettization, weight quantization, pruning) into dense weights and keeps activation fake quantization. The private `ExportBackend._TORCH` name is deprecated in favor of `ExportBackend.Torch`. diff --git a/changelog.d/157.prepare-state-dict.added b/changelog.d/157.prepare-state-dict.added deleted file mode 100644 index d399b8d3..00000000 --- a/changelog.d/157.prepare-state-dict.added +++ /dev/null @@ -1 +0,0 @@ -Added a `state_dict` argument to `prepare()` for `Quantizer` (eager and graph mode) and `MagnitudePruner`, matching `KMeansPalettizer`. It reloads a previously prepared model's compression state (calibrated qparams, pruning masks and step count, centroids), validates it against the config, and leaves the model in the same state as a regular `prepare()` call. diff --git a/changelog.d/157.prepare-state-dict.changed b/changelog.d/157.prepare-state-dict.changed deleted file mode 100644 index 4934fae8..00000000 --- a/changelog.d/157.prepare-state-dict.changed +++ /dev/null @@ -1 +0,0 @@ -Pruning modules now store their step count in a `step_count` buffer, so a `MagnitudePruner` state dict saved before this change cannot be reloaded with `prepare(state_dict=...)`. diff --git a/changelog.d/157.prepare-state-dict.fixed b/changelog.d/157.prepare-state-dict.fixed deleted file mode 100644 index 0dc5173b..00000000 --- a/changelog.d/157.prepare-state-dict.fixed +++ /dev/null @@ -1 +0,0 @@ -Fixed `KMeansPalettizer.prepare(state_dict=...)` failing for configs with a quantized LUT, and `MagnitudePruner` restarting its sparsity schedule from step 0 after a model saved mid-schedule was reloaded. diff --git a/changelog.d/157.prepare-state-dict.removed b/changelog.d/157.prepare-state-dict.removed deleted file mode 100644 index fee8c851..00000000 --- a/changelog.d/157.prepare-state-dict.removed +++ /dev/null @@ -1 +0,0 @@ -Removed the `prev_sparsity` argument from `SparsityScheduleBase.compute_sparsity` and its subclasses. Schedules are now pure functions of the step count; `PolynomialDecaySchedule` returns the last update's sparsity for steps between updates instead of passing through `prev_sparsity` or raising `ValueError` when it is omitted. diff --git a/changelog.d/167.int6-uint6-torch-backend.added b/changelog.d/167.int6-uint6-torch-backend.added deleted file mode 100644 index 52ca8b15..00000000 --- a/changelog.d/167.int6-uint6-torch-backend.added +++ /dev/null @@ -1 +0,0 @@ -Added the `torch.int6` and `torch.uint6` quantization dtypes. They are only supported for `ExportBackend.Torch` for now: `finalize` raises `NotImplementedError` for the Core AI and Core ML backends. diff --git a/changelog.d/168.remove-private-torch-backend.removed b/changelog.d/168.remove-private-torch-backend.removed deleted file mode 100644 index 7af691d6..00000000 --- a/changelog.d/168.remove-private-torch-backend.removed +++ /dev/null @@ -1 +0,0 @@ -Removed the deprecated `ExportBackend._TORCH` alias. Use `ExportBackend.Torch` instead.