diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b7cb8b3..3dec3adf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 +## [0.4.0] - 2026-11-05 + +### Added + +- Added support for Python 3.14. +- Added the public `ExportBackend.Torch` backend for fast torch inference, e.g. running evals. `finalize(backend=ExportBackend.Torch)` folds weight compression (palettization, weight quantization, pruning) into dense weights and keeps activation fake quantization. +- Added a `state_dict` argument to `prepare()` for `Quantizer` (eager and graph mode) and `MagnitudePruner`, matching `KMeansPalettizer`. It reloads a previously prepared model's compression state (calibrated qparams, pruning masks and step count, centroids), validates it against the config, and leaves the model in the same state as a regular `prepare()` call. +- Added MXINT8 quantization (`int8` with `float8_e8m0fnu` scales) in `QuantizationSpec`, supported in eager and graph mode. It is only supported for `ExportBackend.Torch` for now. +- Added the `torch.int6` and `torch.uint6` quantization dtypes. They are only supported for `ExportBackend.Torch` for now: `finalize` raises `NotImplementedError` for the Core AI and Core ML backends. +- Added `VectorKMeansConfig` (`vector_kmeans_config` on `ModuleKMeansPalettizerConfig`) to set the k-means++ candidates per seeding step (`n_local_trials`), restarts, iteration cap and convergence tolerance of vector palettization (`cluster_dim > 1`). Vector k-means now clusters on a CUDA GPU, with flash-kmeans when it is installed and a batched PyTorch loop otherwise or for sensitivity-weighted clustering, and on CPU with scikit-learn, falling back to the PyTorch loop with a warning when scikit-learn is not installed. +- K-means palettization detects weights that are already palettized for their spec (at most `2 ** n_bits` distinct values or vectors per block, e.g. a fake-palettized checkpoint) and uses those unique values as the centroids instead of running k-means. Other weights fall back to k-means after one small probe. +- Added the `sparsity_detection_threshold` option to `PalettizationSpec`. A weight with at least that fraction of exact zeros is treated as already sparse and palettized jointly with its sparsity: only its nonzero values are palettized, the zeros stay exactly zero, and the exported model stores the weight in a sparse representation together with the palettized ops for the nonzero values. Supports per-tensor and per-group-channel granularity, per-channel scales and, on Core AI, quantized LUTs and vector palettization. + +### Changed + +- Graph-mode `Quantizer` now skips activations and states with at most one element (e.g. the 0-d output of `x.sum()`), matching eager mode. This also fixes `prepare` failing on 0-d parameters under a per-channel weight spec. +- Vector palettization (`cluster_dim > 1`) now seeds with greedy k-means++ instead of farthest-point initialization, stops each block once its clustering error stops improving instead of running a fixed number of iterations, and is reproducible without manual seeding, including with `num_workers > 1`. It is much faster on GPU. With `num_workers > 1`, tensors that cluster on the GPU run in the main process while the worker processes handle the others. +- Pruning modules now store their step count in a `step_count` buffer, so a `MagnitudePruner` state dict saved before this change cannot be reloaded with `prepare(state_dict=...)`. + +### Removed + +- Removed the deprecated `ExportBackend._TORCH` alias. Use `ExportBackend.Torch` instead. +- Removed the `prev_sparsity` argument from `SparsityScheduleBase.compute_sparsity` and its subclasses. Schedules are now pure functions of the step count; `PolynomialDecaySchedule` returns the last update's sparsity for steps between updates instead of passing through `prev_sparsity` or raising `ValueError` when it is omitted. + +### Fixed + +- Fixed `KMeansPalettizer.prepare(state_dict=...)` failing for configs with a quantized LUT, and `MagnitudePruner` restarting its sparsity schedule from step 0 after a model saved mid-schedule was reloaded. +- Preserve original floating-point casts that lose precision or range when cleaning up cast chains. + ## [0.3.0] - 2026-09-25 ### Added @@ -59,3 +88,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 [0.2.0]: https://github.com/apple/coreai-optimization/commits/v0.2.0/ [0.2.1]: https://github.com/apple/coreai-optimization/compare/v0.2.0...v0.2.1/ [0.3.0]: https://github.com/apple/coreai-optimization/compare/v0.2.1...v0.3.0/ +[0.4.0]: https://github.com/apple/coreai-optimization/compare/v0.3.0...v0.4.0/ diff --git a/changelog.d/+auto-detect-palettized-weights.added b/changelog.d/+auto-detect-palettized-weights.added deleted file mode 100644 index ab543ed0..00000000 --- a/changelog.d/+auto-detect-palettized-weights.added +++ /dev/null @@ -1 +0,0 @@ -K-means palettization detects weights that are already palettized for their spec (at most `2 ** n_bits` distinct values or vectors per block, e.g. a fake-palettized checkpoint) and uses those unique values as the centroids instead of running k-means. Other weights fall back to k-means after one small probe. diff --git a/changelog.d/+preserve-lossy-float-casts.fixed b/changelog.d/+preserve-lossy-float-casts.fixed deleted file mode 100644 index 6c2172a4..00000000 --- a/changelog.d/+preserve-lossy-float-casts.fixed +++ /dev/null @@ -1 +0,0 @@ -Preserve original floating-point casts that lose precision or range when cleaning up cast chains. diff --git a/changelog.d/+skip-single-element-activation-quant.changed b/changelog.d/+skip-single-element-activation-quant.changed deleted file mode 100644 index 312121ba..00000000 --- a/changelog.d/+skip-single-element-activation-quant.changed +++ /dev/null @@ -1 +0,0 @@ -Graph-mode `Quantizer` now skips activations and states with at most one element (e.g. the 0-d output of `x.sum()`), matching eager mode. This also fixes `prepare` failing on 0-d parameters under a per-channel weight spec. diff --git a/changelog.d/+vector-kmeans.added b/changelog.d/+vector-kmeans.added deleted file mode 100644 index 061d0965..00000000 --- a/changelog.d/+vector-kmeans.added +++ /dev/null @@ -1 +0,0 @@ -Add `VectorKMeansConfig` (`vector_kmeans_config` on `ModuleKMeansPalettizerConfig`) to set the k-means++ candidates per seeding step (`n_local_trials`), restarts, iteration cap and convergence tolerance of vector palettization (`cluster_dim > 1`). Vector k-means now clusters on a CUDA GPU, with flash-kmeans when it is installed and a batched PyTorch loop otherwise or for sensitivity-weighted clustering, and on CPU with scikit-learn, falling back to the PyTorch loop with a warning when scikit-learn is not installed. diff --git a/changelog.d/+vector-kmeans.changed b/changelog.d/+vector-kmeans.changed deleted file mode 100644 index aae90391..00000000 --- a/changelog.d/+vector-kmeans.changed +++ /dev/null @@ -1 +0,0 @@ -Vector palettization (`cluster_dim > 1`) now seeds with greedy k-means++ instead of farthest-point initialization, stops each block once its clustering error stops improving instead of running a fixed number of iterations, and is reproducible without manual seeding, including with `num_workers > 1`. It is much faster on GPU. With `num_workers > 1`, tensors that cluster on the GPU run in the main process while the worker processes handle the others. diff --git a/changelog.d/133.added b/changelog.d/133.added deleted file mode 100644 index ac9be5a2..00000000 --- a/changelog.d/133.added +++ /dev/null @@ -1 +0,0 @@ -Add MXINT8 support. diff --git a/changelog.d/135.added b/changelog.d/135.added deleted file mode 100644 index 6c075498..00000000 --- a/changelog.d/135.added +++ /dev/null @@ -1 +0,0 @@ -Added support for Python 3.14. diff --git a/changelog.d/139.joint-sparsity-palettization.added b/changelog.d/139.joint-sparsity-palettization.added deleted file mode 100644 index 4c082374..00000000 --- a/changelog.d/139.joint-sparsity-palettization.added +++ /dev/null @@ -1 +0,0 @@ -Added the `sparsity_detection_threshold` option to `PalettizationSpec`. A weight with at least that fraction of exact zeros is treated as already sparse and palettized jointly with its sparsity: only its nonzero values are palettized, the zeros stay exactly zero, and the exported model stores the weight in a sparse representation together with the palettized ops for the nonzero values. Supports per-tensor and per-group-channel granularity, per-channel scales and, on Core AI, quantized LUTs and vector palettization. diff --git a/changelog.d/154.public-torch-backend.added b/changelog.d/154.public-torch-backend.added deleted file mode 100644 index d842a834..00000000 --- a/changelog.d/154.public-torch-backend.added +++ /dev/null @@ -1 +0,0 @@ -Added the public `ExportBackend.Torch` backend for fast torch inference, e.g. running evals. `finalize(backend=ExportBackend.Torch)` folds weight compression (palettization, weight quantization, pruning) into dense weights and keeps activation fake quantization. The private `ExportBackend._TORCH` name is deprecated in favor of `ExportBackend.Torch`. diff --git a/changelog.d/157.prepare-state-dict.added b/changelog.d/157.prepare-state-dict.added deleted file mode 100644 index d399b8d3..00000000 --- a/changelog.d/157.prepare-state-dict.added +++ /dev/null @@ -1 +0,0 @@ -Added a `state_dict` argument to `prepare()` for `Quantizer` (eager and graph mode) and `MagnitudePruner`, matching `KMeansPalettizer`. It reloads a previously prepared model's compression state (calibrated qparams, pruning masks and step count, centroids), validates it against the config, and leaves the model in the same state as a regular `prepare()` call. diff --git a/changelog.d/157.prepare-state-dict.changed b/changelog.d/157.prepare-state-dict.changed deleted file mode 100644 index 4934fae8..00000000 --- a/changelog.d/157.prepare-state-dict.changed +++ /dev/null @@ -1 +0,0 @@ -Pruning modules now store their step count in a `step_count` buffer, so a `MagnitudePruner` state dict saved before this change cannot be reloaded with `prepare(state_dict=...)`. diff --git a/changelog.d/157.prepare-state-dict.fixed b/changelog.d/157.prepare-state-dict.fixed deleted file mode 100644 index 0dc5173b..00000000 --- a/changelog.d/157.prepare-state-dict.fixed +++ /dev/null @@ -1 +0,0 @@ -Fixed `KMeansPalettizer.prepare(state_dict=...)` failing for configs with a quantized LUT, and `MagnitudePruner` restarting its sparsity schedule from step 0 after a model saved mid-schedule was reloaded. diff --git a/changelog.d/157.prepare-state-dict.removed b/changelog.d/157.prepare-state-dict.removed deleted file mode 100644 index fee8c851..00000000 --- a/changelog.d/157.prepare-state-dict.removed +++ /dev/null @@ -1 +0,0 @@ -Removed the `prev_sparsity` argument from `SparsityScheduleBase.compute_sparsity` and its subclasses. Schedules are now pure functions of the step count; `PolynomialDecaySchedule` returns the last update's sparsity for steps between updates instead of passing through `prev_sparsity` or raising `ValueError` when it is omitted. diff --git a/changelog.d/167.int6-uint6-torch-backend.added b/changelog.d/167.int6-uint6-torch-backend.added deleted file mode 100644 index 52ca8b15..00000000 --- a/changelog.d/167.int6-uint6-torch-backend.added +++ /dev/null @@ -1 +0,0 @@ -Added the `torch.int6` and `torch.uint6` quantization dtypes. They are only supported for `ExportBackend.Torch` for now: `finalize` raises `NotImplementedError` for the Core AI and Core ML backends. diff --git a/changelog.d/168.remove-private-torch-backend.removed b/changelog.d/168.remove-private-torch-backend.removed deleted file mode 100644 index 7af691d6..00000000 --- a/changelog.d/168.remove-private-torch-backend.removed +++ /dev/null @@ -1 +0,0 @@ -Removed the deprecated `ExportBackend._TORCH` alias. Use `ExportBackend.Torch` instead.