Repository navigation
[Experimental] Add unicode range target mode to the IFT encoder - #277
Merged
Merged
Conversation
… unicode-range segmentations.
- Adds new field target_mode to segmenter_config.proto which configures target mode as either IFT or UNICODE_RANGE. - Adds input validation when in UNICODE_RANGE mode that rejects incompatible segmenter config options.
…E_RANGE. - The generated config is modified to avoid invalid config options for unicode range mode. - CLI utilities are updated to add flags to enable target mode UNICODE_RANGE.
Used in target_mode == UNICODE_RANGE to pre merge input segments to make all conditions disjoint (or optionally all conditions above disjoint_conditions_probability_threshold).
- Generates unicode range css and font subsets from a segmentation plan. - Adds UnicodeRangeCompiler class which font2ur uses.
- Group more aggressively, less categories of codepoints are excluded from grouping. - For the remaining overlap ignored codepoints ensure these are placed into at most one merge group. - Also group based on codepoint interaction as discovered via dep graph analysis.
…roups. - Prevents merges here from causing segments to become part of other merge groups, which would make them shared segments that don't participate in the later cost merging phase.
Multiple passes cause merged segements to get re-evaluated against the probability threshold allowing merges that would have previously been below the threshold.
These are dropped in the produced segmentation plan and so don't contribute to total cost for the plan.
Composite conditions are dropped and don't impact the total cost, so no point in attempting merges around them.
garretrieger
marked this pull request as ready for review
October 17, 2026 01:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Producing a "unicode range" segmentation of a font (where a font is split into distinct subsets that are selected via @font-face's unicode-range property) is a closely related problem to what the IFT encoder is solving for IFT fonts. This PR explores utilizing the IFT encoder to produce unicode range segmentations. It adds a new "target_mode" setting to the segmenter config which can be either IFT (default, current behaviour) or UNICODE_RANGE.
Under UNICODE_RANGE target_mode the segmenter is modified to optimize the segmentation plan as if the font is being served by unicode range:
font2urwhich converts a segmentation plan into a CSS file and set of font subset files.I think this will be potentially a useful tool for optimizing font loading in non-IFT capable browsers. For example (if this ends up producing good results) we could consider having the encoder be able optionally output a combined IFT and unicode range serving package for a font that would contain an IFT font for IFT capable browsers with the unicode range setup as the fallback for non-IFT capable.
At the moment this is still highly experimental. I haven't done much validation on the quality of the unicode ranges it's generating.