Skip to content

Add custom enum values to dataset field annotations - #85

Merged
lukasbindreiter merged 4 commits into
mainfrom
add-dataset-type-definitions
Oct 27, 2026
Merged

lukasbindreiter merged 4 commits into
mainfrom
add-dataset-type-definitions

Conversation

@lukasbindreiter

@lukasbindreiter lukasbindreiter commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Changes

Add FieldAnnotation.enum_values with an extensible EnumValue message containing only a nonempty string value. This replaces the earlier DatasetType.type_definitions proposal.

Clients set TYPE_ENUM, omit type_name, and supply values such as L1 and L2SP. The server generates the native protobuf enum and its numbers. Updates may append values while preserving existing numbers; removal, renaming, and reordering are deferred.

The same annotations are returned with the resolved descriptor. Values correspond in order to its enum members, excluding the generated zero member. Clients pair values with descriptor numbers without knowing generated naming conventions or treating list positions as numbers.

Custom enum filters use string_value for public values. Remove the unsupported enum_name literal and reserve its field name and tag 8. Equality, inequality, and existing OR/null expressions cover the initial use case. Enum queryables do not count toward the string-field limit.

Scope

API definitions and contract comments only. No backend generation, schema compatibility enforcement, ingestion, query compilation, SDK conversions, or generated core clients are included. Backend support must follow before the feature is usable.

Integration note

The removed field already exists on main, although the backend cannot compile it into a working filter. The configured WIRE_JSON compatibility check passes with its name and number reserved, but generated source APIs change. When core regenerates its clients, remove value.HasEnumName() from countQueryValueKinds in datasets-service/cmd/datasets-api-external/dataset-backend/expression_filter.go. No core files are changed in this PR.

Verification

Using Buf 1.73.0:

  • buf format --diff --exit-code .
  • buf build .
  • buf lint .
  • buf breaking . --against ".git#branch=origin/main"
  • git diff --check

All passed.

Add an optional descriptor set to dataset create/update schemas for custom enum definitions and future message types. Document fully qualified references and name-based response resolution.

Amp-Thread-ID: https://ampcode.com/threads/T-01a15368-905e-70db-a93b-31a2b835d742
Co-authored-by: Amp <amp@ampcode.com>
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Lint Protobuf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedOct 27, 2026, 1:14 PM

lukasbindreiter and others added 2 commits October 27, 2026 10:54
Keep full enum descriptors in create/update requests and define reversible public values. Specify server-side string literal resolution, exact enum symbols, supported operators, and null behavior without changing filter message fields.

Amp-Thread-ID: T-01a15368-905e-70db-a93b-31a2b835d742
Replace supporting descriptor input with a string-only EnumValue message. Document server-owned enum generation, append-only updates, response ordering, and public-value filtering.

Amp-Thread-ID: https://ampcode.com/threads/T-01a15368-905e-70db-a93b-31a2b835d742
Co-authored-by: Amp <amp@ampcode.com>
@lukasbindreiter lukasbindreiter changed the title Add dataset-local supporting type definitions Add custom enum values to dataset field annotations Oct 27, 2026
Use public string values as the only custom enum query representation. Reserve the old enum_name field name and tag to prevent accidental reuse.

Amp-Thread-ID: https://ampcode.com/threads/T-01a15368-905e-70db-a93b-31a2b835d742
Co-authored-by: Amp <amp@ampcode.com>
@lukasbindreiter
lukasbindreiter merged commit db2444e into main Oct 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant