Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,24 @@ The expected output is checked in at `examples/expected-selection.avsc`. Its
`data` union contains only `ListEventsResponse`; the `POST /events` JSON response
and `GET /events/{id}/export` `text/csv` response are skipped.

To also emit Confluent Schema Registry referenced schemas while preserving the
bundled `.avsc` output:

```bash
uv run openapi-get-avro generate \
--input examples/minimal.openapi.json \
--namespace com.example.sports \
--rootname SportsEnvelope \
--output build/sports-envelope.avsc \
--references-output-dir build/schema-references \
--root-subject sts.abc.def.avro-value \
--reference-subject-template "{fullname}"
```

The references directory receives one `.avsc` file per generated Avro named
type plus `manifest.json`. The manifest is ordered for registration: dependency
subjects first, then schemas that reference them, with the root envelope last.

Useful policy and naming options:

```bash
Expand Down Expand Up @@ -81,6 +99,10 @@ Accepted CLI values:
- `--unknown-object-policy`: `fail`, `map`, `string`, or `empty-record`
- `--include-status-codes`: comma-separated response codes, evaluated in the order provided
- `--remove-name-suffixes`: comma-separated, case-sensitive suffixes removed from generated Avro named types, for example `Dto`
- `--references-output-dir`: writes Confluent Schema Registry referenced schemas in addition to the bundled output
- `--references-manifest-output`: overrides the referenced-schema manifest path
- `--reference-subject-template`: formats registry subjects with `{fullname}`, `{namespace}`, `{name}`, and `{rootname}`
- `--root-subject`: overrides the root envelope subject; for Confluent's default topic value subject, use `<topic>-value`

The implemented strict behavior is the default: invalid enum values and ambiguous
free-form objects fail. `--any-of-policy union` is implemented when every branch
Expand Down
45 changes: 45 additions & 0 deletions docs/TECHNICAL_SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ Optional options:
--enum-policy fail, string, or sanitize. Default: fail.
--unknown-object-policy fail, map, string, or empty-record. Default: fail.
--remove-name-suffixes Comma-separated generated named-type suffixes to remove. Default: none.
--references-output-dir Also write Confluent Schema Registry referenced schemas to this directory.
--references-manifest-output
Manifest path for referenced schemas. Default: <references-output-dir>/manifest.json.
--reference-subject-template
Subject template. Supports {fullname}, {namespace}, {name}, and {rootname}. Default: {fullname}.
--root-subject Schema Registry subject for the root envelope. For Confluent TopicNameStrategy
topic values, use <topic>-value.
```

## OpenAPI selection rules
Expand Down Expand Up @@ -154,3 +161,41 @@ root envelope fields remain unchanged.
After generating the schema, validate it with `fastavro.parse_schema`.

Validation failures should include enough context to identify the generated type that failed.

## Confluent Schema Registry references

When `--references-output-dir` is set, the CLI must still write the bundled
self-contained schema to `--output` or stdout. It also writes one standalone
schema file per generated Avro named type and a manifest describing Confluent
Schema Registry registration metadata.

Referenced-schema files must use fully qualified Avro names for dependencies.
Reference subjects are formatted with `--reference-subject-template`. The root
envelope subject may be overridden independently with `--root-subject`, which is
the expected option for Confluent's default topic value subject, `<topic>-value`.
The manifest must be deterministic and ordered so dependency subjects appear
before schemas that reference them:

```text
shared leaf named types
-> named types that depend on them
-> GET response records
-> root envelope schema
```

Manifest entries include:

```json
{
"fullname": "com.example.GetMatchResponse",
"subject": "com.example.GetMatchResponse",
"file": "com.example.GetMatchResponse.avsc",
"references": [
{
"name": "com.example.Venue",
"subject": "com.example.Venue",
"version": "latest"
}
]
}
```
93 changes: 90 additions & 3 deletions src/openapi_get_avro/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@
import typer
from ruamel.yaml import YAML

from .converter import convert_openapi_to_avro
from .converter import convert_openapi_to_avro, convert_openapi_to_referenced_avro
from .exceptions import OpenApiAvroError
from .models import (
AnyOfPolicy,
EnumPolicy,
FieldNameCase,
GenerationOptions,
NameStrategy,
ReferencedSchemaSet,
UnknownObjectPolicy,
)

Expand Down Expand Up @@ -91,6 +92,41 @@ def _parse_name_suffixes(value: str) -> tuple[str, ...]:
return suffixes


def _render_json(value: object) -> str:
return json.dumps(value, indent=2, ensure_ascii=False) + "\n"


def _write_referenced_schemas(
schema_set: ReferencedSchemaSet, output_dir: Path, manifest_output: Path | None
) -> None:
output_dir.mkdir(parents=True, exist_ok=True)
for artifact in schema_set.artifacts:
(output_dir / artifact.filename).write_text(
_render_json(artifact.schema),
encoding="utf-8",
)

manifest_path = manifest_output or output_dir / "manifest.json"
manifest_path.parent.mkdir(parents=True, exist_ok=True)
manifest = [
{
"fullname": artifact.fullname,
"subject": artifact.subject,
"file": artifact.filename,
"references": [
{
"name": reference.name,
"subject": reference.subject,
"version": reference.version,
}
for reference in artifact.references
],
}
for artifact in schema_set.artifacts
]
manifest_path.write_text(_render_json(manifest), encoding="utf-8")


@app.command()
def generate(
input: Annotated[
Expand Down Expand Up @@ -149,6 +185,43 @@ def generate(
help="Comma-separated generated Avro named-type suffixes to remove",
),
] = "",
references_output_dir: Annotated[
Path | None,
typer.Option(
"--references-output-dir",
help="Also write Confluent Schema Registry referenced schemas to this directory",
),
] = None,
references_manifest_output: Annotated[
Path | None,
typer.Option(
"--references-manifest-output",
help=(
"Manifest path for referenced schemas; defaults to "
"<references-output-dir>/manifest.json"
),
),
] = None,
reference_subject_template: Annotated[
str,
typer.Option(
"--reference-subject-template",
help=(
"Subject template for referenced schemas. "
"Supports {fullname}, {namespace}, {name}, and {rootname}"
),
),
] = "{fullname}",
root_subject: Annotated[
str | None,
typer.Option(
"--root-subject",
help=(
"Schema Registry subject for the root envelope; for default "
"Confluent topic values use <topic>-value"
),
),
] = None,
) -> None:
"""Generate an Avro envelope schema from GET responses in an OpenAPI document."""
try:
Expand All @@ -174,8 +247,22 @@ def generate(
),
remove_name_suffixes=_parse_name_suffixes(remove_name_suffixes),
)
avro_schema = convert_openapi_to_avro(openapi_doc, options)
rendered = json.dumps(avro_schema, indent=2, ensure_ascii=False) + "\n"
if references_output_dir is None:
avro_schema = convert_openapi_to_avro(openapi_doc, options)
else:
schema_set = convert_openapi_to_referenced_avro(
openapi_doc,
options,
subject_template=reference_subject_template,
root_subject=root_subject,
)
avro_schema = schema_set.bundled_schema
_write_referenced_schemas(
schema_set,
references_output_dir,
references_manifest_output,
)
rendered = _render_json(avro_schema)
if output is None:
typer.echo(rendered, nl=False)
else:
Expand Down
Loading