OpenAPI GET Avro

openapi-get-avro is a deterministic Python CLI for producing Avro schemas from three kinds of input:

• OpenAPI JSON/YAML documents, using only selected GET response schemas.
• JSON files containing arrays of similar objects.
• JSON Schema documents, including local definitions and optional Confluent Schema Registry reference output.

The main OpenAPI workflow generates an Avro envelope record with fixed metadata fields and a data union containing one named record per selected GET response. The inference workflows generate regular Avro records from samples or JSON Schema.

Installation

Install from a built package or package index:

pip install openapi-get-avro
openapi-get-avro --help

When working from this repository, use the checked-in uv workflow instead:

uv sync
uv run openapi-get-avro --help

The package requires Python 3.11 or newer.

Command Overview

Command	Input	Output	Use when
generate	OpenAPI JSON/YAML	Avro envelope schema	You have API response schemas and want a Kafka value contract.
infer-json	JSON array of objects	Avro record schema	You have representative payload samples.
infer-json-schema	JSON Schema	Avro record schema	You already have a JSON Schema contract.

All commands write the generated Avro JSON to --output when provided. If --output is omitted, generate and infer-json write to stdout. infer-json-schema also writes to stdout unless --references-output-dir is provided, in which case omitting --output means references-only mode.

Quick Start

Generate an Avro envelope from an OpenAPI document:

openapi-get-avro generate \
  --input examples/minimal.openapi.json \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --output build/sports-envelope.avsc

Infer an Avro record from JSON samples:

openapi-get-avro infer-json examples/events.json \
  --name Event \
  --namespace com.example.events \
  --output build/Event.avsc

Infer an Avro record from JSON Schema:

openapi-get-avro infer-json-schema examples/event.schema.json \
  --name Event \
  --namespace com.example.events \
  --output build/Event.avsc

From a source checkout, prefix the same commands with uv run.

generate: OpenAPI GET Responses To Avro Envelope

generate reads an OpenAPI JSON or YAML document and emits one Avro envelope schema. Only GET operations are considered. By default it includes status code 200 and response content type application/json.

openapi-get-avro generate \
  --input examples/complex.openapi.yaml \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --output build/complex-sports-envelope.avsc

The root record contains fixed metadata fields: id, timestamp, operation, entity_type, and data. The data field is an Avro union of named records generated from selected GET responses.

Selection Behavior

• Paths are processed deterministically.
• Non-GET methods are ignored.
• Response status codes are selected by --include-status-codes; default is 200.
• Response media type is selected by --content-type; default is application/json.
• Local component refs such as #/components/schemas/Team are resolved.
• Remote refs and file refs are unsupported.

Examples

Select multiple response status codes in an explicit order:

openapi-get-avro generate \
  --input examples/complex.openapi.yaml \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --include-status-codes 200,206,default \
  --output build/sports-envelope.avsc

Select a subset of generated response records:

openapi-get-avro generate \
  --input examples/complex.openapi.yaml \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --include-response-records Venues,MatchParticipants \
  --output build/selected-sports-envelope.avsc

Selectors include the full generated response record name and simplified forms without Response, status suffixes, leading Get, and leading Api. For example, Venues can match GetApiVenues200Response; VenuesAttributes must be specified separately for GetApiVenuesAttributes200Response.

Transform payload field names and remove generated type suffixes:

openapi-get-avro generate \
  --input examples/complex.openapi.yaml \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --field-name-case snake_case \
  --remove-name-suffixes Dto \
  --output build/sports-envelope.avsc

Unwrap REST collection envelopes so the data union contains the item entity record instead of a response wrapper with pagination fields:

openapi-get-avro generate \
  --input examples/complex.openapi.yaml \
  --namespace com.example.sports \
  --rootname SportsEnvelope \
  --response-shape unwrap-collection \
  --output build/sports-envelope.avsc

--response-shape preserve keeps the OpenAPI response shape. This is the default. --response-shape unwrap-array unwraps top-level array responses when the array items map to named records. --response-shape unwrap-collection also unwraps common REST pagination envelopes with one payload array and metadata fields such as pageNumber, totalPages, hasPreviousPage, and hasNextPage.

Options

Required options:

Option	Type	Description
--input, -i	path	OpenAPI JSON/YAML file.
--namespace	string	Avro namespace for generated named schemas.
--rootname	string	Root Avro envelope record name.

General options:

Option	Default	Description
--output, -o	stdout	Output .avsc path.
--strict / --lenient	--strict	Fail on ambiguous constructs by default.

Selection options:

Option	Default	Description
--include-status-codes	200	Comma-separated response codes, evaluated in the order provided.
--include-response-records	include all	Comma-separated generated response record selectors.
--content-type	application/json	Response content type to include.

Naming and field options:

Option	Default	Allowed values	Description
--name-strategy	operationId	operationId, path	Source for generated response record names.
--field-name-case	preserve	preserve, snake_case, camelCase, PascalCase	Transform payload field names only.
--remove-name-suffixes	none	comma-separated Avro name suffixes	Remove exact trailing suffixes from generated named types.

Policy options:

Option	Default	Allowed values	Description
--any-of-policy	fail	fail, union	Map anyOf to an Avro union only when requested.
--enum-policy	fail	fail, string, sanitize	Handle invalid enum values.
--unknown-object-policy	fail	fail, map, string, empty-record	Handle object schemas without clear properties.
--response-shape	preserve	preserve, unwrap-array, unwrap-collection	Normalize selected response root schemas before Avro conversion.
--enforce-timestamp	disabled	flag	Map date, date-time, and timestamp string formats to Avro timestamp-millis.

Reference output options:

Option	Default	Description
--references-output-dir	disabled	Also write Confluent Schema Registry referenced schemas to this directory.
--references-manifest-output	<references-output-dir>/manifest.json	Manifest path for referenced schemas.
--reference-subject-template	{fullname}	Subject template supporting {fullname}, {namespace}, {name}, and {rootname}.
--root-subject	template result	Schema Registry subject for the root envelope; use <topic>-value for TopicNameStrategy.

With generate, providing --references-output-dir does not suppress the bundled schema. If --output is omitted, the bundled schema is still printed to stdout while reference files are written to disk.

infer-json: JSON Samples To Avro Record

infer-json reads a JSON file containing an array of similar objects and infers one Avro record schema.

openapi-get-avro infer-json examples/events.json \
  --name Event \
  --namespace com.example.events \
  --output build/Event.avsc

Fields present in every object are required. Fields missing from at least one object, or present with null at least once, become nullable fields with default: null. Nested objects and arrays are inferred recursively.

Ambiguous samples fail instead of guessing. Examples include mixed observed types such as 123 and "123", fields whose values are always null, and arrays whose observed items are always empty.

Examples

Infer timestamps and enums when samples are strong enough:

openapi-get-avro infer-json examples/events.json \
  --name Event \
  --namespace com.example.events \
  --enforce-timestamp \
  --enforce-enums \
  --output build/Event.avsc

Reuse structurally identical nested records:

openapi-get-avro infer-json examples/events.json \
  --name Event \
  --namespace com.example.events \
  --reuse-record-shapes \
  --output build/Event.avsc

Infer repeating object-valued properties as maps:

openapi-get-avro infer-json examples/tracking.json \
  --name TrackingFrame \
  --namespace com.example.tracking \
  --enforce-map \
  --output build/TrackingFrame.avsc

For example, repeated { "x": 1.0, "y": 2.0, "z": 3.0 } object shapes can be reused as a shared Position record. Objects with primitive entries, such as { "home": 1, "away": 2 }, remain records so fixed field groups are not over-generalized.

Options

Required argument and options:

Argument or option	Type	Description
input	path	JSON file containing an array of objects.
--name	string	Avro record name.
--namespace	string	Avro namespace for inferred named schemas.

Optional options:

Option	Default	Description
--output, -o	stdout	Output .avsc path.
--reuse-record-shapes	disabled	Reuse structurally identical inferred records as Avro named references.
--enforce-timestamp	disabled	Infer ISO date and date-time strings as Avro timestamp-millis.
--enforce-enums	disabled	Infer string fields as Avro enums when all observed values are valid Avro enum symbols.
--enforce-map	disabled	Infer object-valued properties with repeating value shapes as Avro maps.

infer-json-schema: JSON Schema To Avro Record

infer-json-schema reads a JSON Schema document and emits one Avro record schema. It supports object schemas and root array schemas whose items describe an object.

openapi-get-avro infer-json-schema examples/event.schema.json \
  --name Event \
  --namespace com.example.events \
  --output build/Event.avsc

Local JSON Schema refs under #/definitions/... and #/$defs/... are supported. They are converted into Avro named records and enums. Remote refs and file refs are unsupported.

When the JSON Schema root is a pure $ref, the referenced definition becomes the Avro root record name. --root-subject applies to that generated root record when reference output is enabled.

Examples

Map anyOf to Avro unions when every branch maps to a unique named Avro type:

openapi-get-avro infer-json-schema examples/event.schema.json \
  --name Event \
  --namespace com.example.events \
  --any-of-policy union \
  --output build/Event.avsc

Generate a bundled schema for code generation and reference files for Schema Registry:

openapi-get-avro infer-json-schema .local/aed/analytico.saed.review-inbox.schema.json \
  --name AedEvent \
  --namespace com.sts.analytico.saed \
  --output build/com.sts.analytico.saed.review-inbox.avsc \
  --references-output-dir build/review-inbox-references \
  --root-subject analytico.saed.review-inbox-value \
  --reference-subject-template "{fullname}"

Generate only the referenced schemas and manifest:

openapi-get-avro infer-json-schema .local/aed/analytico.saed.review-inbox.schema.json \
  --name AedEvent \
  --namespace com.sts.analytico.saed \
  --references-output-dir build/review-inbox-references \
  --root-subject analytico.saed.review-inbox-value \
  --reference-subject-template "{fullname}"

References-only mode applies only to infer-json-schema: when --references-output-dir is provided and --output is omitted, the command writes reference files and the manifest, and does not print the bundled schema to stdout.

Options

Required argument and options:

Argument or option	Type	Description
input	path	JSON Schema file.
--name	string	Avro record name hint.
--namespace	string	Avro namespace for inferred named schemas.

General and policy options:

Option	Default	Allowed values	Description
--output, -o	stdout, or omitted in references-only mode	path	Output .avsc path.
--any-of-policy	fail	fail, union	Handle JSON Schema anyOf.
--unknown-object-policy	fail	fail, map, string, empty-record	Handle object schemas without clear properties.
--enforce-timestamp	disabled	flag	Map date, date-time, and timestamp string formats to Avro timestamp-millis.

Reference output options:

Option	Default	Description
--references-output-dir	disabled	Write Confluent Schema Registry referenced schemas to this directory.
--references-manifest-output	<references-output-dir>/manifest.json	Manifest path for referenced schemas.
--reference-subject-template	{fullname}	Subject template supporting {fullname}, {namespace}, {name}, and {rootname}.
--root-subject	template result	Schema Registry subject for the inferred root record.

Confluent Schema Registry Reference Output

generate and infer-json-schema can emit Confluent-friendly referenced schemas in addition to, or instead of, a bundled schema depending on command and options.

The references directory contains one .avsc file per generated Avro named type plus a manifest. The manifest is deterministic and ordered for registration: dependencies first, then schemas that reference them, with the root schema last.

Manifest entries look like this:

{
  "fullname": "com.example.Event",
  "subject": "com.example.Event",
  "file": "com.example.Event.avsc",
  "references": [
    {
      "name": "com.example.Venue",
      "subject": "com.example.Venue",
      "version": "latest"
    }
  ]
}

Subject templates support:

Placeholder	Meaning
{fullname}	Fully qualified Avro name, for example com.example.Event.
{namespace}	Avro namespace, for example com.example.
{name}	Short Avro name, for example Event.
{rootname}	Root record name for the current command.

Use --root-subject <topic>-value when publishing the root schema with Confluent's default TopicNameStrategy.

Bundled Schema Versus References

Use the bundled schema from --output for tools that need a self-contained Avro schema, including many code generators such as avrogen. Schema Registry references are intended for registration workflows; a generated .avsc reference file may depend on other files listed in the manifest.

Typical dual-output workflow for JSON Schema:

openapi-get-avro infer-json-schema .local/aed/analytico.saed.review-inbox.schema.json \
  --name AedEvent \
  --namespace com.sts.analytico.saed \
  --output build/com.sts.analytico.saed.review-inbox.avsc \
  --references-output-dir build/review-inbox-references \
  --root-subject analytico.saed.review-inbox-value

Use the bundled file for avrogen, and the manifest plus reference files for Schema Registry registration.

Avro Mapping Notes

• string maps to Avro string.
• string with format: uuid maps to Avro string with logical type uuid.
• string with format: date maps to Avro int with logical type date unless --enforce-timestamp is set.
• string with format: date-time maps to Avro long with logical type timestamp-millis.
• integer maps to long; integer with format: int32 maps to int.
• number maps to double; number with format: float maps to float.
• Optional or nullable fields are emitted as Avro unions with null first and default: null.
• Avro unions are JSON arrays. Avro does not use a literal oneof keyword.
• Enums become Avro enums only when values are valid Avro enum symbols, unless a command option changes that behavior.

Determinism

The generator is designed to behave like a compiler: same input and options should produce byte-for-byte stable output or fail with an actionable error.

Important deterministic behaviors include:

• OpenAPI paths are processed in sorted order.
• Response status code order follows --include-status-codes.
• OpenAPI and JSON Schema property order is preserved for fields.
• Generated names and reference manifests are stable.
• JSON output uses two-space indentation.

Troubleshooting

Most failures include the input path and, for OpenAPI conversion, the selected GET response context.

Common causes:

Message or symptom	Command	Meaning and fix
Unsupported $ref	generate, infer-json-schema	Only local component refs or local JSON Schema definitions are supported. Inline the schema or move it under supported local definitions.
Invalid Avro enum symbol	generate, infer-json-schema, infer-json --enforce-enums	Avro enum symbols must match Avro name rules. Use valid symbols, choose an enum policy where available, or avoid enum inference.
anyOf failure	generate, infer-json-schema	Default policy is fail. Use --any-of-policy union only when every branch maps to a unique named Avro type.
Free-form object failure	generate, infer-json-schema	Object schemas without clear properties are ambiguous. Use --unknown-object-policy map, string, or empty-record when that mapping is acceptable.
Mixed sample types	infer-json	Samples disagree on a field type, for example numbers and strings. Clean the samples or provide a JSON Schema instead.
Always-null field	infer-json	The inferer cannot infer a type from only null. Add representative non-null values or use JSON Schema.
Always-empty array	infer-json	The inferer cannot infer array item type. Include sample items or use JSON Schema.
avrogen cannot resolve types from reference files	reference output	Use the bundled schema produced by --output for code generation. Use reference files for Schema Registry registration.

Local Development

This repository uses uv for dependency management and command execution, Ruff for formatting and linting, mypy for type checking, and pytest for tests.

Set up the environment:

uv sync

Run the standard checks:

uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run pytest

The same checks are available through make:

make check

Useful development commands:

uv run ruff format .
uv run ruff check --fix .
uv lock
uv tree

After uv sync, select .venv/bin/python as the VS Code interpreter on Linux/macOS, or .venv\Scripts\python.exe on Windows. Ruff is configured as the formatter for Python files.