Skip to content

feat(py): propagate type stubs and carry them into venvs - #1538

Merged
jbedard merged 4 commits into
mainfrom
pyi-propagation
Sep 24, 2026
Merged

jbedard merged 4 commits into
mainfrom
pyi-propagation

Conversation

@xangcastle

@xangcastle xangcastle commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

Builds on #1523 (its commit is kept as the first commit here, authorship intact) and makes transitive_pyi_files mean the same thing on both sides of the ruleset boundary.

Closes #558.

Changes

  • .pyi files listed in srcs are classified by extension into PyInfo.transitive_pyi_files, mirroring rules_python where stubs never count as runtime sources. transitive_sources no longer carries them, including through virtual-dependency resolutions. No new pyi_srcs attribute.
  • Stubs carried by a foreign provider (protobuf's py_proto_library _pb2.pyi, a rules_python pyi_srcs) propagate through rules_py's PyInfo. Stubs never enter runfiles.
  • Under --@aspect_rules_py//py:emit_rules_python_providers the emitted rules_python PyInfo also populates direct_pyi_files.
  • A virtual-dependency resolution target that carries PyInfo contributes only its transitive_sources; its DefaultInfo.files are used as sources only for targets without PyInfo.

Changes are visible to end-users: yes

PyInfo.transitive_sources no longer lists .pyi files from srcs; they move to PyInfo.transitive_pyi_files. Resolution targets with PyInfo no longer add their DefaultInfo.files to transitive_sources.

Test plan

  • //py/tests/type-stubs: analysis coverage for first-party, foreign, virtual-resolution, launcher, venv and compatibility-provider shapes, plus a py_test asserting stubs stay out of runfiles at execution time.
  • e2e/rules-python-interop: a rules_python library with pyi_srcs consumed by a rules_py py_library.
  • e2e/rules-python-protobuf: protobuf's py_proto_library generating native_greeting_pb2.pyi, propagated through rules_py's PyInfo.
  • e2e/rules-python-provider-compat: a rules_py stub surviving the merge into rules_python's PyInfo.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: d5d0cf0d-e965-400c-b421-880f5121ecbf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@aspect-workflows

aspect-workflows Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Aspect Workflows Tasks

📅 Thu Sep 24 01:39:04 UTC 2026

Task Results

  • ✅ buildifier · complete (clean) · 17.4s · Aspect · GitHub Actions · Check
  • ✅ gazelle · complete (clean) · 18.9s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-bazel-8 [test] · complete (324/324 passed · 322 cached) · 41.9s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-bazel-9 [test] · complete (314/314 passed · 312 cached) · 43.7s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-crossbuild-bazel-8 [test] · complete (39/39 passed · 39 cached) · 38.8s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-crossbuild-bazel-9 [test] · complete (39/39 passed · 39 cached) · 36.8s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-build-config-bazel-8 [test] · complete (1/1 passed · 1 cached) · 22.1s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-build-config-bazel-9 [test] · complete (1/1 passed · 1 cached) · 24s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-input-validation-bazel-8 [test] · complete (1/1 passed · 1 cached) · 20.3s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-input-validation-bazel-9 [test] · complete (1/1 passed · 1 cached) · 58.8s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-8 [test] · complete (2/2 passed · 2 cached) · 23.9s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-runtime-metadata-bazel-9 [test] · complete (2/2 passed · 2 cached) · 41.7s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-toolchain-settings-bazel-8 [test] · complete (1/1 passed · 1 cached) · 17.8s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-interpreter-toolchain-settings-bazel-9 [test] · complete (1/1 passed · 1 cached) · 47s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-interop-bazel-8 [test] · complete (19/19 passed · 19 cached) · 57.3s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-interop-bazel-9 [test] · complete (19/19 passed · 19 cached) · 2m 14s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-protobuf-bazel-8 [test] · complete (3/3 passed · 3 cached) · 26.3s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-protobuf-bazel-9 [test] · complete (3/3 passed · 3 cached) · 36.6s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-provider-compat-bazel-8 [test] · complete (5/5 passed · 5 cached) · 26.8s · Aspect · GitHub Actions · Check
  • ✅ test-e2e-rules-python-provider-compat-bazel-9 [test] · complete (5/5 passed · 5 cached) · 26.4s · Aspect · GitHub Actions · Check
  • ✅ test-examples-debugger-bazel-8 [test] · complete (1/1 passed · 1 cached) · 25.1s · Aspect · GitHub Actions · Check
  • ✅ test-examples-debugger-bazel-9 [test] · complete (1/1 passed · 1 cached) · 47.9s · Aspect · GitHub Actions · Check
  • ✅ test-examples-dev_deps-bazel-8 [test] · complete (1/1 passed · 1 cached) · 24.9s · Aspect · GitHub Actions · Check
  • ✅ test-examples-dev_deps-bazel-9 [test] · complete (1/1 passed · 1 cached) · 48.6s · Aspect · GitHub Actions · Check
  • ✅ test-examples-django-bazel-8 [test] · complete (1/1 passed · 1 cached) · 23.7s · Aspect · GitHub Actions · Check
  • ✅ test-examples-django-bazel-9 [test] · complete (1/1 passed · 1 cached) · 50.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-multi_version-bazel-8 [test] · complete (2/2 passed · 2 cached) · 25.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-multi_version-bazel-9 [test] · complete (2/2 passed · 2 cached) · 46.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-protobuf-bazel-8 [test] · complete (1/1 passed · 1 cached) · 13.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-protobuf-bazel-9 [test] · complete (1/1 passed · 1 cached) · 19s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_binary-bazel-8 [test] · complete (1/1 passed · 1 cached) · 22.4s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_binary-bazel-9 [test] · complete (1/1 passed · 1 cached) · 55.3s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_pex_binary-bazel-8 [test] · complete (1/1 passed · 1 cached) · 22s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_pex_binary-bazel-9 [test] · complete (1/1 passed · 1 cached) · 40.1s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_venv-bazel-8 [test] · complete (3/3 passed · 3 cached) · 27.2s · Aspect · GitHub Actions · Check
  • ✅ test-examples-py_venv-bazel-9 [test] · complete (3/3 passed · 3 cached) · 45.6s · Aspect · GitHub Actions · Check
  • ✅ test-examples-pytest-bazel-8 [test] · complete (11/11 passed · 11 cached) · 24.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-pytest-bazel-9 [test] · complete (11/11 passed · 11 cached) · 1m 30s · Aspect · GitHub Actions · Check
  • ✅ test-examples-uv_pip_compile-bazel-8 [test] · complete (1/1 passed · 1 cached) · 23s · Aspect · GitHub Actions · Check
  • ✅ test-examples-uv_pip_compile-bazel-9 [test] · complete (1/1 passed · 1 cached) · 1m 14s · Aspect · GitHub Actions · Check
  • ✅ test-examples-virtual_deps-bazel-8 [test] · complete (1/1 passed · 1 cached) · 19.5s · Aspect · GitHub Actions · Check
  • ✅ test-examples-virtual_deps-bazel-9 [test] · complete (1/1 passed · 1 cached) · 1m 30s · Aspect · GitHub Actions · Check
  • ✅ test-root-bazel-8 [test] · complete (396/396 passed · 395 cached) · 4m 7s · Aspect · GitHub Actions · Check
  • ✅ test-root-bazel-9 [test] · complete (395/395 passed · 394 cached) · 2m 57s · Aspect · GitHub Actions · Check

⏱ Last updated Thu Sep 24 01:43:16 UTC 2026 · 📊 GitHub API quota 473/7,700 (6% used, resets in 55m)
🚀 Powered by Aspect CLI (v2026.38.34)  |  Aspect Build · X · LinkedIn · YouTube

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

py_binary startup benchmark

Version Time (ms) vs BCR vs main Build (s)
BCR 1.11.7 163.3/163.3 ±1.4 — — 1.58
main 56.3/56.0 ±1.0 -65.7% — 1.74
PR 55.8/55.6 ±0.5 -65.9% -0.7% 1.45

Time = mean/median ±stddev.
Measured with hyperfine --warmup 5 --runs 50 --shell=none on Linux
Gate: PR vs HEAD main median (threshold: 10%, and must exceed the 2×SE noise floor, here 0.5%). BCR is shown only as a historical baseline.
Build time: cold bazel build //:bench with isolated output base, no disk cache; external repos prefetched so network is excluded.

sys.path quality

Version entries sp roots dupes
BCR 1.11.7 6 1 0
main 7 2 0
PR 7 2 0

sys.path quality measured by bench_syspath inside the assembled venv: sys.path entries, distinct site-packages roots, duplicate realpaths. Duplicates indicate symlink redundancy; many roots suggest an inefficient venv layout.

Bazel analysis benchmark

Version Time (ms) vs BCR vs main Targets Actions Configured (ext) Cfg/target Actions/cfg
BCR 2.0.0-alpha.6 1536/1490 ±283 — — 404 20613 7380 (6675) 18.3 2.79
main 1546/1518 ±211 +1.8% — 404 18363 7379 (6674) 18.3 2.49
PR 1577/1547 ±358 +3.8% +2.0% 404 18363 7379 (6674) 18.3 2.49

Measured with hyperfine --warmup 1 --runs 10 on Linux
Time = mean/median ±stddev. Cfg = configured targets; (ext) = the count in external repos (the @pypi hub machinery).
Gate: PR vs HEAD main median (threshold: 10%, and must exceed the 2×SE noise floor, here 17.3%). BCR is shown only as a historical baseline.
Command: warm-server bazel build --nobuild //workspace/..., analysis cache discarded each run via a fresh --action_env value; no disk cache.
Workload: 100 py_test targets — 2 test files/package in file generation mode (one venv per test file); a single dep_group.

py_image_layer benchmark

Scenario Version Time (s) Actions vs BCR vs main
analysis BCR 2.0.0-alpha.6 0.57/0.54 ±0.10 747 — —
analysis main 0.56/0.52 ±0.10 668 -3.3% —
analysis PR 0.55/0.53 ±0.10 668 -2.0% +1.4%
1p source BCR 2.0.0-alpha.6 0.62/0.60 ±0.11 4 — —
1p source main 0.59/0.58 ±0.07 5 -3.0% —
1p source PR 0.59/0.57 ±0.06 5 -4.4% -1.4%
3p wheel BCR 2.0.0-alpha.6 0.70/0.70 ±0.03 5 — —
3p wheel main 0.73/0.73 ±0.03 6 +3.2% —
3p wheel PR 0.73/0.72 ±0.04 6 +2.7% -0.5%

Time = mean/median ±stddev.
Measured with hyperfine on Linux, building //workspace:image_layers (10 binaries, ~30-wheel dep pool, grouped first-party/pip/interpreter tier) with isolated output base, no disk cache.
Scenarios: analysis = warm-server bazel build --nobuild, re-analyzed each run via a fresh --action_env value; incrementals run against a built state with warm analysis: source = append to the last package's lib.py, wheel = rewrite click post_install_patches content.
Actions: for Analysis, the total action count behind the image target from aquery deps(...); for incrementals, actions re-executed for the mutation, from a single instrumented run's BEP build metrics (deterministic; per-mnemonic breakdown in the *-actions.json artifacts). Informational only, not gated.
Gate: PR vs HEAD main median per scenario (threshold: 10%, and must exceed the 2×SE noise floor). BCR is shown only as a historical baseline.

@xangcastle
xangcastle requested a review from jbedard September 15, 2026 19:49
@xangcastle
xangcastle marked this pull request as ready for review September 15, 2026 19:49
@xangcastle

Copy link
Copy Markdown
Member Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3b746b3f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread py/private/py_info_interop.bzl
Comment thread py/private/py_venv/py_venv.bzl Outdated
runfiles = venv.runtime_runfiles.merge(ctx.runfiles(files = venv_only))
if include_sources:
runfiles = runfiles.merge(ctx.runfiles(transitive_files = venv.transitive_sources))
runfiles = runfiles.merge(ctx.runfiles(transitive_files = depset(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think PYI should be in the runfiles at all...

Comment thread py/private/py_library.bzl Outdated
# A resolution target's default outputs stand in for its sources (a wheel
# install tree has no PyInfo of its own). Stubs among them belong to the
# pyi depset collected alongside, not to the runtime sources.
direct = _runtime_sources(target[DefaultInfo].files.to_list())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't call .to_list()

Comment thread py/private/py_info.bzl Outdated
fields = {
"transitive_sources": "depset[File] — postorder depset of first-party `.py` sources in the transitive closure.",
"transitive_sources": "depset[File] — postorder depset of first-party runtime sources in the transitive closure; `.pyi` stubs are excluded.",
"transitive_pyi_files": "depset[File] — postorder depset of `.pyi` type stubs in the transitive closure: stubs listed in `srcs` plus those carried by deps of either ruleset.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can drop the "stubs listed in srcs plus those carried by deps of either ruleset." - that is an implementation detail of rules, and I don't think we should mention "other rulesets" in these docs

mkanat and others added 3 commits September 23, 2026 18:18
…to venvs

Builds on the transitive_pyi_files plumbing from #1523 and makes the field
mean the same thing on both sides of the ruleset boundary.

- `.pyi` files listed in `srcs` are classified by extension into
  `PyInfo.transitive_pyi_files`, mirroring rules_python where stubs never
  count as runtime sources. `transitive_sources` no longer carries them,
  including through virtual-dependency resolutions.
- Venvs that include sources and launchers put both depsets in runfiles, so
  stubs that only exist in a foreign provider (py_proto_library, a
  rules_python `pyi_srcs`) land beside the modules they annotate.
- Under the migration flag the emitted rules_python PyInfo also populates
  `direct_pyi_files`.
- Drops the redundant resolution wrapper and `has_py_info` guards; the
  interop accessor already returns an empty depset for foreign-less targets.

Tests: analysis coverage in //py/tests/type-stubs for first-party, foreign,
virtual-resolution, launcher, venv and compatibility-provider shapes, plus a
runtime check that stubs sit in runfiles. e2e: a rules_python `pyi_srcs`
library consumed by rules_py, protobuf's py_proto_library `_pb2.pyi`
propagating and landing in runfiles, and rules_python's PyInfo builder
merging a rules_py stub under the compatibility layer.
@jbedard
jbedard merged commit 79cec5a into main Sep 24, 2026
105 checks passed
@jbedard
jbedard deleted the pyi-propagation branch September 24, 2026 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FR]: Support for pyi files?

3 participants