feat: upward bundles

BUILD

@@ -15,6 +15,7 @@ load("//:docs.bzl", "docs")
 
 package(default_visibility = ["//visibility:public"])
 exports_files([
+    "bundle_needs_conf.py.tpl",
     "default_conf.py.tpl",
     "pyproject.toml",
 ])

backlog.md

@@ -0,0 +1,179 @@
+<!--
+  *******************************************************************************
+  Copyright (c) 2026 Contributors to the Eclipse Foundation
+
+  See the NOTICE file(s) distributed with this work for additional
+  information regarding copyright ownership.
+
+  This program and the accompanying materials are made available under the
+  terms of the Apache License 2.0 which is available at
+  https://www.apache.org/licenses/LICENSE-2.0
+
+  SPDX-License-Identifier: Apache-2.0
+  *******************************************************************************
+-->
+
+# Backlog: upstream/main integration and `upward_bundles`
+
+Last updated: 2026-08-28
+
+## Current state
+
+The local `upward_bundle` branch is based on `upstream/main` at `298732e4`.
+The WIP integration is secured in commits `9eba2304` and `1a16863c` and has
+been force-updated to `upstream/upward_bundle`. The working tree is clean; no
+staged changes remain.
+
+The final WIP delta is currently 33 files with approximately 1,947 additions
+and 42 deletions. It is a working integration result, not yet a reviewable PR.
+
+Verification completed before the split:
+
+```text
+bazel test //...                                      21/21 passed
+.venv_docs/bin/python -m pytest -q src/tests/docs_bzl 28 passed
+```
+
+The full pre-commit hook also passed after the final formatting fix.
+
+## Topics contained in the staged delta
+
+### 1. Bundle-local Needs and bundle ownership
+
+`docs_bundle` now distinguishes a bundle's own sources and data from nested
+bundle content. It creates a local Needs export for the bundle's own sources,
+with generated configuration, source-code links, selected metamodel, and the
+correct `entry_doc` as Sphinx master document.
+
+Relevant implementation files:
+
+- `bzl/bundle_rules.bzl`
+- `docs.bzl`
+- `bundle_needs_conf.py.tpl`
+- root `BUILD`
+
+### 2. Hierarchical Needs via `upward_bundles`
+
+The bundle provider carries direct and transitive upward dependencies. A
+bundle can export its own Needs together with explicitly declared ancestors;
+source-less hierarchy groups, multiple parents, diamond-shaped graphs, and
+cyclic declarations are covered.
+
+The top-level `docs()` macro receives the same hierarchy and exposes the
+stable public `docs_source_bundle` and
+`docs_source_bundle_needs_upward` targets. Downward `bundles` composition and
+upward Needs interfaces remain separate concepts.
+
+### 3. Cross-module external Needs
+
+External Needs loading understands named `*_needs_upward` exports and their
+namespaced runfiles paths. The consumer documentation project's
+`project_url` is registered early enough and is used as the canonical base
+URL for imported Needs.
+
+This is covered by the cross-module fixture and the compatibility integration
+tests.
+
+### 4. Documentation and examples
+
+The staged documentation explains the ownership model, hierarchy contract,
+build graph, generated targets, and usage of `upward_bundles`:
+
+- `docs/concepts/hierarchical_bundle_needs.md`
+- `docs/how-to/upward_bundles.rst`
+- `docs/reference/bazel_macros.rst`
+
+The Mermaid-fence handling in `score_sphinx_bundle` supports the diagrams in
+the Markdown documentation. It should remain only if the documentation PR
+needs it and has a rendering regression test.
+
+## Proposed PR plan
+
+The PRs should be stacked on `upstream/main` in this order. Test fixtures and
+tests belong with the feature they verify; the documentation is intentionally
+separate from the implementation review.
+
+### PR 1 — `docs_bundle`: local Needs and hierarchical `upward_bundles`
+
+Approximate size: 900–1,100 LOC including focused tests.
+
+Include:
+
+- direct bundle ownership metadata and `bundle_own_files`;
+- bundle-local `*_needs_local` and merged `*_needs_upward` exports;
+- `upward_bundles` on `docs_bundle` and `docs()`;
+- source-less hierarchy groups and transitive parent propagation;
+- stable source-bundle aliases;
+- cycle, multiple-parent, diamond, and local-versus-upward tests;
+- Sphinx sandbox-safe metamodel inputs and `entry_doc` handling.
+
+This is the main implementation PR. Keep the core hierarchy documentation
+out of this PR except for concise API comments and test descriptions.
+
+### PR 2 — Cross-module external Needs and URL semantics
+
+Approximate size: 200–250 LOC including tests.
+
+Include:
+
+- parsing and resolving named `*_needs_upward` exports;
+- consumer-owned canonical `project_url` handling;
+- cross-module fixture and compatibility tests.
+
+This PR depends on PR 1 because it consumes the generated upward export
+targets.
+
+### PR 3 — Documentation for hierarchical bundles
+
+Approximate size: 700 LOC, mostly documentation.
+
+Include:
+
+- the hierarchy concept document;
+- the `upward_bundles` How-to;
+- the Bazel macro reference updates;
+- Mermaid-fence support if required by the new Markdown diagrams, together
+  with a small rendering test.
+
+## Cleanup before creating the PRs
+
+Remove or split out the following from the current staged integration:
+
+- Mermaid-fence support unless it is required by the documentation PR and is
+  covered by a regression test.
+
+Keep this file as the planning record, but do not mix the planning backlog into
+the implementation PRs unless repository policy requires it.
+
+## Open follow-ups
+
+These points are not blockers for the first hierarchy PR and should become
+separate issues if they remain relevant after review:
+
+- add a black-box test proving that a bundle-local export cannot see a sibling
+  or descendant without an explicit upward dependency;
+- define a negative test for missing or malformed upward export files;
+- define custom-metamodel behavior when an imported Need type is absent from
+  the consumer schema;
+- specify duplicate-ID and URL behavior when a bundle is mounted downward and
+  consumed upward at the same time;
+- review generated target visibility and public naming across repositories;
+- add release notes once the API is approved.
+
+## Architecture contract to preserve
+
+```text
+child local Needs export
+        │
+        └── depends upward on explicit parent exports
+
+parent public composed bundle
+        └── mounts child documentation downward
+```
+
+Bundle-local exports contain no consumer mount paths. Mount paths, backlinks,
+global checks, metrics, and final rendered output are resolved only by the
+composed host documentation build.
+
+`bundles` describes downward documentation composition. `upward_bundles`
+describes the explicit Needs interface available to a bundle's own validation.

bundle_needs_conf.py.tpl

@@ -0,0 +1,23 @@
+# *******************************************************************************
+# Copyright (c) 2026 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# SPDX-License-Identifier: Apache-2.0
+# *******************************************************************************
+# Default Sphinx configuration for a ``docs_bundle`` local Needs export.
+
+project = {PROJECT}
+version = "0.0.0"
+master_doc = {ENTRY_DOC}
+
+extensions = ["score_sphinx_bundle"]
+
+# A bundle-local export intentionally omits mounted descendants. The composed
+# host Needs build resolves those outgoing links later.
+suppress_warnings = ["needs.link_outgoing"]

bzl/bundle_rules.bzl

@@ -70,6 +70,13 @@ DocsBundleInfo = provider(
         # these are resolved at this bundle's mount (for example, a generated
         # index.rst).
         "data": "Bundle-owned generated/supporting files resolved at the bundle's mount.",
+        "own_sources": "Depset of documentation sources owned directly by this bundle.",
+        "own_sourcelinks": "Depset of source-code-link JSON files declared directly by this bundle.",
+        "own_data": "Depset of non-source-tree files declared directly by this bundle.",
+        "entry_doc": "Bundle-relative entry document for the bundle's own sources.",
+        "metamodel": "The metamodel file selected for this bundle's Needs processing.",
+        "direct_upward_bundles": "The bundle targets declared directly in upward_bundles.",
+        "upward_bundles": "Depset containing this bundle's declared upward dependencies and their transitive upward closure.",
     },
 )
 
@@ -238,6 +245,13 @@ def _docs_bundle_impl(ctx):
     own_source_files = []
     own_external_runfiles = []
     own_data = depset(direct = ctx.files.data)
+    upward_bundles = depset(
+        direct = ctx.attr.upward_bundles,
+        transitive = [
+            upward_bundle[DocsBundleInfo].upward_bundles
+            for upward_bundle in ctx.attr.upward_bundles
+        ],
+    )
 
     if ctx.files.srcs:
         runtime_path = _bundle_runtime_path(ctx)
@@ -317,6 +331,13 @@ def _docs_bundle_impl(ctx):
             sourcelinks = sourcelinks,
             external_runfiles = external_runfiles,
             data = all_data,
+            own_sources = depset(ctx.files.srcs),
+            own_sourcelinks = depset(ctx.files.sourcelinks),
+            own_data = own_data,
+            entry_doc = ctx.attr.entry_doc,
+            metamodel = ctx.file.metamodel,
+            direct_upward_bundles = ctx.attr.upward_bundles,
+            upward_bundles = upward_bundles,
         ),
     ]
 
@@ -327,23 +348,31 @@ _docs_bundle = rule(
         "sourcelinks": attr.label_list(allow_files = True),
         "strip_prefix": attr.string(default = ""),
         "entry_doc": attr.string(default = "index"),
+        "metamodel": attr.label(
+            allow_single_file = True,
+            default = Label("@score_docs_as_code//src/extensions/score_metamodel:metamodel_yaml"),
+        ),
         "bundles": attr.label_list(providers = [DocsBundleInfo]),
+        "upward_bundles": attr.label_list(providers = [DocsBundleInfo]),
         "bundle_mount_ats": attr.string_list(),
         "bundle_attach_tos": attr.string_list(),
         "data": attr.label_list(allow_files = True),
     },
     doc = "Internal rule that carries bundle files and their documentation-tree locations.",
 )
 
-def create_bundle(name, bundles, srcs = [], sourcelinks = [], strip_prefix = "", entry_doc = "index", data = [], visibility = None, **kwargs):
+def create_bundle(name, bundles, srcs = [], sourcelinks = [], strip_prefix = "", entry_doc = "index", metamodel = None, upward_bundles = [], data = [], visibility = None, **kwargs):
     """Create a reusable documentation bundle from files and child declarations."""
     parsed_bundles = [_parse_bundle_declaration(declaration) for declaration in bundles]
+    metamodel = metamodel or Label("@score_docs_as_code//src/extensions/score_metamodel:metamodel_yaml")
     _docs_bundle(
         name = name,
         srcs = srcs,
         sourcelinks = sourcelinks,
         strip_prefix = strip_prefix,
         entry_doc = entry_doc,
+        metamodel = metamodel,
+        upward_bundles = upward_bundles,
         bundles = [bundle.bundle for bundle in parsed_bundles],
         bundle_mount_ats = [bundle.mount_at for bundle in parsed_bundles],
         bundle_attach_tos = [bundle.attach_to for bundle in parsed_bundles],
@@ -484,3 +513,28 @@ def generate_code_target_sourcelinks(name, code_targets, visibility = None):
         visibility = visibility,
     )
     return ":" + name
+
+def _bundle_own_files_impl(ctx):
+    """Expose only the files owned directly by a documentation bundle."""
+    bundle = ctx.attr.bundle[DocsBundleInfo]
+    return [DefaultInfo(files = depset(transitive = [
+        bundle.own_sources,
+        bundle.own_data,
+    ]))]
+
+_bundle_own_files = rule(
+    implementation = _bundle_own_files_impl,
+    attrs = {
+        "bundle": attr.label(providers = [DocsBundleInfo]),
+    },
+    doc = "Exposes a bundle's own documentation files without nested bundles.",
+)
+
+def bundle_own_files(name, bundle, visibility = None):
+    """Create a target containing only a bundle's directly owned files."""
+    _bundle_own_files(
+        name = name,
+        bundle = bundle,
+        visibility = visibility,
+    )
+    return ":" + name