Bake pipeline drops EBX overrides and custom blueprints (placement-only today)

[Powback]

The bake pipeline now works end to end on Linux (see 968f523 for the mod.db -> LevelLoaderGen exporter, and #394 for placeholders baking into real ReferenceObjectData). But it only carries placement, and that is worth stating plainly because the editor looks like it supports much more.

What LevelLoaderGen actually reads — the complete set of per-object fields in ebx_json.py:

obj['guid']  obj['origin']  obj['parentData']  obj['originalRef']
obj['blueprintCtrRef']  obj['variation']  obj['transform']  obj['localTransform']

The string override appears nowhere in the generator.

Consequence: MapEditor does save per-instance EBX overrides (ProjectManager.lua, overrides = l_GameObjectSaveData.overrides), and they survive save/reload inside the editor — but they are silently dropped at bake time. Change a light's radius, a colour, a capture radius, then bake: the level gets the stock blueprint at the right position and rotation. Nothing errors, which is the worst part — the loss is invisible.

The same applies to the M1 per-instance blueprint clones. The clone is a DeepCloned DataContainer that exists only in the running game's ResourceManager; nothing serialises it to disk, so there is nothing for the generator to consume even if it wanted to.

Also silently skipped today, for the record:

• origin == 3 (custom children) — # custom children not supported
• vanilla objects whose parentData.typeName is PrefabBlueprint / SpatialPrefabBlueprint — # prefab system not yet implemented

Why this is tractable rather than a rewrite. Both halves already exist, they have just never been connected:

1. Serialising a modified blueprint — PartitionSerializer already turns a live DataContainer into JSON, which is how the inspector shows EBX today.
2. Compiling arbitrary EBX into a bundle — that is exactly what Rime's add_json_partition does, and what the generator already drives for the subworld partition it synthesises.

So the shape of the work is:

• MapEditor: export each per-instance clone as an EBX JSON partition alongside the save (one partition per modified blueprint, keyed by the object guid), rather than only recording the override deltas.
• LevelLoaderGen: emit those partitions into the custom bundle and point that object's ROD Blueprint at the new partition/instance guid instead of the stock one.

Until then, the honest scope of the bake is: stock blueprints, placed and rotated correctly, with variations and vanilla deletions — no data edits.

Related: #394 (placeholders bake correctly — they need no overrides, which is why that path works today), #391 (level-bus wiring is a separate missing dimension: even a correctly baked capture point is not wired into the gamemode).

[Powback]

Per-instance overrides now bake. Verified through Rime into a compiled superbundle, not just up to the JSON — 01704b4 (live-ebx), plus a8e5db8 for the ext side and docs/bake-pipeline.md.

Pipeline: tools/bake.py --project X --generator …/LevelLoaderGen --rimepath …. It wraps LevelLoaderGen rather than forking it — run its save→EBX step unchanged, drop each converted clone partition into the same intermediate folder, repoint that object's ROD, then run its bundle+mod steps. No upstream patch needed, because bundles.py already adds every file in that folder to the bundle.

Verified on WallLamp_01_MP001, ConeInnerAngle 0 → 13.5: 420-instance partition, 0 dangling references, ROD Blueprint matches the new partition guid + primary instance, superbundle compiles (70,960 bytes).

Isolation is structural, per the point raised in review: each overridden instance gets its own partition and only its own ROD is repointed, so sibling instances of the same prefab keep resolving the stock blueprint. There is no shared mutable state to leak through.

Three things Rime rejects, each found by compiling real data, each with a misleading error — all now in docs/bake-pipeline.md §8:

1. $type on inline struct members → Could not find member '$type' on object of type 'EventConnection'. It resolves those from the declared array element type; $type belongs on instances only.
2. Explicit null fields → bare NullReferenceException at fb.OutputNodeData.Serialize. Rime defaults struct-valued fields to new(), so a null replaces a working default and the writer dereferences it. Omit the key instead. PartitionSerializer emits the same {$type,$ref} shape for "genuinely null" and "could not read", so omission is the only correct choice for both.
3. Sound/voice instances serialize with no fields at all — _SerializeFields skips those types because reading them crashes the game. They compile; whether a modified object keeps its audio at runtime is unverified.

Building Rime from source rather than using the circulating 2023 RimeREPL.exe paid for itself here — every one of these was diagnosed by reading the exact line named in the stack trace.

Still open on this issue:

• Apply-to-blueprint is stored but skipped by the baker, and it prints that rather than silently dropping it. It needs to shadow the stock partition under its original name, which bundles.py cannot express (it names every partition CustomLevels/<map>/<file>). Whether a later-loaded bundle can override a stock partition at all is still unverified — if it cannot, that path needs a different design.
• Not yet loaded in-game. The bundle compiles and the references are correct, but I have not booted the baked level and looked at the lamp. That is the real proof and it is the next step.
• origin == 3 custom children and vanilla prefab children remain unsupported upstream.

[Powback]

Objects bake but do not render — root cause identified, and it is the approach, not the toolchain.

Two real bugs were found and fixed by loading in-game (BF3RM/LevelLoaderGen feature/ebx-overrides, cc7f15f): override blueprints were never added to the level's RegistryContainer.BlueprintRegistry, and bundles.py builds one bundle per file so every override sat in a bundle nothing ever loaded — confirmed live, the partition went from Partition not loaded on server to resolving 215KB.

The objects still do not appear, and the reason is structural.

Cloning the blueprint clones everything. Editing one light's colour on a wall lamp produces a partition of 239 containers, including StaticModelEntityData, CompositeMeshAsset, MeshLodGroup, RigidMeshAsset and MeshMaterial. DeepClone is deep, so the whole render chain gets NEW instance guids. The baked object therefore references mesh assets the level has never seen: they are absent from the mesh-variation index, and their resources are not in our bundle. The object is placed and correctly referenced, and draws nothing.

Rime's own source names this failure mode — GenerateRegistryContainerCommand: "a SubWorld's registry has to carry the MeshVariationDatabase so Level:RegisterEntityResources feeds it into the mesh-variation index at load", and EmitSubworldRegistryCommand has an mvdbonly mode "for a SubWorld realized through SWROD", which is exactly how the loader realizes our bundle. This is also why RM's existing baked levels are fine: their objects reference stock blueprints already in that index.

The fix is to stop cloning what did not change. An override touches one container — emit a partition holding only the blueprint plus the containers actually on an override path, and leave every other child as an EXTERNAL reference to the stock container. The mesh chain then stays stock, stays registered, and keeps rendering; only the edited light is new, and a light needs no mesh-variation entry. Partitions drop from ~300KB to a few KB as a side effect.

Implementation sketch: the clone is structurally identical to the original it came from, and m_InstanceClones already stores originalRef alongside the clone. Walking both trees in parallel pairs each cloned container with its stock counterpart, so SerializeCloneSubtree can emit a copy where the override paths land and a reference everywhere else. Chasing a MeshVariationDatabase for cloned meshes would be the wrong fix — it makes the expensive path work instead of not taking it.

Correcting my earlier claim on this issue: what was verified was that the partition compiles and loads. Rendering is a third thing and it does not work yet. Compiling proved very little here — the bundle was valid and self-consistent every time.

Blueprint-wide overrides remain skipped with a warning (partition shadowing), unchanged.