Record dispatches into one batch and submit when a result is read - #4
Open
yujiteshima wants to merge 2 commits into
Open
yujiteshima wants to merge 2 commits into
yujiteshima wants to merge 2 commits into
Conversation
52 vk* call sites, 26 of which return a VkResult, and none were checked. A
failed call left an unusable handle in place and execution carried on, which
showed up in three ways:
- the VM died. With no Vulkan driver present, vkCreateInstance failed, the
null instance went to vkEnumeratePhysicalDevices, and the process aborted
(exit 134) with no Ruby-level error. Same shape in create_buffer: when no
memory type was both host-visible and host-coherent the search fell back
to index 0, and map_buffer then returned NULL straight into a write loop.
- the error named the wrong cause. vkCreateComputePipelines leaves the
handle VK_NULL_HANDLE on failure, which ensure_pipeline read as "the .spv
is missing" -- telling the user to run `make -C shader` for shaders that
were already built and had been rejected by the driver.
- the numbers were quietly wrong. An unchecked vkQueueSubmit meant a lost
device never signalled its fence, vkWaitForFences returned at once, and
the caller read the unwritten buffer back as ordinary Floats.
Adds VK_CHECK / gpu_check, raising a RuntimeError naming the call and the
VkResult. gpu_init and dispatch_compute now take mrb_state so they can raise;
map_buffer too, and it no longer returns NULL. Pipeline state is recorded per
pipeline (PIPE_MISSING_SPV vs PIPE_CREATE_FAILED) so the two cases give
opposite advice. A failed gpu_init sets init_failed rather than being retried
on every later operation, which would leak a fresh context each time.
Since mrb_raise unwinds, buffers are now wrapped before the dispatch that
fills them -- the GC can only free what it owns. That also covers the case
where GPU::SFloat.cast is handed a non-numeric element. narray_sum's partial
buffer is wrapped for the same reason, so destroy_buffer is now only the
finalizer and create_buffer's own unwind path.
Two silent-truncation guards in the same family: GPU::SFloat.new now rejects
a size past UINT32_MAX rather than wrapping it into a different length, and a
dispatch past maxComputeWorkGroupCount is rejected rather than left to the
driver -- lavapipe runs it and returns the right answer, so the limit is easy
to miss in testing, while a hardware-bounded driver may not. GPU.info gains
:max_workgroups so callers and bug reports can see the bound.
Verified on Mesa lavapipe: 46 tests pass (2 new), plus the new
test/shader_error_test.rb in both modes. It is a separate file because it has
to break the process-wide GPU context. Checked by hand that the no-device path
now raises and leaves the VM alive where it previously aborted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAgXRCCQ43jwXzeXKS6hG4
Every operator used to be its own round trip: build a command buffer and a descriptor set, submit, wait on a fence, free. Almost all of the time went into that, not into arithmetic -- a 4-operator chain cost 4 waits. Now dispatch_pipeline() records into one long-lived command buffer with a memory barrier after each dispatch, and gpu_flush() submits and waits once, at the first sync point: a host read or write of a buffer the batch touches (map_buffer), a full descriptor pool, GPU.sync, or shutdown. Buffers carry the epoch they were last bound in, which is how map_buffer decides whether to flush and how the GC finalizer knows to park a still-referenced buffer in a graveyard until the batch has run. GPU.pending shows what is queued; GPU.sync_mode = :eager restores a wait per dispatch for measurement. dispatch_pipeline() is exported so add-on gems (mruby-gpu-kernel) join the same batch. Apple M5, 1024 elements: a * 2 + 1 - 3 + 4 goes from 0.95 ms (4 waits) to 0.25 ms (1 wait); power_spectrum from 2.69 ms (12 waits) to 0.29 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every operator used to be its own round trip: build a command buffer and a descriptor set, submit, wait on a fence, free. Almost all of the time went into that, not into arithmetic —
a * 2 + 1 - 3 + 4cost four waits.This PR records every dispatch into one long-lived command buffer (with a memory barrier after each) and submits once, at the first sync point:
to_a,head,sum,seq,fill, …) —map_bufferis the sync pointGPU.syncBuffers carry the epoch they were last bound in: that is how
map_bufferdecides whether to flush, and how the GC finalizer knows to park a still-referenced buffer in a graveyard until the batch has run.rfftbenefits without changes: its1 + log2(n)passes become one submit.API
GPU.pending— dispatches recorded but not yet submittedGPU.sync— flush explicitly (handy when timing a batch)GPU.sync_mode = :eager | :deferred(default:deferred);:eagerrestores a submit and a wait per dispatch, kept so the two can be measureddispatch_pipeline()is exported ingpu_internal.h, so add-on gems (mruby-gpu-kernel) join the same batch.dispatch_compute()now takesGpuBuffer **.Numbers (Apple M5 / MoltenVK, 100-run mean)
a * 2 + 1 - 3 + 4power_spectrumAt 1M elements the four passes over memory dominate, and batching cannot remove those — fusing the expression into one shader (mruby-gpu-kernel) does: 0.35 ms.
Tests
59/59: the 45 existing tests plus 14 for batching — pending count, flush on read, a host write flushing a queued reader first, an untouched buffer not flushing, a batch longer than the descriptor pool, GC'd intermediates outliving the batch,
sum,rfftdeferred == eager, eager mode,sync_modeerrors. mruby-gpu-kernel's 20 tests pass against this branch.Notes
main.dispatch_pipeline().🤖 Generated with Claude Code