feat: pin RubyLLM's backend with a platform option - #374
Open
TonsOfFun wants to merge 7 commits into
Open
Conversation
The ruby_llm railtie registers RubyLLM as an inflector acronym, which turns "RubyLLM".underscore into "rubyllm", so provider loading required a nonexistent rubyllm_provider.rb and raised a LoadError. Cover that require path with an alias file, the same fix openai_provider.rb applies for OpenAI. Fixes #371. Fix proposed in #372 by @aoki-ryusei. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
RubyLLM resolves which of its providers serves a request from the
model ID, and a model served by more than one -- gemini-2.5-flash
exists on both the Gemini API and Vertex AI -- lands on whichever
RubyLLM's registry prefers, with no way to say otherwise from
ActiveAgent.
Forward a new platform option to RubyLLM's provider: when resolving
the model, for embeddings as well as prompts:
generate_with :ruby_llm, model: "gemini-2.5-flash", platform: :vertexai
It is not named provider: because a provider reference is already the
first argument to generate_with, and not backend: because delegate_to
already uses backend: for the stack that runs a sub-agent. Omitting it
keeps model-based routing unchanged.
Closes #373.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
Takes main's rubyllm_provider.rb from #372 so the alias fix itself stays out of this PR's diff. # Conflicts: # lib/active_agent/providers/rubyllm_provider.rb
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
The helper swapped the :en entry of Inflections' @__instance__ map, but Rails 8.1 keeps the :en instance in a dedicated @__en_instance__, so the swap was a no-op and the registered acronym leaked into later tests. Mutate the live instance in both directions instead — register the acronym, then delete it and rebuild the acronym regexes — which works on both storage layouts, and assert the restoration inside the test so a future leak fails loudly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
TonsOfFun
marked this pull request as ready for review
August 24, 2026 20:58
Edge Rails freezes every Inflections instance after boot (active_support.freeze_inflections), so registering and removing the acronym on the live instance raises FrozenError on the railsmain CI job. Swap in an unfrozen dup for the test -- dup support is what Inflections#initialize_dup exists for -- and restore the original, frozen or not, afterwards. The dup goes in whichever slot the running Rails reads: @__en_instance__ where defined, the @__instance__ map on 7.2. Verified against rails7, rails8, and railsmain gemfiles locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #373. Also adds regression tests for the acronym loading fix that landed in #372 (#371, by @aoki-ryusei) — the branch is merged up with main, so the fix itself is not part of this diff.
Pin RubyLLM's backend with
platform:(#373)RubyLLM resolves which of its providers serves a request from the model ID. A model ID served by more than one backend —
gemini-2.5-flashexists on both the Gemini API and Vertex AI — lands on whichever RubyLLM'sPROVIDER_PREFERENCElists first (gemini), and ActiveAgent had no way to say otherwise: the first argument togenerate_with :ruby_llmselects the ActiveAgent adapter, not RubyLLM's internal backend.The new
platform:option forwards to RubyLLM'sprovider:when resolving the model, for embeddings as well as prompts:or in
config/active_agent.yml:Auth and region stay in
RubyLLM.configure(e.g.vertexai_project_id,vertexai_location). Omittingplatform:passesprovider: niltoRubyLLM::Models.resolve, which is identical to omitting it — today's model-based routing is unchanged.No new version floor:
Models.resolvehas acceptedprovider:since before it accepted theconfig:kwarg the provider already passes (verified against ruby_llm 1.3.0 and 1.16.0).Naming: why
platform:rather thanprovider:orbackend:provider:(RubyLLM's own name, proposed in the issue) collides with ActiveAgent's provider concept —generate_with's first argument is already a provider reference.backend:was the other candidate, butdelegate_toalready usesbackend:for "the provider stack that runs a sub-agent" (delegate_to X, backend: { provider: :anthropic, ... }). Reusing it here gives the word two meanings, one nested inside the other:backend: { provider: :ruby_llm, backend: :vertexai }.platform:is untaken in the codebase and names the thing being chosen — the platform serving the model (Gemini API vs Vertex AI, Bedrock, Azure). It composes cleanly with delegation:backend: { provider: :ruby_llm, platform: :vertexai }.Renaming is a small find/replace (one attribute, one kwarg, tests, docs) if you'd rather have
backend:.Regression tests for the acronym loading fix (#371 / #372)
test/providers/ruby_llm/provider_loading_test.rbcovers what #372 fixed: it registers theRubyLLMacronym the way the ruby_llm railtie does, assertsprovider_load("RubyLLM")resolves, and restores the original inflections afterwards — asserting in-test that nothing leaked. Edge Rails freezes everyInflectionsinstance after boot, so the acronym goes on an unfrozen dup swapped in for the test (Inflections#initialize_dupexists for exactly this), placed in whichever slot the running Rails reads (@__en_instance__on 8.1+, the instance map on 7.2). Reverting #372's alias file makes these tests error with the issue's exactcannot load such file -- active_agent/providers/rubyllm_provider.Changes
lib/active_agent/providers/ruby_llm/options.rb—platformattributelib/active_agent/providers/ruby_llm_provider.rb— forwardprovider: options.platform&.to_syminresolve_ruby_llm_provider!test/providers/ruby_llm/provider_loading_test.rb(new) — 5 tests: both require paths,provider_loadwith and without the acronym registered, remap variantstest/providers/ruby_llm/ruby_llm_provider_test.rb— 5 tests: platform reachesModels.resolvefor prompts and embeddings, symbol and string values, nil default, and an end-to-end test throughgenerate_with→prepare_prompt_parameters→ provider optionsdocs/providers/ruby_llm.md— "Pinning the Platform" section with the Vertex AI exampleAGENTS.md,CHANGELOG.md— notes for both changes (changelog credits fix: load RubyLLM when the gem registers its acronym #372 for the fix)Test plan
bin/teston the ruby_llm tree + provider concern tests, repeated under random seeds against the rails7, rails8, and railsmain gemfiles: 0 failures, 0 errors, 0 skips every run (the railsmain runs exercise the frozen-inflections boot state)bin/testmatches the main baseline exactly — the only failures in that environment are pre-existing missing-API-key integration tests, identical with and without this branchbin/rubocopclean on all touched filesOpen question
RubyLLM pairs
provider:withassume_model_exists:for model IDs not in its registry (fine-tunes, self-hosted endpoints). Left out here to keep scope tight; worth a follow-up if there's demand.🤖 Generated with Claude Code
https://claude.ai/code/session_01S5b2SozepYRbwEcFwkARek