[Feat] Make the decision deadline configurable for slower local System One servers - #36
Open
chaimaerachdi wants to merge 2 commits into
Open
chaimaerachdi wants to merge 2 commits into
chaimaerachdi wants to merge 2 commits into
Conversation
JevModel cut every decision at 5 s, a deadline sized for Jev (0.3 s). A local System One server behind TYPESAFE_API_URL can be slower: on Google Flights, OneJev-27B on an A100 takes about 3.7 s a decision and more on the calendar page, so every run stopped at the eighth step. With 30 s it completed the task with the same 11 actions as Jev. S1A_DECISION_TIMEOUT_S sets the deadline; unset, it stays 5 s, so Jev is unchanged. A value that is not a positive number is a config error. An explicit timeout_s passed to JevModel.from_env still wins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IRONICBo
reviewed
Oct 1, 2026
| value = float(raw) | ||
| except ValueError: | ||
| value = 0.0 | ||
| if value <= 0: |
There was a problem hiding this comment.
float() accepts nan, inf, and overflow such as 1e9999; neither nan nor positive infinity satisfies value <= 0, so they pass this guard. nan then makes the computed deadline and remaining timeout nan, while inf removes the intended bound. Please also reject not math.isfinite(value) and extend the config-error regression with nan, inf, and 1e9999.
Author
float() reads nan, inf and 1e9999; nan passed the `<= 0` guard and made the deadline nan, inf removed the bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
|
Non-blocking:
FYI, out of scope for this PR: an explicit |
MrDongsls
approved these changes
Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
JevModelcuts every decision at 5 s (DECISIONS_TIMEOUT_S), a deadline sized for Jev, which answers in about0.3 s. A local System One server reached through
TYPESAFE_API_URLcan be slower. On Google Flights, OneJev-27B(OmniJev's new 27B model, which speaks the same API) on an A100 takes about 3.7 s a decision and more on the calendar
page, which offers 66 more elements. Every run stopped at the eighth step, right after the calendar opened.
How
s1a/decision_models/wire.py:decisions_timeout_from_env()readsS1A_DECISION_TIMEOUT_S; unset, it returnsDECISIONS_TIMEOUT_S(5 s). A value that is not a finite, positive number (nan,infand1e9999included) isMODEL_SERVICE_CONFIG_ERROR, naming the variable.s1a/decision_models/jev.py:JevModel.from_env(timeout_s=None)uses it; an explicittimeout_sstill wins.What
S1A_DECISION_TIMEOUT_S(docs/configuration.md,.env.example,CHANGELOG.md).Measured (Google Flights, Zurich to London, one way; A100 80 GB; headless Chromium)
Success is judged by a vision model (Gemini 2.5 Flash) on the final screenshot. Time is split with the repo's
profiler (
--profile-out), as indocs/benchmarks.md.qev serve)qev serve)Times are medians over the runs. At 30 s, ten runs were started and three were interrupted by hand before the end;
those three are not counted.
Verification
uv run ruff format --check . && uv run ruff check .: clean.uv run ty check: 1 diagnostic ins1a/decision_models/cua.py, the same onmain.uv run pytest -q --ignore=tests/test_browser_policy.py: 394 passed, 16 failed;mainon this Windows machinefails the same 16 (environment), so none is new. 4 new tests in
tests/test_decision_models_jev.py(
TestDecisionTimeout): default 5 s, the variable, an explicit value winning, invalid values (abc,0,-3,nan,inf,-inf,1e9999).