Skip to content

fix(#529): share /app/work/planner and /app/work/bioqueryai between workbench and celery-worker - #83

Merged
man4ish merged 1 commit into
mainfrom
fix/529-shared-planner-bioqueryai-volumes
Sep 11, 2026
Merged

fix(#529): share /app/work/planner and /app/work/bioqueryai between workbench and celery-worker#83
man4ish merged 1 commit into
mainfrom
fix/529-shared-planner-bioqueryai-volumes

Conversation

@man4ish

@man4ish man4ish commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #529. Workflow Planner's and BioQueryAI's async (Celery) modes were writing real, correct completion data (status.json / .ask_index) inside celery-worker's filesystem, but workbench's HTTP polling handlers (planner_ask_status, bioquery_ask_status) read those same container-internal paths from workbench's own filesystem — where the paths never existed at all, because /app/work/planner and /app/work/bioqueryai were never declared as shared volumes for either service. Every other async-completion path bio_agent uses (runs, objects, workflow_runner_exec, workflow_runner_runs, pathway_enrichment_runs) already is shared; these two specifically were not.

Fix

Adds the two missing bind mounts to both workbench and celery-worker in docker-compose.yml, in the exact same form as the existing runs/objects mounts (${WORK_DIR}/<name>:/app/work/<name>, no extra options):

- ${WORK_DIR}/planner:/app/work/planner
- ${WORK_DIR}/bioqueryai:/app/work/bioqueryai

Scope check (re: #529's own follow-up scan)

Re-confirmed live against the current docker-compose.yml that, with this change, workbench and celery-worker now mount an identical set of 7 /app/work/* paths (runs, objects, workflow_runner_exec, workflow_runner_runs, pathway_enrichment_runs, planner, bioqueryai) — no other divergence between the two services' mount lists. This matches the issue's own follow-up comment, which already swept all 426 plugins and found this bug isolated to bio_agent's two paths (not a systemic pattern) — not re-running that full sweep here, just re-confirming the two services still match after this change.

Testing (real, live, both done after rebuilding + --force-recreateing both workbench and celery-worker; sha256 of the running image config matched a fresh docker compose build of both, confirming no staleness)

Workflow Planner — real submission (Django test client against the live view, real login session, no mocking) → real Celery completion → polled from workbench:

[00] poll -> PENDING
[01] poll -> COMPLETED

Full real multi-plan DAG result returned (ask_id=planner_ffd7008d4a1b). Confirmed the underlying status.json is now byte-identical when read from both containers (previously: present in celery-worker, No such file or directory in workbench).

BioQueryAI — real submission → real Celery task (real deepseek-r1:32b LLM calls via Ollama, ~4m42s) → polled from workbench:

[00] poll -> PENDING
[01..39] poll -> RUNNING   (state genuinely visible mid-run now, not stuck)

Task finished after my poll loop's timeout; re-polled afterward and workbench correctly reports state: COMPLETED (ask_id=ask_1789100190756). The query itself came back status: FAILED / "Time budget exceeded" — that's my test's own overly tight 2-minute bq_time_budget_minutes, not a symptom of this bug; the point being tested (workbench can see real state instead of PENDING-forever) is proven either way. No sign of #530's SIGSEGV during this run.

Historical stuck-PENDING data (pre-fix)

Checked: not recoverable. /app/work/planner and /app/work/bioqueryai were never a named volume or bind mount pre-fix, so anything a Celery task wrote there before this fix lived only in celery-worker's ephemeral container-writable layer. That container has since been recreated (both by the in-progress work this PR finishes, and again by this PR's own --force-recreate verification step) — no stopped container, docker-committed image, or named volume preserves the old layer (docker ps -a / docker images / docker volume ls all checked, nothing found). Any run that completed in celery-worker before this fix and was never separately re-run is gone for good; there is no data-recovery step to run here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V3hyWgSR8wJ3YbRt8zdpqa

…orkbench and celery-worker

Workflow Planner's and BioQueryAI's async (Celery) modes write their
status.json/.ask_index to /app/work/planner and /app/work/bioqueryai
respectively. Neither path was a shared volume between celery-worker
(where the task actually runs and writes the file) and workbench
(where the HTTP polling handler reads it) -- unlike every other
async-completion path bio_agent uses (runs, objects,
workflow_runner_exec, workflow_runner_runs, pathway_enrichment_runs),
which are all already shared.

Confirmed live in #529: the Celery task genuinely completes and
writes a real status.json inside celery-worker's filesystem, but
workbench's filesystem never has the file -- or even the parent
directory -- at all, so planner_ask_status/BioQueryAI polling reports
PENDING forever regardless of actual completion.

Mirrors the existing runs/objects mount exactly: same
${WORK_DIR}/<name>:/app/work/<name> bind-mount form, no extra options,
added to both the workbench and celery-worker services.

Closes #529.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3hyWgSR8wJ3YbRt8zdpqa
@man4ish
man4ish merged commit 48c858d into main Sep 11, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant