From 466871d051cbdf858ec0dffc6587af017f1b76a1 Mon Sep 17 00:00:00 2001 From: Matthew Date: Fri, 2 Oct 2026 09:52:25 +1000 Subject: [PATCH] docs(todo): register the greentechhub v0.12 adoption work Settings, roles, per-user email accounts, a demo data seed and the remaining register_* calls, in order, ahead of the v0.11 items. Closes nothing yet; schedules the two users-API limitations to close. Co-Authored-By: Claude Opus 5.5 --- todo.md | 89 ++++++++++++++++++++++++++++++++++++- web-implementation-brief.md | 4 +- 2 files changed, 89 insertions(+), 4 deletions(-) diff --git a/todo.md b/todo.md index 6846994..98d9a36 100644 --- a/todo.md +++ b/todo.md @@ -66,8 +66,92 @@ Roughly, in order: `register_core`'s `TRUSTED_PROXIES` wired first); real Grafana panels once embedding is configured +## greentechhub v0.12 adoption: settings, roles, email accounts + +greentechhub-core v0.7.0, -fastapi v0.9.0 and -ui v0.12.0 shipped settings, roles, the user menu and the brand +theme. One PR per item, in this order. Items 1–3 can go now. Item 4 waits on the gth secret-settings releases (core +v0.8.0, ui v0.13.0, fastapi v0.10.0: gth roadmap #18–#20); until they ship, items 5–7 can go ahead. The +greentechhub-ui v0.11.0 adoption below continues after item 6, each PR tested against item 2's seed data. Same bar as +the v0.11 items: each PR's tests in their own new file, `pytest` (coverage ≥ 80), `ruff check src tests` and +`mypy src/pyfinbot`. + +- [ ] 1. `build(deps): greentechhub core v0.7.0, fastapi v0.9.0, ui v0.12.0` + - Pin `greentechhub-core[sqlalchemy] @ git+…@v0.7.0` directly; the code imports `greentechhub_core.*` but only + gets core through fastapi today. Bump fastapi to `@v0.9.0` and ui to `@v0.12.0`, then reinstall `.venv` (it has + core **0.6.0**). + - No API changes are needed; both releases are additive. ui 0.12 changes the look, so check every page: + - the brand accent: `btn-primary`, links, focus rings and checked inputs turn brand green; + - `gth_segmented` without option styles becomes the track; + - the favicon is a tile. + - Settle Dependabot #17 (SQLAlchemy 2.1) against core's `[sqlalchemy]` extra (`sqlalchemy>=2`). +- [ ] 2. `feat(dev): demo data seed`, so there's realistic data to test every following PR + - `scripts/seed_demo.py`: + - creates `demo-admin` and `demo-user` with known dev passwords; + - creates about 8 ASX stocks, some archived; + - adds about 150 transactions over three financial years, buys and sells, enough to page; + - adds dividends and a few notes. + - It's idempotent (`--reset` wipes the demo rows) and refuses to run unless `ENVIRONMENT == "development"`. + - README "Development": run it, then log in. + - Tests: two runs don't duplicate; it refuses in production; it produces the expected counts. +- [ ] 3. `feat(settings): user preferences, settings page and the user menu` + - `db/session.py`: expose `get_session_factory()` (an `async_sessionmaker`); `get_session()` is unchanged. + - `settings_table(SQLModel.metadata)` in `models/`, plus migration 4 (`gth_settings`). + - `register_settings(app, settings, registry=SettingsRegistry(USER_PREFERENCES), + store=SQLAlchemySettingsStore(table, async_session_factory=…), views=SettingsViews(templates=templates), + logout_url="/logout")`. No `manage_permission` yet; item 5 adds the App section. + - `web/templating.py`: add `settings_context` to `context_processors`. That brings the user menu (Settings, Log + out), the server-saved theme and `user_settings`, so `|date`, `|money` and `|number` follow the user. Remove + `dashboard.html`'s hand-built logout form. + - `routes/stocks.py`/`transactions.py`: `TableState.from_query(..., user_settings=…)` (via + `get_effective_settings`), so rows per page follow the user. CSV exports keep the plain formats. + - conftest: `register_settings` adds middleware and doesn't use `dependency_overrides`, so + `_ensure_auth_registered` is unaffected. + - Tests: needs login; preferences persist per user; the user menu logs out; page size follows the setting. +- [ ] 4. `feat(email): per-user email accounts for Commsec sync`, after the gth secret-settings releases + - Today one server-wide mailbox (`GMAIL_*`) is imported as whichever user clicks Sync. This moves the account into + each user's settings. + - Bump the gth pins and add core's `[crypto]` extra. Add a `SETTINGS_CIPHER_KEY` setting (a Fernet key) and pass + `FernetCipher` to `register_settings(cipher=…)`. + - An "Email sync" group of USER settings: + - `email.address`; + - `email.app_password` (secret: write-only in the form, encrypted at rest); + - `email.imap_host` (`imap.gmail.com`), `email.imap_port` (993), `email.mailbox` (`INBOX`); + - `email.commsec_sender` (`bounceback@commsec.com.au`). + - `core/email_sync.py`'s `fetch_commsec_emails`/`mark_seen` take an `EmailAccount` value instead of reading the + globals. The `/emails` page and `POST /api/emails/sync` load the current user's account (through + `Settings.get` and `get_secret`). + - The Emails page links to Settings when the account isn't configured. + - **Breaking config change:** remove `GMAIL_*`/`COMMSEC_SENDER` from `Settings`, the README, `.env` examples and + docker-compose. + - Tests: + - two users each sync only their own mailbox (mocked IMAP); + - the password is encrypted in `gth_settings` and appears in no page HTML; + - a blank password on save keeps the stored one; + - not configured gives a link to Settings. +- [ ] 5. `feat(permissions): roles, the admin pages and a locked-down users API` + - The role catalogue: `ADMIN = Role("admin", {users.manage, settings.manage})`. Plus a `ROLE_BOOTSTRAP` setting + (e.g. `demo-admin=admin`). + - `role_grants_table(SQLModel.metadata)` plus migration 5. `register_permissions(app, settings, roles=ROLES, + grants=SQLAlchemyGrantStore(…))`. + - `manage_permission="settings.manage"` turns on Settings › App; `RoleAdminViews(templates=…, + permission="users.manage")` serves `/admin/roles`. Both get nav items with `required_permission`. + - **Users API:** `GET /api/users/` and `POST /api/users/` require `users.manage`. That needs a bearer-JWT twin of + `require_permission`: resolve the `User`, build an `Identity`, and call the app's resolver. + - The first user comes from the seed, or a `create-user` script plus `ROLE_BOOTSTRAP`. + - Update `tests/conftest.py`'s `register_and_login()`, which relies on open registration. + - Closes the two users-API "Known limitations" below, and updates `web-implementation-brief.md` §2 and §10. + - Tests: the bootstrap admin reaches Settings › App and Roles; a normal user gets 403 and no admin nav; the users + API returns 403 without `users.manage`. +- [ ] 6. `chore(web): adopt register_logging, register_health and register_exception_handlers` + - Structured JSON logs, `/health` with core's `check_database` against the async engine, and JSON error envelopes + for `/api`. The pages' login redirect and HX-Redirect flow must be unchanged. +- [ ] 7. Then the greentechhub-ui v0.11.0 adoption below: #21 (date range presets, open) first, then the rest in order. + ## greentechhub-ui v0.11.0 adoption +> Continues after items 1–6 of "greentechhub v0.12 adoption" above (item 7 there). Test each PR against the demo +> seed (item 2). + v0.11.0 (the Data & forms release) adds date range presets, file drop, bulk selection, column view options, CSV export URLs, form-field extras and shared `money`/`number`/`date` filters. One PR per item, in this order (item 1 first: @@ -133,8 +217,9 @@ must pass `pytest` (coverage ≥ 80), `ruff check src tests` and ## Known limitations (accepted, not bugs) -- `GET /users/` requires a valid token but returns every user unfiltered — - no admin/RBAC system built; deliberate scope boundary. +- `GET /users/` requires a valid token but returns every user unfiltered, + and `POST /users/` needs no auth at all (open registration). Scheduled to + close: "greentechhub v0.12 adoption" item 5 (roles, `users.manage`). - Stateless JWT, 24h expiry, no refresh/revocation — a leaked token is valid up to 24h with no force-logout. Acceptable for personal-use scale; would need a blocklist or refresh tokens to harden. diff --git a/web-implementation-brief.md b/web-implementation-brief.md index c84815f..5f92844 100644 --- a/web-implementation-brief.md +++ b/web-implementation-brief.md @@ -14,7 +14,7 @@ PyFinBot currently ships as a FastAPI JSON API only (`/api/...`, JWT bearer auth - **Auth**: `OAuth2PasswordBearer` — `POST /api/auth/login` exchanges credentials for a JWT (`HS256`, 24h expiry, no refresh/revocation). Every route but registration and login requires `Authorization: Bearer `. - **Data model**: `User` (string PK) → `Transaction` (FK `user_id`, cascade delete) → `Stock` (FK `stock_id`, no cascade). `Transaction` carries computed fields (`total_value`, `cost`, `fy`) derived server-side on insert. - **Routers**: `auth`, `users`, `stocks` (CRUD + market sync + `search`), `transactions` (CRUD, paginated via `fastapi-pagination`), `transactions/import` (CSV/Excel bulk import with row-level validation and dedupe), `emails` (`POST /api/emails/sync-commsec` — Gmail IMAP ingestion of Commsec buy/sell confirmation emails), `dividends` (`POST /api/dividends/sync` — yfinance dividend history sync), `reports` (holdings snapshot, FY capital gains, FY dividend income). -- **Known gaps** (from the project's own backlog): no RBAC, stateless JWT with no revocation — unsolved anywhere in the ecosystem yet, not PyFinBot-specific to fix. CORS/env-mode config now exists (`ENVIRONMENT`/`CORS_ORIGINS` in `core/settings.py`), built independently of `greentechhub-fastapi`'s equivalent `register_core` (see §10 for the resulting duplication to clean up). +- **Known gaps** (from the project's own backlog): no RBAC, stateless JWT with no revocation — unsolved anywhere in the ecosystem yet, not PyFinBot-specific to fix. *(Update: roles now exist in greentechhub-core v0.7.0 / -fastapi v0.9.0; adopting them is `todo.md` → "greentechhub v0.12 adoption" item 5.)* CORS/env-mode config now exists (`ENVIRONMENT`/`CORS_ORIGINS` in `core/settings.py`), built independently of `greentechhub-fastapi`'s equivalent `register_core` (see §10 for the resulting duplication to clean up). ## 3. Requested capabilities @@ -105,7 +105,7 @@ Homelab already runs Grafana (`homelab-observe` node, internal URL `http://grafa ## 10. Non-functional / cleanup items -- RBAC and JWT revocation remain unbuilt anywhere in the ecosystem today — not tracked as a blocking dependency, just not yet PyFinBot's to solve either. +- RBAC and JWT revocation remain unbuilt anywhere in the ecosystem today — not tracked as a blocking dependency, just not yet PyFinBot's to solve either. *(Update: RBAC has since shipped in greentechhub-core v0.7.0 / -fastapi v0.9.0 and is scheduled as `todo.md` item 5 of "greentechhub v0.12 adoption"; JWT revocation is still unbuilt.)* - **CORS/env-mode is `greentechhub-fastapi`'s `register_core`, not `greentechhub-core`** (correcting v2, which named the wrong package): `register_core(app, settings)` wires request-ID, timing, security-header, CORS, and trusted-proxy middleware in one call, reading `CORS_ALLOWED_ORIGINS` (a list setting). This is a real, concrete duplication worth flagging: PyFinBot already built its own `CORS_ORIGINS`/`ENVIRONMENT` settings + manual `CORSMiddleware` registration independently (see `todo.md`) — the env var name doesn't even match (`CORS_ORIGINS` vs. `CORS_ALLOWED_ORIGINS`). Adopting `register_core` instead, at the same time as wiring `register_auth`, is a genuine follow-up consolidation — the same shape as BottleBot's `register_health` adoption — not just a nice-to-have. It's also a hard prerequisite for the `forward_auth` swap (§5): `register_core` is what installs `ProxyHeadersMiddleware` and reads `TRUSTED_PROXIES`. - No behavior change to the existing `/api` — the web layer is additive.