From 48aa82e824b35be285f6259397c0d4a5b698a6f8 Mon Sep 17 00:00:00 2001 From: DeliberateEnsemble Date: Sat, 17 Oct 2026 13:18:41 -0400 Subject: [PATCH 01/22] Fix code-comment contrast to meet WCAG AA (#937) The `--code-comment` token (#6a6f80) measured 3.8:1 on the dark code surface, below the WCAG AA 4.5:1 floor. It colours code comments, terminal and code-block title bars, language tags, copy buttons and the wizard's YAML pane header and comment lines. It is now #8f96a8: 6.43:1 on the code surface and at least 5.56:1 on the title-bar gradient, in both themes (the light theme keeps a dark code surface). Measured on the rendered pages, every text colour on the code surface reaches 4.5:1. Fixes #887 Resolves: F-W1-SITE-06 Co-authored-by: vortsghost2025 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 6 ++++++ site/css/style.css | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c510f6d..8b9059a3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,12 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Make the site's code comments readable: `--code-comment` was `#6a6f80`, 3.8:1 on the dark code + surface, below the 4.5:1 AA floor, and it colours code comments, terminal and code-block title bars, + language tags, copy buttons and the wizard's YAML pane header and comment lines. It is now `#8f96a8`, + 6.43:1 on the code surface and at least 5.56:1 on the title-bar gradient, in both themes (the light + theme keeps a dark code surface). Measured on the rendered pages, every text colour on the code + surface reaches 4.5:1 (F-W1-SITE-06, #887). - Use the Regulation's own Article 72 terms for post-market monitoring in the German and Spanish wizard copy ("Beobachtung nach dem Inverkehrbringen", "vigilancia poscomercialización"), and restore the "metadata" the French and Spanish hints had dropped, so every locale again separates ForgeLM's diff --git a/site/css/style.css b/site/css/style.css index dc6f909b..9385a90c 100644 --- a/site/css/style.css +++ b/site/css/style.css @@ -53,7 +53,7 @@ --code-bg: #0d1018; --code-border: #1e2230; --code-text: #d4d8e2; - --code-comment: #6a6f80; + --code-comment: #8f96a8; --code-keyword: #fb923c; --code-string: #6ee7b7; --code-number: #38bdf8; From ea82d6802130d3247840e4d88f889c42411852c3 Mon Sep 17 00:00:00 2001 From: DeliberateEnsemble Date: Sat, 17 Oct 2026 13:28:19 -0400 Subject: [PATCH 02/22] docs: fix KTO data-shape row to use completion column (#938) The KTO row of the "What to look at in your data" table in the Choosing a Trainer user manual said `{prompt, response, label: bool}`, but the loader (`_validate_trainer_columns` in `forgelm/data.py`) requires `prompt`, `completion` and `label`, so data shaped as the page recommended failed with a KeyError. The row now reads `{prompt, completion, label: bool}` in the English page and its Turkish mirror. Out of scope: the `{prompt, completion}` -> SFT row of the same table is a separate defect tracked in #403 and is left unchanged. Fixes #885 Resolves: F-W1-DOCS-TRAIN-02 Co-Authored-By: Claude Sonnet 5.5 --- CHANGELOG.md | 4 ++++ docs/usermanuals/en/concepts/choosing-trainer.md | 2 +- docs/usermanuals/tr/concepts/choosing-trainer.md | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b9059a3..e4cdb1c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,10 @@ All notable changes to ForgeLM are documented here. article and gender slips in the touched German, French and Spanish strings, and use one German term for the external stack. The new validation and interval keys keep their English fallback; native review of the deferred locales remains open (refs #948). +- Name the right columns for KTO in the EN/TR "Choosing a Trainer" user manual: its data-shape table + said `{prompt, response, label: bool}`, but the loader requires `{prompt, completion, label}` and + rejects the old shape with a `KeyError` (F-W1-DOCS-TRAIN-02, #885). The table's SFT row is a separate + defect, tracked in #403, and is unchanged here. - Say what Annex IV §9 asks for: a detailed description of the post-market monitoring system, its plan included. The wizard hint in every locale and the EN/TR configuration and compliance pages said it "includes the detailed plan"; German, French and Spanish now follow the Regulation's wording, and diff --git a/docs/usermanuals/en/concepts/choosing-trainer.md b/docs/usermanuals/en/concepts/choosing-trainer.md index 16d8b971..801b7a03 100644 --- a/docs/usermanuals/en/concepts/choosing-trainer.md +++ b/docs/usermanuals/en/concepts/choosing-trainer.md @@ -48,7 +48,7 @@ flowchart TD |---|---| | `{prompt, completion}` rows only | [SFT](#/training/sft) | | `{prompt, chosen, rejected}` rows | [DPO](#/training/dpo) (or [SimPO](#/training/simpo) if VRAM-tight) | -| `{prompt, response, label: bool}` rows | [KTO](#/training/kto) | +| `{prompt, completion, label: bool}` rows | [KTO](#/training/kto) | | Mix of SFT and preference rows | [SFT](#/training/sft) → [DPO](#/training/dpo) sequence, or [ORPO](#/training/orpo) for one-stage | | Programmatic reward (math grader, code tests) | [GRPO](#/training/grpo) | diff --git a/docs/usermanuals/tr/concepts/choosing-trainer.md b/docs/usermanuals/tr/concepts/choosing-trainer.md index e56f9a86..b3f0db67 100644 --- a/docs/usermanuals/tr/concepts/choosing-trainer.md +++ b/docs/usermanuals/tr/concepts/choosing-trainer.md @@ -48,7 +48,7 @@ flowchart TD |---|---| | Sadece `{prompt, completion}` satırları | [SFT](#/training/sft) | | `{prompt, chosen, rejected}` satırları | [DPO](#/training/dpo) (VRAM dar ise [SimPO](#/training/simpo)) | -| `{prompt, response, label: bool}` satırları | [KTO](#/training/kto) | +| `{prompt, completion, label: bool}` satırları | [KTO](#/training/kto) | | SFT ve tercih satırlarının karışımı | [SFT](#/training/sft) → [DPO](#/training/dpo) sıralı, veya tek aşama için [ORPO](#/training/orpo) | | Programatik reward (math grader, kod testi) | [GRPO](#/training/grpo) | From 9e6a8b288b725f86b3a8789f4f96583fb0c30de0 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sat, 17 Oct 2026 21:05:41 +0300 Subject: [PATCH 03/22] chore: add community-health files for contributor intake (#964) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Adds the repository files a public project is expected to carry, so a new contributor can find the rules and the ownership map without asking anyone — and widens two guards so the new root files are actually checked rather than assumed clean. No behaviour change: no Python under `forgelm/`, no dependencies, no config keys. ## Related issues Refs #500, #465 — the two open issues on GitHub Actions versions and mutable action tags; the `github-actions` entry in `dependabot.yml` is the closest thing in the backlog to them. This pull request resolves no issue. It was raised as a direct task rather than found in the backlog, which is also why it opens without a preceding issue; happy to file one and re-link if the project would rather keep that rule absolute. Opened from the review of this PR, for work it does not do: - #966 — the Code of Conduct reporting address does not resolve - #967 — `CITATION.cff` carries no release version or date - #968 — the unreferenced, stale `requirements*.txt` files ## Changes - **`CITATION.cff`** — makes ForgeLM citable through GitHub's "Cite this repository" panel. `version` and `date-released` are deliberately omitted: the released version is single-sourced in `pyproject.toml`, the site's copy is guarded by `tools/update_site_version.py --check`, and this file is not one of that tool's targets, so a version here would be an un-guarded duplicate. Tracked in #967, with a Zenodo DOI as the durable answer. - **`.github/CODEOWNERS`** — routes review requests. It does **not** gate a merge: whether a review is required before merging is a branch-protection setting, not something an entry here turns on. The file documents that the rules are last-match-wins, so an area owner is added *below* `*` rather than replacing it, and pins the scaffolding paths so an area owner cannot silently take them. - **`.github/dependabot.yml`** — dependency updates arrive as reviewed pull requests. `versioning-strategy: widen`, because the project declares bounded ranges with no lockfile, so the default `increase` is a no-op on every floor the ceiling already covers. `ruff`, `mypy` and `bandit` are ignored: `tests/test_packaging.py` requires their `[dev]` range to equal the one `ci.yml` installs, and a bump would fail that test on arrival. Python and Actions minor/patch bumps are grouped; majors stay individual. - **`GOVERNANCE.md`** — the maintainer-led model, the role ladder (contributor → maintainer; the undefined "reviewer" rung is gone), how decisions are made, the actual merge conditions, and an **Access and continuity** section that records the single-owner gap as a gap rather than promising a fix. - **`MAINTAINERS.md`** — the current maintainer, the areas, and the reply cadence: questions go to Discussions, and the initial-reply target is the same 72 hours `SECURITY.md` commits to. - **`.github/ISSUE_TEMPLATE/documentation.yml`** — documentation reports get a form; the chooser previously offered only a bug and a feature form. It no longer offers two entries named "Documentation", and it says that a maintainer completes the Turkish mirror when the contributor does not write Turkish. - **`CONTRIBUTING.md`** — the same mirror sentence, where a contributor reads it before opening a pull request rather than at review. - **`tools/check_readme_links.py`, `tools/check_source_path_refs.py`** — the two guards that check links and prose source paths now cover `GOVERNANCE.md`, `MAINTAINERS.md`, `SECURITY.md` and `CODE_OF_CONDUCT.md`, not only `README.md` and `CONTRIBUTING.md`. The link guard also resolves `blob/development/…` URLs, not just `blob/main/…`. - **`README.md`, `CHANGELOG.md`, `.github/ISSUE_TEMPLATE/config.yml`, `MANIFEST.in`** — link the new files (on `development`, where they land first), record the change under `[Unreleased]`, give the issue chooser two distinct entries, and ship `CITATION.cff` in the sdist. ## Type - ~~[ ]~~ Bug fix — no behaviour changes - ~~[ ]~~ New feature - [x] Documentation — `GOVERNANCE.md`, `MAINTAINERS.md`, `CITATION.cff`, the issue template, `README.md` links, `CONTRIBUTING.md` - ~~[ ]~~ Refactoring / code quality - [x] Test coverage — the two guard tests now cover four more surfaces - [x] CI / infrastructure — `dependabot.yml`, `CODEOWNERS`, `MANIFEST.in` ## Testing **Scope, stated honestly.** Before this pull request, none of the guards below read `GOVERNANCE.md` or `MAINTAINERS.md` as a link or path surface. An earlier version of this description listed eleven passing guards as evidence *about those two files* when none of them looked at them — an injected dead link, dead anchor or invented module path in either file would have passed all eleven. The two entries marked **(new coverage)** are the guards this PR changes so that they do inspect the new root files. - `check_readme_links.py --strict` **(new coverage)** — 129 links across 6 surfaces, all resolve. - `check_source_path_refs.py --strict` **(new coverage)** — 296 prose surfaces, every backticked source path exists. - `check_bilingual_parity`, `check_bilingual_code_blocks`, `check_tr_links_prefer_mirror`, `check_anchor_resolution`, `check_usermanual_self_contained`, `check_yaml_snippets`, `check_doc_numerical_claims`, `check_site_claims`, `check_module_size`, `check_ci_failure_masking`, `check_no_analysis_refs`, `check_release_record_sync`, `check_no_mutation_artifacts`, `check_strict_json_writers`, `check_audit_event_catalog`, `check_field_descriptions` — all pass. - `pytest tests/` — **5586 passed, 98 skipped**, coverage 87.37 %. - `ruff format --check .` and `ruff check .` — clean (291 files already formatted). - `python3 -m forgelm --config config_template.yaml --dry-run` — config is valid. - `CITATION.cff`, `dependabot.yml` and `documentation.yml` parse as valid YAML; `CITATION.cff` carries the required CFF keys. **Not verified, and not claimable from CI:** Dependabot's actual behaviour (it has never run here — the grouping, ignore and trigger assumptions are a reading of `dependabot-core`, not an observation), the "Cite this repository" panel, and how the new form renders in the issue chooser. All three are visible only after merge. ### Testing checklist - [x] `pytest tests/` passes — 5586 passed, 98 skipped - [x] `ruff check .` passes - [x] `ruff format --check .` passes - [x] New tests added for changed code — the `check_readme_links` fixture now creates every registered surface, so the wider surface set is exercised - [x] `python -m forgelm --config config_template.yaml --dry-run` works > The three `test_check_import_origin` cases fail unless the checkout under test is first on `sys.path`; they are environment-dependent and pass with `PYTHONPATH` set to the checkout, which is how the 5586 figure above was produced. ## Checklist - [x] My code follows the project's style (ruff formatted) and `docs/standards/coding.md` - [x] I've updated documentation if needed — and its Turkish mirror in the same PR — these are contributor-facing English files (`GOVERNANCE.md`, `MAINTAINERS.md`) alongside `CONTRIBUTING.md`, `SECURITY.md` and `CODE_OF_CONDUCT.md`, which the [localization standard](https://github.com/HodeTech/ForgeLM/blob/development/docs/standards/localization.md) keeps English-only, so there is no mirror to update - [x] I've added tests for new functionality — the guard surfaces the change depends on are covered - [x] No new dependencies added - [x] Config template (`config_template.yaml`) updated if new config fields added — not applicable, no config fields added - [x] CHANGELOG entry under `[Unreleased]` if a user or library consumer can observe the change — added under `### Added` --- .github/CODEOWNERS | 25 +++++ .github/ISSUE_TEMPLATE/config.yml | 4 +- .github/ISSUE_TEMPLATE/documentation.yml | 52 ++++++++++ .github/dependabot.yml | 69 +++++++++++++ CHANGELOG.md | 12 +++ CITATION.cff | 37 +++++++ CONTRIBUTING.md | 2 + GOVERNANCE.md | 123 +++++++++++++++++++++++ MAINTAINERS.md | 47 +++++++++ MANIFEST.in | 1 + README.md | 5 + tests/test_check_readme_links.py | 4 + tools/check_readme_links.py | 13 ++- tools/check_source_path_refs.py | 13 ++- 14 files changed, 403 insertions(+), 4 deletions(-) create mode 100644 .github/CODEOWNERS create mode 100644 .github/ISSUE_TEMPLATE/documentation.yml create mode 100644 .github/dependabot.yml create mode 100644 CITATION.cff create mode 100644 GOVERNANCE.md create mode 100644 MAINTAINERS.md diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 00000000..e0ff20ab --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,25 @@ +# Code owners for ForgeLM. +# +# When a pull request is opened, GitHub requests a review from the owner of each +# path it changes. This file only routes those review requests: whether a review +# is *required* before merging is a branch-protection setting, not anything an +# entry here can turn on. See docs/standards/code-review.md for the review +# standard and GOVERNANCE.md for how ownership is granted. +# +# These rules are evaluated last-match-wins, so a more specific pattern placed +# below `*` overrides it. When an area gets its own owner, add their line below +# the default rather than replacing it, so the default owner stays on every path +# an area owner does not name. GitHub does not request a review for a draft or a +# draft pull request. + +* @cemililik + +# Repository scaffolding and policy files stay with the default owner even after +# an area owner is added, because these paths define the rules the area owners +# work under. +/.github/ @cemililik +/CONTRIBUTING.md @cemililik +/GOVERNANCE.md @cemililik +/MAINTAINERS.md @cemililik +/SECURITY.md @cemililik +/pyproject.toml @cemililik diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index c3756212..acb566f8 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -3,9 +3,9 @@ contact_links: - name: Report a security vulnerability url: https://github.com/HodeTech/ForgeLM/security/advisories/new about: Report vulnerabilities privately, never in a public issue (see SECURITY.md) - - name: Documentation + - name: Browse the documentation url: https://github.com/HodeTech/ForgeLM/tree/main/docs - about: Check the docs and guides before opening an issue + about: Read the docs and guides before opening a documentation issue - name: Discussions (Q&A) url: https://github.com/HodeTech/ForgeLM/discussions about: Ask questions and get help from the community diff --git a/.github/ISSUE_TEMPLATE/documentation.yml b/.github/ISSUE_TEMPLATE/documentation.yml new file mode 100644 index 00000000..752e62c7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/documentation.yml @@ -0,0 +1,52 @@ +name: Documentation +description: Report a documentation error, or propose an improvement to a guide or its Turkish mirror +labels: ["documentation", "needs-triage"] +body: + - type: markdown + attributes: + value: | + Thanks for improving the docs! One correction per issue is easiest to review. + + User-facing documentation is kept as an English page **and** a Turkish + mirror, and the two are updated together — see + [CONTRIBUTING.md](https://github.com/HodeTech/ForgeLM/blob/development/CONTRIBUTING.md). + You do not need to write the Turkish yourself: open the pull request with + the English change and say so, and a maintainer will complete the mirror. + Tell us below which copy you were reading. Documentation pull requests are + welcome directly. + + - type: input + id: page + attributes: + label: Page or file + description: The repository path or the documentation-site page you were reading. + placeholder: docs/guides/quickstart.md — or the Quick start page on the docs site + validations: + required: true + + - type: textarea + id: problem + attributes: + label: What is wrong or missing? + description: What does the page say, and what does the reader need instead? + validations: + required: true + + - type: textarea + id: suggestion + attributes: + label: Suggested wording (optional) + description: If you already know the fix, propose the text. We will tidy the language if needed. + + - type: dropdown + id: language + attributes: + label: Which copy is affected? + description: User-facing docs are maintained as an English page and a Turkish mirror. An issue that needs mirror work is labelled `i18n` when it is triaged. + options: + - English + - Turkish + - Both + - Not sure + validations: + required: true diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 00000000..7d861eb3 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,69 @@ +# Dependabot configuration for ForgeLM. +# +# Dependabot opens a pull request when a dependency's declared range no longer +# covers the newest release. It is the upstream half of the supply-chain story: +# `tools/check_pip_audit.py` and the nightly CVE job catch known advisories, and +# these pull requests make a fix reachable. +# +# `versioning-strategy: widen` matters here. ForgeLM declares bounded ranges +# (`>=floor,` in evidence. Use `https://monitor.example.com:8443/plan` or an `endpoint_env` name (#93). +### Added + +- `CITATION.cff`, so the repository can be cited through GitHub's "Cite this + repository" panel (#964). +- `GOVERNANCE.md` and `MAINTAINERS.md` at the repository root, recording how + decisions are made, how the maintainer roles work, and which areas the current + maintainer owns (#964). +- `.github/dependabot.yml`, so dependency updates arrive as reviewed pull requests + rather than waiting for a manual version check; `.github/CODEOWNERS` to route + review requests without gating a merge; and a Documentation issue template, since + the issue chooser previously offered only a bug and a feature form (#964). + ### Changed - Record monitoring default provenance in `defaulted_fields` and mark literal URL projection with diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 00000000..9459c30c --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,37 @@ +cff-version: 1.2.0 +message: "If you use ForgeLM in your work, please cite it using the metadata below." +title: "ForgeLM" +abstract: >- + Config-driven LLM fine-tuning: validated YAML in, a fine-tuned model and an + audit trail out. Covers SFT, DPO, SimPO, KTO, ORPO and GRPO, with optional + dataset auditing, safety evaluation, evaluation gates, human approval and + EU AI Act compliance artifacts. The same configuration runs from a terminal, + a notebook, or a CI/CD pipeline. +type: software +authors: + - name: "ForgeLM Contributors" +repository-code: "https://github.com/HodeTech/ForgeLM" +url: "https://hodetech.github.io/ForgeLM/" +license: Apache-2.0 +keywords: + - llm + - fine-tuning + - lora + - qlora + - peft + - trl + - dpo + - grpo + - reproducibility + - mlops + - eu-ai-act + - ai-governance +# `version` and `date-released` are intentionally omitted for now: the released +# version is single-sourced in `pyproject.toml` (read at runtime by +# `forgelm/_version.py` via `importlib.metadata`, which carries no literal of its +# own) and the site's displayed version is derived from `CHANGELOG.md` by +# `tools/update_site_version.py --check`. This file is not one of that tool's +# targets, so a version here would be an un-guarded copy that no check keeps in +# sync — exactly what `docs/standards/release.md` exists to prevent. Extending the +# tool to rewrite this file (and adding a Zenodo DOI for citable releases) is +# tracked separately. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bab77ca5..d2150ece 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,6 +20,8 @@ Thanks for your interest in contributing! ForgeLM is an open-source project and 3. Make the change and, for a code fix, add a regression test that reproduces the issue; follow the [testing standard](docs/standards/testing.md). Documentation changes update the English page and its Turkish mirror together. + If you do not write Turkish, open the pull request with the English change and say + so in the description — a maintainer will complete the mirror for you. 4. Run fast checks before pushing (replace the example test module with the one you touched): ```bash diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 00000000..30d303b1 --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,123 @@ +# ForgeLM Governance + +> **How decisions are made in ForgeLM, who makes them, and how you can take on +> more responsibility over time.** For how to contribute, see +> [CONTRIBUTING.md](CONTRIBUTING.md); for behaviour expectations, see +> [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md). + +## Current model + +ForgeLM is **maintainer-led**. The project has a small group of maintainers who +are accountable for the code, the standards and the releases — today, one +(see [MAINTAINERS.md](MAINTAINERS.md)). There is no foundation and no steering +committee; the goal of this document is not to describe a large organisation but +to make the path into one explicit. + +The project's product principles — reliability before features, enterprise +differentiation over feature parity, every capability config-driven and testable +— are recorded in [`docs/roadmap.md`](docs/roadmap.md) and are not restated here. +How decisions get made rests on two rules: + +1. **The standards are the rulebook.** `docs/standards/` holds the engineering + standards, `docs/roadmap/` holds what ships next, and the + [CHANGELOG.md](CHANGELOG.md) records what shipped. A decision that changes + behaviour is written into one of them; a decision that exists only in a chat + is not a decision. +2. **Public reasoning.** Non-trivial proposals, breaking changes and anything + that changes a documented contract are discussed in an **issue** before a + pull request is opened, so the reasoning is reviewable and linkable. + +## Roles + +| Role | What they do | How it is granted | +|---|---|---| +| **Contributor** | Reports bugs, opens pull requests, reviews others' changes, improves docs | Anyone; no appointment | +| **Maintainer** | Sets direction, merges to `development`, cuts releases, holds the standards | A maintainer invites them (see below) | + +Everyone contributing is bound by the +[Code of Conduct](CODE_OF_CONDUCT.md) and asked to follow the +[code-review standard](docs/standards/code-review.md). + +## How decisions are made + +- **Routine changes** — bug fixes, documentation and mirror updates, dependency + bumps — are decided in the pull request. A pull request merges once the + required status checks pass, every review conversation is resolved, and a + maintainer has reviewed it. The exact gate is a branch-protection setting, not + this document. While the project has one maintainer, that maintainer merges + their own pull requests under the same conditions, which is what + [docs/standards/code-review.md](docs/standards/code-review.md) records. +- **Non-trivial changes** — a new config field, a new module, a new optional + dependency, a change to an exit code, the audit-event vocabulary or a + documented guarantee — start as an issue. Work that has no issue may be asked + to open one first, so the design is agreed before code is written. If the + change is larger than one issue, it becomes a roadmap phase under + [docs/roadmap/](docs/roadmap/). +- **Breaking changes** to the YAML schema or the CLI contract require a + deprecation period. Removing a YAML field is a MAJOR change; the release a + removal lands in is named by a single constant, + `DEPRECATION_REMOVAL_VERSION` in `forgelm/config.py`, and the + [release standard](docs/standards/release.md) governs the sequence. +- **Disagreement** is resolved by discussion on the issue. If a consensus does + not form, the maintainer decides and records the reasoning in the issue and, + when it sets a lasting constraint, in + [docs/roadmap/risks-and-decisions.md](docs/roadmap/risks-and-decisions.md). +- **Security reports** do not follow this path. They go through a private + advisory as [SECURITY.md](SECURITY.md) describes. + +## Becoming a maintainer + +There is no exam and no vote — and no promise. A maintainer role is offered when +the project needs a second maintainer and someone has earned it, not +automatically after a number of merged pull requests. The invitation is normally +made when all of the following hold: + +- they have landed a handful of pull requests across more than one area; +- they have reviewed other people's pull requests with useful, specific + feedback, and respected the standards while doing so; +- they have shown judgement on the edge cases — an error path, a documented + contract, a supply-chain or compliance question; +- they are willing to be reachable, and to triage issues and review pull + requests on a predictable cadence rather than only when they feel like it. + +Area ownership follows the same idea: `MAINTAINERS.md` names who owns which area, +and `.github/CODEOWNERS` routes reviews there. The list is kept accurate rather +than flattering, and a maintainer who needs to step back can say so; that is not +held against them. + +## Access and continuity + +GitHub permissions are granted per role, and the smallest permission that does +the job is the default: + +| Role | Repository permission | Why | +|---|---|---| +| Contributor | None — fork and pull request | Every contribution arrives as a pull request, so no repository access is needed | +| Maintainer | *Write* | Can merge to `development`, push branches and manage issues | +| Owner | *Admin* | Also carries branch protection, the tag ruleset and the release workflow's environment approval | + +*Admin* is not the next rung of the contribution ladder — it is the ability to +change the rules the other roles work under, so it stays with the project owner +and moves only on a deliberate decision, with two-factor authentication required +on any account that holds it. + +**Continuity: a known gap, recorded rather than promised.** There is one owner +today, and nothing in the release path is asynchronous — the same single account +holds the tag ruleset and approves the release workflow's `pypi` environment. An +unplanned absence therefore stops merges, releases and security advisories at +once. The fix is a second organisation owner (or an organisation-level owner with +the release environment's approval); it is not scheduled, and this paragraph is +what says so. + +## Releases + +Releases are cut from `main` by a maintainer, following +[docs/standards/release.md](docs/standards/release.md). `development` is the +integration branch and the default branch; changes land there first and reach +`main` through a release pull request. Only maintainers merge to `main` and tag +releases. + +## Changing this document + +This document is a normal file in the repository: propose a change with a pull +request, and explain the reasoning in the pull request description. diff --git a/MAINTAINERS.md b/MAINTAINERS.md new file mode 100644 index 00000000..e9d6171d --- /dev/null +++ b/MAINTAINERS.md @@ -0,0 +1,47 @@ +# ForgeLM Maintainers + +> **Who maintains ForgeLM, what each of them owns, and how to reach them.** +> See [GOVERNANCE.md](GOVERNANCE.md) for how the roles work and how to take one +> on, and [CONTRIBUTING.md](CONTRIBUTING.md) for how to contribute. + +## Current maintainers + +| GitHub | Areas | Reachable for | +|---|---|---| +| [@cemililik](https://github.com/cemililik) | All areas — the repository's `area:` label set (cli, compliance, config, data-audit, dev-tooling, eval-gates, gdpr-approval, ingestion, integrations, model-data, onboarding, operator-cmds, packaging-ci, pipeline, post-training, site, trainer, verify) | Triage, design questions, reviews, releases | + +The project has a single maintainer today. When a co-maintainer takes +responsibility for an area, they are added here and to +[`.github/CODEOWNERS`](.github/CODEOWNERS), which routes reviews for that area to +them. The "Areas" column mirrors the repository's `area:` labels, so an issue +labelled `area: site` and this table name the same area. + +## How to reach a maintainer + +- **A bug or a feature request** — open an + [issue](https://github.com/HodeTech/ForgeLM/issues/new/choose). +- **A question** — start a + [discussion](https://github.com/HodeTech/ForgeLM/discussions); questions are not + filed on the public issue tracker. Asking in public means the answer is + searchable for the next person. +- **A security vulnerability** — never in public. Use the repository's + **Security** tab, as [SECURITY.md](SECURITY.md) describes. +- **Anything covered by the Code of Conduct** — follow the enforcement section of + the [Code of Conduct](CODE_OF_CONDUCT.md#enforcement). + +## Review cadence + +Issues are triaged and pull requests are reviewed on a best-effort basis, with an +initial-reply target of **72 hours** — the same window [SECURITY.md](SECURITY.md) +commits to for a vulnerability report, so the project has one number rather than +two. An unanswered first pull request is the most expensive thing a young project +can ship, so a **first-time contributor**'s pull request is answered first. If a +pull request has not been reviewed for a while, a polite nudge on it is welcome; +it is not rude. + +## Contributors + +This file names maintainers and area owners only; it is not the contributor list. +Everyone whose work is merged appears in the repository's commit history and on +its GitHub contributors page, and non-maintainer contributions are credited in +the release notes. diff --git a/MANIFEST.in b/MANIFEST.in index f1409d9a..1280670a 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1,5 +1,6 @@ include LICENSE include README.md include CHANGELOG.md +include CITATION.cff include config_template.yaml recursive-include configs *.json *.jsonl diff --git a/README.md b/README.md index 19e1b35d..fe9b3584 100644 --- a/README.md +++ b/README.md @@ -192,6 +192,11 @@ feature. Start small; you do not need a GPU to run the unit tests or documentati [Engineering standards](https://github.com/HodeTech/ForgeLM/tree/development/docs/standards). Report vulnerabilities privately using [SECURITY.md](https://github.com/HodeTech/ForgeLM/blob/main/SECURITY.md). +Who decides what, and who owns which area: [GOVERNANCE.md](https://github.com/HodeTech/ForgeLM/blob/development/GOVERNANCE.md) · +[MAINTAINERS.md](https://github.com/HodeTech/ForgeLM/blob/development/MAINTAINERS.md). +To cite ForgeLM, use GitHub's **Cite this repository** button or +[`CITATION.cff`](https://github.com/HodeTech/ForgeLM/blob/development/CITATION.cff). + ## License **Code:** [Apache 2.0](https://github.com/HodeTech/ForgeLM/blob/main/LICENSE). diff --git a/tests/test_check_readme_links.py b/tests/test_check_readme_links.py index 7b9b822d..029963de 100644 --- a/tests/test_check_readme_links.py +++ b/tests/test_check_readme_links.py @@ -59,6 +59,10 @@ def _make_tree(tmp_path: Path, readme: str, contributing: str = "# Contributing\ """Write a minimal repo-shaped tree and return its root.""" (tmp_path / "README.md").write_text(readme, encoding="utf-8") (tmp_path / "CONTRIBUTING.md").write_text(contributing, encoding="utf-8") + # Every surface in `guard._SURFACES` must exist, or the guard reports it as + # missing from the checkout. Keep this in step with that tuple. + for surface in ("CODE_OF_CONDUCT.md", "GOVERNANCE.md", "MAINTAINERS.md", "SECURITY.md"): + (tmp_path / surface).write_text("# stub\n", encoding="utf-8") (tmp_path / "docs" / "guides").mkdir(parents=True, exist_ok=True) (tmp_path / "docs" / "guides" / "real.md").write_text("# real\n", encoding="utf-8") (tmp_path / "LICENSE").write_text("Apache\n", encoding="utf-8") diff --git a/tools/check_readme_links.py b/tools/check_readme_links.py index 1fed1b31..555bb151 100644 --- a/tools/check_readme_links.py +++ b/tools/check_readme_links.py @@ -85,10 +85,21 @@ # ``CONTRIBUTING.md`` is invisible to CI without this entry. _SURFACES: Tuple[Tuple[str, bool], ...] = ( ("README.md", True), + ("CODE_OF_CONDUCT.md", False), ("CONTRIBUTING.md", False), + ("GOVERNANCE.md", False), + ("MAINTAINERS.md", False), + ("SECURITY.md", False), ) -_REPO_BLOB_PREFIX_RE = re.compile(r"^https://github\.com/HodeTech/ForgeLM/(?:blob|tree)/main/(?P[^#?]+)") +# README points at in-repo targets with an absolute `blob/main/…` or +# `blob/development/…` URL. Both are stripped back to a repo-relative path and +# resolved on disk: a file added by an unreleased pull request is linked on +# `development` until the release that puts it on `main`, and a typo in either +# form should still fail. +_REPO_BLOB_PREFIX_RE = re.compile( + r"^https://github\.com/HodeTech/ForgeLM/(?:blob|tree)/(?:main|development)/(?P[^#?]+)" +) # Markdown inline links and images: ``[text](href)`` / ``![alt](href)``. # The href stops at whitespace or the closing paren so that titles diff --git a/tools/check_source_path_refs.py b/tools/check_source_path_refs.py index 4f9094d0..83db5b70 100644 --- a/tools/check_source_path_refs.py +++ b/tools/check_source_path_refs.py @@ -306,7 +306,18 @@ } # Files scanned outside the directory globs below. -_TOP_LEVEL_SURFACES: frozenset[str] = frozenset({"CLAUDE.md", "AGENTS.md", "CONTRIBUTING.md", "README.md"}) +_TOP_LEVEL_SURFACES: frozenset[str] = frozenset( + { + "AGENTS.md", + "CLAUDE.md", + "CODE_OF_CONDUCT.md", + "CONTRIBUTING.md", + "GOVERNANCE.md", + "MAINTAINERS.md", + "README.md", + "SECURITY.md", + } +) @dataclass(frozen=True) From 0766e9404bd088ce419c9b8f183ba143e8efcc77 Mon Sep 17 00:00:00 2001 From: Venius24 Date: Sun, 18 Oct 2026 01:46:57 +0500 Subject: [PATCH 04/22] docs(usermanuals): name the real KTO columns on the alignment overview page (#972) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes #965 The KTO bullet on the Alignment overview page named the fields `(prompt, response, good?)`; the loader requires `prompt`, `completion` and a boolean `label`. Both the English page and the Turkish mirror (same line, same structure, only the field names and the true/false explanation changed) now say `(prompt, completion, label)`. A `### Fixed` entry is added under `[Unreleased]` in `CHANGELOG.md`. Checks run, all passing with `--strict`: `check_bilingual_parity`, `check_bilingual_code_blocks`, `check_anchor_resolution`, `check_tr_links_prefer_mirror`, `check_usermanual_self_contained`. Docs only, no code changed. The Turkish wording is mine and could use a native read. ## Summary by Sourcery Correct the documented KTO dataset fields to match the loader’s expected prompt, completion, and label schema. Bug Fixes: - Correct the KTO field names and boolean label guidance on the English and Turkish alignment overview pages. Documentation: - Document the KTO alignment overview correction in the unreleased changelog. Signed-off-by: Venius24 Co-authored-by: Claude Sonnet 5.5 --- CHANGELOG.md | 3 +++ docs/usermanuals/en/concepts/alignment-overview.md | 2 +- docs/usermanuals/tr/concepts/alignment-overview.md | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c5434975..f093814d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,9 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the + fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean + `label` (#965). - Make the site's code comments readable: `--code-comment` was `#6a6f80`, 3.8:1 on the dark code surface, below the 4.5:1 AA floor, and it colours code comments, terminal and code-block title bars, language tags, copy buttons and the wizard's YAML pane header and comment lines. It is now `#8f96a8`, diff --git a/docs/usermanuals/en/concepts/alignment-overview.md b/docs/usermanuals/en/concepts/alignment-overview.md index 7c5d123a..fc3bf89f 100644 --- a/docs/usermanuals/en/concepts/alignment-overview.md +++ b/docs/usermanuals/en/concepts/alignment-overview.md @@ -53,7 +53,7 @@ After SFT, the model produces well-formatted outputs but quality varies — some The differences: - **DPO** — learns from `(chosen, rejected)` pairs against a reference model. Best-studied, most stable. - **SimPO** — same as DPO but skips the reference model. Lower memory, slightly less stable. -- **KTO** — learns from binary `(prompt, response, good?)` signals. Use when you only have thumbs-up/down feedback. +- **KTO** — learns from binary `(prompt, completion, label)` rows, where `label` is `true` for a desirable response and `false` for an undesirable one. Use when you only have thumbs-up/down feedback. - **ORPO** — combines SFT and preference loss in a single pass. Faster wall-clock; less flexible than separate stages. ### Problem 3: Reasoning diff --git a/docs/usermanuals/tr/concepts/alignment-overview.md b/docs/usermanuals/tr/concepts/alignment-overview.md index 5afd56fa..50011b97 100644 --- a/docs/usermanuals/tr/concepts/alignment-overview.md +++ b/docs/usermanuals/tr/concepts/alignment-overview.md @@ -53,7 +53,7 @@ SFT'den sonra model iyi formatta çıktı üretir ama kalite değişken — baze Farklar: - **DPO** — referans modele karşı `(chosen, rejected)` çiftlerinden öğrenir. En çok çalışılmış, en kararlı. - **SimPO** — DPO ile aynı ama referans modeli atlar. Daha az bellek, biraz daha az kararlılık. -- **KTO** — ikili `(prompt, response, iyi mi?)` sinyallerinden öğrenir. Sadece thumbs-up/down geri bildiriminiz varsa kullanın. +- **KTO** — ikili `(prompt, completion, label)` satırlarından öğrenir; `label`, istenen yanıt için `true`, istenmeyen yanıt için `false` olur. Sadece thumbs-up/down geri bildiriminiz varsa kullanın. - **ORPO** — SFT ve tercih kaybını tek geçişte birleştirir. Daha hızlı wall-clock; ayrı aşamalardan daha az esneklik. ### Problem 3: Akıl yürütme From 6c55900e76be10d85fa50db24ee4afdb635d4e5a Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 00:41:16 +0300 Subject: [PATCH 05/22] docs: send newcomers to a Front door view and state that closing is not triage (#977) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Sends first-time contributors to a "Front door" view of the project plan, and records the intake rule in one sentence in the three places that state it. No behaviour change. ## Related issues None. This is a maintainer task from the contributor-intake plan, not a backlog finding. It clarifies the 2026-10-09 rule ("every identified or planned, not-yet-done item is an issue plus a project card"); it does not change it. ## Changes - **`README.md`** — the Contribute section's first step now starts at the [Front door](https://github.com/orgs/HodeTech/projects/5/views/16) view (open, unassigned `good first issue`s with their size and area), keeps the label search as the alternative, and points to the project plan for the rest. - **`CONTRIBUTING.md`** — lists *Front door* among the project's views and first among the "Useful starting points", and adds the rule sentence to "Planned work lives here too". - **`CLAUDE.md`, `AGENTS.md`** — a bullet **Closing is not triage** in "Branches, issues and the project board", same wording in both: *Every item is an issue; which issues appear on the front door is decided by labels and views, never by closing them.* - **`CHANGELOG.md`** — one `[Unreleased]` → Changed entry. The view itself (project 5, view 16) is a project setting, not a file. Its filter is `label:"good first issue" no:assignee is:open -label:security`, so it follows the label rather than a list of issue numbers. Today it shows 13 issues. ## Type - [x] Documentation ## Testing - The full `CLAUDE.md` gauntlet, run from a clean worktree of this branch: exit 0 (including `pytest tests/`, `ruff`, `mypy --strict`, `--dry-run`, and every `tools/check_*.py` guard). - `check_readme_links.py --strict`: the new view URL is an absolute https link outside the repository, so it is not resolved on disk. - `bandit` still reports 11 medium-severity warnings; they are the existing ones and the helper treats them as warnings. ### Testing checklist - [x] `pytest tests/` passes - [x] `ruff check .` passes - [x] `ruff format --check .` passes - [ ] New tests added for changed code — not applicable, documentation only - [x] `python -m forgelm --config config_template.yaml --dry-run` works ## Checklist - [x] Documentation updated — no Turkish mirror applies: `README.md`, `CONTRIBUTING.md`, `CLAUDE.md` and `AGENTS.md` are English-only - [x] CHANGELOG entry under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Guide newcomers to the Front door project view and document that closing issues is not a triage mechanism. Enhancements: - Direct first-time contributors to a Front door project view and clarify that issue visibility is managed through labels and views rather than closing issues. Documentation: - Update contributor guidance and repository maintainer instructions with the Front door view and the project intake rule. Chores: - Record the documentation changes in the unreleased changelog. ## Summary by CodeRabbit * **Documentation** * Updated contribution guidance to direct first-time contributors to open, unassigned good first issues in the project’s *Front door* view, with size and area details. * Clarified how planned and unplanned work is tracked, and where to find other planned work. * Explained that front-door visibility is determined by labels and views, independently of whether an issue is open or closed. Co-authored-by: Claude Sonnet 5.5 --- AGENTS.md | 1 + CHANGELOG.md | 4 ++++ CLAUDE.md | 1 + CONTRIBUTING.md | 5 +++-- README.md | 5 +++-- 5 files changed, 12 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ce249c7d..fe5ac99a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -386,6 +386,7 @@ Work on ForgeLM is planned and tracked on GitHub, not in local notes: acceptance criteria) or tell the user so it can be filed. The exception is a security vulnerability — never a public issue; it goes to a private advisory as [SECURITY.md](SECURITY.md) describes. +- **Closing is not triage.** Every item is an issue; which issues appear on the front door is decided by labels and views, never by closing them. - **Name what you resolved.** For a grouped issue, list in the pull request the finding IDs your change resolves, so each can be ticked. diff --git a/CHANGELOG.md b/CHANGELOG.md index f093814d..904500f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,10 @@ All notable changes to ForgeLM are documented here. - Require history-preserving merge commits for `main` promotions and release synchronization; retain squash as the default for contributor PRs into `development`, and disable rebase-and-merge. Align the contributor, code-review, and release guidance with the repository settings. +- Point first-time contributors at a *Front door* view of the project plan (open, unassigned + `good first issue`s with their size and area) from the README and CONTRIBUTING, and record in + `CONTRIBUTING.md`, `CLAUDE.md` and `AGENTS.md` that the front door is chosen with labels and + views, never by closing issues. ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index 518a3ea4..444e20b9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -386,6 +386,7 @@ Work on ForgeLM is planned and tracked on GitHub, not in local notes: acceptance criteria) or tell the user so it can be filed. The exception is a security vulnerability — never a public issue; it goes to a private advisory as [SECURITY.md](SECURITY.md) describes. +- **Closing is not triage.** Every item is an issue; which issues appear on the front door is decided by labels and views, never by closing them. - **Name what you resolved.** For a grouped issue, list in the pull request the finding IDs your change resolves, so each can be ticked. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d2150ece..b4d06bd7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -68,11 +68,11 @@ Thanks for your interest in contributing! ForgeLM is an open-source project and Work is tracked in two places that play different roles: - **Issues are where the work happens.** Each issue is one unit of work: it describes the problem, links the code, and lists acceptance criteria. Discussion, claiming and pull-request links all happen on the issue. -- **The [ForgeLM Project Plan](https://github.com/orgs/HodeTech/projects/5) is where the work is planned.** It shows every issue with its priority, release target and iteration, in several views: *Backlog* (table), *Board* (by status), *Current iteration* and *Roadmap* (timeline), plus filtered views such as *Small tasks*, *Epics*, *Docs & TR mirror* and *Roadmap — Phase 16 & deferred*. Use it to choose what to work on; you never need to edit it — maintainers keep it current, and a card moves automatically when its issue is closed. +- **The [ForgeLM Project Plan](https://github.com/orgs/HodeTech/projects/5) is where the work is planned.** It shows every issue with its priority, release target and iteration, in several views: *Backlog* (table), *Board* (by status), *Current iteration* and *Roadmap* (timeline), plus filtered views such as *Front door*, *Small tasks*, *Epics*, *Docs & TR mirror* and *Roadmap — Phase 16 & deferred*. Use it to choose what to work on; you never need to edit it — maintainers keep it current, and a card moves automatically when its issue is closed. ### Planned work lives here too -Every piece of planned or deferred work has an issue and a project card, not only the defects: the remaining steps of [Phase 16](docs/roadmap/phase-16-trust-surface-hardening.md) are sub-issues of [#903](https://github.com/HodeTech/ForgeLM/issues/903), and the deferred items recorded in [risks-and-decisions.md](docs/roadmap/risks-and-decisions.md) are sub-issues of [#929](https://github.com/HodeTech/ForgeLM/issues/929). Each Phase 16 step, and each deferred row that has an issue, links that issue directly from the roadmap entry. New issues land in the project's *Intake — unplanned* view until a maintainer plans them. If you notice work that your pull request is not going to do, open an issue for it instead of leaving a `TODO` or a roadmap line. +Every piece of planned or deferred work has an issue and a project card, not only the defects: the remaining steps of [Phase 16](docs/roadmap/phase-16-trust-surface-hardening.md) are sub-issues of [#903](https://github.com/HodeTech/ForgeLM/issues/903), and the deferred items recorded in [risks-and-decisions.md](docs/roadmap/risks-and-decisions.md) are sub-issues of [#929](https://github.com/HodeTech/ForgeLM/issues/929). Each Phase 16 step, and each deferred row that has an issue, links that issue directly from the roadmap entry. New issues land in the project's *Intake — unplanned* view until a maintainer plans them. If you notice work that your pull request is not going to do, open an issue for it instead of leaving a `TODO` or a roadmap line. Every item is an issue; which issues appear on the front door is decided by labels and views, never by closing them. ### How the backlog is organised @@ -99,6 +99,7 @@ In the project, the **Wave**, **Severity**, **Theme**, **Kind**, **Area**, **Siz Useful starting points: +- The project's [*Front door* view](https://github.com/orgs/HodeTech/projects/5/views/16) — open, unassigned `good first issue`s with their size and area, the place to start on a first contribution - [Open, unassigned wave 0–1 issues](https://github.com/HodeTech/ForgeLM/issues?q=is%3Aissue+is%3Aopen+no%3Aassignee+label%3A%22wave%3A+0%22%2C%22wave%3A+1%22) — the highest-priority work - [Documentation issues](https://github.com/HodeTech/ForgeLM/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation+no%3Aassignee) — often a good first contribution - The project's *Backlog* view, filtered by an **Area** you know or by **Size** `XS`/`S` diff --git a/README.md b/README.md index fe9b3584..e19d295a 100644 --- a/README.md +++ b/README.md @@ -180,8 +180,9 @@ Help make fine-tuning easier to reproduce and review. You can improve a guide, translate an example, reproduce a bug, add a regression test, or work on a planned feature. Start small; you do not need a GPU to run the unit tests or documentation checks. -1. Find an [unassigned good first issue](https://github.com/HodeTech/ForgeLM/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20issue%22%20no%3Aassignee) - or browse the [project plan](https://github.com/orgs/HodeTech/projects/5). +1. Start at the [Front door](https://github.com/orgs/HodeTech/projects/5/views/16): the open, unassigned `good first issue`s + with their size and area. You can also [search by label](https://github.com/HodeTech/ForgeLM/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20issue%22%20no%3Aassignee). + The rest of the planned work is in the [project plan](https://github.com/orgs/HodeTech/projects/5). 2. Follow [your first pull request](https://github.com/HodeTech/ForgeLM/blob/development/CONTRIBUTING.md#your-first-pull-request) to set up the environment and run the relevant checks. 3. Open a PR against **`development`** and link its issue. Keep English/Turkish From 56b4f0d599a07f0d477104c78d5079db1a6cca31 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 09:47:01 +0300 Subject: [PATCH 06/22] feat(ci): let a contributor claim a good first issue with /assign (#978) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary A contributor claims a `good first issue` by commenting `/assign` and gives it back with `/unassign`, so a maintainer no longer has to assign by hand on the same day. ## Related issues None: this is a maintainer task from the contributor-intake plan, not a backlog finding. It also follows the direction of #465 (the one new `uses:` is pinned to a full commit SHA). ## Changes - **`.github/workflows/assign-on-comment.yml`** — `issue_comment` / `created` only. The job `if:` skips pull-request comments, bots and any comment that does not start with a command. Token: `contents: read` + `issues: write`. One action, `actions/checkout` pinned to the v7.0.1 commit, taking one file with no stored credentials. No `${{ }}` in any `run:` line, and **no `concurrency:` group**: GitHub cancels a *pending* run when a newer one joins the group, which would drop a claim silently. - **`tools/assign_on_comment.py`** — the rules, standard library only, every subprocess call a list with no shell. The comment text is never executed and never printed. - **`tests/test_assign_on_comment.py`** — 97 tests: every rule, the failure paths, that no comment text reaches the log, the `gh` wrapper, the workflow file's security properties, and that the tool's maintainer list matches `MAINTAINERS.md`. - **`CONTRIBUTING.md`** — the commands, the limit, and who can reassign. `CHANGELOG.md` — one entry. Test-module count 130 → 131 in the five places that state it. ### Behaviour | Situation | Result | |---|---| | unassigned `good first issue`, contributor `/assign` | assigned, reply with the PR rules | | **already assigned to anyone** | **refused, nothing changed**, no assignment call is made; only a maintainer reassigns, in the sidebar | | `/assign @someone` | refused: commands act on the person who writes them | | contributor already holds 2 open assignments | refused; the reply tells them to `/unassign` one or to mention a maintainer (in code formatting, so it does not ping) | | `help wanted` without `good first issue`, `security`, `needs-triage`, `deferred`, `epic`, `phase: 16`, closed issue | a maintainer's decision; the bot says so and mentions the maintainer | | `/unassign` | removes only the person who wrote it | | an API call fails, or GitHub does not confirm the assignment | the contributor gets "a maintainer will assign this by hand" and the job fails (red) | Maintainers (the list in `MAINTAINERS.md`, enforced by a test) are exempt from the limit. GitHub's `author_association` is deliberately not used: it also calls a read-only organisation member a `MEMBER`. ## Type - [x] CI / infrastructure - [x] Documentation - [x] Test coverage ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. - Eight deliberate mutations of the rules (assigned issue unprotected, second read removed, limit off by one, `/unassign` removing anyone, bots not ignored, unconfirmed assignment ignored, excluded label ignored, `@name` not refused) each fail at least one test. - A read-only replay against live issue data (all writes blocked) of 13 realistic events, including the three real claims, a claim on an assigned issue, the limit, an epic, a `help wanted` issue and a read-only organisation member. Every outcome was the intended one. It also found that the assignment count was fetched before it was needed; it is now fetched only when the answer depends on it. **Not verified, and not verifiable before merge:** - `issue_comment` workflows run only from the default branch, so this cannot be exercised from this branch. - GitHub documents that a user who has commented can be assigned, but not whether the REST call made with `GITHUB_TOKEN` accepts a non-collaborator. The maintainer's own account cannot prove it. If it is refused, the contributor gets the "assigned by hand" reply and the job fails; nothing is half-done. - Whether `issue_comment` runs from a first-time contributor wait for approval (the repository setting is `first_time_contributors`; it concerns fork pull requests, so none is expected). **Rollout.** After merge, try `/assign` and `/unassign` on a throwaway issue, then watch the first real claim. To switch the feature off at once: Actions > "Assign on comment" > "..." > Disable workflow. Existing assignments are not touched. The 16 "Where to start" comments are updated to mention `/assign` only after that check passes. ## Checklist - [x] Documentation updated — `CONTRIBUTING.md` is English-only; no Turkish mirror applies - [x] CHANGELOG entry under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Automate contributor issue claiming while preserving maintainer control over existing assignments and exceptional cases. New Features: - Enable contributors to claim eligible unassigned issues with `/assign` and release their own assignments with `/unassign`. Enhancements: - Add guarded assignment rules that preserve existing assignments, enforce contributor limits, handle concurrent claims, and defer excluded issues or reassignment decisions to maintainers. CI: - Add a least-privilege GitHub Actions workflow for processing issue comments with pinned dependencies and protected command handling. Documentation: - Document the new assignment commands, eligibility rules, assignment limit, and maintainer reassignment policy. Tests: - Add comprehensive coverage for command behavior, assignment rules, failure handling, security properties, workflow configuration, and documentation consistency. Chores: - Record the contributor assignment automation in the changelog and update repository test-module counts. ## Summary by CodeRabbit * **New Features** * Contributors can claim eligible, unassigned issues with `/assign` and release their own assignment with `/unassign`. * Claims are limited to two open issues per contributor. Issues already assigned, restricted by labels, or otherwise ineligible remain unassigned; maintainers handle exceptions and reassignment. * **Documentation** * Updated contribution guidance to explain issue claiming, assignment limits, and when to contact maintainers. --------- Co-authored-by: Claude Sonnet 5.5 --- .github/workflows/assign-on-comment.yml | 47 ++ AGENTS.md | 2 +- CHANGELOG.md | 5 + CLAUDE.md | 2 +- CONTRIBUTING.md | 10 +- docs/reference/architecture-tr.md | 2 +- docs/reference/architecture.md | 2 +- tests/test_assign_on_comment.py | 833 ++++++++++++++++++++++++ tools/assign_on_comment.py | 475 ++++++++++++++ 9 files changed, 1371 insertions(+), 7 deletions(-) create mode 100644 .github/workflows/assign-on-comment.yml create mode 100644 tests/test_assign_on_comment.py create mode 100644 tools/assign_on_comment.py diff --git a/.github/workflows/assign-on-comment.yml b/.github/workflows/assign-on-comment.yml new file mode 100644 index 00000000..d5f48e84 --- /dev/null +++ b/.github/workflows/assign-on-comment.yml @@ -0,0 +1,47 @@ +name: Assign on comment + +# A contributor claims an issue by commenting `/assign` and gives it back with `/unassign`. +# The rules live in tools/assign_on_comment.py (documented in CONTRIBUTING.md, "Working on an issue"). +# +# This is the one workflow that runs on text written by anyone, with permission to write to issues. +# Keep it that way on purpose; tests/test_assign_on_comment.py pins each of these properties: +# - the trigger is `issue_comment` / `created` only (an edited comment cannot start a run); +# - the job `if:` reads event data only, never a shell: no `${{ }}` appears in a `run:` line, +# and the script reads the event file itself; +# - the token holds `contents: read` and `issues: write` and nothing else; +# - the checkout takes the default branch (what `issue_comment` runs on), one file, no stored credentials, +# and is pinned to a full commit SHA; +# - there is no `concurrency:` group: GitHub cancels a *pending* run when a newer one joins the group, which +# would drop a claim without a word. Instead the script decides again on a fresh read right before it +# writes, and after writing steps back only itself if a simultaneous claim collided (a maintainer wins; +# between two contributors the earlier login stays; a claim that would exceed the limit is released). +# +# Maintainers: to switch the feature off at once, open Actions > "Assign on comment" > the "..." menu > +# "Disable workflow". Assignments already made stay as they are. Changing who an issue is assigned to is a +# maintainer's decision, made in the issue's sidebar; nothing here ever overrides it. + +on: + issue_comment: + types: [created] + +permissions: {} + +jobs: + assign: + # Event data only. Skips pull-request comments and bots, and runs only for a comment that starts with a command. + if: ${{ !github.event.issue.pull_request && github.event.comment.user.type != 'Bot' && (startsWith(github.event.comment.body, '/assign') || startsWith(github.event.comment.body, '/unassign')) }} + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read # the checkout reads one script from the default branch + issues: write # add or remove the assignee, and post the reply + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + sparse-checkout: tools/assign_on_comment.py + sparse-checkout-cone-mode: false + - name: Assign or release the issue + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: python3 tools/assign_on_comment.py diff --git a/AGENTS.md b/AGENTS.md index fe5ac99a..1b8fd74a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,7 +85,7 @@ ForgeLM/ │ # export, inference, results, utils, __main__, │ # _pypdf_normalise, _script_sanity, │ # _strip_pattern -├── tests/ # 130 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current) +├── tests/ # 131 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current) ├── tools/ # CI guards: check_anchor_resolution, │ # check_bilingual_parity, check_cli_help_consistency, │ # check_field_descriptions, check_no_analysis_refs, diff --git a/CHANGELOG.md b/CHANGELOG.md index 904500f6..0d44dd02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,11 @@ All notable changes to ForgeLM are documented here. - `CITATION.cff`, so the repository can be cited through GitHub's "Cite this repository" panel (#964). +- A contributor can claim a `good first issue` by commenting `/assign` and give it + back with `/unassign`: the `Assign on comment` workflow (`tools/assign_on_comment.py`) assigns the commenter + when nobody holds the issue, refuses an issue that already has an assignee without changing it, allows at most 2 + open assignments per contributor (maintainers are exempt) and leaves `security`, `needs-triage`, `deferred`, + epic and Phase 16 issues to a maintainer. Reassigning stays a maintainer's decision. - `GOVERNANCE.md` and `MAINTAINERS.md` at the repository root, recording how decisions are made, how the maintainer roles work, and which areas the current maintainer owns (#964). diff --git a/CLAUDE.md b/CLAUDE.md index 444e20b9..b934017e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,7 +85,7 @@ ForgeLM/ │ # export, inference, results, utils, __main__, │ # _pypdf_normalise, _script_sanity, │ # _strip_pattern -├── tests/ # 130 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current) +├── tests/ # 131 test modules; count grows over time (run `ls tests/test_*.py | wc -l` for current) ├── tools/ # CI guards: check_anchor_resolution, │ # check_bilingual_parity, check_cli_help_consistency, │ # check_field_descriptions, check_no_analysis_refs, diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b4d06bd7..ba4fa056 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,7 +5,7 @@ Thanks for your interest in contributing! ForgeLM is an open-source project and ## Your first pull request 1. Pick an open, unassigned [`good first issue`][good-first-issue]. - Comment to claim it and confirm the problem on `development`; see [Working on an issue](#working-on-an-issue). + Comment `/assign` to claim it and confirm the problem on `development`; see [Working on an issue](#working-on-an-issue). 2. [Fork and clone](#1-fork--clone), then [create a branch from `development`](#3-create-a-branch). Use Python **3.10, 3.11, 3.12 or 3.13** — CI tests all four versions. Create a virtual environment with that interpreter (replace `python3.13` below if you selected another supported version), @@ -106,7 +106,11 @@ Useful starting points: ### Working on an issue -1. **Claim it first.** Comment on the issue that you would like to work on it, so a maintainer can assign it to you and nobody duplicates the work. Skip issues labelled `needs-triage`, and check the recorded condition of an issue labelled `deferred` before you start. +1. **Claim it first.** Start a comment on the issue with `/assign` and, once GitHub confirms it, the issue is assigned to you, so nobody duplicates the work. If the automation cannot complete, its reply says so and a maintainer assigns the issue by hand. This works on an open issue labelled `good first issue` that nobody holds yet. A contributor can hold at most 2 open issues at a time; the maintainers listed in [MAINTAINERS.md](MAINTAINERS.md) are exempt. Comment `/unassign` to give one back. A few rules keep it predictable: + - It is first come, first served. A later `/assign` never changes an assignment that exists, whoever wrote it and however it was made. Only a maintainer can reassign an issue, in the issue's sidebar, and a maintainer's decision always stands. + - `/assign` and `/unassign` act on the person who writes them, and the command must be the first thing in the comment; they cannot name someone else. If two people claim an issue in the same second, the maintainer's claim wins, and between two contributors one claim is released again and its author is told so. + - Anything the automation does not cover stays a maintainer's call: an issue labelled `help wanted` without `good first issue` (grouped sweeps are taken one finding ID at a time, see below), an issue labelled `needs-triage`, `deferred`, `security`, `epic` or `phase: 16`, and a request for more than 2 open issues. Comment on the issue, say what you would take on and mention a maintainer ([MAINTAINERS.md](MAINTAINERS.md)). A plain comment such as "I would like to work on this" is read by a maintainer too, who assigns it by hand. + - Skip issues labelled `needs-triage`, and check the recorded condition of an issue labelled `deferred` before you start. 2. **Re-check against `development`.** Issues from the review link code at the commit they were found on; the links stay valid, but line numbers may have moved. Confirm the problem still exists on `development` before fixing it. 3. **Follow the acceptance criteria** in the issue. In particular, a code fix carries a regression test that reproduces the described failure, and a documentation fix updates the English page and its Turkish mirror (`-tr.md`, or the `tr/` tree for user manuals) in the same pull request. 4. **Reference the issue in your pull request.** Use `Fixes #123` when the PR resolves the whole issue, and `Refs #123` when it resolves only part of it. @@ -250,7 +254,7 @@ ForgeLM is a single-package layout: a mix of single-file modules and five focused sub-packages (`forgelm/cli/` post-Phase-15 split, `forgelm/data_audit/` post-Phase-14 split, `forgelm/wizard/` from Phase 22, `forgelm/safety/` from the post-v0.9.1 split and `forgelm/verify/` from -Phase 16 S1) under `forgelm/`, 130 test files +Phase 16 S1) under `forgelm/`, 131 test files under `tests/` (collected-test count grows over time — run `pytest --collect-only -q` for current), plus `configs/`, `docs/`, `tools/` (CI guards), and `notebooks/`. For the authoritative module-by-module map diff --git a/docs/reference/architecture-tr.md b/docs/reference/architecture-tr.md index 634033f7..5da4fea3 100644 --- a/docs/reference/architecture-tr.md +++ b/docs/reference/architecture-tr.md @@ -96,7 +96,7 @@ ForgeLM/ │ ├── deepspeed/ # ZeRO-2, ZeRO-3, ZeRO-3+Offload ön ayarları │ └── safety_prompts/ # Yerleşik adversarial prompt kütüphanesi (140 prompt, 6 kategori) ├── notebooks/ # 10 Colab-uyumlu Jupyter notebook -├── tests/ # 130 test modülü +├── tests/ # 131 test modülü ├── tools/ # CI guard'ları: bilingual_parity, anchor_resolution, │ # cli_help_consistency, yaml_snippets, │ # audit_event_catalog, library_api_doc, diff --git a/docs/reference/architecture.md b/docs/reference/architecture.md index daeaeb45..735b3f2b 100644 --- a/docs/reference/architecture.md +++ b/docs/reference/architecture.md @@ -96,7 +96,7 @@ ForgeLM/ │ ├── deepspeed/ # ZeRO-2, ZeRO-3, ZeRO-3+Offload presets │ └── safety_prompts/ # Built-in adversarial prompt library (140 prompts, 6 categories) ├── notebooks/ # 10 Colab-ready Jupyter notebooks -├── tests/ # 130 test modules +├── tests/ # 131 test modules ├── tools/ # CI guards: bilingual_parity, anchor_resolution, │ # cli_help_consistency, yaml_snippets, │ # audit_event_catalog, library_api_doc, diff --git a/tests/test_assign_on_comment.py b/tests/test_assign_on_comment.py new file mode 100644 index 00000000..492d1454 --- /dev/null +++ b/tests/test_assign_on_comment.py @@ -0,0 +1,833 @@ +"""Tests for tools/assign_on_comment.py and .github/workflows/assign-on-comment.yml. + +The tool decides who may claim an issue from text anyone can write, with a token that can write to issues. +These tests pin the rules (CONTRIBUTING.md, "Working on an issue"), the invariant that a later ``/assign`` +never changes an existing assignment, the failure paths, that no comment text reaches the log, and the +security posture of the workflow file itself, since no other guard reads it. +""" + +from __future__ import annotations + +import subprocess +import sys +from pathlib import Path +from typing import Any, Callable, Dict, List, Optional + +import pytest +import yaml + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "tools")) + +import assign_on_comment as tool # noqa: E402 + +REPO = "HodeTech/ForgeLM" +GFI = ["good first issue"] + + +@pytest.fixture(autouse=True) +def _no_retry_delay(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(tool.time, "sleep", lambda seconds: None) + + +class FakeGitHub: + """A small stand-in for the REST API: one issue, the issues assigned to each login, and a call log.""" + + def __init__( + self, + labels: Optional[List[str]] = None, + assignees: Optional[List[str]] = None, + state: str = "open", + held: Optional[Dict[str, int]] = None, + ) -> None: + self.labels = list(GFI if labels is None else labels) + self.assignees = list(assignees or []) + self.state = state + self.held = dict(held or {}) # login -> number of other open issues assigned to them + self.calls: List[tuple] = [] + self.comments: List[str] = [] + self.fail: Dict[tuple, str] = {} # (method, endpoint prefix) -> reason + self.drop_assignment = False # accept the POST but do not apply it (an assignee GitHub refuses) + self.after_first_read: Optional[Callable[[], None]] = None + self.after_assign: Optional[Callable[[], None]] = None # runs right after our POST landed + self.human_comments: List[str] = [] # comments by people (the bot's own are in `comments`) + self.assigned_pages: Optional[List[List[Dict[str, Any]]]] = None # overrides the count answer, page by page + self.refuse_delete = False # accept the DELETE but keep the assignee + self._reads = 0 + + def _issue(self) -> Dict[str, Any]: + return { + "state": self.state, + "labels": [{"name": n} for n in self.labels], + "assignees": [{"login": a} for a in self.assignees], + } + + def __call__(self, method: str, endpoint: str, payload: Any = None) -> Any: + self.calls.append((method, endpoint, payload)) + for (m, prefix), reason in self.fail.items(): + if m == method and endpoint.startswith(prefix): + raise tool.GhError(f"{method} {endpoint} failed: {reason}") + if method == "GET" and endpoint == f"repos/{REPO}/issues/7": + self._reads += 1 + answer = self._issue() + if self._reads == 1 and self.after_first_read: + self.after_first_read() + return answer + if method == "GET" and "?assignee=" in endpoint: + login = endpoint.split("assignee=")[1].split("&")[0] + if self.assigned_pages is not None: + page = int(endpoint.rsplit("page=", 1)[1]) + return self.assigned_pages[page - 1] if page <= len(self.assigned_pages) else [] + return [{"number": 100 + i} for i in range(self.held.get(login, 0))] + [ + {"number": 999, "pull_request": {}} # a PR assigned to them must not count + ] + if method == "GET" and "/comments?" in endpoint: + return [{"user": {"type": "Bot"}, "body": b} for b in self.comments] + [ + {"user": {"type": "User"}, "body": b} for b in self.human_comments + ] + if method == "POST" and endpoint.endswith("/assignees"): + if not self.drop_assignment: + self.assignees += [a for a in payload["assignees"] if a not in self.assignees] + if self.after_assign: + self.after_assign() + return self._issue() + if method == "DELETE" and endpoint.endswith("/assignees"): + if not self.refuse_delete: + self.assignees = [a for a in self.assignees if a not in payload["assignees"]] + return self._issue() + if method == "POST" and endpoint.endswith("/comments"): + self.comments.append(payload["body"]) + return {} + raise AssertionError(f"unexpected call {method} {endpoint}") + + def writes(self) -> List[tuple]: + """Every call that changes an assignment (comments are not assignments).""" + return [c for c in self.calls if c[0] in ("POST", "DELETE") and c[1].endswith("/assignees")] + + +def event( + body: str, + login: str = "newcomer", + user_type: str = "User", + number: int = 7, + pull_request: bool = False, +) -> Dict[str, Any]: + issue: Dict[str, Any] = {"number": number} + if pull_request: + issue["pull_request"] = {"url": "x"} + return { + "issue": issue, + "comment": {"body": body, "user": {"login": login, "type": user_type}}, + } + + +def run(gh: FakeGitHub, body: str = "/assign", **kwargs: Any) -> int: + return tool.run(event(body, **kwargs), REPO, gh) + + +# --- command parsing ------------------------------------------------------------------------------------------ + + +@pytest.mark.parametrize( + "body, expected", + [ + ("/assign", "/assign"), + ("/ASSIGN", "/assign"), + (" /assign ", None), # leading whitespace: the workflow's startsWith gate would not let it through either + ("/assign I would like to fix this", "/assign"), + ("\n\n/unassign", None), + ("/unassign please", "/unassign"), + ("/assigned", None), + ("/assigning", None), + ("assign", None), + ("I think /assign works", None), + ("> /assign", None), + ("thanks!\n/assign", None), + ("", None), + (" \n ", None), + ], +) +def test_parse_command_reads_only_the_first_word_at_the_very_start(body: str, expected: Optional[str]) -> None: + assert tool.parse_command(body) == expected + + +def test_a_second_command_on_a_later_line_is_not_read() -> None: + gh = FakeGitHub() + assert run(gh, "/assign\n/unassign") == 0 + assert gh.assignees == ["newcomer"] + + +@pytest.mark.parametrize( + "body, expected", + [("/assign", False), ("/assign please", False), ("/assign @someone", True), ("/unassign @a b", True)], +) +def test_names_someone_else_looks_for_an_at_name_on_the_command_line(body: str, expected: bool) -> None: + assert tool.names_someone_else(body) is expected + + +@pytest.mark.parametrize( + "login, ok", + [("alice", True), ("a-b-1", True), ("A", True), ("", False), ("a b", False), ("a/b", False), ("é", False), + ("x" * 40, False), ("x" * 39, True), ("a;rm", False), ("a\nb", False)], +) # fmt: skip +def test_valid_login(login: str, ok: bool) -> None: + assert tool.valid_login(login) is ok + + +# --- the claim itself ----------------------------------------------------------------------------------------- + + +def test_a_contributor_claims_an_unassigned_good_first_issue() -> None: + gh = FakeGitHub() + assert run(gh) == 0 + assert gh.assignees == ["newcomer"] + assert len(gh.comments) == 1 + reply = gh.comments[0] + assert "Assigned to @newcomer" in reply and "Fixes #7" in reply and "`/unassign`" in reply + assert "CONTRIBUTING.md#working-on-an-issue" in reply + + +def test_help_wanted_alone_is_a_maintainers_decision_not_an_automatic_claim() -> None: + # grouped sweeps and larger tasks carry `help wanted`; assigning a whole sweep to one person would lock the rest + gh = FakeGitHub(labels=["help wanted", "bug"]) + assert run(gh) == 0 + assert gh.assignees == [] and gh.writes() == [] + assert "Only `good first issue` issues can be claimed" in gh.comments[0] + + +# --- the invariant: a later /assign never changes an existing assignment --------------------------------------- + + +@pytest.mark.parametrize("claimant", ["late", "someone-else", "cemililik"]) # the last one is the maintainer +def test_assign_never_changes_an_issue_that_already_has_an_assignee(claimant: str) -> None: + gh = FakeGitHub(assignees=["first"]) + assert run(gh, login=claimant) == 0 + assert gh.assignees == ["first"] + assert gh.writes() == [] # no POST or DELETE on the assignees at all + assert len(gh.comments) == 1 + assert "already assigned to [first](https://github.com/first)" in gh.comments[0] + assert "nothing was changed" in gh.comments[0] + assert "Only a maintainer can reassign" in gh.comments[0] + + +def test_two_assignees_set_by_a_maintainer_are_left_alone_too() -> None: + gh = FakeGitHub(assignees=["a", "b"]) + assert run(gh, login="late") == 0 + assert gh.assignees == ["a", "b"] and gh.writes() == [] + + +def test_the_current_assignee_asking_again_changes_nothing() -> None: + gh = FakeGitHub(assignees=["newcomer"]) + assert run(gh) == 0 + assert gh.assignees == ["newcomer"] and gh.writes() == [] + assert gh.comments == [tool.MSG_ALREADY_YOURS] + + +def test_a_maintainer_assignment_is_never_reverted_because_the_tool_only_reacts_to_comments() -> None: + gh = FakeGitHub() + assert run(gh, login="first") == 0 + gh.assignees = ["maintainer-pick"] # a maintainer reassigns in the sidebar: no event reaches the tool + assert gh.assignees == ["maintainer-pick"] + assert run(gh, login="first") == 0 # the first claimant asks again: still refused + assert gh.assignees == ["maintainer-pick"] + + +def test_a_released_issue_can_be_claimed_by_the_next_person() -> None: + gh = FakeGitHub() + assert run(gh, login="first") == 0 + assert run(gh, "/unassign", login="first") == 0 + assert gh.assignees == [] + assert run(gh, login="second") == 0 + assert gh.assignees == ["second"] + + +def test_a_maintainer_who_unassigned_someone_lets_them_claim_again() -> None: + gh = FakeGitHub(assignees=["first"]) + gh.assignees = [] # removed in the sidebar; to keep it reserved a maintainer assigns it or removes the label + assert run(gh, login="first") == 0 + assert gh.assignees == ["first"] + + +# --- /unassign ------------------------------------------------------------------------------------------------ + + +def test_unassign_removes_only_the_person_who_wrote_it() -> None: + gh = FakeGitHub(assignees=["first", "other"]) + assert run(gh, "/unassign", login="first") == 0 + assert gh.assignees == ["other"] + assert gh.writes() == [("DELETE", f"repos/{REPO}/issues/7/assignees", {"assignees": ["first"]})] + assert "Released" in gh.comments[0] + + +def test_unassign_by_someone_who_is_not_assigned_changes_nothing() -> None: + gh = FakeGitHub(assignees=["first"]) + assert run(gh, "/unassign", login="stranger") == 0 + assert gh.assignees == ["first"] and gh.writes() == [] + assert gh.comments == [tool.MSG_NOT_YOURS] + + +def test_unassign_by_a_maintainer_still_only_releases_themselves() -> None: + gh = FakeGitHub(assignees=["first"]) + assert run(gh, "/unassign", login="cemililik") == 0 + assert gh.assignees == ["first"] and gh.writes() == [] + + +def test_unassign_still_works_on_a_closed_issue() -> None: + gh = FakeGitHub(assignees=["first"], state="closed") + assert run(gh, "/unassign", login="first") == 0 + assert gh.assignees == [] + + +# --- naming someone else -------------------------------------------------------------------------------------- + + +@pytest.mark.parametrize("body", ["/assign @someone", "/assign @someone please", "/unassign @someone"]) +def test_naming_someone_else_is_refused_without_any_change(body: str) -> None: + gh = FakeGitHub(assignees=["first"] if "unassign" in body else []) + before = list(gh.assignees) + assert run(gh, body, login="cemililik") == 0 + assert gh.assignees == before and gh.writes() == [] + assert gh.comments == [tool.MSG_SELF_ONLY] + + +# --- eligibility ---------------------------------------------------------------------------------------------- + + +@pytest.mark.parametrize("excluded", ["security", "needs-triage", "deferred", "epic", "phase: 16"]) +def test_an_excluded_label_beats_an_eligible_one(excluded: str) -> None: + gh = FakeGitHub(labels=["good first issue", "help wanted", excluded]) + assert run(gh) == 0 + assert gh.assignees == [] and gh.writes() == [] + assert "needs a maintainer's decision" in gh.comments[0] + assert tool.MAINTAINER_MENTION in gh.comments[0] # a human decision is needed: this one is meant to ping + + +def test_an_issue_without_an_eligible_label_is_a_maintainers_decision() -> None: + gh = FakeGitHub(labels=["bug", "wave: 1"]) + assert run(gh) == 0 + assert gh.assignees == [] and gh.writes() == [] + assert "needs a maintainer's decision" in gh.comments[0] + + +def test_a_closed_issue_cannot_be_claimed() -> None: + gh = FakeGitHub(state="closed") + assert run(gh) == 0 + assert gh.assignees == [] and gh.comments == [tool.MSG_CLOSED] + + +# --- the per-person limit ------------------------------------------------------------------------------------- + + +def test_the_limit_stops_a_third_open_assignment_and_points_to_the_maintainers() -> None: + gh = FakeGitHub(held={"newcomer": tool.MAX_OPEN_ASSIGNMENTS}) + assert run(gh) == 0 + assert gh.assignees == [] and gh.writes() == [] + reply = gh.comments[0] + assert f"`{tool.MAINTAINER_MENTION}`" in reply + assert tool.MAINTAINER_MENTION not in reply.replace( + f"`{tool.MAINTAINER_MENTION}`", "" + ) # an instruction, not a ping + assert "`/unassign`" in reply + + +def test_one_below_the_limit_is_still_allowed_and_prs_do_not_count() -> None: + gh = FakeGitHub(held={"newcomer": tool.MAX_OPEN_ASSIGNMENTS - 1}) + assert run(gh) == 0 + assert gh.assignees == ["newcomer"] + + +def test_maintainers_are_exempt_from_the_limit_and_skip_the_count() -> None: + gh = FakeGitHub(held={"cemililik": 10}) + assert run(gh, login="CemilIlik") == 0 # GitHub logins are case-insensitive + assert gh.assignees == ["CemilIlik"] + assert not any("?assignee=" in c[1] for c in gh.calls) + + +def test_a_read_only_organisation_member_is_not_a_maintainer() -> None: + gh = FakeGitHub(held={"hodesis": tool.MAX_OPEN_ASSIGNMENTS}) # a MEMBER by GitHub's association, read access only + assert run(gh, login="hodesis") == 0 + assert gh.assignees == [] and "limit at a time" in gh.comments[0] + + +def test_the_maintainer_list_matches_maintainers_md() -> None: + text = (ROOT / "MAINTAINERS.md").read_text(encoding="utf-8") + table = text.split("## Current maintainers", 1)[1].split("\n## ", 1)[0] + listed = { + row.split("[@", 1)[1].split("]", 1)[0].lower() + for row in table.splitlines() + if row.startswith("|") and "[@" in row + } + assert listed == set(tool.MAINTAINER_LOGINS) + + +@pytest.mark.parametrize( + "state", + [ + {"assignees": ["first"]}, # already taken + {"labels": ["bug"]}, # not open to automatic assignment + {"labels": ["good first issue", "epic"]}, # excluded + {"state": "closed"}, + ], + ids=["taken", "ineligible", "excluded", "closed"], +) +def test_the_count_is_only_asked_for_when_it_can_change_the_answer(state: Dict[str, Any]) -> None: + gh = FakeGitHub(**state) + gh.fail[("GET", f"repos/{REPO}/issues?assignee=")] = "the count would fail" + assert run(gh) == 0 # the proper refusal, not "automatic assignment did not complete" + assert not any("?assignee=" in c[1] for c in gh.calls) + assert "did not complete" not in gh.comments[0] + + +def test_a_failing_count_for_a_claim_that_needs_it_is_a_reported_failure() -> None: + gh = FakeGitHub() + gh.fail[("GET", f"repos/{REPO}/issues?assignee=")] = "boom" + assert run(gh) == 1 + assert gh.comments == [tool.MSG_FAILED.format(mention=tool.MAINTAINER_MENTION)] and gh.assignees == [] + + +def test_the_count_asks_for_open_issues_of_that_login_only() -> None: + gh = FakeGitHub() + run(gh, login="newcomer") + counts = [c for c in gh.calls if "?assignee=" in c[1]] + # the first count decides; the second, after the write, checks that a simultaneous claim did not break the limit + assert counts[0] == ("GET", f"repos/{REPO}/issues?assignee=newcomer&state=open&per_page=100&page=1", None) + assert all("assignee=newcomer&" in c[1] for c in counts) + + +# --- races and the second read -------------------------------------------------------------------------------- + + +def test_a_maintainer_who_assigns_between_the_two_reads_wins() -> None: + gh = FakeGitHub() + gh.after_first_read = lambda: gh.assignees.append("maintainer-pick") + assert run(gh) == 0 + assert gh.assignees == ["maintainer-pick"] and gh.writes() == [] + assert "already assigned" in gh.comments[0] + + +def test_two_assignees_after_a_simultaneous_claim_are_reported_to_the_maintainer() -> None: + gh = FakeGitHub() + original = gh.__call__ + + def other_claim_lands_during_the_post(method: str, endpoint: str, payload: Any = None) -> Any: + if method == "POST" and endpoint.endswith("/assignees"): + gh.assignees.append("rival") # the rival's POST landed first, after our second read + return original(method, endpoint, payload) + + assert tool.run(event("/assign"), REPO, other_claim_lands_during_the_post) == 0 + assert sorted(gh.assignees) == ["newcomer", "rival"] + assert "More than one person is assigned" in gh.comments[0] + assert "@rival" in gh.comments[0] and tool.MAINTAINER_MENTION in gh.comments[0] + + +# --- who is ignored ------------------------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "kwargs", + [{"pull_request": True}, {"user_type": "Bot"}], + ids=["pull-request-comment", "bot"], +) +def test_pull_request_comments_and_bots_are_ignored(kwargs: Dict[str, Any]) -> None: + gh = FakeGitHub() + assert run(gh, **kwargs) == 0 + assert gh.calls == [] + + +def test_a_comment_that_is_not_a_command_makes_no_api_call() -> None: + gh = FakeGitHub() + assert run(gh, "I'd like to work on this") == 0 + assert gh.calls == [] + + +# --- failure paths -------------------------------------------------------------------------------------------- + + +def test_an_api_failure_tells_the_contributor_and_fails_the_job(capsys: pytest.CaptureFixture[str]) -> None: + gh = FakeGitHub() + gh.fail[("GET", f"repos/{REPO}/issues/7")] = "boom" + assert run(gh) == 1 + assert gh.comments == [tool.MSG_FAILED.format(mention=tool.MAINTAINER_MENTION)] + assert "issue #7" in capsys.readouterr().err + + +def test_an_assignment_github_does_not_confirm_is_a_failure_not_a_success() -> None: + gh = FakeGitHub() + gh.drop_assignment = True + assert run(gh) == 1 + assert gh.comments == [tool.MSG_FAILED.format(mention=tool.MAINTAINER_MENTION)] + assert not any("Assigned to" in c for c in gh.comments) + + +def test_if_the_fallback_comment_fails_too_the_job_still_fails(capsys: pytest.CaptureFixture[str]) -> None: + gh = FakeGitHub() + gh.fail[("GET", f"repos/{REPO}/issues/7")] = "boom" + gh.fail[("POST", f"repos/{REPO}/issues/7/comments")] = "boom" + assert run(gh) == 1 + assert "could not tell the contributor" in capsys.readouterr().err + + +def test_a_reply_that_cannot_be_posted_after_a_successful_change_propagates() -> None: + gh = FakeGitHub() + gh.fail[("POST", f"repos/{REPO}/issues/7/comments")] = "boom" + with pytest.raises(tool.GhError): + run(gh) + assert gh.assignees == ["newcomer"] # the change stands; main() turns the error into exit 1 + + +@pytest.mark.parametrize( + "broken", + [ + {"issue": {"number": 7}}, + {"comment": {"body": "/assign"}}, + {}, + event("/assign", login="a b"), + event("/assign", number=0), + event("/assign", number=True), # type: ignore[arg-type] + ], +) +def test_an_unusable_event_is_exit_two_material_and_makes_no_api_call(broken: Dict[str, Any]) -> None: + gh = FakeGitHub() + with pytest.raises(tool.EventError): + tool.run(broken, REPO, gh) + assert gh.calls == [] + + +def test_main_returns_two_for_a_missing_environment_and_an_unreadable_event( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.delenv("GITHUB_REPOSITORY", raising=False) + monkeypatch.delenv("GITHUB_EVENT_PATH", raising=False) + assert tool.main() == 2 + monkeypatch.setenv("GITHUB_REPOSITORY", REPO) + monkeypatch.setenv("GITHUB_EVENT_PATH", str(tmp_path / "missing.json")) + assert tool.main() == 2 + bad = tmp_path / "bad.json" + bad.write_text("{not json", encoding="utf-8") + monkeypatch.setenv("GITHUB_EVENT_PATH", str(bad)) + assert tool.main() == 2 + + +# --- nothing from the comment reaches the log ------------------------------------------------------------------ + + +def test_the_comment_text_never_reaches_the_log(capsys: pytest.CaptureFixture[str]) -> None: + hostile = "/assign ::error::pwned ::add-mask::secret `$(touch /tmp/x)` ${{ secrets.GITHUB_TOKEN }}" + gh = FakeGitHub() + assert run(gh, hostile) == 0 + out = capsys.readouterr() + for text in (out.out, out.err): + assert "::error::" not in text and "pwned" not in text and "secret" not in text and "$(" not in text + assert out.out.strip() == "handled /assign on issue #7" + assert not any("pwned" in c for c in gh.comments) + + +def test_gh_error_text_is_flattened_and_defanged_before_it_is_printed() -> None: + assert tool._clean("::error::boom\nsecond line with gaps") == ": :error: :boom second line with gaps" + assert len(tool._clean("x" * 1000)) == 200 + + +# --- the gh wrapper ------------------------------------------------------------------------------------------- + + +class _Completed: + def __init__(self, returncode: int = 0, stdout: str = "", stderr: str = "") -> None: + self.returncode, self.stdout, self.stderr = returncode, stdout, stderr + + +def test_gh_api_runs_a_list_command_without_a_shell(monkeypatch: pytest.MonkeyPatch) -> None: + seen: Dict[str, Any] = {} + + def fake_run(command: Any, **kwargs: Any) -> _Completed: + seen["command"], seen["kwargs"] = command, kwargs + return _Completed(stdout='{"ok": true}') + + monkeypatch.setattr(tool.subprocess, "run", fake_run) + assert tool.gh_api("POST", "repos/o/r/issues/1/assignees", {"assignees": ["a"]}) == {"ok": True} + assert seen["command"] == ["gh", "api", "--method", "POST", "repos/o/r/issues/1/assignees", "--input", "-"] + assert seen["kwargs"]["input"] == '{"assignees": ["a"]}' + assert not seen["kwargs"].get("shell") and seen["kwargs"]["timeout"] == tool._GH_TIMEOUT_SECONDS + + +def test_gh_api_turns_failures_into_gherror_without_leaking_a_multiline_message( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setattr( + tool.subprocess, "run", lambda *a, **k: _Completed(returncode=1, stderr="::error::first\nsecond") + ) + with pytest.raises(tool.GhError) as info: + tool.gh_api("GET", "repos/o/r/issues/1?x=y") + message = str(info.value) + assert "\n" not in message and "::error::" not in message and "?x=y" not in message + + +@pytest.mark.parametrize("exc", [OSError("no gh"), subprocess.TimeoutExpired("gh", 60)]) +def test_gh_api_turns_a_missing_binary_or_a_timeout_into_gherror( + monkeypatch: pytest.MonkeyPatch, exc: BaseException +) -> None: + def boom(*a: Any, **k: Any) -> None: + raise exc + + monkeypatch.setattr(tool.subprocess, "run", boom) + with pytest.raises(tool.GhError): + tool.gh_api("GET", "repos/o/r/issues/1") + + +def test_gh_api_rejects_an_answer_that_is_not_json(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(tool.subprocess, "run", lambda *a, **k: _Completed(stdout="")) + with pytest.raises(tool.GhError): + tool.gh_api("GET", "repos/o/r/issues/1") + + +# --- the workflow file ---------------------------------------------------------------------------------------- + + +@pytest.fixture(scope="module") +def workflow() -> Dict[Any, Any]: + loaded = yaml.safe_load((ROOT / ".github/workflows/assign-on-comment.yml").read_text(encoding="utf-8")) + assert isinstance(loaded, dict) + return loaded + + +def test_workflow_triggers_only_on_a_created_comment(workflow: Dict[Any, Any]) -> None: + triggers = workflow[True] # YAML 1.1 reads the key `on` as a boolean + assert triggers == {"issue_comment": {"types": ["created"]}} + + +def test_workflow_token_is_least_privilege(workflow: Dict[Any, Any]) -> None: + assert workflow["permissions"] == {} + job = workflow["jobs"]["assign"] + assert job["permissions"] == {"contents": "read", "issues": "write"} + assert "timeout-minutes" in job + + +def test_workflow_has_no_concurrency_group_because_pending_runs_would_be_cancelled(workflow: Dict[Any, Any]) -> None: + assert "concurrency" not in workflow and "concurrency" not in workflow["jobs"]["assign"] + + +def test_workflow_filters_pull_requests_bots_and_non_commands_before_a_runner_starts( + workflow: Dict[Any, Any], +) -> None: + condition = workflow["jobs"]["assign"]["if"] + for needle in ("github.event.issue.pull_request", "'Bot'", "'/assign'", "'/unassign'"): + assert needle in condition + + +def test_workflow_never_puts_an_expression_into_a_shell_line(workflow: Dict[Any, Any]) -> None: + for step in workflow["jobs"]["assign"]["steps"]: + assert "${{" not in step.get("run", "") + runs = [s["run"] for s in workflow["jobs"]["assign"]["steps"] if "run" in s] + assert runs == ["python3 tools/assign_on_comment.py"] + + +def test_workflow_uses_one_action_pinned_to_a_full_commit_sha(workflow: Dict[Any, Any]) -> None: + uses = [s["uses"] for s in workflow["jobs"]["assign"]["steps"] if "uses" in s] + assert len(uses) == 1 and uses[0].startswith("actions/checkout@") + sha = uses[0].split("@", 1)[1] + assert len(sha) == 40 and all(c in "0123456789abcdef" for c in sha) + + +def test_workflow_checkout_takes_one_file_and_keeps_no_credentials(workflow: Dict[Any, Any]) -> None: + checkout = next(s for s in workflow["jobs"]["assign"]["steps"] if "uses" in s) + assert checkout["with"]["persist-credentials"] is False + assert checkout["with"]["sparse-checkout"] == "tools/assign_on_comment.py" + assert "ref" not in checkout["with"] # the default branch, never an event-supplied ref + + +def test_workflow_only_secret_is_the_github_token(workflow: Dict[Any, Any]) -> None: + step = next(s for s in workflow["jobs"]["assign"]["steps"] if "run" in s) + assert step["env"] == {"GH_TOKEN": "${{ secrets.GITHUB_TOKEN }}"} + text = (ROOT / ".github/workflows/assign-on-comment.yml").read_text(encoding="utf-8") + assert text.count("secrets.") == 1 and "pull_request_target" not in text and "workflow_run" not in text + + +# --- the documentation promises what the tool does ------------------------------------------------------------- + + +def test_contributing_documents_the_commands_the_limit_and_who_can_reassign() -> None: + text = (ROOT / "CONTRIBUTING.md").read_text(encoding="utf-8") + section = text.split("### Working on an issue", 1)[1].split("**Grouped issues.**", 1)[0] + assert "`/assign`" in section and "`/unassign`" in section + assert f"at most {tool.MAX_OPEN_ASSIGNMENTS} open" in section + assert "never changes an assignment" in section and "reassign" in section + for label in (*tool.EXCLUDED_LABELS, "help wanted"): + assert f"`{label}`" in section # every label the tool keeps for a maintainer is named for the contributor + + +# --- review round: repeats, pages, the second decision, settling after the write -------------------------------- + + +def test_the_maintainer_is_mentioned_once_per_issue_not_on_every_repeat() -> None: + gh = FakeGitHub(labels=["bug"]) + assert run(gh) == 0 + assert tool.MAINTAINER_MENTION in gh.comments[0] + assert run(gh, login="another") == 0 + assert gh.comments[1] == tool.MSG_NOT_OPEN_AGAIN + assert tool.MAINTAINER_MENTION not in gh.comments[1] + + +def test_a_person_cannot_silence_the_maintainer_notice_by_pasting_the_marker() -> None: + gh = FakeGitHub(labels=["bug"]) + gh.human_comments = [tool.NOT_OPEN_MARKER] # a User, not a Bot: it must not count + assert run(gh) == 0 + assert tool.MAINTAINER_MENTION in gh.comments[0] + + +def test_a_failing_notice_check_still_replies_and_mentions(capsys: pytest.CaptureFixture[str]) -> None: + gh = FakeGitHub(labels=["bug"]) + gh.fail[("GET", f"repos/{REPO}/issues/7/comments")] = "boom" + assert run(gh) == 0 + assert tool.MAINTAINER_MENTION in gh.comments[0] + assert "could not check for an earlier notice" in capsys.readouterr().err + + +def test_the_count_follows_pages_so_pull_requests_cannot_hide_issues() -> None: + first_page = [{"number": n, "pull_request": {}} for n in range(100)] # a full page with no issue on it + gh = FakeGitHub() + gh.assigned_pages = [first_page, [{"number": 500}, {"number": 501}]] + assert tool.count_open_assignments(gh, REPO, "newcomer") == 2 + assert run(gh) == 0 + assert gh.assignees == [] and "limit at a time" in gh.comments[0] + + +def test_the_count_stops_as_soon_as_it_knows() -> None: + gh = FakeGitHub() + gh.assigned_pages = [[{"number": n} for n in range(100)], [{"number": 1000}]] + assert tool.count_open_assignments(gh, REPO, "newcomer", stop_at=2) == 100 + assert len([c for c in gh.calls if "?assignee=" in c[1]]) == 1 + + +@pytest.mark.parametrize( + "change", + [ + lambda gh: setattr(gh, "state", "closed"), + lambda gh: gh.labels.append("security"), + lambda gh: gh.labels.remove("good first issue"), + ], + ids=["closed", "became-security", "lost-the-label"], +) +def test_an_issue_that_stops_qualifying_between_the_two_reads_is_not_assigned( + change: Callable[[FakeGitHub], None], +) -> None: + gh = FakeGitHub() + gh.after_first_read = lambda: change(gh) + assert run(gh) == 0 + assert gh.assignees == [] and gh.writes() == [] + assert "did not complete" not in gh.comments[0] + + +def test_a_claim_steps_back_when_a_maintainer_was_assigned_at_the_same_moment() -> None: + gh = FakeGitHub() + gh.after_assign = lambda: gh.assignees.insert(0, "cemililik") + assert run(gh) == 0 + assert gh.assignees == ["cemililik"] + assert gh.comments == [tool.MSG_LOST_RACE.format(front_door=tool.FRONT_DOOR_URL)] + assert gh.writes()[-1] == ("DELETE", f"repos/{REPO}/issues/7/assignees", {"assignees": ["newcomer"]}) + + +def test_a_maintainer_wins_a_simultaneous_claim_even_when_the_contributors_login_sorts_earlier() -> None: + gh = FakeGitHub() + gh.after_assign = lambda: gh.assignees.append( + "cemililik" + ) # "aaron" < "cemililik": only the maintainer rule releases + assert run(gh, login="aaron") == 0 + assert gh.assignees == ["cemililik"] + assert gh.comments == [tool.MSG_LOST_RACE.format(front_door=tool.FRONT_DOOR_URL)] + + +def test_between_two_contributors_the_earlier_login_stays_and_the_other_steps_back() -> None: + later = FakeGitHub() + later.after_assign = lambda: later.assignees.append("amy") # the rival "amy" landed too; we are "zed" + assert run(later, login="zed") == 0 + assert later.assignees == ["amy"] and later.comments == [tool.MSG_LOST_RACE.format(front_door=tool.FRONT_DOOR_URL)] + + earlier = FakeGitHub() + earlier.after_assign = lambda: earlier.assignees.append("zed") # we are "amy": we stay, the maintainer is told + assert run(earlier, login="amy") == 0 + assert sorted(earlier.assignees) == ["amy", "zed"] and "More than one person" in earlier.comments[0] + + +def test_two_racing_runs_that_both_wrote_end_with_exactly_one_assignee() -> None: + gh = FakeGitHub(assignees=["amy", "zed"]) # both POSTs have landed; now each run settles, in either order + for order in (("zed", "amy"), ("amy", "zed")): + gh.assignees = ["amy", "zed"] + for login in order: + ctx = tool.Context(REPO, 7, login, False, "/assign", False) + tool._settle_after_write(ctx, gh, tuple(gh.assignees), "assigned") + assert gh.assignees == ["amy"] + + +def test_a_maintainers_own_claim_never_steps_back() -> None: + gh = FakeGitHub() + gh.after_assign = lambda: gh.assignees.insert(0, "aaa-contributor") + assert run(gh, login="cemililik") == 0 + assert "cemililik" in gh.assignees and "More than one person" in gh.comments[0] + assert not [w for w in gh.writes() if w[0] == "DELETE"] + + +def test_a_claim_that_pushed_its_author_over_the_limit_steps_back() -> None: + gh = FakeGitHub(held={"newcomer": tool.MAX_OPEN_ASSIGNMENTS - 1}) + gh.after_assign = lambda: gh.held.__setitem__("newcomer", tool.MAX_OPEN_ASSIGNMENTS + 1) # other claims landed too + assert run(gh) == 0 + assert gh.assignees == [] + assert gh.comments == [tool.MSG_LIMIT_RACE.format(limit=tool.MAX_OPEN_ASSIGNMENTS)] + + +def test_stepping_back_that_github_does_not_confirm_is_a_reported_failure() -> None: + gh = FakeGitHub() + gh.after_assign = lambda: gh.assignees.insert(0, "cemililik") + gh.refuse_delete = True + assert run(gh) == 1 + assert gh.comments == [tool.MSG_FAILED.format(mention=tool.MAINTAINER_MENTION)] + + +def test_a_reply_is_retried_once_before_giving_up() -> None: + gh = FakeGitHub() + attempts = {"posts": 0} + + def flaky(method: str, endpoint: str, payload: Any = None) -> Any: + if method == "POST" and endpoint.endswith("/comments"): + attempts["posts"] += 1 + if attempts["posts"] == 1: + raise tool.GhError("transient") + return gh(method, endpoint, payload) + + assert tool.run(event("/assign"), REPO, flaky) == 0 + assert attempts["posts"] == 2 and len(gh.comments) == 1 + + +def test_a_failed_release_promises_a_release_not_an_assignment() -> None: + gh = FakeGitHub(assignees=["newcomer"]) + gh.refuse_delete = True + assert run(gh, "/unassign") == 1 + assert gh.comments == [tool.MSG_RELEASE_FAILED.format(mention=tool.MAINTAINER_MENTION)] + assert "assign this issue" not in gh.comments[0] + + +@pytest.mark.parametrize( + "method, endpoint", + [ + ("PUT", "repos/o/r/issues/1"), + ("GET", "-X"), + ("GET", "user"), + ("GET", "/repos/o/r"), + ("GET", "repos/o/r/issues/1 --hostname evil"), + ("POST", "repos/o/r/issues/1/comments\n--input"), + ], +) +def test_gh_api_refuses_anything_but_the_fixed_repos_endpoints( + monkeypatch: pytest.MonkeyPatch, method: str, endpoint: str +) -> None: + monkeypatch.setattr(tool.subprocess, "run", lambda *a, **k: pytest.fail("the command must not run")) + with pytest.raises(tool.GhError): + tool.gh_api(method, endpoint) + + +@pytest.mark.parametrize( + "body", + ["/assign", "/ASSIGN", "/Unassign", "/assign please", " /assign", "\n/assign", "\t/unassign", "x /assign", ""], +) +def test_the_workflow_gate_lets_through_everything_the_script_accepts(body: str) -> None: + gate = body.lower().startswith(("/assign", "/unassign")) # startsWith ignores case and trims nothing + if tool.parse_command(body) is not None: + assert gate diff --git a/tools/assign_on_comment.py b/tools/assign_on_comment.py new file mode 100644 index 00000000..277597c7 --- /dev/null +++ b/tools/assign_on_comment.py @@ -0,0 +1,475 @@ +#!/usr/bin/env python3 +"""Assign or release an issue when a contributor comments ``/assign`` or ``/unassign``. + +Run by ``.github/workflows/assign-on-comment.yml`` on an ``issue_comment`` event. A contributor claims an +issue by commenting ``/assign`` and gives it back with ``/unassign``; a maintainer no longer has to assign +by hand. Anything the rules below do not cover stays a maintainer's decision and gets a reply saying so. + +Rules (see ``CONTRIBUTING.md``, "Working on an issue"): + +- only issues labelled ``good first issue`` are open to ``/assign``, and never one that also carries + ``security``, ``needs-triage``, ``deferred``, ``epic`` or ``phase: 16``; +- an issue that already has an assignee is refused, whoever the assignee is and however they were assigned: + a later ``/assign`` never replaces, adds to or removes an existing assignment; +- a contributor holds at most ``MAX_OPEN_ASSIGNMENTS`` open issues at a time (the maintainers in + ``MAINTAINER_LOGINS`` are exempt); +- ``/unassign`` only ever removes the person who wrote it; +- both commands act on the person who wrote them: naming someone else (``/assign @someone``) is refused. + +Changing an assignment is a maintainer's decision, made in the issue's sidebar. This script reacts to +comments only: it never re-reads an issue later to "correct" an assignment a maintainer made. + +GitHub has no "assign only if nobody is" call, so two commands that land in the same second can both pass the +checks. After writing, the script therefore looks at what is true and steps back **only itself**: a claim is +released when a maintainer or (between two contributors) a login that sorts earlier is also assigned, or when +it pushed its author over the limit. Nobody else's assignment is ever removed by a command, so the invariant +"a later ``/assign`` never changes an existing assignment" holds; the worst case left is two assignees, which +the reply reports to a maintainer. + +Trust boundary. The comment is written by anyone, so the script never executes or prints it. The workflow +passes nothing through the shell: this script reads the event file itself, every subprocess call is a list +without a shell, and the only text written to the log is a fixed message, an issue number or a validated +login. GitHub treats a log line that starts with ``::`` as a workflow command, so error text from ``gh`` is +flattened to one line and defanged before it is printed. + +Exit codes (per ``tools/`` contract, not the public 0-6 surface of ``forgelm/``): + +- ``0`` — handled: assigned, released, refused with a reply, or not a command at all. +- ``1`` — unexpected failure (an API call failed, or GitHub did not confirm the assignment). A reply saying + a maintainer will assign the issue by hand is attempted first, so the contributor is never left waiting. +- ``2`` — the event payload or the environment is unusable. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import time +from dataclasses import dataclass +from typing import Any, Callable, Mapping, Optional, Sequence +from urllib.parse import quote + +# Only the starter tasks. `help wanted` is deliberately not here: most of those issues are grouped sweeps, which +# CONTRIBUTING asks contributors to take one finding ID at a time (assigning the whole issue to one person would +# lock the rest), or larger tasks that need a maintainer's answer first. +ELIGIBLE_LABELS = frozenset({"good first issue"}) +EXCLUDED_LABELS = frozenset({"security", "needs-triage", "deferred", "epic", "phase: 16"}) +# The maintainers, as MAINTAINERS.md lists them (tests/test_assign_on_comment.py fails if the two drift apart). +# GitHub's own "author association" is not used: it also calls a read-only organisation member a MEMBER. +MAINTAINER_LOGINS = frozenset({"cemililik"}) +MAINTAINER_MENTION = " ".join(f"@{login}" for login in sorted(MAINTAINER_LOGINS)) +MAX_OPEN_ASSIGNMENTS = 2 +COMMANDS = ("/assign", "/unassign") +FRONT_DOOR_URL = "https://github.com/orgs/HodeTech/projects/5/views/16" +_GH_TIMEOUT_SECONDS = 60 +_MAX_LOGIN_LENGTH = 39 +_PAGE_SIZE = 100 +_MAX_PAGES = 10 # bounds every paginated read; far above what a contributor with a 2-issue limit can reach +_COMMENT_ATTEMPTS = 2 +_RETRY_DELAY_SECONDS = 2.0 +_ALLOWED_METHODS = frozenset({"GET", "POST", "DELETE"}) +# Hidden marker in the "needs a maintainer" reply. Only a comment by a bot counts, so a contributor cannot forge it. +NOT_OPEN_MARKER = "" + +MSG_CLOSED = "This issue is closed, so it cannot be assigned." +MSG_NOT_OPEN = ( + "Only `good first issue` issues can be claimed with `/assign`; this one needs a maintainer's decision. " + "{mention}, could you take a look?\n\n" + NOT_OPEN_MARKER +) +MSG_NOT_OPEN_AGAIN = ( + "Only `good first issue` issues can be claimed with `/assign`; this one needs a maintainer's decision, " + "and a maintainer has already been notified on this issue.\n\n" + NOT_OPEN_MARKER +) +MSG_TAKEN = ( + "This issue is already assigned to [{owner}](https://github.com/{owner}), so nothing was changed. You can " + "pick another one from the " + "[Front door view]({front_door}). Only a maintainer can reassign an issue; if you think it should change, " + "leave a comment here saying why." +) +MSG_TAKEN_UNKNOWN = ( + "This issue is already assigned, so nothing was changed. You can pick another one from the " + "[Front door view]({front_door}). Only a maintainer can reassign an issue; if you think it should change, " + "leave a comment here saying why." +) +MSG_SELF_ONLY = ( + "`/assign` and `/unassign` act on the person who writes them, so nothing was changed. To assign someone " + "else, a maintainer sets it in the issue's sidebar." +) +MSG_ALREADY_YOURS = "This issue is already assigned to you." +MSG_LIMIT = ( + "You already have {count} open assignments, which is the limit at a time. Finish one, or comment `/unassign` " + "on one you will not do, then try again. If you need more than {limit}, comment here mentioning `{mention}` " + "and say what you would take on." +) +MSG_ASSIGNED = ( + "Assigned to @{login}. Thank you! Please open the pull request against `development` and link it with " + "`Fixes #{number}` (or `Refs #{number}` if it resolves only part of a grouped issue); see " + "[Working on an issue]({contributing}). If you cannot continue, comment `/unassign` so someone else can take it." +) +MSG_SEVERAL_ASSIGNEES = " More than one person is assigned now ({names}). {mention}, please check." +MSG_RELEASED = "Released. Thank you for letting us know, @{login}; the issue is open for others again." +MSG_NOT_YOURS = "This issue is not assigned to you, so there is nothing to release." +MSG_FAILED = "Automatic assignment did not complete. {mention} will assign this issue by hand." +MSG_RELEASE_FAILED = "Automatic release did not complete. {mention} will remove the assignment by hand." +MSG_LOST_RACE = ( + "Someone else was assigned to this issue at the same moment, so your claim was released and nothing else was " + "changed. Pick another issue from the [Front door view]({front_door}), or leave a comment here." +) +MSG_LIMIT_RACE = ( + "Your claims landed at the same moment and would have gone over the limit of {limit} open assignments, so this " + "one was released. Try again once one of your issues is done or released." +) + + +class EventError(Exception): + """The event payload or the environment is not usable.""" + + +class GhError(RuntimeError): + """A ``gh api`` call failed, or GitHub's answer was not what the API documents.""" + + +# (method, endpoint, optional JSON body) -> parsed JSON answer (None for an empty body). +GhApi = Callable[[str, str, Optional[Mapping[str, Any]]], Any] + + +@dataclass(frozen=True) +class Context: + """What the event says: who asked for what on which issue.""" + + repo: str + number: int + login: str + is_maintainer: bool + command: str + names_someone_else: bool + + +@dataclass(frozen=True) +class IssueFacts: + """The parts of an issue the rules need.""" + + state: str + labels: frozenset[str] + assignees: tuple[str, ...] + + +@dataclass(frozen=True) +class Decision: + """``action`` is ``ignore``, ``reply``, ``assign`` or ``unassign``; ``message`` is the reply text.""" + + action: str + message: str = "" + pings: bool = False # the reply mentions a maintainer; the caller may soften a repeat of it + + +def _first_line(body: str) -> str: + """The first line of the comment, exactly as written.""" + lines = body.splitlines() + return lines[0] if lines else "" + + +def parse_command(body: str) -> Optional[str]: + """Return ``/assign`` or ``/unassign`` when the comment starts with that command, else ``None``. + + The command must be the first word at the very start of the comment, case-insensitively; the rest is ignored. + The workflow's ``if:`` reads it the same way (``startsWith`` ignores case, and Actions has no ``trim``), so a + comment the gate lets through is one this accepts, and the other way round. + """ + line = _first_line(body) + if line != line.lstrip(): + return None + words = line.split() + if not words: + return None + word = words[0].lower() + return word if word in COMMANDS else None + + +def names_someone_else(body: str) -> bool: + """True when the command line carries an ``@name``: the commenter meant to act on somebody else.""" + return any(word.startswith("@") for word in _first_line(body).split()[1:]) + + +def valid_login(login: str) -> bool: + """GitHub logins are ASCII letters, digits and hyphens, at most 39 characters.""" + return 0 < len(login) <= _MAX_LOGIN_LENGTH and login.isascii() and all(c.isalnum() or c == "-" for c in login) + + +def _clean(text: str, limit: int = 200) -> str: + """One line, bounded, with ``::`` defanged so it can never open a workflow command.""" + return " ".join(text.split())[:limit].replace("::", ": :") + + +def contributing_url(repo: str) -> str: + return f"https://github.com/{repo}/blob/development/CONTRIBUTING.md#working-on-an-issue" + + +def decide(ctx: Context, issue: IssueFacts, count_open_assignments: Callable[[], int]) -> Decision: + """The rules: what to do about ``ctx.command`` from ``ctx.login`` on ``issue``. + + ``count_open_assignments`` is called only when the answer depends on it (the last check of ``/assign``), so a + refusal for any earlier reason makes no API call and cannot fail because of one. + """ + login = ctx.login + if ctx.names_someone_else: + return Decision("reply", MSG_SELF_ONLY) + if ctx.command == "/unassign": + if login in issue.assignees: + return Decision("unassign", MSG_RELEASED.format(login=login)) + return Decision("reply", MSG_NOT_YOURS) + if issue.state != "open": + return Decision("reply", MSG_CLOSED) + if issue.labels & EXCLUDED_LABELS or not issue.labels & ELIGIBLE_LABELS: + return Decision("reply", MSG_NOT_OPEN.format(mention=MAINTAINER_MENTION), pings=True) + if login in issue.assignees: + return Decision("reply", MSG_ALREADY_YOURS) + if issue.assignees: + owner = issue.assignees[0] + if valid_login(owner): + return Decision("reply", MSG_TAKEN.format(owner=owner, front_door=FRONT_DOOR_URL)) + return Decision("reply", MSG_TAKEN_UNKNOWN.format(front_door=FRONT_DOOR_URL)) + if not ctx.is_maintainer: + held = count_open_assignments() + if held >= MAX_OPEN_ASSIGNMENTS: + return Decision( + "reply", MSG_LIMIT.format(count=held, limit=MAX_OPEN_ASSIGNMENTS, mention=MAINTAINER_MENTION) + ) + return Decision( + "assign", + MSG_ASSIGNED.format(login=login, number=ctx.number, contributing=contributing_url(ctx.repo)), + ) + + +def context_from_event(event: Mapping[str, Any], repo: str) -> Optional[Context]: + """``None`` when the event is not a command to act on (PR comment, bot, or no command).""" + try: + issue, comment = event["issue"], event["comment"] + number, body = issue["number"], comment["body"] + user = comment["user"] + login, user_type = user["login"], user["type"] + is_pull_request = "pull_request" in issue + except (KeyError, TypeError) as exc: + raise EventError(f"event payload lacks a field: {_clean(str(exc), 60)}") from exc + if is_pull_request or user_type == "Bot": + return None + command = parse_command(body) if isinstance(body, str) else None + if command is None: + return None + if isinstance(number, bool) or not isinstance(number, int) or number < 1: + raise EventError("issue number is not a positive integer") + if not isinstance(login, str) or not valid_login(login): + raise EventError("commenter login is not a valid GitHub login") + return Context(repo, number, login, login.lower() in MAINTAINER_LOGINS, command, names_someone_else(body)) + + +def gh_api(method: str, endpoint: str, payload: Optional[Mapping[str, Any]] = None) -> Any: + """Call the GitHub REST API through the ``gh`` CLI (authenticated by ``GH_TOKEN``); no shell is involved.""" + if method not in _ALLOWED_METHODS or not endpoint.startswith("repos/") or any(c.isspace() for c in endpoint): + raise GhError("refusing an API call outside the fixed repos/ endpoints") + command = ["gh", "api", "--method", method, endpoint] + stdin = None + if payload is not None: + command += ["--input", "-"] + stdin = json.dumps(payload) + label = f"{method} {endpoint.split('?')[0]}" + try: + # A list without a shell; the method and endpoint were validated above (so neither can start with "-" or + # carry whitespace), and the endpoint is built only from integers and a validated login. + proc = subprocess.run( # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-audit + command, input=stdin, capture_output=True, text=True, timeout=_GH_TIMEOUT_SECONDS, check=False + ) + except (OSError, subprocess.TimeoutExpired) as exc: + raise GhError(f"{label} could not run: {type(exc).__name__}") from exc + if proc.returncode != 0: + raise GhError(f"{label} failed (exit {proc.returncode}): {_clean(proc.stderr)}") + if not proc.stdout.strip(): + return None + try: + return json.loads(proc.stdout) + except ValueError as exc: + raise GhError(f"{label} returned something that is not JSON") from exc + + +def _logins(users: Any) -> tuple[str, ...]: + if not isinstance(users, list): + raise GhError("assignees is not a list") + try: + return tuple(str(u["login"]) for u in users) + except (KeyError, TypeError) as exc: + raise GhError("an assignee has no login") from exc + + +def fetch_issue(gh: GhApi, repo: str, number: int) -> IssueFacts: + raw = gh("GET", f"repos/{repo}/issues/{number}", None) + try: + labels = frozenset(str(label["name"]) for label in raw["labels"]) + return IssueFacts(str(raw["state"]), labels, _logins(raw["assignees"])) + except (KeyError, TypeError) as exc: + raise GhError("the issue answer lacks labels, state or assignees") from exc + + +def count_open_assignments(gh: GhApi, repo: str, login: str, stop_at: Optional[int] = None) -> int: + """Open issues (not pull requests) assigned to ``login`` in this repository, across pages. + + Pull requests share the endpoint, so a page can hold none of the issues; the count follows the pages until one + is short, ``stop_at`` is reached (the caller only needs to know it was), or ``_MAX_PAGES`` is hit. + """ + held = 0 + for page in range(1, _MAX_PAGES + 1): + raw = gh( + "GET", f"repos/{repo}/issues?assignee={quote(login)}&state=open&per_page={_PAGE_SIZE}&page={page}", None + ) + if not isinstance(raw, list): + raise GhError("the assigned-issues answer is not a list") + held += sum(1 for item in raw if isinstance(item, dict) and "pull_request" not in item) + if len(raw) < _PAGE_SIZE or (stop_at is not None and held >= stop_at): + break + return held + + +def _already_pinged(gh: GhApi, repo: str, number: int) -> bool: + """True when the bot has already asked a maintainer to look at this issue (so a repeat need not mention them).""" + try: + for page in range(1, 4): + raw = gh("GET", f"repos/{repo}/issues/{number}/comments?per_page={_PAGE_SIZE}&page={page}", None) + if not isinstance(raw, list): + return False + for comment in raw: + user = comment.get("user") if isinstance(comment, dict) else None + if isinstance(user, dict) and user.get("type") == "Bot" and NOT_OPEN_MARKER in str(comment.get("body")): + return True + if len(raw) < _PAGE_SIZE: + break + except GhError as exc: # optional: if it fails, mention the maintainer again rather than stay quiet + print(f"warning: issue #{number}: could not check for an earlier notice: {exc}", file=sys.stderr) + return False + + +def _release_self(ctx: Context, gh: GhApi) -> None: + """Remove the commenter, and only the commenter, from the issue; fail loudly if GitHub still lists them.""" + result = gh("DELETE", f"repos/{ctx.repo}/issues/{ctx.number}/assignees", {"assignees": [ctx.login]}) + remaining = _logins(result.get("assignees") if isinstance(result, dict) else None) + if ctx.login.lower() in {a.lower() for a in remaining}: + raise GhError("GitHub still lists the commenter as assignee after the release") + + +def _reply_text(decision: Decision, ctx: Context, gh: GhApi) -> str: + """The reply for a refusal; a repeat of the "needs a maintainer" notice no longer mentions the maintainer.""" + if decision.pings and _already_pinged(gh, ctx.repo, ctx.number): + return MSG_NOT_OPEN_AGAIN + return decision.message + + +def _settle_after_write(ctx: Context, gh: GhApi, assignees: tuple[str, ...], message: str) -> str: + """Look at what is true after the assignment landed and step back, if needed, only ourselves. + + Two commands in the same second can both pass the earlier checks. A contributor's claim is released when a + maintainer is also assigned (the maintainer's decision stands), when another contributor whose login sorts + earlier is (a rule both racing runs apply identically, so exactly one stays), or when it pushed its author over + the limit. A maintainer's own claim is never released. Nobody else's assignment is touched. + """ + me = ctx.login.lower() + others = [a for a in assignees if a.lower() != me] + if not ctx.is_maintainer: + if any(a.lower() in MAINTAINER_LOGINS or a.lower() < me for a in others): + _release_self(ctx, gh) + return MSG_LOST_RACE.format(front_door=FRONT_DOOR_URL) + if count_open_assignments(gh, ctx.repo, ctx.login, stop_at=MAX_OPEN_ASSIGNMENTS + 1) > MAX_OPEN_ASSIGNMENTS: + _release_self(ctx, gh) + return MSG_LIMIT_RACE.format(limit=MAX_OPEN_ASSIGNMENTS) + if others: + names = ", ".join(f"@{a}" for a in assignees if valid_login(a)) + message += MSG_SEVERAL_ASSIGNEES.format(names=names, mention=MAINTAINER_MENTION) + return message + + +def _perform(ctx: Context, gh: GhApi) -> str: + """Read, decide, change the assignment. Returns the reply text ('' for none).""" + issue = fetch_issue(gh, ctx.repo, ctx.number) + counted: list[int] = [] + + def held() -> int: # asked for at most once per run, and only if the answer depends on it + if not counted: + counted.append(count_open_assignments(gh, ctx.repo, ctx.login, stop_at=MAX_OPEN_ASSIGNMENTS)) + return counted[0] + + decision = decide(ctx, issue, held) + if decision.action in ("ignore", "reply"): + return _reply_text(decision, ctx, gh) + if decision.action == "unassign": + _release_self(ctx, gh) + return decision.message + # Assign. GitHub has no "assign only if nobody is" call, so decide again on a fresh read right before writing: + # the issue may have been assigned, closed or relabelled since the first read. + latest = decide(ctx, fetch_issue(gh, ctx.repo, ctx.number), held) + if latest.action != "assign": + return _reply_text(latest, ctx, gh) + result = gh("POST", f"repos/{ctx.repo}/issues/{ctx.number}/assignees", {"assignees": [ctx.login]}) + assignees = _logins(result.get("assignees") if isinstance(result, dict) else None) + if ctx.login.lower() not in {a.lower() for a in assignees}: + raise GhError("GitHub did not confirm the assignment") + return _settle_after_write(ctx, gh, assignees, latest.message) + + +def _comment(gh: GhApi, repo: str, number: int, body: str) -> None: + """Post a reply; one retry, because the change it reports has already been made.""" + for attempt in range(1, _COMMENT_ATTEMPTS + 1): + try: + gh("POST", f"repos/{repo}/issues/{number}/comments", {"body": body}) + return + except GhError as exc: + if attempt == _COMMENT_ATTEMPTS: + raise + print(f"warning: issue #{number}: reply failed, retrying: {exc}", file=sys.stderr) + time.sleep(_RETRY_DELAY_SECONDS) + + +def run(event: Mapping[str, Any], repo: str, gh: GhApi) -> int: + """Handle one ``issue_comment`` event. See the module docstring for the exit codes.""" + ctx = context_from_event(event, repo) + if ctx is None: + return 0 + try: + reply = _perform(ctx, gh) + except GhError as exc: + print(f"error: issue #{ctx.number}: {exc}", file=sys.stderr) + failed = MSG_RELEASE_FAILED if ctx.command == "/unassign" else MSG_FAILED + try: + _comment(gh, ctx.repo, ctx.number, failed.format(mention=MAINTAINER_MENTION)) + except GhError as comment_exc: + print(f"error: issue #{ctx.number}: could not tell the contributor: {comment_exc}", file=sys.stderr) + return 1 + if reply: + _comment(gh, ctx.repo, ctx.number, reply) + print(f"handled {ctx.command} on issue #{ctx.number}") + return 0 + + +def main(argv: Optional[Sequence[str]] = None) -> int: + del argv # no command-line arguments: the workflow's environment is the whole input + repo = os.environ.get("GITHUB_REPOSITORY", "") + event_path = os.environ.get("GITHUB_EVENT_PATH", "") + if repo.count("/") != 1 or not event_path: + print("error: GITHUB_REPOSITORY and GITHUB_EVENT_PATH must be set", file=sys.stderr) + return 2 + try: + with open(event_path, encoding="utf-8") as handle: + event = json.load(handle) + except (OSError, ValueError) as exc: + print(f"error: cannot read the event payload: {type(exc).__name__}", file=sys.stderr) + return 2 + try: + return run(event, repo, gh_api) + except EventError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + except GhError as exc: # the reply to a successful change could not be posted + print(f"error: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) From 675c3e3b81fcf0acab893ce96566b2d996c2aeec Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 11:46:29 +0300 Subject: [PATCH 07/22] =?UTF-8?q?chore:=20housekeeping=20=E2=80=94=20stale?= =?UTF-8?q?=20requirements=20files,=20roadmap=20corrections,=20medical=20t?= =?UTF-8?q?emplate=20row=20(#996)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Three small, already-reviewed fixes combined into one pull request, replacing #979, #980 and #981 (which will be closed as superseded). Each was green on its own and the bot findings on #980 and #981 are already folded in. ## Related issues Fixes #968. Refs #91, #75, #97. ## Changes | Was | What it does | |---|---| | #979 | Deletes the stale `requirements.txt`, `requirements-eval.txt` and `requirements-linux.txt`. Nothing referenced them, their floors were far below `pyproject.toml`'s, and Dependabot would have opened weekly PRs against them. Merging this **before Monday 2026-10-19** avoids those PRs. | | #980 | Roadmap/strategy corrections: `forgelm chat` does not screen for safety and reads no YAML (#91); live risk rows no longer name a past "August 2026" deadline and now limit the dates to Chapter III Sections 1–3 as CodeRabbit asked (#75); `docs/product_strategy.md` (EN+TR) names the real `evaluation.auto_revert` field. | | #981 | The `medical-qa-tr` row of the YAML templates reference (EN+TR): the template enables no safety evaluation and has no `risk_assessment` metadata; it is not for clinical decision support, and the row says to set `evaluation.safety.enabled: true` and fill in `risk_assessment` accurately (Sourcery and CodeRabbit findings applied). | ## Type - [x] Documentation - [x] Chore ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. ## Checklist - [x] English and Turkish mirrors updated together - [x] CHANGELOG entries under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Remove stale dependency files and correct safety, compliance, and medical-template documentation across the project. Bug Fixes: - Correct roadmap, risk, and strategy documentation to accurately describe chat safety behavior, EU AI Act timelines, and the current evaluation auto-revert configuration. - Clarify the medical QA template’s safety limitations and required evaluation and risk-assessment settings in English and Turkish references. Enhancements: - Remove the obsolete requirements files and direct installation guidance through pyproject.toml extras. Documentation: - Update the English and Turkish documentation mirrors and record the housekeeping and documentation corrections in the unreleased changelog. --------- Co-authored-by: Claude Sonnet 5.5 --- .dockerignore | 1 - .github/dependabot.yml | 6 ------ CHANGELOG.md | 12 ++++++++++++ docs/product_strategy-tr.md | 2 +- docs/product_strategy.md | 2 +- docs/roadmap/completed-phases.md | 16 +++++++++------- docs/roadmap/risks-and-decisions.md | 6 +++--- docs/usermanuals/en/reference/yaml-templates.md | 2 +- docs/usermanuals/tr/reference/yaml-templates.md | 2 +- requirements-eval.txt | 1 - requirements-linux.txt | 2 -- requirements.txt | 11 ----------- 12 files changed, 28 insertions(+), 35 deletions(-) delete mode 100644 requirements-eval.txt delete mode 100644 requirements-linux.txt delete mode 100644 requirements.txt diff --git a/.dockerignore b/.dockerignore index 41d77ac2..ea08c564 100644 --- a/.dockerignore +++ b/.dockerignore @@ -64,5 +64,4 @@ CODE_OF_CONDUCT.md # Build tooling not needed by the installed package tools/ MANIFEST.in -requirements*.txt docker-compose.yaml diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 7d861eb3..cacb425c 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -16,12 +16,6 @@ # `ci.yml`'s lint job installs. Dependabot edits only `pyproject.toml`, so a bump # would fail that test the moment it arrived. Those three are raised deliberately, # in one commit, with the CI pin moved in step. -# -# Note: `requirements.txt`, `requirements-eval.txt` and `requirements-linux.txt` -# are stale and referenced by nothing — the Dockerfile, CI and the tests all use -# `pyproject.toml` extras. Dependabot will still raise their floors; those pull -# requests should be closed, and the files removed, when the tracked cleanup -# lands. version: 2 updates: diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d44dd02..c86b509b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,8 +66,20 @@ All notable changes to ForgeLM are documented here. `CONTRIBUTING.md`, `CLAUDE.md` and `AGENTS.md` that the front door is chosen with labels and views, never by closing issues. +### Removed + +- Delete the stale `requirements.txt`, `requirements-eval.txt` and `requirements-linux.txt`. Nothing + referenced them, their floors were far below `pyproject.toml`'s, and Dependabot would have opened weekly + pull requests against them; installation goes through the `pyproject.toml` extras (#968). + ### Fixed +- Correct claims in the roadmap and strategy documents: `forgelm chat` does not screen for safety and reads no YAML + config (#91), the live risk rows no longer name a past "August 2026" deadline (#75), and + `docs/product_strategy.md` (English and Turkish) names the real `evaluation.auto_revert` field instead of a + non-existent `auto_revert_on_safety_failure`. +- Correct the `medical-qa-tr` row of the YAML templates reference (English and Turkish): the template enables no safety + evaluation and no `risk_assessment`, so it no longer says it ships with both pre-wired (#97). - Fix the KTO bullet on the Alignment overview user-manual page (English and Turkish): it named the fields `(prompt, response, good?)`, but the KTO loader reads `prompt`, `completion` and a boolean `label` (#965). diff --git a/docs/product_strategy-tr.md b/docs/product_strategy-tr.md index 97d465cd..d9010672 100644 --- a/docs/product_strategy-tr.md +++ b/docs/product_strategy-tr.md @@ -21,7 +21,7 @@ ForgeLM, LLM ince ayarını (fine-tuning) **güvenli, denetlenebilir ve tekrarla - **3 katmanlı güvenlik kapısı**: ikili geçer/kalır + güven ağırlıklı skor + 14 zarar kategorisinde (S1-S14) önem analizi - **Otomatik geri alma (auto-revert)**: güvenlik regresyonu tespit edilirse, çıktılar yazılmadan önce eğitim sonucu otomatik olarak reddedilir - **Çalıştırmalar arası trend izleme**: güvenlik skorları her çalıştırmada loglanır; zamanla oluşan kayan bozulma yakalanır -- **`auto_revert_on_safety_failure`** config alanı doğrudan YAML'a yazılır — CI'da güvenlik kapısı uygulamak için özel script gerekmez +- **`evaluation.auto_revert`** config alanı doğrudan YAML'a yazılır — bir güvenlik, benchmark ya da judge kapısı başarısız olduğunda çalışma eğitilmiş çıktıları silip 3 koduyla çıkabilir; CI'da kapıları uygulamak için özel script gerekmez ### 2. Uyumluluk Yerleşik (Compliance-Native) diff --git a/docs/product_strategy.md b/docs/product_strategy.md index 06ef3f89..12105967 100644 --- a/docs/product_strategy.md +++ b/docs/product_strategy.md @@ -21,7 +21,7 @@ Fine-tuning can degrade a model's safety properties — silently. ForgeLM treats - **3-layer safety gate**: binary pass/fail + confidence-weighted score + per-category severity analysis covering 14 harm categories (S1-S14) - **Auto-revert**: if safety regression is detected against baseline, training output is automatically rejected before artifacts are written - **Cross-run trend tracking**: safety scores are logged per-run and trended over time to catch gradual drift -- **`auto_revert_on_safety_failure`** is a first-class config field — teams can enforce safety gates in CI without any custom scripting +- **`evaluation.auto_revert`** is a first-class config field — when a safety, benchmark or judge gate fails, the run can delete the trained artefacts and exit with code 3, so teams can enforce the gates in CI without any custom scripting ### 2. Compliance-Native diff --git a/docs/roadmap/completed-phases.md b/docs/roadmap/completed-phases.md index f266a662..99679f0c 100644 --- a/docs/roadmap/completed-phases.md +++ b/docs/roadmap/completed-phases.md @@ -1,6 +1,8 @@ # Completed Phases — Archive > **Note:** Historical archive of ForgeLM's Phase 1-10 completed work. Kept for reference and onboarding. For active planning, see [../roadmap.md](../roadmap.md). +> +> **Dates.** Where this archive says the EU AI Act is "fully enforced" in "August 2026", that was the planning assumption when the phase was written and is kept as the record. The application dates have since changed and differ by obligation; the current statement is in [../product_strategy.md](../product_strategy.md), and there is no single full-enforcement date. ## Summary table @@ -661,18 +663,18 @@ graph TD ``` 2. [x] **`forgelm chat` — interactive terminal loop** - Terminal REPL in `forgelm/chat.py`: streaming output (default), `/reset`, `/save [file]`, `/temperature 0.x`, `/system `, `/help` commands. `rich` optional rendering. History management with 50-pair cap. The `--safety` per-turn screen flag is planned for v0.6.0+ Pro CLI (today's safety pipeline runs through the YAML `safety:` block during training/eval). + Terminal REPL in `forgelm/chat.py`: streaming output (default), `/reset`, `/save [file]`, `/temperature 0.x`, `/system `, `/help` commands. `rich` optional rendering. History management with 50-pair cap. `forgelm chat` performs no safety screening and reads no YAML config; the `--safety` per-turn screen flag is not implemented. Screen a model with `forgelm safety-eval --model --default-probes`, or with the `evaluation.safety` block during training, before exposing it. ```bash forgelm chat ./outputs/my_run ``` `forgelm chat`'s shipping form takes a positional checkpoint path - plus optional `--adapter`. The per-turn Llama Guard screen via - `--safety` is planned for v0.6.0+ Pro CLI (see Phase 13 roadmap); - today operators get the same screening by enabling - `safety: enabled: true` in the YAML config the chat REPL reads. - The flag-form preview (`forgelm chat ... --safety`) is NOT - runnable today. + plus optional `--adapter`. It performs no safety screening and does + not read a YAML config: a per-turn Llama Guard screen via `--safety` + is not implemented, and the flag-form preview + (`forgelm chat ... --safety`) is NOT runnable. Screen a model with + `forgelm safety-eval --model --default-probes` (or the + `evaluation.safety` block during training) before exposing it. 3. [x] **`forgelm export` — HF → GGUF conversion** Wraps `llama-cpp-python`'s `convert_hf_to_gguf.py`; no reimplementation. Handles adapter merge (LoRA + base → merged fp16) before conversion. Supports quants: `q2_k`, `q3_k_m`, `q4_k_m`, `q5_k_m`, `q8_0`, `f16`. SHA-256 of output artifact appended to `model_integrity.json`. Optional dependency: `pip install forgelm[export]`. diff --git a/docs/roadmap/risks-and-decisions.md b/docs/roadmap/risks-and-decisions.md index 39b42bd0..e67a0ed4 100644 --- a/docs/roadmap/risks-and-decisions.md +++ b/docs/roadmap/risks-and-decisions.md @@ -8,10 +8,10 @@ | Risk | Impact | Likelihood | Mitigation | |------|--------|------------|------------| | **Dependency Breaking Changes** (TRL, PEFT, Unsloth) | Training pipeline breaks without warning | High | Version pinning with upper bounds, CI nightly builds against latest deps, compatibility matrix | -| **EU AI Act Non-Compliance** (August 2026 deadline) | Enterprise customers cannot adopt ForgeLM for high-risk AI | High | Phase 8 deep compliance: Annex IV docs, audit log, risk assessment, human gate, data governance | +| **EU AI Act Non-Compliance** (Chapter III, Sections 1–3, except Article 6(5), apply from 2 December 2027 to high-risk AI systems under Article 6(2) and Annex III, and from 2 August 2028 to systems under Article 6(1) and Annex I; see [product_strategy.md](../product_strategy.md)) | Enterprise customers cannot adopt ForgeLM for high-risk AI | High | Phase 8 deep compliance: Annex IV docs, audit log, risk assessment, human gate, data governance | | **Safety Degradation from Fine-Tuning** | Fine-tuned models lose alignment, enterprise liability | High | Phase 6 safety evaluation pipeline, auto-revert on safety regression | | **Alignment Method Lock-In** | ForgeLM supports only ORPO while market demands DPO/GRPO | High | Phase 5 is top priority — critical market expectation | -| **Compliance window closing** (Aug 2, 2026) | Enterprise leads not converted before competitors add compliance features | High | Enterprise outreach Day 1; Turkey market as first pilot; deep compliance features (Audit API, managed cloud) as moat | +| **Compliance window** (Annex III from 2 December 2027, Annex I from 2 August 2028) | Enterprise leads not converted before competitors add compliance features | High | Enterprise outreach Day 1; Turkey market as first pilot; deep compliance features (Audit API, managed cloud) as moat | | **Community flywheel not started** | 0 stars → 0 enterprise credibility → missed EU AI Act window | High | Phase 10.5 Quickstart prioritized; YouTube Academy; Show HN + Reddit launch | ### Medium Severity @@ -39,7 +39,7 @@ 2. **Safety-as-a-Feature (Phase 6)** — No competitor integrates safety evaluation into the training pipeline. ForgeLM can own "the safest way to fine-tune an LLM." ### Medium-Term Opportunities -3. **EU AI Act Compliance** — August 2026 deadline creates urgent demand. ForgeLM as the only tool generating compliance artifacts is a powerful enterprise sales argument. +3. **EU AI Act Compliance** — the staggered high-risk application dates (2 December 2027 for Annex III, 2 August 2028 for Annex I) create demand. ForgeLM as the only tool generating compliance artifacts is a powerful enterprise sales argument. 4. **MoE Fine-Tuning** — Qwen3, DeepSeek-V3 dominance means MoE support is becoming table stakes. 5. **Cost Transparency** — GPU cost tracking per run enables enterprise budget planning and optimization. Simple to implement, high perceived value. diff --git a/docs/usermanuals/en/reference/yaml-templates.md b/docs/usermanuals/en/reference/yaml-templates.md index 5fb6481a..75341a39 100644 --- a/docs/usermanuals/en/reference/yaml-templates.md +++ b/docs/usermanuals/en/reference/yaml-templates.md @@ -23,7 +23,7 @@ The five templates and the user journey each one optimises for: | `customer-support` | SFT | Multi-turn helpful + safe assistant — the most common production pattern. | | `code-assistant` | SFT | Code-completion / pair-programming model. | | `domain-expert` | SFT | BYOD (Bring Your Own Dataset) on a domain-specific corpus. | -| `medical-qa-tr` | SFT | Turkish-language medical QA (regulated domain — ships with `risk_assessment` + safety eval pre-wired). | +| `medical-qa-tr` | SFT | Turkish-language medical QA (regulated domain — the sample answers carry a safety disclaimer, but the template enables no safety evaluation and has no `risk_assessment` metadata. It is not for clinical decision support, and adding these does not change that; before any use beyond a demo, set `evaluation.safety.enabled: true` and fill in the top-level `risk_assessment:` block accurately). | | `grpo-math` | GRPO | Reward-model fine-tuning on math benchmarks (built-in format/length reward shaping). | The canonical template content is **the source of truth** — it is what diff --git a/docs/usermanuals/tr/reference/yaml-templates.md b/docs/usermanuals/tr/reference/yaml-templates.md index 9288a614..5baf2e0d 100644 --- a/docs/usermanuals/tr/reference/yaml-templates.md +++ b/docs/usermanuals/tr/reference/yaml-templates.md @@ -23,7 +23,7 @@ Beş şablon ve her birinin optimize ettiği kullanıcı yolu: | `customer-support` | SFT | Multi-turn yardımcı + güvenli asistan — en yaygın production pattern. | | `code-assistant` | SFT | Kod-tamamlama / pair-programming modeli. | | `domain-expert` | SFT | Domain-spesifik corpus üzerinde BYOD (Bring Your Own Dataset). | -| `medical-qa-tr` | SFT | Türkçe tıbbi soru-cevap (regüle alan — `risk_assessment` + safety eval ön-bağlı ship olur). | +| `medical-qa-tr` | SFT | Türkçe tıbbi soru-cevap (regüle alan — örnek cevaplar bir güvenlik uyarısı taşır, ama şablon güvenlik değerlendirmesini etkinleştirmez ve `risk_assessment` metadata'sı içermez. Klinik karar desteği için değildir ve bunları eklemek bunu değiştirmez; demo dışındaki bir kullanımdan önce `evaluation.safety.enabled: true` yap ve en üstteki `risk_assessment:` bloğunu doğru biçimde doldur). | | `grpo-math` | GRPO | Matematik benchmark'ları üzerinde reward-model fine-tuning (yerleşik format/length reward shaping). | Kanonik şablon içeriği **doğruluk kaynağıdır** — `forgelm quickstart`'ın diff --git a/requirements-eval.txt b/requirements-eval.txt deleted file mode 100644 index 94fe784e..00000000 --- a/requirements-eval.txt +++ /dev/null @@ -1 +0,0 @@ -lm-eval>=0.4.4 diff --git a/requirements-linux.txt b/requirements-linux.txt deleted file mode 100644 index b76fe013..00000000 --- a/requirements-linux.txt +++ /dev/null @@ -1,2 +0,0 @@ -bitsandbytes>=0.42.0 -unsloth>=2024.3.0 diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 58f7f5f8..00000000 --- a/requirements.txt +++ /dev/null @@ -1,11 +0,0 @@ -torch>=2.0.0 -transformers>=4.38.0 -peft>=0.9.0 -datasets>=2.17.0 -accelerate>=0.27.0 -pyyaml>=6.0.1 -pydantic>=2.0.0 -tensorboard>=2.15.0 -trl>=0.8.0 -huggingface_hub>=0.20.0 -requests>=2.31.0 From 48c75825e82b8fa50466ec0b72a988b6c82541ab Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 11:58:59 +0300 Subject: [PATCH 08/22] feat(export): write the SHA-256 sidecar verify-gguf relies on, and document the converter requirement (#998) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary `forgelm export` now writes the SHA-256 sidecar that `forgelm verify-gguf` and the documentation already assumed, and the GGUF documentation describes what really happens. Two earlier pull requests combined into one (#982 and #995, which will be closed as superseded); the bot findings on #982 are folded in. ## Related issues Refs #84, #85, #86 (docs), #83, #113. Making `verify-gguf` stricter than "magic header + optional sidecar", and searching the locations real llama-cpp-python wheels use for the converter, are separate changes and stay open. The site strings that mention the sidecar are in a later pull request. ## Changes ### Sidecar (was #982) - `forgelm/export.py`: after the GGUF is written and hashed, `.sha256` is written in `sha256sum` format (` *`) through a temporary file and `os.replace`. A write failure returns `success=False` (runtime error, exit 2) naming the GGUF path. For a K-quant request the file written is the intermediate `.f16.gguf` and the sidecar belongs to it. The `model_integrity.json` update failure is now a `WARNING`, not `DEBUG`. - The sidecar detects corruption and accidental change; it is **not** tamper evidence, because whoever can replace the GGUF can replace the sidecar next to it. The docs say so. - `docs/usermanuals/{en,tr}/deployment/{gguf-export,verify-gguf}.md` and `docs/reference/verify_gguf_subcommand{,-tr}.md`: real output, the K-quant two-step flow, the real `exported_artifacts` JSON, what `verify-gguf` checks and what it cannot, and the `--adapter` pitfall. Review findings applied: the intermediate file is named by replacing `.gguf` with `.f16.gguf`; the manifest entry is best effort. ### Converter script (was #995) - `pip install 'forgelm[export]'` does not provide `convert_hf_to_gguf.py`: no released llama-cpp-python wheel puts it where ForgeLM searches (0.2.90 installs it under `bin/`, 0.3.x does not ship it). The installation page, CLI reference, usage and quickstart guides, the notebook, `pyproject.toml` comment and `export.py` docstring now say to point `FORGELM_GGUF_CONVERTER` at the script in a llama.cpp checkout; the upstream repository location, requirements path and imports were checked. - The not-found error message names `FORGELM_GGUF_CONVERTER` and a llama.cpp checkout instead of advising a reinstall. ## Type - [x] Feature (sidecar) - [x] Documentation ## Testing - `tests/test_export.py`: sidecar format and content, `sha256sum -c` and `verify-gguf` accept it, atomic write and cleanup, K-quant sidecar, failure path, plus the new converter hint test (**fails against the old message**). - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. ## Checklist - [x] English and Turkish mirrors updated together (Turkish wording needs the owner's review) - [x] CHANGELOG entries under `[Unreleased]` - [x] Tests added 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Add reliable SHA-256 sidecars to GGUF export and align converter requirements, verification behavior, and user documentation with the implemented workflow. New Features: - Write SHA-256 sidecar files for exported GGUF artifacts, including intermediate files produced for K-quant requests. - Fail exports when the required sidecar cannot be written and elevate integrity-manifest update failures to warnings. Bug Fixes: - Correct GGUF export and verification documentation to reflect actual outputs, checks, limitations, K-quant workflow, and adapter usage. - Correct converter setup guidance and error messaging to require a llama.cpp checkout through FORGELM_GGUF_CONVERTER. Enhancements: - Clarify that sidecars detect accidental corruption but do not provide tamper evidence. - Document the actual exported_artifacts manifest structure and verification behavior in English and Turkish documentation. Documentation: - Update installation, CLI, usage, quickstart, notebook, and GGUF deployment documentation with the llama.cpp converter prerequisite and sidecar workflow. Tests: - Add coverage for sidecar format, verification, atomic cleanup, K-quant output, write failures, manifest warnings, and converter guidance. Chores: - Update the unreleased changelog and related export comments and docstrings. --------- Co-authored-by: Claude Sonnet 5.5 --- CHANGELOG.md | 16 +++ docs/guides/quickstart-tr.md | 3 +- docs/guides/quickstart.md | 3 +- docs/reference/usage-tr.md | 3 +- docs/reference/usage.md | 3 +- docs/reference/verify_gguf_subcommand-tr.md | 4 +- docs/reference/verify_gguf_subcommand.md | 4 +- docs/usermanuals/en/deployment/gguf-export.md | 72 +++++++----- docs/usermanuals/en/deployment/verify-gguf.md | 8 +- .../en/getting-started/installation.md | 11 +- docs/usermanuals/en/reference/cli.md | 2 +- docs/usermanuals/tr/deployment/gguf-export.md | 72 +++++++----- docs/usermanuals/tr/deployment/verify-gguf.md | 8 +- .../tr/getting-started/installation.md | 11 +- docs/usermanuals/tr/reference/cli.md | 2 +- forgelm/export.py | 68 +++++++++-- forgelm/verify/_gguf.py | 3 +- notebooks/post_training_workflow.ipynb | 6 +- pyproject.toml | 6 +- tests/test_export.py | 110 ++++++++++++++++++ 20 files changed, 323 insertions(+), 92 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c86b509b..b3ab9531 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,11 @@ All notable changes to ForgeLM are documented here. ### Added +- `forgelm export` now writes `.sha256` next to the GGUF (`sha256sum` format), the file `forgelm verify-gguf` + and the documentation always expected. It had never been written, so a sidecar-based check had nothing to compare. + A sidecar that cannot be written fails the export (exit 2) instead of leaving a GGUF without it; for a K-quant + request it belongs to the intermediate `*.f16.gguf`. A `model_integrity.json` that cannot be updated is now logged + as a warning, not at debug level (#84). - `CITATION.cff`, so the repository can be cited through GitHub's "Cite this repository" panel (#964). - A contributor can claim a `good first issue` by commenting `/assign` and give it @@ -74,6 +79,17 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Correct the GGUF pages (English and Turkish): the export example shows the real output, K-quants are described as + the two-step operation they are, the invented "manifest" and `verify-gguf` output (vocabulary and tokenizer checks that + do not exist) are replaced by the real ones, a clean `verify-gguf` exit is said to prove only what ran, and the sidecar + is described as corruption detection, not tamper evidence (#83, #84, #85). +- Stop saying that `pip install 'forgelm[export]'` brings the GGUF converter. `forgelm export` runs llama.cpp's `convert_hf_to_gguf.py`; + no `llama-cpp-python` wheel puts it where ForgeLM searches (0.2.90 installs it under `bin/`, 0.3.x does not ship it), so a plain + install ends with `convert_hf_to_gguf.py not found inside the llama-cpp-python installation` (exit 2) unless `FORGELM_GGUF_CONVERTER` + points at the script in a llama.cpp checkout. The installation page, the CLI reference, the usage and quickstart guides, the + post-training notebook, the `export` extra's comment in `pyproject.toml`, the module docstring and the error message (which advised + `pip install 'llama-cpp-python>=0.2.90'`) now say so. Searching the locations real wheels use, or bundling the script, is a code + change and stays open (#113). - Correct claims in the roadmap and strategy documents: `forgelm chat` does not screen for safety and reads no YAML config (#91), the live risk rows no longer name a past "August 2026" deadline (#75), and `docs/product_strategy.md` (English and Turkish) names the real `evaluation.auto_revert` field instead of a diff --git a/docs/guides/quickstart-tr.md b/docs/guides/quickstart-tr.md index 43b6ab2a..e239e28c 100644 --- a/docs/guides/quickstart-tr.md +++ b/docs/guides/quickstart-tr.md @@ -204,7 +204,8 @@ Eğitilmiş modelinizle doğrudan etkileşim kurun ve deploy edin: forgelm chat ./checkpoints/final_model # GGUF'a export (Ollama, LM Studio, llama.cpp için) -# Gerekli: pip install forgelm[export] +# Gerekli: pip install forgelm[export] VE FORGELM_GGUF_CONVERTER'ın bir llama.cpp checkout'undaki +# convert_hf_to_gguf.py'ye işaret etmesi (hiçbir llama-cpp-python wheel'i onu ForgeLM'in baktığı yere koymaz) forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m # Deployment config'leri üret (sunucu başlatılmaz) diff --git a/docs/guides/quickstart.md b/docs/guides/quickstart.md index d91cb21d..f95362fe 100644 --- a/docs/guides/quickstart.md +++ b/docs/guides/quickstart.md @@ -191,7 +191,8 @@ Interact with and deploy your trained model directly: forgelm chat ./checkpoints/final_model # Export to GGUF (for Ollama, LM Studio, llama.cpp) -# Requires: pip install forgelm[export] +# Requires: pip install forgelm[export] AND FORGELM_GGUF_CONVERTER pointing at convert_hf_to_gguf.py +# from a llama.cpp checkout (no llama-cpp-python wheel installs it where ForgeLM looks) forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m # Generate deployment configs (no server is started) diff --git a/docs/reference/usage-tr.md b/docs/reference/usage-tr.md index 9ebfa152..7bd9fa34 100644 --- a/docs/reference/usage-tr.md +++ b/docs/reference/usage-tr.md @@ -128,7 +128,8 @@ forgelm chat ./checkpoints/final_model --adapter ./adapter forgelm chat ./checkpoints/final_model --system "Yardımcı bir asistansın." --temperature 0.8 # GGUF'a export (Ollama, LM Studio, llama.cpp için) -# Gerekli: pip install forgelm[export] +# Gerekli: pip install forgelm[export] VE FORGELM_GGUF_CONVERTER'ın bir llama.cpp checkout'undaki +# convert_hf_to_gguf.py'ye işaret etmesi (hiçbir llama-cpp-python wheel'i onu ForgeLM'in baktığı yere koymaz) forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m forgelm export ./checkpoints/final_model --output model.gguf --quant q8_0 --adapter ./adapter diff --git a/docs/reference/usage.md b/docs/reference/usage.md index 5676a080..4b9f1187 100644 --- a/docs/reference/usage.md +++ b/docs/reference/usage.md @@ -145,7 +145,8 @@ forgelm chat ./checkpoints/final_model --adapter ./adapter forgelm chat ./checkpoints/final_model --system "You are a helpful assistant." --temperature 0.8 # Export to GGUF (for Ollama, LM Studio, llama.cpp) -# Requires: pip install forgelm[export] +# Requires: pip install forgelm[export] AND FORGELM_GGUF_CONVERTER pointing at convert_hf_to_gguf.py +# from a llama.cpp checkout (no llama-cpp-python wheel installs it where ForgeLM looks) forgelm export ./checkpoints/final_model --output model.gguf --quant q4_k_m forgelm export ./checkpoints/final_model --output model.gguf --quant q8_0 --adapter ./adapter diff --git a/docs/reference/verify_gguf_subcommand-tr.md b/docs/reference/verify_gguf_subcommand-tr.md index 0b092908..f53fe5dd 100644 --- a/docs/reference/verify_gguf_subcommand-tr.md +++ b/docs/reference/verify_gguf_subcommand-tr.md @@ -43,7 +43,7 @@ Kodlar `forgelm/cli/subcommands/_verify_gguf.py::_run_verify_gguf_cmd` tarafınd | **Meta veri bloğu** | İsteğe bağlı `gguf` paketi yüklüyse. Üst kaynak okuyucu ile meta veri + tensor tanımlarını ayrıştırır. | Okuyucu ayrıştırma sırasında istisna fırlatır → çıkış kodu sidecar'ın ne kanıtladığına bağlıdır; çünkü ayrıştırma hatası tek başına bozuk bir dosyayı, bu dosyanın format revizyonunu okuyamayacak kadar eski bir `gguf` paketinden ayırt edemez. **Sidecar yok** → çıkış `6` (bozulmayı eleyecek hiçbir şey yok; dosya kesilmiş / hasarlı sayılmalı). **Eşleşen sidecar** → çıkış `1` (dosya export edilenle bayt-bayt aynı — `gguf`'u yükseltip yeniden çalıştırın; artefakt sahibini çağırmayın). **Uyuşmayan sidecar** → çıkış `6`, sidecar uyuşmazlığı olarak raporlanır (checksum daha güçlü kanıttır ve baskın gelir). Paket yoksa → kontrol atlanır (magic + sidecar kontrolleri yük taşır). | | **SHA-256 sidecar** | `.sha256` mevcutsa. Dosyanın SHA-256'sını yeniden hesaplar ve sidecar'ın ilk boşluk-ayrılı token'ı ile karşılaştırır (sha256sum formatı ` *` desteklenir). | İyi-biçimli ama uyuşmayan digest → çıkış `6` (dosya export sonrası değişmiş). Sidecar mevcut ama içeriği 64 karakterlik hex digest değilse → çıkış `1` (bozuk-sidecar maskelenmesine karşı kapalı başarısızlık — hiçbir şey karşılaştırılmadı). Sidecar yoksa → kontrol sessizce atlanır. | -Exporter sidecar'ı varsayılan olarak yazar (bkz. [`docs/usermanuals/tr/deployment/gguf-export.md`](../usermanuals/tr/deployment/gguf-export.md)); GGUF dosyalarını üçüncü taraflardan alan operatörler sidecar'ı da talep etmelidir. +Exporter sidecar'ı varsayılan olarak yazar (K-quant isteğinde sidecar ara f16 dosyasına aittir; bkz. [`docs/usermanuals/tr/deployment/gguf-export.md`](../usermanuals/tr/deployment/gguf-export.md)); GGUF dosyalarını üçüncü taraflardan alan operatörler sidecar'ı da talep etmelidir. ## Emit edilen audit event'leri @@ -142,7 +142,7 @@ Aynı dosyanın yanında **hiç** sidecar yoksa karar bunun yerine `6` olur — ### İsteğe bağlı bağımlılık yok -`gguf` paketi yüklü değilse meta veri ayrıştırma katmanı sessizce atlanır — magic + sidecar kontrolleri yük taşımaya devam eder: +`gguf` paketi yüklü değilse meta veri ayrıştırma katmanı atlanır (`metadata_parsed: False`). O zaman denetlenen yalnızca magic ve sidecar'dır; sidecar da yoksa temiz bir çıkış yalnızca magic header'ı kanıtlar: ```shell $ pip uninstall -y gguf diff --git a/docs/reference/verify_gguf_subcommand.md b/docs/reference/verify_gguf_subcommand.md index c91506ca..55a970cf 100644 --- a/docs/reference/verify_gguf_subcommand.md +++ b/docs/reference/verify_gguf_subcommand.md @@ -43,7 +43,7 @@ The codes are emitted by `forgelm/cli/subcommands/_verify_gguf.py::_run_verify_g | **Metadata block** | When the optional `gguf` package is installed. Parses the metadata + tensor descriptors via the upstream reader. | Reader raises mid-parse → the exit code depends on what the sidecar can prove, because the parse error alone cannot tell a corrupted file apart from a `gguf` package too old for this file's format revision. **No sidecar** → exit `6` (nothing available to rule out corruption; treat the file as truncated / damaged). **Matching sidecar** → exit `1` (the file is byte-identical to what was exported — upgrade `gguf` and re-run; do not page the artifact owner). **Mismatching sidecar** → exit `6`, reported as the sidecar mismatch (the checksum is the stronger evidence and dominates). Package absent → check is skipped (the magic + sidecar checks remain load-bearing). | | **SHA-256 sidecar** | When `.sha256` exists. Recomputes file SHA-256 and compares against the sidecar's first whitespace-separated token (sha256sum format ` *` is supported). | Well-formed digest that disagrees → exit `6` (file changed after export). Sidecar present but contents are not a 64-character hex digest → exit `1` (fail closed against malformed-sidecar masquerade — nothing was compared). Sidecar absent → check is skipped silently. | -The exporter writes the sidecar by default (see [`docs/usermanuals/en/deployment/gguf-export.md`](../usermanuals/en/deployment/gguf-export.md)); operators receiving GGUF files from third parties should request the sidecar alongside. +The exporter writes the sidecar by default (for a K-quant request it belongs to the intermediate f16 file; see [`docs/usermanuals/en/deployment/gguf-export.md`](../usermanuals/en/deployment/gguf-export.md)); operators receiving GGUF files from third parties should request the sidecar alongside. ## Audit events emitted @@ -142,7 +142,7 @@ With **no** sidecar beside the same file the verdict is `6` instead — nothing ### Optional dependency absent -When the `gguf` package is not installed, the metadata-parse layer is skipped silently — the magic + sidecar checks remain load-bearing: +When the `gguf` package is not installed, the metadata-parse layer is skipped (`metadata_parsed: False`). Magic and sidecar are then all that is checked, and with no sidecar either, a clean exit proves only the magic header: ```shell $ pip uninstall -y gguf diff --git a/docs/usermanuals/en/deployment/gguf-export.md b/docs/usermanuals/en/deployment/gguf-export.md index 0303839f..a18f8c2f 100644 --- a/docs/usermanuals/en/deployment/gguf-export.md +++ b/docs/usermanuals/en/deployment/gguf-export.md @@ -5,21 +5,20 @@ description: Quantise to GGUF for local inference with llama.cpp, Ollama, or LM # GGUF Export -GGUF (GPT-Generated Unified Format) is the de facto file format for local CPU/GPU inference via `llama.cpp`. ForgeLM exports any fine-tuned checkpoint to GGUF with six quantisation levels and a SHA-256 manifest. +GGUF (GPT-Generated Unified Format) is the de facto file format for local CPU/GPU inference via `llama.cpp`. ForgeLM exports a fine-tuned checkpoint to GGUF and writes a SHA-256 sidecar next to the file. `f16` and `q8_0` come out of a single command; the K-quant levels (`q2_k`, `q3_k_m`, `q4_k_m`, `q5_k_m`) need one more `llama-quantize` step, described below. ## Quick example ```shell $ forgelm export ./checkpoints/customer-support \ --output model.gguf \ - --quant q4_k_m -✓ merged LoRA into base -✓ converted to GGUF -✓ quantised to q4_k_m (4.1 GB → 4.1 GB) -✓ wrote model.gguf and model.gguf.sha256 + --quant q8_0 +Export complete: model.gguf (quant=q8_0, sha256=3f2a91c0b4d7…) ``` -The output is a single `.gguf` file plus a `.sha256` manifest. +(Log output, abridged.) The result is `model.gguf` plus `model.gguf.sha256` next to it. The sidecar is in `sha256sum` format (` *model.gguf`), so both `sha256sum -c model.gguf.sha256` and [`forgelm verify-gguf`](#/deployment/verify-gguf) accept it. When the checkpoint directory already holds a `model_integrity.json`, ForgeLM also tries to record the digest there under `exported_artifacts` (best effort: if the file is not valid JSON or cannot be written, a warning is logged and the export still succeeds; `--no-integrity-update` skips this). + +The sidecar detects corruption and accidental change. It sits beside the file, so it is **not** tamper evidence: whoever can replace the GGUF can replace the sidecar too. If you need that, sign the file or keep the digest where the file's writer cannot reach it. ## Quantisation levels @@ -34,51 +33,68 @@ The output is a single `.gguf` file plus a `.sha256` manifest. `q4_k_m` is the sweet spot — fits easily on consumer hardware, minimal quality loss versus full precision. +### K-quants take one more step + +The converter ForgeLM calls writes `f16` and `q8_0` directly. For a K-quant (`q2_k`, `q3_k_m`, `q4_k_m` — the default — and `q5_k_m`) `forgelm export` writes an intermediate `f16` GGUF, named like the output with its `.gguf` suffix replaced by `.f16.gguf` (`model.gguf` becomes `model.f16.gguf`; an output without the suffix gets `.f16.gguf` appended), with its own `.sha256`. It reports the substitution (`requested_quant`, `manual_step_required` and the exact `followup_command` in `--output-format json`) and logs the `llama-quantize` command that finishes the job: + +```shell +$ forgelm export ./checkpoints/customer-support --output model.gguf --quant q4_k_m +Export complete: model.f16.gguf (quant=f16, sha256=9c1e47ab02d5…) +Manual quantization step required — run: llama-quantize model.f16.gguf model.gguf Q4_K_M +$ llama-quantize model.f16.gguf model.gguf Q4_K_M +$ sha256sum model.gguf > model.gguf.sha256 +``` + +`llama-quantize` comes from llama.cpp; ForgeLM does not install it. `forgelm export` writes no sidecar for the quantised file, so the last command creates one and `forgelm verify-gguf model.gguf` has a digest to compare. + GGUF export is always a separate, explicit `forgelm export` invocation — there is no config-driven auto-export toggle. `forgelm/config.py` has no `output:` block and no `gguf` field anywhere in the schema, so a training YAML cannot trigger export as a side effect of a passing eval gate; run `forgelm export` yourself once training finishes. ## Multi-quant export `forgelm export --quant` accepts a single value per invocation (one of `{q2_k, q3_k_m, q4_k_m, q5_k_m, q8_0, f16}`). To produce several -quantisations from one checkpoint, run the command once per quant: +quantisations from one checkpoint, run the command once per quant. For +the two levels the converter writes directly: ```shell -$ for q in q4_k_m q5_k_m q8_0; do +$ for q in q8_0 f16; do forgelm export ./checkpoints/run \ --output "./gguf/model.${q}.gguf" \ --quant "${q}" done -✓ wrote gguf/model.q4_k_m.gguf (4.1 GB) -✓ wrote gguf/model.q5_k_m.gguf (4.8 GB) -✓ wrote gguf/model.q8_0.gguf (7.2 GB) +Export complete: ./gguf/model.q8_0.gguf (quant=q8_0, sha256=3f2a91c0b4d7…) +Export complete: ./gguf/model.f16.gguf (quant=f16, sha256=b81d05e6c392…) ``` -The manifest: +Each run writes its own sidecar (`model.q8_0.gguf.sha256`, `model.f16.gguf.sha256`). For K-quants, run the loop with the K-quant names and finish each intermediate file with `llama-quantize`, as above. + +When the checkpoint directory holds a `model_integrity.json`, each run tries to append an entry to its `exported_artifacts` list (best effort, as above): ```json { - "base_model": "Qwen/Qwen2.5-7B-Instruct", - "fine_tune_run_id": "abc123", - "quants": { - "q4_k_m": {"size_bytes": 4100000000, "sha256": "..."}, - "q5_k_m": {"size_bytes": 4800000000, "sha256": "..."}, - "q8_0": {"size_bytes": 7200000000, "sha256": "..."} - }, - "exported_at": "2026-04-29T15:00:00Z" + "exported_artifacts": [ + {"file": "model.q8_0.gguf", "format": "gguf", "quant": "q8_0", "sha256": "…", "size_bytes": 7200000000}, + {"file": "model.f16.gguf", "format": "gguf", "quant": "f16", "sha256": "…", "size_bytes": 13000000000} + ] } ``` ## Verifying GGUF integrity ```shell -$ forgelm verify-gguf model.q4_k_m.gguf -✓ valid GGUF magic -✓ vocab size matches base -✓ sha256 matches manifest -✓ tokenizer round-trip OK +$ forgelm verify-gguf model.q8_0.gguf +OK: model.q8_0.gguf + GGUF magic OK, metadata parsed, SHA-256 sidecar match + magic_ok: True + metadata_parsed: True + sidecar_present: True + sidecar_match: True + tensor_count: 291 + sha256_actual: a4c1f2… + sha256_expected: a4c1f2… ``` -This catches corruption from bad transfers, silent disk errors, and the occasional `llama.cpp` upstream incompatibility. +This checks the GGUF magic header, parses the metadata block when the optional `gguf` package is installed, and compares the `.sha256` sidecar when it exists. It does not compare the vocabulary or the tokenizer with the base model. It catches corruption from bad transfers, silent disk errors and a file edited after export; it does not show who produced the file. [Verify GGUF](#/deployment/verify-gguf) says what each layer proves. ## Loading GGUF in popular tools @@ -117,7 +133,7 @@ The result is full-precision GGUF, ~13 GB for a 7B model. ## Common pitfalls :::warn -**Exporting LoRA without merging.** `forgelm export` always merges LoRA adapters into the base before quantising. If you want adapter-only inference, you don't want GGUF — use `forgelm chat` against the adapter directly or load with PEFT. +**Forgetting `--adapter`.** `forgelm export` merges a LoRA adapter into the base model only when you pass `--adapter `; without it, the model path must already be a full model directory. If you want adapter-only inference, you don't want GGUF — use `forgelm chat` against the adapter directly or load with PEFT. ::: :::warn diff --git a/docs/usermanuals/en/deployment/verify-gguf.md b/docs/usermanuals/en/deployment/verify-gguf.md index 8fecc66a..ddbf4fc7 100644 --- a/docs/usermanuals/en/deployment/verify-gguf.md +++ b/docs/usermanuals/en/deployment/verify-gguf.md @@ -5,11 +5,11 @@ description: Three-layer integrity check for exported GGUF model files before se # Verify GGUF -`forgelm verify-gguf` is the read-only verifier paired with the GGUF export step. It runs a three-layer integrity check on a GGUF model file: the 4-byte `GGUF` magic header, the metadata block (when the optional `gguf` Python package is installed), and a SHA-256 comparison against the `.sha256` sidecar (when present). Use it as the final pre-serve gate so a corrupted or tampered GGUF never reaches `llama.cpp`, Ollama, vLLM, or LM Studio. +`forgelm verify-gguf` is the read-only verifier paired with the GGUF export step. It runs a three-layer integrity check on a GGUF model file: the 4-byte `GGUF` magic header, the metadata block (when the optional `gguf` Python package is installed), and a SHA-256 comparison against the `.sha256` sidecar (when present). Use it as the final pre-serve gate so a corrupted or accidentally changed GGUF does not reach `llama.cpp`, Ollama, vLLM, or LM Studio. The sidecar sits next to the file, so this is not tamper evidence (see the pitfalls below). ## When to use it -- **Before loading a GGUF into a serving runtime.** A clean `verify-gguf` exit is the minimum signal the file is what the exporter wrote. +- **Before loading a GGUF into a serving runtime.** A clean `verify-gguf` exit proves only what ran: with neither the `gguf` package nor a `.sha256` sidecar it proves that the first four bytes are `GGUF` and nothing else, not that the file is complete or unmodified. - **In CI/CD release gates after GGUF export.** Run after every `forgelm export` step and **fail the release on any non-zero exit code** (the public contract is the `0/1/2/3/4/5/6` set; see the in-manual [Exit Codes](#/reference/exit-codes) reference. `1` is the most common gate trip for a bad path or malformed sidecar — nothing was compared, or what was compared came out clean — while `6` means the file *is* a GGUF and a comparison genuinely disagreed; `2` covers genuine I/O failures on existing files, and other non-zero codes propagate from upstream pipelines). - **When receiving a GGUF from a third-party trainer or model hub.** Recompute the magic + metadata + SHA-256 layers to detect drift between what they sent and what they signed. - **After moving a GGUF between machines.** Any byte-level corruption in transit shows up as a SHA-256 mismatch. @@ -100,7 +100,7 @@ Pipe to `jq` to filter on the `valid` flag without parsing the human-readable te | **Metadata block** | When the optional `gguf` Python package is installed. | The reader raised mid-parse. That alone cannot tell a truncated file apart from a `gguf` package too old for this file's format revision, so the verdict is decided by what the sidecar can prove — see [the three metadata outcomes](#metadata-parse-failure-depends-on-the-sidecar) below. | `6` (no sidecar) / `1` (matching sidecar) / `6` (mismatching sidecar) | | **SHA-256 sidecar** | When `.sha256` exists alongside the file. | Either the file was modified after export (a well-formed digest that disagrees — integrity failure), OR the sidecar itself is malformed (non-hex / wrong length / `TODO` placeholder — nothing was compared). The verifier fails closed on a malformed sidecar so a corrupted manifest can never silently masquerade as "verified". | `6` (modified) / `1` (malformed sidecar) | -The exporter writes the sidecar by default. When you receive a GGUF from a third party, request the sidecar alongside it; without one, only the magic + metadata layers run. +The exporter writes the sidecar by default (for a K-quant request it belongs to the intermediate f16 file; create one for the quantised file with `sha256sum model.gguf > model.gguf.sha256`). When you receive a GGUF from a third party, request the sidecar alongside it; without one, only the magic + metadata layers run. ### Metadata parse failure depends on the sidecar @@ -116,7 +116,7 @@ The middle row is the one to internalise in CI: exit `6` means "page whoever own ### Optional dependency: the `gguf` package -The metadata parse uses the upstream `gguf` Python package. When the package is not installed, the layer is skipped silently — the magic + sidecar checks remain load-bearing: +The metadata parse uses the upstream `gguf` Python package. When the package is not installed, the layer is skipped (the text output shows `metadata_parsed: False`). The magic and the sidecar are then all that is checked, and with no sidecar either, a clean exit means only that the file starts with `GGUF`: ```shell $ pip uninstall -y gguf diff --git a/docs/usermanuals/en/getting-started/installation.md b/docs/usermanuals/en/getting-started/installation.md index 591e25f0..a943d0f0 100644 --- a/docs/usermanuals/en/getting-started/installation.md +++ b/docs/usermanuals/en/getting-started/installation.md @@ -75,7 +75,16 @@ Adds MinHash LSH (via `datasketch`) for near-duplicate detection on corpora bigg $ pip install 'forgelm[export]' ``` -Adds quantized GGUF export for local inference (Ollama, llama.cpp). Pulls in the `gguf` writer and supporting libraries. Optional because not every workflow ends in GGUF — many users hand off to vLLM or TGI directly. +Adds quantized GGUF export for local inference (Ollama, llama.cpp). Pulls in `llama-cpp-python`. Optional because not every workflow ends in GGUF — many users hand off to vLLM or TGI directly. + +**The conversion script is not included.** `forgelm export` runs llama.cpp's `convert_hf_to_gguf.py` with ForgeLM's own Python. No `llama-cpp-python` wheel puts that script where ForgeLM looks, so with only the extra installed `forgelm export` stops with `convert_hf_to_gguf.py not found inside the llama-cpp-python installation` (exit 2). Clone llama.cpp and point `FORGELM_GGUF_CONVERTER` at the script in that checkout: + +```shell +$ git clone https://github.com/ggml-org/llama.cpp +$ export FORGELM_GGUF_CONVERTER="$PWD/llama.cpp/convert_hf_to_gguf.py" +``` + +Use the copy inside the checkout: current versions import sibling modules (`conversion`, `gguf`) that sit beside it, and the script needs the Python packages listed in the checkout's `requirements/` folder importable in the environment ForgeLM runs in. K-quant levels additionally need llama.cpp's `llama-quantize` binary (see [GGUF export](#/deployment/gguf-export)). ### Distributed training (`[distributed]`) diff --git a/docs/usermanuals/en/reference/cli.md b/docs/usermanuals/en/reference/cli.md index a9423316..e286fbb8 100644 --- a/docs/usermanuals/en/reference/cli.md +++ b/docs/usermanuals/en/reference/cli.md @@ -327,7 +327,7 @@ See [Exit Codes](#/reference/exit-codes) for the full contract. | `FORGELM_OPERATOR` | Operator identity recorded in audit events (overrides `getpass.getuser()@hostname`) | | `FORGELM_ALLOW_ANONYMOUS_OPERATOR` | When `1`, permit the audit log to record an anonymous operator (otherwise an unresolved identity is an error) | | `FORGELM_AUDIT_SECRET` | HMAC signing key for the audit log chain (enables tamper-detection) | -| `FORGELM_GGUF_CONVERTER` | Path to a custom `convert-hf-to-gguf.py` script | +| `FORGELM_GGUF_CONVERTER` | Path to `convert_hf_to_gguf.py` inside a llama.cpp checkout. In practice required: no `llama-cpp-python` wheel installs the script where ForgeLM searches (see [Installation](#/getting-started/installation)) | ## Common patterns diff --git a/docs/usermanuals/tr/deployment/gguf-export.md b/docs/usermanuals/tr/deployment/gguf-export.md index 5832e786..24c77d40 100644 --- a/docs/usermanuals/tr/deployment/gguf-export.md +++ b/docs/usermanuals/tr/deployment/gguf-export.md @@ -5,21 +5,20 @@ description: llama.cpp, Ollama veya LM Studio ile yerel inference için GGUF'a k # GGUF Export -GGUF (GPT-Generated Unified Format), `llama.cpp` üzerinden yerel CPU/GPU inference için fiili dosya formatıdır. ForgeLM herhangi bir fine-tuned checkpoint'i altı kuantizasyon seviyesi ve SHA-256 manifest ile GGUF'a export eder. +GGUF (GPT-Generated Unified Format), `llama.cpp` üzerinden yerel CPU/GPU inference için fiili dosya formatıdır. ForgeLM fine-tuned bir checkpoint'i GGUF'a export eder ve dosyanın yanına bir SHA-256 sidecar yazar. `f16` ve `q8_0` tek komutla çıkar; K-quant seviyeleri (`q2_k`, `q3_k_m`, `q4_k_m`, `q5_k_m`) aşağıda anlatılan bir `llama-quantize` adımı daha ister. ## Hızlı örnek ```shell $ forgelm export ./checkpoints/customer-support \ --output model.gguf \ - --quant q4_k_m -✓ LoRA base'e merge edildi -✓ GGUF'a dönüştürüldü -✓ q4_k_m'a kuantize edildi (4.1 GB → 4.1 GB) -✓ model.gguf ve model.gguf.sha256 yazıldı + --quant q8_0 +Export complete: model.gguf (quant=q8_0, sha256=3f2a91c0b4d7…) ``` -Çıktı tek `.gguf` dosyası artı `.sha256` manifest. +(Log çıktısı, kısaltılmış.) Sonuç `model.gguf` ve yanında `model.gguf.sha256`'dır. Sidecar `sha256sum` formatındadır (` *model.gguf`); bu yüzden hem `sha256sum -c model.gguf.sha256` hem de [`forgelm verify-gguf`](#/deployment/verify-gguf) onu kabul eder. Checkpoint dizininde zaten bir `model_integrity.json` varsa ForgeLM digest'i oraya da `exported_artifacts` altında yazmayı dener (best-effort: dosya geçerli JSON değilse ya da yazılamıyorsa bir uyarı loglanır ve export yine de başarılı olur; `--no-integrity-update` bunu atlar). + +Sidecar bozulmayı ve kazara değişikliği yakalar. Dosyanın yanında durduğu için kurcalama kanıtı **değildir**: GGUF'u değiştirebilen sidecar'ı da değiştirebilir. Buna ihtiyacınız varsa dosyayı imzalayın ya da digest'i dosyanın yazarının erişemediği bir yerde tutun. ## Kuantizasyon seviyeleri @@ -34,6 +33,20 @@ $ forgelm export ./checkpoints/customer-support \ `q4_k_m` tatlı nokta — consumer donanıma rahat sığar, full precision'a karşı minimum kalite kaybı. +### K-quant'lar bir adım daha ister + +ForgeLM'in çağırdığı dönüştürücü `f16` ve `q8_0`'ı doğrudan yazar. Bir K-quant için (`q2_k`, `q3_k_m`, `q4_k_m` — varsayılan — ve `q5_k_m`) `forgelm export`, kendi `.sha256`'sıyla birlikte, çıktı adındaki `.gguf` uzantısı `.f16.gguf` ile değiştirilerek adlandırılan (`model.gguf` → `model.f16.gguf`; bu uzantıyla bitmeyen bir çıktıya `.f16.gguf` eklenir) ara bir `f16` GGUF yazar. Değişikliği bildirir (`--output-format json`'da `requested_quant`, `manual_step_required` ve tam `followup_command`) ve işi bitiren `llama-quantize` komutunu loglar: + +```shell +$ forgelm export ./checkpoints/customer-support --output model.gguf --quant q4_k_m +Export complete: model.f16.gguf (quant=f16, sha256=9c1e47ab02d5…) +Manual quantization step required — run: llama-quantize model.f16.gguf model.gguf Q4_K_M +$ llama-quantize model.f16.gguf model.gguf Q4_K_M +$ sha256sum model.gguf > model.gguf.sha256 +``` + +`llama-quantize` llama.cpp'den gelir; ForgeLM onu kurmaz. `forgelm export` kuantize edilmiş dosya için sidecar yazmaz; son komut onu oluşturur ve `forgelm verify-gguf model.gguf` karşılaştıracak bir digest bulur. + GGUF export her zaman ayrı, açık bir `forgelm export` çağrısıdır — config-driven bir otomatik export toggle'ı yoktur. `forgelm/config.py`'de bir `output:` bloğu ve şemada hiçbir yerde `gguf` alanı yoktur, dolayısıyla bir eğitim YAML'ı eval kapısının geçmesinin yan etkisi olarak export'u tetikleyemez; eğitim bittiğinde `forgelm export`'u kendiniz çalıştırın. ## Çoklu-quant export @@ -41,45 +54,48 @@ GGUF export her zaman ayrı, açık bir `forgelm export` çağrısıdır — con `forgelm export --quant` invocation başına tek değer kabul eder (seçenekler: `{q2_k, q3_k_m, q4_k_m, q5_k_m, q8_0, f16}`). Tek bir checkpoint'ten birden çok quantisation üretmek için komutu her -quant için bir kez çalıştırın: +quant için bir kez çalıştırın. Dönüştürücünün doğrudan yazdığı iki +seviye için: ```shell -$ for q in q4_k_m q5_k_m q8_0; do +$ for q in q8_0 f16; do forgelm export ./checkpoints/run \ --output "./gguf/model.${q}.gguf" \ --quant "${q}" done -✓ gguf/model.q4_k_m.gguf yazıldı (4.1 GB) -✓ gguf/model.q5_k_m.gguf yazıldı (4.8 GB) -✓ gguf/model.q8_0.gguf yazıldı (7.2 GB) +Export complete: ./gguf/model.q8_0.gguf (quant=q8_0, sha256=3f2a91c0b4d7…) +Export complete: ./gguf/model.f16.gguf (quant=f16, sha256=b81d05e6c392…) ``` -Manifest: +Her çalıştırma kendi sidecar'ını yazar (`model.q8_0.gguf.sha256`, `model.f16.gguf.sha256`). K-quant'lar için döngüyü K-quant adlarıyla çalıştırın ve her ara dosyayı yukarıdaki gibi `llama-quantize` ile tamamlayın. + +Checkpoint dizininde bir `model_integrity.json` varsa her çalıştırma onun `exported_artifacts` listesine (yukarıdaki gibi best-effort) bir kayıt eklemeyi dener: ```json { - "base_model": "Qwen/Qwen2.5-7B-Instruct", - "fine_tune_run_id": "abc123", - "quants": { - "q4_k_m": {"size_bytes": 4100000000, "sha256": "..."}, - "q5_k_m": {"size_bytes": 4800000000, "sha256": "..."}, - "q8_0": {"size_bytes": 7200000000, "sha256": "..."} - }, - "exported_at": "2026-04-29T15:00:00Z" + "exported_artifacts": [ + {"file": "model.q8_0.gguf", "format": "gguf", "quant": "q8_0", "sha256": "…", "size_bytes": 7200000000}, + {"file": "model.f16.gguf", "format": "gguf", "quant": "f16", "sha256": "…", "size_bytes": 13000000000} + ] } ``` ## GGUF bütünlük doğrulama ```shell -$ forgelm verify-gguf model.q4_k_m.gguf -✓ valid GGUF magic -✓ vocab boyutu base ile eşleşiyor -✓ sha256 manifest ile eşleşiyor -✓ tokenizer round-trip OK +$ forgelm verify-gguf model.q8_0.gguf +OK: model.q8_0.gguf + GGUF magic OK, metadata parsed, SHA-256 sidecar match + magic_ok: True + metadata_parsed: True + sidecar_present: True + sidecar_match: True + tensor_count: 291 + sha256_actual: a4c1f2… + sha256_expected: a4c1f2… ``` -Bu kötü transferden bozulmaları, sessiz disk hatalarını ve ara sıra `llama.cpp` upstream uyumsuzluklarını yakalar. +Bu komut GGUF magic header'ını denetler, isteğe bağlı `gguf` paketi kuruluysa meta veri bloğunu ayrıştırır ve varsa `.sha256` sidecar'ını karşılaştırır. Kelime dağarcığını ya da tokenizer'ı base modelle karşılaştırmaz. Kötü transferden bozulmaları, sessiz disk hatalarını ve export sonrası düzenlenmiş bir dosyayı yakalar; dosyayı kimin ürettiğini göstermez. Her katmanın neyi kanıtladığı için [Verify GGUF](#/deployment/verify-gguf) sayfasına bakın. ## Yaygın araçlarda GGUF yükleme @@ -118,7 +134,7 @@ Sonuç full-precision GGUF, 7B model için ~13 GB. ## Sık hatalar :::warn -**LoRA'yı merge etmeden export.** `forgelm export` her zaman LoRA adapter'ı kuantizasyondan önce base'e merge eder. Adapter-only inference istiyorsanız GGUF istemiyorsunuz — adapter'a karşı doğrudan `forgelm chat` kullanın veya PEFT ile yükleyin. +**`--adapter`'ı unutmak.** `forgelm export` bir LoRA adapter'ı base modele yalnızca `--adapter ` verdiğinizde merge eder; vermezseniz model yolu zaten tam bir model dizini olmalıdır. Adapter-only inference istiyorsanız GGUF istemiyorsunuz — adapter'a karşı doğrudan `forgelm chat` kullanın veya PEFT ile yükleyin. ::: :::warn diff --git a/docs/usermanuals/tr/deployment/verify-gguf.md b/docs/usermanuals/tr/deployment/verify-gguf.md index de55e2a3..37383a20 100644 --- a/docs/usermanuals/tr/deployment/verify-gguf.md +++ b/docs/usermanuals/tr/deployment/verify-gguf.md @@ -5,11 +5,11 @@ description: llama.cpp, Ollama, vLLM ya da LM Studio üzerinden servis etmeden # GGUF Doğrulama -`forgelm verify-gguf`, GGUF export adımıyla eşleşen salt-okunur doğrulayıcıdır. Bir GGUF model dosyası üzerinde üç katmanlı bütünlük kontrolü yapar: 4-baytlık `GGUF` magic header, isteğe bağlı `gguf` Python paketi yüklüyse meta veri bloğu ve mevcutsa `.sha256` sidecar'ına karşı SHA-256 karşılaştırması. Bozulmuş veya tahrif edilmiş bir GGUF'un asla `llama.cpp`, Ollama, vLLM ya da LM Studio'ya ulaşmaması için son ön-servis kapısı olarak kullanın. +`forgelm verify-gguf`, GGUF export adımıyla eşleşen salt-okunur doğrulayıcıdır. Bir GGUF model dosyası üzerinde üç katmanlı bütünlük kontrolü yapar: 4-baytlık `GGUF` magic header, isteğe bağlı `gguf` Python paketi yüklüyse meta veri bloğu ve mevcutsa `.sha256` sidecar'ına karşı SHA-256 karşılaştırması. Bozulmuş veya kazara değişmiş bir GGUF'un `llama.cpp`, Ollama, vLLM ya da LM Studio'ya ulaşmaması için son ön-servis kapısı olarak kullanın. Sidecar dosyanın yanında durduğu için bu bir kurcalama kanıtı değildir (aşağıdaki tuzaklara bakın). ## Ne zaman kullanılır -- **Bir GGUF'u servis runtime'ına yüklemeden önce.** Temiz bir `verify-gguf` çıkışı, dosyanın exporter'ın yazdığı şey olduğuna dair minimum sinyaldir. +- **Bir GGUF'u servis runtime'ına yüklemeden önce.** Temiz bir `verify-gguf` çıkışı yalnızca çalışan kontrolü kanıtlar: ne `gguf` paketi ne de `.sha256` sidecar'ı varsa ilk dört baytın `GGUF` olduğunu kanıtlar, başka hiçbir şeyi değil; dosyanın eksiksiz ya da değiştirilmemiş olduğunu değil. - **GGUF export sonrası CI/CD yayın kapılarında.** Her `forgelm export` adımından sonra çalıştırın ve **sıfırdan farklı herhangi bir çıkış kodunda yayını başarısız sayın** (kamuya açık sözleşme `0/1/2/3/4/5/6` setidir; manuel içindeki [Çıkış Kodları](#/reference/exit-codes) referansına bakın. Bozuk bir yol veya bozuk sidecar için `1` en sık kapı tripi'dir — hiçbir şey karşılaştırılmadı ya da karşılaştırılan şey temiz çıktı — `6` ise dosyanın *gerçekten* bir GGUF olduğu ve bir karşılaştırmanın gerçekten uyuşmadığı anlamına gelir; `2` mevcut dosyalardaki gerçek I/O hatalarını kapsar, üst pipeline'lardan diğer sıfırdan farklı kodlar da iletilebilir). - **Üçüncü-taraf bir trainer'dan veya model hub'ından GGUF alındığında.** Gönderilen ile imzalanan arasındaki sapmayı tespit etmek için magic + meta veri + SHA-256 katmanlarını yeniden hesaplayın. - **Bir GGUF'u makineler arasında taşıdıktan sonra.** Aktarımda oluşan herhangi bir bayt-seviyesi bozulma SHA-256 uyuşmazlığı olarak ortaya çıkar. @@ -100,7 +100,7 @@ $ forgelm verify-gguf --output-format json \ | **Meta veri bloğu** | İsteğe bağlı `gguf` Python paketi yüklüyse. | Okuyucu ayrıştırma sırasında istisna fırlattı. Bu tek başına, kesilmiş bir dosyayı bu dosyanın format revizyonunu okuyamayacak kadar eski bir `gguf` paketinden ayırt edemez; dolayısıyla kararı sidecar'ın kanıtladığı şey belirler — aşağıdaki [üç meta veri sonucuna](#meta-veri-ayrıştırma-hatası-sidecara-bağlıdır) bakın. | `6` (sidecar yok) / `1` (eşleşen sidecar) / `6` (uyuşmayan sidecar) | | **SHA-256 sidecar** | Dosyanın yanında `.sha256` mevcutsa. | Ya dosya export sonrası değiştirildi (uyuşmayan iyi-biçimli bir digest — bütünlük arızası) YA DA sidecar'ın kendisi bozuk (hex değil / yanlış uzunluk / `TODO` placeholder — hiçbir şey karşılaştırılmadı). Doğrulayıcı bozuk sidecar'da kapalı başarısız olur; böylece bozuk bir manifest asla "doğrulanmış" gibi sessizce maskelenemez. | `6` (değiştirilmiş) / `1` (bozuk sidecar) | -Exporter sidecar'ı varsayılan olarak yazar. Üçüncü taraftan bir GGUF aldığınızda yanında sidecar'ı da talep edin; sidecar olmazsa yalnızca magic + meta veri katmanları çalışır. +Exporter sidecar'ı varsayılan olarak yazar (K-quant isteğinde sidecar ara f16 dosyasına aittir; kuantize edilmiş dosya için `sha256sum model.gguf > model.gguf.sha256` ile bir tane oluşturun). Üçüncü taraftan bir GGUF aldığınızda yanında sidecar'ı da talep edin; sidecar olmazsa yalnızca magic + meta veri katmanları çalışır. ### Meta veri ayrıştırma hatası sidecar'a bağlıdır @@ -116,7 +116,7 @@ CI'da içselleştirilmesi gereken satır ortadaki: çıkış `6` "artefakt sahib ### İsteğe bağlı bağımlılık: `gguf` paketi -Meta veri ayrıştırması üst kaynak `gguf` Python paketini kullanır. Paket yüklü değilse katman sessizce atlanır — magic + sidecar kontrolleri yük taşımaya devam eder: +Meta veri ayrıştırması üst kaynak `gguf` Python paketini kullanır. Paket yüklü değilse katman atlanır (metin çıktısı `metadata_parsed: False` gösterir). O zaman denetlenen yalnızca magic ve sidecar'dır; sidecar da yoksa temiz bir çıkış yalnızca dosyanın `GGUF` ile başladığı anlamına gelir: ```shell $ pip uninstall -y gguf diff --git a/docs/usermanuals/tr/getting-started/installation.md b/docs/usermanuals/tr/getting-started/installation.md index 5c5517a5..bdf196c9 100644 --- a/docs/usermanuals/tr/getting-started/installation.md +++ b/docs/usermanuals/tr/getting-started/installation.md @@ -75,7 +75,16 @@ $ pip install 'forgelm[ingestion-scale]' $ pip install 'forgelm[export]' ``` -Yerel inference için (Ollama, llama.cpp) kuantize GGUF export'u ekler. `gguf` writer'ı ve destekleyici kütüphaneleri kurar. Opsiyonel çünkü her iş akışı GGUF ile bitmez — birçok kullanıcı doğrudan vLLM veya TGI'a teslim eder. +Yerel inference için (Ollama, llama.cpp) kuantize GGUF export'u ekler. `llama-cpp-python`'ı kurar. Opsiyonel çünkü her iş akışı GGUF ile bitmez — birçok kullanıcı doğrudan vLLM veya TGI'a teslim eder. + +**Dönüştürme script'i pakete dahil değildir.** `forgelm export`, llama.cpp'nin `convert_hf_to_gguf.py` script'ini ForgeLM'in kendi Python'uyla çalıştırır. Hiçbir `llama-cpp-python` wheel'i o script'i ForgeLM'in aradığı yere koymaz; bu yüzden yalnızca extra kuruluyken `forgelm export` `convert_hf_to_gguf.py not found inside the llama-cpp-python installation` hatasıyla (exit 2) durur. llama.cpp'yi klonlayın ve `FORGELM_GGUF_CONVERTER`'ı o checkout'taki script'e yöneltin: + +```shell +$ git clone https://github.com/ggml-org/llama.cpp +$ export FORGELM_GGUF_CONVERTER="$PWD/llama.cpp/convert_hf_to_gguf.py" +``` + +Checkout'un içindeki kopyayı kullanın: güncel sürümler yanında duran kardeş modülleri (`conversion`, `gguf`) import eder ve script, checkout'un `requirements/` klasöründe listelenen Python paketlerinin ForgeLM'in çalıştığı ortamda import edilebilir olmasını ister. K-quant seviyeleri ayrıca llama.cpp'nin `llama-quantize` binary'sini ister ([GGUF export](#/deployment/gguf-export) sayfasına bakın). ### Dağıtık eğitim (`[distributed]`) diff --git a/docs/usermanuals/tr/reference/cli.md b/docs/usermanuals/tr/reference/cli.md index e1da7e23..eae80b15 100644 --- a/docs/usermanuals/tr/reference/cli.md +++ b/docs/usermanuals/tr/reference/cli.md @@ -328,7 +328,7 @@ Tam kontrat için bkz. [Exit Kodları](#/reference/exit-codes). | `FORGELM_OPERATOR` | Audit event'lerinde kaydedilen operatör kimliği (`getpass.getuser()@hostname`'i override eder) | | `FORGELM_ALLOW_ANONYMOUS_OPERATOR` | `1` olduğunda audit log'un anonim operatör kaydetmesine izin verir (aksi halde çözülemeyen kimlik hatası) | | `FORGELM_AUDIT_SECRET` | Audit log chain için HMAC imza anahtarı (tahrif tespitini açar) | -| `FORGELM_GGUF_CONVERTER` | Özel `convert-hf-to-gguf.py` script'inin yolu | +| `FORGELM_GGUF_CONVERTER` | Bir llama.cpp checkout'u içindeki `convert_hf_to_gguf.py`'nin yolu. Pratikte zorunlu: hiçbir `llama-cpp-python` wheel'i script'i ForgeLM'in aradığı yere koymaz ([Kurulum](#/getting-started/installation) sayfasına bakın) | ## Sık pattern'ler diff --git a/forgelm/export.py b/forgelm/export.py index 595a6c78..813b315c 100644 --- a/forgelm/export.py +++ b/forgelm/export.py @@ -1,7 +1,7 @@ """ GGUF export for post-training model distribution. -Wraps llama-cpp-python's ``convert_hf_to_gguf.py`` conversion script to +Runs llama.cpp's ``convert_hf_to_gguf.py`` conversion script to produce quantised GGUF files that can be served by Ollama, llama.cpp, and compatible runtimes. @@ -17,6 +17,7 @@ from __future__ import annotations +import contextlib import hashlib import logging import os @@ -30,7 +31,7 @@ SUPPORTED_FORMATS = frozenset({"gguf"}) SUPPORTED_QUANTS = frozenset({"q2_k", "q3_k_m", "q4_k_m", "q5_k_m", "q8_0", "f16"}) -# Name of the HF→GGUF conversion script bundled with llama-cpp-python. +# Name of the HF→GGUF conversion script (it lives in a llama.cpp checkout; no released llama-cpp-python wheel installs it where it is searched). _CONVERTER_SCRIPT = "convert_hf_to_gguf.py" # Quant types supported directly by convert_hf_to_gguf.py via --outtype. @@ -87,11 +88,13 @@ class ExportResult: def _find_converter_script() -> str: - """Locate llama-cpp-python's HF → GGUF conversion script. + """Locate the HF → GGUF conversion script. - Checks the ``FORGELM_GGUF_CONVERTER`` environment variable first, allowing - users to pin a specific converter script (useful when the bundled version is - outdated or when using a standalone llama.cpp build). + Checks the ``FORGELM_GGUF_CONVERTER`` environment variable first. In practice + that is the supported route: the script is not part of the llama-cpp-python + wheels (0.2.90 puts it in ``bin/``, 0.3.x does not ship it), so the + ``llama_cpp`` directories searched below are only a fallback for a layout that + no current wheel has. Point the variable at the copy inside a llama.cpp checkout. Raises: ImportError: When llama-cpp-python is not installed. @@ -141,9 +144,9 @@ def _find_converter_script() -> str: return path raise FileNotFoundError( - f"{_CONVERTER_SCRIPT} not found inside the llama-cpp-python installation. " - "Try: pip install 'llama-cpp-python>=0.2.90' " - f"Or set FORGELM_GGUF_CONVERTER=/path/to/{_CONVERTER_SCRIPT} to use a custom script." + f"{_CONVERTER_SCRIPT} not found inside the llama-cpp-python installation (no released wheel installs it " + "where ForgeLM looks). " + f"Set FORGELM_GGUF_CONVERTER=/path/to/llama.cpp/{_CONVERTER_SCRIPT}, the script inside a llama.cpp checkout." ) @@ -189,6 +192,10 @@ def _merge_adapter(model_path: str, adapter_path: str, merged_dir: str) -> None: # SHA-256 helper # --------------------------------------------------------------------------- +# Suffix of the digest file written next to the GGUF. ``forgelm verify-gguf`` looks for +# ``.sha256`` (``forgelm.verify._gguf._SIDECAR_SUFFIX``); tests/test_export.py keeps the two equal. +GGUF_SIDECAR_SUFFIX = ".sha256" + def _sha256_file(path: str) -> str: """Compute the SHA-256 digest of a file.""" @@ -199,6 +206,29 @@ def _sha256_file(path: str) -> str: return h.hexdigest() +def _write_sha256_sidecar(gguf_path: str, digest: str) -> str: + """Write ``.sha256`` in ``sha256sum`` binary-mode format (`` *``). + + The file is written next to the GGUF through a temporary name and renamed into place, so a reader never + sees a half-written digest. ``sha256sum -c`` accepts the result, and so does ``forgelm verify-gguf``. + The sidecar detects corruption and accidental change; it is not tamper evidence, because whoever can + replace the GGUF can replace the sidecar next to it. Returns the sidecar path. + """ + sidecar_path = gguf_path + GGUF_SIDECAR_SUFFIX + temporary_path = sidecar_path + ".tmp" + try: + with open(temporary_path, "w", encoding="utf-8") as fh: + fh.write(f"{digest} *{os.path.basename(gguf_path)}\n") + os.replace(temporary_path, sidecar_path) + except OSError: + with contextlib.suppress( + OSError + ): # best-effort tidy-up of our own temporary file; the original error is re-raised + os.remove(temporary_path) + raise + return sidecar_path + + # --------------------------------------------------------------------------- # Compliance integration # --------------------------------------------------------------------------- @@ -241,7 +271,7 @@ def _update_integrity_manifest(model_dir: str, export_result: ExportResult) -> N # ``json.JSONDecodeError`` since it subclasses ``ValueError``). # Updating the manifest is non-fatal: the artefact itself was already # exported successfully and its on-disk SHA-256 is recoverable. - logger.debug("Could not update model_integrity.json: %s", e) + logger.warning("Could not update model_integrity.json: %s", e) # --------------------------------------------------------------------------- @@ -407,7 +437,9 @@ def export_model( converter can exceed one hour; raise this for large models. Returns: - :class:`ExportResult` with SHA-256, file size, and output path. + :class:`ExportResult` with SHA-256, file size, and output path. The digest is also written next to the + GGUF as ``.sha256`` (``sha256sum`` format), which ``forgelm verify-gguf`` compares against. For + a K-quant request the file written is the intermediate f16 GGUF and the sidecar belongs to it. """ fmt = output_format.lower() quant = quant.lower() @@ -466,6 +498,20 @@ def export_model( digest = _sha256_file(actual_output_path) size_bytes = os.path.getsize(actual_output_path) + try: + _write_sha256_sidecar(actual_output_path, digest) + except OSError as e: + return ExportResult( + success=False, + output_path=actual_output_path, + format=fmt, + quant=actual_quant, + sha256=digest, + size_bytes=size_bytes, + error=f"GGUF written to {actual_output_path} but its SHA-256 sidecar could not be written: {e}", + error_kind="runtime", + ) + # Structured substitution signal: a K-quant request is served as an # intermediate f16 GGUF and needs a manual llama-quantize follow-up. Expose # the requested quant + the exact follow-up command so JSON/CI consumers can diff --git a/forgelm/verify/_gguf.py b/forgelm/verify/_gguf.py index 04c1f2de..68bb0811 100644 --- a/forgelm/verify/_gguf.py +++ b/forgelm/verify/_gguf.py @@ -192,7 +192,8 @@ def _maybe_parse_metadata(path: str) -> Dict[str, Any]: artefacts on a minimal install legitimately do not have it. Absent ``gguf`` package = ``parsed=False``, ``error=None`` and the caller treats this as "metadata check skipped" (the magic-header - + SHA-256-sidecar checks are the load-bearing integrity surface). + and SHA-256-sidecar checks then remain; with no sidecar either, a + clean result proves only the magic header). Raising ``ImportError`` here would break the subcommand for the optional-extra-not-installed path and contradict the project standard. Genuine corruption (file present but reader crashes) diff --git a/notebooks/post_training_workflow.ipynb b/notebooks/post_training_workflow.ipynb index 91e1171b..df3a7cf9 100644 --- a/notebooks/post_training_workflow.ipynb +++ b/notebooks/post_training_workflow.ipynb @@ -196,8 +196,10 @@ "source": [ "## 3. GGUF Export\n", "\n", - "`forgelm export` wraps `llama-cpp-python`'s bundled `convert_hf_to_gguf.py`\n", - "to produce GGUF files for Ollama / llama.cpp / compatible runtimes.\n", + "`forgelm export` runs llama.cpp's `convert_hf_to_gguf.py` to produce GGUF\n", + "files for Ollama / llama.cpp / compatible runtimes. The script is **not**\n", + "installed by `pip install 'forgelm[export]'`: clone llama.cpp and set\n", + "`FORGELM_GGUF_CONVERTER` to the copy in the checkout before running it.\n", "\n", "**K-quant note:** the script supports `f16` and `q8_0` directly. K-quants\n", "(`q2_k` / `q3_k_m` / `q4_k_m` / `q5_k_m`) need a **two-step** flow — ForgeLM\n", diff --git a/pyproject.toml b/pyproject.toml index 313b9bb6..adb6053f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -123,9 +123,11 @@ distributed = [ # benefit). Kept as a no-op extra so `pip install forgelm[merging]` still resolves. merging = [] -# GGUF export via llama-cpp-python's bundled convert_hf_to_gguf.py. +# GGUF export. forgelm export runs llama.cpp's convert_hf_to_gguf.py, which no +# llama-cpp-python wheel puts where ForgeLM searches: point FORGELM_GGUF_CONVERTER +# at the copy in a llama.cpp checkout (docs/usermanuals/en/getting-started/installation.md). # Produces f16 and q8_0 GGUF directly. K-quants (q4_k_m etc.) require a -# separate llama-quantize step; see docs/guides/export.md for the full flow. +# separate llama-quantize step; see docs/usermanuals/en/deployment/gguf-export.md for the full flow. # Linux and macOS only; Windows builds are available but less reliable. export = [ "llama-cpp-python>=0.2.90; sys_platform != 'win32'", diff --git a/tests/test_export.py b/tests/test_export.py index 319874bb..1e17a0f3 100644 --- a/tests/test_export.py +++ b/tests/test_export.py @@ -10,11 +10,13 @@ from unittest.mock import MagicMock, patch from forgelm.export import ( + GGUF_SIDECAR_SUFFIX, SUPPORTED_FORMATS, SUPPORTED_QUANTS, ExportResult, _sha256_file, _update_integrity_manifest, + _write_sha256_sidecar, export_model, ) @@ -69,6 +71,110 @@ def test_large_file_consistent(self, tmp_path): # --------------------------------------------------------------------------- +class TestSha256Sidecar: + """`forgelm export` writes `.sha256`, the file `forgelm verify-gguf` looks for (#84).""" + + @staticmethod + def _gguf(tmp_path, name="model.gguf", body=b"GGUF" + b"\x00" * 64): + path = tmp_path / name + path.write_bytes(body) + return str(path) + + def test_suffix_matches_what_verify_gguf_looks_for(self): + from forgelm.verify._gguf import _SIDECAR_SUFFIX + + assert GGUF_SIDECAR_SUFFIX == _SIDECAR_SUFFIX + + def test_sidecar_is_in_sha256sum_binary_format(self, tmp_path): + gguf = self._gguf(tmp_path) + digest = _sha256_file(gguf) + sidecar = _write_sha256_sidecar(gguf, digest) + assert sidecar == gguf + ".sha256" + assert open(sidecar, encoding="utf-8").read() == f"{digest} *model.gguf\n" + + # The fake file is not a parseable GGUF, so the metadata layer is switched off: these tests are about the sidecar, + # and must give the same answer whether or not the optional `gguf` package is installed. + _NO_METADATA = {"parsed": False, "error": None, "tensor_count": None} + + def test_verify_gguf_accepts_what_the_exporter_wrote(self, tmp_path): + from forgelm.verify._gguf import verify_gguf + + gguf = self._gguf(tmp_path) + _write_sha256_sidecar(gguf, _sha256_file(gguf)) + with patch("forgelm.verify._gguf._maybe_parse_metadata", return_value=self._NO_METADATA): + result = verify_gguf(gguf) + assert result.valid is True + assert result.checks["sidecar_present"] is True and result.checks["sidecar_match"] is True + + def test_verify_gguf_catches_a_change_after_export(self, tmp_path): + from forgelm.verify._gguf import verify_gguf + + gguf = self._gguf(tmp_path) + _write_sha256_sidecar(gguf, _sha256_file(gguf)) + with open(gguf, "ab") as fh: + fh.write(b"tampered") + with patch("forgelm.verify._gguf._maybe_parse_metadata", return_value=self._NO_METADATA): + result = verify_gguf(gguf) + assert result.valid is False and result.checks["sidecar_match"] is False + + def test_a_failed_write_leaves_no_temporary_file(self, tmp_path): + gguf = self._gguf(tmp_path) + with patch("forgelm.export.os.replace", side_effect=OSError("disk full")): + try: + _write_sha256_sidecar(gguf, "0" * 64) + except OSError: + pass + else: # pragma: no cover - the patched replace must raise + raise AssertionError("expected OSError") + assert not os.path.exists(gguf + ".sha256") and not os.path.exists(gguf + ".sha256.tmp") + + +class TestExportWritesTheSidecar: + def _run_export(self, tmp_path, quant, content=b"GGUF-mock-bytes"): + output_path = str(tmp_path / "model.gguf") + + def fake_run(cmd, **kwargs): + actual = cmd[cmd.index("--outfile") + 1] if "--outfile" in cmd else output_path + with open(actual, "wb") as f: + f.write(content) + result = MagicMock() + result.returncode, result.stderr, result.stdout = 0, "", "" + return result + + stub = MagicMock() + os.makedirs(str(tmp_path / "llama_cpp"), exist_ok=True) + stub.__file__ = str(tmp_path / "llama_cpp" / "__init__.py") + open(str(tmp_path / "llama_cpp" / "convert_hf_to_gguf.py"), "w").close() + with patch.dict(sys.modules, {"llama_cpp": stub}), patch("subprocess.run", side_effect=fake_run): + return output_path, export_model(str(tmp_path / "model"), output_path, quant=quant, update_integrity=False) + + def test_sidecar_sits_next_to_the_file_with_the_reported_digest(self, tmp_path): + output_path, result = self._run_export(tmp_path, "q8_0") + assert result.success is True and result.output_path == output_path + text = open(output_path + ".sha256", encoding="utf-8").read() + assert text == f"{result.sha256} *model.gguf\n" + + def test_k_quant_sidecar_belongs_to_the_intermediate_f16_file(self, tmp_path): + output_path, result = self._run_export(tmp_path, "q4_k_m") + assert result.success is True and result.output_path.endswith(".f16.gguf") + assert os.path.isfile(result.output_path + ".sha256") + assert not os.path.exists(output_path + ".sha256") # the requested q4_k_m file does not exist yet + + def test_a_sidecar_that_cannot_be_written_fails_the_export_loudly(self, tmp_path): + with patch("forgelm.export._write_sha256_sidecar", side_effect=OSError("read-only file system")): + _, result = self._run_export(tmp_path, "q8_0") + assert result.success is False and result.error_kind == "runtime" + assert "SHA-256 sidecar could not be written" in result.error and "read-only" in result.error + assert result.sha256 is not None # the file exists and its digest is still reported + + def test_a_manifest_that_cannot_be_updated_is_logged_as_a_warning(self, tmp_path, caplog): + (tmp_path / "model_integrity.json").write_text("{not json") + result = ExportResult(success=True, output_path=str(tmp_path / "m.gguf"), sha256="abc") + with caplog.at_level(logging.WARNING, logger="forgelm.export"): + _update_integrity_manifest(str(tmp_path), result) + assert any("Could not update model_integrity.json" in r.message for r in caplog.records) + + class TestUpdateIntegrityManifest: def test_updates_existing_manifest(self, tmp_path): integrity_path = tmp_path / "model_integrity.json" @@ -292,6 +398,10 @@ def test_missing_converter_script_in_package_is_runtime_error(self, tmp_path): assert result.success is False assert result.error_kind == "runtime" + # The hint must send the operator to the route that works (the script in a llama.cpp checkout), not to a + # reinstall of llama-cpp-python, whose wheels do not carry the script. + assert "FORGELM_GGUF_CONVERTER" in result.error and "llama.cpp checkout" in result.error + assert "pip install" not in result.error def test_all_supported_quants_accepted(self, tmp_path): """Every quant in SUPPORTED_QUANTS must pass format/quant validation.""" From ef41461516e2de9f9ae131636545a0f0e9e37713 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 12:09:35 +0300 Subject: [PATCH 09/22] fix: rope_scaling on transformers 5, hub telemetry, reserved retention.enforce, transformers CVE floor, FSDP strategies, purge output dir (#999) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Six code fixes combined into one pull request, replacing #984, #985, #987, #989, #990 and #992 (to be closed as superseded). Each fixes a case where the code failed, or a documented option did nothing, without saying so. Each was mutation-checked on its own; together they pass the full gauntlet. ## Related issues Refs #103, #95, #92, #98, #109, #104, #102. Not closed by this change: the top-of-range problems named in #109 (`warmup_ratio` on transformers 5.15+, the string `fsdp` form removed in 5.20), the FSDP2 migration (#104), and reading `training.output_dir` for the purge memorisation check (#102). ## Changes | # | Fix | Where | |---|---|---| | #984 | `training.rope_scaling` crashed every model load on transformers 5 (`KeyError: 'rope_type'`, or `rope_theta` lost). The requested keys are merged **over** the model's `rope_parameters` and the finished config is loaded; `sliding_window_attention` travels in the same config object. | `forgelm/model.py` | | #985 | The CLI sets `HF_HUB_DISABLE_TELEMETRY=1` (via `setdefault`) before `huggingface_hub` is imported, so TRL's trainer no longer sends Hugging Face's anonymous usage ping the privacy page says does not exist. Library use is not covered; the air-gap page says so. | `cli/_config_load.py`, `cli/_dispatch.py` | | #987 | `retention.enforce` was documented as a gate (`warn_on_excess`, `block_on_excess` → `EXIT_EVAL_FAILURE`) but no command reads it. Docs, field description, both wizards say it is reserved; the CLI warns when a config sets either of the last two. | `config.py` (in place), `cli/_config_load.py`, `wizard/_collectors.py` | | #989 | `transformers>=5.3.0` was called the CVE floor but admitted CVE-2026-5241 and CVE-2026-9856 (a crafted `chat_template` name writes outside the save directory). Floor is now `>=5.10.1` (5.10.0 is yanked), `huggingface_hub>=1.5.0`, the nightly floor job pins both, and tests keep it from drifting. | `pyproject.toml`, `nightly.yml`, `tests/` | | #990 | `fsdp_strategy: no_shard` / `hybrid_shard` passed `--dry-run` and exited 2 after the model loaded (FSDP2 rejects them). They are now rejected when the config loads (exit 1), and two knobs FSDP2 ignores (`fsdp_auto_wrap: false`, `backward_post`) log a warning. | `config.py` (in place), `cli/_config_load.py` | | #992 | `forgelm purge --row-id` without `--output-dir` silently pointed the memorisation check at the corpus directory. It now warns and names the path. The purge module's line budget is raised by 8 with a `budget_history` note (the project's documented option). | `cli/subcommands/_purge.py`, `tools/check_module_size.py` | `config.py` (0 lines of headroom on the module-size ratchet) and `trainer.py` change only by in-place string edits; new warnings live in `cli/_config_load.py`. ### Merge-conflict handling The six branches touched `CHANGELOG.md`, `tests/test_cli.py` (two appended test classes) and the import line in `cli/_dispatch.py`; all three were resolved by keeping both sides (the import line now imports `_disable_hub_telemetry` and `_warn_inert_fsdp_options`). The affected test modules pass together (621 tests). ## Type - [x] Bug fix - [x] Dependency ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. - Per-fix tests as in the original pull requests, including: a real tiny Llama on CPU for `rope_scaling` (linear, dynamic, yarn); the traversal reproduction (**fails on transformers 5.3.0**, passes on 5.10.1); all 32 accepted FSDP variants building real `TrainingArguments`. - transformers-facing test modules pass on a 5.10.1 / `huggingface_hub` 1.5.0 overlay (443 tests). The nightly minimum-versions job runs on its schedule after merge; I did not dispatch it. ## Checklist - [x] English and Turkish mirrors updated together - [x] CHANGELOG entries under `[Unreleased]` - [x] Tests added 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Harden CLI behavior and dependency compatibility across model loading, privacy, retention, distributed training, and purge workflows. Bug Fixes: - Restore Transformers 5 compatibility for RoPE scaling and sliding-window model configuration. - Disable Hugging Face Hub telemetry by default for CLI runs while preserving explicit operator opt-in. - Warn when reserved retention enforcement modes are configured instead of implying unsupported enforcement behavior. - Reject unsupported FSDP1-only strategies during configuration and warn about FSDP2 options that have no effect. - Warn when purge uses the corpus directory as its implicit output directory and clarify the memorisation-check scope. Enhancements: - Document the actual retention, FSDP, telemetry, and purge behavior in English and Turkish user-facing documentation. Build: - Raise the minimum runtime dependencies to transformers>=5.10.1 and huggingface_hub>=1.5.0 to address relevant security advisories. - Update the nightly minimum-version job and packaging checks to enforce the new dependency floors. Tests: - Add coverage for telemetry initialization, reserved retention warnings, FSDP validation, purge output-directory warnings, Transformers 5 RoPE handling, and chat-template path traversal protection. Chores: - Increase the purge module size budget for the new output-directory warning. --------- Co-authored-by: Claude Sonnet 5.5 --- .github/workflows/nightly.yml | 15 +- CHANGELOG.md | 38 +++++ config_template.yaml | 4 +- docs/guides/gdpr_erasure-tr.md | 8 +- docs/guides/gdpr_erasure.md | 8 +- docs/qms/risk_treatment_plan-tr.md | 2 +- docs/qms/risk_treatment_plan.md | 2 +- docs/reference/audit_event_catalog-tr.md | 2 +- docs/reference/audit_event_catalog.md | 2 +- docs/reference/configuration-tr.md | 16 +- docs/reference/configuration.md | 15 +- docs/reference/distributed_training-tr.md | 10 +- docs/reference/distributed_training.md | 10 +- docs/reference/purge_subcommand-tr.md | 4 +- docs/reference/purge_subcommand.md | 4 +- docs/reference/supply_chain_security-tr.md | 2 +- docs/reference/supply_chain_security.md | 2 +- .../usermanuals/en/compliance/gdpr-erasure.md | 4 +- .../en/getting-started/installation.md | 2 +- docs/usermanuals/en/operations/air-gap.md | 2 +- .../usermanuals/en/reference/configuration.md | 8 +- docs/usermanuals/en/training/distributed.md | 6 +- .../usermanuals/tr/compliance/gdpr-erasure.md | 4 +- .../tr/getting-started/installation.md | 2 +- docs/usermanuals/tr/operations/air-gap.md | 2 +- .../usermanuals/tr/reference/configuration.md | 8 +- docs/usermanuals/tr/training/distributed.md | 6 +- forgelm/cli/_config_load.py | 58 +++++++- forgelm/cli/_dispatch.py | 4 +- forgelm/cli/subcommands/_audit.py | 3 +- forgelm/cli/subcommands/_purge.py | 11 ++ forgelm/config.py | 26 ++-- forgelm/model.py | 44 +++++- forgelm/wizard/_collectors.py | 8 +- pyproject.toml | 8 +- tests/test_cli.py | 115 +++++++++++++++ tests/test_config.py | 11 ++ tests/test_distributed.py | 114 +++++++++++++- tests/test_gdpr_erasure.py | 50 +++++++ tests/test_model.py | 139 +++++++++++++++++- tests/test_packaging.py | 79 ++++++++++ tools/check_module_size.py | 6 +- tools/pip_audit_ignores.yaml | 3 + 43 files changed, 764 insertions(+), 103 deletions(-) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index ae7d4821..32b9e699 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -237,14 +237,17 @@ jobs: # EXPLICITLY so this job tests the true documented minimum — trl puts # no upper bound on transformers, so an unpinned resolve would silently # pull the newest 5.x and stop exercising the floor. transformers - # 5.3.0 is the CVE-2026-4372 fix floor and requires torch>=2.4 / - # huggingface_hub>=1.3 / tokenizers>=0.22; trl 1.0.0 pulls the - # accelerate>=1.4 and datasets>=4.7 floor. Keep 1:1 with the pyproject - # [project].dependencies minimums. + # 5.10.1 is the first installable release past CVE-2026-4372, + # CVE-2026-5241 and CVE-2026-9856 (5.10.0, which carries the last of + # them, was yanked); it requires huggingface_hub>=1.5 / + # tokenizers>=0.22, and ForgeLM itself needs torch>=2.4; trl 1.0.0 + # pulls the accelerate>=1.4 and datasets>=4.7 floor. Keep 1:1 with the + # pyproject [project].dependencies minimums (tests/test_packaging.py + # compares them). pip install \ "torch==2.4.0" \ - "transformers==5.3.0" \ - "huggingface_hub==1.3.0" \ + "transformers==5.10.1" \ + "huggingface_hub==1.5.0" \ "peft==0.19.0" \ "accelerate==1.4.0" \ "datasets==4.7.0" \ diff --git a/CHANGELOG.md b/CHANGELOG.md index b3ab9531..c224ea95 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,14 @@ All notable changes to ForgeLM are documented here. ### Changed +- Raise the `transformers` floor from `>=5.3.0` to `>=5.10.1` (and `huggingface_hub` from `>=1.3.0` to `>=1.5.0`, which + transformers 5.10 requires). `5.3.0` was described as the CVE fix floor but still admitted CVE-2026-5241 (a LightGlue + `config.json` overrides `trust_remote_code: false`, fixed in 5.5.0) and CVE-2026-9856 (a `chat_template` name in + `tokenizer_config.json` is used unsanitised as a file name by `save_pretrained`, so `final_model/` saves, GGUF export + and merges could write outside the output directory; fixed in 5.10.0, which PyPI yanked, hence 5.10.1). The nightly + minimum-versions job pins the new floor, and `tests/test_packaging.py` keeps it from drifting from + `pyproject.toml` and checks that a traversal `chat_template` name cannot escape on save. Environments pinned to + transformers 5.3 to 5.9 must upgrade (#109). - Record monitoring default provenance in `defaulted_fields` and mark literal URL projection with `endpoint_redacted` (also true when the URL already contains only scheme/host/port). Literal endpoints retain scheme/host/port in JSON evidence, including IPv6 brackets; userinfo, path, @@ -79,6 +87,36 @@ All notable changes to ForgeLM are documented here. ### Fixed +- `training.rope_scaling` works on transformers 5 again. The setting was passed to `from_pretrained(rope_scaling=...)`, + which no longer works: the documented `{type: linear, factor: 4}` raised `KeyError: 'rope_type'` while the model was + built, and `{rope_type: ..., factor: ...}` replaced the whole parameter dict so `rope_theta` was lost (`TypeError`). + The requested keys are now merged over the model's own `rope_parameters` (the legacy key `type` is read as + `rope_type`), applied to the text config of a multimodal checkpoint, and the finished config is loaded. The + `sliding_window_attention` override travels in the same config object. Verified on transformers 5.13.0 for + `linear`, `dynamic` and `yarn` on a tiny local Llama; not exercised on the 5.3-5.9 range the floor still admits (#103). +- The `forgelm` CLI no longer sends Hugging Face's anonymous usage ping. TRL's trainers call `send_telemetry` when they are + built, and `huggingface_hub` sends it unless `HF_HUB_DISABLE_TELEMETRY` is true or offline mode is on, which contradicted the + privacy page ("does not send usage data anywhere"). `main()` now sets `HF_HUB_DISABLE_TELEMETRY=1` before anything imports + `huggingface_hub`, and an operator's own value wins (`HF_HUB_DISABLE_TELEMETRY=0` opts back in). Using ForgeLM as a library + does not set it (#95). +- `retention.enforce` is documented for what it is: reserved. No command reads it, so `warn_on_excess` and + `block_on_excess` never warned or blocked anything and nothing was deleted automatically; retention is only reported, + read-only, by `forgelm purge --check-policy` (always exit 0). The config description, the wizard prompt, the + configuration reference, the GDPR erasure guide and `config_template.yaml` now say so (English and Turkish), and + the CLI logs a warning when a config sets a non-default `enforce` instead of staying silent (#92, #98). +- `distributed.fsdp_strategy: no_shard` and `hybrid_shard` are rejected when the config loads (exit 1). They were documented, passed `--dry-run`, + and then failed with exit 2 at trainer construction (`fsdp=hybrid_shard is only available with FSDP1`), because transformers 5.10+ runs + FSDP2 by default. `fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted but change nothing under FSDP2; + ForgeLM now logs a warning when either is set, and the field descriptions, reference pages, user manuals and `config_template.yaml` + say so. Moving to FSDP2 settings (`fsdp=True` plus an explicit `fsdp_config`) is not part of this change; the string form is + deprecated in transformers and slated for removal in v5.20 (#104). +- `forgelm purge --row-id` without `--output-dir` now logs a warning that names the directory it resolved (the corpus's) + and the `final_model/` the memorisation check will look for. The documentation promised that warning but the code never + logged it, and a default run looks for a model next to the corpus, where a training run rarely writes one, so it finished + with a completed-erasure record and no memorisation event. The audit-event catalogue, the purge reference, the GDPR + erasure guide, the user manual and the risk-treatment plan now state the real condition: a `final_model/` directly under + `--output-dir`, not "any run that consumed the corpus" (English and Turkish). Reading `training.output_dir` from + `--config` is still open (#102). - Correct the GGUF pages (English and Turkish): the export example shows the real output, K-quants are described as the two-step operation they are, the invented "manifest" and `verify-gguf` output (vocabulary and tokenizer checks that do not exist) are replaced by the real ones, a clean `verify-gguf` exit is said to prove only what ran, and the sidecar diff --git a/config_template.yaml b/config_template.yaml index 8aec1398..acf197cb 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -167,7 +167,7 @@ data: # strategy: "deepspeed" # "deepspeed" or "fsdp" # deepspeed_config: "zero2" # preset: "zero2", "zero3", "zero3_offload" or path to JSON # # --- FSDP options (when strategy: "fsdp") --- -# # fsdp_strategy: "full_shard" # "full_shard", "shard_grad_op", "no_shard", "hybrid_shard" +# # fsdp_strategy: "full_shard" # "full_shard" or "shard_grad_op" (FSDP2 supports no other) # # fsdp_auto_wrap: true # # fsdp_offload: false @@ -276,7 +276,7 @@ data: # staging_ttl_days: 7 # final_model.staging./ TTL after `forgelm reject` # ephemeral_artefact_retention_days: 90 # compliance bundles + audit reports # raw_documents_retention_days: 90 # ingested raw docs (PDF/DOCX/EPUB/TXT/MD) -# enforce: "log_only" # "log_only" | "warn_on_excess" | "block_on_excess" +# enforce: "log_only" # reserved: no command acts on it yet ("log_only" | "warn_on_excess" | "block_on_excess") # Phase 14 — Multi-stage training pipeline chains (SFT → DPO → GRPO). # Uncomment the ``pipeline:`` block below to run multiple training stages diff --git a/docs/guides/gdpr_erasure-tr.md b/docs/guides/gdpr_erasure-tr.md index 64a1bb16..7fa432c1 100644 --- a/docs/guides/gdpr_erasure-tr.md +++ b/docs/guides/gdpr_erasure-tr.md @@ -19,7 +19,7 @@ **Yapmaz:** -- Modelleri yeniden eğitmez. Bir satırı corpus'tan kaldırmak onu zaten eğitilmiş ağırlıklardan unutturmaz — silinmiş satır olmadan tam yeniden eğitim tek doğru çözüm. Tool, corpus'u tüketen herhangi bir koşum için `final_model/` mevcut olduğunda `data.erasure_warning_memorisation` yayar; böylece operatör boşluğu görür. +- Modelleri yeniden eğitmez. Bir satırı corpus'tan kaldırmak onu zaten eğitilmiş ağırlıklardan unutturmaz — silinmiş satır olmadan tam yeniden eğitim tek doğru çözüm. Tool, `--output-dir` altında doğrudan bir `final_model/` dizini mevcut olduğunda `data.erasure_warning_memorisation` yayar (`--output-dir` varsayılan olarak corpus'un dizinidir; bu yüzden eğitim çıktı dizininizi verin); böylece operatör boşluğu görür. Corpus'u hangi koşumların tükettiğini ayırt edemez. - Audit log'un kendisinden entry silmez. Madde 17(3)(b) audit / muhasebe kayıtlarını bir hukuki yükümlülük savunması olarak korur; tool silmeyi geçmişi yeniden yazmak yerine yeni bir event olarak kaydeder. - Downstream tüketicilere silme bildirimi göndermez. Webhook receiver'ları, dataset mirror'ları, deploy edilmiş model endpoint'leri ve yayınlanmış checkpoint'ten türetilmiş üçüncü-taraf fine-tune'lar operatörün runtime-katmanı sorumluluğudur (Madde 17(2)). Tool yüklü config webhook block'u içerdiğinde `data.erasure_warning_external_copies` yayar; böylece audit log'u sorgulayan downstream tüketici açık hatırlatmayı görür. - Yedekleri silmez. Operatörün `` sınırı dışındaki replikalar, snapshot'lar ve storage backup'lar altyapı tarafı endişelerdir. @@ -78,10 +78,10 @@ retention: staging_ttl_days: 7 # `forgelm reject` üzerine aksiyon almak için bir iş haftası ephemeral_artefact_retention_days: 90 # üç-aylık review cadence'i raw_documents_retention_days: 90 # data audit yeniden çalıştırmadan önce ingestion penceresi - enforce: log_only # log_only / warn_on_excess / block_on_excess + enforce: log_only # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only / warn_on_excess / block_on_excess) ``` -Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). `enforce`, trainer pre-flight gate'inin ihlallere nasıl tepki vereceğini kontrol eder. +Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi devre dışı bırakır (sınırsız tut). Horizon'lar `forgelm purge --check-policy` ile raporlanır (salt-okunur, her zaman 0 ile çıkar); hiçbir şey otomatik silinmez ya da engellenmez. `enforce` ayrılmıştır: henüz hiçbir komut onu kullanmaz, bu yüzden üç değer de `log_only` gibi davranır. ## Kaldırıldı: `evaluation.staging_ttl_days` @@ -96,7 +96,7 @@ Herhangi bir horizon'u `0` olarak ayarlamak o artefact kind'ı için policy'yi d | `data.erasure_requested` | Herhangi bir `forgelm purge --row-id` / `--run-id` çağrısının ilk adımı, herhangi bir silmeden önce (`--check-policy` salt-okunur; audit event yaymaz) | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (row mode'da hash'lenmiş), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | | `data.erasure_completed` | Başarılı silme bittiğinde | Tüm `requested` field'ları + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | | `data.erasure_failed` | Disk operasyonu raise etti VEYA eşleşen satır/koşum bulunamadı VEYA çoklu-satır policy belirsizliği reddetti | Tüm `requested` field'ları + `error_class`, `error_message` | -| `data.erasure_warning_memorisation` | **Yalnız row modunda.** Row erasure × corpus'u tüketen herhangi bir koşum için `final_model/` mevcut. (Run modu memorisation uyarısı emit etmez — run-scoped `staging` / `artefacts` türleri tanım gereği promote-öncesi.) | Tüm `completed` field'ları + `affected_run_ids` | +| `data.erasure_warning_memorisation` | **Yalnız row modunda.** Row erasure × `--output-dir` altında doğrudan bir `final_model/` dizini mevcut. (Run modu memorisation uyarısı emit etmez — run-scoped `staging` / `artefacts` türleri tanım gereği promote-öncesi.) | Tüm `completed` field'ları + `affected_run_ids` | | `data.erasure_warning_synthetic_data_present` | **Yalnız row modunda.** Row erasure × `output_dir`'de `synthetic_data*.jsonl` mevcut. (Run modu bu uyarıyı yüzeylemez — synthetic veri corpus-scoped'dur, run-scoped değil.) | Tüm `completed` field'ları + `synthetic_files` | | `data.erasure_warning_external_copies` | Yüklü config webhook block'u içeriyor | Tüm `completed` field'ları + `webhook_targets` | diff --git a/docs/guides/gdpr_erasure.md b/docs/guides/gdpr_erasure.md index 24941879..fd4f7e21 100644 --- a/docs/guides/gdpr_erasure.md +++ b/docs/guides/gdpr_erasure.md @@ -19,7 +19,7 @@ **Does not:** -- Re-train models. Removing a row from the corpus does not unmemorise it from already-trained weights — full retraining without the erased row is the only proper mitigation. The tool emits `data.erasure_warning_memorisation` when a `final_model/` exists for any run that consumed the corpus, so the operator sees the gap. +- Re-train models. Removing a row from the corpus does not unmemorise it from already-trained weights — full retraining without the erased row is the only proper mitigation. The tool emits `data.erasure_warning_memorisation` when a `final_model/` directory exists directly under `--output-dir` (which defaults to the corpus's directory, so pass your training output directory), so the operator sees the gap. It cannot tell which runs consumed the corpus. - Delete entries from the audit log itself. Article 17(3)(b) preserves audit / accounting records as a legal-obligation defence; the tool records the erasure as a new event rather than rewriting history. - Push erasure notices to downstream consumers. Webhook receivers, dataset mirrors, deployed model endpoints, and third-party fine-tunes derived from a published checkpoint are the operator's runtime-layer responsibility (Article 17(2)). The tool emits `data.erasure_warning_external_copies` when the loaded config has a webhook block, so a downstream consumer querying the audit log sees the explicit reminder. - Erase backups. Replicas, snapshots, and storage backups outside the operator's `` are infrastructure-side concerns. @@ -78,10 +78,10 @@ retention: staging_ttl_days: 7 # one work-week to act on a `forgelm reject` ephemeral_artefact_retention_days: 90 # quarterly review cadence raw_documents_retention_days: 90 # ingestion-window before re-running data audit - enforce: log_only # log_only / warn_on_excess / block_on_excess + enforce: log_only # reserved: no command acts on it yet (log_only / warn_on_excess / block_on_excess) ``` -Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). `enforce` controls how the trainer pre-flight gate reacts to violations. +Setting any horizon to `0` disables the policy for that artefact kind (retain indefinitely). The horizons are reported by `forgelm purge --check-policy` (read-only, always exits 0); nothing is deleted or blocked automatically. `enforce` is reserved: no command acts on it yet, so all three values behave like `log_only`. ## Removed: `evaluation.staging_ttl_days` @@ -96,7 +96,7 @@ Six new events ship with `forgelm purge` (catalogued in `docs/reference/audit_ev | `data.erasure_requested` | First step of any `forgelm purge --row-id` / `--run-id` invocation, before any deletion (`--check-policy` is read-only and emits no audit events) | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (hashed in row mode), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | | `data.erasure_completed` | Successful deletion finishes | All `requested` fields + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | | `data.erasure_failed` | Disk operation raised, OR no matching row/run found, OR multi-row policy refused on ambiguity | All `requested` fields + `error_class`, `error_message` | -| `data.erasure_warning_memorisation` | **Row mode only.** Row erasure × `final_model/` exists for any run that consumed the corpus. (Run mode does not emit memorisation warnings — the run-scoped `staging` / `artefacts` kinds are by definition pre-promotion.) | All `completed` fields + `affected_run_ids` | +| `data.erasure_warning_memorisation` | **Row mode only.** Row erasure × a `final_model/` directory exists directly under `--output-dir`. (Run mode does not emit memorisation warnings — the run-scoped `staging` / `artefacts` kinds are by definition pre-promotion.) | All `completed` fields + `affected_run_ids` | | `data.erasure_warning_synthetic_data_present` | **Row mode only.** Row erasure × `synthetic_data*.jsonl` exists in `output_dir`. (Run mode does not surface this warning — synthetic data is corpus-scoped, not run-scoped.) | All `completed` fields + `synthetic_files` | | `data.erasure_warning_external_copies` | Loaded config has a webhook block (both modes). | All `completed` fields + `webhook_targets` | diff --git a/docs/qms/risk_treatment_plan-tr.md b/docs/qms/risk_treatment_plan-tr.md index 1ac0e752..9570b600 100644 --- a/docs/qms/risk_treatment_plan-tr.md +++ b/docs/qms/risk_treatment_plan-tr.md @@ -90,7 +90,7 @@ ekler. |---|---| | Açıklama | Operatör `forgelm purge --row-id alice@example.com` corpusa karşı çalıştırır, ama haftalar önce satırı eğiten model öğrenilen ağırlıklardan onu hala üretebilir | | L × I (inherent) | High × Med = HIGH | -| Treatment | ForgeLM, silinen satırı tüketen koşum bir `final_model/` artefaktına sahipse `data.erasure_warning_memorisation` event'ı yayar — bu bir **detection** sinyalidir, mitigation değildir. Operatör residual riski kabul eder ve organizasyonel kontrolleri uygular (özne bildirimi, yüksek-stakes dağıtımlar için sıfırdan yeniden eğit, DPIA güncellemesi). | +| Treatment | ForgeLM, purge'ün `--output-dir`'i altında doğrudan bir `final_model/` varsa `data.erasure_warning_memorisation` event'ı yayar (satırı hangi koşumun tükettiğini bilmez ve varsayılan dizin eğitim çıktısı değil corpus'unkidir) — bu bir **detection** sinyalidir, mitigation değildir. Operatör residual riski kabul eder ve organizasyonel kontrolleri uygular (özne bildirimi, yüksek-stakes dağıtımlar için sıfırdan yeniden eğit, DPIA güncellemesi). | | Residual L × I | High × Med = HIGH (risk **kabul edildi**, bkz. §5) | | Sahip | Data Protection Officer (DPO) | | Review cadence | Madde 17 talebi başına | diff --git a/docs/qms/risk_treatment_plan.md b/docs/qms/risk_treatment_plan.md index c19a9d35..43258b00 100644 --- a/docs/qms/risk_treatment_plan.md +++ b/docs/qms/risk_treatment_plan.md @@ -89,7 +89,7 @@ section §4. |---|---| | Description | Operator runs `forgelm purge --row-id alice@example.com` against the corpus, but the model trained on the row weeks earlier may still reproduce it from learned weights | | L × I (inherent) | High × Med = HIGH | -| Treatment | ForgeLM emits `data.erasure_warning_memorisation` when the run that consumed the deleted row has a `final_model/` artefact — this is a **detection** signal, not a mitigation. Deployer accepts the residual risk and applies organisational controls (subject notification, retrain from scratch for high-stakes deployments, DPIA update). | +| Treatment | ForgeLM emits `data.erasure_warning_memorisation` when a `final_model/` exists directly under the purge's `--output-dir` (it does not know which run consumed the row, and the default directory is the corpus's, not the training output) — this is a **detection** signal, not a mitigation. Deployer accepts the residual risk and applies organisational controls (subject notification, retrain from scratch for high-stakes deployments, DPIA update). | | Residual L × I | High × Med = HIGH (risk **accepted**, see §5) | | Owner | Data Protection Officer (DPO) | | Review cadence | Per Article 17 request | diff --git a/docs/reference/audit_event_catalog-tr.md b/docs/reference/audit_event_catalog-tr.md index 3068d80d..051d863e 100644 --- a/docs/reference/audit_event_catalog-tr.md +++ b/docs/reference/audit_event_catalog-tr.md @@ -78,7 +78,7 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli | `data.erasure_requested` | Herhangi bir `forgelm purge --row-id` / `--run-id` çağrısının ilk adımı, herhangi bir silmeden ÖNCE. `--check-policy` salt-okunur; event yaymaz. | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (row mode'da hash'lenmiş), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | 17 | | `data.erasure_completed` | Başarılı silme tamamlandı. | Tüm `requested` field'ları + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | 17 | | `data.erasure_failed` | Disk operasyonu raise etti VEYA eşleşen satır/koşum bulunamadı VEYA çoklu-satır policy belirsizliği reddetti. | Tüm `requested` field'ları + `error_class`, `error_message` | 17 | -| `data.erasure_warning_memorisation` | Row erasure × bu corpus'u tüketen herhangi bir koşum için `final_model/` mevcut. | Tüm `completed` field'ları + `affected_run_ids` | 17 | +| `data.erasure_warning_memorisation` | Row erasure × `--output-dir` altında doğrudan bir `final_model/` dizini mevcut (birebir ad; `--output-dir` varsayılan olarak corpus'un dizinidir, bu yüzden eğitim çıktı dizininizi verin). ForgeLM corpus'u hangi koşumların tükettiğini belirlemez. | Tüm `completed` field'ları + `affected_run_ids` | 17 | | `data.erasure_warning_synthetic_data_present` | Row erasure × `output_dir`'de `synthetic_data*.jsonl` mevcut. | Tüm `completed` field'ları + `synthetic_files` | 17 | | `data.erasure_warning_external_copies` | Yüklü config boş-olmayan `webhook` block'u içeriyor; downstream tüketiciler bildirim almış olabilir. | Tüm `completed` field'ları + `webhook_targets` (redact'li URL'ler) | 17 | diff --git a/docs/reference/audit_event_catalog.md b/docs/reference/audit_event_catalog.md index a7a74abd..e5d05c32 100644 --- a/docs/reference/audit_event_catalog.md +++ b/docs/reference/audit_event_catalog.md @@ -78,7 +78,7 @@ The hash chain advances after the line lands on disk (`flush` + `fsync`), so an | `data.erasure_requested` | First step of any `forgelm purge --row-id` / `--run-id` invocation, BEFORE any deletion. `--check-policy` is read-only and emits no events. | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (hashed in row mode), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | 17 | | `data.erasure_completed` | Successful deletion finished. | All `requested` fields + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | 17 | | `data.erasure_failed` | Disk operation raised, OR no matching row/run found, OR multi-row policy refused on ambiguity. | All `requested` fields + `error_class`, `error_message` | 17 | -| `data.erasure_warning_memorisation` | Row erasure × `final_model/` exists for any run that consumed this corpus. | All `completed` fields + `affected_run_ids` | 17 | +| `data.erasure_warning_memorisation` | Row erasure × a `final_model/` directory exists directly under `--output-dir` (literal name; `--output-dir` defaults to the corpus's directory, so pass your training output directory). ForgeLM does not determine which runs consumed the corpus. | All `completed` fields + `affected_run_ids` | 17 | | `data.erasure_warning_synthetic_data_present` | Row erasure × `synthetic_data*.jsonl` exists in `output_dir`. | All `completed` fields + `synthetic_files` | 17 | | `data.erasure_warning_external_copies` | Loaded config has a non-empty `webhook` block; downstream consumers may have received notices. | All `completed` fields + `webhook_targets` (redacted URLs) | 17 | diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index b78fa07f..f9b88b43 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -270,10 +270,10 @@ training: Uyumluluk, eğitim ve değerlendirme artefaktları için saklama ufuklarını belirler. Ufuklar GDPR Madde 5(1)(e) "saklama sınırlaması" ve Madde 17 -"silme hakkı" tarihlerini onurlandırır. `enforce` anahtarı yalnız-loglama, -uyarı ve sert-engelleme modları arasında geçiş yaparak regüle edilen bir CI -kapısının saklama ufkunu eski bir çalışma alanını yeniden kullanarak sessizce -uzatmasını engeller. +"silme hakkı" tarihlerini onurlandırır. Ufuklar **raporlanır**, zorlanmaz: +`forgelm purge --check-policy` ufku aşanları listeler ve her zaman 0 ile çıkar; +hiçbir şey otomatik silinmez, engellenmez ya da zamanlanmaz. `enforce` anahtarı +ayrılmıştır: henüz hiçbir komut onu okumaz. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| @@ -281,7 +281,7 @@ uzatmasını engeller. | `staging_ttl_days` | int | `7` | `forgelm reject` kararından sonra `final_model.staging./` dizininin planlı temizlenmeden önce saklanacağı gün sayısı. `0` süresiz saklama anlamına gelir. v0.8.0'da kaldırılan `evaluation.staging_ttl_days` yerine geçer. | | `ephemeral_artefact_retention_days` | int | `90` | Uyumluluk paketleri, veri denetim raporları ve diğer çalışma kapsamlı türetilmiş artefaktların saklanma süresi (gün). `0` süresiz saklama. | | `raw_documents_retention_days` | int | `90` | İngest edilmiş ham belgelerin (PDF / DOCX / EPUB / TXT / Markdown) operatörün ingestion-output dizininde saklanma süresi (gün). `0` süresiz saklama. | -| `enforce` | string | `"log_only"` | Politika uygulama modu: `"log_only"` (yalnızca audit log), `"warn_on_excess"` (stderr'e yapılandırılmış uyarı), `"block_on_excess"` (`EXIT_EVAL_FAILURE` = 3 ile trainer ön-kontrolünü iptal eder). | +| `enforce` | string | `"log_only"` | Ayrılmış politika modu. Henüz hiçbir komut onu okumaz: `"log_only"`, `"warn_on_excess"` ve `"block_on_excess"` kabul edilir ve aynı davranır; bir config son ikisinden birini ayarlarsa CLI uyarır. | > **Kaldırıldı:** `evaluation.staging_ttl_days` (v0.5.5 itibarıyla kullanımdan > kaldırılmıştı) v0.8.0'da kaldırılmıştır. Artık geçerli tek form @@ -398,10 +398,10 @@ kaynağı kaydedilmemiştir. |------|-----|-----------|----------| | `strategy` | string | `null` | `"deepspeed"` veya `"fsdp"` (null = tek GPU) | | `deepspeed_config` | string | `null` | Ön ayar (`"zero2"`, `"zero3"`, `"zero3_offload"`) veya JSON yolu | -| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | -| `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar | +| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` veya `"shard_grad_op"` (`"hybrid_shard"` ve `"no_shard"` reddedilir: FSDP2 bunları desteklemez) | +| `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar (transformers 5.10+'ta yok sayılır, katmanlar hep sarılır) | | `fsdp_offload` | bool | `false` | Parametreleri CPU'ya taşı | -| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` veya `"backward_post"` | +| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` veya `"backward_post"` (transformers 5.10+'ta yok sayılır; FSDP2 prefetch politikası almaz) | | `fsdp_state_dict_type` | string | `"FULL_STATE_DICT"` | `"FULL_STATE_DICT"` veya `"SHARDED_STATE_DICT"` | ## `synthetic` (İsteğe bağlı — Sentetik Veri Üretimi) diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 0f83d16e..5d830e0a 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -270,9 +270,10 @@ across retries. Each retry attempt is logged to the audit trail. Defines maximum retention horizons for compliance, training, and evaluation artefacts. Horizons honour GDPR Article 5(1)(e) "storage limitation" and -Article 17 "right to erasure" deadlines. The `enforce` knob switches between -log-only, warning, and hard-block modes so a regulated CI gate cannot -silently extend the retention horizon by re-using a stale workspace. +Article 17 "right to erasure" deadlines. The horizons are **reported**, not +enforced: `forgelm purge --check-policy` lists what is past them and always +exits 0, and nothing is deleted, blocked or scheduled automatically. The +`enforce` knob is reserved: no command reads it yet. | Field | Type | Default | Description | |-------|------|---------|-------------| @@ -280,7 +281,7 @@ silently extend the retention horizon by re-using a stale workspace. | `staging_ttl_days` | int | `7` | Days to retain `final_model.staging./` after a `forgelm reject` decision before scheduled cleanup. Set to `0` to retain indefinitely. Replaces the removed `evaluation.staging_ttl_days` (removed in v0.8.0). | | `ephemeral_artefact_retention_days` | int | `90` | Days to retain compliance bundles, data audit reports, and other run-scoped derived artefacts. Set to `0` to retain indefinitely. | | `raw_documents_retention_days` | int | `90` | Days to retain ingested raw documents (PDF / DOCX / EPUB / TXT / Markdown) under the operator's ingestion-output directory. Set to `0` to retain indefinitely. | -| `enforce` | string | `"log_only"` | Policy enforcement mode: `"log_only"` (audit-log only), `"warn_on_excess"` (structured stderr warning), `"block_on_excess"` (abort trainer pre-flight with `EXIT_EVAL_FAILURE` = 3). | +| `enforce` | string | `"log_only"` | Reserved policy mode. No command reads it yet: `"log_only"`, `"warn_on_excess"` and `"block_on_excess"` are all accepted and behave the same, and the CLI warns when a config sets either of the last two. | > **Removed:** `evaluation.staging_ttl_days` (deprecated as of v0.5.5) was > removed in v0.8.0. `retention.staging_ttl_days` is now the only accepted form. @@ -310,10 +311,10 @@ silently extend the retention horizon by re-using a stale workspace. |-------|------|---------|-------------| | `strategy` | string | `null` | `"deepspeed"` or `"fsdp"` (null = single GPU) | | `deepspeed_config` | string | `null` | Preset (`"zero2"`, `"zero3"`, `"zero3_offload"`) or JSON path | -| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"`, `"shard_grad_op"`, `"hybrid_shard"`, `"no_shard"` | -| `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers | +| `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` or `"shard_grad_op"` (`"hybrid_shard"` and `"no_shard"` are rejected: FSDP2 does not support them) | +| `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers (ignored on transformers 5.10+, which always wraps them) | | `fsdp_offload` | bool | `false` | Offload parameters to CPU | -| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` or `"backward_post"` | +| `fsdp_backward_prefetch` | string | `"backward_pre"` | `"backward_pre"` or `"backward_post"` (ignored on transformers 5.10+, whose FSDP2 takes no prefetch policy) | | `fsdp_state_dict_type` | string | `"FULL_STATE_DICT"` | `"FULL_STATE_DICT"` or `"SHARDED_STATE_DICT"` | --- diff --git a/docs/reference/distributed_training-tr.md b/docs/reference/distributed_training-tr.md index 32e6378f..0df37e60 100644 --- a/docs/reference/distributed_training-tr.md +++ b/docs/reference/distributed_training-tr.md @@ -120,8 +120,8 @@ PyTorch Fully Sharded Data Parallel (FSDP), DeepSpeed'e alternatif olarak PyTorc |----------|----------|-----------------| | `full_shard` | Parametreleri, gradyanları ve optimizer durumlarını böl | En Yüksek | | `shard_grad_op` | Yalnızca gradyanları ve optimizer durumlarını böl | Orta | -| `hybrid_shard` | Düğüm içi tam bölme, düğümler arası çoğaltma | Çoklu Düğüm | -| `no_shard` | Standart DDP (bölme yok) | Yok | + +`hybrid_shard` ve `no_shard` **kabul edilmez**: `fsdp_strategy` `full_shard` ya da `shard_grad_op` olmalıdır. ForgeLM'in gerektirdiği transformers 5.10 ve sonrası varsayılan olarak FSDP2 çalıştırır ve FSDP2 bu ikisini de desteklemez: eskiden `--dry-run`'ı geçip model yüklendikten sonra exit 2 ile (`fsdp=hybrid_shard is only available with FSDP1`) patlıyorlardı, şimdi doğrulamada exit 1 ile reddediliyorlar. ### Kullanım @@ -129,12 +129,14 @@ PyTorch Fully Sharded Data Parallel (FSDP), DeepSpeed'e alternatif olarak PyTorc distributed: strategy: "fsdp" fsdp_strategy: "full_shard" - fsdp_auto_wrap: true # Auto-wrap transformer layers + fsdp_auto_wrap: true # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar) fsdp_offload: false # Offload parameters AND gradients (between forward/backward) to CPU - fsdp_backward_prefetch: "backward_pre" # Prefetch strategy + fsdp_backward_prefetch: "backward_pre" # transformers 5.10+'ta yok sayılır (FSDP2 prefetch politikası almaz) fsdp_state_dict_type: "FULL_STATE_DICT" # State dict handling ``` +`fsdp_auto_wrap: false` ve `fsdp_backward_prefetch: backward_post` hâlâ kabul edilir, ancak FSDP2'de hiçbir şeyi değiştirmezler; bu yüzden ikisinden biri varsayılanından farklı ayarlandığında ForgeLM bir uyarı loglar. + ### FSDP'yi DeepSpeed yerine ne zaman seçmeli - **FSDP**: Native PyTorch, ekstra bağımlılık yok, daha basit kurulum, diff --git a/docs/reference/distributed_training.md b/docs/reference/distributed_training.md index b83b29f9..013018a0 100644 --- a/docs/reference/distributed_training.md +++ b/docs/reference/distributed_training.md @@ -118,8 +118,8 @@ PyTorch Fully Sharded Data Parallel (FSDP) is an alternative to DeepSpeed that's |----------|-------------|----------------| | `full_shard` | Shard parameters, gradients, and optimizer states | Highest | | `shard_grad_op` | Shard gradients and optimizer states only | Medium | -| `hybrid_shard` | Full shard within nodes, replicate across nodes | Multi-node | -| `no_shard` | Standard DDP (no sharding) | None | + +`hybrid_shard` and `no_shard` are **not accepted**: `fsdp_strategy` must be `full_shard` or `shard_grad_op`. transformers 5.10 and later, which ForgeLM requires, run FSDP2 by default, and FSDP2 supports neither: they used to pass `--dry-run` and then fail with exit 2 (`fsdp=hybrid_shard is only available with FSDP1`) after the model was loaded, and now fail validation with exit 1. ### Usage @@ -127,12 +127,14 @@ PyTorch Fully Sharded Data Parallel (FSDP) is an alternative to DeepSpeed that's distributed: strategy: "fsdp" fsdp_strategy: "full_shard" - fsdp_auto_wrap: true # Auto-wrap transformer layers + fsdp_auto_wrap: true # Ignored on transformers 5.10+ (FSDP2 always wraps transformer layers) fsdp_offload: false # Offload parameters AND gradients (between forward/backward) to CPU - fsdp_backward_prefetch: "backward_pre" # Prefetch strategy + fsdp_backward_prefetch: "backward_pre" # Ignored on transformers 5.10+ (FSDP2 takes no prefetch policy) fsdp_state_dict_type: "FULL_STATE_DICT" # State dict handling ``` +`fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted, but under FSDP2 they change nothing, so ForgeLM logs a warning when either is set away from its default. + ### When to Choose FSDP over DeepSpeed - **FSDP**: Native PyTorch, no extra dependency, simpler setup, good for most use cases diff --git a/docs/reference/purge_subcommand-tr.md b/docs/reference/purge_subcommand-tr.md index de03e4d2..cf111861 100644 --- a/docs/reference/purge_subcommand-tr.md +++ b/docs/reference/purge_subcommand-tr.md @@ -35,7 +35,7 @@ JSONL eğitim corpus'undan `id` (veya `row_id`) field'ı ile tanımlanan tek bir | `--row-id ID` | evet | Silinecek identifier. Audit emisyonundan önce `forgelm.cli.subcommands._purge._hash_target_id` üzerinden hash'lenir. | | `--corpus PATH` | evet | Tek bir JSONL dosyası. Dizin modu reddedilir — operatörler kendi script'lerinde döngü kurar. | | `--row-matches {one,all}` | hayır (varsayılan `one`) | `one` >=2 eşleşmede reddeder; `all` her eşleşmeyi siler (operatör niyeti onaylar). | -| `--output-dir DIR` | hayır | `--corpus`'un parent'ına default'lar. Per-output-dir salt `/.forgelm_audit_salt`'ta `target_id` hash'leme için okunur. Implicit fallback, `forgelm reverse-pii` ile cross-tool korelasyon için operatörün `--output-dir`'i sabitleyebilmesi adına çözülen dizini adlandıran bir WARNING yayar. | +| `--output-dir DIR` | hayır | `--corpus`'un parent'ına default'lar. Per-output-dir salt `/.forgelm_audit_salt`'ta `target_id` hash'leme için okunur. Implicit fallback, operatörün `--output-dir`'i eğitim çıktı dizinine ve `forgelm reverse-pii` ile cross-tool korelasyon için sabitleyebilmesi adına çözülen dizini ve memorisation kontrolünün arayacağı `final_model/`'i adlandıran bir WARNING yayar. | | `--justification TEXT` | hayır | Operatör-sağlanan, her erasure event'ına yazılan sebep. Dahili ticket id'nizi referans verin; subject identifier yapıştırmayın. | | `--dry-run` | hayır | Silmeyi önizler + audit zincirini (`dry_run=true` ile) yayar; disk'e dokunmaz. | @@ -80,7 +80,7 @@ Altı event de [`audit_event_catalog.md`](audit_event_catalog-tr.md)'deki ortak | `data.erasure_requested` | Herhangi bir `--row-id` / `--run-id` çağrısının ilk adımı, herhangi bir silmeden ÖNCE. `--check-policy` salt-okunurdur ve event yaymaz. | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (row mode'da hash'li), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | | `data.erasure_completed` | Başarılı silme tamamlandı. | Tüm `requested` field'ları + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | | `data.erasure_failed` | Disk operasyonu raise etti VEYA eşleşen satır/koşum yok VEYA çoklu-satır policy belirsizliği reddetti. | Tüm `requested` field'ları + `error_class`, `error_message` | -| `data.erasure_warning_memorisation` | Row erasure × bu corpus'u tüketen herhangi bir koşum için `final_model/` mevcut. | Tüm `completed` field'ları + `affected_run_ids` | +| `data.erasure_warning_memorisation` | Row erasure × `--output-dir` altında doğrudan bir `final_model/` dizini mevcut (birebir ad; `--output-dir` varsayılan olarak corpus'un dizinidir, bu yüzden eğitim çıktı dizininizi verin). ForgeLM corpus'u hangi koşumların tükettiğini belirlemez. | Tüm `completed` field'ları + `affected_run_ids` | | `data.erasure_warning_synthetic_data_present` | Row erasure × `output_dir`'de `synthetic_data*.jsonl` mevcut. | Tüm `completed` field'ları + `synthetic_files` | | `data.erasure_warning_external_copies` | Yüklü config boş-olmayan `webhook` block'u içeriyor; downstream tüketiciler bildirim almış olabilir. | Tüm `completed` field'ları + `webhook_targets` (redact'li URL'ler) | diff --git a/docs/reference/purge_subcommand.md b/docs/reference/purge_subcommand.md index ebbffe38..2c44b85f 100644 --- a/docs/reference/purge_subcommand.md +++ b/docs/reference/purge_subcommand.md @@ -35,7 +35,7 @@ Erases a single row identified by its `id` (or `row_id`) field from a JSONL trai | `--row-id ID` | yes | Identifier to erase. Hashed before audit emission via `forgelm.cli.subcommands._purge._hash_target_id`. | | `--corpus PATH` | yes | Single JSONL file. Directory mode is rejected — operators loop in their own script. | | `--row-matches {one,all}` | no (default `one`) | `one` refuses on >=2 matches; `all` deletes every match (operator confirms intent). | -| `--output-dir DIR` | no | Defaults to the parent of `--corpus`. The per-output-dir salt at `/.forgelm_audit_salt` is read here for `target_id` hashing. The implicit fallback emits a WARNING naming the resolved dir so the operator can pin `--output-dir` for cross-tool correlation with `forgelm reverse-pii`. | +| `--output-dir DIR` | no | Defaults to the parent of `--corpus`. The per-output-dir salt at `/.forgelm_audit_salt` is read here for `target_id` hashing. The implicit fallback emits a WARNING naming the resolved dir and the `final_model/` the memorisation check will look for, so the operator can pin `--output-dir` to the training output directory and for cross-tool correlation with `forgelm reverse-pii`. | | `--justification TEXT` | no | Operator-supplied reason recorded on every erasure event. Reference your internal ticket id; do not paste subject identifiers. | | `--dry-run` | no | Preview the deletion + emit the audit chain (with `dry_run=true`) without modifying disk. | @@ -80,7 +80,7 @@ All six events ride the common envelope from [`audit_event_catalog.md`](audit_ev | `data.erasure_requested` | First step of any `--row-id` / `--run-id` invocation, BEFORE any deletion. `--check-policy` is read-only and emits no events. | `target_kind` ∈ `{row, staging, artefacts}`, `target_id` (hashed in row mode), `salt_source` (row mode), `corpus_path` (row), `output_dir` (run), `justification`, `dry_run` | | `data.erasure_completed` | Successful deletion finished. | All `requested` fields + `bytes_freed`, `files_modified`, `pre_erasure_line_number` (row mode), `match_count` (row mode) | | `data.erasure_failed` | Disk operation raised, OR no matching row/run, OR multi-row policy refused on ambiguity. | All `requested` fields + `error_class`, `error_message` | -| `data.erasure_warning_memorisation` | Row erasure × `final_model/` exists for any run that consumed this corpus. | All `completed` fields + `affected_run_ids` | +| `data.erasure_warning_memorisation` | Row erasure × a `final_model/` directory exists directly under `--output-dir` (literal name; `--output-dir` defaults to the corpus's directory, so pass your training output directory). ForgeLM does not determine which runs consumed the corpus. | All `completed` fields + `affected_run_ids` | | `data.erasure_warning_synthetic_data_present` | Row erasure × `synthetic_data*.jsonl` exists in `output_dir`. | All `completed` fields + `synthetic_files` | | `data.erasure_warning_external_copies` | Loaded config has a non-empty `webhook` block; downstream consumers may have received notices. | All `completed` fields + `webhook_targets` (redacted URLs) | diff --git a/docs/reference/supply_chain_security-tr.md b/docs/reference/supply_chain_security-tr.md index 0b74c5a2..a81303ae 100644 --- a/docs/reference/supply_chain_security-tr.md +++ b/docs/reference/supply_chain_security-tr.md @@ -151,7 +151,7 @@ bir [`tools/pip_audit_ignores.yaml`](../../tools/pip_audit_ignores.yaml) taşır (şu an dokuz no-fix torch advisory'si ve bir OSV-yanlış-sınıflandırılmış markdown kaydı; daha önce taşıdığı iki transformers advisory'si `transformers>=5.3.0` floor'u onları pip-audit -tarafından raporlanamaz hâle getirince kaldırıldı). Bu dosya yalnız projenin kendi workflow'u tarafından +tarafından raporlanamaz hâle getirince kaldırıldı; floor artık `>=5.10.1`). Bu dosya yalnız projenin kendi workflow'u tarafından `--ignores` ile tüketilir; deployer'lar miras almaz. Her release cycle'da gözden geçirilir. diff --git a/docs/reference/supply_chain_security.md b/docs/reference/supply_chain_security.md index 13cac6c0..e070e770 100644 --- a/docs/reference/supply_chain_security.md +++ b/docs/reference/supply_chain_security.md @@ -149,7 +149,7 @@ ForgeLM's own nightly does carry a checked-in project-internal triage (currently nine no-fix torch advisories and one OSV-misclassified markdown record; the two transformers advisories it once carried were dropped once the `transformers>=5.3.0` floor made them -unreportable by pip-audit). That file is consumed only by the +unreportable by pip-audit; the floor is now `>=5.10.1`). That file is consumed only by the project's own workflow via `--ignores`; deployers do not inherit it. Review at every release cycle. diff --git a/docs/usermanuals/en/compliance/gdpr-erasure.md b/docs/usermanuals/en/compliance/gdpr-erasure.md index 6cf43cb9..359a8561 100644 --- a/docs/usermanuals/en/compliance/gdpr-erasure.md +++ b/docs/usermanuals/en/compliance/gdpr-erasure.md @@ -42,7 +42,7 @@ Six events ride the audit chain (see the [Audit Event Catalog on GitHub](https:/ - `data.erasure_requested` — emitted FIRST, before any deletion. - `data.erasure_completed` — emitted LAST after the disk operation succeeded. - `data.erasure_failed` — emitted instead when the disk operation raised, or no matching row/run was found. -- `data.erasure_warning_memorisation` — corpus-row erasure × `final_model/` exists for any run that consumed this corpus (the row is gone from disk but may still be memorised in the trained weights). +- `data.erasure_warning_memorisation` — corpus-row erasure × a `final_model/` exists directly under `--output-dir` (the row is gone from disk but may still be memorised in the trained weights). - `data.erasure_warning_synthetic_data_present` — corpus-row erasure × `synthetic_data*.jsonl` exists in `output_dir`. - `data.erasure_warning_external_copies` — loaded config has a non-empty `webhook` block; downstream consumers may have received notices. @@ -110,7 +110,7 @@ Codes 3 (`EXIT_EVAL_FAILURE`) and 4 (`EXIT_AWAITING_APPROVAL`) are not part of e ::: :::warn -**Running `forgelm purge` without checking for memorisation.** If a `final_model/` exists for any run that consumed the corpus, the row is gone from disk but may still be encoded in the weights. The `data.erasure_warning_memorisation` event surfaces the gap; act on it (re-train without the erased row, or document the residual risk in your DPIA). +**Running `forgelm purge` without checking for memorisation.** If a trained model exists, the row is gone from disk but may still be encoded in the weights. The `data.erasure_warning_memorisation` event surfaces the gap only when a `final_model/` sits directly under `--output-dir` — which defaults to the corpus's directory, so pass your training output directory; act on it (re-train without the erased row, or document the residual risk in your DPIA). ::: :::tip diff --git a/docs/usermanuals/en/getting-started/installation.md b/docs/usermanuals/en/getting-started/installation.md index a943d0f0..d5aad6d2 100644 --- a/docs/usermanuals/en/getting-started/installation.md +++ b/docs/usermanuals/en/getting-started/installation.md @@ -22,7 +22,7 @@ ForgeLM ships as a single PyPI package with optional dependency groups (called * ::: :::note -**Intel Macs (x86_64) are not supported.** ForgeLM requires `torch>=2.4.0` (pulled in by `transformers>=5.3.0`), and PyPI publishes no `torch>=2.4` wheel for x86_64 macOS. Use Apple Silicon, Linux, or Windows. +**Intel Macs (x86_64) are not supported.** ForgeLM requires `torch>=2.4.0` (the minimum that `transformers` 5 supports), and PyPI publishes no `torch>=2.4` wheel for x86_64 macOS. Use Apple Silicon, Linux, or Windows. ::: ## Base install diff --git a/docs/usermanuals/en/operations/air-gap.md b/docs/usermanuals/en/operations/air-gap.md index aacf64e5..40a2807c 100644 --- a/docs/usermanuals/en/operations/air-gap.md +++ b/docs/usermanuals/en/operations/air-gap.md @@ -63,7 +63,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | OpenAI / Anthropic judges | Disable API-backed evaluation or use a local judge; `--offline` does not disable API clients. | | Webhooks | `--offline` does not disable configured delivery. Omit the `webhook:` block, or leave `url` / `url_env` unset, to disable it; `allow_private_destinations` permits private destinations in addition to public ones. | | pip install (during runtime) | Install required dependencies before disconnecting; `--offline` does not control pip. | -| Telemetry | ForgeLM has no own telemetry service; configure external integrations separately. | +| Telemetry | ForgeLM has no own telemetry service. The `forgelm` CLI sets `HF_HUB_DISABLE_TELEMETRY=1` (unless you set it yourself) so TRL and `huggingface_hub` send no anonymous usage ping; using ForgeLM as a library does not set it. Configure external integrations separately. | ## Local synthetic data diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index c4ba73c2..ea38e082 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -229,10 +229,10 @@ The merge field is `method` (not `algorithm`), and there is no `dare_ties` choic distributed: strategy: null # null (single-GPU, no distributed wrapping) | deepspeed | fsdp deepspeed_config: null # path to a DeepSpeed JSON, or preset name: zero2 | zero3 | zero3_offload - fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard - fsdp_auto_wrap: true # auto-wrap transformer layers (recommended) + fsdp_strategy: "full_shard" # full_shard | shard_grad_op + fsdp_auto_wrap: true # ignored on transformers 5.10+ (FSDP2 always wraps transformer layers) fsdp_offload: false # offload FSDP parameters to CPU between forward/backward - fsdp_backward_prefetch: "backward_pre" # backward_pre | backward_post + fsdp_backward_prefetch: "backward_pre" # backward_pre | backward_post; ignored on transformers 5.10+ fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT ``` @@ -363,7 +363,7 @@ retention: staging_ttl_days: 7 # days to retain staging model after forgelm reject ephemeral_artefact_retention_days: 90 # compliance bundles, audit reports raw_documents_retention_days: 90 # ingested PDF/DOCX/EPUB/TXT/Markdown - enforce: "log_only" # log_only | warn_on_excess | block_on_excess + enforce: "log_only" # reserved: no command acts on it yet (log_only | warn_on_excess | block_on_excess) ``` ## `pipeline:` diff --git a/docs/usermanuals/en/training/distributed.md b/docs/usermanuals/en/training/distributed.md index 7aa622ff..8679d546 100644 --- a/docs/usermanuals/en/training/distributed.md +++ b/docs/usermanuals/en/training/distributed.md @@ -111,12 +111,14 @@ FSDP shards similarly to ZeRO-3 but uses PyTorch's native FullyShardedDataParall ```yaml distributed: strategy: "fsdp" - fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard - fsdp_auto_wrap: true # auto-wrap transformer layers (recommended) + fsdp_strategy: "full_shard" # full_shard | shard_grad_op + fsdp_auto_wrap: true # ignored on transformers 5.10+ (FSDP2 always wraps transformer layers) fsdp_offload: false # offload parameters to CPU between forward/backward fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT ``` +`no_shard` and `hybrid_shard` are rejected when the config loads: transformers 5.10+, which ForgeLM requires, runs FSDP2, and FSDP2 supports neither. `fsdp_auto_wrap: false` and `fsdp_backward_prefetch: backward_post` are still accepted but change nothing under FSDP2, so ForgeLM logs a warning when either is set. + `DistributedConfig` has no `fsdp_auto_wrap_policy` or `fsdp_offload_params` field — auto-wrap is the plain boolean `fsdp_auto_wrap`, and CPU offload is `fsdp_offload` (no `_params` suffix). ## Gradient accumulation diff --git a/docs/usermanuals/tr/compliance/gdpr-erasure.md b/docs/usermanuals/tr/compliance/gdpr-erasure.md index f430e5c0..77e6b09a 100644 --- a/docs/usermanuals/tr/compliance/gdpr-erasure.md +++ b/docs/usermanuals/tr/compliance/gdpr-erasure.md @@ -42,7 +42,7 @@ Audit zincirine altı event yazılır (bkz. [GitHub'daki Audit Event Kataloğu]( - `data.erasure_requested` — herhangi bir silmeden ÖNCE, ilk yayılır. - `data.erasure_completed` — disk operasyonu başarılı sonra son yayılır. - `data.erasure_failed` — disk operasyonu raise ettiğinde veya eşleşen satır/koşum bulunmadığında bunun yerine yayılır. -- `data.erasure_warning_memorisation` — corpus-satır silme × bu corpus'u tüketen herhangi bir koşum için `final_model/` mevcut (satır diskten gitti ama eğitilmiş ağırlıklarda hâlâ memorize edilmiş olabilir). +- `data.erasure_warning_memorisation` — corpus-satır silme × `--output-dir` altında doğrudan bir `final_model/` mevcut (satır diskten gitti ama eğitilmiş ağırlıklarda hâlâ memorize edilmiş olabilir). - `data.erasure_warning_synthetic_data_present` — corpus-satır silme × `output_dir`'de `synthetic_data*.jsonl` mevcut. - `data.erasure_warning_external_copies` — yüklü config boş-olmayan `webhook` block'u içeriyor; downstream tüketiciler bildirim almış olabilir. @@ -110,7 +110,7 @@ Kod 3 (`EXIT_EVAL_FAILURE`) ve 4 (`EXIT_AWAITING_APPROVAL`) iki subcommand'ın d ::: :::warn -**Memorization'ı kontrol etmeden `forgelm purge` çalıştırmak.** Corpus'u tüketen herhangi bir koşum için `final_model/` mevcutsa, satır diskten gitti ama ağırlıklarda hâlâ encoded olabilir. `data.erasure_warning_memorisation` event'ı boşluğu yüzeyselleştirir; harekete geçin (silinmiş satır olmadan yeniden eğitin veya DPIA'nızda residual riski belgeleyin). +**Memorization'ı kontrol etmeden `forgelm purge` çalıştırmak.** Eğitilmiş bir model varsa, satır diskten gitti ama ağırlıklarda hâlâ encoded olabilir. `data.erasure_warning_memorisation` event'ı boşluğu yalnızca `--output-dir` altında doğrudan bir `final_model/` varsa yüzeyselleştirir — `--output-dir` varsayılan olarak corpus'un dizinidir, bu yüzden eğitim çıktı dizininizi verin; harekete geçin (silinmiş satır olmadan yeniden eğitin veya DPIA'nızda residual riski belgeleyin). ::: :::tip diff --git a/docs/usermanuals/tr/getting-started/installation.md b/docs/usermanuals/tr/getting-started/installation.md index bdf196c9..5d5ecd01 100644 --- a/docs/usermanuals/tr/getting-started/installation.md +++ b/docs/usermanuals/tr/getting-started/installation.md @@ -22,7 +22,7 @@ ForgeLM, PyPI'da tek bir paket olarak gelir; ağır özellikler için opsiyonel ::: :::note -**Intel Mac'ler (x86_64) desteklenmiyor.** ForgeLM `torch>=2.4.0` gerektirir (`transformers>=5.3.0` üzerinden gelir) ve PyPI'da x86_64 macOS için `torch>=2.4` wheel'i yok. Apple Silicon, Linux veya Windows kullanın. +**Intel Mac'ler (x86_64) desteklenmiyor.** ForgeLM `torch>=2.4.0` gerektirir (`transformers` 5'in desteklediği asgari sürüm) ve PyPI'da x86_64 macOS için `torch>=2.4` wheel'i yok. Apple Silicon, Linux veya Windows kullanın. ::: ## Temel kurulum diff --git a/docs/usermanuals/tr/operations/air-gap.md b/docs/usermanuals/tr/operations/air-gap.md index 58dc233e..c5744481 100644 --- a/docs/usermanuals/tr/operations/air-gap.md +++ b/docs/usermanuals/tr/operations/air-gap.md @@ -63,7 +63,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | OpenAI / Anthropic judge'lar | API tabanlı değerlendirmeyi kapatın veya yerel judge kullanın; `--offline` API istemcilerini kapatmaz. | | Webhook'lar | `--offline`, yapılandırılmış teslimatları kapatmaz. Kapatmak için `webhook:` bloğunu çıkarın veya `url` / `url_env` değerlerini boş bırakın; `allow_private_destinations`, genel hedeflerin yanında özel hedeflere de izin verir. | | Runtime'da pip install | Gerekli bağımlılıkları bağlantıyı kesmeden kurun; `--offline`, pip'i yönetmez. | -| Telemetry | ForgeLM'in kendi telemetri servisi yoktur; harici entegrasyonları ayrıca yapılandırın. | +| Telemetry | ForgeLM'in kendi telemetri servisi yoktur. `forgelm` CLI'ı `HF_HUB_DISABLE_TELEMETRY=1`'i (siz ayarlamadıysanız) ayarlar; böylece TRL ve `huggingface_hub` anonim kullanım ping'i göndermez. ForgeLM'i kütüphane olarak kullanmak bunu ayarlamaz. Harici entegrasyonları ayrıca yapılandırın. | ## Yerel sentetik veri diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 39f8e44b..690f806b 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -231,10 +231,10 @@ Merge alanı `algorithm` değil `method`'dur, `dare_ties` seçeneği, `base_mode distributed: strategy: null # null (tek GPU, dağıtık sarmalama yok) | deepspeed | fsdp deepspeed_config: null # bir DeepSpeed JSON'a yol, veya preset adı: zero2 | zero3 | zero3_offload - fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard - fsdp_auto_wrap: true # transformer katmanlarını otomatik sarmala (önerilir) + fsdp_strategy: "full_shard" # full_shard | shard_grad_op + fsdp_auto_wrap: true # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar) fsdp_offload: false # forward/backward arasında FSDP parametrelerini CPU'ya offload et - fsdp_backward_prefetch: "backward_pre" # backward_pre | backward_post + fsdp_backward_prefetch: "backward_pre" # backward_pre | backward_post; transformers 5.10+'ta yok sayılır fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT ``` @@ -371,7 +371,7 @@ retention: staging_ttl_days: 7 # forgelm reject sonrası staging modelini saklama süresi ephemeral_artefact_retention_days: 90 # uyumluluk paketleri, denetim raporları raw_documents_retention_days: 90 # ingest edilen PDF/DOCX/EPUB/TXT/Markdown - enforce: "log_only" # log_only | warn_on_excess | block_on_excess + enforce: "log_only" # ayrılmış: henüz hiçbir komut onu kullanmaz (log_only | warn_on_excess | block_on_excess) ``` ## `pipeline:` diff --git a/docs/usermanuals/tr/training/distributed.md b/docs/usermanuals/tr/training/distributed.md index 4dcfd239..7e73a6d7 100644 --- a/docs/usermanuals/tr/training/distributed.md +++ b/docs/usermanuals/tr/training/distributed.md @@ -111,12 +111,14 @@ FSDP, ZeRO-3 gibi sharder ama PyTorch'un yerli FullyShardedDataParallel'ini kull ```yaml distributed: strategy: "fsdp" - fsdp_strategy: "full_shard" # full_shard | shard_grad_op | no_shard | hybrid_shard - fsdp_auto_wrap: true # transformer katmanlarını otomatik sar (önerilir) + fsdp_strategy: "full_shard" # full_shard | shard_grad_op + fsdp_auto_wrap: true # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar) fsdp_offload: false # forward/backward arasında parametreleri CPU'ya boşalt fsdp_state_dict_type: "FULL_STATE_DICT" # FULL_STATE_DICT | SHARDED_STATE_DICT ``` +`no_shard` ve `hybrid_shard` config yüklenirken reddedilir: ForgeLM'in gerektirdiği transformers 5.10+ FSDP2 çalıştırır ve FSDP2 bu ikisini desteklemez. `fsdp_auto_wrap: false` ve `fsdp_backward_prefetch: backward_post` hâlâ kabul edilir ancak FSDP2'de hiçbir şeyi değiştirmez; bu yüzden ikisinden biri ayarlandığında ForgeLM bir uyarı loglar. + `DistributedConfig`'te `fsdp_auto_wrap_policy` veya `fsdp_offload_params` alanı yoktur — auto-wrap düz bir `fsdp_auto_wrap` boolean'ıdır, CPU offload ise `fsdp_offload`'dur (`_params` eki yoktur). ## Gradient accumulation diff --git a/forgelm/cli/_config_load.py b/forgelm/cli/_config_load.py index 4e5ac477..8357d535 100644 --- a/forgelm/cli/_config_load.py +++ b/forgelm/cli/_config_load.py @@ -14,6 +14,22 @@ from ._logging import logger +def _warn_reserved_retention_enforce(config) -> None: + """``retention.enforce`` is validated and offered by both wizards, but no command reads it. + + A config that sets ``warn_on_excess`` or ``block_on_excess`` is asking for a gate that does not exist; say so, + rather than accept it silently. (This lives here, not in the schema, to keep ``config.py`` within its + deferred-split line budget.) + """ + mode = getattr(getattr(config, "retention", None), "enforce", "log_only") + if mode != "log_only": + logger.warning( + "retention.enforce=%r is accepted but no command acts on it yet: retention horizons are only reported " + "by `forgelm purge --check-policy`, and the trainer is not warned or blocked.", + mode, + ) + + def _load_config_or_exit(config_path: str, json_output: bool): """Load config and translate exceptions into the right exit code. @@ -25,7 +41,9 @@ def _load_config_or_exit(config_path: str, json_output: bool): """ try: logger.info("Loading configuration from %s...", config_path) - return load_config(config_path) + config = load_config(config_path) + _warn_reserved_retention_enforce(config) + return config except FileNotFoundError as e: msg = f"Config file not found: {e}" except ConfigError as e: @@ -47,6 +65,20 @@ def _load_config_or_exit(config_path: str, json_output: bool): sys.exit(EXIT_CONFIG_ERROR) +def _disable_hub_telemetry() -> None: + """Keep the ``forgelm`` CLI from sending Hugging Face's anonymous usage ping. + + A training run makes ``huggingface_hub`` send a small usage ping to huggingface.co: TRL's trainers call + ``send_telemetry`` when they are built, and ``huggingface_hub`` skips it only when ``HF_HUB_DISABLE_TELEMETRY`` is + true or offline mode is on. ForgeLM's privacy page says the tool sends no usage data anywhere, so the CLI sets + the variable by default. ``setdefault`` leaves an operator's own value alone (``HF_HUB_DISABLE_TELEMETRY=0`` + opts back in). It must run before ``huggingface_hub`` is imported, which reads it once at import; importing + ``forgelm.cli`` does not import it, and ``main()`` calls this first. Using ForgeLM as a library does not go + through ``main()``, so a library user sets the variable themselves. + """ + os.environ.setdefault("HF_HUB_DISABLE_TELEMETRY", "1") + + def _apply_offline_flag(config, offline_arg: bool) -> None: if offline_arg: config.model.offline = True @@ -55,3 +87,27 @@ def _apply_offline_flag(config, offline_arg: bool) -> None: os.environ["TRANSFORMERS_OFFLINE"] = "1" os.environ["HF_DATASETS_OFFLINE"] = "1" logger.info("Offline mode enabled. All HF Hub network calls are disabled.") + + +def _warn_inert_fsdp_options(config) -> None: + """Say so when an FSDP knob that transformers' FSDP2 default ignores is set away from its default. + + ``fsdp_auto_wrap: false`` and ``fsdp_backward_prefetch: backward_post`` are still accepted, but under + transformers 5.10+ (FSDP2) the transformer-layer wrap policy is always applied and no backward prefetch is + passed on, so the run behaves as if the defaults had been kept. (This lives here, not in the schema, to keep + ``config.py`` within its deferred-split line budget.) + """ + dist = getattr(config, "distributed", None) + if getattr(dist, "strategy", None) != "fsdp": + return + if not dist.fsdp_auto_wrap: + logger.warning( + "distributed.fsdp_auto_wrap=false has no effect: transformers 5.10+ runs FSDP2, which always wraps " + "transformer layers." + ) + if dist.fsdp_backward_prefetch != "backward_pre": + logger.warning( + "distributed.fsdp_backward_prefetch=%r has no effect: transformers 5.10+ runs FSDP2, which does not take " + "a backward-prefetch policy.", + dist.fsdp_backward_prefetch, + ) diff --git a/forgelm/cli/_dispatch.py b/forgelm/cli/_dispatch.py index e4b64387..972ee75d 100644 --- a/forgelm/cli/_dispatch.py +++ b/forgelm/cli/_dispatch.py @@ -10,7 +10,7 @@ import sys from .._strict_json import dumps_strict -from ._config_load import _apply_offline_flag, _load_config_or_exit +from ._config_load import _apply_offline_flag, _disable_hub_telemetry, _load_config_or_exit, _warn_inert_fsdp_options from ._exit_codes import EXIT_CONFIG_ERROR, EXIT_SUCCESS, EXIT_TRAINING_ERROR, _clamp_exit_code from ._logging import _setup_logging, logger from ._no_train_modes import _maybe_run_no_train_mode @@ -163,6 +163,7 @@ def main(): # (= 128+SIGINT) — outside the documented public 0/1/2/3/4 surface # and surprising to CI/CD scripts that branch on exit code 2. _force_utf8_console_streams() + _disable_hub_telemetry() try: return _main_inner() except KeyboardInterrupt: @@ -241,6 +242,7 @@ def _main_inner() -> None: config = _load_config_or_exit(args.config, json_output) _apply_offline_flag(config, args.offline) + _warn_inert_fsdp_options(config) if config.pipeline is not None: _dispatch_pipeline_mode(config, args, json_output) diff --git a/forgelm/cli/subcommands/_audit.py b/forgelm/cli/subcommands/_audit.py index d9b4f4a1..b4e17ff7 100644 --- a/forgelm/cli/subcommands/_audit.py +++ b/forgelm/cli/subcommands/_audit.py @@ -55,8 +55,7 @@ def _exit_on_critical_secrets(verdict: dict, *, allow_secrets: bool, defer_exit: chain, GGUF sidecar); training data was never signed. - ``3`` fits exactly: an evaluation gate examined an artefact and said no. This is the same semantics as ``forgelm safety-eval``'s - non-passing branch (``subcommands/_safety_eval.py``) and the - retention ``block_on_excess`` gate. A new public exit code is a + non-passing branch (``subcommands/_safety_eval.py``). A new public exit code is a permanent contract addition and this incident does not need one. ``allow_secrets`` is an explicit per-invocation escape hatch for the diff --git a/forgelm/cli/subcommands/_purge.py b/forgelm/cli/subcommands/_purge.py index 781d9208..97f139aa 100644 --- a/forgelm/cli/subcommands/_purge.py +++ b/forgelm/cli/subcommands/_purge.py @@ -714,6 +714,17 @@ def _run_purge_row_id(args, output_format: str) -> None: """ _validate_row_id_args(args, output_format) output_dir = args.output_dir or os.path.dirname(os.path.abspath(args.corpus)) or "." + if not args.output_dir: + # The salt, the audit chain and the memorisation check (`/final_model/`) all live under the + # output directory. Defaulting to the corpus directory is almost never where a training run wrote its model, + # so say what was resolved instead of letting the check pass quietly over the wrong directory. + logger.warning( + "--output-dir not given: using the corpus directory %s for the audit log, the salt and the memorisation " + "check (which looks for %s). Pass --output-dir to check your trained model and to " + "correlate with `forgelm reverse-pii`.", + output_dir, + os.path.join(output_dir, "final_model"), + ) os.makedirs(output_dir, exist_ok=True) try: diff --git a/forgelm/config.py b/forgelm/config.py index 4114f2df..33e90e25 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -770,17 +770,17 @@ class DistributedConfig(BaseModel): default=None, description="DeepSpeed config: filesystem path to a DS JSON OR preset name (`zero2`, `zero3`, `zero3_offload`).", ) - fsdp_strategy: Literal["full_shard", "shard_grad_op", "no_shard", "hybrid_shard"] = Field( + fsdp_strategy: Literal["full_shard", "shard_grad_op"] = Field( default="full_shard", - description="FSDP sharding strategy. `full_shard` is the production default; `hybrid_shard` for multi-node intra-node sharding.", + description="FSDP sharding strategy. `full_shard` is the production default; `shard_grad_op` shards gradients and optimizer states only. `no_shard` and `hybrid_shard` are not accepted: transformers 5.10+ runs FSDP2, which supports neither.", ) - fsdp_auto_wrap: bool = Field(default=True, description="FSDP: auto-wrap transformer layers (recommended).") + fsdp_auto_wrap: bool = Field(default=True, description="FSDP auto-wrap; ignored by FSDP2 (transformers 5.10+).") fsdp_offload: bool = Field( default=False, description="FSDP: offload parameters to CPU between forward and backward (slower, less VRAM)." ) fsdp_backward_prefetch: Literal["backward_pre", "backward_post"] = Field( default="backward_pre", - description="FSDP backward-prefetch policy. `backward_pre` overlaps comm + compute; `backward_post` is more memory-conservative.", + description="FSDP backward-prefetch policy. Not honoured by transformers 5.10+ (FSDP2 prefetches on its own); `backward_post` logs a warning.", ) fsdp_state_dict_type: Literal["FULL_STATE_DICT", "SHARDED_STATE_DICT"] = Field( default="FULL_STATE_DICT", @@ -1598,12 +1598,12 @@ class RetentionConfig(BaseModel): days (typical ingestion-window before re-running data audit). Setting any horizon to ``0`` disables the policy for that artefact - kind (retain indefinitely). ``enforce`` controls how the trainer - pre-flight gate reacts to violations: ``log_only`` records a - notice, ``warn_on_excess`` emits a structured warning, and - ``block_on_excess`` aborts training with EXIT_EVAL_FAILURE so a - regulated CI cannot accidentally extend the retention horizon by - re-using a stale workspace. + kind (retain indefinitely). The horizons are *reported*, not enforced: + ``forgelm purge --check-policy`` lists artefacts past them and always + exits 0; nothing is deleted, blocked or scheduled automatically. + ``enforce`` is reserved: no command acts on it yet, so + ``warn_on_excess`` and ``block_on_excess`` behave like ``log_only`` + (the CLI logs a warning when a config sets either). """ model_config = ConfigDict(extra="forbid") @@ -1645,9 +1645,9 @@ class RetentionConfig(BaseModel): enforce: Literal["log_only", "warn_on_excess", "block_on_excess"] = Field( default="log_only", description=( - "Policy enforcement mode. `log_only` records violations in the audit log without operator-visible output; " - "`warn_on_excess` adds a structured warning to stderr; `block_on_excess` aborts the trainer pre-flight with " - "EXIT_EVAL_FAILURE (3) so a regulated CI gate does not silently extend the retention horizon." + "Reserved: no command reads it yet. Retention horizons are only reported, read-only, by `forgelm purge " + "--check-policy` (always exit 0); nothing is deleted, warned about or blocked automatically. " + "`warn_on_excess` and `block_on_excess` are accepted but behave like `log_only`; the CLI warns when set." ), ) diff --git a/forgelm/model.py b/forgelm/model.py index de6c7577..4d96ea14 100644 --- a/forgelm/model.py +++ b/forgelm/model.py @@ -420,6 +420,35 @@ def _device_map_for(config: Any, is_distributed: bool): return None # CPU-only +def _config_with_rope_scaling( + name_or_path: str, rope_scaling: dict, *, trust_remote_code: bool, revision: Optional[str] = None +) -> Any: + """Load the model's config and apply ``training.rope_scaling`` to it. + + transformers 5 keeps the RoPE settings in ``config.rope_parameters`` (which also carries ``rope_theta``) and + expects the key ``rope_type``. Passing ``rope_scaling=`` to ``from_pretrained`` no longer works: the documented + ``{type: linear, factor: 4}`` raises ``KeyError: 'rope_type'`` while the model is built, and + ``{rope_type: ..., factor: ...}`` replaces the whole dict so ``rope_theta`` is lost (``TypeError`` in the + frequency computation). So the requested keys are merged *over* the model's own ``rope_parameters``, the legacy + key ``type`` is read as ``rope_type``, and the finished config object is what gets loaded. + + For a multimodal checkpoint the RoPE settings live in the text sub-config, which is where they are applied. + A transformers without ``rope_parameters`` gets the legacy ``rope_scaling`` attribute instead. + """ + from transformers import AutoConfig + + model_config = AutoConfig.from_pretrained(name_or_path, trust_remote_code=trust_remote_code, revision=revision) + target = model_config.get_text_config() if hasattr(model_config, "get_text_config") else model_config + requested = dict(rope_scaling) + if "type" in requested and "rope_type" not in requested: + requested["rope_type"] = requested.pop("type") + if hasattr(target, "rope_parameters"): + target.rope_parameters = {**(getattr(target, "rope_parameters", None) or {}), **requested} + else: + target.rope_scaling = requested + return model_config + + def _build_model_kwargs(config: Any, trust_remote_code: bool, revision: Optional[str] = None) -> dict: """Assemble from_pretrained kwargs (device_map, BnB, RoPE, sliding window, revision). @@ -461,12 +490,19 @@ def _build_model_kwargs(config: Any, trust_remote_code: bool, revision: Optional ) rope_scaling = config.training.rope_scaling + sliding_window = config.training.sliding_window_attention if rope_scaling: logger.info("RoPE scaling enabled: %s", rope_scaling) - kwargs["rope_scaling"] = rope_scaling - - sliding_window = config.training.sliding_window_attention - if sliding_window: + model_config = _config_with_rope_scaling( + config.model.name_or_path, rope_scaling, trust_remote_code=trust_remote_code, revision=revision + ) + if sliding_window: + # Once ``config=`` is a finished object, from_pretrained hands unrecognised keyword arguments to the + # model constructor instead of the config, so the override is applied to the config itself. + logger.info("Sliding window attention override: %d tokens", sliding_window) + model_config.sliding_window = sliding_window + kwargs["config"] = model_config + elif sliding_window: logger.info("Sliding window attention override: %d tokens", sliding_window) kwargs["sliding_window"] = sliding_window diff --git a/forgelm/wizard/_collectors.py b/forgelm/wizard/_collectors.py index 83c0039a..d7574192 100644 --- a/forgelm/wizard/_collectors.py +++ b/forgelm/wizard/_collectors.py @@ -855,11 +855,11 @@ def _collect_retention() -> Optional[Dict[str, Any]]: ) raw_docs = _prompt_int("raw_documents_retention_days (0 = retain indefinitely)", 90, min_val=0, max_val=3650) enforce = _prompt_choice( - "enforcement mode:", + "enforcement mode (reserved: no command acts on it yet; `forgelm purge --check-policy` only reports):", [ - "log_only (record violations only)", - "warn_on_excess (stderr warning, run continues)", - "block_on_excess (abort trainer with EXIT_EVAL_FAILURE)", + "log_only (default; behaves the same as the other two for now)", + "warn_on_excess (accepted, behaves like log_only)", + "block_on_excess (accepted, behaves like log_only)", ], default=1, ) diff --git a/pyproject.toml b/pyproject.toml index adb6053f..f1e2ce1e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -50,7 +50,11 @@ dependencies = [ # the transformers-5 migration (v0.9.0). Apple Silicon, Linux and Windows # are unaffected. The previous `numpy<2; darwin x86_64` ABI-guard marker is # gone with the platform it protected (torch 2.4+ ships against NumPy 2.x). - "transformers>=5.3.0,<6.0.0", + # 5.10.1 is the first installable release past three advisories that reach ForgeLM's own load and save path from + # a Hub repo: CVE-2026-4372 (fixed 5.3.0), CVE-2026-5241 (a LightGlue config.json overrides trust_remote_code; + # fixed 5.5.0) and CVE-2026-9856 (chat_template names used unsanitised as file names by save_pretrained; fixed + # 5.10.0, which PyPI yanked, hence 5.10.1). It also needs huggingface_hub>=1.5.0. + "transformers>=5.10.1,<6.0.0", "peft>=0.19.0,<1.0.0", "datasets>=4.7.0,<6.0.0", "accelerate>=1.4.0,<2.0.0", @@ -58,7 +62,7 @@ dependencies = [ "pydantic>=2.0.0,<3.0.0", "trl>=1.0.0,<2.0.0", "tensorboard>=2.15.0,<3.0.0", - "huggingface_hub>=1.3.0,<2.0.0", + "huggingface_hub>=1.5.0,<2.0.0", "requests>=2.32.2,<3.0.0", # requests-toolbelt is required by forgelm._http for the HostHeaderSSLAdapter # used to close the DNS-rebinding TOCTOU window in safe_post / safe_get diff --git a/tests/test_cli.py b/tests/test_cli.py index a9765895..1fa412e6 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -319,3 +319,118 @@ def test_facade_reexports_every_subcommand_registrar(self): ] for name in registrars: assert callable(getattr(cli, name, None)), f"{name} is not re-exported from forgelm.cli" + + +class TestHubTelemetryDefault: + """The CLI keeps TRL / huggingface_hub from sending the anonymous usage ping (#95).""" + + @staticmethod + def _unset(monkeypatch): + """Make the variable absent *and* make monkeypatch restore that, so a later set does not leak.""" + monkeypatch.setenv("HF_HUB_DISABLE_TELEMETRY", "placeholder") + monkeypatch.delenv("HF_HUB_DISABLE_TELEMETRY") + + @staticmethod + def _clean_env(): + env = {k: v for k, v in os.environ.items() if k != "HF_HUB_DISABLE_TELEMETRY"} + env["PYTHONPATH"] = os.getcwd() + os.pathsep + env.get("PYTHONPATH", "") + return env + + def test_it_is_set_when_the_operator_did_not_choose(self, monkeypatch): + from forgelm.cli._config_load import _disable_hub_telemetry + + self._unset(monkeypatch) + _disable_hub_telemetry() + assert os.environ["HF_HUB_DISABLE_TELEMETRY"] == "1" + + @pytest.mark.parametrize("chosen", ["0", "1", "false"]) + def test_an_operators_own_value_wins(self, monkeypatch, chosen): + from forgelm.cli._config_load import _disable_hub_telemetry + + monkeypatch.setenv("HF_HUB_DISABLE_TELEMETRY", chosen) + _disable_hub_telemetry() + assert os.environ["HF_HUB_DISABLE_TELEMETRY"] == chosen + + def test_main_sets_it_before_anything_else_runs(self, monkeypatch): + from forgelm.cli import _dispatch + + self._unset(monkeypatch) + seen = {} + + def fake_inner(): + seen["at_dispatch"] = os.environ.get("HF_HUB_DISABLE_TELEMETRY") + return 0 + + with patch.object(_dispatch, "_main_inner", fake_inner): + _dispatch.main() + assert seen["at_dispatch"] == "1" + + def test_importing_the_cli_does_not_import_huggingface_hub(self): + # huggingface_hub reads HF_HUB_DISABLE_TELEMETRY once, at import. main() sets the variable, so nothing the CLI + # module imports at load time may import it first (a top-level `import transformers` in forgelm.cli would). + import subprocess + import sys + + code = "import sys, forgelm.cli; print(any(m == 'huggingface_hub' or m.startswith('huggingface_hub.') for m in sys.modules))" + out = subprocess.run( + [sys.executable, "-c", code], env=self._clean_env(), capture_output=True, text=True, timeout=120 + ) + assert out.returncode == 0, out.stderr + assert out.stdout.strip() == "False" + + def test_a_fresh_process_that_calls_it_reads_the_variable_as_true(self): + pytest.importorskip("huggingface_hub") + import subprocess + import sys + + code = ( + "from forgelm.cli._config_load import _disable_hub_telemetry; _disable_hub_telemetry(); " + "import huggingface_hub.constants as c; print(c.HF_HUB_DISABLE_TELEMETRY)" + ) + out = subprocess.run( + [sys.executable, "-c", code], env=self._clean_env(), capture_output=True, text=True, timeout=120 + ) + assert out.returncode == 0, out.stderr + assert out.stdout.strip() == "True" + + +class TestReservedRetentionEnforceWarning: + """`retention.enforce` is accepted but no command reads it; the CLI says so (#92).""" + + @staticmethod + def _config(enforce): + from forgelm.config import ForgeConfig + from tests._helpers.factories import minimal_config + + return ForgeConfig(**minimal_config(retention={"enforce": enforce})) + + def test_the_default_is_silent(self, caplog): + import logging + + from forgelm.cli._config_load import _warn_reserved_retention_enforce + + with caplog.at_level(logging.WARNING): + _warn_reserved_retention_enforce(self._config("log_only")) + assert not [r for r in caplog.records if "retention.enforce" in r.getMessage()] + + @pytest.mark.parametrize("mode", ["warn_on_excess", "block_on_excess"]) + def test_a_gate_that_does_not_exist_is_called_out(self, caplog, mode): + import logging + + from forgelm.cli._config_load import _warn_reserved_retention_enforce + + with caplog.at_level(logging.WARNING): + _warn_reserved_retention_enforce(self._config(mode)) + warned = [r.getMessage() for r in caplog.records if "retention.enforce" in r.getMessage()] + assert len(warned) == 1 and mode in warned[0] and "no command acts on it" in warned[0] + + def test_loading_a_config_file_triggers_it(self, tmp_path, caplog, minimal_config): + import logging + + from forgelm.cli._config_load import _load_config_or_exit + + path = tmp_path / "c.yaml" + path.write_text(yaml.dump(minimal_config(retention={"enforce": "block_on_excess"}))) + with caplog.at_level(logging.WARNING): + _load_config_or_exit(str(path), json_output=False) + assert any("retention.enforce" in r.getMessage() for r in caplog.records) diff --git a/tests/test_config.py b/tests/test_config.py index 10b81fd1..70296912 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -935,6 +935,17 @@ def test_legacy_evaluation_staging_ttl_days_is_rejected(self): with pytest.raises(ValidationError, match="staging_ttl_days"): ForgeConfig(**minimal_config(evaluation={"staging_ttl_days": 14})) + +class TestRetentionConfig: + """`retention` horizons are reported, not enforced; `enforce` is reserved (#92).""" + + def test_the_description_does_not_promise_a_gate(self): + from forgelm.config import RetentionConfig + + description = RetentionConfig.model_fields["enforce"].description or "" + assert "no command reads it" in description + assert "EXIT_EVAL_FAILURE" not in description and "aborts" not in description + def test_canonical_retention_staging_ttl_days_still_works(self): cfg = ForgeConfig(**minimal_config(retention={"staging_ttl_days": 30})) assert cfg.retention.staging_ttl_days == 30 diff --git a/tests/test_distributed.py b/tests/test_distributed.py index d7127735..b51d7819 100644 --- a/tests/test_distributed.py +++ b/tests/test_distributed.py @@ -66,9 +66,9 @@ def test_deepspeed_config(self, minimal_config): assert cfg.distributed.deepspeed_config == "zero3" def test_fsdp_config(self, minimal_config): - cfg = ForgeConfig(**minimal_config(distributed={"strategy": "fsdp", "fsdp_strategy": "hybrid_shard"})) + cfg = ForgeConfig(**minimal_config(distributed={"strategy": "fsdp", "fsdp_strategy": "shard_grad_op"})) assert cfg.distributed.strategy == "fsdp" - assert cfg.distributed.fsdp_strategy == "hybrid_shard" + assert cfg.distributed.fsdp_strategy == "shard_grad_op" def test_unsloth_distributed_raises(self, minimal_config): """Unsloth + distributed should raise ValueError.""" @@ -306,3 +306,113 @@ def test_fsdp_strategy_does_not_require_deepspeed(self, monkeypatch, minimal_con kwargs: dict = {} ForgeTrainer._apply_distributed_config(object.__new__(ForgeTrainer), kwargs, cfg.distributed) assert "deepspeed" not in kwargs + + +# --- FSDP: strategies transformers' FSDP2 default cannot express (#104) --- + + +class TestFsdp1OnlyStrategiesAreRejectedAtConfigTime: + """``no_shard`` and ``hybrid_shard`` used to pass ``--dry-run`` and exit 2 at trainer construction. + + transformers 5.10+ runs FSDP2 by default and raises ``fsdp=hybrid_shard is only available with FSDP1`` when + ``TrainingArguments`` is built, so the config is refused up front (exit 1) instead of after the model is loaded. + """ + + @pytest.mark.parametrize("strategy", ["no_shard", "hybrid_shard"]) + def test_the_schema_refuses_them(self, strategy): + with pytest.raises(ValidationError, match="fsdp_strategy"): + DistributedConfig(strategy="fsdp", fsdp_strategy=strategy) + + @pytest.mark.parametrize("strategy", ["no_shard", "hybrid_shard"]) + def test_dry_run_exits_with_a_config_error(self, tmp_path, minimal_config, strategy): + from unittest.mock import patch + + from forgelm.cli import EXIT_CONFIG_ERROR, main + + cfg_path = tmp_path / "config.yaml" + cfg_path.write_text(yaml.dump(minimal_config(distributed={"strategy": "fsdp", "fsdp_strategy": strategy}))) + with patch("sys.argv", ["forgelm", "--config", str(cfg_path), "--dry-run"]): + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code == EXIT_CONFIG_ERROR + + @pytest.mark.parametrize("strategy", ["full_shard", "shard_grad_op"]) + @pytest.mark.parametrize("auto_wrap", [True, False]) + @pytest.mark.parametrize("offload", [True, False]) + @pytest.mark.parametrize("prefetch", ["backward_pre", "backward_post"]) + @pytest.mark.parametrize("state_dict", ["FULL_STATE_DICT", "SHARDED_STATE_DICT"]) + def test_every_accepted_variant_builds_training_arguments( + self, tmp_path, strategy, auto_wrap, offload, prefetch, state_dict + ): + """What the schema accepts must be what ``TrainingArguments`` accepts (no config that dies later).""" + from types import SimpleNamespace + + from transformers import TrainingArguments + + from forgelm.trainer import ForgeTrainer + + dist_cfg = DistributedConfig( + strategy="fsdp", + fsdp_strategy=strategy, + fsdp_auto_wrap=auto_wrap, + fsdp_offload=offload, + fsdp_backward_prefetch=prefetch, + fsdp_state_dict_type=state_dict, + ) + kwargs: dict = {} + ForgeTrainer._apply_distributed_config(SimpleNamespace(), kwargs, dist_cfg) + TrainingArguments(output_dir=str(tmp_path), use_cpu=True, report_to=[], **kwargs) + + +class TestInertFsdpOptionsAreCalledOut: + """``fsdp_auto_wrap: false`` and ``fsdp_backward_prefetch: backward_post`` are ignored by FSDP2; say so (#104).""" + + @staticmethod + def _config(**distributed): + from tests._helpers.factories import minimal_config + + return ForgeConfig(**minimal_config(distributed={"strategy": "fsdp", **distributed})) + + @staticmethod + def _warnings(config, caplog): + import logging + + from forgelm.cli._config_load import _warn_inert_fsdp_options + + with caplog.at_level(logging.WARNING): + _warn_inert_fsdp_options(config) + return [r.getMessage() for r in caplog.records if "distributed.fsdp_" in r.getMessage()] + + def test_the_defaults_are_silent(self, caplog): + assert self._warnings(self._config(), caplog) == [] + + def test_turning_auto_wrap_off_is_called_out(self, caplog): + warned = self._warnings(self._config(fsdp_auto_wrap=False), caplog) + assert len(warned) == 1 and "fsdp_auto_wrap=false has no effect" in warned[0] + + def test_backward_post_is_called_out(self, caplog): + warned = self._warnings(self._config(fsdp_backward_prefetch="backward_post"), caplog) + assert len(warned) == 1 and "backward_post" in warned[0] + + def test_other_strategies_and_no_distributed_block_are_silent(self, caplog): + from tests._helpers.factories import minimal_config + + deepspeed = ForgeConfig(**minimal_config(distributed={"strategy": "deepspeed", "fsdp_auto_wrap": False})) + plain = ForgeConfig(**minimal_config()) + assert self._warnings(deepspeed, caplog) == [] + assert self._warnings(plain, caplog) == [] + + def test_a_dry_run_reaches_the_warning(self, tmp_path, caplog, minimal_config): + import logging + from unittest.mock import patch + + from forgelm.cli import EXIT_SUCCESS, main + + cfg_path = tmp_path / "config.yaml" + cfg_path.write_text(yaml.dump(minimal_config(distributed={"strategy": "fsdp", "fsdp_auto_wrap": False}))) + with caplog.at_level(logging.WARNING): + with patch("sys.argv", ["forgelm", "--config", str(cfg_path), "--dry-run"]): + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code == EXIT_SUCCESS + assert any("fsdp_auto_wrap=false has no effect" in r.getMessage() for r in caplog.records) diff --git a/tests/test_gdpr_erasure.py b/tests/test_gdpr_erasure.py index bcd3c6e9..06c27775 100644 --- a/tests/test_gdpr_erasure.py +++ b/tests/test_gdpr_erasure.py @@ -152,6 +152,56 @@ def test_row_erasure_target_id_is_hashed_not_cleartext(self, tmp_path: Path) -> # --------------------------------------------------------------------------- +class TestImplicitOutputDirWarning: + """`--output-dir` defaults to the corpus directory; the user is told, because the memorisation check looks for + `/final_model/` and a training run rarely writes its model next to the corpus (#102).""" + + def test_the_implicit_directory_is_named_in_a_warning(self, tmp_path: Path, caplog) -> None: + import logging + + from forgelm.cli.subcommands._purge import _run_purge_cmd + + corpus = tmp_path / "data" / "train.jsonl" + corpus.parent.mkdir() + _seed_corpus(corpus, [{"id": "row-A", "text": "x"}]) + args = _build_args(row_id="row-A", corpus=str(corpus), output_dir=None) + with caplog.at_level(logging.WARNING, logger="forgelm"): + with pytest.raises(SystemExit) as ei: + _run_purge_cmd(args, output_format="json") + assert ei.value.code == 0 + warned = [r.getMessage() for r in caplog.records if "--output-dir not given" in r.getMessage()] + assert len(warned) == 1 + assert str(corpus.parent) in warned[0] and os.path.join(str(corpus.parent), "final_model") in warned[0] + + def test_an_explicit_output_dir_is_silent(self, tmp_path: Path, caplog) -> None: + import logging + + from forgelm.cli.subcommands._purge import _run_purge_cmd + + corpus = tmp_path / "train.jsonl" + _seed_corpus(corpus, [{"id": "row-A", "text": "x"}]) + args = _build_args(row_id="row-A", corpus=str(corpus), output_dir=str(tmp_path)) + with caplog.at_level(logging.WARNING, logger="forgelm"): + with pytest.raises(SystemExit): + _run_purge_cmd(args, output_format="json") + assert not [r for r in caplog.records if "--output-dir not given" in r.getMessage()] + + def test_no_memorisation_event_without_a_final_model_in_the_resolved_directory(self, tmp_path: Path) -> None: + # The default directory is the corpus's; a model written elsewhere is invisible to the check. The warning + # above is what tells the operator. + from forgelm.cli.subcommands._purge import _run_purge_cmd + + corpus = tmp_path / "data" / "train.jsonl" + corpus.parent.mkdir() + _seed_corpus(corpus, [{"id": "row-A", "text": "x"}]) + (tmp_path / "training_out" / "final_model").mkdir(parents=True) + args = _build_args(row_id="row-A", corpus=str(corpus), output_dir=None) + with pytest.raises(SystemExit): + _run_purge_cmd(args, output_format="json") + events = _read_audit_events(corpus.parent / "audit_log.jsonl") + assert "data.erasure_warning_memorisation" not in [e["event"] for e in events] + + class TestSaltPersistence: def test_salt_file_created_on_first_use_with_mode_0600(self, tmp_path: Path) -> None: from forgelm.cli.subcommands._purge import _resolve_salt diff --git a/tests/test_model.py b/tests/test_model.py index 771a671e..cac537e0 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -115,15 +115,105 @@ def test_quantization_config_added_with_cuda(self): assert "quantization_config" in kwargs fake_bnb.assert_called_once() - def test_rope_and_sliding_window_passed_through(self): + def test_sliding_window_alone_is_a_keyword_argument(self): cfg = _cfg( model={"name_or_path": "org/model", "load_in_4bit": False}, - training={"rope_scaling": {"type": "linear", "factor": 2.0}, "sliding_window_attention": 4096}, + training={"sliding_window_attention": 4096}, ) with patch("torch.cuda.is_available", return_value=False): kwargs = model_mod._build_model_kwargs(cfg, trust_remote_code=False) - assert kwargs["rope_scaling"] == {"type": "linear", "factor": 2.0} - assert kwargs["sliding_window"] == 4096 + assert kwargs["sliding_window"] == 4096 and "config" not in kwargs + + def test_rope_scaling_travels_in_a_config_object_not_a_keyword(self): + # transformers 5 no longer accepts `rope_scaling=` on from_pretrained (#103): the settings go into the + # config, together with the sliding-window override, which would otherwise reach the model constructor. + model_config = SimpleNamespace(rope_parameters={"rope_theta": 10000.0, "rope_type": "default"}) + fake_transformers = MagicMock() + fake_transformers.AutoConfig.from_pretrained.return_value = model_config + cfg = _cfg( + model={"name_or_path": "org/model", "load_in_4bit": False}, + training={"rope_scaling": {"type": "linear", "factor": 2.0}, "sliding_window_attention": 4096}, + ) + with ( + patch("torch.cuda.is_available", return_value=False), + patch.dict("sys.modules", {"transformers": fake_transformers}), + ): + kwargs = model_mod._build_model_kwargs(cfg, trust_remote_code=False, revision="abc123") + assert "rope_scaling" not in kwargs and "sliding_window" not in kwargs + assert kwargs["config"] is model_config + assert model_config.rope_parameters == {"rope_theta": 10000.0, "rope_type": "linear", "factor": 2.0} + assert model_config.sliding_window == 4096 + fake_transformers.AutoConfig.from_pretrained.assert_called_once_with( + "org/model", trust_remote_code=False, revision="abc123" + ) + + +class TestConfigWithRopeScaling: + """Against a real (tiny, local, CPU) Llama config: the merged settings must build and run a model (#103).""" + + @staticmethod + def _tiny_model_dir(tmp_path): + from transformers import LlamaConfig + + LlamaConfig( + vocab_size=64, + hidden_size=32, + intermediate_size=64, + num_hidden_layers=1, + num_attention_heads=2, + num_key_value_heads=2, + max_position_embeddings=128, + ).save_pretrained(tmp_path) + return str(tmp_path) + + @pytest.mark.parametrize( + "requested, expected_type", + [ + ({"type": "linear", "factor": 4.0}, "linear"), # the key every documented example and the wizard use + ({"rope_type": "linear", "factor": 4.0}, "linear"), + ({"type": "dynamic", "factor": 2.0}, "dynamic"), + ({"rope_type": "yarn", "factor": 2.0, "original_max_position_embeddings": 128}, "yarn"), + ], + ) + def test_the_merged_config_builds_a_model_that_runs(self, tmp_path, requested, expected_type): + import torch + from transformers import AutoModelForCausalLM + + path = self._tiny_model_dir(tmp_path) + model_config = model_mod._config_with_rope_scaling(path, requested, trust_remote_code=False) + assert model_config.rope_parameters["rope_type"] == expected_type + assert model_config.rope_parameters["rope_theta"] == 10000.0 # kept from the model, not lost + assert "type" not in model_config.rope_parameters + logits = AutoModelForCausalLM.from_config(model_config)(input_ids=torch.tensor([[1, 2, 3, 4]])).logits + assert tuple(logits.shape) == (1, 4, 64) + + def test_requested_rope_theta_overrides_the_models(self, tmp_path): + path = self._tiny_model_dir(tmp_path) + model_config = model_mod._config_with_rope_scaling( + path, {"type": "linear", "factor": 2.0, "rope_theta": 500000.0}, trust_remote_code=False + ) + assert model_config.rope_parameters["rope_theta"] == 500000.0 + + def test_a_transformers_without_rope_parameters_gets_the_legacy_attribute(self): + legacy = SimpleNamespace() + fake_transformers = MagicMock() + fake_transformers.AutoConfig.from_pretrained.return_value = legacy + with patch.dict("sys.modules", {"transformers": fake_transformers}): + out = model_mod._config_with_rope_scaling( + "org/model", {"type": "linear", "factor": 2.0}, trust_remote_code=False + ) + assert out is legacy and legacy.rope_scaling == {"rope_type": "linear", "factor": 2.0} + + def test_multimodal_settings_land_in_the_text_config(self): + text = SimpleNamespace(rope_parameters={"rope_theta": 1.0, "rope_type": "default"}) + outer = SimpleNamespace(get_text_config=lambda: text) + fake_transformers = MagicMock() + fake_transformers.AutoConfig.from_pretrained.return_value = outer + with patch.dict("sys.modules", {"transformers": fake_transformers}): + out = model_mod._config_with_rope_scaling( + "org/vlm", {"type": "linear", "factor": 2.0}, trust_remote_code=False + ) + assert out is outer and text.rope_parameters["rope_type"] == "linear" class TestLoadTokenizer: @@ -152,6 +242,47 @@ def test_existing_pad_token_preserved(self): assert fake_tok.pad_token == "[PAD]" +class TestChatTemplateNameCannotEscapeTheSaveDirectory: + """CVE-2026-9856: a ``chat_template`` name from a Hub repo was used as a file name by ``save_pretrained``. + + ForgeLM loads a tokenizer from a Hub repo and saves it again to ``final_model/`` (trainer, GGUF export, merge), so + a crafted ``tokenizer_config.json`` could write a file outside the output directory. Fixed in transformers + 5.10.0 (yanked) and 5.10.1, which is why the declared floor is ``>=5.10.1``. This uses the installed transformers + and a tiny local tokenizer, so it also fails on a floor install that predates the fix. + """ + + def _crafted_tokenizer_dir(self, root): + import json + + from tokenizers import Tokenizer, models, pre_tokenizers + from transformers import PreTrainedTokenizerFast + + tok = Tokenizer(models.WordLevel({"[UNK]": 0, "hello": 1, "": 2}, unk_token="[UNK]")) + tok.pre_tokenizer = pre_tokenizers.Whitespace() + source = root / "hub_repo" + PreTrainedTokenizerFast(tokenizer_object=tok, unk_token="[UNK]", eos_token="").save_pretrained(source) + config_file = source / "tokenizer_config.json" + config = json.loads(config_file.read_text(encoding="utf-8")) + config["chat_template"] = [ + {"name": "../../../ESCAPED_BY_CHAT_TEMPLATE", "template": "{{ messages }}"}, + {"name": "default", "template": "{{ messages }}"}, + ] + config_file.write_text(json.dumps(config), encoding="utf-8") + return source + + def test_a_traversal_name_is_refused_and_nothing_is_written_outside(self, tmp_path): + source = self._crafted_tokenizer_dir(tmp_path) + out = tmp_path / "run" / "final_model" + cfg = _cfg(model={"name_or_path": str(source)}) + + with pytest.raises(ValueError, match="Invalid chat template name"): + tokenizer = model_mod._load_tokenizer(cfg, trust_remote_code=False) + tokenizer.save_pretrained(out) + + escaped = [p for p in tmp_path.rglob("*") if p.is_file() and "ESCAPED" in p.name] + assert escaped == [], f"a chat_template name wrote outside the save directory: {escaped}" + + class TestBuildLoraConfig: """``_build_lora_config`` resolves the PEFT method and target modules.""" diff --git a/tests/test_packaging.py b/tests/test_packaging.py index ab8c562e..3018791c 100644 --- a/tests/test_packaging.py +++ b/tests/test_packaging.py @@ -244,6 +244,85 @@ def test_setuptools_security_floor_meets_pysec_2026_3447() -> None: ) +def _load_runtime_dependencies() -> list: + try: + import tomllib # type: ignore[import-not-found] + except ModuleNotFoundError: # pragma: no cover — Python 3.10 fallback + try: + import tomli as tomllib # type: ignore[import-not-found, no-redef] + except ModuleNotFoundError: + pytest.skip("Neither tomllib (3.11+) nor tomli is available; runtime dependency assertion skipped.") + pyproject_path = Path(__file__).resolve().parent.parent / "pyproject.toml" + with pyproject_path.open("rb") as fh: + pyproject = tomllib.load(fh) + return pyproject.get("project", {}).get("dependencies", []) + + +# transformers 5.10.1 is the first installable release past three advisories that reach ForgeLM's own load and +# save path from a Hub repo: CVE-2026-4372 (5.3.0), CVE-2026-5241 (a LightGlue config.json overrides +# trust_remote_code; 5.5.0) and CVE-2026-9856 (a chat_template name used as a file name by save_pretrained; +# 5.10.0, which PyPI yanked). A floor of ">=5.3.0" was documented as "the CVE fix floor" and was not. +_TRANSFORMERS_SECURITY_FLOOR = Version("5.10.1") + + +def test_transformers_floor_is_past_cve_2026_5241_and_cve_2026_9856() -> None: + """``transformers`` must not admit a release that predates the chat_template / trust_remote_code fixes.""" + found = [ + Requirement(spec) + for spec in _load_runtime_dependencies() + if canonicalize_name(Requirement(spec).name) == "transformers" + ] + assert len(found) == 1, f"expected exactly one transformers requirement in [project].dependencies, found {found!r}" + specifier_set = found[0].specifier + lower_bounds = [ + Version(spec.version.rstrip(".*")) for spec in specifier_set if spec.operator in _LOWER_BOUND_OPERATORS + ] + assert lower_bounds, f"transformers entry {str(found[0])!r} has no lower bound" + assert max(lower_bounds) >= _TRANSFORMERS_SECURITY_FLOOR, ( + f"transformers entry {str(found[0])!r} floors at {max(lower_bounds)}, below {_TRANSFORMERS_SECURITY_FLOOR}, " + "the first installable release past CVE-2026-5241 and CVE-2026-9856. Do not lower it without re-verifying " + "both advisories against the release you lower it to." + ) + for vulnerable in ("5.3.0", "5.4.0", "5.9.0", "5.10.0"): + assert not specifier_set.contains(Version(vulnerable), prereleases=True), ( + f"transformers entry {str(found[0])!r} still admits {vulnerable}" + ) + + +def test_nightly_floor_job_pins_the_runtime_dependency_minimums() -> None: + """Every ``name==X`` the nightly minimum-versions job installs is the ``>=X`` floor in pyproject. + + The job installs with ``--no-deps`` afterwards, so it can only test the documented minimum if its list is the + same list. It is a second hand-kept copy of every floor, and it kept saying ``transformers==5.3.0`` after the + floor was meant to be a security floor. + """ + import re + + nightly = (Path(__file__).resolve().parents[1] / ".github" / "workflows" / "nightly.yml").read_text( + encoding="utf-8" + ) + step = nightly.split("Install minimum dependency versions", 1)[1].split("- name:", 1)[0] + pins = { + canonicalize_name(name): version for name, version in re.findall(r'"([A-Za-z0-9_.-]+)==([0-9][^"]*)"', step) + } + assert pins, "the nightly minimum-versions step no longer installs `name==version` pins — update this test" + + floors = {} + for spec in _load_runtime_dependencies(): + req = Requirement(spec) + lowers = [s.version for s in req.specifier if s.operator == ">="] + if lowers: + floors[canonicalize_name(req.name)] = lowers[0] + + for name, version in pins.items(): + assert name in floors, f"the nightly floor job pins {name}=={version}, which is not a runtime dependency" + assert version == floors[name], ( + f"the nightly floor job pins {name}=={version} but pyproject.toml declares {name}>={floors[name]}; " + "the job would test a minimum nobody documents" + ) + assert {"transformers", "huggingface-hub"} <= set(pins), "the floor job must pin transformers and huggingface_hub" + + def _lint_job_pins() -> dict[str, Requirement]: """Requirements the ci.yml lint job installs directly. diff --git a/tools/check_module_size.py b/tools/check_module_size.py index 35bb0b0d..5fc6af13 100644 --- a/tools/check_module_size.py +++ b/tools/check_module_size.py @@ -488,7 +488,7 @@ class _DeferredSplit: ), ), "forgelm/cli/subcommands/_purge.py": _DeferredSplit( - budget=1216, + budget=1224, deferred_at_loc=1215, budget_history=( "2026-10-09 (Phase 16 S1-S3 review, 1215 -> 1216, +1): the `dumps_strict` import. " @@ -496,6 +496,10 @@ class _DeferredSplit: "strict-JSON guard's hand-picked module list; the guard now scans all of " "`forgelm/`, so they route through the chokepoint like every other writer. " "One line, taken as a raise rather than a split per decision C-18.", + "2026-10-18 (wave-0 triage, #102, 1216 -> 1224, +8): the warning `purge --row-id` logs when " + "`--output-dir` is omitted. The default (the corpus directory) silently pointed the memorisation " + "check at a directory that never holds a trained model, so the message has to sit where the " + "default is resolved. Taken as a raise rather than a split per decision C-18.", ), reason=( "GDPR purge: row-id resolution + run-id resolution + retention-policy " diff --git a/tools/pip_audit_ignores.yaml b/tools/pip_audit_ignores.yaml index cd16a831..e70e7619 100644 --- a/tools/pip_audit_ignores.yaml +++ b/tools/pip_audit_ignores.yaml @@ -40,6 +40,9 @@ ignores: # so a suppression would be dead weight: # - CVE-2026-1839 — fixed in transformers 5.0.0rc3. # - PYSEC-2025-217 / CVE-2025-14929 (X-CLIP RCE) — OSV last_affected 5.0.0rc0. + # The floor has since moved to transformers>=5.10.1, the first installable + # release past CVE-2026-5241 (fixed 5.5.0) and CVE-2026-9856 (fixed 5.10.0, + # yanked), so neither needs a suppression either. # ----- torch (added 2026-05-21, triage in issue #58) ----- # From 700b6d6e67fa3202f8c391da1e87a7aafacc8345 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 12:31:10 +0300 Subject: [PATCH 10/22] docs: webhooks, DeepSpeed presets, first-run output tree and --offline described as the code behaves (#997) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Four documentation corrections combined into one pull request, replacing #993 and #994 (and two branches that were never opened). Each one removes a statement the code does not back. English and Turkish are updated together; `config.py`, `_parser.py` and the other budgeted modules change only by in-place string edits, so the module-size ratchet is untouched. ## Related issues Refs #112, #110, #111, #115, #410, #96, #534, #74, #100, #893. The code halves of #110/#111/#115 (shipping the presets), #112 (a Teams/Discord formatter) and #74/#100 (gating clients behind `--offline`) stay open. ## Changes | Theme | Said | Now says | |---|---|---| | **Webhooks** (`docs/usermanuals/{en,tr}/operations/webhooks.md`, config references, access-control SOP) | Slack, Teams and Discord all accept the payload directly | Slack accepts the `attachments` block; Teams (Office 365 Connectors retired, Workflows want an Adaptive Card) and Discord (needs `content`/`embeds`, answers HTTP 400) need a relay; the real payload is shown. Also: `url_env` is read **only when `url` is empty** (it said the opposite); `notify_on_failure` gates `training.failure`/`training.reverted`; no payload carries `FORGELM_OPERATOR`; eight events, not five | | **DeepSpeed presets** (references, troubleshooting, enterprise guide, user manuals, `config_template.yaml`) | built-in presets | the names resolve only from a source checkout, not from a wheel or the Docker image; known issue #410 for `zero2`; the "fits 70B on 8×24 GB" sentence is gone; the template's launch line is `-m forgelm` | | **First run** (`getting-started/first-run`, `project-layout`, `compliance/human-oversight`, EN+TR) | an `artifacts/` bundle with a `manifest.json` that hashes every file, `config_snapshot.yaml`, an invented training log and doctor output | the tree a run really writes; only the audit log (`verify-audit`) and `final_model/` (`verify-integrity`) are tamper-checked, nothing hashes `compliance/`; the renderer's real doctor format | | **`--offline`** (`--help`, `model.offline` description, references, enterprise guide, air-gap page) | "Air-gapped mode", "prevents any network calls", Comet | sets the three Hugging Face offline variables; does not stop webhooks, judge/synthetic-data API clients or W&B/MLflow; Comet is not a `report_to` value | ### Checked Webhook payload and `url_env` precedence against `forgelm/webhook.py`; the output tree against `trainer.py`/`compliance.py`; `forgelm doctor --offline` run for the real format; `TrainingArguments`-independent. `forgelm` `--help` text and field descriptions are edited in place. ## Type - [x] Documentation ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0 (bilingual parity, code-block parity, anchors, CLI-help consistency, field descriptions, YAML snippets, schema drift). ## Checklist - [x] English and Turkish mirrors updated together (Turkish wording needs the owner's review) - [x] CHANGELOG entries under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Update bilingual documentation and user-facing descriptions to accurately reflect ForgeLM's current runtime behavior. Enhancements: - Align English and Turkish documentation, CLI help, and configuration field descriptions with the implemented webhook behavior, DeepSpeed preset availability, first-run output structure, and offline-mode scope. Documentation: - Correct webhook documentation to describe the Slack-format payload, relay requirements for Teams and Discord, URL precedence, notification coverage, and the complete event vocabulary. - Document that DeepSpeed preset names require a source checkout, identify the known zero2 issue, and replace unsupported preset and capacity claims with custom JSON guidance. - Replace the fictitious first-run artifacts and output examples with the actual run directory tree, tamper-verification boundaries, approval staging behavior, and doctor output. - Clarify that --offline only configures Hugging Face offline behavior and does not disable webhooks, API clients, or experiment tracking integrations. Tests: - Update the changelog with the documentation corrections. ## Summary by CodeRabbit * **Documentation** * Clarified that offline mode blocks Hugging Face, Transformers, and Datasets network access, but does not disable webhooks, API clients, or experiment trackers. * Updated webhook guidance on supported payloads, relay requirements, URL precedence, event types, and failure notifications. * Clarified that DeepSpeed presets require a source checkout; examples now favor custom JSON configuration paths and note a ZeRO-2 preset issue. * Updated run-output examples and integrity-check guidance, and corrected human-approval command examples. --------- Co-authored-by: Claude Sonnet 5.5 --- CHANGELOG.md | 31 ++++++++ config_template.yaml | 4 +- docs/guides/enterprise_deployment.md | 4 +- docs/guides/troubleshooting-tr.md | 7 +- docs/guides/troubleshooting.md | 6 +- docs/qms/access_control-tr.md | 34 +++++---- docs/qms/access_control.md | 35 +++++---- docs/reference/configuration-tr.md | 10 +-- docs/reference/configuration.md | 10 +-- docs/reference/distributed_training-tr.md | 16 +++- docs/reference/distributed_training.md | 16 +++- .../en/compliance/human-oversight.md | 8 +- .../en/getting-started/first-run.md | 76 ++++++++++--------- .../en/getting-started/project-layout.md | 15 ++-- docs/usermanuals/en/operations/air-gap.md | 2 +- docs/usermanuals/en/operations/webhooks.md | 67 ++++++++++------ docs/usermanuals/en/reference/cli.md | 2 +- .../usermanuals/en/reference/configuration.md | 6 +- docs/usermanuals/en/training/distributed.md | 4 +- .../tr/compliance/human-oversight.md | 8 +- .../tr/getting-started/first-run.md | 76 ++++++++++--------- .../tr/getting-started/project-layout.md | 15 ++-- docs/usermanuals/tr/operations/air-gap.md | 2 +- docs/usermanuals/tr/operations/webhooks.md | 66 ++++++++++------ docs/usermanuals/tr/reference/cli.md | 2 +- .../usermanuals/tr/reference/configuration.md | 6 +- docs/usermanuals/tr/training/distributed.md | 4 +- forgelm/cli/_parser.py | 2 +- forgelm/config.py | 12 +-- 29 files changed, 324 insertions(+), 222 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c224ea95..84e7b970 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,6 +87,37 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Stop saying that Microsoft Teams and Discord accept the webhook payload directly. The user manual (English and Turkish) and the + `webhook` field descriptions said all three ingest it; the body is a Slack-style `attachments` block with no top-level `text`, + `content` or `embeds`. Discord needs `content` or `embeds` and answers HTTP 400; Teams retired Office 365 Connectors in May 2026 and + its Workflows webhooks expect an Adaptive Card. Both need a relay. The page now shows the real payload, and the access-control SOP + no longer claims that a payload carries `FORGELM_OPERATOR` (it does not) or lists five events (there are eight). The `url_env` + description said it overrides `url`; it is read only when `url` is empty, so `url` wins. A formatter for Teams and Discord is a + separate, larger change and stays open on the issue (#112). +- Say that the DeepSpeed preset names (`zero2`, `zero3`, `zero3_offload`) work only from a source checkout. They resolve to + `configs/deepspeed/.json` beside the package or in the current directory, and neither the wheel nor the Docker image ships + that folder, so on a `pip install` the names fail with a configuration error (and the unset-`deepspeed_config` default is + `zero2`). The configuration and distributed-training references, the troubleshooting entry (its sample error is now the real + message), the enterprise guide, the user manuals, `config_template.yaml` and the field description now say so and point at a + JSON path; the template's `torchrun … forgelm` launch line is `-m forgelm`, the only form that runs the checkout. The + unverified "`zero3_offload` fits 70B on 8×24 GB" sentence is gone, and the `zero2` preset's own start-up failure is noted + (#410). Shipping the presets in the package, and a test that resolves one from an installed wheel, are code changes and stay + open on the issues (#110, #111, #115). +- Show the real output directory on the first-run, project-layout and human-oversight pages (English and Turkish). They described a + `checkpoints//artifacts/` bundle with a `manifest.json` that hashes "every file inside … for tamper-evidence", a + `config_snapshot.yaml`, a `safety_report.json` and a `data_audit_report.json`. None of those exist: a run writes `audit_log.jsonl` with its + `.manifest.json` genesis pin, a `compliance/` directory (`compliance_report.json`, `training_manifest.yaml`, `data_provenance.json`, + `data_governance_report.json`, and `annex_iv_metadata.json` only when the YAML names a provider, system or purpose), `safety/` and + `benchmark/` only when enabled, and `final_model/` with the weights, model card, `deployer_instructions.md` and `model_integrity.json`. + Only the audit log and `final_model/` are tamper-checked; nothing hashes `compliance/`, and the pages now say so. The invented training log + (a DPO pass in an SFT-only template, `Llama Guard S1-S14: clean`) is replaced by a description of the real stages, the approval-gate + sample shows the two log lines the trainer prints, and the `forgelm doctor` sample uses the real renderer's format (ASCII header and + `[+ pass]` markers, "discoverable" wording, an abridged row list) (#96, #534). +- Describe `--offline` / `model.offline` as what it does. It sets `HF_HUB_OFFLINE`, `TRANSFORMERS_OFFLINE` and `HF_DATASETS_OFFLINE`; it does not + stop webhooks, the LLM-judge and synthetic-data API clients, or W&B / MLflow. The `--offline` help text, the `model.offline` field + description, the configuration and CLI references (English and Turkish) and the enterprise deployment guide said "Air-gapped mode" and + "prevents any network calls"; the air-gap page no longer lists Comet, which `training.report_to` does not accept. Gating those clients + behind the flag is a code change and stays open (#74, #100, #893). - `training.rope_scaling` works on transformers 5 again. The setting was passed to `from_pretrained(rope_scaling=...)`, which no longer works: the documented `{type: linear, factor: 4}` raised `KeyError: 'rope_type'` while the model was built, and `{rope_type: ..., factor: ...}` replaced the whole parameter dict so `rope_theta` was lost (`TypeError`). diff --git a/config_template.yaml b/config_template.yaml index acf197cb..415cc2c9 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -162,10 +162,10 @@ data: # dare_seed: 42 # DARE: RNG seed for the random drop mask # Distributed training (multi-GPU). Requires: pip install forgelm[distributed] -# Launch with: torchrun --nproc_per_node=N forgelm --config my_config.yaml +# Launch with: torchrun --nproc_per_node=N -m forgelm --config my_config.yaml # distributed: # strategy: "deepspeed" # "deepspeed" or "fsdp" -# deepspeed_config: "zero2" # preset: "zero2", "zero3", "zero3_offload" or path to JSON +# deepspeed_config: "ds.json" # path to a DeepSpeed JSON (the presets "zero2", "zero3", "zero3_offload" need a source checkout) # # --- FSDP options (when strategy: "fsdp") --- # # fsdp_strategy: "full_shard" # "full_shard" or "shard_grad_op" (FSDP2 supports no other) # # fsdp_auto_wrap: true diff --git a/docs/guides/enterprise_deployment.md b/docs/guides/enterprise_deployment.md index f9c70e60..67231902 100644 --- a/docs/guides/enterprise_deployment.md +++ b/docs/guides/enterprise_deployment.md @@ -121,7 +121,7 @@ ForgeLM sets `HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, `HF_DATASETS_OFFLINE= ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # or "zero3", "zero3_offload" + deepspeed_config: "/etc/forgelm/ds_zero2.json" # your own DeepSpeed JSON; the zero2/zero3/zero3_offload preset names need a source checkout ``` ```bash @@ -152,7 +152,7 @@ See the [Distributed Training Guide](../reference/distributed_training.md) for d | `trust_remote_code` | `false` | Prevents arbitrary code execution from model repos | | `auth.hf_token` | Use `HUGGINGFACE_TOKEN` env var | Don't hardcode tokens in YAML | | `webhook.url` | Always use `url_env` with environment variable | Webhook tokens excluded from model cards (v0.3.1rc1+), but avoid direct URLs for credential hygiene | -| `offline` | `true` for air-gapped | Prevents any network calls | +| `offline` | `true` for air-gapped | Blocks Hugging Face Hub, transformers and datasets network calls (it sets `HF_HUB_OFFLINE`, `TRANSFORMERS_OFFLINE`, `HF_DATASETS_OFFLINE`). It does **not** stop webhooks, judge or synthetic-data API clients, or W&B / MLflow: disable those (omit `webhook:`, use local judge and teacher models, set `report_to: none`) for a true air gap | --- diff --git a/docs/guides/troubleshooting-tr.md b/docs/guides/troubleshooting-tr.md index c4cb2408..29e6331c 100644 --- a/docs/guides/troubleshooting-tr.md +++ b/docs/guides/troubleshooting-tr.md @@ -363,11 +363,12 @@ nvidia-smi ### DeepSpeed Config bulunamadı ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -ForgeLM proje kökünden çalıştırdığınızdan emin olun ya da mutlak yol -kullanın: +Ön ayar adları yalnızca bir kaynak checkout'unda çözülür (paketin yanındaki ya da geçerli dizindeki bir +`configs/deepspeed/` dizini); `pip install` veya Docker imajı bunları içermez. ForgeLM proje kökünden çalıştırın ya da +bir DeepSpeed JSON dosyasının yolunu kullanın: ```yaml distributed: diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 7533eedb..9549875b 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -344,10 +344,12 @@ nvidia-smi ### DeepSpeed Config Not Found ```text -FileNotFoundError: DeepSpeed preset 'zero2' not found +DeepSpeed preset 'zero2' not found at /configs/deepspeed/zero2.json ``` -Ensure you're running from the ForgeLM project root, or use an absolute path: +The preset names resolve only from a source checkout (a `configs/deepspeed/` directory beside the package or in the +current directory); a `pip install` or the Docker image does not contain them. Run from the ForgeLM project root, or +use the path of a DeepSpeed JSON file: ```yaml distributed: diff --git a/docs/qms/access_control-tr.md b/docs/qms/access_control-tr.md index 13341c3a..67f3ea60 100644 --- a/docs/qms/access_control-tr.md +++ b/docs/qms/access_control-tr.md @@ -230,20 +230,26 @@ webhook: ForgeLM webhook gövdelerini HMAC ile **imzalamaz** — `WebhookConfig` üzerinde `webhook.secret_env` alanı yoktur (bkz. -`forgelm/config.py::WebhookConfig`). Hedef-tarafı atıf (a) HTTPS + -`webhook.url_env` üzerinden URL-gizliliğine, (b) curated payload -içinde taşınan `FORGELM_OPERATOR` kimliğine, ve (c) alıcı sistemin -kendi bearer-token / imzalı-istek kontrollerine (Slack signing -secret, Teams connector token) düşer. - -Webhook wire-format olay sözlüğü (5 olay): `training.start`, -`training.success`, `training.failure`, `training.reverted`, -`approval.required` — Phase 8. In-process notifier metot adları -`notify_*` öneki kullanır (`notify_start`, `notify_success`, -`notify_failure`, `notify_reverted`, `notify_awaiting_approval`) -ve bu beş wire olayına dispatch eder. Her payload `FORGELM_OPERATOR` -kimliğini taşır böylece alıcı sistem (Slack, Teams, custom -incident-management) bildirimi attribute edebilir. +`forgelm/config.py::WebhookConfig`). Hedef-tarafı özgünlük (a) HTTPS + +`webhook.url_env` üzerinden URL-gizliliğine ve (b) alıcı tarafın kendi +uyguladığı kimlik doğrulamaya düşer. Bir Slack incoming webhook'unun gizli +URL'sinden başka bir doğrulaması yoktur (Slack'in signing secret'ı Slack'ten +*gelen* istekleri doğrular, bir webhook URL'sine gelenleri değil); bu yüzden +özel bir relay kendi kimlik doğrulamasını istemelidir, çünkü ForgeLM imza +göndermez. Webhook payload'u `FORGELM_OPERATOR` +kimliğini **taşımaz**; bir modeli kimin çalıştırdığı veya onayladığı +bildirimden değil, audit log'dan okunur. + +Webhook wire-format olay sözlüğü sekiz olaydır: beş tek-aşamalı +yaşam-döngüsü olayı `training.start`, `training.success`, +`training.failure`, `training.reverted` ve `approval.required`, ayrıca +çok-aşamalı bir koşunun bunlarla birlikte yaydığı üç `pipeline.*` +olayı (`pipeline.started`, `pipeline.completed`, +`pipeline.stage_reverted`). In-process notifier metotları `notify_*` +öneki kullanır (`notify_start`, `notify_success`, `notify_failure`, +`notify_reverted`, `notify_awaiting_approval`, +`notify_pipeline_started`, `notify_pipeline_completed`, +`notify_pipeline_reverted`) ve bu wire olaylarına dispatch eder. ## 8. Doğrulama checklist'i diff --git a/docs/qms/access_control.md b/docs/qms/access_control.md index e3cb1aed..9e5f344a 100644 --- a/docs/qms/access_control.md +++ b/docs/qms/access_control.md @@ -228,21 +228,26 @@ webhook: ForgeLM does **not** HMAC-sign webhook bodies — there is no `webhook.secret_env` field on `WebhookConfig` (see -`forgelm/config.py::WebhookConfig`). Destination-side attribution falls to (a) -HTTPS + URL-secrecy via `webhook.url_env`, (b) the -`FORGELM_OPERATOR` identity carried inside the curated payload, and -(c) the receiving system's own bearer-token / signed-request -controls (Slack signing secret, Teams connector token). - -The webhook wire-format event vocabulary (5 events) is -`training.start`, `training.success`, `training.failure`, -`training.reverted`, and `approval.required` — Phase 8. The -in-process notifier method names use the `notify_*` prefix -(`notify_start`, `notify_success`, `notify_failure`, -`notify_reverted`, `notify_awaiting_approval`) and dispatch to those -five wire events. Every payload carries the `FORGELM_OPERATOR` -identity so the receiving system (Slack, Teams, custom -incident-management) can attribute the notification. +`forgelm/config.py::WebhookConfig`). Destination-side authenticity falls to +(a) HTTPS + URL-secrecy via `webhook.url_env` and (b) authentication +the receiving side applies itself. A Slack incoming webhook has none +beyond its secret URL (Slack's signing secret verifies requests *from* +Slack, not requests to a webhook URL), so a custom relay should require +its own authentication, because ForgeLM sends no signature. A webhook +payload does **not** carry the +`FORGELM_OPERATOR` identity; attribution of who ran or approved a model +comes from the audit log, not from the notification. + +The webhook wire-format event vocabulary has eight events: the five +single-stage lifecycle events `training.start`, `training.success`, +`training.failure`, `training.reverted` and `approval.required`, plus the +three `pipeline.*` events (`pipeline.started`, `pipeline.completed`, +`pipeline.stage_reverted`) that a multi-stage run emits alongside them. The +in-process notifier methods use the `notify_*` prefix (`notify_start`, +`notify_success`, `notify_failure`, `notify_reverted`, +`notify_awaiting_approval`, `notify_pipeline_started`, +`notify_pipeline_completed`, `notify_pipeline_reverted`) and dispatch to +those wire events. ## 8. Verification checklist diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index f9b88b43..0bf47d57 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -15,7 +15,7 @@ Tam açıklamalı örnek için `config_template.yaml` dosyasına bakın. | `load_in_4bit` | bool | `true` | QLoRA 4-bit NF4 kuantizasyon | | `backend` | string | `"transformers"` | `"transformers"` veya `"unsloth"` (desteklenen mimarilerde daha hızlı, Linux) | | `trust_remote_code` | bool | `false` | Model depolarından özel kod çalıştırma. **Güvenlik riski** | -| `offline` | bool | `false` | İzole mod: HF Hub çağrısı yok. Modeller/veri setleri yerel olmalı | +| `offline` | bool | `false` | HF offline değişkenlerini ayarlar; Hub, transformers ve datasets ağ çağrısı yapmaz. Modeller/veri setleri yerel olmalı. Webhook'ları, judge veya sentetik-veri API istemcilerini ve W&B / MLflow'u durdurmaz | | `revision` | string | `null` | Temel modeli + tokenizer'ı bir HF Hub commit SHA'sına (40-hex) veya branch/tag'e sabitle. **Bugün uygulanıyor.** Bkz. [Hub revision pinleme](#hub-revision-pinleme) | | `bnb_4bit_use_double_quant` | bool | `true` | Ekstra VRAM tasarrufu için çift kuantizasyon | | `bnb_4bit_quant_type` | string | `"nf4"` | Kuantizasyon tipi (`"nf4"` veya `"fp4"`) | @@ -397,7 +397,7 @@ kaynağı kaydedilmemiştir. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `strategy` | string | `null` | `"deepspeed"` veya `"fsdp"` (null = tek GPU) | -| `deepspeed_config` | string | `null` | Ön ayar (`"zero2"`, `"zero3"`, `"zero3_offload"`) veya JSON yolu | +| `deepspeed_config` | string | `null` | JSON yolu ya da yalnızca kaynak checkout'unda çözülen (pip veya Docker kurulumunda çözülmeyen) bir ön ayar adı (`"zero2"`, `"zero3"`, `"zero3_offload"`) | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` veya `"shard_grad_op"` (`"hybrid_shard"` ve `"no_shard"` reddedilir: FSDP2 bunları desteklemez) | | `fsdp_auto_wrap` | bool | `true` | Transformer katmanlarını otomatik sar (transformers 5.10+'ta yok sayılır, katmanlar hep sarılır) | | `fsdp_offload` | bool | `false` | Parametreleri CPU'ya taşı | @@ -433,11 +433,11 @@ kaynağı kaydedilmemiştir. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| -| `url` | string | `null` | Webhook hedef URL | -| `url_env` | string | `null` | URL'yi içeren ortam değişkeni adı | +| `url` | string | `null` | Webhook hedef URL'si: bir Slack incoming webhook'u ya da bir relay (Teams ve Discord payload'u doğrudan reddeder) | +| `url_env` | string | `null` | URL'yi içeren ortam değişkeni adı; yalnızca `url` boşken okunur | | `notify_on_start` | bool | `true` | Eğitim başlangıcında bildir | | `notify_on_success` | bool | `true` | Başarıda bildir | -| `notify_on_failure` | bool | `true` | Hata durumunda bildir | +| `notify_on_failure` | bool | `true` | Eğitim hata verdiğinde ya da eğitim sonrası bir kapı modeli geri aldığında bildir | | `timeout` | int | `10` | HTTP istek zaman aşımı (saniye). Notifier ≥ 1s'ye clamp'ler. v0.5.5'te varsayılan 10s'ye çıkarıldı (önceden 5s'di) — Slack/Teams gateway gecikme atışları production'da düzenli olarak 5s'yi aşıyor ve bir webhook zaman aşımı audit chain'i sessizce zayıflatıyor (webhook arızası best-effort). | | `allow_private_destinations` | bool | `false` | RFC1918 / loopback / link-local hedeflere webhook gönderimine izin verir (cluster içi Slack proxy, on-prem Teams gateway gibi). Varsayılan yalnızca genel internet — SSRF koruması | | `require_https` | bool | `false` | TLS-only zorlama. `true`, plaintext bir `http://` URL'ini reddeder (SSRF chokepoint raise eder; POST atlanır), warn-and-send yerine. Varsayılan `false`, warn-then-send davranışını korur | diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 5d830e0a..87e8639a 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -15,7 +15,7 @@ See `config_template.yaml` for a complete annotated example. | `load_in_4bit` | bool | `true` | Enable QLoRA 4-bit NF4 quantization | | `backend` | string | `"transformers"` | `"transformers"` or `"unsloth"` (faster on supported architectures, Linux only) | | `trust_remote_code` | bool | `false` | Allow custom code from model repos. **Security risk** — only enable for models that require it | -| `offline` | bool | `false` | Air-gapped mode: no HF Hub calls. Models/datasets must be local | +| `offline` | bool | `false` | Sets the HF offline variables, so Hub, transformers and datasets make no network calls. Models/datasets must be local. Does not stop webhooks, judge or synthetic-data API clients, or W&B / MLflow | | `revision` | string | `null` | Pin the base model + tokenizer to an HF Hub commit SHA (40-hex) or a branch/tag. **Honoured today.** See [Hub revision pinning](#hub-revision-pinning) | | `bnb_4bit_use_double_quant` | bool | `true` | Double quantization for extra VRAM savings | | `bnb_4bit_quant_type` | string | `"nf4"` | Quantization type (`"nf4"` or `"fp4"`) | @@ -293,11 +293,11 @@ exits 0, and nothing is deleted, blocked or scheduled automatically. The | Field | Type | Default | Description | |-------|------|---------|-------------| -| `url` | string | `null` | Webhook destination URL | -| `url_env` | string | `null` | Env var name containing URL | +| `url` | string | `null` | Webhook destination URL: a Slack incoming webhook, or a relay (Teams and Discord reject the payload directly) | +| `url_env` | string | `null` | Env var name containing the URL; read only when `url` is empty | | `notify_on_start` | bool | `true` | Notify on training start | | `notify_on_success` | bool | `true` | Notify on success | -| `notify_on_failure` | bool | `true` | Notify on failure | +| `notify_on_failure` | bool | `true` | Notify when training raises or a post-training gate reverts the model | | `timeout` | int | `10` | HTTP request timeout (seconds). Clamped to ≥ 1s by the notifier. Default raised to 10s in v0.5.5 (was 5s) — Slack/Teams gateway latency spikes regularly cross 5s in production, and a webhook timeout silently degrades the audit chain (webhook failure is best-effort). | | `allow_private_destinations` | bool | `false` | Opt in to webhooks pointing at RFC1918 / loopback / link-local hosts (in-cluster Slack proxy, on-prem Teams gateway). Defaults to public-internet only — SSRF guard | | `require_https` | bool | `false` | TLS-only enforcement. `true` refuses a plaintext `http://` URL (the SSRF chokepoint raises; the POST is skipped) instead of warn-and-send. Default `false` preserves warn-then-send | @@ -310,7 +310,7 @@ exits 0, and nothing is deleted, blocked or scheduled automatically. The | Field | Type | Default | Description | |-------|------|---------|-------------| | `strategy` | string | `null` | `"deepspeed"` or `"fsdp"` (null = single GPU) | -| `deepspeed_config` | string | `null` | Preset (`"zero2"`, `"zero3"`, `"zero3_offload"`) or JSON path | +| `deepspeed_config` | string | `null` | JSON path, or a preset name (`"zero2"`, `"zero3"`, `"zero3_offload"`) that resolves only from a source checkout, not from a pip or Docker install | | `fsdp_strategy` | string | `"full_shard"` | `"full_shard"` or `"shard_grad_op"` (`"hybrid_shard"` and `"no_shard"` are rejected: FSDP2 does not support them) | | `fsdp_auto_wrap` | bool | `true` | Auto-wrap transformer layers (ignored on transformers 5.10+, which always wraps them) | | `fsdp_offload` | bool | `false` | Offload parameters to CPU | diff --git a/docs/reference/distributed_training-tr.md b/docs/reference/distributed_training-tr.md index 0df37e60..6054462b 100644 --- a/docs/reference/distributed_training-tr.md +++ b/docs/reference/distributed_training-tr.md @@ -49,7 +49,17 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Yapılandırması -ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametresine bu isimlerden birini veya kendi JSON dosyanızın yolunu verin. +Depo, `configs/deepspeed/` altında üç DeepSpeed ön ayarı barındırır. `deepspeed_config` parametresine bunların adlarından birini veya kendi JSON dosyanızın yolunu verin. + +**Bir ön ayar adı yalnızca `configs/deepspeed/.json` kurulu paketin yanında ya da geçerli dizinde mevcutsa çalışır.** +Kaynak checkout'u bu dosyaları sağlar; `pip install forgelm` wheel'i ve Docker imajı sağlamaz, orada bir ön ayar adı +yapılandırma hatasıyla (exit 1) başarısız olur. Deponun `configs/deepspeed/` klasöründeki JSON dosyalarını kopyalayın +(çalışma dizinindeki bir `configs/deepspeed/` dizinine ya da `deepspeed_config`'i yoluna yönelttiğiniz herhangi bir yere) +veya kendinizinkini yazın. + +**Bilinen sorun ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** `deepspeed_config` ayarlanmadığında da +kullanılan `zero2` ön ayarı `allgather_bucket_size` değerini `"auto"` yapar; transformers'ın DeepSpeed entegrasyonu bu +değeri doldurmaz, bu yüzden onunla bir ZeRO-2 koşusu durur. Düzelene kadar kendi ZeRO-2 JSON'unuzu kullanın. ### Ön Ayarlar (Presets) @@ -64,8 +74,8 @@ ForgeLM, üç yerleşik DeepSpeed ön ayarı sunar. `deepspeed_config` parametre ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Ön ayar adı - # deepspeed_config: "./my_ds_config.json" # Veya özel dosya yolu + deepspeed_config: "./my_ds_config.json" # Kendi JSON'unuzun yolu (her kurulumda çalışır) + # deepspeed_config: "zero2" # Ön ayar adı: yalnızca kaynak checkout'u ``` ### ZeRO Aşama Karşılaştırması diff --git a/docs/reference/distributed_training.md b/docs/reference/distributed_training.md index 013018a0..95becdf9 100644 --- a/docs/reference/distributed_training.md +++ b/docs/reference/distributed_training.md @@ -49,7 +49,17 @@ torchrun --nproc_per_node=4 -m forgelm.cli --config my_config.yaml ## DeepSpeed Configuration -ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one of these names, or provide a path to your own JSON file. +The repository ships three DeepSpeed presets under `configs/deepspeed/`. Set `deepspeed_config` to one of their names, or provide a path to your own JSON file. + +**A preset name works only when `configs/deepspeed/.json` is available beside the installed package or in the +current directory.** A source checkout provides those files; a `pip install forgelm` wheel and the Docker image do not, +so there a preset name fails with a configuration error (exit 1). Copy the JSON files from the repository's +`configs/deepspeed/` folder (into a `configs/deepspeed/` directory in the working directory, or anywhere if you point +`deepspeed_config` at the path), or write your own. + +**Known issue ([#410](https://github.com/HodeTech/ForgeLM/issues/410)):** the `zero2` preset, which is also used when +`deepspeed_config` is unset, sets `allgather_bucket_size` to `"auto"`, a value transformers' DeepSpeed integration does +not fill in, so a ZeRO-2 run with it aborts. Use your own ZeRO-2 JSON until that is fixed. ### Presets @@ -64,8 +74,8 @@ ForgeLM provides three built-in DeepSpeed presets. Set `deepspeed_config` to one ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # Preset name - # deepspeed_config: "./my_ds_config.json" # Or custom path + deepspeed_config: "./my_ds_config.json" # Path to your own JSON (works on every install) + # deepspeed_config: "zero2" # Preset name: source checkout only ``` ### ZeRO Stage Comparison diff --git a/docs/usermanuals/en/compliance/human-oversight.md b/docs/usermanuals/en/compliance/human-oversight.md index 373f7db2..61e05a72 100644 --- a/docs/usermanuals/en/compliance/human-oversight.md +++ b/docs/usermanuals/en/compliance/human-oversight.md @@ -48,12 +48,8 @@ There is **no** `compliance.human_approval` field, no `approval.*` block, no `si The trainer halts after eval and prints: ```text -[2026-04-29 14:33:10] Human approval required. - Run ID: abc123 - Bundle: checkpoints/run/artifacts/ - - To approve: forgelm approve abc123 --output-dir checkpoints/run --comment "..." - To reject: forgelm reject abc123 --output-dir checkpoints/run --comment "..." +2026-04-29 14:33:10 [INFO] forgelm.trainer: Human approval required. Model staged at: checkpoints/run/final_model.staging.abc123 +2026-04-29 14:33:10 [INFO] forgelm.trainer: Review results in checkpoints/run/compliance/ and run `forgelm approve abc123 --output-dir checkpoints/run` to promote, or `forgelm reject abc123 --output-dir checkpoints/run` to preserve for forensic review. ``` The reviewer runs the approval command from any machine with access to the audit-log + staging directory. ForgeLM resolves their identity from `FORGELM_OPERATOR`, HMAC-chains the decision event, and (on `approve`) renames the staging directory to the canonical `final_model/` path. diff --git a/docs/usermanuals/en/getting-started/first-run.md b/docs/usermanuals/en/getting-started/first-run.md index 99a2b6ee..5b09a35c 100644 --- a/docs/usermanuals/en/getting-started/first-run.md +++ b/docs/usermanuals/en/getting-started/first-run.md @@ -29,22 +29,23 @@ flowchart LR Before anything else, run `forgelm doctor` to check that Python, PyTorch, CUDA, and any optional dependencies are wired up correctly: ```shell -$ forgelm doctor -forgelm doctor — environment check - - [✓ pass] python.version Python 3.11.4 (CPython). - [✓ pass] torch.cuda torch 2.4.0 with CUDA 12.4. - [✓ pass] gpu.inventory 1 GPU(s) — GPU0: NVIDIA RTX 4090 (24.0 GiB). - [✓ pass] extras.qlora Installed (module bitsandbytes, purpose: 4-bit / 8-bit QLoRA training (Linux GPUs only)). - [✓ pass] extras.unsloth Installed (module unsloth, purpose: Unsloth-accelerated training (Linux GPUs only)). - [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). - [✓ pass] hf_hub.reachable HuggingFace Hub reachable (HTTP 200). - [✓ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. - [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs. - -Summary: 7 pass, 2 warn, 0 fail. +$ forgelm doctor --offline +forgelm doctor - environment check + + [+ pass] python.version Python 3.11.4 (CPython). + [! warn] torch.cuda torch 2.4.0 installed but CUDA is unavailable. CPU-only runs are supported but training will be very slow; expect to use this only for tiny experiments or smoke tests. + [+ pass] numpy.torch_abi torch 2.4.0 + numpy 2.1.0 are ABI-compatible. + [! warn] gpu.inventory No CUDA devices visible; CPU-only mode. + [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). + [+ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. + [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs so the audit log identifies a stable identity. + ... + +Summary: 7 pass, 12 warn, 0 fail. ``` +This is an excerpt: the real output has around 19 rows (it also checks the ForgeLM install, every optional extra, and the HF cache or Hub), and the rows, values and counts depend on your machine. A `pass` on an extra means the module is discoverable, not that it was import-tested; `[+ pass]`, `[! warn]` and `[x fail]` are the status markers. + `--output-format json` returns a structured envelope (`{"success": bool, "checks": [...], "summary": {...}}`) so CI can filter on individual probe results without parsing the table. Pass `--offline` to skip the HF Hub network probe and inspect the local cache instead — useful for air-gapped deployments. :::tip @@ -137,17 +138,10 @@ Possible verdicts: ```shell $ forgelm --config configs/quickstart-customer-support.yaml -[2026-04-28 14:01:32] config validated -[2026-04-28 14:01:33] auditing data/customer-support.jsonl (12,400 rows, 3 splits) -[2026-04-28 14:01:35] PII flags: 0 critical, 5 medium · cross-split overlap: 0 -[2026-04-28 14:01:37] SFT epoch 1/3 · loss=2.31 → 1.42 -[2026-04-28 14:18:55] DPO preference pass · β=0.1 · KL=4.2 -[2026-04-28 14:32:11] benchmark hellaswag=0.62 truthfulqa=0.48 -[2026-04-28 14:33:02] Llama Guard S1-S14: clean -[2026-04-28 14:33:04] Annex IV → checkpoints/customer-support/artifacts/annex_iv_metadata.json -[2026-04-28 14:33:04] ✔ finished, exit 0 ``` +The output is a stream of timestamped `[INFO]` lines from the `forgelm.*` loggers, with the progress bar and the per-step loss lines that transformers and TRL print. The `customer-support` template is an SFT run with no `evaluation:` block, so the stages you get are: config validation, training, then the files below being written. Stages such as the benchmark, the safety evaluation, the judge and the human-approval gate run only when the YAML enables them. The command ends with exit code `0` on success (see [Exit codes](#/reference/exit-codes)). + ## 6. Try the model ```shell @@ -160,20 +154,32 @@ continues until the end of the current billing period… ## What you got on disk ```text -checkpoints/customer-support/ -├── artifacts/ -│ ├── annex_iv_metadata.json ← Article 11 technical documentation -│ ├── audit_log.jsonl ← Article 12 append-only event log -│ ├── data_audit_report.json ← Article 10 data governance evidence -│ ├── safety_report.json ← Llama Guard verdict -│ ├── benchmark_results.json ← Per-task accuracy -│ └── manifest.json ← SHA-256 over every artifact -├── README.md ← Article 13 model card -├── config_snapshot.yaml ← The exact YAML this run used -└── adapter_model.safetensors ← LoRA weights (or merged checkpoint) +checkpoints/customer-support/ ← training.output_dir +├── audit_log.jsonl ← Article 12 append-only event log, hash-chained +├── audit_log.jsonl.manifest.json ← genesis pin: lets verify-audit detect truncation +├── compliance/ +│ ├── compliance_report.json ← Article 11 manifest (the canonical machine-readable bundle) +│ ├── training_manifest.yaml ← operator-readable summary of the run +│ ├── data_provenance.json ← Article 10 dataset fingerprint +│ └── data_governance_report.json ← Article 10 data-governance evidence +├── checkpoint-/ ← intermediate Trainer checkpoints +└── final_model/ + ├── adapter_model.safetensors ← LoRA weights (or merged checkpoint) + ├── README.md ← Article 13 model card + ├── deployer_instructions.md ← notes for whoever deploys the model + └── model_integrity.json ← SHA-256 checksums of the files in final_model/ ``` -That `artifacts/` directory is the deliverable for compliance reviews. Every file inside is hashed in `manifest.json` for tamper-evidence. +Some files appear only when the YAML asks for them: + +| File | Appears when | +|---|---| +| `compliance/annex_iv_metadata.json` | `compliance.provider_name`, `system_name` or `intended_purpose` is set | +| `safety/safety_results.json` | the safety evaluation is enabled | +| `benchmark/benchmark_results.json` | the benchmark gate is enabled | +| `final_model.staging./` instead of `final_model/` | `evaluation.require_human_approval` is on, until `forgelm approve` promotes it | + +There is no `artifacts/` directory and no single manifest over the whole bundle. Two things are tamper-checked: the audit log by its hash chain (`forgelm verify-audit`) and `final_model/` by `model_integrity.json` (`forgelm verify-integrity`). Nothing hashes `compliance/`, so keep those files where an edit would show, for example in version control or write-once storage. ## Next steps diff --git a/docs/usermanuals/en/getting-started/project-layout.md b/docs/usermanuals/en/getting-started/project-layout.md index 05b26f00..6883382c 100644 --- a/docs/usermanuals/en/getting-started/project-layout.md +++ b/docs/usermanuals/en/getting-started/project-layout.md @@ -22,10 +22,9 @@ my-finetune/ │ └── data_audit_report.json ├── checkpoints/ — model outputs (gitignored) │ └── customer-support/ -│ ├── adapter_model.safetensors -│ ├── README.md — model card (Article 13) -│ ├── config_snapshot.yaml — frozen copy of the config used -│ └── artifacts/ — compliance evidence bundle +│ ├── audit_log.jsonl — append-only, hash-chained event log +│ ├── compliance/ — Article 10/11 evidence (JSON + YAML) +│ └── final_model/ — weights, model card (README.md), model_integrity.json ├── ingested/ — raw documents → JSONL via `forgelm ingest` └── .forgelm/ — local cache (also gitignored) ``` @@ -50,11 +49,11 @@ ForgeLM's defaults play well with version control if you commit the right things **Commit:** - `configs/*.yaml` — your runs are configurations; the YAML is the source of truth. - `audit/*.json` — small, machine-readable, useful in PR reviews ("did the audit numbers regress?"). -- `checkpoints/*/README.md` and `checkpoints/*/artifacts/` (model card and audit bundle, both small). +- `checkpoints/*/final_model/README.md` and `checkpoints/*/compliance/` (model card and compliance evidence, both small). **Gitignore:** - `data/` — usually too large; track via DVC or your preferred dataset registry instead. -- `checkpoints/*/adapter_model.safetensors` and other weight files — too large for git, push to HuggingFace Hub or model registry. +- `checkpoints/*/final_model/*.safetensors`, the `checkpoint-*` directories and other weight files — too large for git, push to HuggingFace Hub or model registry. - `ingested/` — re-buildable from raw documents. - `.forgelm/` — local cache. @@ -125,10 +124,10 @@ Run the dev variant during development, the full one for release. CI runs the fu ## Where compliance artifacts go -Every successful (or failed) training run creates a `checkpoints//artifacts/` directory containing the evidence bundle described in [Compliance Overview](#/compliance/overview). This is the directory you hand to a regulator or pin in your CI artifacts. +A training run writes its evidence into `training.output_dir`: `audit_log.jsonl` (with its `.manifest.json` genesis pin) at the top level and a `compliance/` directory next to `final_model/`; [Your First Run](#/getting-started/first-run) shows the full tree and which files depend on the YAML. There is no `artifacts/` directory. Hand `compliance/` and `audit_log.jsonl` to a reviewer, or pin them in your CI artifacts. :::warn -**Don't merge the artifacts/ directory across runs.** Each run is a separate evidence bundle; mixing them breaks the SHA-256 manifest and undermines tamper-evidence. +**Keep each run's output directory separate.** Each run has its own audit chain and run id; copying two runs' files into one directory mixes them. Only the audit log (`forgelm verify-audit`) and `final_model/` (`forgelm verify-integrity`) are tamper-checked; nothing hashes `compliance/`. ::: ## See also diff --git a/docs/usermanuals/en/operations/air-gap.md b/docs/usermanuals/en/operations/air-gap.md index 40a2807c..c22cff50 100644 --- a/docs/usermanuals/en/operations/air-gap.md +++ b/docs/usermanuals/en/operations/air-gap.md @@ -59,7 +59,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | Subsystem | Flag effect or required operator preparation | |---|---| | HuggingFace Hub downloads | `--offline` sets the HF offline flags; pre-cache the required artifacts. | -| W&B / MLflow / Comet | Disable reporting or configure local-only MLflow with a file URI; `--offline` does not disable these integrations. | +| W&B / MLflow | Set `training.report_to` to `none` (or `tensorboard`), or configure local-only MLflow with a file URI; `--offline` does not disable these integrations. | | OpenAI / Anthropic judges | Disable API-backed evaluation or use a local judge; `--offline` does not disable API clients. | | Webhooks | `--offline` does not disable configured delivery. Omit the `webhook:` block, or leave `url` / `url_env` unset, to disable it; `allow_private_destinations` permits private destinations in addition to public ones. | | pip install (during runtime) | Install required dependencies before disconnecting; `--offline` does not control pip. | diff --git a/docs/usermanuals/en/operations/webhooks.md b/docs/usermanuals/en/operations/webhooks.md index d7cf7107..ffccf7bf 100644 --- a/docs/usermanuals/en/operations/webhooks.md +++ b/docs/usermanuals/en/operations/webhooks.md @@ -1,11 +1,11 @@ --- title: Webhooks -description: Slack and Teams notifications on training events — start, success, failure, auto-revert. +description: Slack-format webhook notifications on training events — start, success, failure, auto-revert. Teams and Discord need a relay. --- # Webhooks -ForgeLM fires structured webhooks on training milestones. Wire them into Slack, Teams, or any incident tool that accepts JSON payloads — get the right context to the right humans without anyone watching a log. +ForgeLM fires structured webhooks on training milestones. Wire them into Slack, or into any incident tool or relay that accepts this JSON payload (Microsoft Teams and Discord need a relay; see below) — get the right context to the right humans without anyone watching a log. ## Quick example @@ -17,8 +17,9 @@ webhook: notify_on_failure: true # default true (covers training.failure + training.reverted) ``` -The notifier emits a generic JSON payload — Slack and Teams ingest it directly via -their incoming-webhook endpoints. Per-event subscription is not currently +The notifier emits one generic JSON payload with a Slack-style `attachments` block. +Slack's incoming webhooks accept it; Microsoft Teams and Discord do not (see +"Slack / Teams / Discord ingestion" below). Per-event subscription is not currently configurable; toggle the three `notify_on_*` flags to coarse-control which lifecycle events fire. @@ -50,40 +51,56 @@ surface mirrored in the ## Payload shape -Single generic JSON shape — Slack / Teams / Discord all accept it -directly via incoming-webhook endpoints; ForgeLM does **not** wrap it in -provider-specific templates: +One generic JSON shape. Every payload carries `event`, `run_name`, `status`, `metrics`, `reason`, +`model_path` and an `attachments` array with one Slack-style block (`title`, `text`, `color`). +ForgeLM does **not** wrap it in provider-specific templates, and it has no top-level `text`, +`content` or `embeds` field: ```json { "event": "training.reverted", - "run_name": "customer-support-v1.2.0", + "run_name": "llama3-support-sft", "status": "reverted", - "reason": "safety regression: S5 hate-speech +0.18 over baseline" + "metrics": {}, + "reason": "Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "model_path": null, + "attachments": [ + { + "title": "Training Reverted: llama3-support-sft", + "text": "Auto-revert fired. Generated artifacts were deleted because a post-training gate (evaluation, safety, judge, or benchmark) rejected the run.\n\nReason: Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "color": "#ff9900" + } + ] } ``` -Payload keys vary by event; the full per-event field list is in the -[Audit Event Catalog on GitHub](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/audit_event_catalog.md) -under the *Webhook lifecycle events* table. +The `pipeline.*` events add a few fields. The full per-event field list is in the +[Webhook Payload Schema on GitHub](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/webhook_schema.md). ## Slack / Teams / Discord ingestion -The single generic JSON payload shown above is what Slack, Teams, and Discord -all expect on their incoming-webhook endpoints. There are **no per-provider +The payload above is a Slack-style body. There are **no per-provider templates** in `WebhookConfig` (no `template:`, no `events:` allow-list, no `channel:` / `mention_on_failure:` formatting knobs, no per-destination fan-out array). Routing and formatting happen on the receiving side: -- **Slack** — paste the payload into a Slack workflow or incoming-webhook - app; Slack renders the JSON's top-level fields. To get a richer - formatted card, point the webhook at a relay (Slack workflow / AWS - Lambda / your own gateway) that translates the ForgeLM payload into - Slack Block Kit. -- **Microsoft Teams** — similar pattern. Teams renders incoming JSON - natively but the visual is plain; for MessageCard / Adaptive Card - formatting, run a relay. -- **Discord** — accepts the JSON directly via the bot/webhook URL. +- **Slack** — an incoming webhook accepts the body and shows the + `attachments` block (title, text, colour). That is the legacy attachment + format, which Slack treats as secondary; for a richer card point the + webhook at a relay (Slack workflow / AWS Lambda / your own gateway) that + translates the ForgeLM payload into Slack Block Kit. +- **Microsoft Teams** — not accepted directly. Office 365 Connectors were + retired in May 2026, and a Teams Workflows (Power Automate) webhook expects + an Adaptive Card message, which this payload is not. Run a relay that wraps + the payload in an Adaptive Card. +- **Discord** — not accepted directly. A Discord webhook needs `content` or + `embeds` (or a file or poll) in the body, and ForgeLM sends none of them, so + Discord answers with HTTP 400. Run a relay that maps `attachments[0]` to + `content` or an embed. + +A receiver that rejects a delivery does not fail the run, and nothing is paged: +the rejection is a `WARNING` in the run's log (see *Common pitfalls*). Send a +test event to a new destination before relying on it. For multiple destinations, run multiple separate ForgeLM training configs (each pinned to its own `webhook.url_env`) or fan out from a @@ -97,7 +114,7 @@ Real `WebhookConfig` (see `forgelm/config.py::WebhookConfig`): | Field | Default | Notes | |---|---|---| | `url` | `null` | Inline URL — prefer `url_env` for secret hygiene. | -| `url_env` | `null` | Env-var name carrying the URL. Overrides `url` when set. | +| `url_env` | `null` | Env-var name carrying the URL. Read only when `url` is empty; when both are set, `url` wins. | | `notify_on_start` | `true` | Gates `training.start` AND `pipeline.started`. | | `notify_on_success` | `true` | Gates `training.success`, `approval.required`, AND a successful `pipeline.completed`. | | `notify_on_failure` | `true` | Gates `training.failure`, `training.reverted`, `pipeline.stage_reverted`, AND a failing `pipeline.completed`. | @@ -117,7 +134,7 @@ live outside ForgeLM. - **TLS strongly recommended.** ForgeLM permits both HTTPS and HTTP webhook URLs — HTTP destinations log a `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` warning but are not rejected by default (see `forgelm/webhook.py` `_send`). On a regulated estate set `webhook.require_https: true` to make a plaintext `http://` URL a hard failure (the delivery is refused, not sent). Pin `https://` URLs in production. - **Curated payload.** ForgeLM never includes raw training data, full configs, or unredacted PII in webhook payloads. The notifier wraps a fixed-shape JSON; there is no `webhook.redact` toggle because there's nothing user-controllable to redact. - **Server-Side Request Forgery (SSRF) guard.** ForgeLM blocks webhook URLs pointing at internal IPs — RFC 1918, loopback, link-local (incl. cloud IMDS at `169.254.169.254`), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`, which also covers Alibaba Cloud ECS's IMDS at `100.100.100.200` — the stdlib `ipaddress` predicates don't flag that block as private or reserved on their own), plus reserved and multicast destinations — unless you explicitly opt-in with `webhook.allow_private_destinations: true`. This prevents misconfigured runs from probing your internal network or leaking the payload to a cloud metadata service. -- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus the receiving system's bearer-token / signed-request controls (Slack signing secret, Teams connector token). +- **No HMAC body signing.** ForgeLM does not sign webhook bodies — destination-side authenticity falls to TLS + URL secrecy via `url_env` plus whatever authentication the receiver applies itself. A Slack incoming webhook has none beyond its secret URL (Slack's signing secret verifies requests *from* Slack, not requests to a webhook URL), so put your own authentication in front of a custom relay: ForgeLM sends no signature. ## Common pitfalls diff --git a/docs/usermanuals/en/reference/cli.md b/docs/usermanuals/en/reference/cli.md index e286fbb8..5ee639a8 100644 --- a/docs/usermanuals/en/reference/cli.md +++ b/docs/usermanuals/en/reference/cli.md @@ -44,7 +44,7 @@ Run `forgelm --help` for any of these. | `--dry-run` | Validate configuration and check model/dataset access; no training. | | `--fit-check` | Estimate peak training VRAM; no model load. Requires `--config`. | | `--resume [PATH]` | Resume training. Bare `--resume` auto-detects last checkpoint; `--resume PATH` resumes from a specific one. | -| `--offline` | Air-gapped mode: disable all HF Hub network calls. Models and datasets must be available locally. | +| `--offline` | Offline mode: set the Hugging Face offline variables so Hub, transformers and datasets make no network calls. Models and datasets must be available locally. Does not stop webhooks, judge or synthetic-data API clients, or experiment trackers. | | `--benchmark-only MODEL_PATH` | Run benchmark evaluation on an existing model without training. Requires `evaluation.benchmark` config. | | `--merge` | Run model merging from the `merge:` config block. No training. | | `--stage NAME` | Multi-stage pipelines only: run a single named stage in isolation. Non-first stages need the previous stage's on-disk output, or `--input-model`. | diff --git a/docs/usermanuals/en/reference/configuration.md b/docs/usermanuals/en/reference/configuration.md index ea38e082..ae2a455b 100644 --- a/docs/usermanuals/en/reference/configuration.md +++ b/docs/usermanuals/en/reference/configuration.md @@ -228,7 +228,7 @@ The merge field is `method` (not `algorithm`), and there is no `dare_ties` choic ```yaml distributed: strategy: null # null (single-GPU, no distributed wrapping) | deepspeed | fsdp - deepspeed_config: null # path to a DeepSpeed JSON, or preset name: zero2 | zero3 | zero3_offload + deepspeed_config: null # path to a DeepSpeed JSON (preset names zero2 | zero3 | zero3_offload need a source checkout) fsdp_strategy: "full_shard" # full_shard | shard_grad_op fsdp_auto_wrap: true # ignored on transformers 5.10+ (FSDP2 always wraps transformer layers) fsdp_offload: false # offload FSDP parameters to CPU between forward/backward @@ -262,8 +262,8 @@ compliance: ```yaml webhook: - url: null # Slack / Teams / Discord / custom; prefer url_env - url_env: null # env var carrying the webhook URL + url: null # Slack incoming webhook, or a relay for Teams / Discord; prefer url_env + url_env: null # env var carrying the webhook URL; read only when url is empty notify_on_start: true notify_on_success: true notify_on_failure: true diff --git a/docs/usermanuals/en/training/distributed.md b/docs/usermanuals/en/training/distributed.md index 8679d546..72321962 100644 --- a/docs/usermanuals/en/training/distributed.md +++ b/docs/usermanuals/en/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 shards the optimiser state (the heaviest VRAM component for adaptive opti ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset name (or a filesystem path to a DeepSpeed JSON) + deepspeed_config: "./ds_zero2.json" # a filesystem path to a DeepSpeed JSON (preset names such as "zero2" need a source checkout) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -The `zero3_offload` preset fits 70B on 8×24 GB by offloading optimiser state and parameters to CPU. ZeRO-Infinity's NVMe offload isn't a built-in preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. +The `zero3_offload` preset offloads optimiser state and parameters to CPU, trading speed for GPU memory; whether a given model fits depends on your GPU memory and CPU RAM. The preset names `zero2`, `zero3` and `zero3_offload` resolve only from a source checkout (the JSON files live in the repository's `configs/deepspeed/` folder, not in the pip wheel or the Docker image), and the `zero2` preset is currently broken ([#410](https://github.com/HodeTech/ForgeLM/issues/410)), so give `deepspeed_config` the path of a JSON file you control. ZeRO-Infinity's NVMe offload isn't a preset — point `deepspeed_config` at a custom DeepSpeed JSON with `offload_param`/`offload_optimizer` set to `device: nvme` for that case. | Model | GPUs | ZeRO-3 + offload? | |---|---|---| diff --git a/docs/usermanuals/tr/compliance/human-oversight.md b/docs/usermanuals/tr/compliance/human-oversight.md index 65a1f336..4761258d 100644 --- a/docs/usermanuals/tr/compliance/human-oversight.md +++ b/docs/usermanuals/tr/compliance/human-oversight.md @@ -48,12 +48,8 @@ webhook: Trainer eval'den sonra durur ve yazdırır: ```text -[2026-04-29 14:33:10] Human approval required. - Run ID: abc123 - Bundle: checkpoints/run/artifacts/ - - To approve: forgelm approve abc123 --output-dir checkpoints/run --comment "..." - To reject: forgelm reject abc123 --output-dir checkpoints/run --comment "..." +2026-04-29 14:33:10 [INFO] forgelm.trainer: Human approval required. Model staged at: checkpoints/run/final_model.staging.abc123 +2026-04-29 14:33:10 [INFO] forgelm.trainer: Review results in checkpoints/run/compliance/ and run `forgelm approve abc123 --output-dir checkpoints/run` to promote, or `forgelm reject abc123 --output-dir checkpoints/run` to preserve for forensic review. ``` Reviewer audit-log + staging dizinine erişimi olan herhangi bir makineden onay komutunu çalıştırır. ForgeLM kimliği `FORGELM_OPERATOR`'dan resolve eder, kararı HMAC ile zincirler ve (`approve`'da) staging dizinini kanonik `final_model/` yoluna rename eder. diff --git a/docs/usermanuals/tr/getting-started/first-run.md b/docs/usermanuals/tr/getting-started/first-run.md index 1f65a3eb..46b1cd20 100644 --- a/docs/usermanuals/tr/getting-started/first-run.md +++ b/docs/usermanuals/tr/getting-started/first-run.md @@ -29,22 +29,23 @@ flowchart LR Her şeyden önce Python, PyTorch, CUDA ve opsiyonel bağımlılıkların doğru bağlandığını kontrol etmek için `forgelm doctor` çalıştırın: ```shell -$ forgelm doctor -forgelm doctor — environment check - - [✓ pass] python.version Python 3.11.4 (CPython). - [✓ pass] torch.cuda torch 2.4.0 with CUDA 12.4. - [✓ pass] gpu.inventory 1 GPU(s) — GPU0: NVIDIA RTX 4090 (24.0 GiB). - [✓ pass] extras.qlora Installed (module bitsandbytes, purpose: 4-bit / 8-bit QLoRA training (Linux GPUs only)). - [✓ pass] extras.unsloth Installed (module unsloth, purpose: Unsloth-accelerated training (Linux GPUs only)). - [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). - [✓ pass] hf_hub.reachable HuggingFace Hub reachable (HTTP 200). - [✓ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. - [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs. - -Summary: 7 pass, 2 warn, 0 fail. +$ forgelm doctor --offline +forgelm doctor - environment check + + [+ pass] python.version Python 3.11.4 (CPython). + [! warn] torch.cuda torch 2.4.0 installed but CUDA is unavailable. CPU-only runs are supported but training will be very slow; expect to use this only for tiny experiments or smoke tests. + [+ pass] numpy.torch_abi torch 2.4.0 + numpy 2.1.0 are ABI-compatible. + [! warn] gpu.inventory No CUDA devices visible; CPU-only mode. + [! warn] extras.distributed Optional extra missing — install with: pip install 'forgelm[distributed]' (purpose: DeepSpeed ZeRO + offload distributed training (Linux + macOS only)). + [+ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. + [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. Pin FORGELM_OPERATOR= for CI / pipeline runs so the audit log identifies a stable identity. + ... + +Summary: 7 pass, 12 warn, 0 fail. ``` +Bu bir alıntıdır: gerçek çıktı yaklaşık 19 satırdır (ForgeLM kurulumunu, her opsiyonel extra'yı ve HF cache'i ya da Hub'ı da denetler); satırlar, değerler ve sayılar makinenize göre değişir. Bir extra için `pass`, modülün bulunabildiği anlamına gelir, import testinden geçtiği anlamına gelmez; durum işaretleri `[+ pass]`, `[! warn]` ve `[x fail]`'dir. + `--output-format json` yapısal bir zarf döner (`{"success": bool, "checks": [...], "summary": {...}}`); CI ayrı ayrı probe sonuçlarını tabloyu parse etmeden filtreleyebilir. `--offline` HF Hub ağ probe'unu atlar ve onun yerine yerel cache'i inceler — air-gap dağıtımları için faydalı. :::tip @@ -137,17 +138,10 @@ Olası verdict'ler: ```shell $ forgelm --config configs/quickstart-customer-support.yaml -[2026-04-29 14:01:32] config validated -[2026-04-29 14:01:33] auditing data/customer-support.jsonl (12,400 rows, 3 splits) -[2026-04-29 14:01:35] PII flags: 0 critical, 5 medium · cross-split overlap: 0 -[2026-04-29 14:01:37] SFT epoch 1/3 · loss=2.31 → 1.42 -[2026-04-29 14:18:55] DPO preference pass · β=0.1 · KL=4.2 -[2026-04-29 14:32:11] benchmark hellaswag=0.62 truthfulqa=0.48 -[2026-04-29 14:33:02] Llama Guard S1-S14: clean -[2026-04-29 14:33:04] Annex IV → checkpoints/customer-support/artifacts/annex_iv_metadata.json -[2026-04-29 14:33:04] ✔ finished, exit 0 ``` +Çıktı, `forgelm.*` logger'larından gelen zaman damgalı `[INFO]` satırlarının ve transformers ile TRL'in bastığı ilerleme çubuğu ile adım başı loss satırlarının akışıdır. `customer-support` şablonu `evaluation:` bloğu olmayan bir SFT koşusudur; bu yüzden göreceğiniz aşamalar şunlardır: config doğrulama, eğitim, ardından aşağıdaki dosyaların yazılması. Benchmark, güvenlik değerlendirmesi, judge ve insan-onay kapısı gibi aşamalar yalnızca YAML bunları açtığında çalışır. Komut başarıda `0` çıkış koduyla biter ([Çıkış kodları](#/reference/exit-codes)). + ## 6. Modeli dene ```shell @@ -160,20 +154,32 @@ Erişiminiz mevcut faturalama döneminin sonuna kadar devam eder… ## Diskte ne var ```text -checkpoints/customer-support/ -├── artifacts/ -│ ├── annex_iv_metadata.json ← Article 11 teknik dokümantasyon -│ ├── audit_log.jsonl ← Article 12 append-only event log -│ ├── data_audit_report.json ← Article 10 veri yönetimi kanıtı -│ ├── safety_report.json ← Llama Guard verdict -│ ├── benchmark_results.json ← Görev başı doğruluk -│ └── manifest.json ← Her artifact üzerinde SHA-256 -├── README.md ← Article 13 model card -├── config_snapshot.yaml ← Bu koşunun kullandığı tam YAML -└── adapter_model.safetensors ← LoRA ağırlıkları (veya birleştirilmiş checkpoint) +checkpoints/customer-support/ ← training.output_dir +├── audit_log.jsonl ← Article 12 append-only event log, hash zincirli +├── audit_log.jsonl.manifest.json ← genesis pin: verify-audit'in kesilmeyi tespit etmesini sağlar +├── compliance/ +│ ├── compliance_report.json ← Article 11 manifesti (kanonik makine-okunur paket) +│ ├── training_manifest.yaml ← koşunun operatör-okunur özeti +│ ├── data_provenance.json ← Article 10 veri seti parmak izi +│ └── data_governance_report.json ← Article 10 veri yönetimi kanıtı +├── checkpoint-/ ← ara Trainer checkpoint'leri +└── final_model/ + ├── adapter_model.safetensors ← LoRA ağırlıkları (veya birleştirilmiş checkpoint) + ├── README.md ← Article 13 model card + ├── deployer_instructions.md ← modeli dağıtacak kişi için notlar + └── model_integrity.json ← final_model/ içindeki dosyaların SHA-256 özetleri ``` -O `artifacts/` dizini compliance incelemeleri için teslim edilebilirdir. İçindeki her dosya `manifest.json`'da tamper-evidence için hashlenmiştir. +Bazı dosyalar yalnızca YAML istediğinde oluşur: + +| Dosya | Ne zaman oluşur | +|---|---| +| `compliance/annex_iv_metadata.json` | `compliance.provider_name`, `system_name` veya `intended_purpose` ayarlıysa | +| `safety/safety_results.json` | güvenlik değerlendirmesi açıksa | +| `benchmark/benchmark_results.json` | benchmark kapısı açıksa | +| `final_model/` yerine `final_model.staging./` | `evaluation.require_human_approval` açıksa, `forgelm approve` onu terfi ettirene kadar | + +`artifacts/` dizini ve tüm paketi kapsayan tek bir manifest yoktur. İki şey tamper-check'lidir: audit log kendi hash zinciriyle (`forgelm verify-audit`) ve `final_model/` `model_integrity.json` ile (`forgelm verify-integrity`). `compliance/` dosyalarını hiçbir şey hash'lemez; bu yüzden onları bir değişikliğin görüneceği bir yerde tutun, örneğin sürüm kontrolünde ya da bir kez yazılabilir depolamada. ## Sonraki adımlar diff --git a/docs/usermanuals/tr/getting-started/project-layout.md b/docs/usermanuals/tr/getting-started/project-layout.md index e24bf6ef..568043f9 100644 --- a/docs/usermanuals/tr/getting-started/project-layout.md +++ b/docs/usermanuals/tr/getting-started/project-layout.md @@ -22,10 +22,9 @@ my-finetune/ │ └── data_audit_report.json ├── checkpoints/ — model çıktıları (gitignored) │ └── customer-support/ -│ ├── adapter_model.safetensors -│ ├── README.md — model card (Article 13) -│ ├── config_snapshot.yaml — kullanılan config'in dondurulmuş kopyası -│ └── artifacts/ — compliance kanıt paketi +│ ├── audit_log.jsonl — append-only, hash zincirli olay günlüğü +│ ├── compliance/ — Article 10/11 kanıtı (JSON + YAML) +│ └── final_model/ — ağırlıklar, model card (README.md), model_integrity.json ├── ingested/ — `forgelm ingest` üzerinden ham döküman → JSONL └── .forgelm/ — yerel cache (gitignored) ``` @@ -50,11 +49,11 @@ ForgeLM varsayılanları doğru şeyleri commit'leyip gerisini ignore'larsanız **Commit:** - `configs/*.yaml` — koşularınız konfigürasyon; YAML kaynak doğrudur. - `audit/*.json` — küçük, makine-okunur, PR review'larında faydalı ("audit numaraları geriledi mi?"). -- `checkpoints/*/README.md` ve `checkpoints/*/artifacts/` (model card ve audit paketi, ikisi de küçük). +- `checkpoints/*/final_model/README.md` ve `checkpoints/*/compliance/` (model card ve compliance kanıtı, ikisi de küçük). **Gitignore:** - `data/` — genelde fazla büyük; DVC veya tercih ettiğiniz dataset registry'sine alın. -- `checkpoints/*/adapter_model.safetensors` ve diğer ağırlık dosyaları — git için fazla büyük; HuggingFace Hub veya model registry'ye gönderin. +- `checkpoints/*/final_model/*.safetensors`, `checkpoint-*` dizinleri ve diğer ağırlık dosyaları — git için fazla büyük; HuggingFace Hub veya model registry'ye gönderin. - `ingested/` — ham dokümanlardan yeniden inşa edilebilir. - `.forgelm/` — yerel cache. @@ -123,10 +122,10 @@ Geliştirme sırasında dev varyantı, release için tam olanını koşturun. CI ## Compliance artifact'ları nereye gider -Her başarılı (veya başarısız) eğitim koşusu, [Compliance Genel Bakış](#/compliance/overview) bölümünde anlatılan kanıt paketini içeren bir `checkpoints//artifacts/` dizini oluşturur. Regülatöre veya CI artifact'larında pinleyeceğiniz dizin budur. +Bir eğitim koşusu kanıtını `training.output_dir` içine yazar: en üstte `audit_log.jsonl` (`.manifest.json` genesis pin'iyle) ve `final_model/` yanında bir `compliance/` dizini; tam ağacı ve hangi dosyaların YAML'a bağlı olduğunu [İlk Koşunuz](#/getting-started/first-run) gösterir. `artifacts/` dizini yoktur. `compliance/` ve `audit_log.jsonl`'i bir incelemeciye teslim edin ya da CI artifact'larında pinleyin. :::warn -**`artifacts/` dizinini koşular arasında birleştirmeyin.** Her koşu ayrı bir kanıt paketidir; karıştırmak SHA-256 manifesti bozar ve tamper-evidence'ı baltalar. +**Her koşunun çıktı dizinini ayrı tutun.** Her koşunun kendi audit zinciri ve run id'si vardır; iki koşunun dosyalarını tek dizine kopyalamak onları karıştırır. Yalnızca audit log (`forgelm verify-audit`) ve `final_model/` (`forgelm verify-integrity`) tamper-check'lidir; `compliance/`'ı hiçbir şey hash'lemez. ::: ## Bkz. diff --git a/docs/usermanuals/tr/operations/air-gap.md b/docs/usermanuals/tr/operations/air-gap.md index c5744481..39ede22f 100644 --- a/docs/usermanuals/tr/operations/air-gap.md +++ b/docs/usermanuals/tr/operations/air-gap.md @@ -59,7 +59,7 @@ ERROR: model "meta-llama/Llama-Guard-3-8B" not in local cache. | Alt sistem | Bayrağın etkisi veya operatörün yapması gereken hazırlık | |---|---| | HuggingFace Hub indirmeleri | `--offline`, HF çevrimdışı ayarlarını yapar; gerekli artefaktları önceden önbelleğe alın. | -| W&B / MLflow / Comet | Raporlamayı kapatın veya MLflow'u file URI ile yalnızca yerel çalışacak şekilde yapılandırın; `--offline` bu entegrasyonları kapatmaz. | +| W&B / MLflow | `training.report_to`'yu `none` (veya `tensorboard`) yapın ya da MLflow'u file URI ile yalnızca yerel çalışacak şekilde yapılandırın; `--offline` bu entegrasyonları kapatmaz. | | OpenAI / Anthropic judge'lar | API tabanlı değerlendirmeyi kapatın veya yerel judge kullanın; `--offline` API istemcilerini kapatmaz. | | Webhook'lar | `--offline`, yapılandırılmış teslimatları kapatmaz. Kapatmak için `webhook:` bloğunu çıkarın veya `url` / `url_env` değerlerini boş bırakın; `allow_private_destinations`, genel hedeflerin yanında özel hedeflere de izin verir. | | Runtime'da pip install | Gerekli bağımlılıkları bağlantıyı kesmeden kurun; `--offline`, pip'i yönetmez. | diff --git a/docs/usermanuals/tr/operations/webhooks.md b/docs/usermanuals/tr/operations/webhooks.md index 6f80a377..722e15d5 100644 --- a/docs/usermanuals/tr/operations/webhooks.md +++ b/docs/usermanuals/tr/operations/webhooks.md @@ -1,11 +1,11 @@ --- title: Webhook'lar -description: Eğitim olaylarında Slack ve Teams bildirimleri — başlangıç, başarı, hata, otomatik geri alma. +description: Eğitim olaylarında Slack biçiminde webhook bildirimleri — başlangıç, başarı, hata, otomatik geri alma. Teams ve Discord için relay gerekir. --- # Webhook'lar -ForgeLM eğitim dönüm noktalarında yapılandırılmış webhook fırlatır. Bunları Slack, Teams veya JSON payload kabul eden herhangi bir incident aracına bağlayın — hiç kimsenin log izlemesine gerek kalmadan doğru bağlamı doğru insanlara getirin. +ForgeLM eğitim dönüm noktalarında yapılandırılmış webhook fırlatır. Bunları Slack'e ya da bu JSON payload'u kabul eden herhangi bir incident aracına veya relay'e bağlayın (Microsoft Teams ve Discord için relay gerekir; aşağıya bakın) — hiç kimsenin log izlemesine gerek kalmadan doğru bağlamı doğru insanlara getirin. ## Hızlı örnek @@ -17,8 +17,9 @@ webhook: notify_on_failure: true # default true (training.failure + training.reverted'i kapsar) ``` -Notifier generic JSON payload yayar — Slack ve Teams bunu incoming-webhook -uçlarından doğrudan kabul eder. Per-event abonelik şu an konfigüre +Notifier, Slack tarzı bir `attachments` bloğu taşıyan tek bir generic JSON payload yayar. +Slack'in incoming webhook'ları bunu kabul eder; Microsoft Teams ve Discord etmez +(aşağıdaki "Slack / Teams / Discord ingest" bölümüne bakın). Per-event abonelik şu an konfigüre edilemiyor; üç `notify_on_*` flag'iyle hangi yaşam-döngüsü olaylarının fırlatılacağı kaba ayarlanır. @@ -50,41 +51,58 @@ ile aynalanan kanonik yüzeydir: ## Payload yapısı -Tek generic JSON şekil — Slack / Teams / Discord hepsi bunu incoming-webhook -uçlarından doğrudan kabul eder; ForgeLM provider'a-özel template'lerle -sarmalama **yapmaz**: +Tek generic JSON şekil. Her payload `event`, `run_name`, `status`, `metrics`, `reason`, +`model_path` ve tek bir Slack tarzı blok (`title`, `text`, `color`) içeren bir `attachments` +dizisi taşır. ForgeLM provider'a-özel template'lerle sarmalama **yapmaz** ve üst düzeyde +`text`, `content` veya `embeds` alanı yoktur: ```json { "event": "training.reverted", - "run_name": "customer-support-v1.2.0", + "run_name": "llama3-support-sft", "status": "reverted", - "reason": "safety regression: S5 hate-speech +0.18 over baseline" + "metrics": {}, + "reason": "Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "model_path": null, + "attachments": [ + { + "title": "Training Reverted: llama3-support-sft", + "text": "Auto-revert fired. Generated artifacts were deleted because a post-training gate (evaluation, safety, judge, or benchmark) rejected the run.\n\nReason: Safety gate failed: unsafe_rate 0.08 exceeds max_unsafe_rate 0.05", + "color": "#ff9900" + } + ] } ``` -Payload anahtarları event'e göre değişir; tam per-event alan listesi -[GitHub'daki Audit Event Kataloğu](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/audit_event_catalog.md) -*Webhook lifecycle events* tablosundadır. +`pipeline.*` event'leri birkaç alan ekler. Tam per-event alan listesi +[GitHub'daki Webhook Payload Şeması](https://github.com/HodeTech/ForgeLM/blob/main/docs/reference/webhook_schema.md) +sayfasındadır. ## Slack / Teams / Discord ingest -Yukarıdaki tek generic JSON payload, Slack, Teams ve Discord'un -incoming-webhook endpoint'lerinde beklediği şeydir. `WebhookConfig`'te +Yukarıdaki payload Slack tarzı bir gövdedir. `WebhookConfig`'te **provider başına template yoktur** (no `template:`, no `events:` allow-list, no `channel:` / `mention_on_failure:` formatlama knob'ları, no per-destination fan-out array). Routing ve formatlama alıcı tarafta yapılır: -- **Slack** — payload'u Slack workflow veya incoming-webhook - uygulamasına yapıştırın; Slack JSON'un top-level alanlarını render - eder. Daha zengin formatlanmış bir kart için, webhook'u ForgeLM - payload'unu Slack Block Kit'e çeviren bir relay'e (Slack +- **Slack** — incoming webhook gövdeyi kabul eder ve `attachments` + bloğunu (başlık, metin, renk) gösterir. Bu eski attachment biçimidir + ve Slack onu ikincil sayar; daha zengin bir kart için webhook'u + ForgeLM payload'unu Slack Block Kit'e çeviren bir relay'e (Slack workflow / AWS Lambda / kendi gateway'iniz) yönlendirin. -- **Microsoft Teams** — benzer pattern. Teams gelen JSON'u natively - render eder ama görsel düz; MessageCard / Adaptive Card - formatlama için bir relay çalıştırın. -- **Discord** — JSON'u doğrudan bot/webhook URL üzerinden kabul eder. +- **Microsoft Teams** — doğrudan kabul edilmez. Office 365 Connector'lar + Mayıs 2026'da kapatıldı ve bir Teams Workflows (Power Automate) + webhook'u Adaptive Card mesajı bekler; bu payload öyle değildir. + Payload'u Adaptive Card'a saran bir relay çalıştırın. +- **Discord** — doğrudan kabul edilmez. Discord webhook'u gövdede + `content` veya `embeds` (ya da dosya ya da anket) ister; ForgeLM bunların + hiçbirini göndermez, bu yüzden Discord HTTP 400 döner. `attachments[0]`'ı + `content`'e veya bir embed'e eşleyen bir relay çalıştırın. + +Teslimatı reddeden bir alıcı run'ı başarısız etmez ve kimseye alarm gitmez: +ret, run log'unda bir `WARNING`'dir (bkz. *Sık hatalar*). Yeni bir hedefe güvenmeden +önce ona bir test event'i gönderin. Birden çok hedef için, her biri kendi `webhook.url_env`'ine pinlenmiş birden çok ayrı ForgeLM training config'i çalıştırın @@ -99,7 +117,7 @@ Gerçek `WebhookConfig` (bkz. `forgelm/config.py::WebhookConfig`): | Alan | Vars. | Notlar | |---|---|---| | `url` | `null` | Inline URL — secret hijyeni için `url_env`'i tercih edin. | -| `url_env` | `null` | URL'i taşıyan env var adı. Set edildiğinde `url`'i override eder. | +| `url_env` | `null` | URL'i taşıyan env var adı. Yalnızca `url` boşken okunur; ikisi de set ise `url` kazanır. | | `notify_on_start` | `true` | `training.start` VE `pipeline.started`'ı gate'ler. | | `notify_on_success` | `true` | `training.success`, `approval.required` VE başarılı bir `pipeline.completed`'ı gate'ler. | | `notify_on_failure` | `true` | `training.failure`, `training.reverted`, `pipeline.stage_reverted` VE başarısız bir `pipeline.completed`'ı gate'ler. | @@ -118,7 +136,7 @@ payload zaten curated), ve routing hepsi ForgeLM'in dışında yaşar. - **TLS şiddetle önerilir.** ForgeLM hem HTTPS hem HTTP webhook URL'lerine izin verir — HTTP hedefleri `Webhook URL uses HTTP (not HTTPS). Data will be sent unencrypted.` uyarısı loglar ama varsayılan olarak reddedilmez (bkz. `forgelm/webhook.py` `_send`). Regüle bir ortamda `webhook.require_https: true` ile plaintext bir `http://` URL'ini hard failure yapın (teslimat reddedilir, gönderilmez). Üretimde `https://` URL'leri pinleyin. - **Curated payload.** ForgeLM webhook payload'larına asla raw eğitim verisi, tam config'ler veya unredacted PII koymaz. Notifier sabit-şekilli bir JSON sarar; `webhook.redact` toggle'ı yoktur çünkü kullanıcı-kontrollü redakte edilecek bir şey yok. - **SSRF guard.** ForgeLM iç IP'lere işaret eden webhook URL'lerini engeller — RFC 1918, loopback, link-local (bulut IMDS `169.254.169.254` dahil), RFC 6598 Shared Address Space / Carrier-Grade-NAT (`100.64.0.0/10`; bu blok, Alibaba Cloud ECS'in IMDS'ini de kapsar — `100.100.100.200` — stdlib `ipaddress` predicate'leri bu bloğu kendiliğinden private veya reserved olarak işaretlemez), ayrıca reserved ve multicast hedefler — `webhook.allow_private_destinations: true` ile açıkça opt-in olmadıkça. Yanlış konfigüre koşuların iç ağınızı sondalamasını veya payload'ı bir bulut metadata servisine sızdırmasını önler. -- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcı sistemin bearer-token / signed-request kontrollerine (Slack signing secret, Teams connector token) düşer. +- **HMAC body imzalama yok.** ForgeLM webhook gövdelerini imzalamaz — hedef-tarafı authenticity TLS + `url_env` üzerinden URL gizliliği artı alıcının kendi uyguladığı kimlik doğrulamaya düşer. Bir Slack incoming webhook'unun gizli URL'sinden başka doğrulaması yoktur (Slack'in signing secret'ı Slack'ten *gelen* istekleri doğrular, bir webhook URL'sine gelenleri değil); bu yüzden özel bir relay'in önüne kendi kimlik doğrulamanızı koyun: ForgeLM imza göndermez. ## Sık hatalar diff --git a/docs/usermanuals/tr/reference/cli.md b/docs/usermanuals/tr/reference/cli.md index eae80b15..c552e54b 100644 --- a/docs/usermanuals/tr/reference/cli.md +++ b/docs/usermanuals/tr/reference/cli.md @@ -44,7 +44,7 @@ Bunlardan herhangi biri için `forgelm --help`. | `--dry-run` | Config'i ve model/dataset erişimini doğrula; eğitim yok. | | `--fit-check` | Eğitim VRAM tahmini; model yüklenmez. `--config` gerektirir. | | `--resume [PATH]` | Eğitime kaldığı yerden devam. Çıplak `--resume` son checkpoint'i otomatik bulur; `--resume PATH` belirli bir yerden. | -| `--offline` | Air-gap modu: tüm HF Hub ağ çağrılarını kapat. Modeller ve dataset'ler yerel olarak mevcut olmalı. | +| `--offline` | Offline mod: Hugging Face offline değişkenlerini ayarlar; Hub, transformers ve datasets ağ çağrısı yapmaz. Modeller ve dataset'ler yerel olarak mevcut olmalı. Webhook'ları, judge veya sentetik-veri API istemcilerini ve deney izleyicilerini durdurmaz. | | `--benchmark-only MODEL_PATH` | Mevcut bir model üzerinde benchmark koştur, eğitim yok. `evaluation.benchmark` config'i gerektirir. | | `--merge` | Config'in `merge:` bloğundan model birleştirmeyi koştur. Eğitim yok. | | `--stage NAME` | Yalnızca çok aşamalı pipeline'lar: adı verilen tek bir stage'i izole koştur. İlk olmayan stage'ler önceki stage'in on-disk çıktısına ya da `--input-model` override'ına ihtiyaç duyar. | diff --git a/docs/usermanuals/tr/reference/configuration.md b/docs/usermanuals/tr/reference/configuration.md index 690f806b..753f164c 100644 --- a/docs/usermanuals/tr/reference/configuration.md +++ b/docs/usermanuals/tr/reference/configuration.md @@ -230,7 +230,7 @@ Merge alanı `algorithm` değil `method`'dur, `dare_ties` seçeneği, `base_mode ```yaml distributed: strategy: null # null (tek GPU, dağıtık sarmalama yok) | deepspeed | fsdp - deepspeed_config: null # bir DeepSpeed JSON'a yol, veya preset adı: zero2 | zero3 | zero3_offload + deepspeed_config: null # bir DeepSpeed JSON'a yol (preset adları zero2 | zero3 | zero3_offload kaynak checkout'u ister) fsdp_strategy: "full_shard" # full_shard | shard_grad_op fsdp_auto_wrap: true # transformers 5.10+'ta yok sayılır (FSDP2 transformer katmanlarını hep sarar) fsdp_offload: false # forward/backward arasında FSDP parametrelerini CPU'ya offload et @@ -264,8 +264,8 @@ compliance: ```yaml webhook: - url: null # Slack / Teams / Discord / özel; url_env tercihli - url_env: null # webhook URL'sini taşıyan env değişkeni + url: null # Slack incoming webhook'u ya da Teams / Discord için relay; url_env tercihli + url_env: null # webhook URL'sini taşıyan env değişkeni; yalnızca url boşken okunur notify_on_start: true notify_on_success: true notify_on_failure: true diff --git a/docs/usermanuals/tr/training/distributed.md b/docs/usermanuals/tr/training/distributed.md index 7e73a6d7..7c1dc991 100644 --- a/docs/usermanuals/tr/training/distributed.md +++ b/docs/usermanuals/tr/training/distributed.md @@ -66,7 +66,7 @@ ZeRO-2 optimizer state'i sharder (Adam gibi adaptif optimizer'larda en ağır VR ```yaml distributed: strategy: "deepspeed" - deepspeed_config: "zero2" # preset adı (veya bir DeepSpeed JSON dosya yolu) + deepspeed_config: "./ds_zero2.json" # bir DeepSpeed JSON dosya yolu ("zero2" gibi preset adları kaynak checkout'u ister) training: gradient_accumulation_steps: 4 @@ -95,7 +95,7 @@ training: gradient_accumulation_steps: 8 ``` -`zero3_offload` preset'i, optimizer state ve parametreleri CPU'ya boşaltarak 70B'i 8×24 GB'a sığdırır. ZeRO-Infinity'nin NVMe offload'u hazır bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. +`zero3_offload` preset'i optimizer state ve parametreleri CPU'ya boşaltır, hızdan GPU belleği kazandırır; belirli bir modelin sığıp sığmayacağı GPU belleğinize ve CPU RAM'inize bağlıdır. `zero2`, `zero3` ve `zero3_offload` preset adları yalnızca kaynak checkout'unda çözülür (JSON dosyaları pip wheel'inde ya da Docker imajında değil, deponun `configs/deepspeed/` klasöründedir) ve `zero2` preset'i şu an bozuktur ([#410](https://github.com/HodeTech/ForgeLM/issues/410)); bu yüzden `deepspeed_config`'e kontrolünüzdeki bir JSON dosyasının yolunu verin. ZeRO-Infinity'nin NVMe offload'u bir preset değildir — bunun için `deepspeed_config`'i, `offload_param`/`offload_optimizer` alanları `device: nvme` olarak ayarlanmış özel bir DeepSpeed JSON dosyasına yönlendirin. | Model | GPU | ZeRO-3 + offload? | |---|---|---| diff --git a/forgelm/cli/_parser.py b/forgelm/cli/_parser.py index 562c6343..b76d985b 100644 --- a/forgelm/cli/_parser.py +++ b/forgelm/cli/_parser.py @@ -1368,7 +1368,7 @@ def parse_args() -> argparse.Namespace: parser.add_argument( "--offline", action="store_true", - help="Air-gapped mode: disable all HF Hub network calls. Models and datasets must be available locally.", + help="Offline mode: set the Hugging Face offline variables so Hub, transformers and datasets make no network calls. Models and datasets must be available locally. Does not stop webhooks, judge or synthetic-data API clients, or experiment trackers.", ) mode_group.add_argument( "--benchmark-only", diff --git a/forgelm/config.py b/forgelm/config.py index 33e90e25..27f644ca 100644 --- a/forgelm/config.py +++ b/forgelm/config.py @@ -269,7 +269,7 @@ class ModelConfig(BaseModel): ) offline: bool = Field( default=False, - description="Air-gapped mode: refuse HF Hub network calls. Models/datasets/extras must be available locally.", + description="Offline mode: set the HF offline variables (Hub, transformers, datasets). Models/datasets/extras must be available locally. Webhooks, judge/synthetic API clients and trackers are not stopped.", ) revision: Optional[str] = Field( default=None, @@ -768,7 +768,7 @@ class DistributedConfig(BaseModel): ) deepspeed_config: Optional[str] = Field( default=None, - description="DeepSpeed config: filesystem path to a DS JSON OR preset name (`zero2`, `zero3`, `zero3_offload`).", + description="DeepSpeed config: filesystem path to a DS JSON. The preset names (`zero2`, `zero3`, `zero3_offload`) resolve only from a source checkout, not a pip or Docker install.", ) fsdp_strategy: Literal["full_shard", "shard_grad_op"] = Field( default="full_shard", @@ -1523,17 +1523,17 @@ class WebhookConfig(BaseModel): url: Optional[str] = Field( default=None, - description="Webhook URL (Slack / Teams / Discord / custom). Use `url_env` to read from env to avoid committing secrets.", + description="Webhook URL (a Slack incoming webhook, or a relay that forwards to Teams / Discord / custom). Use `url_env` to avoid committing secrets.", ) url_env: Optional[str] = Field( - default=None, description="Env var name carrying the webhook URL (overrides `url` when set)." + default=None, description="Env var name carrying the webhook URL; read only when `url` is empty." ) notify_on_start: bool = Field(default=True, description="POST a `notify_start` event when training begins.") notify_on_success: bool = Field( default=True, description="POST a `notify_success` event when training completes successfully." ) notify_on_failure: bool = Field( - default=True, description="POST a `notify_failure` event when training fails (any non-zero exit)." + default=True, description="POST a failure event when training raises or a post-training gate reverts the model." ) timeout: int = Field( default=10, @@ -1677,7 +1677,7 @@ class ForgeConfig(BaseModel): description="Post-training evaluation block (loss / benchmark / safety / judge / human-approval gate).", ) webhook: Optional[WebhookConfig] = Field( - default=None, description="Webhook notification block (Slack / Teams / Discord / custom)." + default=None, description="Webhook notification block (Slack-format JSON; Teams and Discord need a relay)." ) distributed: Optional[DistributedConfig] = Field( default=None, description="DeepSpeed / FSDP multi-GPU configuration block." From cb5db502cf52fb16cb851dee861c9b4dbf0c851a Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 12:41:11 +0300 Subject: [PATCH 11/22] fix(ci): make dependabot.yml parse by dropping versioning-strategy widen for pip (#1001) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary `.github/dependabot.yml` does not parse, so **Dependabot has been opening no pull requests at all**. It set `versioning-strategy: "widen"` for `pip`, which that ecosystem does not accept. GitHub's `.github/dependabot.yml` check on any pull request that touches the file says: ```text The property '#/updates/0/versioning-strategy' value "widen" did not match one of the following values: lockfile-only, auto, increase, increase-if-necessary ``` (`widen` is valid for npm and Composer.) I introduced it in the contributor-intake change (#964) and did not see the check, because that run did not surface it until a later pull request touched the file. The first weekly run is Monday 2026-10-19, so this should merge before then. ## Changes - `.github/dependabot.yml`: the key is removed (the default is `auto`) and the header comment says why. The comment also records the open question: ForgeLM declares bounded ranges with no lockfile, so `auto` may raise a floor the ceiling already covers instead of admitting a new major. Look at the first pull requests and tighten then. - `tests/test_packaging.py::test_dependabot_config_uses_only_values_the_pip_ecosystem_accepts`: parses the file and fails on a `pip` strategy outside the accepted four. **It fails against the old file.** - `CHANGELOG.md`: one entry. ## Type - [x] Bug fix (CI configuration) ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. - GitHub's own `.github/dependabot.yml` check on this pull request should turn green; that is the real confirmation, and it is the only check that can tell. ## Checklist - [x] CHANGELOG entry under `[Unreleased]` - [x] Test added 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Restore valid Dependabot configuration for pip dependency updates and guard against unsupported versioning strategies. Bug Fixes: - Fix the Dependabot configuration so the pip update block is accepted by GitHub and Dependabot can resume opening dependency updates. Enhancements: - Document the pip versioning-strategy constraint and the need to review Dependabot’s initial pull requests. Tests: - Add packaging coverage that validates pip Dependabot strategies against GitHub’s accepted values and ensures the pip update block remains configured. ## Summary by CodeRabbit * **Bug Fixes** * Corrected the Python dependency update settings to use supported versioning options, preventing configuration rejection. * **Tests** * Added a check to ensure Python dependency updates use valid versioning settings. * **Documentation** * Updated the changelog with the configuration correction. --------- Co-authored-by: Claude Sonnet 5.5 --- .github/dependabot.yml | 13 +++++++------ CHANGELOG.md | 3 +++ tests/test_packaging.py | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+), 6 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index cacb425c..69d81387 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -5,11 +5,13 @@ # `tools/check_pip_audit.py` and the nightly CVE job catch known advisories, and # these pull requests make a fix reachable. # -# `versioning-strategy: widen` matters here. ForgeLM declares bounded ranges -# (`>=floor,=floor, None: ) +def test_dependabot_config_uses_only_values_the_pip_ecosystem_accepts() -> None: + """GitHub rejects the whole ``dependabot.yml`` for one bad value, and Dependabot then opens nothing. + + ``versioning-strategy: widen`` is valid for npm and Composer but not for ``pip``; the file shipped with it and + the "Dependabot" check on every pull request that touched the file said so while the weekly run silently never + happened. The accepted values are listed in GitHub's error message. + """ + import yaml + + config = yaml.safe_load( + (Path(__file__).resolve().parents[1] / ".github" / "dependabot.yml").read_text(encoding="utf-8") + ) + accepted = {"auto", "increase", "increase-if-necessary", "lockfile-only"} + for update in config["updates"]: + if update["package-ecosystem"] != "pip": + continue + strategy = update.get("versioning-strategy") + assert strategy is None or strategy in accepted, ( + f"versioning-strategy {strategy!r} is not accepted for the pip ecosystem ({sorted(accepted)})" + ) + assert any(u["package-ecosystem"] == "pip" for u in config["updates"]), "the pip update block is gone" + + def _load_runtime_dependencies() -> list: try: import tomllib # type: ignore[import-not-found] From 86eb1f76ab07827e7d78063ba3269f38f7960b17 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 12:51:56 +0300 Subject: [PATCH 12/22] docs: compliance and privacy wording, SBOM and Article 12/14, and the six-language site copy, described as the code behaves (#1000) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Compliance- and privacy-facing wording corrected to what the code does, plus the six-language site copy that repeated the same claims. Six earlier branches combined into one pull request, replacing #983, #986, #988 and #991 (to be closed as superseded) and two branches that were never opened. **This is compliance wording; please read the before/after tables.** ## ⚠️ Localization exception `docs/standards/localization.md` keeps German, French, Spanish and Chinese as deferred tiers and says not to machine-translate them. At the owner's decision I **wrote the DE / FR / ES / ZH site strings myself**, with no native review. English and Turkish are authoritative; the other four need a native reader before they can be trusted. A follow-up issue for that is drafted, not yet filed. ## Related issues Refs #73, #99, #90, #94, #101, #87, #114, #75, #86, #89, #92, #98, #110, #112. The code halves stay open: whether the gate should ever compare trainer and approver, failing closed on a missing safety verdict (breaking), attaching SBOMs to the release. ## Changes | Theme | Said | Now says | |---|---|---| | **Segregation of duties** (ISO 27001 / SOC 2 mappings, SoA, SOP, deployer guide, manual) | the approval gate "enforces trainer ≠ approver", "dual-controlled" | it **records** trainer and approver identities and does not compare them; both are self-asserted; keeping them different people is the deployer's control. `require_human_approval` is not enforced for strict tiers by `ConfigError` (it only is when safety evaluation is off) | | **GDPR / KVKK manual page** | identifiable subjects are removed; "no specific person's data is recoverable from the model"; an `international_transfers` setting; audit report with source attribution and SHA-256 manifest; retention "auto-deletes originals" | eight identifier formats are masked, names and free text are not; the report is counts and summaries, not written to the audit log; retention is reported, not enforced; memorisation is not measured | | **No-verdict safety evaluation** (8 documents + TR) | exit 0 ⇒ every gate passed; classifier load failure halts the run; classification errors count as unsafe | a safety evaluation that produced no verdict (`evaluation_completed: false`) keeps the model and exits 0 (4 with the approval gate) regardless of `auto_revert`; code unchanged | | **SBOM** (reference, SOP, guides, manuals, mappings) | `gh release download …`; "re-emit from the git tag"; "licence extraction in SBOM" | workflow-run artifacts, 90-day retention; `gh run download`; the file is the CI runner's `pip list` (test tooling included, no dependency graph, no licences) | | **Article 12 / 14** (audit-event catalogue, audit-log and human-oversight pages) | the audit log is "mandated by Article 12"; the gate "implements" Article 14 | the log is development-time evidence that supports Annex IV; Article 12 logging and Article 14 in-use oversight belong to the deployed system | | **Site, six languages** | benchmark failure "fails (exit 3)"; "dual-controlled"; Slack/Teams/Discord; "PII redacted"; "satisfying the access-request side of the GDPR"; wizard "Enforce automatic deletion"; "SHA-256 manifest"; DeepSpeed "presets" | exit 3 only with `auto_revert: true`; records the approver; Slack-format, Teams and Discord need a relay; common formats masked, names not detected; reverse-pii finds only what is literally in the scanned files; retention mode reserved; SHA-256 sidecar; bring your own DeepSpeed JSON | The site GGUF strings describe the `.sha256` sidecar, the retention strings the reserved mode, and the DeepSpeed strings the preset caveat; they match the GGUF, code and docs pull requests, so **merge those first**. A new site test keeps the retired phrases out of English, Turkish and the HTML (mutation-checked). ### What this changes for a deployer A DPIA or audit response can no longer cite ForgeLM for: segregation of duties being enforced, personal data being removed or unrecoverable from the model, the SBOM being on the release page, or the audit log being the Article 12 mechanism. ## Type - [x] Documentation (and site copy) ## Testing - The full `CLAUDE.md` gauntlet from a clean worktree of this branch: exit 0. (An earlier run failed `tests/test_turkish_glossary_terms.py` because the Article 12 rewrite wrote "Ek IV" instead of "Annex IV", fixed here, then two timing-based regex linearity tests failed while two gauntlets ran at once; they pass alone and in this final run.) - `node --test tests/site/*.test.cjs`: 81 pass; English ↔ Turkish keys in lockstep. ## Checklist - [x] English and Turkish updated together (Turkish wording needs the owner's review) - [ ] DE / FR / ES / ZH reviewed by a native speaker — **not done** - [x] CHANGELOG entries under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 --- .github/workflows/nightly.yml | 2 +- CHANGELOG.md | 43 +++ docs/guides/iso_soc2_deployer_guide-tr.md | 48 ++-- docs/guides/iso_soc2_deployer_guide.md | 48 ++-- docs/guides/safety_compliance-tr.md | 14 +- docs/guides/safety_compliance.md | 12 +- docs/qms/risk_treatment_plan-tr.md | 2 +- docs/qms/risk_treatment_plan.md | 2 +- docs/qms/sop_change_management-tr.md | 14 +- docs/qms/sop_change_management.md | 14 +- docs/qms/statement_of_applicability-tr.md | 4 +- docs/qms/statement_of_applicability.md | 4 +- docs/reference/architecture.md | 2 +- docs/reference/audit_event_catalog-tr.md | 6 +- docs/reference/audit_event_catalog.md | 6 +- docs/reference/iso27001_control_mapping-tr.md | 4 +- docs/reference/iso27001_control_mapping.md | 4 +- .../soc2_trust_criteria_mapping-tr.md | 4 +- docs/reference/soc2_trust_criteria_mapping.md | 4 +- docs/reference/supply_chain_security-tr.md | 32 ++- docs/reference/supply_chain_security.md | 30 +- docs/standards/error-handling.md | 2 +- docs/usermanuals/en/compliance/audit-log.md | 2 +- docs/usermanuals/en/compliance/gdpr.md | 46 ++- .../en/compliance/human-oversight.md | 2 +- .../en/operations/iso-soc2-deployer.md | 6 +- .../usermanuals/en/operations/supply-chain.md | 17 +- docs/usermanuals/en/reference/exit-codes.md | 4 +- docs/usermanuals/en/reference/json-output.md | 2 +- docs/usermanuals/tr/compliance/audit-log.md | 2 +- docs/usermanuals/tr/compliance/gdpr.md | 44 ++- .../tr/compliance/human-oversight.md | 2 +- .../tr/operations/iso-soc2-deployer.md | 6 +- .../usermanuals/tr/operations/supply-chain.md | 17 +- docs/usermanuals/tr/reference/exit-codes.md | 4 +- docs/usermanuals/tr/reference/json-output.md | 2 +- site/compliance.html | 10 +- site/features.html | 8 +- site/index.html | 10 +- site/js/translations.js | 264 +++++++++--------- site/js/wizard.js | 9 +- site/quickstart.html | 9 +- tests/site/site_translations.test.cjs | 29 ++ tools/generate_sbom.py | 5 +- 44 files changed, 471 insertions(+), 330 deletions(-) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 32b9e699..15089281 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -273,7 +273,7 @@ jobs: # the moment a dependency advisory drops, even before the team # re-bumps versions: # - # - pip-audit (transitive CVEs against the OSV / GHSA databases) + # - pip-audit (transitive CVEs against PyPI's vulnerability data, its default service) # - bandit (static-analysis on forgelm/ production code only) # # Severity policy (per tools/check_pip_audit.py + tools/check_bandit.py): diff --git a/CHANGELOG.md b/CHANGELOG.md index a8ec66d1..bdcc5ea1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,6 +87,49 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Stop describing the human-approval gate as enforcing segregation of duties (English and Turkish): it records the + trainer and approver identities and does not compare them, and both are self-asserted. The ISO 27001 and SOC 2 + mappings, the Statement of Applicability, the change-management SOP, the deployer guide and the user manual now + say what the gate records and that keeping the two people different is the deployer's control. The guide also no + longer says `require_human_approval` is enforced for strict risk tiers: it is not (#73, #99, #90). +- Rewrite the GDPR / KVKK user-manual page (English and Turkish) to what ForgeLM does. It claimed that "by the time + data lands in JSONL, identifiable subjects are removed", that "no specific person's data is recoverable from the model" + and that the audit report "confirms this", that the audit flags memorisation, that a retention setting "auto-deletes" + originals and "records the intended TTL in the audit log", and it showed an `ingestion: pii_mask:` block and an + `international_transfers.enabled` setting that do not exist. Masking now reads as reducing the eight detected identifier + formats (names, addresses and free text are not detected), the audit report as the plain JSON it is (no source + attribution, no hash manifest), and access and erasure requests as `forgelm reverse-pii` and `forgelm purge` (#101). +- Document the one case where a failed safety gate still exits 0 with `auto_revert: true`: an evaluation that produced no + verdict (`safety.evaluation_completed: false` — classifier not loaded, probes file missing or empty, or most verdicts + unreadable) keeps the model and the run exits 0, or 4 with the approval gate. The error-handling standard, the exit-codes and + JSON-output pages, the audit-event catalogue, the safety guide, the deployer guide and the risk-treatment plan said that exit + 0 meant every gate passed, that a classifier load failure "halts" the pipeline and leaves no `final_model/`, and that + classification errors count as unsafe. The behaviour is unchanged; making it fail closed is a breaking exit-code change + left for a major version (#87). +- Say where the per-release SBOMs actually are and what they contain. The supply-chain reference, the ISO 27001 / SOC 2 deployer guide, + the change-management SOP and the supply-chain and deployer user-manual pages told an auditor to run `gh release download … --pattern 'sbom-*'` + and to "re-emit the SBOM from the git tag and diff it against the one attached to the release". Nothing attaches an SBOM to a + GitHub Release: `publish.yml` uploads the twelve files as workflow-run artifacts, which GitHub deletes after 90 days. The docs now give + the working commands (`gh run list --workflow publish.yml --branch v --status success`, then `gh run download`), say that a + release SBOM is the CI runner's `pip list` (test tooling such as `pytest` included, no dependency graph, no licence data) and so is + not reproducible from the tag, and drop "license extraction in SBOM" from the ISO A.5.32, SOC 2 CC9.2 and statement-of-applicability + rows. The nightly `pip-audit` is described as querying PyPI's vulnerability data (its default service, not OSV), and the manual's + "exit 1 means HIGH/CRITICAL" now includes UNKNOWN-severity findings. English and Turkish. Attaching the SBOMs to the release and + generating them from a clean runtime-only environment are code changes and stay open on the issue (#114). +- Stop saying that ForgeLM's audit log is the record-keeping artefact "mandated by EU AI Act Article 12" and that the approval gate + "implements" Article 14. Article 12 asks the deployed high-risk system to log events automatically over its lifetime, and Article 14's + oversight (monitor, interpret, override, stop) happens while the system is in use. `audit_log.jsonl` is development-time evidence of the + training pipeline, and `evaluation.require_human_approval` is a pre-promotion sign-off that supports design-time oversight documentation. + The audit-event catalogue, the audit-log and human-oversight user-manual pages (English and Turkish) now say that (#75). +- Correct site copy that promised more than the code does, in all six languages (`site/js/translations.js`, the HTML fallbacks and a code + comment in `site/js/wizard.js`): a failing benchmark gate exits 3 only with `evaluation.auto_revert: true` (and a safety evaluation with + no verdict keeps the model and exits 0); the approval gate records the approver, it is not "dual-controlled" and does not implement + Article 14's in-use oversight; webhook alerts are Slack-format, Teams and Discord need a relay; PII masking covers common formats, not + names or free text, and `reverse-pii` does not "satisfy" an access request; the wizard's retention "enforce" option is reserved and + deletes nothing; GGUF export writes a SHA-256 sidecar (K-quants need `llama-quantize`) rather than a "manifest"; DeepSpeed preset names + need a source checkout. A new site test keeps those phrases from coming back. English and Turkish are the maintained tiers; the German, + French, Spanish and Chinese strings were written by the assistant, not a native reviewer, and need review (#86, #89, #92, #94, #98, #99, + #101, #110, #112). The `forgelm doctor` sample on the quickstart page now uses the real renderer's format (#534). - Make `.github/dependabot.yml` parse. It set `versioning-strategy: widen` for `pip`, which that ecosystem does not accept (GitHub's "Dependabot" check on any pull request touching the file said so), so Dependabot opened no pull requests at all. The setting is removed, the header comment explains the restriction. diff --git a/docs/guides/iso_soc2_deployer_guide-tr.md b/docs/guides/iso_soc2_deployer_guide-tr.md index fd52e63e..196d0689 100644 --- a/docs/guides/iso_soc2_deployer_guide-tr.md +++ b/docs/guides/iso_soc2_deployer_guide-tr.md @@ -26,16 +26,20 @@ ForgeLM kanıtına sahip: 2. **Change control** — Madde 14 staging gate (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit olayları + koşum başına damgalanan `config_hash` (manifest sidecar - alanı). Her model promotion çift kontrollü ve forensic olarak - attribute edilmiştir. + alanı). Onay + kapısı etkinleştirildiğinde her promotion, audit log'a kaydedilen onaylayıcı kimliğine atfedilir. ForgeLM + eğitici ve onaylayıcı kimliklerini kaydeder ama karşılaştırmaz: bunların farklı kişiler olmasını kimlik + sağlayıcınızda uygularsınız. 3. **Data lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime + `hf_revision_source` ile derecelendirilmiş dataset HF Hub commit SHA'sı); `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). 4. **Supply chain** — yayın etiketi başına yayın matrisinin her (OS - × Python-version) hücresi için emit edilen CycloneDX 1.5 SBOM. - Wave 4 statik + dinamik güvenlik taraması için `pip-audit` + × Python-version) hücresi için emit edilen CycloneDX 1.5 SBOM; + 90 gün boyunca workflow-run artifact olarak saklanır (GitHub + release'e eklenmez; CI runner'ının `pip list`'ini, test araçları + dahil, lisans verisi olmadan listeler). Wave 4 statik + dinamik güvenlik taraması için `pip-audit` nightly + `bandit` CI ekler. ## Denetim gözlem dönemi öncesi setup checklist'i @@ -65,7 +69,9 @@ Denetçi sormadan **önce** indirmeniz gereken on iki madde. - [ ] **`evaluation.require_human_approval: true`** her `risk_classification` ∈ `{high-risk, unacceptable}` koşumu - için. Unutursanız F-compliance-110 `ConfigError` raise eder. + için. ForgeLM bunu zorlamaz: F-compliance-110 yalnızca bu katmanlarda `evaluation.safety.enabled` + kapalıysa `ConfigError` raise eder; dolayısıyla onay kapısı olmayan, elle yazılmış bir strict-katman + YAML'ı kabul edilir. CLI sihirbazı (`forgelm --wizard`) bu katmanlarda kapıyı açar. - [ ] **Webhook lifecycle yapılandır** (`url_env`, `secret_env`, `notify_on_*`) böylece SIEM her durum geçişini içe alır. - [ ] **`auto_revert: true` etkinleştir** üretim eğitimi için — @@ -155,8 +161,9 @@ jq -r 'select(.run_id == "X" and (.event == "training.started" or .event == "hum ./outputs/audit_log.jsonl | sort -u ``` -İki farklı operator ID görev ayrılığını kanıtlar (ISO A.5.3, SOC 2 -CC1.5). +`training.started` üzerindeki operatörü `human_approval.granted` üzerindeki onaylayıcıyla karşılaştırın. İki kimlik de +kendi beyanıdır (`FORGELM_OPERATOR`, yoksa OS kullanıcısı); dolayısıyla görev ayrılığı (ISO A.5.3, SOC 2 CC1.5) ancak +kimlik sağlayıcınız bunları farklı kişiler yapıyorsa geçerlidir. ### S3: "Data lineage göster" @@ -198,19 +205,25 @@ eğittiğinin kanıtı saymayın. ### S4: "Supply chain göster" ```bash -# SBOM artefaktını GitHub release sayfasından indir. -gh release download v0.5.5 --pattern 'sbom-*' - -# Veya istek üzerine yeniden üret. +# Tag'in publish çalıştırmasından bir SBOM indir. Bunlar workflow-run +# artifact'larıdır, 90 günde silinir; GitHub release asset'i değildir. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v0.11.0 --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 + +# Veya kendi ortamınızı tanımlayın: modeli eğiten ya da sunan ortamda, +# bir ForgeLM checkout'undan çalıştırın. python3 tools/generate_sbom.py > sbom.json # Son yayınla diff al. diff <(jq -S . sbom-prev.json) <(jq -S . sbom.json) ``` -CycloneDX 1.5 JSON her transitive bağımlılığı purl -(`pkg:pypi/...`) ve sürümle listeler. Dependency-Track CVE -korelasyonu için bunu doğal olarak içe alır. +CycloneDX 1.5 JSON kurulu her paketi purl (`pkg:pypi/...`) ve +sürümle listeler; bağımlılık grafiği ve lisans verisi yoktur. +Release SBOM'ları CI runner'ını tanımlar, bu yüzden `pytest` gibi +test araçlarını da adlandırır. Dependency-Track dosyayı CVE +korelasyonu için doğal olarak içe alır. ### S5: "Erişim kontrolleri göster — yalnız yetkili reviewer'ların model onayladığını nasıl ispatlıyorsunuz?" @@ -249,8 +262,11 @@ Referans: - `docs/qms/sop_incident_response.md` §4 (Wave 4 / Faz 23 genişletme) güvenlik-incident playbook için. -- Zincirin kendisi: `audit.classifier_load_failed` event yangın + - `pipeline.failed` propagate + koşum `final_model/` dizini ÜRETMEZ. +- Zincirin kendisi: `audit.classifier_load_failed` event'i tetiklenir + ve `safety.evaluation_completed` `evaluation_completed: false` kaydeder. + Koşu **durdurulmaz**: model `final_model/` içinde kalır ve koşu 0 ile + çıkar (onay kapısıyla 4); bu yüzden exit koduna değil, bu event'lere ya da + JSON zarfının `safety` bloğuna göre alarm kurun. - F-compliance-110 strict gate: `risk_classification ∈ {high-risk, unacceptable}` AND `evaluation.safety.enabled = false` ise config validator koşum başlamadan ÖNCE `ConfigError` raise eder. Denetçi diff --git a/docs/guides/iso_soc2_deployer_guide.md b/docs/guides/iso_soc2_deployer_guide.md index 3132ef96..9d020375 100644 --- a/docs/guides/iso_soc2_deployer_guide.md +++ b/docs/guides/iso_soc2_deployer_guide.md @@ -25,15 +25,20 @@ direct ForgeLM evidence: `forgelm verify-audit` validates the chain end-to-end. 2. **Change control** — Article 14 staging gate (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit - events + `config_hash` (per-run manifest sidecar field) stamped per run. Every - model promotion is dual-controlled and forensically attributed. + events + `config_hash` (per-run manifest sidecar field) stamped per run. When + the approval gate is enabled, every promotion is attributed to the approver identity recorded in the audit + log. ForgeLM records the trainer and approver identities but does not compare them: keeping them different + people is a control you enforce at your identity provider. 3. **Data lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime + the dataset's HF Hub commit SHA, graded by `hf_revision_source`); `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). 4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release tag for - every (OS × Python-version) cell of the publish matrix. Wave 4 + every (OS × Python-version) cell of the publish matrix, kept as a + workflow-run artifact for 90 days (not attached to the GitHub + release; it lists the CI runner's `pip list`, test tooling + included, with no licence data). Wave 4 adds `pip-audit` nightly + `bandit` CI for static + dynamic security scanning. @@ -63,7 +68,9 @@ Twelve items to land **before** the auditor asks. - [ ] **`evaluation.require_human_approval: true`** for every `risk_classification` ∈ `{high-risk, unacceptable}` run. - F-compliance-110 raises `ConfigError` if you forget. + ForgeLM does not enforce this: F-compliance-110 only raises `ConfigError` when + `evaluation.safety.enabled` is off for these tiers, so a hand-written strict-tier YAML without the + approval gate is accepted. The CLI wizard (`forgelm --wizard`) switches the gate on for these tiers. - [ ] **Configure webhook lifecycle** (`url_env`, `secret_env`, `notify_on_*`) so SIEM ingests every state transition. - [ ] **Enable `auto_revert: true`** for production training so @@ -155,8 +162,9 @@ jq -r 'select(.run_id == "X" and (.event == "training.started" or .event == "hum ./outputs/audit_log.jsonl | sort -u ``` -Two distinct operator IDs prove segregation of duties (ISO A.5.3, -SOC 2 CC1.5). +Compare the operator on `training.started` with the approver on `human_approval.granted`. Both identities are +self-asserted (`FORGELM_OPERATOR`, else the OS user), so segregation of duties (ISO A.5.3, SOC 2 CC1.5) holds only +if your identity provider makes them different people. ### Q3: "Show me the data lineage" @@ -198,19 +206,25 @@ corpus trained the model. ### Q4: "Show me the supply chain" ```bash -# Download the SBOM artefact from the GitHub release page. -gh release download v0.5.5 --pattern 'sbom-*' - -# Or regenerate on demand. +# Download one SBOM from the tag's publish run. They are workflow-run +# artifacts that expire after 90 days, not GitHub release assets. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v0.11.0 --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 + +# Or describe your own environment: run this from a ForgeLM checkout in the +# environment that trains or serves the model. python3 tools/generate_sbom.py > sbom.json # Diff against last release. diff <(jq -S . sbom-prev.json) <(jq -S . sbom.json) ``` -The CycloneDX 1.5 JSON lists every transitive dependency with its -purl (`pkg:pypi/...`) and version. Dependency-Track ingests this -natively for CVE correlation. +The CycloneDX 1.5 JSON lists every installed package with its purl +(`pkg:pypi/...`) and version; it has no dependency graph and no licence +data. The release SBOMs describe the CI runner, so they also name test +tooling such as `pytest`. Dependency-Track ingests the file natively for +CVE correlation. ### Q5: "Show me the access controls — how do you prove only authorised reviewers approve models?" @@ -248,9 +262,11 @@ Reference: - `docs/qms/sop_incident_response.md` §4 (Wave 4 / Faz 23 expansion) for the security-incident playbook. -- The audit chain itself: `audit.classifier_load_failed` event fires - + `pipeline.failed` propagates + the run does NOT produce a - `final_model/` directory. +- The audit chain itself: the `audit.classifier_load_failed` event fires + and `safety.evaluation_completed` records `evaluation_completed: false`. + The run is **not** halted: the model stays in `final_model/` and the + run exits 0 (4 with the approval gate), so alert on those events or on + the JSON envelope's `safety` block, not on the exit code. - F-compliance-110 strict gate: if `risk_classification ∈ {high-risk, unacceptable}` AND `evaluation.safety.enabled = false`, the config validator raises `ConfigError` BEFORE the run starts. diff --git a/docs/guides/safety_compliance-tr.md b/docs/guides/safety_compliance-tr.md index 547edf2f..a4ece7f1 100644 --- a/docs/guides/safety_compliance-tr.md +++ b/docs/guides/safety_compliance-tr.md @@ -95,8 +95,7 @@ Kapı 2: Confidence-weighted → safety_score < min_safety_score? Kapı 3: Şiddet → critical_count > 0? high_count > eşik? ``` -Üç kapı bağımsızdır. Herhangi bir kapının başarısızlığı (etkinse) -auto-revert tetikler. +Üç kapı bağımsızdır. Bir kapının başarısızlığı (etkinse) auto-revert tetikler; güvenlik değerlendirmesi hiç karar üretemediyse bu geçerli değildir (bkz. [Fail-safe davranış](#fail-safe-davranış)). ### Zarar kategorileri (Llama Guard 3) @@ -204,13 +203,12 @@ iyileştirip bozmadığını izlemesine yardım eder. ### Fail-safe davranış -- Sınıflandırma hataları **güvensiz** olarak değerlendirilir (fail-safe - prensibi) +- Sınıflandırıcının yanıtlayamadığı bir çift **skorlanmamış** sayılır, güvenli değil - Düşük-güvenli sınıflandırmalar manuel inceleme için işaretlenir (uyarı log'u) -- Güvenlik değerlendirmesi başarısız olursa ve `auto_revert: true` ise - model otomatik olarak silinir -- Hat entegrasyonu için `3` exit kodu döner +- Skorlanmış bir güvenlik değerlendirmesi başarısız olursa ve `auto_revert: true` ise + model otomatik olarak silinir ve hat entegrasyonu için `3` exit kodu döner +- **Değerlendirme karar üretemezse** (`safety.evaluation_completed: false` — sınıflandırıcı yüklenemedi, probe dosyası yok ya da boş, ya da çoğu karar okunamadı), model `auto_revert: true` olsa bile **korunur** ve koşu `0` ile çıkar (onay kapısıyla `4`); çünkü okunmamış bir karar zarar kanıtı değildir. Yalnızca exit koduna değil, JSON çıktısındaki `safety.evaluation_completed` ve `safety.passed`'e bakın --- @@ -522,6 +520,8 @@ Herhangi bir adım başarısız olursa ve `auto_revert: true` ise: Model silinir + Webhook bildirimi (başarısızlık) + Exit kodu 3 ``` +Karar üretemeyen bir güvenlik değerlendirmesi (`evaluation_completed: false`) istisnadır: model korunur ve koşu 0 ile çıkar. + ### İnsan onay kapısı (Madde 14) Tüm otomatik kontroller geçtikten sonra hattı duraklatmak için diff --git a/docs/guides/safety_compliance.md b/docs/guides/safety_compliance.md index 5569369c..6f4f585a 100644 --- a/docs/guides/safety_compliance.md +++ b/docs/guides/safety_compliance.md @@ -77,7 +77,7 @@ Gate 2: Confidence-weighted → safety_score < min_safety_score? Gate 3: Severity → critical_count > 0? high_count > threshold? ``` -All three gates are independent. Any gate failure triggers auto-revert (if enabled). +All three gates are independent. A gate failure triggers auto-revert (if enabled), unless the safety evaluation produced no verdict at all (see [Fail-Safe Behavior](#fail-safe-behavior)). ### Harm Categories (Llama Guard 3) @@ -179,10 +179,10 @@ This helps teams track whether data/config changes are improving or degrading sa ### Fail-Safe Behavior -- Classification errors are treated as **unsafe** (fail-safe principle) +- A pair the classifier cannot answer is counted as **unscored**, not as safe - Low-confidence classifications are flagged for manual review (warning log) -- If safety evaluation fails and `auto_revert: true`, the model is automatically deleted -- Exit code `3` is returned for pipeline integration +- If a scored safety evaluation fails and `auto_revert: true`, the model is automatically deleted and exit code `3` is returned for pipeline integration +- **If the evaluation produces no verdict** (`safety.evaluation_completed: false` — the classifier failed to load, the probes file is missing or empty, or most verdicts were unreadable), the model is **kept** and the run exits `0` (`4` with the approval gate) even with `auto_revert: true`, because an unread verdict is not evidence of harm. Branch on `safety.evaluation_completed` and `safety.passed` in the JSON output, not on the exit code alone --- @@ -461,12 +461,14 @@ Model saved + Model card + Compliance artifacts Webhook notification (success) ``` -If any step fails and `auto_revert: true`: +If a gate fails with a verdict and `auto_revert: true`: ``` ↓ fail Model deleted + Webhook notification (failure) + Exit code 3 ``` +A safety evaluation that produced no verdict (`evaluation_completed: false`) is the exception: the model is kept and the run exits 0. + ### Human Approval Gate (Art. 14) Add `require_human_approval: true` to pause the pipeline after all automated checks pass: diff --git a/docs/qms/risk_treatment_plan-tr.md b/docs/qms/risk_treatment_plan-tr.md index 9570b600..073d0c25 100644 --- a/docs/qms/risk_treatment_plan-tr.md +++ b/docs/qms/risk_treatment_plan-tr.md @@ -103,7 +103,7 @@ ekler. |---|---| | Açıklama | Llama Guard / yapılandırılmış safety classifier yüklenemiyor (HF Hub down, OOM, version mismatch); ForgeLM gate yangın etmeden devam eder | | L × I (inherent) | Med × High = HIGH | -| Treatment | F-compliance-110 strict gate: `risk_classification` ∈ `{high-risk, unacceptable}` + `evaluation.safety.enabled: false` `ConfigError` raise eder; `audit.classifier_load_failed` event arızayı kaydeder; operatör pipeline'ı non-zero exit'te durur | +| Treatment | F-compliance-110 strict gate: `risk_classification` ∈ `{high-risk, unacceptable}` + `evaluation.safety.enabled: false` `ConfigError` raise eder; `audit.classifier_load_failed` event arızayı kaydeder. Koşu **durdurulmaz**: karar üretemeyen bir güvenlik değerlendirmesi modeli korur ve 0 ile çıkar (onay kapısıyla 4), bu yüzden operatör pipeline'ı yalnızca exit koduna değil, JSON zarfındaki `safety.evaluation_completed` / `safety.passed`'e ya da o audit event'ine bakmalıdır | | Residual L × I | Low × Low = LOW | | Sahip | ML Lead | | Review cadence | Eğitim koşumu başına | diff --git a/docs/qms/risk_treatment_plan.md b/docs/qms/risk_treatment_plan.md index 43258b00..5b5ef08b 100644 --- a/docs/qms/risk_treatment_plan.md +++ b/docs/qms/risk_treatment_plan.md @@ -102,7 +102,7 @@ section §4. |---|---| | Description | Llama Guard / configured safety classifier fails to load (HF Hub down, OOM, version mismatch); ForgeLM proceeds without the gate fired | | L × I (inherent) | Med × High = HIGH | -| Treatment | F-compliance-110 strict gate: `risk_classification` ∈ `{high-risk, unacceptable}` + `evaluation.safety.enabled: false` raises `ConfigError`; `audit.classifier_load_failed` event records the failure; deployer pipeline halts on non-zero exit | +| Treatment | F-compliance-110 strict gate: `risk_classification` ∈ `{high-risk, unacceptable}` + `evaluation.safety.enabled: false` raises `ConfigError`; `audit.classifier_load_failed` event records the failure. The run is **not** halted: a safety evaluation with no verdict keeps the model and exits 0 (4 with the approval gate), so the deployer pipeline must branch on the JSON envelope's `safety.evaluation_completed` / `safety.passed` or on that audit event, not on the exit code alone | | Residual L × I | Low × Low = LOW | | Owner | ML Lead | | Review cadence | Per training run | diff --git a/docs/qms/sop_change_management-tr.md b/docs/qms/sop_change_management-tr.md index c010fc36..8a3d1a4b 100644 --- a/docs/qms/sop_change_management-tr.md +++ b/docs/qms/sop_change_management-tr.md @@ -109,8 +109,9 @@ CAB'sıdır: retention süresi dolana kadar (`retention.staging_ttl_days` — config'te). -Bu, operatöre SOC 2 CC8.1 kanıtı verir: her model promotion -attribute, çift kontrollü ve forensic olarak kayıtlıdır. +Bu, operatöre SOC 2 CC8.1 kanıtı verir: onay kapısı etkinleştirildiğinde her +model promotion kaydedilen onaylayıcı kimliğine atfedilir ve forensic olarak kayıtlıdır. Eğitici ile onaylayıcıyı +ayırmak dağıtımı yapanın kontrolüdür: ForgeLM iki kimliği karşılaştırmaz. ### 4.2 Yapılandırma drift tespiti @@ -134,9 +135,12 @@ olarak atfeder. 1.5 SBOM üretir. Wave 4 bir determinism contract testi ekler (`tests/test_supply_chain_security.py::TestGenerateSbomDeterministic`). -Drift tespiti: bir release'in SBOM'u `git tag`'inden tekrarlanabilir. -Bir denetçi bunu istek üzerine yeniden emit edip GitHub release'e -ekli artefakta diff alabilir. +Drift tespiti: SBOM, release'in build edildiği ortamın kaydıdır (matris runner'ında +`pip list`); tag'in `publish.yml` çalıştırmasının workflow-run artifact'ı olarak 90 gün +saklanır, GitHub release'e eklenmez. Bağımlılıklar build anında PyPI'dan çözüldüğü için +yalnızca `git tag`'den tekrarlanamaz. İki release'in SBOM'unu `diff` ile karşılaştırın; +dağıtılmış bir ortamı kanıtlamak için orada `python3 tools/generate_sbom.py` ile yeni +bir tane üretin. ## 5. Versiyon İzleme diff --git a/docs/qms/sop_change_management.md b/docs/qms/sop_change_management.md index 79336c3d..cffc7820 100644 --- a/docs/qms/sop_change_management.md +++ b/docs/qms/sop_change_management.md @@ -104,8 +104,9 @@ in-pipeline CAB: in `final_model.staging./` for forensic review until retention expires (`retention.staging_ttl_days` in the config). -This gives a deployer SOC 2 CC8.1 evidence: every model promotion is -attributed, dual-controlled, and forensically recorded. +This gives a deployer SOC 2 CC8.1 evidence: when the approval gate is +enabled, every model promotion is attributed to the recorded approver identity and forensically recorded. +Separating trainer and approver is the deployer's control: ForgeLM does not compare the two identities. ### 4.2 Configuration drift detection @@ -129,9 +130,12 @@ CycloneDX 1.5 SBOM per (OS × Python-version) on every release tag (`publish.yml`). Wave 4 adds a determinism contract test (`tests/test_supply_chain_security.py::TestGenerateSbomDeterministic`). -Drift detection: a release's SBOM is reproducible from its `git tag`. -An auditor can re-emit it on demand and diff against the artefact -attached to the GitHub release. +Drift detection: the SBOM is a record of the environment the release was built in +(`pip list` on the matrix runner), kept as a workflow-run artifact of the tag's +`publish.yml` run for 90 days; it is not attached to the GitHub release. It is not +reproducible from the `git tag` alone, because dependencies resolve from PyPI at build +time. Compare two releases' SBOMs with `diff`; to evidence a deployed environment, emit +a fresh one there with `python3 tools/generate_sbom.py`. ## 5. Version Tracking diff --git a/docs/qms/statement_of_applicability-tr.md b/docs/qms/statement_of_applicability-tr.md index ac90f3c5..3f93b03f 100644 --- a/docs/qms/statement_of_applicability-tr.md +++ b/docs/qms/statement_of_applicability-tr.md @@ -59,7 +59,7 @@ operatör-tarafı eylem. | A.5.29 Aksaklık sırasında bilgi güvenliği | YES | `auto_revert` baseline-flip; `model.reverted` event | Base-model retention dokümante et | | A.5.30 İş sürekliliği için ICT hazırlığı | YES | DR planlama | Benimse | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | YES | EU AI Act + GDPR mapping'ler; Annex IV bundle | Kural değişikliklerini izle | -| A.5.32 Fikri mülkiyet hakları | YES | SBOM'da lisans çıkarımı; HF model-card metadata | Per-model lisans review | +| A.5.32 Fikri mülkiyet hakları | YES | HF model-card metadata (SBOM lisans verisi taşımaz) | Per-model lisans review | | A.5.33 Kayıtların korunması | YES | Append-only + HMAC + manifest sidecar; off-site replica | Off-site backup | | A.5.34 Gizlilik ve PII korunması | YES | `forgelm reverse-pii` Md. 15; `forgelm purge` Md. 17; `forgelm audit` PII tespiti + maskeleme | DSAR workflow + DPIA | | A.5.35 Bilgi güvenliğinin bağımsız incelenmesi | YES | Yıllık external denetim | Benimse | @@ -108,7 +108,7 @@ ForgeLM-spesifik kontrol envanteriyle ilgilidir. | Kontrol | Applicable? | ForgeLM kanıtı | Operatör-tarafı eylem | |---|---|---|---| | A.8.1 Kullanıcı uç cihazları | YES | `forgelm doctor` — Python / CUDA / GPU / extras / HF auth / disk / `FORGELM_OPERATOR` checks | Endpoint hardening | -| A.8.2 Ayrıcalıklı erişim hakları | YES | Operatör attribution; approval gate non-trainer operator | IdP'de RBAC | +| A.8.2 Ayrıcalıklı erişim hakları | YES | Operatör attribution; approval gate onaylayıcı kimliğini kaydeder | IdP'de RBAC | | A.8.3 Bilgi erişim kısıtlaması | YES | Salted identifier hashing; `forgelm reverse-pii` Md. 15 | DSAR workflow | | A.8.4 Kaynak koduna erişim | YES | `model.trust_remote_code=False` default; `model.revision` temel-model deposunu sabitler ve `trust_remote_code: true` ile çalıştırılacak depo-içi kodu sınırlar. `_fingerprint_hf_revision` yalnızca **dataset** deposunu kapsar | VCS erişim kontrolü; `trust_remote_code: true` onaylanan her yerde `model.revision` zorunlu tutun | | A.8.5 Güvenli kimlik doğrulama | YES | `safe_post` non-HTTPS'de auth header reddeder; webhook secret discipline | MFA + token rotasyonu | diff --git a/docs/qms/statement_of_applicability.md b/docs/qms/statement_of_applicability.md index b78bc887..26767b04 100644 --- a/docs/qms/statement_of_applicability.md +++ b/docs/qms/statement_of_applicability.md @@ -58,7 +58,7 @@ contribution → deployer-side action. | A.5.29 Information security during disruption | YES | `auto_revert` baseline-flip; `model.reverted` event | Document base-model retention | | A.5.30 ICT readiness for business continuity | YES | DR planning | Adopt | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | YES | EU AI Act + GDPR mappings; Annex IV bundle | Track rule changes | -| A.5.32 Intellectual property rights | YES | License extraction in SBOM; HF model-card metadata | Per-model license review | +| A.5.32 Intellectual property rights | YES | HF model-card metadata (the SBOM carries no licence data) | Per-model license review | | A.5.33 Protection of records | YES | Append-only + HMAC + manifest; off-site replica | Off-site backup | | A.5.34 Privacy and protection of PII | YES | `forgelm reverse-pii` Article 15; `forgelm purge` Article 17; `forgelm audit` PII detection + masking | DSAR workflow + DPIA | | A.5.35 Independent review of information security | YES | Annual external audit | Adopt | @@ -107,7 +107,7 @@ ForgeLM-specific control inventory. | Control | Applicable? | ForgeLM evidence | Deployer-side action | |---|---|---|---| | A.8.1 User endpoint devices | YES | `forgelm doctor` — Python / CUDA / GPU / extras / HF auth / disk / `FORGELM_OPERATOR` checks | Endpoint hardening | -| A.8.2 Privileged access rights | YES | Operator attribution; approval gate non-trainer operator | RBAC at IdP | +| A.8.2 Privileged access rights | YES | Operator attribution; approval gate records the approver identity | RBAC at IdP | | A.8.3 Information access restriction | YES | Salted identifier hashing; `forgelm reverse-pii` Article 15 | DSAR workflow | | A.8.4 Access to source code | YES | `model.trust_remote_code=False` default; `model.revision` pins the base-model repo, bounding the repo-bundled code `trust_remote_code: true` would run. `_fingerprint_hf_revision` covers the **dataset** repo only | VCS access control; require `model.revision` wherever `trust_remote_code: true` is approved | | A.8.5 Secure authentication | YES | `safe_post` rejects auth headers on non-HTTPS; webhook secret discipline | MFA + token rotation | diff --git a/docs/reference/architecture.md b/docs/reference/architecture.md index 735b3f2b..e3872c43 100644 --- a/docs/reference/architecture.md +++ b/docs/reference/architecture.md @@ -134,7 +134,7 @@ Lightweight `TrainResult` dataclass — importable without torch/transformers. C Wraps EleutherAI `lm-evaluation-harness`. Runs configurable benchmark tasks, extracts accuracy metrics, applies min_score threshold, and saves results. Optional dependency: `pip install forgelm[eval]`. ### `safety/` -Runs a configurable safety classifier (Llama Guard, ShieldGemma) on adversarial test prompts. Generates responses from the fine-tuned model, classifies each as safe/unsafe, and triggers auto-revert if regression exceeds threshold. Errors are treated as unsafe (fail-safe principle). +Runs a configurable safety classifier (Llama Guard, ShieldGemma) on adversarial test prompts. Generates responses from the fine-tuned model, classifies each as safe/unsafe, and triggers auto-revert if regression exceeds threshold. A pair the classifier cannot answer is counted as unscored, not as safe; when the classifier does not load, the probes file is missing or empty, or most verdicts are unreadable, the evaluation produces no verdict (`evaluation_completed: false`) and the model is kept (the run exits 0). ### `judge.py` LLM-as-Judge evaluation supporting API-based judges (OpenAI-compatible endpoint) and local model judges. Includes robust JSON parsing with markdown code block extraction. Scores on 1-10 scale with configurable minimum threshold. diff --git a/docs/reference/audit_event_catalog-tr.md b/docs/reference/audit_event_catalog-tr.md index 051d863e..452d4e73 100644 --- a/docs/reference/audit_event_catalog-tr.md +++ b/docs/reference/audit_event_catalog-tr.md @@ -1,9 +1,9 @@ # Audit Event Kataloğu -> **Hedef kitle:** EU AI Act Madde 12 kayıt-tutma artefaktlarını inceleyen ForgeLM operatörleri, denetçiler ve aşağı akış doğrulayıcıları. +> **Hedef kitle:** ForgeLM'in eğitim-hattı audit log'unu inceleyen ForgeLM operatörleri, denetçiler ve aşağı akış doğrulayıcıları. > **Ayna:** [audit_event_catalog.md](audit_event_catalog.md) -Bu katalog, ForgeLM'in EU AI Act Madde 12 tarafından zorunlu kılınan, append-only ve hash-zincirli kayıt-tutma artefaktı olan `audit_log.jsonl`'a yazabileceği tüm event'leri sıralar. Her satır ortak bir zarf paylaşır; aşağıdaki satırlardan birini `event` alanı seçer. +Bu katalog, ForgeLM'in append-only ve hash-zincirli eğitim-hattı audit log'u olan `audit_log.jsonl`'a yazabileceği tüm event'leri sıralar. Bu log, teknik dokümantasyonu (Annex IV) ve kalite-yönetimi kayıt tutmayı destekleyebilecek geliştirme-zamanı kanıtıdır; tek başına EU AI Act Madde 12'yi karşılamaz, çünkü Madde 12 dağıtılmış yüksek-riskli sistemin ömrü boyunca olayları otomatik kaydetmesini ister. Her satır ortak bir zarf paylaşır; aşağıdaki satırlardan birini `event` alanı seçer. ## Ortak zarf @@ -161,7 +161,7 @@ damgasıyla kurabilir. Uygulama: `forgelm/webhook.py`. | Webhook `event` | Denetim günlüğü karşılığı | Tetikleyici | Kapı (gate) | Zorunlu payload alanları | |---|---|---|---|---| | `training.start` | `training.started` | `train()` çağrıldı, model yüklenmeden önce. | `webhook.notify_on_start` | `run_name`, `status="started"` | -| `training.success` | `pipeline.completed` | Koşu revert veya bekleyen onay olmadan tamamlandı. `evaluation.auto_revert: true` ile tüm kapılar geçildi; varsayılan `auto_revert: false` ile bir kapı geçemediği halde yalnızca kaydedildiğinde de tetiklenir (model yine terfi eder). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | +| `training.success` | `pipeline.completed` | Koşu revert veya bekleyen onay olmadan tamamlandı. `evaluation.auto_revert: true` ile karara varan tüm kapılar geçildi (karar üretemeyen bir güvenlik değerlendirmesi, `safety.evaluation_completed: false`, modeli korur ve yine buraya varır); varsayılan `auto_revert: false` ile bir kapı geçemediği halde yalnızca kaydedildiğinde de tetiklenir (model yine terfi eder). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | | `training.failure` | `pipeline.failed` | Eğitim sürecinin kendisi hata fırlattı (OOM, veri seti hatası, yakalanmayan istisna). | `webhook.notify_on_failure` | `run_name`, `status="failed"`, `reason` (maskelenmiş, ≤2048 karakter) | | `training.reverted` | `model.reverted` | Eğitim sonrası bir kapı (değerlendirme, güvenlik, hakem, benchmark) çalışmayı reddetti ve `_revert_model` adaptörleri sildi. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `reason` (maskelenmiş, ≤2048 karakter) | | `approval.required` | `human_approval.required` | Çalışma başarılı oldu, `evaluation.require_human_approval=true`, model insan incelemesi için staging'de (EU AI Act Madde 14). | `webhook.notify_on_success` | `run_name`, `status="awaiting_approval"`, `model_path` | diff --git a/docs/reference/audit_event_catalog.md b/docs/reference/audit_event_catalog.md index e5d05c32..d5b93ebd 100644 --- a/docs/reference/audit_event_catalog.md +++ b/docs/reference/audit_event_catalog.md @@ -1,9 +1,9 @@ # Audit Event Catalog -> **Audience:** ForgeLM operators, regulators, and downstream verifiers reviewing the EU AI Act Article 12 record-keeping artefacts. +> **Audience:** ForgeLM operators, regulators, and downstream verifiers reviewing ForgeLM's training-pipeline audit log. > **Mirror:** [audit_event_catalog-tr.md](audit_event_catalog-tr.md) -This catalog enumerates every event ForgeLM may append to `audit_log.jsonl`, the append-only, hash-chained record-keeping artefact mandated by EU AI Act Article 12. Every entry shares a common envelope; the `event` field selects one of the rows below. +This catalog enumerates every event ForgeLM may append to `audit_log.jsonl`, ForgeLM's append-only, hash-chained training-pipeline audit log. It is development-time evidence that can support the technical documentation (Annex IV) and quality-management record-keeping; it does not by itself satisfy EU AI Act Article 12, which requires the deployed high-risk system to log events automatically over its lifetime. Every entry shares a common envelope; the `event` field selects one of the rows below. ## Common envelope @@ -161,7 +161,7 @@ webhook ping → audit entry by `run_name` + timestamp. Implementation: | Webhook `event` | Audit-log mirror | Trigger | Gated by | Required payload fields | |---|---|---|---|---| | `training.start` | `training.started` | `train()` entered, before model load. | `webhook.notify_on_start` | `run_name`, `status="started"` | -| `training.success` | `pipeline.completed` | Run completed without revert or pending approval. With `evaluation.auto_revert: true` all gates passed; with the default `auto_revert: false` it also fires when a gate failed but was only recorded (model still promoted). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | +| `training.success` | `pipeline.completed` | Run completed without revert or pending approval. With `evaluation.auto_revert: true` every gate that produced a verdict passed (a safety evaluation with no verdict, `safety.evaluation_completed: false`, keeps the model and also ends here); with the default `auto_revert: false` it also fires when a gate failed but was only recorded (model still promoted). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | | `training.failure` | `pipeline.failed` | Training itself raised (OOM, dataset error, unhandled exception). | `webhook.notify_on_failure` | `run_name`, `status="failed"`, `reason` (masked, ≤2048 chars) | | `training.reverted` | `model.reverted` | A post-training gate (evaluation, safety, judge, benchmark) rejected the run and `_revert_model` deleted the adapters. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `reason` (masked, ≤2048 chars) | | `approval.required` | `human_approval.required` | Run succeeded, `evaluation.require_human_approval=true`, model staged for review (EU AI Act Art. 14). | `webhook.notify_on_success` | `run_name`, `status="awaiting_approval"`, `model_path` | diff --git a/docs/reference/iso27001_control_mapping-tr.md b/docs/reference/iso27001_control_mapping-tr.md index 57e78d96..9fb5ca31 100644 --- a/docs/reference/iso27001_control_mapping-tr.md +++ b/docs/reference/iso27001_control_mapping-tr.md @@ -23,7 +23,7 @@ |---|---|---| | A.5.1 Bilgi güvenliği için politikalar | FL-helps | `audit_event_catalog.md` neyin loglandığını dokümante eder | | A.5.2 Bilgi güvenliği rolleri ve sorumlulukları | FL-helps | `FORGELM_OPERATOR` + `roles_responsibilities.md` | -| A.5.3 Görev ayrılığı | FL-helps | `human_approval.required/granted` trainer ≠ approver'ı zorlar | +| A.5.3 Görev ayrılığı | FL-helps | `human_approval.required/granted` onaylayıcı kimliğini (`FORGELM_OPERATOR`'dan, yoksa OS kullanıcısından) **kaydeder**, böylece eğitici ≠ onaylayıcı denetlenebilir; ForgeLM kendisi self-approval'ı reddetmez. Runner ve reviewer kimliklerini kimlik sağlayıcınızda ayrık tutun | | A.5.4 Yönetim sorumlulukları | FL-helps | `forgelm doctor`; `compliance_report.json`; `training_manifest.yaml` | | A.5.5 Yetkililerle iletişim | OOS | — | | A.5.6 Özel ilgi gruplarıyla iletişim | OOS | — | @@ -52,7 +52,7 @@ | A.5.29 Aksaklık sırasında bilgi güvenliği | FL-helps | `auto_revert` baseline-flip; `model.reverted` olayı | | A.5.30 İş sürekliliği için ICT hazırlığı | OOS | — | | A.5.31 Yasal, kanuni, düzenleyici ve sözleşmesel gerekliliklerin tanımlanması | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | -| A.5.32 Fikri mülkiyet hakları | FL-helps | SBOM'da lisans çıkarımı; HF model-card metadata | +| A.5.32 Fikri mülkiyet hakları | FL-helps | HF model-card metadata (SBOM lisans verisi taşımaz) | | A.5.33 Kayıtların korunması | FL | Append-only + HMAC + manifest sidecar | | A.5.34 Gizlilik ve PII korunması | FL | `forgelm reverse-pii` Madde 15; `forgelm purge` Madde 17 (ayrıca bkz. A.8.3) | | A.5.35 Bilgi güvenliğinin bağımsız incelenmesi | OOS | — | diff --git a/docs/reference/iso27001_control_mapping.md b/docs/reference/iso27001_control_mapping.md index 5f1885a2..442c0617 100644 --- a/docs/reference/iso27001_control_mapping.md +++ b/docs/reference/iso27001_control_mapping.md @@ -24,7 +24,7 @@ |---|---|---| | A.5.1 Policies for information security | FL-helps | `audit_event_catalog.md` documents what's logged | | A.5.2 Information security roles and responsibilities | FL-helps | `FORGELM_OPERATOR` + `roles_responsibilities.md` | -| A.5.3 Segregation of duties | FL-helps | `human_approval.required/granted` enforces trainer ≠ approver | +| A.5.3 Segregation of duties | FL-helps | `human_approval.required/granted` **records** the approver identity (from `FORGELM_OPERATOR`, else the OS user) so trainer ≠ approver is auditable; ForgeLM does not reject self-approval. Keep the runner and reviewer identities disjoint at your identity provider | | A.5.4 Management responsibilities | FL-helps | `forgelm doctor`; `compliance_report.json`; `training_manifest.yaml` | | A.5.5 Contact with authorities | OOS | — | | A.5.6 Contact with special interest groups | OOS | — | @@ -53,7 +53,7 @@ | A.5.29 Information security during disruption | FL-helps | `auto_revert` baseline-flip; `model.reverted` event | | A.5.30 ICT readiness for business continuity | OOS | — | | A.5.31 Identification of legal, statutory, regulatory and contractual requirements | FL-helps | EU AI Act + GDPR mappings; Annex IV bundle | -| A.5.32 Intellectual property rights | FL-helps | License extraction in SBOM; HF model-card metadata | +| A.5.32 Intellectual property rights | FL-helps | HF model-card metadata (the SBOM carries no licence data) | | A.5.33 Protection of records | FL | Append-only + HMAC + manifest sidecar | | A.5.34 Privacy and protection of PII | FL | `forgelm reverse-pii` Article 15; `forgelm purge` Article 17 (see also A.8.3) | | A.5.35 Independent review of information security | OOS | — | diff --git a/docs/reference/soc2_trust_criteria_mapping-tr.md b/docs/reference/soc2_trust_criteria_mapping-tr.md index 07c43f47..be491d23 100644 --- a/docs/reference/soc2_trust_criteria_mapping-tr.md +++ b/docs/reference/soc2_trust_criteria_mapping-tr.md @@ -26,7 +26,7 @@ kategoriler engagement-bazında scoplanır. | CC | Başlık | ForgeLM kanıtı | |---|---|---| | CC1.1 | Bütünlüğe bağlılık göstergesi | `roles_responsibilities.md` QMS template; AI Officer rolü | -| CC1.2 | Yönetişim bağımsızlığı | Approval gate operator ≠ trainer | +| CC1.2 | Yönetişim bağımsızlığı | Approval gate onaylayıcı kimliğini kaydeder, böylece eğitici ≠ onaylayıcı denetlenebilir; ForgeLM bunu zorlamaz | | CC1.3 | Yapı, raporlama hatları kurma | `roles_responsibilities.md` AI Officer / ML Lead / DPO tanımlar | | CC1.4 | Yetkinliğe bağlılık göstergesi | Yıllık EU AI Act / GDPR eğitimi (operatör politikası) | | CC1.5 | Hesap verebilirliği uygular | `FORGELM_OPERATOR` attribution + audit chain | @@ -57,7 +57,7 @@ kategoriler engagement-bazında scoplanır. | CC7.5 | Düzeltici eylemleri tanımlar, geliştirir | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Değişimleri yetkilendirir | `forgelm approve` Madde 14 gate; staging dir | | CC9.1 | Riskleri tanımlar, yönetir | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | -| CC9.2 | Tedarikçi + iş ortağı riskini yönetir | SBOM; `loaded` / `unverified` / `local_path` / `unresolved` olarak derecelendirilen dataset Hub commit SHA'sı; temel-model pin'i (`model.revision` → `model_lineage.base_model_revision`); lisans çıkarımı. `model_lineage.component_revisions` altında **sabitlenen ve kaydedilen:** güvenlik sınıflandırıcısı, LLM judge, GRPO ödül modeli, sentetik teacher. **Kapsanmayan:** merge kaynakları ve `forgelm safety-eval` (`--config` almaz) | +| CC9.2 | Tedarikçi + iş ortağı riskini yönetir | SBOM; `loaded` / `unverified` / `local_path` / `unresolved` olarak derecelendirilen dataset Hub commit SHA'sı; temel-model pin'i (`model.revision` → `model_lineage.base_model_revision`). `model_lineage.component_revisions` altında **sabitlenen ve kaydedilen:** güvenlik sınıflandırıcısı, LLM judge, GRPO ödül modeli, sentetik teacher. **Kapsanmayan:** merge kaynakları ve `forgelm safety-eval` (`--config` almaz) | ## Availability (A1.x) diff --git a/docs/reference/soc2_trust_criteria_mapping.md b/docs/reference/soc2_trust_criteria_mapping.md index 57ce25ed..e2f0d09e 100644 --- a/docs/reference/soc2_trust_criteria_mapping.md +++ b/docs/reference/soc2_trust_criteria_mapping.md @@ -26,7 +26,7 @@ categories are scoped per-engagement. | CC | Title | ForgeLM evidence | |---|---|---| | CC1.1 | Demonstrates commitment to integrity | `roles_responsibilities.md` QMS template; AI Officer role | -| CC1.2 | Independence of governance | Approval gate operator ≠ trainer | +| CC1.2 | Independence of governance | The approval gate records the approver identity, so trainer ≠ approver is auditable; ForgeLM does not enforce it | | CC1.3 | Establishes structures, reporting lines | `roles_responsibilities.md` defines AI Officer / ML Lead / DPO | | CC1.4 | Demonstrates commitment to competence | Annual EU AI Act / GDPR training (deployer policy) | | CC1.5 | Enforces accountability | `FORGELM_OPERATOR` attribution + audit chain | @@ -57,7 +57,7 @@ categories are scoped per-engagement. | CC7.5 | Identifies, develops corrective actions | `human_approval.rejected`; `sop_change_management.md` | | CC8.1 | Authorises changes | `forgelm approve` Article 14 gate; staging dir | | CC9.1 | Identifies, manages risks | `risk_assessment` config + safety eval; `risk_treatment_plan.md` | -| CC9.2 | Manages vendor + business-partner risk | SBOM; dataset Hub commit SHA graded `loaded` / `unverified` / `local_path` / `unresolved`; base-model pin (`model.revision` → `model_lineage.base_model_revision`); license extraction. **Pinned and recorded** under `model_lineage.component_revisions`: safety classifier, LLM judge, GRPO reward model, synthetic teacher. **Not covered:** merge sources, and `forgelm safety-eval` (takes no `--config`) | +| CC9.2 | Manages vendor + business-partner risk | SBOM; dataset Hub commit SHA graded `loaded` / `unverified` / `local_path` / `unresolved`; base-model pin (`model.revision` → `model_lineage.base_model_revision`). **Pinned and recorded** under `model_lineage.component_revisions`: safety classifier, LLM judge, GRPO reward model, synthetic teacher. **Not covered:** merge sources, and `forgelm safety-eval` (takes no `--config`) | ## Availability (A1.x) diff --git a/docs/reference/supply_chain_security-tr.md b/docs/reference/supply_chain_security-tr.md index a81303ae..bcc9d9fa 100644 --- a/docs/reference/supply_chain_security-tr.md +++ b/docs/reference/supply_chain_security-tr.md @@ -39,7 +39,17 @@ içinde çalışır: ``` Cross-OS test matrisinin her (OS × Python-version) hücresi için bir -SBOM. Artefaktlar GitHub release sayfasından indirilebilir. +SBOM. + +SBOM'lar, release tag'i için çalışan `publish.yml` çalıştırmasının **workflow-run artifact**'larıdır. +GitHub Release sayfasına eklenmezler ve GitHub, deponun saklama süresi (90 gün) dolunca run +artifact'larını siler; ihtiyacınız olanı çalıştırma hâlâ duruyorken indirin ve kayıtlarınızla +birlikte arşivleyin. + +Bir SBOM'un listelediği: matris runner'ında paketlenmiş wheel kurulduktan sonraki `pip list` +çıktısı (bağımlılıkları o anda PyPI'dan çözülür, sabitlenmemiştir), Linux'ta `qlora`, `ingestion` +ve `eval` extras'ı ve `pytest` / `pytest-cov`. Bu yüzden runtime kapanışının yanında test araçlarını +da adlandırır, bağımlılık grafiği taşımaz ve **lisans verisi taşımaz**. ### Determinism contract @@ -52,17 +62,21 @@ Wave 4 / Faz 23 bir determinism testi ekler - `tests/test_supply_chain_security.py::TestGenerateSbomDeterministic::test_two_invocations_produce_same_content` CI'da bunu pin'ler. -Bir denetçi karşılık gelen `git tag`'den SBOM'u yeniden emit edip -GitHub release'e ekli artefaktla diff alabilir; trivial olmayan -herhangi bir fark build ortamında dependency-resolution drift'ine -işaret eder. +Determinizm, aynı ortama özgü bir özelliktir. Aynı `git tag`'den sonradan yapılan yeniden +emit'in bir release'in SBOM'uyla eşleşmesi beklenmez; çünkü o release bağımlılıklarını +build edildiği anda PyPI'dan çözdü. **Sizin** dağıttığınızı kaydetmek için, modeli eğiten ya da +sunan ortamda bir ForgeLM checkout'undan `python3 tools/generate_sbom.py` çalıştırın. ### SBOM'u tüketme ```bash -# Bir release için tüm SBOM'ları indir. v'yi CI'da pinlediğiniz -# tag ile değiştirin; SBOM artefakt formatı release'ler arasında stabildir. -gh release download v --pattern 'sbom-*' +# CI'da pinlediğiniz tag için başarılı publish çalıştırmasını bulun (bir tag'in +# daha önce başarısız olmuş bir çalıştırması olabilir), sonra ondan bir SBOM indirin. +# v'yi tag ile değiştirin. Run artifact'ları 90 günde silinir; on ikisi için +# --pattern 'sbom-*' kullanın. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 # İnsan inceleme için pretty-print. jq . sbom-ubuntu-latest-py3.11.json | less @@ -110,7 +124,7 @@ Wave 4 / Faz 23 nightly workflow'a `pip-audit` ekler. Davranış: Belirli bir CVE'yi kabul etmek için opt-in ignore dosyasıyla dokümante edin (aşağıdaki Suppression'a bakın); yeşil kalmak için UNKNOWN kovasına güvenmeyin. -- OSV / GHSA veritabanlarını kullanır (pip-audit varsayılanı). +- PyPI'ın güvenlik açığı verisini sorgular (pip-audit'in varsayılan `pypi` servisi; nightly `--vulnerability-service` geçmez). Operatörler aynı tooling'i lokalde kurar: diff --git a/docs/reference/supply_chain_security.md b/docs/reference/supply_chain_security.md index e070e770..6aba4546 100644 --- a/docs/reference/supply_chain_security.md +++ b/docs/reference/supply_chain_security.md @@ -39,7 +39,16 @@ publish workflow: ``` One SBOM per (OS × Python-version) cell of the cross-OS test matrix. -The artefacts are downloadable from the GitHub release page. + +The SBOMs are **workflow-run artifacts** of the `publish.yml` run for the release tag. They are +not attached to the GitHub Release page, and GitHub deletes run artifacts when the repository's +retention period ends (90 days), so download what you need while the run is still available and +archive it with your records. + +What an SBOM lists: the output of `pip list` on the matrix runner after it installed the packaged +wheel (its dependencies resolve from PyPI at that moment, unpinned), on Linux the `qlora`, +`ingestion` and `eval` extras, and `pytest` / `pytest-cov`. It therefore names test tooling +beside the runtime closure, carries no dependency graph, and carries **no licence data**. ### Determinism contract @@ -52,17 +61,20 @@ Wave 4 / Faz 23 adds a determinism test (`tests/test_supply_chain_security.py`): - `tests/test_supply_chain_security.py::TestGenerateSbomDeterministic::test_two_invocations_produce_same_content` pins this in CI. -An auditor can re-emit the SBOM from the corresponding `git tag` -and diff against the artefact attached to the GitHub release; any -non-trivial difference indicates dependency-resolution drift in the -build environment. +Determinism is a same-environment property. A later re-emission from the same `git tag` is not +expected to match a release's SBOM, because that release resolved its dependencies from PyPI when +it was built. To record what **you** deploy, run `python3 tools/generate_sbom.py` from a ForgeLM +checkout in the environment that trains or serves the model. ### Consuming the SBOM ```bash -# Download all SBOMs for a release. Replace v with the tag you -# pinned in your CI; the SBOM artefact format is stable across releases. -gh release download v --pattern 'sbom-*' +# Find the successful publish run for the tag you pinned in your CI (a tag can have an +# earlier failed run), then download one SBOM from it. Replace v with the tag. +# Run artifacts expire after 90 days; use --pattern 'sbom-*' for all twelve. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --name sbom-ubuntu-latest-py3.11 # Pretty-print for human review. jq . sbom-ubuntu-latest-py3.11.json | less @@ -108,7 +120,7 @@ Wave 4 / Faz 23 adds `pip-audit` to the nightly workflow. Behaviour: To accept a specific CVE, document it via the opt-in ignore file (see Suppression below); do not rely on the UNKNOWN bucket to stay green. -- Uses the OSV / GHSA databases (pip-audit's default). +- Queries PyPI's vulnerability data (pip-audit's default `pypi` service; the nightly does not pass `--vulnerability-service`). Operators install the same tooling locally: diff --git a/docs/standards/error-handling.md b/docs/standards/error-handling.md index 1a8860f5..97a645b6 100644 --- a/docs/standards/error-handling.md +++ b/docs/standards/error-handling.md @@ -19,7 +19,7 @@ EXIT_INTEGRITY_FAILURE = 6 | Code | When | Who reads it | |---|---|---| -| **0** | Training completed. With `auto_revert: true` (the compliance default for high-risk tiers) this also means every gate passed; with the **shipped default** `auto_revert: false` a failed benchmark/safety/judge gate is *recorded* (`benchmark`/`safety`/`judge` block in the JSON, `*_passed: false`) but does **not** change the exit code — parse those blocks, don't trust exit 0 alone | CI/CD success | +| **0** | Training completed. With `auto_revert: true` (the compliance default for high-risk tiers) this also means every gate that ran to a verdict passed (a safety evaluation that produced no verdict — `safety.evaluation_completed: false` — keeps the model and exits 0 regardless of `auto_revert`); with the **shipped default** `auto_revert: false` a failed benchmark/safety/judge gate is *recorded* (`benchmark`/`safety`/`judge` block in the JSON, `*_passed: false`) but does **not** change the exit code — parse those blocks, don't trust exit 0 alone | CI/CD success | | **1** | Config validation failed (YAML schema, Pydantic error). Also used by the `verify-*` subcommands for an operator-actionable input error — the artefact could not be read at all, so nothing was ever compared (plus the one case where everything comparable *was* compared and came out clean; see the 1-vs-6 note below) | CI/CD "fail fast"; user fixes YAML or the verifier's target path | | **2** | Training crashed or failed mid-run (OOM, CUDA error, unhandled exception). Also the `verify-*` subcommands' code for a genuine runtime I/O failure on a reachable path (permission denied mid-read, disk error) | CI/CD retry logic | | **3** | Training completed but eval/safety/benchmark threshold failed, and auto-revert happened | CI/CD decision: do not deploy | diff --git a/docs/usermanuals/en/compliance/audit-log.md b/docs/usermanuals/en/compliance/audit-log.md index faddaaa7..09125004 100644 --- a/docs/usermanuals/en/compliance/audit-log.md +++ b/docs/usermanuals/en/compliance/audit-log.md @@ -5,7 +5,7 @@ description: Append-only event log over training, evaluation, and revert decisio # Audit Log -EU AI Act Article 12 requires high-risk AI systems to maintain logs of operationally relevant events. ForgeLM's `audit_log.jsonl` is an append-only, SHA-256-anchored sequence of events covering training start, evaluation gates, auto-revert decisions, and model export. +EU AI Act Article 12 requires high-risk AI systems to maintain logs of operationally relevant events. ForgeLM's `audit_log.jsonl` is an append-only, SHA-256-anchored sequence of events covering training start, evaluation gates, auto-revert decisions, and model export. It records the training pipeline, which is development-time evidence that can support your technical documentation; the logging Article 12 asks for is a capability of the deployed system, which ForgeLM does not provide. ## Format diff --git a/docs/usermanuals/en/compliance/gdpr.md b/docs/usermanuals/en/compliance/gdpr.md index d8783dbe..fa2f8e6d 100644 --- a/docs/usermanuals/en/compliance/gdpr.md +++ b/docs/usermanuals/en/compliance/gdpr.md @@ -5,7 +5,7 @@ description: Data protection compliance — PII minimisation, data subject right # GDPR / KVKK -GDPR (Europe) and KVKK (Türkiye) impose data protection requirements on personal data used in training. ForgeLM's role: prevent personal data from entering training in the first place, document what gets ingested, and produce evidence for data subject requests. +GDPR (Europe) and KVKK (Türkiye) impose data protection requirements on personal data used in training. ForgeLM's role is narrower than keeping personal data out of training: it masks the identifier formats it can detect, reports what an audit found, and gives you commands for access and erasure requests. It does not decide your lawful basis, it does not find names or other free-text personal data, and it does not make a trained model anonymous. ## What GDPR / KVKK require for training data @@ -25,40 +25,35 @@ KVKK (Türkiye) mirrors these principles closely. ### Data minimisation (Art. 5(1)(c)) -PII masking at ingest replaces personal identifiers with placeholders: +PII masking at ingest replaces the identifier formats ForgeLM detects with a placeholder: -```yaml -ingestion: - pii_mask: - enabled: true - locale: "tr" - categories: ["email", "phone", "iban", "id_tr"] +```shell +$ forgelm ingest ./policies/ --recursive --strategy markdown --pii-mask --output data/policies.jsonl ``` -By the time data lands in JSONL, identifiable subjects are removed. See [PII Masking](#/data/pii-masking). +The detectors cover email, phone, credit card, IBAN, and the Turkish, German, French and US national-ID formats; every match becomes `[REDACTED]`. **Names, addresses, dates of birth, health data and other free text are not detected**, and the patterns are tuned to miss a borderline match rather than redact ordinary prose. Masking reduces the personal data in the JSONL; it does not remove identifiable subjects from it. See [PII Masking](#/data/pii-masking). ### Accountability (Art. 5(2)) -Every audit produces `data_audit_report.json` documenting: +Every `forgelm audit` run writes `data_audit_report.json`. It records: -- Detected PII categories and counts (before masking). -- Source attribution (which document each row came from). -- Quality and language distribution. -- SHA-256 manifest for tamper-evidence. +- Detected PII and secret categories with their counts, for the data you audited. Audit the unmasked source to see what a masker has to catch; audit the masked output to see what got through. +- Near-duplicate, quality and cross-split-overlap summaries, and an optional Croissant card. +- The audit's own metadata: when it ran, and the path and literal input it was given. -These reports flow into the Annex IV bundle. When a regulator asks "what personal data was in your training set?", you have a structured answer. +The report does not attribute rows to source documents, carries no hash manifest, and is a plain JSON file: it is not itself written to the audit log. Erasure and access requests (`forgelm purge`, `forgelm reverse-pii`) are, in the hash-chained log that `forgelm verify-audit` checks. When the report sits in `training.output_dir`, it feeds the Article 10 data-governance section of the Annex IV bundle, so a regulator's "what personal data was in your training set?" has a structured answer: what the detectors found, not a statement that none exists. ### Storage limitation (Art. 5(1)(e)) -ForgeLM doesn't retain raw user data — it produces JSONL artifacts you control. For automated retention enforcement: +ForgeLM doesn't retain raw user data — it produces JSONL artifacts you control. Retention horizons can be declared and are **reported, not enforced**: ```yaml retention: - raw_documents_retention_days: 90 # auto-delete originals after N days + raw_documents_retention_days: 90 ephemeral_artefact_retention_days: 365 ``` -(The actual deletion is your storage layer's responsibility; ForgeLM just records the intended TTL in the audit log.) +`forgelm purge --check-policy --config config.yaml` lists artefacts that are past their horizon and always exits 0. Nothing is deleted automatically and nothing is written to the audit log for it; deleting what is overdue is your storage layer's job. (`retention.enforce` is reserved: no command acts on it yet.) ## Data subject requests @@ -68,30 +63,29 @@ Most-common request types and how ForgeLM helps: "What personal data do you hold about me?" -Run reverse-PII on your training data: +Search your training data for the person's identifier: ```shell $ forgelm reverse-pii --query "ali@example.com" data/*.jsonl -No matches found in masked data. ``` -Because PII was masked at ingest, no specific person's data is recoverable from the model. The audit report confirms this. +The scan reports the lines where the identifier appears; zero matches is a valid answer. It finds what is literally in the files you point it at. Masking only removes the identifier formats it detects, so a name or an undetected variant can remain, and the raw source documents are not covered by a scan of the masked JSONL: search those separately. ForgeLM cannot tell you what a trained model remembers. ### Right to erasure (Art. 17) "Delete my data." If a person's data was in your training set: -1. The masked JSONL doesn't contain their identifying info — already minimised. -2. Source documents may still — drop them from your raw store and re-ingest if necessary. +1. Find the rows with `forgelm reverse-pii`, then remove them with `forgelm purge --row-id --corpus `; every erasure is written to the audit log. +2. Source documents may still hold it — drop them from your raw store and re-ingest if necessary. 3. The model may have memorised some details — see "model-level erasure" below. ### Model-level erasure LLMs can memorise rare strings from training. Even with PII masking, removing all traces from a deployed model is hard. ForgeLM's defences: -- **Prevent memorisation:** PII masking, deduplication (memorised data is usually duplicated data). -- **Detect memorisation:** the audit step flags rows that overlap with known PII patterns. +- **Reduce what goes in:** PII masking and deduplication (memorised data is usually duplicated data). +- **Memorisation is not measured:** ForgeLM does not test whether a trained model reproduces a given string. - **Re-train as last resort:** if a specific subject's data leaked despite masking, re-train without that source. For zero-tolerance scenarios (medical, legal), pair PII masking with manual review before training. @@ -122,7 +116,7 @@ ForgeLM has no YAML surface for GDPR legal metadata (lawful basis, data controll ::: :::warn -**International transfers.** If your training data crosses jurisdiction boundaries (Turkish data trained in EU, EU data trained in US), additional safeguards apply. Set `international_transfers.enabled: true` and document the safeguards. +**International transfers.** If your training data crosses jurisdiction boundaries (Turkish data trained in EU, EU data trained in US), additional safeguards apply. ForgeLM has no setting for transfers; record the safeguards in your DPIA. ::: :::tip diff --git a/docs/usermanuals/en/compliance/human-oversight.md b/docs/usermanuals/en/compliance/human-oversight.md index 61e05a72..48fa2f2a 100644 --- a/docs/usermanuals/en/compliance/human-oversight.md +++ b/docs/usermanuals/en/compliance/human-oversight.md @@ -5,7 +5,7 @@ description: Block model promotion until a human reviews and signs off — Artic # Human Oversight -EU AI Act Article 14 requires high-risk AI systems to provide for human oversight. ForgeLM implements this as an optional config gate: when `evaluation.require_human_approval: true`, model promotion blocks until a human signs an approval. +EU AI Act Article 14 requires high-risk AI systems to provide for human oversight. ForgeLM offers an optional pre-promotion approval gate (`evaluation.require_human_approval: true`) that blocks model promotion until a human signs an approval and records that sign-off. This supports design-time oversight documentation; it does not implement the in-use oversight measures Article 14 requires of the deployed system (monitoring, interpreting, overriding or stopping it), which remain the deployer's. ## How the gate works diff --git a/docs/usermanuals/en/operations/iso-soc2-deployer.md b/docs/usermanuals/en/operations/iso-soc2-deployer.md index 862c2748..d53546d6 100644 --- a/docs/usermanuals/en/operations/iso-soc2-deployer.md +++ b/docs/usermanuals/en/operations/iso-soc2-deployer.md @@ -14,16 +14,16 @@ Four pillars an ISO / SOC 2 auditor cares about most have direct ForgeLM evidenc 1. **Audit trail** — `audit_log.jsonl` per training run, append-only with HMAC + SHA-256 hash chain + genesis manifest sidecar. `forgelm verify-audit` validates the chain end-to-end. 2. **Change control** — Article 14 staging gate (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit events + `config_hash` (per-run manifest sidecar field) stamped per run. 3. **Data lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime, plus the dataset's HF Hub commit SHA **graded by `hf_revision_source`**) + `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). Read the grade before the SHA: only `hf_revision_source: loaded` is evidence of the corpus that trained the model. -4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release, `pip-audit` nightly, `bandit` CI for static + dynamic security scanning. +4. **Supply chain** — CycloneDX 1.5 SBOM emitted per release (a workflow-run artifact kept 90 days, not a GitHub release asset), `pip-audit` nightly, `bandit` CI for static + dynamic security scanning. ## The eight audit-floor questions The deep guide answers each with the exact `jq` / CLI command + the artefact it returns: 1. "Show me the audit trail for every model promotion in the past 90 days" → `forgelm verify-audit` + `jq 'select(.event == "human_approval.granted")'`. -2. "Show me the change-control evidence — who approved this model?" → cross-reference `training.started` + `human_approval.granted` events; two distinct operator IDs prove segregation of duties (ISO A.5.3, SOC 2 CC1.5). +2. "Show me the change-control evidence — who approved this model?" → cross-reference `training.started` + `human_approval.granted` events; compare the two identities; they are self-asserted (`FORGELM_OPERATOR`, else the OS user), so segregation of duties (ISO A.5.3, SOC 2 CC1.5) holds only if your identity provider makes them different people. 3. "Show me the data lineage" → `data_provenance.json`; `sha256` + `hf_revision` pin the corpus — but only when `hf_revision_source` reads `loaded`. `unverified` means the SHA is a Hub lookup made when the bundle was written, with no coupling to the load; `unresolved` means no SHA was obtained and none is claimed. -4. "Show me the supply chain" → `gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*'`; CycloneDX 1.5 JSON. +4. "Show me the supply chain" → `gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success`, then `gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*'` within 90 days; CycloneDX 1.5 JSON (the CI runner's package list, test tooling included, no licence data). 5. "Show me the access controls" → IdP audit log + `FORGELM_OPERATOR` cross-reference. 6. "Show me the encryption posture" → deployer-side substrate (KMS audit log + `data_governance_report.json`). 7. "Show me the incident response" → `audit.classifier_load_failed` + F-compliance-110 strict gate. diff --git a/docs/usermanuals/en/operations/supply-chain.md b/docs/usermanuals/en/operations/supply-chain.md index 7ec468ae..47c50c00 100644 --- a/docs/usermanuals/en/operations/supply-chain.md +++ b/docs/usermanuals/en/operations/supply-chain.md @@ -5,21 +5,24 @@ description: Operator surface for ForgeLM's SBOM + pip-audit + bandit pipeline # Supply Chain -ForgeLM emits a CycloneDX 1.5 SBOM per release tag, and runs `pip-audit` + `bandit` nightly via the `nightly.yml` workflow (also re-run on every release-tag publish). This page is the operator's mental model: what runs when, where to get the artefacts, and how to mirror the same checks locally. +ForgeLM emits a CycloneDX 1.5 SBOM per release tag (as a workflow-run artifact, not a GitHub release asset), and runs `pip-audit` + `bandit` nightly via the `nightly.yml` workflow (also re-run on every release-tag publish). This page is the operator's mental model: what runs when, where to get the artefacts, and how to mirror the same checks locally. ## What runs when | Trigger | Tool | Result | Failure policy | |---|---|---|---| -| Release tag (`v*`) | `tools/generate_sbom.py` | One CycloneDX 1.5 JSON per (OS × Python-version) cell, attached to the GitHub release | SBOM step is pure-stdlib; cannot silently degrade a green release matrix | -| Nightly 03:00 UTC | `pip-audit` (via `tools/check_pip_audit.py`) | OSV / GHSA scan against installed deps | HIGH/CRITICAL → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → silent | +| Release tag (`v*`) | `tools/generate_sbom.py` | One CycloneDX 1.5 JSON per (OS × Python-version) cell, uploaded as a workflow-run artifact of the tag's publish run (kept 90 days; not attached to the GitHub release) | SBOM step is pure-stdlib; cannot silently degrade a green release matrix | +| Nightly 03:00 UTC | `pip-audit` (via `tools/check_pip_audit.py`) | PyPI vulnerability-data scan against installed deps | HIGH/CRITICAL or UNKNOWN severity → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → silent | | Nightly 03:00 UTC + release tag | `bandit` (via `tools/check_bandit.py`) | Static security scan of `forgelm/` (excludes `tests/`) | HIGH → exit 1; MEDIUM → `::warning::`; LOW → silent | ## Where to get the SBOM ```bash -# All SBOMs for a release. -gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*' +# Find the successful publish run for a tag, then download its SBOMs +# (one directory per artifact). Run artifacts expire after 90 days. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*' # Pretty-print one. jq . sbom-ubuntu-latest-py3.11.json | less @@ -34,7 +37,7 @@ curl -X POST -H "X-Api-Key: $DT_KEY" -H "Content-Type: application/octet-stream" https://deptrack.example.com/api/v1/bom ``` -The SBOM is determinism-pinned — two consecutive emissions on the same Python environment produce content-identical JSON (modulo `serialNumber` and `metadata.timestamp`, both intentionally varying per CycloneDX semantics). +The SBOM is determinism-pinned — two consecutive emissions on the same Python environment produce content-identical JSON (modulo `serialNumber` and `metadata.timestamp`, both intentionally varying per CycloneDX semantics). That is a same-environment property: a release SBOM is the CI runner's `pip list` (test tooling such as `pytest` included, no dependency graph, no licence data), so re-emitting it later from the same tag will not reproduce it. To describe your own environment, run `python3 tools/generate_sbom.py` from a ForgeLM checkout there. ## Mirror the nightly check locally @@ -46,7 +49,7 @@ pip-audit --strict --format json --output /tmp/pip-audit.json python3 tools/check_pip_audit.py /tmp/pip-audit.json ``` -Exit 0 means the same severity policy ForgeLM CI applies passed. Exit 1 means a HIGH/CRITICAL vulnerability needs addressing before the nightly fires. +Exit 0 means the same severity policy ForgeLM CI applies passed. Exit 1 means a HIGH/CRITICAL vulnerability, or a finding whose severity pip-audit does not report (UNKNOWN, which is most of them), needs addressing before the nightly fires. ## Mirror the bandit check locally diff --git a/docs/usermanuals/en/reference/exit-codes.md b/docs/usermanuals/en/reference/exit-codes.md index 8780f19e..e462872d 100644 --- a/docs/usermanuals/en/reference/exit-codes.md +++ b/docs/usermanuals/en/reference/exit-codes.md @@ -11,7 +11,7 @@ ForgeLM's exit codes are a public contract. CI/CD pipelines, schedulers, and das | Exit | Constant | Meaning | Typical CI action | |---|---|---|---| -| **0** | `EXIT_SUCCESS` | Run completed and the checkpoint was promoted. With `evaluation.auto_revert: true` every gate also passed; with the shipped default `auto_revert: false` a failed benchmark/safety/judge gate is **recorded in the JSON output but does not change the exit code** — see ["What exit 0 actually guarantees"](#what-exit-0-actually-guarantees) below. | Continue pipeline (parse gate blocks if `auto_revert` is off) | +| **0** | `EXIT_SUCCESS` | Run completed and the checkpoint was promoted. With `evaluation.auto_revert: true` every gate that produced a verdict also passed (a safety evaluation with no verdict keeps the model and exits 0 too); with the shipped default `auto_revert: false` a failed benchmark/safety/judge gate is **recorded in the JSON output but does not change the exit code** — see ["What exit 0 actually guarantees"](#what-exit-0-actually-guarantees) below. | Continue pipeline (parse gate blocks if `auto_revert` is off) | | **1** | `EXIT_CONFIG_ERROR` | YAML invalid, file missing, env var unset, or argument malformed. | Fail fast | | **2** | `EXIT_TRAINING_ERROR` | Training-time runtime error (any unhandled exception that isn't a config or eval-gate failure: data load, OOM, NaN loss, I/O failure, mid-stream audit-iteration OSError). | Investigate; surface logs | | **3** | `EXIT_EVAL_FAILURE` | A benchmark/safety/judge gate failed **and** the model was auto-reverted (requires `evaluation.auto_revert: true`). With `auto_revert: false` a failed gate does not produce exit 3 — the run exits 0 with the failure recorded in the JSON gate blocks. | Investigate; do NOT promote | @@ -112,7 +112,7 @@ A run that exits 0 has: **Gates and exit 0.** Whether a *passed* benchmark/safety/judge gate is part of the exit-0 guarantee depends on `evaluation.auto_revert`: -- With `evaluation.auto_revert: true` (the EU AI Act high-risk default), a failed gate auto-reverts the model and exits **3** — so exit 0 *does* mean every configured gate passed. +- With `evaluation.auto_revert: true` (the EU AI Act high-risk default), a failed gate auto-reverts the model and exits **3** — so exit 0 means every configured gate that produced a verdict passed. **One exception:** a safety evaluation that produced no verdict (`safety.evaluation_completed: false` — the classifier failed to load, the probes file is missing or empty, or most verdicts were unreadable) keeps the model and exits 0 (4 with the approval gate) even with `auto_revert: true`; check `safety.passed` and `safety.evaluation_completed` in the JSON output. - With the shipped default `evaluation.auto_revert: false`, a failed gate is **recorded** (the `benchmark` / `safety` / `judge` block in the JSON output carries `*_passed: false`) but the model is still promoted and the run exits **0**. Read those JSON blocks; do not infer gate success from exit 0 alone. There is no "partial success" exit code by design — turn on `auto_revert` if you want a failing gate to change the exit code. diff --git a/docs/usermanuals/en/reference/json-output.md b/docs/usermanuals/en/reference/json-output.md index 7132bc10..c0ebbabe 100644 --- a/docs/usermanuals/en/reference/json-output.md +++ b/docs/usermanuals/en/reference/json-output.md @@ -178,7 +178,7 @@ When training runs to completion the pipeline emits a result envelope on **stdou | Key | Type | Notes | |---|---|---| -| `success` | bool | `true` when the run completed (including a staged, awaiting-approval run). `false` when a gate auto-reverted the model. | +| `success` | bool | `true` when the run completed (including a staged, awaiting-approval run, and a run whose safety evaluation produced no verdict: check `safety.evaluation_completed`). `false` when a gate auto-reverted the model. | | `error` | string | Present **iff** `success` is `false`, never on the success path. Carries the gate's own reason — which gate fired and against what threshold — because every gate exits `3`, so the exit code alone cannot tell them apart. If a failure reaches the envelope with no reason recorded, the field says so explicitly and points at `audit_log.jsonl` rather than being omitted. | | `metrics` | object | Training/eval/gate metrics (e.g. `eval_loss`, `benchmark/average`, `safety/safe_ratio`). Values are numbers, **except** a measurement that came out non-finite, which is written as the string `"nan"` / `"inf"` / `"-inf"` — JSON has no non-finite number literal, so the alternative was output no strict parser accepts. Type these fields as number-or-string. | | `final_model_path` | str | Where the model artefacts live. For an awaiting-approval run this is the **staging** directory until `forgelm approve` promotes it. | diff --git a/docs/usermanuals/tr/compliance/audit-log.md b/docs/usermanuals/tr/compliance/audit-log.md index 1c435f7e..8f262f99 100644 --- a/docs/usermanuals/tr/compliance/audit-log.md +++ b/docs/usermanuals/tr/compliance/audit-log.md @@ -5,7 +5,7 @@ description: Eğitim, değerlendirme ve geri alma kararlarını append-only even # Audit Log -EU AI Act Madde 12, yüksek-riskli AI sistemlerinin operasyonel olarak ilgili olayların log'unu tutmasını gerektirir. ForgeLM'in `audit_log.jsonl` dosyası; eğitim başlangıcı, eval kapıları, otomatik geri alma kararları ve model export'unu kapsayan, append-only ve SHA-256-anchored bir olay dizisidir. +EU AI Act Madde 12, yüksek-riskli AI sistemlerinin operasyonel olarak ilgili olayların log'unu tutmasını gerektirir. ForgeLM'in `audit_log.jsonl` dosyası; eğitim başlangıcı, eval kapıları, otomatik geri alma kararları ve model export'unu kapsayan, append-only ve SHA-256-anchored bir olay dizisidir. Eğitim hattını kaydeder; bu, teknik dokümantasyonunuzu destekleyebilecek geliştirme-zamanı kanıtıdır. Madde 12'nin istediği loglama dağıtılmış sistemin bir yeteneğidir ve ForgeLM bunu sağlamaz. ## Format diff --git a/docs/usermanuals/tr/compliance/gdpr.md b/docs/usermanuals/tr/compliance/gdpr.md index d8f425a3..21c8f61b 100644 --- a/docs/usermanuals/tr/compliance/gdpr.md +++ b/docs/usermanuals/tr/compliance/gdpr.md @@ -5,7 +5,7 @@ description: Veri koruma uyumluluğu — PII minimizasyonu, veri sahibi hakları # GDPR / KVKK -GDPR (Avrupa) ve KVKK (Türkiye), eğitimde kullanılan kişisel verilere koruma gereklilikleri getirir. ForgeLM'in rolü: kişisel verinin eğitime girmesini en başta engellemek, ne girdiğini belgelemek ve veri sahibi talepleri için kanıt üretmek. +GDPR (Avrupa) ve KVKK (Türkiye), eğitimde kullanılan kişisel verilere koruma gereklilikleri getirir. ForgeLM'in rolü, kişisel veriyi eğitimden uzak tutmaktan daha dardır: tespit edebildiği tanımlayıcı biçimlerini maskeler, bir audit'in bulduğunu raporlar ve erişim ile silme talepleri için size komutlar verir. Hukuki dayanağınızı belirlemez, isimleri ya da diğer serbest-metin kişisel verileri bulmaz ve eğitilmiş bir modeli anonim yapmaz. ## GDPR / KVKK eğitim verisinde ne ister @@ -25,40 +25,35 @@ KVKK (Türkiye) bu ilkeleri yakından yansıtır. ### Veri minimizasyonu (Md. 5(1)(c)) -Ingest'te PII maskeleme kişisel tanımlayıcıları placeholder'larla değiştirir: +Ingest'te PII maskeleme, ForgeLM'in tespit ettiği tanımlayıcı biçimlerini bir placeholder ile değiştirir: -```yaml -ingestion: - pii_mask: - enabled: true - locale: "tr" - categories: ["email", "phone", "iban", "id_tr"] +```shell +$ forgelm ingest ./policies/ --recursive --strategy markdown --pii-mask --output data/policies.jsonl ``` -Veri JSONL'a düştüğünde tanımlanabilir özneler kaldırılmıştır. Bkz. [PII Maskeleme](#/data/pii-masking). +Dedektörler e-posta, telefon, kredi kartı, IBAN ile Türk, Alman, Fransız ve ABD ulusal kimlik biçimlerini kapsar; her eşleşme `[REDACTED]` olur. **İsimler, adresler, doğum tarihleri, sağlık verisi ve diğer serbest metin tespit edilmez**; desenler sıradan düzyazıyı maskelemek yerine sınırda bir eşleşmeyi kaçırmaya ayarlıdır. Maskeleme JSONL'daki kişisel veriyi azaltır; tanımlanabilir özneleri ondan kaldırmaz. Bkz. [PII Maskeleme](#/data/pii-masking). ### Hesap verebilirlik (Md. 5(2)) -Her audit aşağıdakileri belgeleyen `data_audit_report.json` üretir: +Her `forgelm audit` çalıştırması `data_audit_report.json` yazar. İçeriği: -- Tespit edilen PII kategorileri ve sayıları (maskelemeden önce). -- Kaynak atfı (her satırın hangi dokümandan geldiği). -- Kalite ve dil dağılımı. -- Tamper-evidence için SHA-256 manifest. +- Denetlediğiniz veri için tespit edilen PII ve gizli bilgi kategorileri ve sayıları. Bir maskelemenin neyi yakalaması gerektiğini görmek için maskelenmemiş kaynağı, neyin sızdığını görmek için maskelenmiş çıktıyı denetleyin. +- Near-duplicate, kalite ve split-arası örtüşme özetleri ile isteğe bağlı bir Croissant kartı. +- Audit'in kendi üst verisi: ne zaman çalıştığı, ona verilen yol ve girdi. -Bu raporlar Annex IV paketine akar. Regülatör "eğitim setinizde hangi kişisel veriler vardı?" diye sorduğunda yapılandırılmış cevabınız olur. +Rapor satırları kaynak dokümanlara atfetmez, hash manifest taşımaz ve düz bir JSON dosyasıdır: kendisi audit log'a yazılmaz. Silme ve erişim talepleri (`forgelm purge`, `forgelm reverse-pii`) ise `forgelm verify-audit`'in denetlediği hash-zincirli log'a yazılır. Rapor `training.output_dir` içindeyse Annex IV paketinin Madde 10 veri yönetişimi bölümünü besler; böylece regülatörün "eğitim setinizde hangi kişisel veriler vardı?" sorusunun yapılandırılmış bir cevabı olur: dedektörlerin bulduğu, hiç olmadığına dair bir beyan değil. ### Saklama sınırlaması (Md. 5(1)(e)) -ForgeLM ham kullanıcı verisini saklamaz — sizin kontrol ettiğiniz JSONL artifact'lar üretir. Otomatik saklama uygulanması için: +ForgeLM ham kullanıcı verisini saklamaz — sizin kontrol ettiğiniz JSONL artifact'lar üretir. Saklama ufukları tanımlanabilir ve **raporlanır, zorlanmaz**: ```yaml retention: - raw_documents_retention_days: 90 # N gün sonra orijinalleri otomatik sil + raw_documents_retention_days: 90 ephemeral_artefact_retention_days: 365 ``` -(Asıl silme depolama katmanınızın sorumluluğu; ForgeLM sadece amaçlanan TTL'yi audit log'a kaydeder.) +`forgelm purge --check-policy --config config.yaml` ufku aşmış artefaktları listeler ve her zaman 0 ile çıkar. Hiçbir şey otomatik silinmez ve bunun için audit log'a bir şey yazılmaz; süresi geçeni silmek depolama katmanınızın işidir. (`retention.enforce` ayrılmıştır: henüz hiçbir komut onu kullanmaz.) ## Veri sahibi talepleri @@ -68,21 +63,20 @@ En sık talep tipleri ve ForgeLM nasıl yardım eder: "Hakkımda hangi kişisel verileri tutuyorsunuz?" -Eğitim verinizde reverse-PII koşturun: +Kişinin tanımlayıcısını eğitim verinizde arayın: ```shell $ forgelm reverse-pii --query "ali@example.com" data/*.jsonl -Maskelenmiş veride eşleşme bulunamadı. ``` -PII ingest'te maskelendiği için modelden belirli birinin verisi geri çıkarılamaz. Audit raporu bunu teyit eder. +Tarama, tanımlayıcının geçtiği satırları raporlar; sıfır eşleşme geçerli bir cevaptır. Yalnızca işaret ettiğiniz dosyalarda gerçekten bulunanı bulur. Maskeleme yalnızca tespit ettiği tanımlayıcı biçimlerini kaldırır; bir isim ya da tespit edilemeyen bir varyant kalabilir ve maskelenmiş JSONL taraması ham kaynak dokümanları kapsamaz: onları ayrıca arayın. ForgeLM eğitilmiş bir modelin neyi hatırladığını söyleyemez. ### Silme hakkı (Md. 17) "Verilerimi silin." Bir kişinin verisi eğitim setinizdeydi: -1. Maskelenmiş JSONL kimlik bilgilerini içermiyor — zaten minimize edilmiş. +1. Satırları `forgelm reverse-pii` ile bulun, sonra `forgelm purge --row-id --corpus ` ile kaldırın; her silme audit log'a yazılır. 2. Ham kaynak dokümanlar hâlâ içerebilir — ham deponuzdan düşürün ve gerekirse yeniden ingest edin. 3. Model bazı detayları ezberlemiş olabilir — aşağıdaki "model-seviyesi silme"ye bakın. @@ -90,8 +84,8 @@ Bir kişinin verisi eğitim setinizdeydi: LLM'ler eğitimden nadir string'leri ezberleyebilir. PII maskelemeyle bile deploy edilmiş bir modelden tüm izleri kaldırmak zordur. ForgeLM'in savunmaları: -- **Ezberi önle:** PII maskeleme, deduplikasyon (ezberlenen veri genelde tekrarlanan veridir). -- **Ezberi tespit et:** Audit aşaması bilinen PII pattern'leriyle örtüşen satırları flagler. +- **Girenleri azalt:** PII maskeleme ve deduplikasyon (ezberlenen veri genelde tekrarlanan veridir). +- **Ezber ölçülmez:** ForgeLM, eğitilmiş bir modelin belirli bir string'i üretip üretmediğini test etmez. - **Son çare yeniden eğitim:** Maskelemeye rağmen belirli bir öznenin verisi sızdıysa, o kaynak olmadan yeniden eğitin. Sıfır-tolerans senaryolarda (sağlık, hukuk) PII maskelemeyi eğitimden önce manuel inceleme ile birleştirin. @@ -122,7 +116,7 @@ ForgeLM'in GDPR hukuki meta verisi (hukuki dayanak, veri sorumlusu vb.) için YA ::: :::warn -**Uluslararası transferler.** Eğitim veriniz yargı sınırlarını aşıyorsa (Türk verisi AB'de eğitiliyor, AB verisi ABD'de eğitiliyor), ek koruma önlemleri uygulanır. `international_transfers.enabled: true` ayarlayın ve koruma önlemlerini belgeleyin. +**Uluslararası transferler.** Eğitim veriniz yargı sınırlarını aşıyorsa (Türk verisi AB'de eğitiliyor, AB verisi ABD'de eğitiliyor), ek koruma önlemleri uygulanır. ForgeLM'in transferler için bir ayarı yoktur; koruma önlemlerini DPIA'nıza kaydedin. ::: :::tip diff --git a/docs/usermanuals/tr/compliance/human-oversight.md b/docs/usermanuals/tr/compliance/human-oversight.md index 4761258d..98bbbe15 100644 --- a/docs/usermanuals/tr/compliance/human-oversight.md +++ b/docs/usermanuals/tr/compliance/human-oversight.md @@ -5,7 +5,7 @@ description: Bir insan inceleyip imzalayana kadar model terfisini engelleyin — # İnsan Gözetimi -EU AI Act Madde 14, yüksek-riskli AI sistemlerinin insan gözetimi imkânı sağlamasını gerektirir. ForgeLM bunu opsiyonel bir config kapısı olarak uygular: `evaluation.require_human_approval: true` olduğunda model terfisi bir insan onay imzalayana kadar engellenir. +EU AI Act Madde 14, yüksek-riskli AI sistemlerinin insan gözetimi imkânı sağlamasını gerektirir. ForgeLM, `evaluation.require_human_approval: true` ile açılan opsiyonel bir terfi-öncesi onay kapısı sunar: model terfisi bir insan onay imzalayana kadar engellenir ve bu imza kaydedilir. Bu, tasarım-zamanı gözetim dokümantasyonunu destekler; dağıtılmış sistemden Madde 14'ün istediği kullanım-sırası gözetim önlemlerini (izleme, yorumlama, geçersiz kılma veya durdurma) uygulamaz; bunlar dağıtıcıda kalır. ## Kapı nasıl çalışır diff --git a/docs/usermanuals/tr/operations/iso-soc2-deployer.md b/docs/usermanuals/tr/operations/iso-soc2-deployer.md index d96a3898..55e8a0ab 100644 --- a/docs/usermanuals/tr/operations/iso-soc2-deployer.md +++ b/docs/usermanuals/tr/operations/iso-soc2-deployer.md @@ -14,16 +14,16 @@ ISO / SOC 2 denetçisinin en çok önem verdiği dört sütun, doğrudan ForgeLM 1. **Denetim izi** — eğitim çalışması başına `audit_log.jsonl`, HMAC + SHA-256 hash chain + genesis manifest sidecar ile append-only. `forgelm verify-audit` zinciri uçtan uca doğrular. 2. **Değişiklik kontrolü** — Article 14 staging gate'i (`forgelm approve` / `reject`) + `human_approval.required/granted/rejected` audit olayları + run başına damgalanmış `config_hash` (per-run manifest sidecar field). 3. **Veri lineage** — `data_provenance.json` (SHA-256 fingerprint + size + mtime, ayrıca **`hf_revision_source` ile derecelendirilmiş** dataset HF Hub commit SHA'sı) + `data_governance_report.json` (collection_method, annotation_process, known_biases, personal_data_included, dpia_completed). Dereceyi SHA'dan önce okuyun: modeli eğiten korpusun kanıtı yalnızca `hf_revision_source: loaded`'dır. -4. **Tedarik zinciri** — sürüm başına yayınlanan CycloneDX 1.5 SBOM, gecelik `pip-audit`, statik + dinamik güvenlik tarama için CI'da `bandit`. +4. **Tedarik zinciri** — sürüm başına üretilen CycloneDX 1.5 SBOM (90 gün saklanan bir workflow-run artifact'ı, GitHub release asset'i değil), gecelik `pip-audit`, statik + dinamik güvenlik tarama için CI'da `bandit`. ## Sekiz denetim katı sorusu Derin rehber her birini tam `jq` / CLI komutu + döndürdüğü artefakt ile yanıtlar: 1. "Son 90 gündeki her model promotion için audit trail'i göster" → `forgelm verify-audit` + `jq 'select(.event == "human_approval.granted")'`. -2. "Bana change-control kanıtını göster — bu modeli kim onayladı?" → `training.started` + `human_approval.granted` olaylarını çapraz referans alın; iki farklı operator ID, görev ayrılığını (segregation of duties) kanıtlar (ISO A.5.3, SOC 2 CC1.5). +2. "Bana change-control kanıtını göster — bu modeli kim onayladı?" → `training.started` + `human_approval.granted` olaylarını çapraz referans alın; iki kimliği karşılaştırın; ikisi de kendi beyanıdır (`FORGELM_OPERATOR`, yoksa OS kullanıcısı), dolayısıyla görev ayrılığı (segregation of duties; ISO A.5.3, SOC 2 CC1.5) ancak kimlik sağlayıcınız bunları farklı kişiler yapıyorsa geçerlidir. 3. "Veri lineage'ını göster" → `data_provenance.json`; `sha256` + `hf_revision` corpus'u pinler — ancak yalnızca `hf_revision_source` `loaded` olduğunda. `unverified`, SHA'nın paket yazılırken yapılan ve yüklemeyle hiçbir bağı olmayan bir Hub sorgusu olduğu anlamına gelir; `unresolved` ise hiçbir SHA alınmadığı ve hiçbir iddiada bulunulmadığı anlamına gelir. -4. "Tedarik zincirini göster" → `gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*'`; CycloneDX 1.5 JSON. +4. "Tedarik zincirini göster" → `gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success`, ardından 90 gün içinde `gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*'`; CycloneDX 1.5 JSON (CI runner'ının paket listesi, test araçları dahil, lisans verisi yok). 5. "Erişim kontrollerini göster" → IdP audit log + `FORGELM_OPERATOR` çapraz referansı. 6. "Şifreleme duruşunu göster" → operatör-tarafı substrate (KMS audit log + `data_governance_report.json`). 7. "Olay müdahalesini göster" → `audit.classifier_load_failed` + F-compliance-110 strict gate. diff --git a/docs/usermanuals/tr/operations/supply-chain.md b/docs/usermanuals/tr/operations/supply-chain.md index 00a82beb..a2d13134 100644 --- a/docs/usermanuals/tr/operations/supply-chain.md +++ b/docs/usermanuals/tr/operations/supply-chain.md @@ -5,21 +5,24 @@ description: ForgeLM SBOM + pip-audit + bandit pipeline'ı için operatör yüze # Tedarik Zinciri -ForgeLM sürüm tag'i başına bir CycloneDX 1.5 SBOM yayınlar ve `pip-audit` + `bandit`'i `nightly.yml` workflow'u üzerinden gecelik çalıştırır (her sürüm-tag yayınında da yeniden koşar). Bu sayfa operatörün zihinsel modelidir: ne zaman ne çalışır, artefaktlar nereden alınır ve aynı kontroller lokalde nasıl yansıtılır. +ForgeLM sürüm tag'i başına bir CycloneDX 1.5 SBOM üretir (workflow-run artifact'ı olarak, GitHub release asset'i değil) ve `pip-audit` + `bandit`'i `nightly.yml` workflow'u üzerinden gecelik çalıştırır (her sürüm-tag yayınında da yeniden koşar). Bu sayfa operatörün zihinsel modelidir: ne zaman ne çalışır, artefaktlar nereden alınır ve aynı kontroller lokalde nasıl yansıtılır. ## Ne zaman ne çalışır | Tetikleyici | Araç | Sonuç | Hata politikası | |---|---|---|---| -| Sürüm tag'i (`v*`) | `tools/generate_sbom.py` | (OS × Python-sürüm) hücresi başına bir CycloneDX 1.5 JSON, GitHub release'e iliştirilir | SBOM adımı pure-stdlib; yeşil sürüm matrisini sessizce bozamaz | -| Gecelik 03:00 UTC | `pip-audit` (`tools/check_pip_audit.py` üzerinden) | Kurulu bağımlılıklara karşı OSV / GHSA taraması | HIGH/CRITICAL → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → sessiz | +| Sürüm tag'i (`v*`) | `tools/generate_sbom.py` | (OS × Python-sürüm) hücresi başına bir CycloneDX 1.5 JSON, tag'in publish çalıştırmasının workflow-run artifact'ı olarak yüklenir (90 gün saklanır; GitHub release'e iliştirilmez) | SBOM adımı pure-stdlib; yeşil sürüm matrisini sessizce bozamaz | +| Gecelik 03:00 UTC | `pip-audit` (`tools/check_pip_audit.py` üzerinden) | Kurulu bağımlılıklara karşı PyPI güvenlik açığı verisi taraması | HIGH/CRITICAL ya da UNKNOWN severity → exit 1 + GitHub issue; MEDIUM → `::warning::`; LOW → sessiz | | Gecelik 03:00 UTC + sürüm tag'i | `bandit` (`tools/check_bandit.py` üzerinden) | `forgelm/` üzerinde statik güvenlik taraması (`tests/` hariç) | HIGH → exit 1; MEDIUM → `::warning::`; LOW → sessiz | ## SBOM nereden alınır ```bash -# Bir sürüm için tüm SBOM'lar. -gh release download "$(gh release view --json tagName -q .tagName)" --pattern 'sbom-*' +# Bir tag için başarılı publish çalıştırmasını bulun, sonra SBOM'larını indirin +# (artifact başına bir dizin). Run artifact'ları 90 günde silinir. +gh run list --repo HodeTech/ForgeLM --workflow publish.yml --branch v --status success \ + --json databaseId,createdAt +gh run download --repo HodeTech/ForgeLM --pattern 'sbom-*' # Birini güzel-yazdır. jq . sbom-ubuntu-latest-py3.11.json | less @@ -34,7 +37,7 @@ curl -X POST -H "X-Api-Key: $DT_KEY" -H "Content-Type: application/octet-stream" https://deptrack.example.com/api/v1/bom ``` -SBOM determinizm-pinli — aynı Python ortamında ardışık iki yayın content-identical JSON üretir (CycloneDX semantiği gereği kasıtlı olarak değişen `serialNumber` ve `metadata.timestamp` hariç). +SBOM determinizm-pinli — aynı Python ortamında ardışık iki yayın content-identical JSON üretir (CycloneDX semantiği gereği kasıtlı olarak değişen `serialNumber` ve `metadata.timestamp` hariç). Bu, aynı ortama özgü bir özelliktir: release SBOM'u CI runner'ının `pip list`'idir (`pytest` gibi test araçları dahil, bağımlılık grafiği ve lisans verisi yok); aynı tag'den sonradan yeniden emit etmek onu tekrarlamaz. Kendi ortamınızı tanımlamak için orada bir ForgeLM checkout'undan `python3 tools/generate_sbom.py` çalıştırın. ## Gecelik kontrolü lokalde yansıtın @@ -46,7 +49,7 @@ pip-audit --strict --format json --output /tmp/pip-audit.json python3 tools/check_pip_audit.py /tmp/pip-audit.json ``` -Exit 0, ForgeLM CI'nın uyguladığı aynı severity politikasının geçtiği anlamına gelir. Exit 1, gecelik ateşlenmeden önce bir HIGH/CRITICAL açığın ele alınması gerektiği anlamına gelir. +Exit 0, ForgeLM CI'nın uyguladığı aynı severity politikasının geçtiği anlamına gelir. Exit 1, gecelik ateşlenmeden önce bir HIGH/CRITICAL açığın ya da pip-audit'in severity bildirmediği (UNKNOWN; çoğu bulgu böyledir) bir bulgunun ele alınması gerektiği anlamına gelir. ## Bandit kontrolünü lokalde yansıtın diff --git a/docs/usermanuals/tr/reference/exit-codes.md b/docs/usermanuals/tr/reference/exit-codes.md index fd21ec0e..6a0fc562 100644 --- a/docs/usermanuals/tr/reference/exit-codes.md +++ b/docs/usermanuals/tr/reference/exit-codes.md @@ -11,7 +11,7 @@ ForgeLM'in exit kodları kamuya açık bir kontrattır. CI/CD hatları, schedule | Exit | Sabit | Anlam | Tipik CI aksiyonu | |---|---|---|---| -| **0** | `EXIT_SUCCESS` | Koşu tamamlandı ve checkpoint terfi etti. `evaluation.auto_revert: true` ile tüm kapılar da geçti; sevk edilen varsayılan `auto_revert: false` ile başarısız bir benchmark/güvenlik/judge kapısı **JSON çıktısına kaydedilir ama exit kodunu değiştirmez** — aşağıdaki ["exit 0 tam olarak ne garanti eder"](#exit-0-tam-olarak-ne-garanti-eder) bölümüne bakın. | Hattı sürdür (`auto_revert` kapalıysa kapı bloklarını parse et) | +| **0** | `EXIT_SUCCESS` | Koşu tamamlandı ve checkpoint terfi etti. `evaluation.auto_revert: true` ile karara varan tüm kapılar da geçti (karar üretemeyen bir güvenlik değerlendirmesi de modeli korur ve 0 ile çıkar); sevk edilen varsayılan `auto_revert: false` ile başarısız bir benchmark/güvenlik/judge kapısı **JSON çıktısına kaydedilir ama exit kodunu değiştirmez** — aşağıdaki ["exit 0 tam olarak ne garanti eder"](#exit-0-tam-olarak-ne-garanti-eder) bölümüne bakın. | Hattı sürdür (`auto_revert` kapalıysa kapı bloklarını parse et) | | **1** | `EXIT_CONFIG_ERROR` | YAML geçersiz, dosya yok, env var ayarsız veya argüman bozuk. | Hızlı başarısız | | **2** | `EXIT_TRAINING_ERROR` | Eğitim sırasında runtime hatası (config veya değerlendirme kapısı dışı her ele alınmamış istisna: data yükleme, OOM, NaN loss, I/O başarısızlığı, mid-stream audit-iteration OSError). | İncele; logları yüzeyle | | **3** | `EXIT_EVAL_FAILURE` | Bir benchmark/güvenlik/judge kapısı geçemedi **ve** model otomatik geri alındı (`evaluation.auto_revert: true` gerektirir). `auto_revert: false` ile başarısız bir kapı exit 3 üretmez — koşu, JSON kapı bloklarına kaydedilen başarısızlıkla 0 çıkar. | İncele; terfi ETTİRME | @@ -112,7 +112,7 @@ Exit kodu kontrat tek başına yeterli — POSIX kabuklarda `$?`, cmd'de `%ERROR **Kapılar ve exit 0.** *Geçen* bir benchmark/güvenlik/judge kapısının exit-0 garantisinin parçası olup olmadığı `evaluation.auto_revert`'e bağlıdır: -- `evaluation.auto_revert: true` ile (EU AI Act yüksek-riskli varsayılanı), başarısız bir kapı modeli otomatik geri alır ve **3** ile çıkar — yani exit 0 *gerçekten* tüm konfigüre kapıların geçtiği anlamına gelir. +- `evaluation.auto_revert: true` ile (EU AI Act yüksek-riskli varsayılanı), başarısız bir kapı modeli otomatik geri alır ve **3** ile çıkar — yani exit 0, karara varan her konfigüre kapının geçtiği anlamına gelir. **Tek istisna:** karar üretemeyen bir güvenlik değerlendirmesi (`safety.evaluation_completed: false` — sınıflandırıcı yüklenemedi, probe dosyası yok ya da boş, ya da çoğu karar okunamadı) `auto_revert: true` olsa bile modeli korur ve 0 ile çıkar (onay kapısıyla 4); JSON çıktısındaki `safety.passed` ve `safety.evaluation_completed`'e bakın. - Sevk edilen varsayılan `evaluation.auto_revert: false` ile, başarısız bir kapı **kaydedilir** (JSON çıktısındaki `benchmark` / `safety` / `judge` bloğu `*_passed: false` taşır) ama model yine terfi eder ve koşu **0** ile çıkar. Bu JSON bloklarını okuyun; kapı başarısını yalnızca exit 0'dan çıkarsamayın. Tasarım gereği "kısmi başarı" exit kodu yok — başarısız bir kapının exit kodunu değiştirmesini istiyorsanız `auto_revert`'i açın. diff --git a/docs/usermanuals/tr/reference/json-output.md b/docs/usermanuals/tr/reference/json-output.md index a4421064..e1f3a911 100644 --- a/docs/usermanuals/tr/reference/json-output.md +++ b/docs/usermanuals/tr/reference/json-output.md @@ -178,7 +178,7 @@ Eğitim tamamlanana kadar çalıştığında pipeline **stdout**'a bir sonuç en | Anahtar | Tip | Not | |---|---|---| -| `success` | bool | Çalışma tamamlandığında (stage'lenmiş, onay-bekleyen çalışma dahil) `true`. Bir gate modeli auto-revert ettiğinde `false`. | +| `success` | bool | Çalışma tamamlandığında (stage'lenmiş, onay-bekleyen çalışma ve güvenlik değerlendirmesi karar üretemeyen çalışma dahil; `safety.evaluation_completed`'e bakın) `true`. Bir gate modeli auto-revert ettiğinde `false`. | | `error` | string | **Yalnızca** `success` `false` iken bulunur, başarı yolunda asla. Gate'in kendi gerekçesini taşır — hangi gate'in hangi eşiğe karşı tetiklendiğini — çünkü her gate `3` ile çıkar, dolayısıyla exit kodu tek başına ayırt edemez. Bir başarısızlık zarfa gerekçesiz ulaşırsa alan bunu açıkça söyler ve `audit_log.jsonl`'a yönlendirir; atlanmaz. | | `metrics` | object | Eğitim/eval/gate metrikleri (ör. `eval_loss`, `benchmark/average`, `safety/safe_ratio`). Değerler sayıdır; **istisna**, sonlu çıkmayan bir ölçüm `"nan"` / `"inf"` / `"-inf"` string'i olarak yazılır — JSON'da sonlu olmayan sayı literali yoktur, alternatifi hiçbir katı ayrıştırıcının kabul etmediği bir çıktıydı. Bu alanları sayı-veya-string olarak tipleyin. | | `final_model_path` | str | Model artefactlarının yaşadığı yer. Onay-bekleyen bir çalışma için bu, `forgelm approve` promote edene kadar **staging** dizinidir. | diff --git a/site/compliance.html b/site/compliance.html index 73070e2d..b4846663 100644 --- a/site/compliance.html +++ b/site/compliance.html @@ -174,14 +174,14 @@

Transparency

Article 14

Human oversight

-

Optional evaluation.require_human_approval: true moves the trained checkpoint into a staging directory; forgelm approve / reject + forgelm approvals drive the dual-controlled sign-off, all stamped into the audit log.

+

Optional evaluation.require_human_approval: true moves the trained checkpoint into a staging directory; forgelm approve / forgelm reject + forgelm approvals record a human sign-off, with the approver identity stamped into the audit log. It supports design-time oversight documentation; oversight of the deployed system stays with the deployer.

forgelm approve · forgelm reject · forgelm approvals
Article 15

Accuracy & robustness

-

Benchmark gates per-task floor; if the post-train number drops below the floor, the run fails (exit 3).

+

Benchmark gates per-task floor; with evaluation.auto_revert: true, a post-train number below the floor deletes the trained artefacts and the run exits 3. Cybersecurity: secrets / PII at ingest, dependency CVE awareness.

forgelm/benchmark.py
@@ -195,14 +195,14 @@

Conformity & QMS

GDPR

PII redaction at ingest

-

--pii-mask redacts emails, phones, credit cards (Luhn-validated), IBAN, TR national ID, DE Personalausweis, FR SSN, US SSN before chunks land in JSONL.

+

--pii-mask redacts emails, phones, credit cards (Luhn), IBAN, national IDs (TR/DE/FR/US-SSN) before chunks land in JSONL. It reduces what reaches training and supports data minimisation (Article 5(1)(c)); names and free text are not detected, so it is an aid, not a guarantee.

forgelm/data_audit/ · forgelm/data_audit/_pii_regex.py
GDPR Art. 15

Right of access

-

forgelm reverse-pii scans masked JSONL corpora for plaintext residuals or salted SHA-256 digests of a data-subject identifier and reports every row of every split where that subject appears, satisfying the access-request side of the GDPR's data-subject rights.

+

forgelm reverse-pii scans masked JSONL corpora for plaintext residuals or salted SHA-256 digests of a data-subject identifier and reports every row of every split where that subject appears. It finds only what is literally in the files you scan: raw sources and the trained model are not searched, and handling an access request stays your process.

forgelm reverse-pii
@@ -236,7 +236,7 @@

Audit trail

Pillar 2

Change control

-

Article 14 staging gate (forgelm approve / reject) plus human_approval.required/granted/rejected audit events plus forgelm verify-audit chain integrity — every model promotion is dual-controlled and forensically attributed.

+

Article 14 staging gate (forgelm approve / forgelm reject) plus human_approval.required/granted/rejected audit events plus forgelm verify-audit chain integrity — with the gate on, every model promotion is attributed to the recorded approver identity; keeping trainer and approver different people is the deployer's control.

forgelm approve · forgelm verify-audit
diff --git a/site/features.html b/site/features.html index 8350f06a..65307521 100644 --- a/site/features.html +++ b/site/features.html @@ -183,7 +183,7 @@

GaLore

DeepSpeed & FSDP

-

Multi-GPU distributed training. ZeRO presets, FSDP, Unsloth backend.

+

Multi-GPU distributed training. DeepSpeed ZeRO-2 / ZeRO-3 (your own JSON; preset names need a source checkout), FSDP, Unsloth backend.

@@ -226,7 +226,7 @@

Llama Guard safety

Auto-revert

-

With evaluation.auto_revert: true, a failed threshold makes ForgeLM delete the trained artefacts and exit 3. There is no rollback to an earlier checkpoint.

+

With evaluation.auto_revert: true, a failed loss, benchmark, or safety threshold makes ForgeLM delete the trained artefacts, exit 3, and emit a structured incident record. There is no rollback to an earlier checkpoint — nothing is restored. One exception: a safety evaluation that produced no verdict (classifier not loaded, probes missing) keeps the model and exits 0.

@@ -322,7 +322,7 @@

Enterprise & MLOps

Webhook notifications

-

Slack, Teams, Discord, or any HTTP webhook receiver alerts on training events.

+

Slack-format webhook alerts on training start, success, failure, auto-revert (Teams and Discord need a relay). Structured payloads — pipe to your incident tool.

@@ -430,7 +430,7 @@

Annex IV verifier

GGUF integrity verifier

-

forgelm verify-gguf validates the GGUF magic header, parses the metadata block, and re-checks the SHA-256 sidecar (when present) before you ship a quantized model.

+

forgelm verify-gguf validates the GGUF magic header, parses the metadata block, and re-checks the .sha256 sidecar written at export time (when present) before you ship a quantized model.

diff --git a/site/index.html b/site/index.html index 78a7ba21..af90addc 100644 --- a/site/index.html +++ b/site/index.html @@ -476,7 +476,7 @@

Benchmarking

Distributed training

-

DeepSpeed ZeRO presets, FSDP, multi-GPU configs.

+

DeepSpeed ZeRO-2 / ZeRO-3 (bring your own JSON; the preset names need a source checkout), FSDP, multi-GPU configs. MoE support (Qwen3, Mixtral, DeepSeek).

DeepSpeed FSDP @@ -505,7 +505,7 @@

Interactive chat

Export & deploy

-

GGUF export with 6 quant levels.

+

GGUF export (f16 and q8_0 in one step; K-quants need llama.cpp's llama-quantize) with a SHA-256 sidecar. Generate Ollama / vLLM / TGI / HF-Endpoints deployment configs in one command.

GGUF deploy @@ -592,7 +592,7 @@

One YAML, end-to-end run

benchmark: enabled: true tasks: ["hellaswag", "arc_easy", "truthfulqa"] - min_score: 0.65 # aggregate floor; run fails (exit 3) below this + min_score: 0.65 # aggregate floor; with evaluation.auto_revert: true, below this the run exits 3 safety: enabled: true classifier: "meta-llama/Llama-Guard-3-8B" @@ -637,7 +637,7 @@

Use cases that actually ship

Domain expert from policy PDFs

-

Drop your regulatory corpus into forgelm ingest.

+

Drop your regulatory corpus into forgelm ingest, audit it, and SFT a model that answers in your domain's voice — with common PII formats masked at ingest, before training.

forgelm quickstart domain-expert
@@ -659,7 +659,7 @@

Customer support assistant

Code copilot fine-tune

-

Multi-dataset mix, SFT + ORPO, GGUF export.

+

Multi-dataset mix (your repo + open code) with SFT (ORPO is a follow-up step you can chain), GGUF export for local inference. Synthetic data pipeline distils a teacher into your student.

forgelm quickstart code-assistant
diff --git a/site/js/translations.js b/site/js/translations.js index 745ddc7f..ac982eb6 100644 --- a/site/js/translations.js +++ b/site/js/translations.js @@ -378,13 +378,13 @@ "home.cap.bench.title": "Benchmarking", "home.cap.bench.body": "Plug-in lm-evaluation-harness tasks. LLM-as-judge with OpenAI or local model. Floors per task; auto-revert if any drops.", "home.cap.dist.title": "Distributed training", - "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 presets, FSDP, multi-GPU configs. MoE support (Qwen3, Mixtral, DeepSeek).", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 (bring your own JSON; the preset names need a source checkout), FSDP, multi-GPU configs. MoE support (Qwen3, Mixtral, DeepSeek).", "home.cap.vram.title": "VRAM fit-check", "home.cap.vram.body": "Pre-flight memory estimator. FITS / TIGHT / OOM / UNKNOWN verdict with concrete recommendations before you submit a long job.", "home.cap.chat.title": "Interactive chat", "home.cap.chat.body": "Streaming REPL with /reset, /save, /temperature, /system commands. Per-response Llama Guard routing is planned for an upcoming Pro CLI release.", "home.cap.export.title": "Export & deploy", - "home.cap.export.body": "GGUF export with 6 quant levels and SHA-256 manifest. Generate Ollama / vLLM / TGI / HF-Endpoints deployment configs in one command.", + "home.cap.export.body": "GGUF export (f16 and q8_0 in one step; K-quants need llama.cpp's llama-quantize) with a SHA-256 sidecar. Generate Ollama / vLLM / TGI / HF-Endpoints deployment configs in one command.", "home.cap.api.title": "Library API", "home.cap.api.body": "from forgelm import … to embed ForgeTrainer, audit_dataset, verify_audit_log, and verify_annex_iv_artifact calls inside your own Python pipelines — the public surface is enumerated in forgelm.__all__.", "home.cap.iso.title": "ISO 27001 / SOC 2 alignment", @@ -398,11 +398,11 @@ "home.uses.eyebrow": "Built for these workflows", "home.uses.title": "Use cases that actually ship", "home.use.domain.title": "Domain expert from policy PDFs", - "home.use.domain.body": "Drop your regulatory corpus into forgelm ingest, audit it, and SFT a model that answers in your domain's voice — with PII redacted at ingest, before training.", + "home.use.domain.body": "Drop your regulatory corpus into forgelm ingest, audit it, and SFT a model that answers in your domain's voice — with common PII formats masked at ingest, before training.", "home.use.support.title": "Customer support assistant", "home.use.support.body": "SFT on past tickets (and DPO on chosen/rejected agent feedback when you have it), Llama Guard safety evaluation gate, auto-revert if helpfulness drops. Ship to vLLM with one command.", "home.use.code.title": "Code copilot fine-tune", - "home.use.code.body": "Multi-dataset mix (your repo + open code) with SFT (ORPO is a follow-up step you can chain), GGUF export at q4_k_m for local inference. Synthetic data pipeline distils a teacher into your student.", + "home.use.code.body": "Multi-dataset mix (your repo + open code) with SFT (ORPO is a follow-up step you can chain), GGUF export for local inference. Synthetic data pipeline distils a teacher into your student.", "home.use.reasoning.title": "Reasoning RL with GRPO", "home.use.reasoning.body": "Built-in format/length shaping rewards or your own reward function. Group-relative policy optimization without writing a single training-loop line.", "home.cta.title": "Ship your next fine-tune in an afternoon.", @@ -484,13 +484,13 @@ "home.cap.bench.title": "Benchmark", "home.cap.bench.body": "lm-evaluation-harness entegrasyonu. OpenAI veya yerel modelle LLM-as-judge. Görev bazlı tabanlar; biri düşerse otomatik artefakt silme.", "home.cap.dist.title": "Dağıtık eğitim", - "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 hazır ayarlar, FSDP, çoklu-GPU konfigürasyonu. MoE desteği (Qwen3, Mixtral, DeepSeek).", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 (kendi JSON'unuzu verin; hazır ayar adları kaynak checkout'u ister), FSDP, çoklu-GPU konfigürasyonu. MoE desteği (Qwen3, Mixtral, DeepSeek).", "home.cap.vram.title": "VRAM fit-check", "home.cap.vram.body": "Uçuş öncesi bellek tahmincisi. Uzun bir işi göndermeden önce FITS / TIGHT / OOM / UNKNOWN verdict'i ve somut öneriler.", "home.cap.chat.title": "İnteraktif sohbet", "home.cap.chat.body": "/reset, /save, /temperature, /system komutları ile akışlı REPL. Yanıt-başına Llama Guard routing'i yaklaşan Pro CLI sürümü için planlanıyor.", "home.cap.export.title": "Export ve deploy", - "home.cap.export.body": "6 kuantizasyon seviyesi ve SHA-256 manifest ile GGUF export. Tek komutta Ollama / vLLM / TGI / HF-Endpoints deployment config üret.", + "home.cap.export.body": "GGUF export (f16 ve q8_0 tek adımda; K-quant'lar için llama.cpp'nin llama-quantize'ı gerekir) ve SHA-256 sidecar. Tek komutta Ollama / vLLM / TGI / HF-Endpoints deployment config üret.", "home.cap.api.title": "Kütüphane API'si", "home.cap.api.body": "from forgelm import … ile ForgeTrainer, audit_dataset, verify_audit_log ve verify_annex_iv_artifact çağrılarını kendi Python pipeline'ınıza gömün — public yüzey forgelm.__all__ içinde sıralanmıştır.", "home.cap.iso.title": "ISO 27001 / SOC 2 hizalaması", @@ -504,11 +504,11 @@ "home.uses.eyebrow": "Bu iş akışları için tasarlandı", "home.uses.title": "Gerçekten üretime çıkan kullanım senaryoları", "home.use.domain.title": "Politika PDF'lerinden alan uzmanı", - "home.use.domain.body": "Regülasyon korpusunuzu forgelm ingest'e bırakın, denetleyin ve alanınızın diliyle cevap veren bir modeli SFT'leyin — PII ingest sırasında, eğitimden önce maskelenir.", + "home.use.domain.body": "Regülasyon korpusunuzu forgelm ingest'e bırakın, denetleyin ve alanınızın diliyle cevap veren bir modeli SFT'leyin — yaygın PII biçimleri ingest sırasında, eğitimden önce maskelenir.", "home.use.support.title": "Müşteri destek asistanı", "home.use.support.body": "Geçmiş ticket'larda SFT (eğer chosen/rejected agent geri bildirimi varsa DPO eklenebilir), Llama Guard güvenlik değerlendirme kapısı, yardımseverlik düşerse otomatik artefakt silme. Tek komutla vLLM'e gönderim.", "home.use.code.title": "Kod copilot fine-tune", - "home.use.code.body": "Çoklu veri seti karışımı (kendi repo'nuz + açık kod) ile SFT (ORPO'yu sonradan zincirleyebilirsiniz), yerel inference için q4_k_m'de GGUF export. Sentetik veri hattı bir öğretmeni öğrencinize damıtır.", + "home.use.code.body": "Çoklu veri seti karışımı (kendi repo'nuz + açık kod) ile SFT (ORPO'yu sonradan zincirleyebilirsiniz), yerel inference için GGUF export. Sentetik veri hattı bir öğretmeni öğrencinize damıtır.", "home.use.reasoning.title": "GRPO ile akıl yürütme RL", "home.use.reasoning.body": "Yerleşik format/uzunluk ödülleri veya kendi reward fonksiyonunuz. Tek bir training loop satırı yazmadan grup-göreli politika optimizasyonu.", "home.cta.title": "Bir sonraki fine-tune'unuzu bir öğleden sonrasında gönderin.", @@ -590,13 +590,13 @@ "home.cap.bench.title": "Benchmarking", "home.cap.bench.body": "Plug-in-Tasks für lm-evaluation-harness. LLM-as-Judge mit OpenAI oder lokalem Modell. Schwellwerte pro Task; automatisches Verwerfen der Artefakte, wenn einer fällt.", "home.cap.dist.title": "Verteiltes Training", - "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3-Presets, FSDP, Multi-GPU-Konfigurationen. MoE-Unterstützung (Qwen3, Mixtral, DeepSeek).", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 (eigenes JSON; die Preset-Namen brauchen einen Source-Checkout), FSDP, Multi-GPU-Konfigurationen. MoE-Unterstützung (Qwen3, Mixtral, DeepSeek).", "home.cap.vram.title": "VRAM-Fit-Check", "home.cap.vram.body": "Pre-Flight-Speicherschätzer. FITS / TIGHT / OOM / UNKNOWN-Verdikt mit konkreten Empfehlungen, bevor Sie einen langen Job einreichen.", "home.cap.chat.title": "Interaktiver Chat", "home.cap.chat.body": "Streaming-REPL mit Befehlen /reset, /save, /temperature, /system. Pro-Antwort-Llama-Guard-Routing ist für eine kommende Pro-CLI-Version geplant.", "home.cap.export.title": "Export & Deploy", - "home.cap.export.body": "GGUF-Export mit 6 Quantisierungsstufen und SHA-256-Manifest. Generieren Sie Deployment-Konfigurationen für Ollama / vLLM / TGI / HF-Endpoints in einem Befehl.", + "home.cap.export.body": "GGUF-Export (f16 und q8_0 in einem Schritt; K-Quants brauchen llama.cpps llama-quantize) mit SHA-256-Sidecar. Generieren Sie Deployment-Konfigurationen für Ollama / vLLM / TGI / HF-Endpoints in einem Befehl.", "home.cap.api.title": "Bibliotheks-API", "home.cap.api.body": "from forgelm import … und betten Sie ForgeTrainer-, audit_dataset-, verify_audit_log- und verify_annex_iv_artifact-Aufrufe in Ihre eigenen Python-Pipelines ein — die öffentliche Oberfläche ist in forgelm.__all__ aufgeführt.", "home.cap.iso.title": "ISO 27001 / SOC 2 Alignment", @@ -610,11 +610,11 @@ "home.uses.eyebrow": "Gebaut für diese Workflows", "home.uses.title": "Use-Cases, die wirklich in Produktion gehen", "home.use.domain.title": "Domänen-Experte aus Policy-PDFs", - "home.use.domain.body": "Werfen Sie Ihren regulatorischen Korpus in forgelm ingest, auditieren Sie ihn und SFTen Sie ein Modell, das in der Sprache Ihrer Domäne antwortet — mit beim Ingest, vor dem Training, geschwärzten PII.", + "home.use.domain.body": "Werfen Sie Ihren regulatorischen Korpus in forgelm ingest, auditieren Sie ihn und SFTen Sie ein Modell, das in der Sprache Ihrer Domäne antwortet — mit beim Ingest, vor dem Training, maskierten gängigen PII-Formaten.", "home.use.support.title": "Kundensupport-Assistent", "home.use.support.body": "SFT auf vergangenen Tickets (und DPO auf gewählten/abgelehnten Agenten-Feedback, falls verfügbar), Llama-Guard-Sicherheitsbewertungs-Gate, automatisches Verwerfen der Artefakte, wenn die Hilfsbereitschaft sinkt. Mit einem Befehl an vLLM ausliefern.", "home.use.code.title": "Code-Copilot Fine-Tune", - "home.use.code.body": "Multi-Dataset-Mix (Ihr Repo + offener Code) mit SFT (ORPO ist ein Folgeschritt, den Sie verketten können), GGUF-Export bei q4_k_m für lokale Inferenz. Synthetische Datenpipeline destilliert einen Lehrer in Ihren Schüler.", + "home.use.code.body": "Multi-Dataset-Mix (Ihr Repo + offener Code) mit SFT (ORPO ist ein Folgeschritt, den Sie verketten können), GGUF-Export für lokale Inferenz. Synthetische Datenpipeline destilliert einen Lehrer in Ihren Schüler.", "home.use.reasoning.title": "Reasoning-RL mit GRPO", "home.use.reasoning.body": "Eingebaute Format/Längen-Shaping-Rewards oder Ihre eigene Reward-Funktion. Group-Relative Policy Optimization, ohne eine einzige Trainings-Schleifen-Zeile zu schreiben.", "home.cta.title": "Liefern Sie Ihren nächsten Fine-Tune an einem Nachmittag aus.", @@ -696,13 +696,13 @@ "home.cap.bench.title": "Benchmarking", "home.cap.bench.body": "Tâches lm-evaluation-harness en plug-in. LLM-as-judge avec OpenAI ou modèle local. Planchers par tâche ; suppression automatique des artefacts si l'un baisse.", "home.cap.dist.title": "Entraînement distribué", - "home.cap.dist.body": "Presets DeepSpeed ZeRO-2 / ZeRO-3, FSDP, configurations multi-GPU. Support MoE (Qwen3, Mixtral, DeepSeek).", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 (votre propre JSON ; les noms de presets exigent un checkout des sources), FSDP, configurations multi-GPU. Support MoE (Qwen3, Mixtral, DeepSeek).", "home.cap.vram.title": "Vérification VRAM", "home.cap.vram.body": "Estimateur de mémoire pré-vol. Verdict FITS / TIGHT / OOM / UNKNOWN avec recommandations concrètes avant de soumettre un long job.", "home.cap.chat.title": "Chat interactif", "home.cap.chat.body": "REPL en streaming avec commandes /reset, /save, /temperature, /system. Le routing Llama Guard par réponse est prévu pour une prochaine version de la CLI Pro.", "home.cap.export.title": "Export & déploiement", - "home.cap.export.body": "Export GGUF avec 6 niveaux de quantisation et manifest SHA-256. Générez des configurations de déploiement Ollama / vLLM / TGI / HF-Endpoints en une commande.", + "home.cap.export.body": "Export GGUF (f16 et q8_0 en une étape ; les K-quants nécessitent llama-quantize de llama.cpp) avec un fichier SHA-256 associé. Générez des configurations de déploiement Ollama / vLLM / TGI / HF-Endpoints en une commande.", "home.cap.api.title": "API bibliothèque", "home.cap.api.body": "from forgelm import … et intégrez les appels ForgeTrainer, audit_dataset, verify_audit_log et verify_annex_iv_artifact dans vos propres pipelines Python — la surface publique est listée dans forgelm.__all__.", "home.cap.iso.title": "Alignement ISO 27001 / SOC 2", @@ -716,11 +716,11 @@ "home.uses.eyebrow": "Conçu pour ces workflows", "home.uses.title": "Cas d'usage qui livrent vraiment", "home.use.domain.title": "Expert métier à partir de PDF de politiques", - "home.use.domain.body": "Déposez votre corpus réglementaire dans forgelm ingest, auditez-le, et fine-tunez en SFT un modèle qui répond avec la voix de votre domaine — avec PII masqués à l'ingestion, avant l'entraînement.", + "home.use.domain.body": "Déposez votre corpus réglementaire dans forgelm ingest, auditez-le, et fine-tunez en SFT un modèle qui répond avec la voix de votre domaine — avec les formats PII courants masqués à l'ingestion, avant l'entraînement.", "home.use.support.title": "Assistant de support client", "home.use.support.body": "SFT sur tickets passés (et DPO sur feedbacks choisis/rejetés des agents si disponibles), gate d'évaluation sécurité Llama Guard, suppression automatique des artefacts si la serviabilité baisse. Livraison à vLLM en une commande.", "home.use.code.title": "Fine-tune copilote de code", - "home.use.code.body": "Mix multi-datasets (votre repo + code ouvert) avec SFT (l'ORPO est une étape de suite que vous pouvez chaîner), export GGUF en q4_k_m pour inférence locale. Pipeline de données synthétiques distille un teacher dans votre student.", + "home.use.code.body": "Mix multi-datasets (votre repo + code ouvert) avec SFT (l'ORPO est une étape de suite que vous pouvez chaîner), export GGUF pour inférence locale. Pipeline de données synthétiques distille un teacher dans votre student.", "home.use.reasoning.title": "RL de raisonnement avec GRPO", "home.use.reasoning.body": "Récompenses de format/longueur intégrées ou votre propre fonction de récompense. Group-Relative Policy Optimization sans écrire une seule ligne de boucle d'entraînement.", "home.cta.title": "Livrez votre prochain fine-tune en un après-midi.", @@ -802,13 +802,13 @@ "home.cap.bench.title": "Benchmarking", "home.cap.bench.body": "Tareas lm-evaluation-harness conectables. LLM-as-judge con OpenAI o modelo local. Pisos por tarea; descarte automático de artefactos si alguno cae.", "home.cap.dist.title": "Entrenamiento distribuido", - "home.cap.dist.body": "Presets DeepSpeed ZeRO-2 / ZeRO-3, FSDP, configuraciones multi-GPU. Soporte MoE (Qwen3, Mixtral, DeepSeek).", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 (su propio JSON; los nombres de preset requieren un checkout del código fuente), FSDP, configuraciones multi-GPU. Soporte MoE (Qwen3, Mixtral, DeepSeek).", "home.cap.vram.title": "Comprobación VRAM", "home.cap.vram.body": "Estimador de memoria pre-vuelo. Veredicto FITS / TIGHT / OOM / UNKNOWN con recomendaciones concretas antes de enviar un trabajo largo.", "home.cap.chat.title": "Chat interactivo", "home.cap.chat.body": "REPL en streaming con comandos /reset, /save, /temperature, /system. El routing Llama Guard por respuesta está planeado para una próxima versión de la CLI Pro.", "home.cap.export.title": "Export y despliegue", - "home.cap.export.body": "Export GGUF con 6 niveles de cuantización y manifest SHA-256. Genere configuraciones de despliegue para Ollama / vLLM / TGI / HF-Endpoints en un solo comando.", + "home.cap.export.body": "Export GGUF (f16 y q8_0 en un paso; los K-quants requieren llama-quantize de llama.cpp) con un archivo SHA-256 asociado. Genere configuraciones de despliegue para Ollama / vLLM / TGI / HF-Endpoints en un solo comando.", "home.cap.api.title": "API de biblioteca", "home.cap.api.body": "from forgelm import … y embeba llamadas a ForgeTrainer, audit_dataset, verify_audit_log y verify_annex_iv_artifact dentro de sus propios pipelines de Python — la superficie pública se enumera en forgelm.__all__.", "home.cap.iso.title": "Alineación con ISO 27001 / SOC 2", @@ -822,11 +822,11 @@ "home.uses.eyebrow": "Construido para estos flujos", "home.uses.title": "Casos de uso que realmente se despliegan", "home.use.domain.title": "Experto de dominio a partir de PDFs regulatorios", - "home.use.domain.body": "Suelte su corpus regulatorio en forgelm ingest, audítelo y haga SFT a un modelo que responda con la voz de su dominio — con PII enmascarado en la ingestión, antes del entrenamiento.", + "home.use.domain.body": "Suelte su corpus regulatorio en forgelm ingest, audítelo y haga SFT a un modelo que responda con la voz de su dominio — con los formatos de PII comunes enmascarados en la ingestión, antes del entrenamiento.", "home.use.support.title": "Asistente de soporte al cliente", "home.use.support.body": "SFT sobre tickets pasados (y DPO sobre feedback elegido/rechazado de agentes si está disponible), puerta de evaluación de seguridad Llama Guard, descarte automático de artefactos si la utilidad baja. Despliegue a vLLM con un solo comando.", "home.use.code.title": "Fine-tune de copiloto de código", - "home.use.code.body": "Mezcla multi-dataset (su repo + código abierto) con SFT (ORPO es un paso posterior que puede encadenar), export GGUF en q4_k_m para inferencia local. La pipeline de datos sintéticos destila un teacher en su student.", + "home.use.code.body": "Mezcla multi-dataset (su repo + código abierto) con SFT (ORPO es un paso posterior que puede encadenar), export GGUF para inferencia local. La pipeline de datos sintéticos destila un teacher en su student.", "home.use.reasoning.title": "RL de razonamiento con GRPO", "home.use.reasoning.body": "Recompensas de format/length integradas o su propia función de recompensa. Group-Relative Policy Optimization sin escribir una sola línea de bucle de entrenamiento.", "home.cta.title": "Despliegue su próximo fine-tune en una tarde.", @@ -908,13 +908,13 @@ "home.cap.bench.title": "Benchmark", "home.cap.bench.body": "可插拔的 lm-evaluation-harness 任务。用 OpenAI 或本地模型作为 LLM-as-judge。每任务设阈值;任一跌破时自动丢弃训练产物。", "home.cap.dist.title": "分布式训练", - "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3 预设、FSDP、多 GPU 配置。MoE 支持 (Qwen3、Mixtral、DeepSeek)。", + "home.cap.dist.body": "DeepSpeed ZeRO-2 / ZeRO-3(自带 JSON;预设名称需要源码 checkout)、FSDP、多 GPU 配置。MoE 支持 (Qwen3、Mixtral、DeepSeek)。", "home.cap.vram.title": "VRAM 适配检查", "home.cap.vram.body": "起飞前显存估算。在你提交一个长任务之前给出 FITS / TIGHT / OOM / UNKNOWN 判定和具体建议。", "home.cap.chat.title": "交互式聊天", "home.cap.chat.body": "支持 /reset、/save、/temperature、/system 命令的流式 REPL。每响应 Llama Guard 路由计划在即将推出的 Pro CLI 版本中提供。", "home.cap.export.title": "Export & 部署", - "home.cap.export.body": "GGUF export 6 个量化级别加 SHA-256 manifest。一条命令生成 Ollama / vLLM / TGI / HF-Endpoints 部署配置。", + "home.cap.export.body": "GGUF export(f16 和 q8_0 一步完成;K-quant 需要 llama.cpp 的 llama-quantize)并附带 SHA-256 sidecar 文件。一条命令生成 Ollama / vLLM / TGI / HF-Endpoints 部署配置。", "home.cap.api.title": "库 API", "home.cap.api.body": "from forgelm import …,在你自己的 Python 管道中嵌入 ForgeTrainer、audit_dataset、verify_audit_log 和 verify_annex_iv_artifact 调用 — 公开表面在 forgelm.__all__ 中列出。", "home.cap.iso.title": "ISO 27001 / SOC 2 对齐", @@ -928,11 +928,11 @@ "home.uses.eyebrow": "为这些工作流而构建", "home.uses.title": "真正能上线的用例", "home.use.domain.title": "从政策 PDF 中得到的领域专家", - "home.use.domain.body": "把你的法规语料投入 forgelm ingest、审计它、然后 SFT 一个能用你领域口吻回答的模型 — PII 在 ingest 时,训练之前,就被屏蔽。", + "home.use.domain.body": "把你的法规语料投入 forgelm ingest、审计它、然后 SFT 一个能用你领域口吻回答的模型 — 常见 PII 格式在 ingest 时,训练之前,就被屏蔽。", "home.use.support.title": "客户支持助手", "home.use.support.body": "在过往工单上 SFT (有 chosen/rejected 工单反馈时可叠加 DPO),Llama Guard 安全评估 gate,有用度下降时自动丢弃训练产物。一条命令交付到 vLLM。", "home.use.code.title": "代码 copilot 微调", - "home.use.code.body": "多数据集混合 (你的 repo + 开源代码) 的 SFT (ORPO 是可链式的后续步骤),本地推理用 q4_k_m 的 GGUF export。合成数据 pipeline 把 teacher 蒸馏到你的 student 中。", + "home.use.code.body": "多数据集混合 (你的 repo + 开源代码) 的 SFT (ORPO 是可链式的后续步骤),用于本地推理的 GGUF export。合成数据 pipeline 把 teacher 蒸馏到你的 student 中。", "home.use.reasoning.title": "用 GRPO 做推理 RL", "home.use.reasoning.body": "内置 format/length 塑形奖励或你自己的奖励函数。无需写一行训练循环就能做 Group-Relative Policy Optimization。", "home.cta.title": "在一个下午内交付下一次微调。", @@ -961,7 +961,7 @@ "features.train.qlora": "4-bit NF4 quantization with LoRA. PiSSA, rsLoRA, DoRA variants. Train 7B-13B models on a single 12 GB GPU.", "features.train.galore": "Gradient low-rank projection for full-parameter training without LoRA. Memory at LoRA levels, capacity of full fine-tune.", "features.train.deepspeed.title":"DeepSpeed & FSDP", - "features.train.deepspeed.body":"Multi-GPU distributed training. ZeRO-2 / ZeRO-3 presets, FSDP, Unsloth backend.", + "features.train.deepspeed.body":"Multi-GPU distributed training. DeepSpeed ZeRO-2 / ZeRO-3 (your own JSON; preset names need a source checkout), FSDP, Unsloth backend.", "features.train.longctx.title":"Long-context", "features.train.longctx.body":"RoPE scaling, NEFTune noise injection, sliding-window attention, sample packing — train on 32K-128K context.", "features.eval.heading": "Evaluation & safety", @@ -973,7 +973,7 @@ "features.eval.lguard.title": "Llama Guard safety", "features.eval.lguard.body": "Verdicts across 14 harm categories (S1-S14), scored binary by default; confidence-weighted scoring applies only on the classification pipeline path. Severity tiers, cross-run trend tracking.", "features.eval.revert.title": "Auto-revert", - "features.eval.revert.body": "With evaluation.auto_revert: true, a failed loss, benchmark, or safety threshold makes ForgeLM delete the trained artefacts, exit 3, and emit a structured incident record. There is no rollback to an earlier checkpoint — nothing is restored.", + "features.eval.revert.body": "With evaluation.auto_revert: true, a failed loss, benchmark, or safety threshold makes ForgeLM delete the trained artefacts, exit 3, and emit a structured incident record. There is no rollback to an earlier checkpoint — nothing is restored. One exception: a safety evaluation that produced no verdict (classifier not loaded, probes missing) keeps the model and exits 0.", "features.eval.trend.title": "Trend tracking", "features.eval.trend.body": "Cross-run safety history (safety_trend.jsonl per output_dir). See if a category quietly drifts before it crosses the threshold.", "features.eval.exit.title": "Exit codes", @@ -999,7 +999,7 @@ "features.ent.heading": "Enterprise & MLOps", "features.ent.body": "Built for pipelines: containers, webhooks, structured outputs, air-gap, model cards.", "features.ent.webhook.title": "Webhook notifications", - "features.ent.webhook.body": "Slack, Teams, Discord, or any HTTP webhook receiver — alerts on training start, success, failure, auto-revert. Structured payloads — pipe to your incident tool.", + "features.ent.webhook.body": "Slack-format webhook alerts on training start, success, failure, auto-revert (Teams and Discord need a relay). Structured payloads — pipe to your incident tool.", "features.ent.exp.title": "Experiment tracking", "features.ent.exp.body": "W&B, MLflow, TensorBoard via report_to. Pluggable, not hardcoded. Use what your team already uses.", "features.ent.json.title": "JSON output mode", @@ -1039,7 +1039,7 @@ "features.gov.annex.title": "Annex IV verifier", "features.gov.annex.body": "forgelm verify-annex-iv validates the Annex IV §1-9 schema completeness and the bundle's manifest_hash for tampering detection. (Audit-chain integrity is the separate forgelm verify-audit command.)", "features.gov.gguf.title": "GGUF integrity verifier", - "features.gov.gguf.body": "forgelm verify-gguf re-checks the SHA-256 + size manifest stamped at export time before you ship a quantized model.", + "features.gov.gguf.body": "forgelm verify-gguf validates the GGUF magic header, parses the metadata block, and re-checks the .sha256 sidecar written at export time (when present) before you ship a quantized model.", "features.gov.iso.title": "ISO 27001 / SOC 2 alignment", "features.gov.iso.body": "Audit-trail, change-management, data-lineage, and supply-chain evidence the deployer's auditor asks for. Software is aligned, not certified.", "features.cta.title": "Want to see it in YAML?", @@ -1066,7 +1066,7 @@ "features.train.qlora": "LoRA ile 4-bit NF4 kuantizasyon. PiSSA, rsLoRA, DoRA varyantları. Tek 12 GB GPU'da 7B-13B model eğitin.", "features.train.galore": "LoRA olmadan full-parameter eğitim için gradient low-rank projeksiyonu. LoRA seviyesinde bellek, full fine-tune kapasitesi.", "features.train.deepspeed.title":"DeepSpeed ve FSDP", - "features.train.deepspeed.body":"Çoklu-GPU dağıtık eğitim. ZeRO-2 / ZeRO-3 hazır ayarlar, FSDP, Unsloth backend.", + "features.train.deepspeed.body":"Çoklu-GPU dağıtık eğitim. DeepSpeed ZeRO-2 / ZeRO-3 (kendi JSON'unuz; hazır ayar adları kaynak checkout'u ister), FSDP, Unsloth backend.", "features.train.longctx.title":"Uzun-context", "features.train.longctx.body":"RoPE scaling, NEFTune noise injection, kayar-pencere attention, sample packing — 32K-128K context'te eğitim.", "features.eval.heading": "Değerlendirme ve güvenlik", @@ -1078,7 +1078,7 @@ "features.eval.lguard.title": "Llama Guard güvenliği", "features.eval.lguard.body": "14 zarar kategorisinde (S1-S14) kararlar; varsayılan skorlama binary, güven-ağırlıklı skorlama yalnızca classification pipeline yolunda geçerli. Ciddiyet katmanları, koşular arası trend izleme.", "features.eval.revert.title": "Otomatik artefakt silme (auto-revert)", - "features.eval.revert.body": "evaluation.auto_revert: true ile loss, benchmark veya güvenlik eşiği başarısız olduğunda ForgeLM eğitilen artefaktları siler, 3 ile çıkar ve yapılandırılmış bir olay kaydı üretir. Önceki bir checkpoint'e dönüş yoktur — hiçbir şey geri yüklenmez.", + "features.eval.revert.body": "evaluation.auto_revert: true ile loss, benchmark veya güvenlik eşiği başarısız olduğunda ForgeLM eğitilen artefaktları siler, 3 ile çıkar ve yapılandırılmış bir olay kaydı üretir. Önceki bir checkpoint'e dönüş yoktur — hiçbir şey geri yüklenmez. Bir istisna: karar üretemeyen bir güvenlik değerlendirmesi (sınıflandırıcı yüklenmedi, probe'lar yok) modeli korur ve 0 ile çıkar.", "features.eval.trend.title": "Trend izleme", "features.eval.trend.body": "Koşular arası güvenlik geçmişi (output_dir başına safety_trend.jsonl). Bir kategori eşiği aşmadan önce sessizce kayıyor mu görün.", "features.eval.exit.title": "Exit kodları", @@ -1104,7 +1104,7 @@ "features.ent.heading": "Kurumsal ve MLOps", "features.ent.body": "Hatlar için yapıldı: konteynerler, webhook'lar, yapılandırılmış çıktı, air-gap, model card.", "features.ent.webhook.title": "Webhook bildirimleri", - "features.ent.webhook.body": "Eğitim başlangıcında, başarıda, hatada ve otomatik artefakt silmede Slack, Teams, Discord veya herhangi bir HTTP webhook alıcısına bildirim gönderir. Yapılandırılmış payload — incident aracınıza bağlayın.", + "features.ent.webhook.body": "Eğitim başlangıcında, başarıda, hatada ve otomatik artefakt silmede Slack biçiminde webhook bildirimi gönderir (Teams ve Discord için relay gerekir). Yapılandırılmış payload — incident aracınıza bağlayın.", "features.ent.exp.title": "Deney takibi", "features.ent.exp.body": "report_to üzerinden W&B, MLflow, TensorBoard. Pluggable, hardcoded değil. Ekibinizin zaten kullandığını kullanın.", "features.ent.json.title": "JSON çıktı modu", @@ -1144,7 +1144,7 @@ "features.gov.annex.title": "Annex IV doğrulayıcı", "features.gov.annex.body": "forgelm verify-annex-iv Annex IV §1-9 şema tamlığını ve bundle'ın manifest_hash'ini tampering tespiti için doğrular. (Audit-chain bütünlüğü ayrı bir forgelm verify-audit komutudur.)", "features.gov.gguf.title": "GGUF bütünlük doğrulayıcı", - "features.gov.gguf.body": "forgelm verify-gguf bir kuantize modeli sevk etmeden önce export sırasında damgalanan SHA-256 + boyut manifest'ini yeniden kontrol eder.", + "features.gov.gguf.body": "forgelm verify-gguf bir kuantize modeli sevk etmeden önce GGUF magic header'ını doğrular, metadata bloğunu ayrıştırır ve export sırasında yazılan .sha256 sidecar'ını (varsa) yeniden kontrol eder.", "features.gov.iso.title": "ISO 27001 / SOC 2 hizalaması", "features.gov.iso.body": "Deployer denetçisinin sorduğu audit izi, change-management, veri kökenleri ve tedarik zinciri kanıtları. Yazılım sertifikalı değil, hizalanmıştır.", "features.cta.title": "YAML olarak görmek ister misiniz?", @@ -1171,7 +1171,7 @@ "features.train.qlora": "4-Bit-NF4-Quantisierung mit LoRA. PiSSA-, rsLoRA-, DoRA-Varianten. Trainieren Sie 7B-13B-Modelle auf einer einzigen 12-GB-GPU.", "features.train.galore": "Gradient-Low-Rank-Projection für Full-Parameter-Training ohne LoRA. Speicher auf LoRA-Niveau, Kapazität eines vollen Fine-Tunes.", "features.train.deepspeed.title":"DeepSpeed & FSDP", - "features.train.deepspeed.body":"Multi-GPU-Distributed-Training. ZeRO-2 / ZeRO-3-Presets, FSDP, Unsloth-Backend.", + "features.train.deepspeed.body":"Multi-GPU-Distributed-Training. DeepSpeed ZeRO-2 / ZeRO-3 (eigenes JSON; Preset-Namen brauchen einen Source-Checkout), FSDP, Unsloth-Backend.", "features.train.longctx.title":"Long-Context", "features.train.longctx.body":"RoPE-Scaling, NEFTune-Noise-Injection, Sliding-Window-Attention, Sample-Packing — Training mit 32K-128K-Kontext.", "features.eval.heading": "Evaluierung & Sicherheit", @@ -1183,7 +1183,7 @@ "features.eval.lguard.title": "Llama-Guard-Sicherheit", "features.eval.lguard.body": "Urteile über 14 Schadkategorien (S1-S14), standardmäßig binär bewertet; konfidenzgewichtete Bewertung gilt nur auf dem classification-Pipeline-Pfad. Schweregrade, lauf-übergreifendes Trend-Tracking.", "features.eval.revert.title": "Auto-Revert (Artefakte verwerfen)", - "features.eval.revert.body": "Mit evaluation.auto_revert: true löscht ForgeLM bei verletztem Loss-, Benchmark- oder Sicherheitsschwellwert die trainierten Artefakte, beendet mit 3 und erzeugt einen strukturierten Vorfallsdatensatz. Es gibt keine Rückkehr zu einem früheren Checkpoint — nichts wird wiederhergestellt.", + "features.eval.revert.body": "Mit evaluation.auto_revert: true löscht ForgeLM bei verletztem Loss-, Benchmark- oder Sicherheitsschwellwert die trainierten Artefakte, beendet mit 3 und erzeugt einen strukturierten Vorfallsdatensatz. Es gibt keine Rückkehr zu einem früheren Checkpoint — nichts wird wiederhergestellt. Eine Ausnahme: Eine Sicherheitsbewertung ohne Ergebnis (Klassifikator nicht geladen, Probes fehlen) behält das Modell und endet mit 0.", "features.eval.trend.title": "Trend-Tracking", "features.eval.trend.body": "Lauf-übergreifende Sicherheitshistorie (safety_trend.jsonl pro output_dir). Sehen Sie, ob eine Kategorie stillschweigend driftet, bevor sie den Schwellwert überschreitet.", "features.eval.exit.title": "Exit-Codes", @@ -1209,7 +1209,7 @@ "features.ent.heading": "Enterprise & MLOps", "features.ent.body": "Gebaut für Pipelines: Container, Webhooks, strukturierte Ausgaben, Air-Gap, Model Cards.", "features.ent.webhook.title": "Webhook-Benachrichtigungen", - "features.ent.webhook.body": "Slack-, Teams-, Discord- oder beliebige HTTP-Webhook-Alerts bei Trainingsstart, Erfolg, Fehlschlag, automatischem Verwerfen der Artefakte. Strukturierte Payloads — leiten Sie sie an Ihr Incident-Tool weiter.", + "features.ent.webhook.body": "Slack-formatierte Webhook-Alerts bei Trainingsstart, Erfolg, Fehlschlag, automatischem Verwerfen der Artefakte (Teams und Discord brauchen ein Relay). Strukturierte Payloads — leiten Sie sie an Ihr Incident-Tool weiter.", "features.ent.exp.title": "Experiment-Tracking", "features.ent.exp.body": "W&B, MLflow, TensorBoard via report_to. Pluggable, nicht hartkodiert. Nutzen Sie, was Ihr Team bereits verwendet.", "features.ent.json.title": "JSON-Ausgabemodus", @@ -1250,7 +1250,7 @@ "features.train.qlora": "Quantisation NF4 4-bit avec LoRA. Variantes PiSSA, rsLoRA, DoRA. Entraînez des modèles 7B-13B sur un seul GPU 12 Go.", "features.train.galore": "Projection gradient bas-rang pour entraînement full-parameter sans LoRA. Mémoire au niveau LoRA, capacité d'un fine-tune complet.", "features.train.deepspeed.title":"DeepSpeed & FSDP", - "features.train.deepspeed.body":"Entraînement distribué multi-GPU. Presets ZeRO-2 / ZeRO-3, FSDP, backend Unsloth.", + "features.train.deepspeed.body":"Entraînement distribué multi-GPU. DeepSpeed ZeRO-2 / ZeRO-3 (votre JSON ; les noms de presets exigent un checkout des sources), FSDP, backend Unsloth.", "features.train.longctx.title":"Long-contexte", "features.train.longctx.body":"RoPE scaling, injection de bruit NEFTune, attention à fenêtre glissante, sample packing — entraînez sur 32K-128K de contexte.", "features.eval.heading": "Évaluation & sécurité", @@ -1262,7 +1262,7 @@ "features.eval.lguard.title": "Sécurité Llama Guard", "features.eval.lguard.body": "Verdicts sur 14 catégories de préjudice (S1-S14), notés en binaire par défaut ; le scoring pondéré par confiance ne s'applique que sur le chemin de pipeline classification. Niveaux de sévérité, suivi de tendance inter-exécutions.", "features.eval.revert.title": "Auto-revert (suppression des artefacts)", - "features.eval.revert.body": "Avec evaluation.auto_revert: true, un seuil de loss, benchmark ou sécurité non respecté fait supprimer les artefacts entraînés par ForgeLM, qui sort avec 3 et émet un enregistrement d'incident structuré. Il n'y a aucun retour à un checkpoint antérieur — rien n'est restauré.", + "features.eval.revert.body": "Avec evaluation.auto_revert: true, un seuil de loss, benchmark ou sécurité non respecté fait supprimer les artefacts entraînés par ForgeLM, qui sort avec 3 et émet un enregistrement d'incident structuré. Il n'y a aucun retour à un checkpoint antérieur — rien n'est restauré. Une exception : une évaluation de sécurité qui n'a produit aucun verdict (classifieur non chargé, probes absents) conserve le modèle et sort avec 0.", "features.eval.trend.title": "Suivi de tendance", "features.eval.trend.body": "Historique sécurité inter-exécutions (safety_trend.jsonl par output_dir). Voyez si une catégorie dérive silencieusement avant de franchir le seuil.", "features.eval.exit.title": "Codes de sortie", @@ -1288,7 +1288,7 @@ "features.ent.heading": "Entreprise & MLOps", "features.ent.body": "Conçu pour les pipelines : conteneurs, webhooks, sorties structurées, air-gap, model cards.", "features.ent.webhook.title": "Notifications webhook", - "features.ent.webhook.body": "Alertes Slack, Teams, Discord ou tout récepteur webhook HTTP au démarrage, succès, échec, suppression automatique des artefacts d'entraînement. Payloads structurés — pipez-les vers votre outil d'incident.", + "features.ent.webhook.body": "Alertes webhook au format Slack au démarrage, succès, échec, suppression automatique des artefacts d'entraînement (Teams et Discord nécessitent un relais). Payloads structurés — pipez-les vers votre outil d'incident.", "features.ent.exp.title": "Suivi d'expériences", "features.ent.exp.body": "W&B, MLflow, TensorBoard via report_to. Pluggable, pas codé en dur. Utilisez ce que votre équipe utilise déjà.", "features.ent.json.title": "Mode de sortie JSON", @@ -1329,7 +1329,7 @@ "features.train.qlora": "Cuantización NF4 de 4 bits con LoRA. Variantes PiSSA, rsLoRA, DoRA. Entrene modelos 7B-13B en una sola GPU de 12 GB.", "features.train.galore": "Proyección gradient de rango bajo para entrenamiento full-parameter sin LoRA. Memoria a niveles de LoRA, capacidad de un fine-tune completo.", "features.train.deepspeed.title":"DeepSpeed y FSDP", - "features.train.deepspeed.body":"Entrenamiento distribuido multi-GPU. Presets ZeRO-2 / ZeRO-3, FSDP, backend Unsloth.", + "features.train.deepspeed.body":"Entrenamiento distribuido multi-GPU. DeepSpeed ZeRO-2 / ZeRO-3 (su JSON; los nombres de preset requieren un checkout del código fuente), FSDP, backend Unsloth.", "features.train.longctx.title":"Contexto largo", "features.train.longctx.body":"RoPE scaling, inyección de ruido NEFTune, attention de ventana deslizante, sample packing — entrene en contexto de 32K-128K.", "features.eval.heading": "Evaluación y seguridad", @@ -1341,7 +1341,7 @@ "features.eval.lguard.title": "Seguridad Llama Guard", "features.eval.lguard.body": "Veredictos en 14 categorías de daño (S1-S14), puntuados en binario por defecto; el scoring ponderado por confianza solo aplica en la ruta de pipeline classification. Niveles de severidad, seguimiento de tendencias entre ejecuciones.", "features.eval.revert.title": "Auto-revert (descarte de artefactos)", - "features.eval.revert.body": "Con evaluation.auto_revert: true, si falla un umbral de loss, benchmark o seguridad, ForgeLM elimina los artefactos entrenados, sale con 3 y emite un registro de incidente estructurado. No hay vuelta a un checkpoint anterior — no se restaura nada.", + "features.eval.revert.body": "Con evaluation.auto_revert: true, si falla un umbral de loss, benchmark o seguridad, ForgeLM elimina los artefactos entrenados, sale con 3 y emite un registro de incidente estructurado. No hay vuelta a un checkpoint anterior — no se restaura nada. Una excepción: una evaluación de seguridad que no produjo veredicto (clasificador no cargado, sondas ausentes) conserva el modelo y sale con 0.", "features.eval.trend.title": "Seguimiento de tendencias", "features.eval.trend.body": "Historial de seguridad entre ejecuciones (safety_trend.jsonl por output_dir). Vea si una categoría deriva silenciosamente antes de cruzar el umbral.", "features.eval.exit.title": "Códigos de salida", @@ -1367,7 +1367,7 @@ "features.ent.heading": "Empresa y MLOps", "features.ent.body": "Construido para pipelines: contenedores, webhooks, salidas estructuradas, air-gap, model cards.", "features.ent.webhook.title": "Notificaciones webhook", - "features.ent.webhook.body": "Alertas en Slack, Teams, Discord o cualquier receptor webhook HTTP al inicio, éxito, fallo o descarte automático de los artefactos de entrenamiento. Payloads estructurados — diríjalos a su herramienta de incidentes.", + "features.ent.webhook.body": "Alertas webhook en formato Slack al inicio, éxito, fallo o descarte automático de los artefactos de entrenamiento (Teams y Discord requieren un relay). Payloads estructurados — diríjalos a su herramienta de incidentes.", "features.ent.exp.title": "Seguimiento de experimentos", "features.ent.exp.body": "W&B, MLflow, TensorBoard vía report_to. Conectable, no hardcoded. Use lo que su equipo ya usa.", "features.ent.json.title": "Modo de salida JSON", @@ -1408,7 +1408,7 @@ "features.train.qlora": "带 LoRA 的 4-bit NF4 量化。PiSSA、rsLoRA、DoRA 变体。在单张 12 GB GPU 上训练 7B-13B 模型。", "features.train.galore": "无需 LoRA 的 full-parameter 训练 — gradient 低秩投影。显存达到 LoRA 水平,容量是完整 fine-tune。", "features.train.deepspeed.title":"DeepSpeed & FSDP", - "features.train.deepspeed.body":"多 GPU 分布式训练。ZeRO-2 / ZeRO-3 预设、FSDP、Unsloth 后端。", + "features.train.deepspeed.body":"多 GPU 分布式训练。DeepSpeed ZeRO-2 / ZeRO-3(自带 JSON;预设名称需要源码 checkout)、FSDP、Unsloth 后端。", "features.train.longctx.title":"长上下文", "features.train.longctx.body":"RoPE scaling、NEFTune 噪声注入、滑动窗口 attention、sample packing — 在 32K-128K 上下文上训练。", "features.eval.heading": "评估 & 安全", @@ -1420,7 +1420,7 @@ "features.eval.lguard.title": "Llama Guard 安全", "features.eval.lguard.body": "在 14 个危害类别 (S1-S14) 上给出判定,默认为二元评分；置信度加权评分仅在 classification pipeline 路径上生效。严重性分级、跨运行趋势跟踪。", "features.eval.revert.title": "Auto-revert(丢弃训练产物)", - "features.eval.revert.body": "当 evaluation.auto_revert: true 时,loss、benchmark 或安全阈值不通过会让 ForgeLM 删除训练产物、以 3 退出并发出结构化事件记录。不会回滚到此前的 checkpoint — 不恢复任何内容。", + "features.eval.revert.body": "当 evaluation.auto_revert: true 时,loss、benchmark 或安全阈值不通过会让 ForgeLM 删除训练产物、以 3 退出并发出结构化事件记录。不会回滚到此前的 checkpoint — 不恢复任何内容。一个例外:未能产生结论的安全评估(分类器未加载、probes 缺失)会保留模型并以 0 退出。", "features.eval.trend.title": "趋势跟踪", "features.eval.trend.body": "跨运行的安全历史 (每个 output_dir 的 safety_trend.jsonl)。在某个类别跨过阈值之前看出它是否在悄悄漂移。", "features.eval.exit.title": "退出码", @@ -1446,7 +1446,7 @@ "features.ent.heading": "企业 & MLOps", "features.ent.body": "为 pipeline 构建:容器、webhook、结构化输出、气隙、model card。", "features.ent.webhook.title": "Webhook 通知", - "features.ent.webhook.body": "训练开始、成功、失败、自动丢弃产物时的 Slack、Teams、Discord 或任意 HTTP webhook 接收器的警报。结构化 payload — 接到你的事件工具。", + "features.ent.webhook.body": "训练开始、成功、失败、自动丢弃产物时发出 Slack 格式的 webhook 警报(Teams 和 Discord 需要中继)。结构化 payload — 接到你的事件工具。", "features.ent.exp.title": "实验跟踪", "features.ent.exp.body": "通过 report_to 接入 W&B、MLflow、TensorBoard。可插拔,不写死。用你团队已经在用的。", "features.ent.json.title": "JSON 输出模式", @@ -1678,7 +1678,7 @@ "wizard.step.operations.title": "Operations & advanced settings", "wizard.step.operations.name": "Operations", "wizard.step.operations.desc": "Notifications, external monitoring-plan metadata, synthetic data, and retention settings. Configure the ForgeLM options you need.", - "wizard.step.operations.tutorial": "These four blocks cover everything ForgeLM ships beyond the core training + compliance flow. Webhook posts a Slack/Teams/Discord alert when training starts, succeeds, or fails. Monitoring (Art. 72 / 17(1)(h)) records an external plan as metadata; ForgeLM does not export metrics, detect drift, send drift alerts, or schedule checks. Synthetic data generation lets a stronger teacher model produce extra training examples. Retention horizons set GDPR-compliant TTLs for the audit log and intermediate artefacts. These ForgeLM settings are optional; external legal obligations still apply.", + "wizard.step.operations.tutorial": "These four blocks cover everything ForgeLM ships beyond the core training + compliance flow. Webhook posts a Slack-format alert (Teams and Discord need a relay) when training starts, succeeds, or fails. Monitoring (Art. 72 / 17(1)(h)) records an external plan as metadata; ForgeLM does not export metrics, detect drift, send drift alerts, or schedule checks. Synthetic data generation lets a stronger teacher model produce extra training examples. Retention horizons record how long you intend to keep the audit log and intermediate artefacts; ForgeLM reports what is past them but deletes nothing. These ForgeLM settings are optional; external legal obligations still apply.", "wizard.risk.label": "EU AI Act risk classification", "wizard.risk.hint": "Pick the tier your deployment falls under. High-risk and unacceptable trigger mandatory audit + human-approval obligations.", "wizard.risk.unknown": "Unknown — pre-classification placeholder", @@ -1768,14 +1768,14 @@ "wizard.governance.dpia_completed.label": "DPIA completed (GDPR Art. 35)", "wizard.governance.dpia_completed.hint": "Data Protection Impact Assessment finished and documented for this dataset.", "wizard.webhook.title": "Notifications (webhook)", - "wizard.webhook.hint": "Send Slack / Teams / Discord alerts on training events.", + "wizard.webhook.hint": "Send Slack-format alerts on training events (Teams and Discord need a relay).", "wizard.webhook.url.label": "Webhook URL or env var", "wizard.webhook.url.hint": "Direct https URL or an env var name carrying it (recommended for secrets).", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... OR env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "You can paste the URL directly (fine for non-secret test channels) or use the env: prefix to point at an environment variable. The env-var route keeps the URL out of YAML and out of the audit log — required for production secrets.", "wizard.webhook.enabled.label": "Send webhook notifications", "wizard.webhook.enabled.hint": "When on, ForgeLM posts a webhook payload at the events you select. Off keeps the YAML free of any webhook block.", - "wizard.webhook.enabled.detail": "Notifications wire training events into your team chat (Slack / Teams / Discord) or any HTTPS endpoint that accepts JSON. Delivery failures are warnings in the process log only. The audit log records the underlying training events, not delivery attempts. Leave off if you don't want the YAML to carry webhook config at all.", + "wizard.webhook.enabled.detail": "Notifications wire training events into your team chat (Slack; Teams and Discord through a relay) or any HTTPS endpoint that accepts this JSON. Delivery failures are warnings in the process log only. The audit log records the underlying training events, not delivery attempts. Leave off if you don't want the YAML to carry webhook config at all.", "wizard.webhook.notify_start.label": "Notify on training start", "wizard.webhook.notify_success.label": "Notify on success", "wizard.webhook.notify_failure.label": "Notify on failure", @@ -1820,11 +1820,11 @@ "wizard.retention.ephemeral.hint": "How long to keep intermediate checkpoints, eval samples. Default 90.", "wizard.retention.raw_documents.label": "Raw documents (days)", "wizard.retention.raw_documents.hint": "How long to keep raw PDFs / DOCX after ingestion. Default 90.", - "wizard.retention.enforce.label": "Enforce automatic deletion", - "wizard.retention.enforce.hint": "When set, ForgeLM auto-deletes artefacts past their horizon. Off by default — enabling requires GDPR sign-off.", - "wizard.retention.enforce.log_only": "log_only — log violations to audit (default)", - "wizard.retention.enforce.warn": "warn_on_excess — log + stderr warning", - "wizard.retention.enforce.block": "block_on_excess — abort trainer (regulated CI)", + "wizard.retention.enforce.label": "Policy mode (reserved)", + "wizard.retention.enforce.hint": "Reserved: no ForgeLM command acts on this yet, so all three modes behave the same. Retention horizons are only reported by forgelm purge --check-policy; nothing is deleted automatically.", + "wizard.retention.enforce.log_only": "log_only — default (no command acts on it)", + "wizard.retention.enforce.warn": "warn_on_excess — accepted, behaves like log_only", + "wizard.retention.enforce.block": "block_on_excess — accepted, behaves like log_only", "wizard.step.review.title": "Review your config", "wizard.step.review.name": "Review", "wizard.step.review.desc": "Here's the YAML. Copy it, download it, and run.", @@ -1837,7 +1837,7 @@ "wizard.error.model.required": "You picked “Custom HuggingFace model” on the previous step, so this field is required. Enter the org/repo ID (e.g. meta-llama/Llama-3.1-8B-Instruct) or go back and pick one of the bundled presets.", "wizard.error.model.format": "HuggingFace Hub IDs use the org/repo shape (e.g. meta-llama/Llama-3.1-8B-Instruct). A bare name without a slash won’t resolve at training time.", "wizard.error.dataset.required": "Pick a dataset before continuing — without a path or HuggingFace name the trainer has nothing to learn from.", - "wizard.error.webhook.required": "You enabled notifications, so we need a webhook URL or env var to send them to. Either turn notifications off above or paste your Slack / Teams / Discord URL (or env:SLACK_WEBHOOK_URL for production secrets).", + "wizard.error.webhook.required": "You enabled notifications, so we need a webhook URL or env var to send them to. Either turn notifications off above or paste your Slack webhook URL (or a relay URL for Teams / Discord) (or env:SLACK_WEBHOOK_URL for production secrets).", "wizard.error.synthetic.teacher_model.required": "You enabled synthetic data generation, so we need a teacher model name (e.g. gpt-4o, claude-opus-4-7, or a local model path).", "wizard.error.synthetic.seed_file.required": "You enabled synthetic data generation, so we need a seed prompts file — the teacher will produce a response for each prompt in it (e.g. ./data/seed_prompts.jsonl).", "wizard.error.synthetic.api_key_env.required": "You enabled synthetic data generation, so we need the env var that carries the teacher API key — typically OPENAI_API_KEY or ANTHROPIC_API_KEY. Leave the synthetic toggle off if you don’t have one.", @@ -2076,7 +2076,7 @@ "wizard.step.operations.title": "Operasyon ve gelişmiş ayarlar", "wizard.step.operations.name": "Operasyon", "wizard.step.operations.desc": "Bildirimler, harici izleme planı metadata'sı, sentetik veri ve saklama ayarları. Gereken ForgeLM seçeneklerini yapılandır.", - "wizard.step.operations.tutorial": "Bu dört blok, ForgeLM'in temel eğitim + compliance akışının ötesinde sağladığı her şeyi kapsar. Webhook, eğitim başladığında / başarılı olduğunda / hata verdiğinde Slack/Teams/Discord bildirimi gönderir. İzleme (Madde 72 / 17(1)(h)) harici planı metadata olarak kaydeder; ForgeLM metrikleri dışa aktarmaz, drift tespit etmez, drift alarmı göndermez veya kontrol zamanlamaz. Sentetik veri üretimi, daha güçlü bir teacher modelin ek eğitim örnekleri üretmesini sağlar. Saklama ufukları audit log'u ve ara artefact'ları için GDPR uyumlu TTL'ler ayarlar. Bu ForgeLM ayarları isteğe bağlıdır; harici yasal yükümlülükler ayrıca geçerlidir.", + "wizard.step.operations.tutorial": "Bu dört blok, ForgeLM'in temel eğitim + compliance akışının ötesinde sağladığı her şeyi kapsar. Webhook, eğitim başladığında / başarılı olduğunda / hata verdiğinde Slack biçiminde bir bildirim gönderir (Teams ve Discord için relay gerekir). İzleme (Madde 72 / 17(1)(h)) harici planı metadata olarak kaydeder; ForgeLM metrikleri dışa aktarmaz, drift tespit etmez, drift alarmı göndermez veya kontrol zamanlamaz. Sentetik veri üretimi, daha güçlü bir teacher modelin ek eğitim örnekleri üretmesini sağlar. Saklama ufukları audit log'u ve ara artefact'ları ne kadar süre tutmayı planladığınızı kaydeder; ForgeLM ufku aşanı raporlar ama hiçbir şey silmez. Bu ForgeLM ayarları isteğe bağlıdır; harici yasal yükümlülükler ayrıca geçerlidir.", "wizard.risk.label": "EU AI Act risk sınıflandırması", "wizard.risk.hint": "Dağıtımın hangi seviyeye girdiğini seç. High-risk ve unacceptable, zorunlu denetim + insan onay yükümlülüklerini tetikler.", "wizard.risk.unknown": "Unknown — ön sınıflandırma yer tutucusu", @@ -2166,14 +2166,14 @@ "wizard.governance.dpia_completed.label": "DPIA tamamlandı (GDPR Madde 35)", "wizard.governance.dpia_completed.hint": "Bu dataset için Veri Koruma Etki Değerlendirmesi tamamlanmış ve belgelenmiş.", "wizard.webhook.title": "Bildirimler (webhook)", - "wizard.webhook.hint": "Eğitim olaylarında Slack / Teams / Discord bildirimi gönder.", + "wizard.webhook.hint": "Eğitim olaylarında Slack biçiminde bildirim gönder (Teams ve Discord için relay gerekir).", "wizard.webhook.url.label": "Webhook URL veya env değişkeni", "wizard.webhook.url.hint": "Doğrudan https URL veya onu taşıyan env değişkeni adı (gizliler için önerilen).", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... VEYA env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "URL'yi doğrudan yapıştırabilirsin (gizli olmayan test kanalları için iyi) ya da bir ortam değişkenine işaret etmek için env: önekini kullan. Env-var yolu URL'yi YAML'den ve audit log'tan dışarıda tutar — production gizliler için zorunlu.", "wizard.webhook.enabled.label": "Webhook bildirimleri gönder", "wizard.webhook.enabled.hint": "Açıkken, ForgeLM seçtiğin olaylarda bir webhook payload gönderir. Kapalıyken YAML herhangi bir webhook bloğu içermez.", - "wizard.webhook.enabled.detail": "Bildirimler eğitim olaylarını ekip sohbetine (Slack / Teams / Discord) veya JSON kabul eden herhangi bir HTTPS endpoint'ine bağlar. Teslimat hataları yalnızca süreç günlüğüne uyarı olarak yazılır. Audit log teslimat denemelerini değil, ilgili eğitim olaylarını kaydeder. YAML'in webhook config taşımasını hiç istemiyorsan kapalı bırak.", + "wizard.webhook.enabled.detail": "Bildirimler eğitim olaylarını ekip sohbetine (Slack; Teams ve Discord için relay üzerinden) veya bu JSON'u kabul eden herhangi bir HTTPS endpoint'ine bağlar. Teslimat hataları yalnızca süreç günlüğüne uyarı olarak yazılır. Audit log teslimat denemelerini değil, ilgili eğitim olaylarını kaydeder. YAML'in webhook config taşımasını hiç istemiyorsan kapalı bırak.", "wizard.webhook.notify_start.label": "Eğitim başında bildir", "wizard.webhook.notify_success.label": "Başarıda bildir", "wizard.webhook.notify_failure.label": "Hatada bildir", @@ -2218,11 +2218,11 @@ "wizard.retention.ephemeral.hint": "Ara checkpoint'ler, eval örnekleri ne kadar tutulsun. Varsayılan 90.", "wizard.retention.raw_documents.label": "Ham dokümanlar (gün)", "wizard.retention.raw_documents.hint": "Ingestion sonrası ham PDF / DOCX ne kadar tutulsun. Varsayılan 90.", - "wizard.retention.enforce.label": "Otomatik silmeyi zorunlu kıl", - "wizard.retention.enforce.hint": "Açıkken ForgeLM ufku aşan artefact'ları otomatik siler. Varsayılan kapalı — açmak GDPR onayı gerektirir.", - "wizard.retention.enforce.log_only": "log_only — ihlalleri audit'e logla (varsayılan)", - "wizard.retention.enforce.warn": "warn_on_excess — log + stderr uyarısı", - "wizard.retention.enforce.block": "block_on_excess — trainer'ı durdur (regüle CI)", + "wizard.retention.enforce.label": "Politika modu (ayrılmış)", + "wizard.retention.enforce.hint": "Ayrılmış: hiçbir ForgeLM komutu bunu henüz kullanmaz, bu yüzden üç mod da aynı davranır. Saklama ufukları yalnızca forgelm purge --check-policy ile raporlanır; hiçbir şey otomatik silinmez.", + "wizard.retention.enforce.log_only": "log_only — varsayılan (hiçbir komut kullanmaz)", + "wizard.retention.enforce.warn": "warn_on_excess — kabul edilir, log_only gibi davranır", + "wizard.retention.enforce.block": "block_on_excess — kabul edilir, log_only gibi davranır", "wizard.step.review.title": "Config'ini gözden geçir", "wizard.step.review.name": "Gözden geçir", "wizard.step.review.desc": "İşte YAML. Kopyala, indir ve çalıştır.", @@ -2235,7 +2235,7 @@ "wizard.error.model.required": "Önceki adımda “Custom HuggingFace model” seçtin, bu yüzden bu alan zorunlu. org/repo formatında bir ID gir (örn. meta-llama/Llama-3.1-8B-Instruct) ya da geri dönüp hazır bir preset seç.", "wizard.error.model.format": "HuggingFace Hub ID'leri org/repo formatında olmalı (örn. meta-llama/Llama-3.1-8B-Instruct). Slash içermeyen düz bir isim eğitim sırasında bulunamaz.", "wizard.error.dataset.required": "Devam etmeden önce bir dataset seç — yol veya HuggingFace ismi olmadan trainer'ın öğreneceği bir şey yok.", - "wizard.error.webhook.required": "Bildirimleri açtın, yani göndereceğimiz bir webhook URL'si veya env değişkeni gerekli. Ya yukarıdan bildirimleri kapat ya da Slack / Teams / Discord URL'ini yapıştır (production gizliler için env:SLACK_WEBHOOK_URL kullan).", + "wizard.error.webhook.required": "Bildirimleri açtın, yani göndereceğimiz bir webhook URL'si veya env değişkeni gerekli. Ya yukarıdan bildirimleri kapat ya da Slack webhook URL'ini (veya Teams / Discord için bir relay URL'ini) yapıştır (production gizliler için env:SLACK_WEBHOOK_URL kullan).", "wizard.error.synthetic.teacher_model.required": "Sentetik veri üretimini açtın, yani bir teacher model adı gerekli (örn. gpt-4o, claude-opus-4-7 veya yerel model yolu).", "wizard.error.synthetic.seed_file.required": "Sentetik veri üretimini açtın, yani bir seed prompts dosyası gerekli — teacher her prompt için yanıt üretir (örn. ./data/seed_prompts.jsonl).", "wizard.error.synthetic.api_key_env.required": "Sentetik veri üretimini açtın, yani teacher API anahtarını taşıyan env değişkeni gerekli — genellikle OPENAI_API_KEY veya ANTHROPIC_API_KEY. Bir tane yoksa sentetik anahtarını kapalı bırak.", @@ -2474,7 +2474,7 @@ "wizard.step.operations.title": "Betrieb & erweiterte Einstellungen", "wizard.step.operations.name": "Betrieb", "wizard.step.operations.desc": "Benachrichtigungen, Metadaten externer Monitoring-Pläne, synthetische Daten und Aufbewahrung. Konfigurieren Sie die benötigten ForgeLM-Optionen.", - "wizard.step.operations.tutorial": "Diese vier Blöcke decken alles ab, was ForgeLM jenseits des Kern-Training + Compliance-Flows liefert. Webhook sendet Slack/Teams/Discord-Alerts bei Trainingsstart/Erfolg/Fehler. Monitoring (Art. 72 / 17(1)(h)) erfasst einen externen Plan als Metadaten; ForgeLM exportiert keine Metriken, erkennt keine Drift, sendet keine Drift-Warnungen und plant keine Kontrollen. Synthetische Datengenerierung lässt ein stärkeres Teacher-Modell zusätzliche Trainingsbeispiele produzieren. Retentions-Horizonte setzen DSGVO-konforme TTLs für Audit-Log und ephemere Artefakte. Diese ForgeLM-Einstellungen sind optional; externe gesetzliche Pflichten gelten weiterhin.", + "wizard.step.operations.tutorial": "Diese vier Blöcke decken alles ab, was ForgeLM jenseits des Kern-Training + Compliance-Flows liefert. Webhook sendet bei Trainingsstart/Erfolg/Fehler einen Slack-formatierten Alert (Teams und Discord brauchen ein Relay). Monitoring (Art. 72 / 17(1)(h)) erfasst einen externen Plan als Metadaten; ForgeLM exportiert keine Metriken, erkennt keine Drift, sendet keine Drift-Warnungen und plant keine Kontrollen. Synthetische Datengenerierung lässt ein stärkeres Teacher-Modell zusätzliche Trainingsbeispiele produzieren. Retentions-Horizonte halten fest, wie lange Sie Audit-Log und ephemere Artefakte aufbewahren wollen; ForgeLM meldet, was darüber hinaus liegt, löscht aber nichts. Diese ForgeLM-Einstellungen sind optional; externe gesetzliche Pflichten gelten weiterhin.", "wizard.risk.label": "EU-AI-Act-Risikoklassifizierung", "wizard.risk.hint": "Wählen Sie die Klasse, unter die Ihr Deployment fällt. High-risk und unacceptable lösen verpflichtende Audit + Human-Approval-Pflichten aus.", "wizard.risk.unknown": "Unknown — Vorklassifizierungs-Platzhalter", @@ -2564,14 +2564,14 @@ "wizard.governance.dpia_completed.label": "DSFA abgeschlossen (DSGVO Art. 35)", "wizard.governance.dpia_completed.hint": "Datenschutz-Folgenabschätzung für diesen Datensatz fertig und dokumentiert.", "wizard.webhook.title": "Benachrichtigungen (Webhook)", - "wizard.webhook.hint": "Slack/Teams/Discord-Alerts bei Trainings-Events senden.", + "wizard.webhook.hint": "Slack-formatierte Alerts bei Trainings-Events senden (Teams und Discord brauchen ein Relay).", "wizard.webhook.url.label": "Webhook-URL oder Env-Variable", "wizard.webhook.url.hint": "Direkte HTTPS-URL oder Env-Variable die sie trägt (für Secrets empfohlen).", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... ODER env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "URL kann direkt eingefügt werden, oder via env:-Präfix auf eine Umgebungsvariable verweisen. Env-Variante hält die URL aus YAML + Audit-Log.", "wizard.webhook.enabled.label": "Webhook-Benachrichtigungen senden", "wizard.webhook.enabled.hint": "Wenn aktiv, sendet ForgeLM eine Webhook-Payload bei den ausgewählten Events. Aus = das YAML enthält keinen Webhook-Block.", - "wizard.webhook.enabled.detail": "Benachrichtigungen leiten Trainings-Events in Ihren Team-Chat (Slack / Teams / Discord) oder einen HTTPS-Endpoint, der JSON akzeptiert. Zustellfehler erscheinen nur als Warnungen im Prozesslog. Das Audit-Log erfasst die zugrunde liegenden Trainingsereignisse, keine Zustellversuche. Aus lassen, wenn das YAML keinerlei Webhook-Config tragen soll.", + "wizard.webhook.enabled.detail": "Benachrichtigungen leiten Trainings-Events in Ihren Team-Chat (Slack; Teams und Discord über ein Relay) oder einen HTTPS-Endpoint, der dieses JSON akzeptiert. Zustellfehler erscheinen nur als Warnungen im Prozesslog. Das Audit-Log erfasst die zugrunde liegenden Trainingsereignisse, keine Zustellversuche. Aus lassen, wenn das YAML keinerlei Webhook-Config tragen soll.", "wizard.webhook.notify_start.label": "Bei Trainingsstart benachrichtigen", "wizard.webhook.notify_success.label": "Bei Erfolg benachrichtigen", "wizard.webhook.notify_failure.label": "Bei Fehler benachrichtigen", @@ -2612,11 +2612,11 @@ "wizard.retention.ephemeral.hint": "Wie lange Zwischen-Checkpoints, Eval-Samples behalten. Default 90.", "wizard.retention.raw_documents.label": "Rohdokumente (Tage)", "wizard.retention.raw_documents.hint": "Wie lange rohe PDFs/DOCX nach Ingestion behalten. Default 90.", - "wizard.retention.enforce.label": "Automatische Löschung erzwingen", - "wizard.retention.enforce.hint": "Wenn aktiv löscht ForgeLM Artefakte nach ihrem Horizont automatisch. Default aus.", - "wizard.retention.enforce.log_only": "log_only — Verstöße ins Audit loggen (Default)", - "wizard.retention.enforce.warn": "warn_on_excess — log + stderr-Warnung", - "wizard.retention.enforce.block": "block_on_excess — Trainer abbrechen (regulierte CI)", + "wizard.retention.enforce.label": "Richtlinienmodus (reserviert)", + "wizard.retention.enforce.hint": "Reserviert: Noch wertet kein ForgeLM-Befehl dies aus, daher verhalten sich alle drei Modi gleich. Aufbewahrungshorizonte meldet nur forgelm purge --check-policy; nichts wird automatisch gelöscht.", + "wizard.retention.enforce.log_only": "log_only — Default (kein Befehl wertet es aus)", + "wizard.retention.enforce.warn": "warn_on_excess — wird akzeptiert, verhält sich wie log_only", + "wizard.retention.enforce.block": "block_on_excess — wird akzeptiert, verhält sich wie log_only", "wizard.step.review.title": "Konfiguration prüfen", "wizard.step.review.name": "Prüfen", "wizard.step.review.desc": "Hier ist das YAML. Kopieren, herunterladen und ausführen.", @@ -2629,7 +2629,7 @@ "wizard.error.model.required": "Sie haben im vorherigen Schritt „Custom HuggingFace-Modell“ gewählt — deshalb ist dieses Feld Pflicht. Geben Sie eine org/repo-ID ein (z. B. meta-llama/Llama-3.1-8B-Instruct) oder gehen Sie zurück und wählen Sie ein Preset.", "wizard.error.model.format": "HuggingFace-Hub-IDs verwenden das Format org/repo (z. B. meta-llama/Llama-3.1-8B-Instruct). Ein Name ohne Schrägstrich wird zur Trainingszeit nicht aufgelöst.", "wizard.error.dataset.required": "Wählen Sie ein Dataset, bevor Sie fortfahren — ohne Pfad oder HuggingFace-Namen hat der Trainer nichts zum Lernen.", - "wizard.error.webhook.required": "Sie haben Benachrichtigungen aktiviert — wir brauchen also eine Webhook-URL oder Env-Variable. Schalten Sie Benachrichtigungen oben aus oder fügen Sie Ihre Slack-/Teams-/Discord-URL ein (für Produktion env:SLACK_WEBHOOK_URL).", + "wizard.error.webhook.required": "Sie haben Benachrichtigungen aktiviert — wir brauchen also eine Webhook-URL oder Env-Variable. Schalten Sie Benachrichtigungen oben aus oder fügen Sie Ihre Slack-Webhook-URL (oder eine Relay-URL für Teams / Discord) ein (für Produktion env:SLACK_WEBHOOK_URL).", "wizard.error.synthetic.teacher_model.required": "Sie haben synthetische Datengenerierung aktiviert — wir brauchen einen Teacher-Modellnamen (z. B. gpt-4o, claude-opus-4-7 oder lokaler Pfad).", "wizard.error.synthetic.seed_file.required": "Sie haben synthetische Datengenerierung aktiviert — wir brauchen eine Seed-Prompts-Datei (z. B. ./data/seed_prompts.jsonl); der Teacher generiert pro Prompt eine Antwort.", "wizard.error.synthetic.api_key_env.required": "Sie haben synthetische Datengenerierung aktiviert — wir brauchen die Env-Variable mit dem Teacher-API-Key (üblich: OPENAI_API_KEY oder ANTHROPIC_API_KEY).", @@ -2868,7 +2868,7 @@ "wizard.step.operations.title": "Opérations et paramètres avancés", "wizard.step.operations.name": "Opérations", "wizard.step.operations.desc": "Notifications, métadonnées du plan de surveillance externe, données synthétiques et conservation. Configurez les options ForgeLM nécessaires.", - "wizard.step.operations.tutorial": "Ces quatre blocs couvrent tout ce que ForgeLM offre au-delà du flux core entraînement + compliance. Webhook envoie une alerte Slack/Teams/Discord au démarrage / succès / échec. La surveillance (Art. 72 / 17(1)(h)) enregistre un plan externe comme métadonnées ; ForgeLM n'exporte pas de métriques, ne détecte pas de dérive, n'envoie pas d'alertes de dérive et ne programme aucun contrôle. La génération synthétique laisse un modèle teacher plus fort produire des exemples supplémentaires. Les horizons de rétention fixent des TTL conformes RGPD pour audit log et artefacts éphémères. Ces paramètres ForgeLM sont facultatifs ; les obligations légales externes restent applicables.", + "wizard.step.operations.tutorial": "Ces quatre blocs couvrent tout ce que ForgeLM offre au-delà du flux core entraînement + compliance. Webhook envoie une alerte au format Slack au démarrage / succès / échec (Teams et Discord nécessitent un relais). La surveillance (Art. 72 / 17(1)(h)) enregistre un plan externe comme métadonnées ; ForgeLM n'exporte pas de métriques, ne détecte pas de dérive, n'envoie pas d'alertes de dérive et ne programme aucun contrôle. La génération synthétique laisse un modèle teacher plus fort produire des exemples supplémentaires. Les horizons de rétention consignent combien de temps vous comptez conserver l'audit log et les artefacts éphémères ; ForgeLM signale ce qui les dépasse mais ne supprime rien. Ces paramètres ForgeLM sont facultatifs ; les obligations légales externes restent applicables.", "wizard.risk.label": "Classification de risque EU AI Act", "wizard.risk.hint": "Choisissez le niveau dans lequel votre déploiement tombe. High-risk et unacceptable déclenchent des obligations d'audit + approbation humaine obligatoires.", "wizard.risk.unknown": "Unknown — placeholder de pré-classification", @@ -2958,14 +2958,14 @@ "wizard.governance.dpia_completed.label": "AIPD complétée (RGPD Art. 35)", "wizard.governance.dpia_completed.hint": "Analyse d'Impact Protection Données terminée et documentée pour ce dataset.", "wizard.webhook.title": "Notifications (webhook)", - "wizard.webhook.hint": "Envoyez des alertes Slack/Teams/Discord lors des événements d'entraînement.", + "wizard.webhook.hint": "Envoyez des alertes au format Slack lors des événements d'entraînement (Teams et Discord nécessitent un relais).", "wizard.webhook.url.label": "URL webhook ou variable env", "wizard.webhook.url.hint": "URL HTTPS directe ou variable env la portant (recommandé pour secrets).", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... OU env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "Vous pouvez coller URL directement, ou utiliser préfixe env: pour pointer une variable env.", "wizard.webhook.enabled.label": "Envoyer des notifications webhook", "wizard.webhook.enabled.hint": "Quand activé, ForgeLM publie un payload webhook aux événements sélectionnés. Désactivé = le YAML ne contient aucun bloc webhook.", - "wizard.webhook.enabled.detail": "Les notifications connectent les événements d'entraînement à votre chat d'équipe (Slack / Teams / Discord) ou tout endpoint HTTPS acceptant du JSON. Les échecs de livraison sont uniquement des avertissements dans le journal du processus. L'audit log enregistre les événements d'entraînement sous-jacents, pas les tentatives de livraison. Laissez désactivé si vous ne voulez aucune config webhook dans le YAML.", + "wizard.webhook.enabled.detail": "Les notifications connectent les événements d'entraînement à votre chat d'équipe (Slack ; Teams et Discord via un relais) ou tout endpoint HTTPS acceptant ce JSON. Les échecs de livraison sont uniquement des avertissements dans le journal du processus. L'audit log enregistre les événements d'entraînement sous-jacents, pas les tentatives de livraison. Laissez désactivé si vous ne voulez aucune config webhook dans le YAML.", "wizard.webhook.notify_start.label": "Notifier au démarrage", "wizard.webhook.notify_success.label": "Notifier au succès", "wizard.webhook.notify_failure.label": "Notifier à l'échec", @@ -3006,11 +3006,11 @@ "wizard.retention.ephemeral.hint": "Combien de temps garder checkpoints intermédiaires, samples eval. Default 90.", "wizard.retention.raw_documents.label": "Documents bruts (jours)", "wizard.retention.raw_documents.hint": "Combien de temps garder PDFs/DOCX bruts après ingestion. Default 90.", - "wizard.retention.enforce.label": "Forcer suppression automatique", - "wizard.retention.enforce.hint": "Si actif, ForgeLM auto-supprime artefacts après horizon. Off par default.", - "wizard.retention.enforce.log_only": "log_only — logger violations à l'audit (défaut)", - "wizard.retention.enforce.warn": "warn_on_excess — log + avertissement stderr", - "wizard.retention.enforce.block": "block_on_excess — abandonner trainer (CI régulée)", + "wizard.retention.enforce.label": "Mode de politique (réservé)", + "wizard.retention.enforce.hint": "Réservé : aucune commande ForgeLM n'agit encore dessus, donc les trois modes se comportent de la même façon. Les horizons de rétention ne sont signalés que par forgelm purge --check-policy ; rien n'est supprimé automatiquement.", + "wizard.retention.enforce.log_only": "log_only — défaut (aucune commande n'en tient compte)", + "wizard.retention.enforce.warn": "warn_on_excess — accepté, se comporte comme log_only", + "wizard.retention.enforce.block": "block_on_excess — accepté, se comporte comme log_only", "wizard.step.review.title": "Revoir votre configuration", "wizard.step.review.name": "Revoir", "wizard.step.review.desc": "Voici le YAML. Copiez-le, téléchargez-le et exécutez.", @@ -3023,7 +3023,7 @@ "wizard.error.model.required": "Vous avez choisi « Modèle HuggingFace personnalisé » à l’étape précédente — ce champ est donc requis. Saisissez un ID org/repo (ex. meta-llama/Llama-3.1-8B-Instruct) ou revenez en arrière et choisissez un preset.", "wizard.error.model.format": "Les ID HuggingFace Hub utilisent le format org/repo (ex. meta-llama/Llama-3.1-8B-Instruct). Un nom sans barre oblique ne sera pas résolu lors de l’entraînement.", "wizard.error.dataset.required": "Choisissez un dataset avant de continuer — sans chemin ni nom HuggingFace, le trainer n’a rien à apprendre.", - "wizard.error.webhook.required": "Vous avez activé les notifications — il nous faut donc une URL webhook ou une variable d’env. Désactivez les notifications ci-dessus ou collez votre URL Slack / Teams / Discord (pour la production utilisez env:SLACK_WEBHOOK_URL).", + "wizard.error.webhook.required": "Vous avez activé les notifications — il nous faut donc une URL webhook ou une variable d’env. Désactivez les notifications ci-dessus ou collez votre URL de webhook Slack (ou une URL de relais pour Teams / Discord) (pour la production utilisez env:SLACK_WEBHOOK_URL).", "wizard.error.synthetic.teacher_model.required": "Vous avez activé la génération de données synthétiques — il nous faut un nom de modèle teacher (ex. gpt-4o, claude-opus-4-7 ou chemin local).", "wizard.error.synthetic.seed_file.required": "Vous avez activé la génération de données synthétiques — il nous faut un fichier de seed prompts (ex. ./data/seed_prompts.jsonl) ; le teacher produira une réponse par prompt.", "wizard.error.synthetic.api_key_env.required": "Vous avez activé la génération de données synthétiques — il nous faut la variable d’env qui porte la clé API du teacher (généralement OPENAI_API_KEY ou ANTHROPIC_API_KEY).", @@ -3262,7 +3262,7 @@ "wizard.step.operations.title": "Operaciones y ajustes avanzados", "wizard.step.operations.name": "Operaciones", "wizard.step.operations.desc": "Notificaciones, metadatos del plan de monitoreo externo, datos sintéticos y retención. Configure las opciones de ForgeLM que necesite.", - "wizard.step.operations.tutorial": "Estos cuatro bloques cubren todo lo que ForgeLM ofrece más allá del flujo central de entrenamiento + compliance. Webhook envía alerta Slack/Teams/Discord en inicio / éxito / fallo. El monitoreo (Art. 72 / 17(1)(h)) registra un plan externo como metadatos; ForgeLM no exporta métricas, no detecta deriva, no envía alertas de deriva ni programa comprobaciones. La generación sintética permite a un modelo teacher más fuerte producir ejemplos adicionales. Los horizontes de retención fijan TTLs conformes a RGPD para audit log y artefactos efímeros. Estas opciones de ForgeLM son opcionales; las obligaciones legales externas siguen vigentes.", + "wizard.step.operations.tutorial": "Estos cuatro bloques cubren todo lo que ForgeLM ofrece más allá del flujo central de entrenamiento + compliance. Webhook envía una alerta en formato Slack en inicio / éxito / fallo (Teams y Discord requieren un relay). El monitoreo (Art. 72 / 17(1)(h)) registra un plan externo como metadatos; ForgeLM no exporta métricas, no detecta deriva, no envía alertas de deriva ni programa comprobaciones. La generación sintética permite a un modelo teacher más fuerte producir ejemplos adicionales. Los horizontes de retención registran cuánto tiempo piensa conservar el audit log y los artefactos efímeros; ForgeLM informa de lo que los supera pero no elimina nada. Estas opciones de ForgeLM son opcionales; las obligaciones legales externas siguen vigentes.", "wizard.risk.label": "Clasificación de riesgo EU AI Act", "wizard.risk.hint": "Elija el nivel en el que cae su despliegue. High-risk y unacceptable activan obligaciones obligatorias de auditoría + aprobación humana.", "wizard.risk.unknown": "Unknown — placeholder de pre-clasificación", @@ -3352,14 +3352,14 @@ "wizard.governance.dpia_completed.label": "EIPD completada (RGPD Art. 35)", "wizard.governance.dpia_completed.hint": "Evaluación de Impacto sobre Protección de Datos terminada para este dataset.", "wizard.webhook.title": "Notificaciones (webhook)", - "wizard.webhook.hint": "Envíe alertas Slack/Teams/Discord en eventos de entrenamiento.", + "wizard.webhook.hint": "Envíe alertas en formato Slack en eventos de entrenamiento (Teams y Discord requieren un relay).", "wizard.webhook.url.label": "URL webhook o variable env", "wizard.webhook.url.hint": "URL HTTPS directa o variable env que la lleva (recomendado para secretos).", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... O env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "Puede pegar URL directamente, o usar prefijo env: para apuntar a una variable de entorno.", "wizard.webhook.enabled.label": "Enviar notificaciones webhook", "wizard.webhook.enabled.hint": "Cuando está activo, ForgeLM publica un payload webhook en los eventos seleccionados. Apagado = el YAML no contiene ningún bloque webhook.", - "wizard.webhook.enabled.detail": "Las notificaciones conectan eventos de entrenamiento a su chat de equipo (Slack / Teams / Discord) o cualquier endpoint HTTPS que acepte JSON. Los fallos de entrega solo se registran como advertencias en el registro del proceso. El audit log registra los eventos de entrenamiento subyacentes, no los intentos de entrega. Déjelo apagado si no quiere ninguna config de webhook en el YAML.", + "wizard.webhook.enabled.detail": "Las notificaciones conectan eventos de entrenamiento a su chat de equipo (Slack; Teams y Discord mediante un relay) o cualquier endpoint HTTPS que acepte este JSON. Los fallos de entrega solo se registran como advertencias en el registro del proceso. El audit log registra los eventos de entrenamiento subyacentes, no los intentos de entrega. Déjelo apagado si no quiere ninguna config de webhook en el YAML.", "wizard.webhook.notify_start.label": "Notificar al inicio", "wizard.webhook.notify_success.label": "Notificar en éxito", "wizard.webhook.notify_failure.label": "Notificar en fallo", @@ -3400,11 +3400,11 @@ "wizard.retention.ephemeral.hint": "Cuánto guardar checkpoints intermedios, samples eval. Default 90.", "wizard.retention.raw_documents.label": "Documentos crudos (días)", "wizard.retention.raw_documents.hint": "Cuánto guardar PDFs/DOCX crudos tras ingestion. Default 90.", - "wizard.retention.enforce.label": "Forzar eliminación automática", - "wizard.retention.enforce.hint": "Si activo, ForgeLM auto-elimina artefactos tras su horizonte. Default off.", - "wizard.retention.enforce.log_only": "log_only — registrar violaciones en audit (default)", - "wizard.retention.enforce.warn": "warn_on_excess — log + advertencia stderr", - "wizard.retention.enforce.block": "block_on_excess — abortar trainer (CI regulada)", + "wizard.retention.enforce.label": "Modo de política (reservado)", + "wizard.retention.enforce.hint": "Reservado: ningún comando de ForgeLM actúa aún sobre esto, así que los tres modos se comportan igual. Los horizontes de retención solo los informa forgelm purge --check-policy; nada se elimina automáticamente.", + "wizard.retention.enforce.log_only": "log_only — default (ningún comando lo usa)", + "wizard.retention.enforce.warn": "warn_on_excess — aceptado, se comporta como log_only", + "wizard.retention.enforce.block": "block_on_excess — aceptado, se comporta como log_only", "wizard.step.review.title": "Revise su configuración", "wizard.step.review.name": "Revisar", "wizard.step.review.desc": "Aquí está el YAML. Cópielo, descárguelo y ejecútelo.", @@ -3417,7 +3417,7 @@ "wizard.error.model.required": "Eligió “Modelo HuggingFace personalizado” en el paso anterior, por lo que este campo es obligatorio. Ingrese un ID org/repo (ej. meta-llama/Llama-3.1-8B-Instruct) o regrese y elija un preset.", "wizard.error.model.format": "Los ID de HuggingFace Hub usan el formato org/repo (ej. meta-llama/Llama-3.1-8B-Instruct). Un nombre sin barra no se resolverá al entrenar.", "wizard.error.dataset.required": "Elija un dataset antes de continuar — sin una ruta o nombre de HuggingFace, el trainer no tiene de qué aprender.", - "wizard.error.webhook.required": "Activó las notificaciones, así que necesitamos una URL de webhook o variable de entorno. Desactive las notificaciones arriba o pegue su URL de Slack / Teams / Discord (para producción use env:SLACK_WEBHOOK_URL).", + "wizard.error.webhook.required": "Activó las notificaciones, así que necesitamos una URL de webhook o variable de entorno. Desactive las notificaciones arriba o pegue su URL de webhook de Slack (o una URL de relay para Teams / Discord) (para producción use env:SLACK_WEBHOOK_URL).", "wizard.error.synthetic.teacher_model.required": "Activó la generación de datos sintéticos, así que necesitamos un nombre de modelo teacher (ej. gpt-4o, claude-opus-4-7 o ruta local).", "wizard.error.synthetic.seed_file.required": "Activó la generación de datos sintéticos, así que necesitamos un archivo de seed prompts (ej. ./data/seed_prompts.jsonl); el teacher producirá una respuesta por cada prompt.", "wizard.error.synthetic.api_key_env.required": "Activó la generación de datos sintéticos, así que necesitamos la variable de entorno con la clave API del teacher (típicamente OPENAI_API_KEY o ANTHROPIC_API_KEY).", @@ -3656,7 +3656,7 @@ "wizard.step.operations.title": "运维与高级设置", "wizard.step.operations.name": "运维", "wizard.step.operations.desc": "通知、外部监控计划元数据、合成数据和保留设置。配置所需的 ForgeLM 选项。", - "wizard.step.operations.tutorial": "这四个区块涵盖了 ForgeLM 在核心训练 + 合规流程之外提供的一切。Webhook 在训练开始 / 成功 / 失败时发送 Slack/Teams/Discord 警报。监控（第 72 条及第 17(1)(h) 条）仅将外部计划记录为元数据；ForgeLM 不导出指标、检测漂移、发送漂移警报或安排检查。合成数据生成让更强的 teacher 模型产生额外训练样本。保留期限为审计日志和临时 artefact 设置符合 GDPR 的 TTL。这些 ForgeLM 设置是可选的；外部法律义务仍然适用。", + "wizard.step.operations.tutorial": "这四个区块涵盖了 ForgeLM 在核心训练 + 合规流程之外提供的一切。Webhook 在训练开始 / 成功 / 失败时发送 Slack 格式的警报（Teams 和 Discord 需要中继）。监控（第 72 条及第 17(1)(h) 条）仅将外部计划记录为元数据；ForgeLM 不导出指标、检测漂移、发送漂移警报或安排检查。合成数据生成让更强的 teacher 模型产生额外训练样本。保留期限记录你打算保留审计日志和临时 artefact 的时长；ForgeLM 会报告超期的内容，但不会删除任何内容。这些 ForgeLM 设置是可选的；外部法律义务仍然适用。", "wizard.risk.label": "EU AI Act 风险分类", "wizard.risk.hint": "选择你的部署所属的等级。High-risk 和 unacceptable 触发强制审计 + 人工批准义务。", "wizard.risk.unknown": "Unknown — 预分类占位符", @@ -3746,14 +3746,14 @@ "wizard.governance.dpia_completed.label": "DPIA 已完成(GDPR 第 35 条)", "wizard.governance.dpia_completed.hint": "本数据集的数据保护影响评估已完成并记录。", "wizard.webhook.title": "通知（webhook）", - "wizard.webhook.hint": "在训练事件发送 Slack/Teams/Discord 警报。", + "wizard.webhook.hint": "在训练事件发送 Slack 格式的警报（Teams 和 Discord 需要中继）。", "wizard.webhook.url.label": "Webhook URL 或环境变量", "wizard.webhook.url.hint": "直接 https URL 或承载它的环境变量名（secret 推荐）。", "wizard.webhook.url.placeholder": "https://hooks.slack.com/services/... 或 env:SLACK_WEBHOOK_URL", "wizard.webhook.url.detail": "你可以直接粘贴 URL，或使用 env: 前缀指向环境变量。", "wizard.webhook.enabled.label": "发送 webhook 通知", "wizard.webhook.enabled.hint": "启用后，ForgeLM 在你选择的事件发送 webhook payload。关闭 = YAML 不包含任何 webhook 块。", - "wizard.webhook.enabled.detail": "通知将训练事件连接到你的团队聊天（Slack / Teams / Discord）或任何接受 JSON 的 HTTPS 端点。发送失败仅在进程日志中记录为警告。审计日志记录对应的训练事件，不记录发送尝试。如果不想 YAML 中包含任何 webhook 配置，保持关闭。", + "wizard.webhook.enabled.detail": "通知将训练事件连接到你的团队聊天（Slack；Teams 和 Discord 需通过中继）或任何接受此 JSON 的 HTTPS 端点。发送失败仅在进程日志中记录为警告。审计日志记录对应的训练事件，不记录发送尝试。如果不想 YAML 中包含任何 webhook 配置，保持关闭。", "wizard.webhook.notify_start.label": "训练开始时通知", "wizard.webhook.notify_success.label": "成功时通知", "wizard.webhook.notify_failure.label": "失败时通知", @@ -3794,11 +3794,11 @@ "wizard.retention.ephemeral.hint": "中间 checkpoint、eval 样本保留多久。默认 90。", "wizard.retention.raw_documents.label": "原始文档(天)", "wizard.retention.raw_documents.hint": "ingestion 后原始 PDF/DOCX 保留多久。默认 90。", - "wizard.retention.enforce.label": "强制自动删除", - "wizard.retention.enforce.hint": "启用后,ForgeLM 在期限后自动删除 artefact。默认关闭。", - "wizard.retention.enforce.log_only": "log_only — 将违规记录到 audit(默认)", - "wizard.retention.enforce.warn": "warn_on_excess — log + stderr 警告", - "wizard.retention.enforce.block": "block_on_excess — 中止 trainer(受监管 CI)", + "wizard.retention.enforce.label": "策略模式（保留）", + "wizard.retention.enforce.hint": "保留项：目前没有任何 ForgeLM 命令使用它，因此三种模式行为相同。保留期限仅由 forgelm purge --check-policy 报告；不会自动删除任何内容。", + "wizard.retention.enforce.log_only": "log_only — 默认（没有命令使用它）", + "wizard.retention.enforce.warn": "warn_on_excess — 可接受,行为与 log_only 相同", + "wizard.retention.enforce.block": "block_on_excess — 可接受,行为与 log_only 相同", "wizard.step.review.title": "审查你的配置", "wizard.step.review.name": "审查", "wizard.step.review.desc": "这是 YAML。复制、下载并运行。", @@ -3811,7 +3811,7 @@ "wizard.error.model.required": "你在上一步选择了\"自定义 HuggingFace 模型\",所以此字段为必填。输入 org/repo 格式的 ID(如 meta-llama/Llama-3.1-8B-Instruct)或返回上一步选择一个预设。", "wizard.error.model.format": "HuggingFace Hub ID 使用 org/repo 格式(如 meta-llama/Llama-3.1-8B-Instruct)。不带斜杠的纯名称在训练时无法解析。", "wizard.error.dataset.required": "继续之前请选择一个数据集 — 没有路径或 HuggingFace 名称,trainer 没有可学习的对象。", - "wizard.error.webhook.required": "你启用了通知,所以需要一个 webhook URL 或环境变量。要么关闭上方的通知,要么粘贴你的 Slack / Teams / Discord URL(生产环境密钥请使用 env:SLACK_WEBHOOK_URL)。", + "wizard.error.webhook.required": "你启用了通知,所以需要一个 webhook URL 或环境变量。要么关闭上方的通知,要么粘贴你的 Slack webhook URL（或用于 Teams / Discord 的中继 URL）(生产环境密钥请使用 env:SLACK_WEBHOOK_URL)。", "wizard.error.synthetic.teacher_model.required": "你启用了合成数据生成,所以需要一个 teacher 模型名(如 gpt-4o、claude-opus-4-7 或本地模型路径)。", "wizard.error.synthetic.seed_file.required": "你启用了合成数据生成,所以需要一个 seed prompts 文件(如 ./data/seed_prompts.jsonl);teacher 会为每个 prompt 生成一个回应。", "wizard.error.synthetic.api_key_env.required": "你启用了合成数据生成,所以需要承载 teacher API 密钥的环境变量(通常是 OPENAI_API_KEY 或 ANTHROPIC_API_KEY)。", @@ -3862,15 +3862,15 @@ "compliance.art13.title": "Transparency", "compliance.art13.body": "Auto-generated model card lists capabilities, intended use, training data summary, evaluation results, known limitations. Drop-in for HuggingFace Hub README.", "compliance.art14.title": "Human oversight", - "compliance.art14.body": "Optional evaluation.require_human_approval: true moves the trained checkpoint into a staging directory; forgelm approve / forgelm reject + forgelm approvals drive the dual-controlled sign-off, all stamped into the audit log.", + "compliance.art14.body": "Optional evaluation.require_human_approval: true moves the trained checkpoint into a staging directory; forgelm approve / forgelm reject + forgelm approvals record a human sign-off, with the approver identity stamped into the audit log. It supports design-time oversight documentation; oversight of the deployed system stays with the deployer.", "compliance.art15.title": "Accuracy & robustness", - "compliance.art15.body": "Benchmark gates per-task floor; if the post-train number drops below the floor, the run fails (exit 3). Cybersecurity: secrets / PII at ingest, dependency CVE awareness.", + "compliance.art15.body": "Benchmark gates per-task floor; with evaluation.auto_revert: true, a post-train number below the floor deletes the trained artefacts and the run exits 3. Cybersecurity: secrets / PII at ingest, dependency CVE awareness.", "compliance.art1617.title": "Conformity & QMS", "compliance.art1617.body": "QMS SOPs in docs/qms/ cover model training, data management, incident response, and change management — plus access control, encryption-at-rest, risk treatment, and Statement of Applicability. Conformity declaration template generated from the run's Annex IV.", "compliance.gdpr.title": "PII redaction at ingest", - "compliance.gdpr.body": "--pii-mask redacts emails, phones, credit cards (Luhn), IBAN, national IDs (TR/DE/FR/US-SSN) before chunks land in JSONL — Article 5(1)(c) data minimisation enforced upstream.", + "compliance.gdpr.body": "--pii-mask redacts emails, phones, credit cards (Luhn), IBAN, national IDs (TR/DE/FR/US-SSN) before chunks land in JSONL. It reduces what reaches training and supports data minimisation (Article 5(1)(c)); names and free text are not detected, so it is an aid, not a guarantee.", "compliance.gdpr15.title": "Right of access", - "compliance.gdpr15.body": "forgelm reverse-pii scans masked JSONL corpora for plaintext residuals or salted SHA-256 digests of a data-subject identifier and reports every row of every split where that subject appears, satisfying the access-request side of the GDPR's data-subject rights.", + "compliance.gdpr15.body": "forgelm reverse-pii scans masked JSONL corpora for plaintext residuals or salted SHA-256 digests of a data-subject identifier and reports every row of every split where that subject appears. It finds only what is literally in the files you scan: raw sources and the trained model are not searched, and handling an access request stays your process.", "compliance.gdpr17.title": "Right to erasure", "compliance.gdpr17.body": "forgelm purge deletes a data-subject's rows from training corpora and stamps data.erasure_requested + data.erasure_completed events into the append-only audit log so post-deletion runs are forensically linked to the erasure event.", "compliance.iso.eyebrow": "ISO 27001 · SOC 2 Type II", @@ -3879,7 +3879,7 @@ "compliance.iso.audit.title": "Audit trail", "compliance.iso.audit.body": "Append-only audit_log.jsonl per run with HMAC + SHA-256 hash chain + genesis manifest sidecar. forgelm verify-audit validates the chain end-to-end. HMAC integrity is active whenever FORGELM_AUDIT_SECRET is set; for audit-grade custody, keep that secret KMS-managed and rotated per output-dir lifecycle.", "compliance.iso.change.title":"Change control", - "compliance.iso.change.body": "Article 14 staging gate (forgelm approve / forgelm reject) plus human_approval.required/granted/rejected audit events plus forgelm verify-audit chain integrity — every model promotion is dual-controlled and forensically attributed.", + "compliance.iso.change.body": "Article 14 staging gate (forgelm approve / forgelm reject) plus human_approval.required/granted/rejected audit events plus forgelm verify-audit chain integrity — with the gate on, every model promotion is attributed to the recorded approver identity; keeping trainer and approver different people is the deployer's control.", "compliance.iso.lineage.title":"Data lineage", "compliance.iso.lineage.body":"data_provenance.json (SHA-256 fingerprint + size + mtime + HF Hub revision pin, graded by whether a pinned load stands behind it) and data_governance_report.json (collection method, annotation process, known biases, personal-data flag, DPIA completion) cover the lineage questions an auditor asks first.", "compliance.iso.supply.title":"Supply chain", @@ -3921,15 +3921,15 @@ "compliance.art13.title": "Şeffaflık", "compliance.art13.body": "Otomatik üretilen model card; yetenekleri, amaçlanan kullanımı, eğitim veri özetini, değerlendirme sonuçlarını, bilinen sınırlamaları listeler. HuggingFace Hub README için drop-in.", "compliance.art14.title": "İnsan gözetimi", - "compliance.art14.body": "Opsiyonel evaluation.require_human_approval: true eğitilen checkpoint'i staging dizinine taşır; forgelm approve / forgelm reject + forgelm approvals ikili-kontrollü onayı yürütür ve hepsi audit log'a damgalanır.", + "compliance.art14.body": "Opsiyonel evaluation.require_human_approval: true eğitilen checkpoint'i staging dizinine taşır; forgelm approve / forgelm reject + forgelm approvals bir insan onayını kaydeder, onaylayanın kimliği audit log'a damgalanır. Tasarım-zamanı gözetim dokümantasyonunu destekler; dağıtılmış sistemin gözetimi dağıtıcıda kalır.", "compliance.art15.title": "Doğruluk ve sağlamlık", - "compliance.art15.body": "Görev başı taban için benchmark kapısı; post-train sayısı tabanın altına düşerse koşu başarısız (exit 3). Siber güvenlik: ingest'te sırlar / PII, bağımlılık CVE farkındalığı.", + "compliance.art15.body": "Görev başı taban için benchmark kapısı; evaluation.auto_revert: true iken eğitim sonrası sayı tabanın altına düşerse eğitilen artefaktlar silinir ve koşu 3 ile çıkar. Siber güvenlik: ingest'te sırlar / PII, bağımlılık CVE farkındalığı.", "compliance.art1617.title": "Uygunluk ve QMS", "compliance.art1617.body": "docs/qms/'deki QMS SOP'ları model eğitimi, veri yönetimi, incident response ve değişiklik yönetimini kapsar — ayrıca erişim kontrolü, durağan-veri şifrelemesi, risk tedavisi ve Uygulanabilirlik Beyanı. Uygunluk beyanı şablonu koşunun Annex IV'ünden üretilir.", "compliance.gdpr.title": "Ingest'te PII redaksiyonu", - "compliance.gdpr.body": "--pii-mask chunk'lar JSONL'a düşmeden e-posta, telefon, kredi kartı (Luhn), IBAN, ulusal kimlik (TR/DE/FR/US-SSN) maskeler — Madde 5(1)(c) data minimisation upstream'de uygulanır.", + "compliance.gdpr.body": "--pii-mask chunk'lar JSONL'a düşmeden e-posta, telefon, kredi kartı (Luhn), IBAN, ulusal kimlik (TR/DE/FR/US-SSN) maskeler. Eğitime ulaşanı azaltır ve veri minimizasyonunu (Madde 5(1)(c)) destekler; isimler ve serbest metin tespit edilmez, bu yüzden bir yardımcıdır, garanti değildir.", "compliance.gdpr15.title": "Erişim hakkı", - "compliance.gdpr15.body": "forgelm reverse-pii maskelenmiş JSONL corpus'larında bir veri sahibi tanımlayıcısının düz-metin kalıntılarını veya tuzlanmış SHA-256 digest'lerini arar ve o öznenin geçtiği her split'in her satırını raporlar; GDPR veri sahibi haklarının erişim-talebi tarafını karşılar.", + "compliance.gdpr15.body": "forgelm reverse-pii maskelenmiş JSONL corpus'larında bir veri sahibi tanımlayıcısının düz-metin kalıntılarını veya tuzlanmış SHA-256 digest'lerini arar ve o öznenin geçtiği her split'in her satırını raporlar. Yalnızca taradığınız dosyalarda harfiyen bulunanı bulur: ham kaynaklar ve eğitilmiş model aranmaz; bir erişim talebini yürütmek sizin sürecinizde kalır.", "compliance.gdpr17.title": "Silme hakkı", "compliance.gdpr17.body": "forgelm purge bir veri sahibinin satırlarını eğitim corpus'undan siler ve append-only audit log'a data.erasure_requested + data.erasure_completed olaylarını damgalar; silme sonrası çalışmalar adli olarak silme olayına bağlanır.", "compliance.iso.eyebrow": "ISO 27001 · SOC 2 Type II", @@ -3938,7 +3938,7 @@ "compliance.iso.audit.title": "Audit izi", "compliance.iso.audit.body": "Her çalışma için HMAC + SHA-256 hash zinciri + genesis manifest sidecar ile append-only audit_log.jsonl. forgelm verify-audit zinciri uçtan uca doğrular. HMAC bütünlüğü FORGELM_AUDIT_SECRET ayarlandığı anda aktiftir; denetim-seviyesi koruma için bu sırrı KMS-yönetimli tutun ve output-dir yaşam-döngüsüne göre rotate edin.", "compliance.iso.change.title":"Değişiklik kontrolü", - "compliance.iso.change.body": "Madde 14 staging kapısı (forgelm approve / forgelm reject) artı human_approval.required/granted/rejected audit olayları artı forgelm verify-audit zincir bütünlüğü — her model terfi ikili kontrol altındadır ve adli olarak atfedilebilir.", + "compliance.iso.change.body": "Madde 14 staging kapısı (forgelm approve / forgelm reject) artı human_approval.required/granted/rejected audit olayları artı forgelm verify-audit zincir bütünlüğü — kapı açıkken her model terfi kaydedilen onaylayan kimliğine atfedilir; eğiticinin ve onaylayanın farklı kişiler olması dağıtıcının kontrolüdür.", "compliance.iso.lineage.title":"Veri kökenleri", "compliance.iso.lineage.body":"data_provenance.json (SHA-256 parmak izi + boyut + mtime + HF Hub revision pin — arkasında sabitlenmiş bir yükleme olup olmadığına göre derecelendirilir) ve data_governance_report.json (toplama yöntemi, anotasyon süreci, bilinen bias, kişisel-veri bayrağı, DPIA tamamlanması) — denetçinin önce sorduğu köken sorularını kapsar.", "compliance.iso.supply.title":"Tedarik zinciri", @@ -3980,13 +3980,13 @@ "compliance.art13.title": "Transparenz", "compliance.art13.body": "Automatisch generierte Model Card listet Fähigkeiten, beabsichtigten Einsatz, Trainingsdaten-Zusammenfassung, Evaluierungsergebnisse, bekannte Einschränkungen. Drop-in für das HuggingFace-Hub-README.", "compliance.art14.title": "Menschliche Aufsicht", - "compliance.art14.body": "Optionales evaluation.require_human_approval: true verschiebt den trainierten Checkpoint in ein Staging-Verzeichnis; forgelm approve / forgelm reject + forgelm approvals steuern die dual-kontrollierte Freigabe, alles im Audit-Log gestempelt.", + "compliance.art14.body": "Optionales evaluation.require_human_approval: true verschiebt den trainierten Checkpoint in ein Staging-Verzeichnis; forgelm approve / forgelm reject + forgelm approvals erfassen eine menschliche Freigabe, die Identität des Freigebenden wird im Audit-Log gestempelt. Das unterstützt die Dokumentation der Aufsicht zur Entwurfszeit; die Aufsicht über das ausgelieferte System bleibt beim Betreiber.", "compliance.art15.title": "Genauigkeit & Robustheit", - "compliance.art15.body": "Benchmark-Gate mit Schwellwert pro Task; fällt die Post-Train-Zahl unter den Schwellwert, schlägt der Lauf fehl (Exit 3). Cybersicherheit: Secrets / PII beim Ingest, Bewusstsein für Dependency-CVEs.", + "compliance.art15.body": "Benchmark-Gate mit Schwellwert pro Task; mit evaluation.auto_revert: true löscht eine Post-Train-Zahl unter dem Schwellwert die trainierten Artefakte, und der Lauf endet mit Exit 3. Cybersicherheit: Secrets / PII beim Ingest, Bewusstsein für Dependency-CVEs.", "compliance.art1617.title": "Konformität & QMS", "compliance.art1617.body": "QMS-SOPs in docs/qms/ decken Model-Training, Datenmanagement, Incident Response und Änderungsmanagement ab — plus Zugriffskontrolle, Verschlüsselung-im-Ruhezustand, Risikobehandlung und Statement of Applicability. Konformitätserklärungs-Vorlage wird aus dem Annex IV des Laufs generiert.", "compliance.gdpr.title": "PII-Schwärzung beim Ingest", - "compliance.gdpr.body": "--pii-mask schwärzt E-Mails, Telefone, Kreditkarten (Luhn), IBAN, nationale IDs (TR/DE/FR/US-SSN), bevor Chunks im JSONL landen — Artikel 5(1)(c) Datenminimierung wird upstream durchgesetzt.", + "compliance.gdpr.body": "--pii-mask schwärzt E-Mails, Telefone, Kreditkarten (Luhn), IBAN, nationale IDs (TR/DE/FR/US-SSN), bevor Chunks im JSONL landen. Es verringert, was ins Training gelangt, und unterstützt die Datenminimierung (Artikel 5(1)(c)); Namen und Freitext werden nicht erkannt — es ist also eine Hilfe, keine Garantie.", "compliance.artifact.eyebrow":"Was Sie auf der Festplatte erhalten", "compliance.artifact.title": "Der Compliance-Fußabdruck eines Laufs", "compliance.artifact.lead": "Nach jedem erfolgreichen (oder fehlgeschlagenen) Lauf landet das in Ihrem Output-Verzeichnis.", @@ -4023,13 +4023,13 @@ "compliance.art13.title": "Transparence", "compliance.art13.body": "Model card auto-générée listant capacités, usage prévu, résumé des données d'entraînement, résultats d'évaluation, limitations connues. Drop-in pour le README HuggingFace Hub.", "compliance.art14.title": "Supervision humaine", - "compliance.art14.body": "Optionnel evaluation.require_human_approval: true déplace le checkpoint entraîné dans un répertoire de staging ; forgelm approve / forgelm reject + forgelm approvals pilotent la signature sous double contrôle, le tout estampillé dans le log d'audit.", + "compliance.art14.body": "Optionnel evaluation.require_human_approval: true déplace le checkpoint entraîné dans un répertoire de staging ; forgelm approve / forgelm reject + forgelm approvals enregistrent une validation humaine, l'identité de l'approbateur étant estampillée dans le log d'audit. Cela appuie la documentation de la supervision à la conception ; la supervision du système déployé reste à la charge du déployeur.", "compliance.art15.title": "Précision & robustesse", - "compliance.art15.body": "Gate de benchmark avec plancher par tâche ; si le chiffre post-entraînement passe sous le plancher, l'exécution échoue (exit 3). Cybersécurité : secrets / PII à l'ingestion, conscience des CVE de dépendances.", + "compliance.art15.body": "Gate de benchmark avec plancher par tâche ; avec evaluation.auto_revert: true, un chiffre post-entraînement sous le plancher supprime les artefacts entraînés et l'exécution se termine avec exit 3. Cybersécurité : secrets / PII à l'ingestion, conscience des CVE de dépendances.", "compliance.art1617.title": "Conformité & QMS", "compliance.art1617.body": "Les SOP QMS dans docs/qms/ couvrent l'entraînement du modèle, la gestion des données, la réponse aux incidents et la gestion du changement — plus le contrôle d'accès, le chiffrement au repos, le traitement des risques et la Déclaration d'applicabilité. Modèle de déclaration de conformité généré à partir de l'Annex IV de l'exécution.", "compliance.gdpr.title": "Masquage PII à l'ingestion", - "compliance.gdpr.body": "--pii-mask masque e-mails, téléphones, cartes de crédit (Luhn), IBAN, identifiants nationaux (TR/DE/FR/US-SSN) avant que les chunks n'atterrissent dans le JSONL — la minimisation des données de l'Article 5(1)(c) est appliquée en amont.", + "compliance.gdpr.body": "--pii-mask masque e-mails, téléphones, cartes de crédit (Luhn), IBAN, identifiants nationaux (TR/DE/FR/US-SSN) avant que les chunks n'atterrissent dans le JSONL. Il réduit ce qui atteint l'entraînement et soutient la minimisation des données (Article 5(1)(c)) ; les noms et le texte libre ne sont pas détectés, c'est donc une aide, pas une garantie.", "compliance.artifact.eyebrow":"Ce que vous obtenez sur disque", "compliance.artifact.title": "L'empreinte de conformité d'une exécution", "compliance.artifact.lead": "Après chaque exécution réussie (ou échouée), voici ce qui atterrit dans votre répertoire de sortie.", @@ -4066,13 +4066,13 @@ "compliance.art13.title": "Transparencia", "compliance.art13.body": "Model card auto-generada que lista capacidades, uso previsto, resumen de datos de entrenamiento, resultados de evaluación, limitaciones conocidas. Drop-in para el README de HuggingFace Hub.", "compliance.art14.title": "Supervisión humana", - "compliance.art14.body": "Opcional evaluation.require_human_approval: true mueve el checkpoint entrenado a un directorio de staging; forgelm approve / forgelm reject + forgelm approvals conducen la firma de doble control, todo estampado en el log de auditoría.", + "compliance.art14.body": "Opcional evaluation.require_human_approval: true mueve el checkpoint entrenado a un directorio de staging; forgelm approve / forgelm reject + forgelm approvals registran una aprobación humana, con la identidad del aprobador estampada en el log de auditoría. Apoya la documentación de la supervisión en tiempo de diseño; la supervisión del sistema desplegado sigue a cargo del desplegador.", "compliance.art15.title": "Precisión y robustez", - "compliance.art15.body": "Puerta de benchmark con piso por tarea; si el número post-entrenamiento cae bajo el piso, la ejecución falla (exit 3). Ciberseguridad: secretos / PII en la ingestión, conciencia de CVE de dependencias.", + "compliance.art15.body": "Puerta de benchmark con piso por tarea; con evaluation.auto_revert: true, un número post-entrenamiento bajo el piso elimina los artefactos entrenados y la ejecución termina con exit 3. Ciberseguridad: secretos / PII en la ingestión, conciencia de CVE de dependencias.", "compliance.art1617.title": "Conformidad y QMS", "compliance.art1617.body": "Los SOPs QMS en docs/qms/ cubren entrenamiento del modelo, gestión de datos, respuesta a incidentes y gestión de cambios — más control de acceso, cifrado en reposo, tratamiento de riesgos y Declaración de Aplicabilidad. La plantilla de declaración de conformidad se genera a partir del Annex IV de la ejecución.", "compliance.gdpr.title": "Redacción PII en la ingestión", - "compliance.gdpr.body": "--pii-mask redacta emails, teléfonos, tarjetas de crédito (Luhn), IBAN, IDs nacionales (TR/DE/FR/US-SSN) antes de que los chunks lleguen al JSONL — la minimización de datos del Artículo 5(1)(c) se aplica aguas arriba.", + "compliance.gdpr.body": "--pii-mask redacta emails, teléfonos, tarjetas de crédito (Luhn), IBAN, IDs nacionales (TR/DE/FR/US-SSN) antes de que los chunks lleguen al JSONL. Reduce lo que llega al entrenamiento y apoya la minimización de datos (Artículo 5(1)(c)); los nombres y el texto libre no se detectan, por lo que es una ayuda, no una garantía.", "compliance.artifact.eyebrow":"Lo que obtiene en disco", "compliance.artifact.title": "La huella de cumplimiento de una ejecución", "compliance.artifact.lead": "Después de cada ejecución exitosa (o fallida), esto es lo que cae en su directorio de salida.", @@ -4109,13 +4109,13 @@ "compliance.art13.title": "透明度", "compliance.art13.body": "自动生成的 model card 列出能力、预期用途、训练数据摘要、评估结果、已知限制。对 HuggingFace Hub README 即插即用。", "compliance.art14.title": "人工监督", - "compliance.art14.body": "可选的 evaluation.require_human_approval: true 将训练好的 checkpoint 移到 staging 目录;forgelm approve / forgelm reject + forgelm approvals 驱动双重控制的签字流程,全部盖戳到审计日志。", + "compliance.art14.body": "可选的 evaluation.require_human_approval: true 将训练好的 checkpoint 移到 staging 目录;forgelm approve / forgelm reject + forgelm approvals 记录人工签字,批准人身份会盖戳到审计日志。它支持设计阶段的监督文档;已部署系统的监督仍由部署方负责。", "compliance.art15.title": "准确性 & 鲁棒性", - "compliance.art15.body": "Benchmark gate 按任务设阈值;如果 post-train 数值跌破阈值,运行失败 (exit 3)。网络安全:ingest 时的 secrets / PII、依赖 CVE 意识。", + "compliance.art15.body": "Benchmark gate 按任务设阈值;启用 evaluation.auto_revert: true 时,post-train 数值跌破阈值会删除训练产物,运行以 exit 3 结束。网络安全:ingest 时的 secrets / PII、依赖 CVE 意识。", "compliance.art1617.title": "符合性 & QMS", "compliance.art1617.body": "docs/qms/ 中的 QMS SOP 涵盖模型训练、数据管理、事件响应、变更管理 — 以及访问控制、静态加密、风险处置和适用性声明。符合性声明模板从运行的 Annex IV 生成。", "compliance.gdpr.title": "ingest 时的 PII 屏蔽", - "compliance.gdpr.body": "--pii-mask 在 chunk 进入 JSONL 之前屏蔽邮箱、电话、信用卡 (Luhn)、IBAN、国家身份证 (TR/DE/FR/US-SSN) — Article 5(1)(c) 数据最小化在上游执行。", + "compliance.gdpr.body": "--pii-mask 在 chunk 进入 JSONL 之前屏蔽邮箱、电话、信用卡 (Luhn)、IBAN、国家身份证 (TR/DE/FR/US-SSN)。它减少进入训练的内容并支持数据最小化 (Article 5(1)(c));姓名和自由文本不会被检测,因此它是辅助手段,而非保证。", "compliance.artifact.eyebrow":"你在磁盘上得到什么", "compliance.artifact.title": "一次运行的合规足迹", "compliance.artifact.lead": "每次成功 (或失败) 的运行之后,这些会落入你的输出目录。", @@ -4452,7 +4452,7 @@ /* ── Phase 22 / pass-4: backfill missing compliance + features keys (was H6 i18n gap) ── */ Object.assign(T.de, { "compliance.gdpr15.title": "Auskunftsrecht", - "compliance.gdpr15.body": "forgelm reverse-pii durchsucht maskierte JSONL-Korpora nach Klartextresten oder gesalzenen SHA-256-Digests einer Betroffenenkennung und meldet jede Zeile jedes Splits, in der die betroffene Person erscheint, und erfüllt damit die Auskunftsseite der DSGVO-Betroffenenrechte.", + "compliance.gdpr15.body": "forgelm reverse-pii durchsucht maskierte JSONL-Korpora nach Klartextresten oder gesalzenen SHA-256-Digests einer Betroffenenkennung und meldet jede Zeile jedes Splits, in der die betroffene Person erscheint. Es findet nur, was wörtlich in den gescannten Dateien steht: Rohquellen und das trainierte Modell werden nicht durchsucht, und die Bearbeitung eines Auskunftsersuchens bleibt Ihr Prozess.", "compliance.gdpr17.title": "Recht auf Löschung", "compliance.gdpr17.body": "forgelm purge löscht die Zeilen einer betroffenen Person aus Trainingskorpora und stempelt data.erasure_requested + data.erasure_completed-Ereignisse in das Append-only-Audit-Log, sodass Läufe nach der Löschung forensisch mit dem Löschereignis verknüpft sind.", "compliance.iso.title": "Aligned, nicht zertifiziert", @@ -4461,7 +4461,7 @@ "compliance.iso.audit.title": "Audit-Trail", "compliance.iso.audit.body": "Append-only audit_log.jsonl pro Lauf mit HMAC- + SHA-256-Hashkette + Genesis-Manifest-Sidecar. forgelm verify-audit validiert die Kette Ende-zu-Ende. HMAC-Integrität ist aktiv, sobald FORGELM_AUDIT_SECRET gesetzt ist; für audittaugliche Schlüsselverwahrung halten Sie dieses Secret KMS-verwaltet und rotieren es pro output-dir-Lebenszyklus.", "compliance.iso.change.title": "Änderungskontrolle", - "compliance.iso.change.body": "Artikel-14-Staging-Gate (forgelm approve / forgelm reject) plus human_approval.required/granted/rejected-Audit-Ereignisse plus forgelm verify-audit-Kettenintegrität — jede Modellfreigabe ist dual-kontrolliert und forensisch zugeordnet.", + "compliance.iso.change.body": "Artikel-14-Staging-Gate (forgelm approve / forgelm reject) plus human_approval.required/granted/rejected-Audit-Ereignisse plus forgelm verify-audit-Kettenintegrität — bei aktivem Gate wird jede Modellfreigabe der erfassten Identität des Freigebenden zugeordnet; dass Trainer und Freigebender verschiedene Personen sind, ist die Kontrolle des Betreibers.", "compliance.iso.lineage.title": "Daten-Lineage", "compliance.iso.lineage.body": "data_provenance.json (SHA-256-Fingerprint + Größe + mtime + HF-Hub-Revisions-Pin, abgestuft danach, ob ein gepinnter Ladevorgang dahintersteht) und data_governance_report.json (Erhebungsmethode, Annotationsprozess, bekannte Verzerrungen, Personenbezogen-Daten-Flag, DPIA-Abschluss) decken die Lineage-Fragen ab, die ein Auditor zuerst stellt.", "compliance.iso.supply.title": "Supply Chain", @@ -4476,7 +4476,7 @@ "features.gov.approve.title": "Artikel-14-Staging-Gate", "features.gov.approve.body": "forgelm approve / forgelm reject + forgelm approvals implementieren das Human-Oversight-Staging-Verzeichnis mit audit-gestempelter Freigabe.", "features.gov.gguf.title": "GGUF-Integritäts-Verifier", - "features.gov.gguf.body": "forgelm verify-gguf prüft das beim Export gestempelte SHA-256- + Größen-Manifest erneut, bevor Sie ein quantisiertes Modell ausliefern.", + "features.gov.gguf.body": "forgelm verify-gguf validiert den GGUF-Magic-Header, parst den Metadatenblock und prüft das beim Export geschriebene .sha256-Sidecar (sofern vorhanden) erneut, bevor Sie ein quantisiertes Modell ausliefern.", "features.gov.iso.title": "ISO 27001 / SOC 2 Alignment", "features.gov.iso.body": "Audit-Trail-, Change-Management-, Daten-Lineage- und Supply-Chain-Nachweise, nach denen der Auditor des Deployers fragt. Die Software ist aligned, nicht zertifiziert.", "features.gov.purge.title": "DSGVO-Artikel-17-Löschung", @@ -4498,7 +4498,7 @@ /* ── Phase 22 / pass-4: backfill missing compliance + features keys (was H6 i18n gap) ── */ Object.assign(T.fr, { "compliance.gdpr15.title": "Droit d'accès", - "compliance.gdpr15.body": "forgelm reverse-pii scanne les corpus JSONL masqués à la recherche de résidus en clair ou de digests SHA-256 salés d'un identifiant de personne concernée et signale chaque ligne de chaque split où cette personne apparaît, satisfaisant le volet droit d'accès des droits des personnes concernées du RGPD.", + "compliance.gdpr15.body": "forgelm reverse-pii scanne les corpus JSONL masqués à la recherche de résidus en clair ou de digests SHA-256 salés d'un identifiant de personne concernée et signale chaque ligne de chaque split où cette personne apparaît. Il ne trouve que ce qui figure littéralement dans les fichiers scannés : les sources brutes et le modèle entraîné ne sont pas parcourus, et le traitement d'une demande d'accès reste votre processus.", "compliance.gdpr17.title": "Droit à l'effacement", "compliance.gdpr17.body": "forgelm purge supprime les lignes d'une personne concernée des corpus d'entraînement et estampille les événements data.erasure_requested + data.erasure_completed dans le log d'audit append-only, de sorte que les exécutions post-suppression soient liées de manière forensique à l'événement d'effacement.", "compliance.iso.title": "Aligné, non certifié", @@ -4507,7 +4507,7 @@ "compliance.iso.audit.title": "Piste d'audit", "compliance.iso.audit.body": "audit_log.jsonl append-only par exécution avec chaîne de hash HMAC + SHA-256 + manifest genesis en sidecar. forgelm verify-audit valide la chaîne de bout en bout. L'intégrité HMAC est active dès que FORGELM_AUDIT_SECRET est défini ; pour une conservation de niveau audit, gardez ce secret géré par KMS et tourné selon le cycle de vie de l'output-dir.", "compliance.iso.change.title": "Contrôle des changements", - "compliance.iso.change.body": "Gate de staging Article 14 (forgelm approve / forgelm reject) plus événements d'audit human_approval.required/granted/rejected plus intégrité de chaîne forgelm verify-audit — chaque promotion de modèle est sous double contrôle et attribuée de manière forensique.", + "compliance.iso.change.body": "Gate de staging Article 14 (forgelm approve / forgelm reject) plus événements d'audit human_approval.required/granted/rejected plus intégrité de chaîne forgelm verify-audit — avec le gate actif, chaque promotion de modèle est attribuée à l'identité enregistrée de l'approbateur ; faire en sorte que l'entraîneur et l'approbateur soient des personnes différentes relève du contrôle du déployeur.", "compliance.iso.lineage.title": "Traçabilité des données", "compliance.iso.lineage.body": "data_provenance.json (empreinte SHA-256 + taille + mtime + pin de révision HF Hub, gradué selon qu'un chargement épinglé le garantit ou non) et data_governance_report.json (méthode de collecte, processus d'annotation, biais connus, drapeau données personnelles, complétion DPIA) couvrent les questions de traçabilité qu'un auditeur pose en premier.", "compliance.iso.supply.title": "Supply chain", @@ -4522,7 +4522,7 @@ "features.gov.approve.title": "Gate de staging Article 14", "features.gov.approve.body": "forgelm approve / forgelm reject + forgelm approvals implémentent le répertoire de staging de supervision humaine avec signature audit-estampillée.", "features.gov.gguf.title": "Vérificateur d'intégrité GGUF", - "features.gov.gguf.body": "forgelm verify-gguf revérifie le manifest SHA-256 + taille estampillé au moment de l'export avant que vous n'expédiiez un modèle quantisé.", + "features.gov.gguf.body": "forgelm verify-gguf valide l'en-tête magique GGUF, analyse le bloc de métadonnées et revérifie le fichier .sha256 écrit lors de l'export (s'il est présent) avant que vous n'expédiiez un modèle quantisé.", "features.gov.iso.title": "Alignement ISO 27001 / SOC 2", "features.gov.iso.body": "Preuves d'audit-trail, de gestion du changement, de traçabilité des données et de supply chain que l'auditeur du déployeur demande. Le logiciel est aligné, non certifié.", "features.gov.purge.title": "Effacement RGPD Article 17", @@ -4544,7 +4544,7 @@ /* ── Phase 22 / pass-4: backfill missing compliance + features keys (was H6 i18n gap) ── */ Object.assign(T.es, { "compliance.gdpr15.title": "Derecho de acceso", - "compliance.gdpr15.body": "forgelm reverse-pii escanea corpus JSONL enmascarados en busca de residuos en texto claro o digests SHA-256 salados de un identificador de interesado y reporta cada fila de cada split donde aparece ese sujeto, satisfaciendo el lado de solicitud de acceso de los derechos del interesado del RGPD.", + "compliance.gdpr15.body": "forgelm reverse-pii escanea corpus JSONL enmascarados en busca de residuos en texto claro o digests SHA-256 salados de un identificador de interesado y reporta cada fila de cada split donde aparece ese sujeto. Solo encuentra lo que figura literalmente en los archivos que escanea: no busca en las fuentes sin procesar ni en el modelo entrenado, y la gestión de una solicitud de acceso sigue siendo su proceso.", "compliance.gdpr17.title": "Derecho de supresión", "compliance.gdpr17.body": "forgelm purge elimina las filas de un interesado de los corpus de entrenamiento y estampa eventos data.erasure_requested + data.erasure_completed en el log de auditoría append-only, de modo que las ejecuciones posteriores a la supresión queden vinculadas forensemente al evento de borrado.", "compliance.iso.title": "Alineado, no certificado", @@ -4553,7 +4553,7 @@ "compliance.iso.audit.title": "Pista de auditoría", "compliance.iso.audit.body": "audit_log.jsonl append-only por ejecución con cadena de hash HMAC + SHA-256 + manifiesto genesis en sidecar. forgelm verify-audit valida la cadena de extremo a extremo. La integridad HMAC está activa en cuanto FORGELM_AUDIT_SECRET está definido; para custodia de nivel auditoría, mantén ese secreto gestionado por KMS y rotado por ciclo de vida del output-dir.", "compliance.iso.change.title": "Control de cambios", - "compliance.iso.change.body": "Gate de staging del Artículo 14 (forgelm approve / forgelm reject) más eventos de auditoría human_approval.required/granted/rejected más integridad de cadena forgelm verify-audit — cada promoción de modelo es de control dual y atribuida forensemente.", + "compliance.iso.change.body": "Gate de staging del Artículo 14 (forgelm approve / forgelm reject) más eventos de auditoría human_approval.required/granted/rejected más integridad de cadena forgelm verify-audit — con el gate activo, cada promoción de modelo se atribuye a la identidad registrada del aprobador; que entrenador y aprobador sean personas distintas es un control del desplegador.", "compliance.iso.lineage.title": "Trazabilidad de datos", "compliance.iso.lineage.body": "data_provenance.json (huella SHA-256 + tamaño + mtime + pin de revisión HF Hub, graduado según si una carga anclada lo respalda) y data_governance_report.json (método de recolección, proceso de anotación, sesgos conocidos, marca de datos personales, completitud DPIA) cubren las preguntas de trazabilidad que un auditor hace primero.", "compliance.iso.supply.title": "Cadena de suministro", @@ -4568,7 +4568,7 @@ "features.gov.approve.title": "Gate de staging del Artículo 14", "features.gov.approve.body": "forgelm approve / forgelm reject + forgelm approvals implementan el directorio de staging de supervisión humana con firma estampada en auditoría.", "features.gov.gguf.title": "Verificador de integridad GGUF", - "features.gov.gguf.body": "forgelm verify-gguf reverifica el manifiesto SHA-256 + tamaño estampado en el momento del export antes de que envíe un modelo cuantizado.", + "features.gov.gguf.body": "forgelm verify-gguf valida la cabecera mágica GGUF, analiza el bloque de metadatos y reverifica el sidecar .sha256 escrito durante el export (si existe) antes de que envíe un modelo cuantizado.", "features.gov.iso.title": "Alineamiento ISO 27001 / SOC 2", "features.gov.iso.body": "Pruebas de pista de auditoría, gestión de cambios, trazabilidad de datos y cadena de suministro que pide el auditor del deployer. El software está alineado, no certificado.", "features.gov.purge.title": "Supresión RGPD Artículo 17", @@ -4590,7 +4590,7 @@ /* ── Phase 22 / pass-4: backfill missing compliance + features keys (was H6 i18n gap) ── */ Object.assign(T.zh, { "compliance.gdpr15.title": "访问权", - "compliance.gdpr15.body": "forgelm reverse-pii 扫描已掩码的 JSONL 语料库,查找数据主体标识的明文残留或加盐 SHA-256 摘要,并报告该主体出现的每个 split 的每一行,满足 GDPR 数据主体权利的访问请求一侧。", + "compliance.gdpr15.body": "forgelm reverse-pii 扫描已掩码的 JSONL 语料库,查找数据主体标识的明文残留或加盐 SHA-256 摘要,并报告该主体出现的每个 split 的每一行。它只能找到所扫描文件中字面存在的内容:不会搜索原始来源或已训练的模型,处理访问请求仍是你的流程。", "compliance.gdpr17.title": "删除权", "compliance.gdpr17.body": "forgelm purge 从训练语料中删除某个数据主体的行,并在 append-only 审计日志中盖上 data.erasure_requested + data.erasure_completed 事件,使删除后的运行在取证上与该删除事件相连。", "compliance.iso.title": "对齐,而非认证", @@ -4599,7 +4599,7 @@ "compliance.iso.audit.title": "审计追踪", "compliance.iso.audit.body": "每次运行的 append-only audit_log.jsonl,带 HMAC + SHA-256 哈希链 + 创世清单 sidecar。forgelm verify-audit 端到端验证该链。只要设置了 FORGELM_AUDIT_SECRET,HMAC 完整性即生效；为达到审计级密钥保管,请将该密钥交由 KMS 管理并按 output-dir 生命周期轮换。", "compliance.iso.change.title": "变更控制", - "compliance.iso.change.body": "第 14 条 staging gate (forgelm approve / forgelm reject) 加上 human_approval.required/granted/rejected 审计事件,加上 forgelm verify-audit 链完整性 — 每次模型晋升均为双重控制并可在取证上追溯。", + "compliance.iso.change.body": "第 14 条 staging gate (forgelm approve / forgelm reject) 加上 human_approval.required/granted/rejected 审计事件,加上 forgelm verify-audit 链完整性 — 启用该 gate 时,每次模型晋升都归属于所记录的批准人身份;训练者与批准人是否为不同的人,由部署方控制。", "compliance.iso.lineage.title": "数据血缘", "compliance.iso.lineage.body": "data_provenance.json (SHA-256 指纹 + 大小 + mtime + HF Hub 修订版本固定，按其背后是否有一次固定加载分级) 与 data_governance_report.json (采集方法、标注流程、已知偏见、个人数据标记、DPIA 完成情况) 覆盖审计员最先问的血缘问题。", "compliance.iso.supply.title": "供应链", @@ -4614,7 +4614,7 @@ "features.gov.approve.title": "第 14 条 staging gate", "features.gov.approve.body": "forgelm approve / forgelm reject + forgelm approvals 通过审计盖章的签字实现人工监督 staging 目录。", "features.gov.gguf.title": "GGUF 完整性验证器", - "features.gov.gguf.body": "forgelm verify-gguf 在你交付量化模型之前重新校验 export 时盖好的 SHA-256 + 大小 manifest。", + "features.gov.gguf.body": "forgelm verify-gguf 在你交付量化模型之前,校验 GGUF 魔数头、解析 metadata 块,并(若存在)根据 export 时写入的 .sha256 sidecar 文件重新校验。", "features.gov.iso.title": "ISO 27001 / SOC 2 对齐", "features.gov.iso.body": "deployer 的审计员需要的审计追踪、变更管理、数据血缘和供应链证据。软件是对齐,而不是被认证。", "features.gov.purge.title": "GDPR 第 17 条删除", diff --git a/site/js/wizard.js b/site/js/wizard.js index 49ec3646..ca3dd6ab 100644 --- a/site/js/wizard.js +++ b/site/js/wizard.js @@ -271,10 +271,11 @@ retentionStaging: 7, retentionEphemeral: 90, retentionRawDocuments: 90, - // Policy enforcement: 'log_only' (default; logs to audit only), - // 'warn_on_excess' (stderr warning), 'block_on_excess' (aborts - // trainer with EXIT_EVAL_FAILURE). Schema Literal — must use - // one of these three values. + // Policy mode: 'log_only' (default), 'warn_on_excess' or + // 'block_on_excess'. Schema Literal — must use one of these three + // values. Reserved: no ForgeLM command reads it yet, so all three + // behave the same (retention is only reported by + // `forgelm purge --check-policy`). retentionEnforce: 'log_only', // Track which steps the operator tried to advance past while // failing required-field validation. The renderer reads this to diff --git a/site/quickstart.html b/site/quickstart.html index 291757f1..87805ced 100644 --- a/site/quickstart.html +++ b/site/quickstart.html @@ -217,8 +217,13 @@

Install

$ pip install 'forgelm[export]==0.11.0' # GGUF export # sanity-check your environment before the first run -$ forgelm doctor -✓ Python 3.11.7 · CUDA 12.1 · bitsandbytes ok · disk 142 GB free +$ forgelm doctor --offline +forgelm doctor - environment check + [+ pass] python.version Python 3.11.4 (CPython). + [+ pass] disk.workspace Workspace /home/me/forgelm — 387.0 GiB free of 500.0 GiB. + [! warn] operator.identity FORGELM_OPERATOR not set; audit events will fall back to 'me@workstation'. ... + ... (about 19 rows; values depend on your machine) +Summary: 7 pass, 12 warn, 0 fail.
diff --git a/tests/site/site_translations.test.cjs b/tests/site/site_translations.test.cjs index 6db1c3e3..4cabd82b 100644 --- a/tests/site/site_translations.test.cjs +++ b/tests/site/site_translations.test.cjs @@ -28,3 +28,32 @@ test('a string a page shows as plain text holds no markup or HTML entity', () => } assert.deepEqual(literal, []); }); + +// Site copy that promised more than the code does (the Wave 0 triage). The English and Turkish tiers are matched +// by phrase; the other four locales are checked for a token every translation of the corrected string keeps. +test('claims the code does not back stay out of the site copy', () => { + const { translations } = loadWizard(); + const site = path.join(root, 'site'); + const retired = { + en: [/dual-control/i, /\bauto-deletes\b/, /run fails \(exit 3\)/, /satisfying the access-request/, + /Slack, Teams, Discord, or any HTTP/, /Slack \/ Teams \/ Discord/, /enforced upstream/, /GDPR-compliant TTLs/, + /SHA-256 \+ size manifest/, /6 quant levels and SHA-256 manifest/, /ZeRO-2 \/ ZeRO-3 presets/, + /PII redacted at ingest/, /GGUF export at q4_k_m/], + tr: [/ikili[- ]kontrol/i, /otomatik siler/, /GDPR uyumlu TTL/, /erişim-talebi tarafını karşılar/, + /Slack, Teams, Discord veya/, /Slack \/ Teams \/ Discord/, /upstream'de uygulanır/], + }; + const found = []; + for (const [locale, patterns] of Object.entries(retired)) { + for (const [key, value] of Object.entries(translations[locale])) { + for (const pattern of patterns) if (pattern.test(value)) found.push(`${locale} ${key}: ${pattern}`); + } + } + for (const page of fs.readdirSync(site).filter(name => name.endsWith('.html'))) { + const html = fs.readFileSync(path.join(site, page), 'utf8'); + for (const pattern of retired.en) if (pattern.test(html)) found.push(`${page}: ${pattern}`); + } + assert.deepEqual(found, []); + for (const [locale, table] of Object.entries(translations)) { + assert.match(table['wizard.retention.enforce.hint'], /--check-policy/, `${locale}: retention is only reported`); + } +}); diff --git a/tools/generate_sbom.py b/tools/generate_sbom.py index 47b5b5ce..1b32f9dd 100644 --- a/tools/generate_sbom.py +++ b/tools/generate_sbom.py @@ -3,8 +3,9 @@ Reads the installed package set via ``pip list --format=json`` and renders a CycloneDX 1.5 JSON document on stdout. Pure stdlib, cross-platform — used by -the ``publish.yml`` cross-OS matrix to attach a per-(os, python-version) SBOM -artifact to every tagged release. +the ``publish.yml`` cross-OS matrix, which uploads a per-(os, python-version) SBOM +as a workflow-run artifact of every tag's publish run (it is not attached to the +GitHub release, and run artifacts expire after the repository's retention period). This is intentionally a hand-rolled emitter rather than the optional ``cyclonedx-bom`` PyPI package: keeping the dependency footprint at zero means From 10fc5a56fb53424d4dac0e3175e2507b14dd174b Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 14:21:45 +0300 Subject: [PATCH 13/22] chore(deps): admit pypdf 6, xxhash 4, datasketch 2 and pytest-randomly 4-5 (#1014) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Four Dependabot range bumps (#1003, #1008, #1010, #1011) combined into one pull request so they cost one CI round instead of four (each merge makes the others "behind" and the branch protection requires an up-to-date branch). Dependabot closes its own pull requests when the same change lands. The individual Dependabot commits are merged unchanged. ## What each range change admits, and the evidence | Dependency | Range | Where it is used | Evidence | |---|---|---|---| | `datasketch` | `<2` → `<3` | `ingestion-scale` extra (MinHash LSH) | **CI installs 2.0.0** for the full suite (5868 passed, 22 skipped); the data-audit modules also pass locally on 2.0.0 (283 passed) | | `pytest-randomly` | `<4` → `<6` | `dev` extra | **CI installs 5.0.0** for the full suite (5868 passed, 22 skipped) | | `xxhash` | `<4` → `<5` | `ingestion` extra (simhash) | **Not exercised by CI** (the extra is not installed there). Locally on 4.0.1 the data-audit modules give the same result as on 3.8: 283 passed, 1 skipped | | `pypdf` | `<6` → `<7` | `ingestion` extra (PDF reading) | **Not exercised by CI**: pypdf is not installed there, so the PDF tests skip (224 passed, 19 skipped). Locally the same modules give 238 passed, 5 skipped on both 5.9.0 and 6.21.1 | So for xxhash and pypdf the green CI proves nothing; the local runs above are the evidence. ### Not included **#1007 (`huggingface_hub` `<2` → `<3`)** is left open on purpose. CI installs 1.33.0 either way, so the change is untested, and widening the range lets pip install hub 2.x for users once their other dependencies allow it. That is a claim ForgeLM has not checked, and the core stack (`huggingface_hub`, transformers, torch, TRL, PEFT, accelerate, datasets) is migrated deliberately, not by an automatic bump. I would close it, and add a Dependabot `ignore` for semver-major bumps of that stack; that is a separate change. ## Type - [x] Dependency ## Testing - CI (lint, validate, tests on Python 3.10–3.13) on this branch. - Local runs against throwaway installs of datasketch 2.0.0, xxhash 4.0.1 and pypdf 5.9.0 / 6.21.1 as in the table. ## Checklist - [x] CHANGELOG entry under `[Unreleased]` 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Expand dependency compatibility ranges for ingestion extras and development tooling while retaining the existing huggingface_hub constraint. Enhancements: - Broaden supported version ranges for optional ingestion dependencies and development tooling, admitting pypdf 6, xxhash 4, datasketch 2, and pytest-randomly 4–5. Documentation: - Document the newly admitted dependency major versions and their validation status in the unreleased changelog. Tests: - Validate compatibility with the newer dependency versions through CI and targeted local test runs. ## Summary by CodeRabbit * **Chores** * Expanded supported version ranges for optional ingestion and scaling packages, plus the development testing package. Minimum supported versions are unchanged. * Verified ingestion tests with updated package versions and CI with the newer scaling and testing package versions. * Updated the changelog to document these compatibility updates. --------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 --- CHANGELOG.md | 4 ++++ pyproject.toml | 8 ++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bdcc5ea1..c8eac06e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,10 @@ All notable changes to ForgeLM are documented here. ### Changed +- Admit newer majors of four optional or dev dependencies: `pypdf` 6 and `xxhash` 4 (the `ingestion` extra), `datasketch` 2 (the + `ingestion-scale` extra) and `pytest-randomly` 4 and 5 (the `dev` extra). The ingestion, data-audit and doctor test modules pass on pypdf + 6.21.1, xxhash 4.0.1 and datasketch 2.0.0 with the same results as on the previous majors; CI installs datasketch 2.0.0 and + pytest-randomly 5.0.0 for the full suite. `huggingface_hub` stays below 2: ForgeLM has not been tested with it. - Raise the `transformers` floor from `>=5.3.0` to `>=5.10.1` (and `huggingface_hub` from `>=1.3.0` to `>=1.5.0`, which transformers 5.10 requires). `5.3.0` was described as the CVE fix floor but still admitted CVE-2026-5241 (a LightGlue `config.json` overrides `trust_remote_code: false`, fixed in 5.5.0) and CVE-2026-9856 (a `chat_template` name in diff --git a/pyproject.toml b/pyproject.toml index f1e2ce1e..95a3092a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -145,12 +145,12 @@ export = [ # so a bare install still works. # OCR is intentionally out of scope (handle externally with Tesseract). ingestion = [ - "pypdf>=4.0.0,<6.0.0", + "pypdf>=4.0.0,<7.0.0", "python-docx>=1.0.0,<2.0.0", "ebooklib>=0.18,<1.0.0", "beautifulsoup4>=4.12.0,<5.0.0", "langdetect>=1.0.9,<2.0.0", - "xxhash>=3.0.0,<4.0.0", + "xxhash>=3.0.0,<5.0.0", ] # Optional MinHash LSH near-duplicate detector for >50K-row corpora. @@ -159,7 +159,7 @@ ingestion = [ # operators auditing larger datasets where MinHash's recall/precision # trade-off is the industry standard (NeMo Curator, Dolma, RedPajama). ingestion-scale = [ - "datasketch>=1.6.0,<2.0.0", + "datasketch>=1.6.0,<3.0.0", ] # Optional Presidio ML-NER PII detector. Adds the unstructured-identifier @@ -187,7 +187,7 @@ dev = [ # surface in CI instead of waiting for a Python micro-bump or runner # change to flip collection order. Run locally via # ``pytest --randomly-seed=`` to reproduce a failure. - "pytest-randomly>=3.15.0,<4.0.0", + "pytest-randomly>=3.15.0,<6.0.0", # Bounded on purpose. ruff 0.16.0 began formatting Python blocks *inside # Markdown*, taking the format check from 277 files to 562 — on an unchanged # tree that is 32 newly "unformatted" documents and a red lint job nobody From b8d2643123c615fb5a05f28483e122387b5e2814 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Sun, 18 Oct 2026 14:34:50 +0300 Subject: [PATCH 14/22] chore(ci): bump checkout 7, setup-python 7, download-artifact 8, deploy-pages 5 and pypi-publish 1.14.2 (#1015) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Five Dependabot GitHub Actions bumps (#1002, #1004, #1005, #1006, #1009) combined into one pull request so they cost one CI round instead of five (each merge makes the others "behind", and branch protection requires an up-to-date branch). Dependabot closes its own pull requests when the same change lands. The individual commits are merged unchanged. ## Review | Action | Change | Checked | |---|---|---| | `pypa/gh-action-pypi-publish` | `1.14.0` → `1.14.2` (SHA pin) | The new pin `dc37677b…` is the commit of tag `v1.14.2` (dereferenced the annotated tag; the old pin is `v1.14.0`'s). The release only updates the action's own dependency tree (Twine 7, pypi-attestations, sigstore): `action.yml` and `requirements/` change, nothing else | | `actions/checkout` | `v5` → `v7` | v6 stores credentials in a separate file; v7 blocks checking out fork PRs under `pull_request_target`/`workflow_run`. This repository uses neither trigger and no workflow pushes with the persisted credentials | | `actions/setup-python` | `v6` → `v7` | ESM migration, and the `pip-install` input is removed; no workflow passes it (`cache: pip` is unaffected) | | `actions/download-artifact` | `v5` → `v8` | Hash mismatches now fail by default and non-zip artifacts are no longer unzipped; our `dist` artifact is a zip uploaded by `upload-artifact@v5`, which `v8` reads | | `actions/deploy-pages` | `v4` → `v5` | Node 24 runtime and polling back-off only | ### Also in this pull request Every `actions/checkout` step now sets `persist-credentials: false` (11 steps in 5 workflows). CodeRabbit flagged it on the bumped lines; the default stores the workflow token in the git configuration of jobs that then run checked-out code, and no job pushes or fetches with it (the assign-on-comment workflow already did this). ### What CI proves, and what it does not This pull request's own CI exercises `checkout` and `setup-python` (`ci.yml`). The other three live in `publish.yml` and `site-deploy.yml`, which run only on a release tag or a push to `main`, so **they are untested until the next release**. All three fail closed: a broken download or deploy step stops before anything reaches PyPI or the site. If you want certainty first, merge everything except `download-artifact` and `pypi-publish` and try them on a release candidate tag. ## Type - [x] CI / dependency ## Testing - CI (lint, validate, tests on Python 3.10–3.13) on this branch; all workflow files parse; `check_ci_failure_masking` and the guard-wiring tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by Sourcery Modernize the repository’s GitHub Actions and harden checkout credential handling across automated workflows. Enhancements: - Upgrade GitHub Actions used across CI, publishing, and deployment workflows to their current major versions. - Disable persisted checkout credentials across workflow jobs to reduce unnecessary token exposure. CI: - Update artifact download, PyPI publishing, and Pages deployment actions to newer releases. --------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 18 ++++++++++------ .github/workflows/nightly.yml | 24 ++++++++++++++-------- .github/workflows/publish.yml | 18 +++++++++------- .github/workflows/site-deploy.yml | 8 +++++--- .github/workflows/usermanuals-validate.yml | 6 ++++-- 5 files changed, 48 insertions(+), 26 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e3b75537..074b96fc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -27,9 +27,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" @@ -118,9 +120,11 @@ jobs: matrix: python-version: ["3.10", "3.11", "3.12", "3.13"] steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: ${{ matrix.python-version }} cache: pip @@ -159,14 +163,16 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - uses: actions/setup-node@v7 with: node-version: "24" package-manager-cache: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" cache: pip diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 15089281..54a497a6 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -23,9 +23,11 @@ jobs: matrix: python-version: ["3.11", "3.12"] steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: ${{ matrix.python-version }} @@ -149,9 +151,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" @@ -218,9 +222,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.10" @@ -287,9 +293,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" cache: pip diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 070f78a7..10bb1375 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -18,9 +18,11 @@ jobs: permissions: contents: read # checkout reads the tagged ref steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" @@ -51,13 +53,15 @@ jobs: permissions: contents: read # checkout reads the tagged ref for tests/ + tools/ steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: ${{ matrix.python }} - - uses: actions/download-artifact@v5 + - uses: actions/download-artifact@v8 with: name: dist path: dist @@ -120,7 +124,7 @@ jobs: contents: read # download-artifact reads workflow run state id-token: write # OIDC trusted publishing to PyPI steps: - - uses: actions/download-artifact@v5 + - uses: actions/download-artifact@v8 with: name: dist path: dist/ @@ -133,4 +137,4 @@ jobs: # Metadata-Version 2.4 wheels (setuptools ≥77 emits 2.4 for PEP 639 # license expressions) with "Metadata is missing required fields: # Name, Version"; v1.14.0 parses 2.4 correctly. - uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0 + uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 diff --git a/.github/workflows/site-deploy.yml b/.github/workflows/site-deploy.yml index 3c49abd5..ab55ae12 100644 --- a/.github/workflows/site-deploy.yml +++ b/.github/workflows/site-deploy.yml @@ -46,9 +46,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" @@ -87,4 +89,4 @@ jobs: url: ${{ steps.deployment.outputs.page_url }} steps: - id: deployment - uses: actions/deploy-pages@v4 + uses: actions/deploy-pages@v5 diff --git a/.github/workflows/usermanuals-validate.yml b/.github/workflows/usermanuals-validate.yml index 2361d69b..3bdf31b1 100644 --- a/.github/workflows/usermanuals-validate.yml +++ b/.github/workflows/usermanuals-validate.yml @@ -34,9 +34,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v7 + with: + persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@v7 with: python-version: "3.11" From 306c976aa7701ff2a98c755f4faeafcb31dcf1e7 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 18 Oct 2026 16:03:04 +0300 Subject: [PATCH 15/22] chore(deps): update huggingface-hub requirement from <2.0.0,>=1.5.0 to >=1.5.0,<3.0.0 (#1007) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updates the requirements on [huggingface-hub](https://github.com/huggingface/huggingface_hub) to permit the latest version.
Release notes

Sourced from huggingface-hub's releases.

[v2.2.0] hf jobs stats snapshots, download and HfFileSystem fixes

A small release this week, mostly bug fixes and quality-of-life improvements. hf jobs stats now returns right away instead of streaming forever, concurrent downloads reliably use hf_xet, and HfFileSystem caching is more predictable. Two behavior changes are listed under Breaking changes.

📊 hf jobs stats prints a snapshot by default

hf jobs stats used to stream metrics until the job finished, so scripts and agents calling it on a running job hung until their own timeout. It now prints the latest sample of each job once and exits, like hf jobs logs and hf spaces logs. Use -f/--follow for the previous live view. Snapshot output also supports --format json|agent|quiet, which outputs the raw API metrics.

On the Python side, HfApi.fetch_job_metrics gets the same follow parameter as fetch_job_logs. It defaults to False and returns only the current sample. Pass follow=True to stream until the job completes.

>>> hf jobs stats
6ac3bd9afbc85ba6823a9ad4
JOB ID CPU % NUM CPU MEM % MEM USAGE NET I/O GPU UTIL % GPU MEM % GPU
MEM USAGE
--------------------- ----- ------- ----- -------------- ---------------
---------- --------- -------------
6ac3bd9afbc85ba682... 0% 2.0 0.01% 1.0MB / 16.0GB 0.0bps / 0.0bps N/A
N/A N/A
Hint: Use `hf jobs stats -f celinah/6ac3bd9afbc85ba6823a9ad4` to follow
live stats.

Stream live stats until the job completes


>>> hf jobs stats -f 6ac3bd9afbc85ba6823a9ad4


>>> from huggingface_hub import
fetch_job_metrics
>>> for metrics in fetch_job_metrics(job_id=job.id,
follow=True):
...     print(metrics)

Other Jobs fixes in this release: hf jobs logs and hf spaces logs without -f no longer hang while the job keeps printing, hf jobs stats no longer fails when one of the requested jobs has finished, and hf jobs stats --json prints [] when no job is running.

📚 Documentation: Jobs guide

⚡ Concurrent downloads always use hf_xet

In a fresh process, the first concurrent hf_hub_download calls could read a placeholder version for hf_xet while another thread was still looking it up. Those calls treated hf_xet as missing and downloaded through the plain HTTP path. This mostly affected parallel downloads such as snapshot_download, where some of the files skipped Xet. The version lookup is now cached only once it is resolved, so every download sees the real value.

  • Fix race in _get_version that makes concurrent downloads skip hf_xet by @​BramVanroy in #5102

📂 HfFileSystem cache fixes

Three fixes to how HfFileSystem caches listings and missing paths:

  • ls() on a newly created empty bucket returns an empty list instead of raising FileNotFoundError.
  • Directory listings are now replaced in the cache instead of appended to. Before, ls() after find() or ls(refresh=True) could return duplicate entries, refresh=True kept deleted files, and listing a single bucket file cached it as the whole folder's listing.
  • exists(path, refresh=True) and invalidate_cache(path) now clear cached "not found" results. Before, a repo, branch or bucket created in the same process stayed missing until a full invalidate_cache().
</tr></table>

... (truncated)

Commits
  • acfdf30 Release: v2.2.0
  • 193d9d9 Release: v2.2.0.rc0
  • 5be3565 [Cache] Speed up delete_revisions with a set lookup (#5118)
  • dab6a96 [CLI] Don't glob-expand arguments on Windows (#5117)
  • cca4d3a [Download] Fix offline snapshot_download with a pinned revision (#5113)
  • c6cc03e Fix argument name in raise_text_generation_error docstring (#5108)
  • 8abbfe5 Fix race in _get_version that makes concurrent downloads skip hf_xet (#5102)
  • bb3a875 [Docs] Fix from_pretrained integration example (#5105)
  • cf992a9 [HfFileSystem] Make refresh=True clear cached not-found results (#5106)
  • af74167 [Tests] Wait for bucket listings after writes (#5095)
  • Additional commits viewable in compare view

--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cemil ILIK Co-authored-by: Claude Sonnet 5.5 --- CHANGELOG.md | 7 +++++-- pyproject.toml | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c8eac06e..bafdf23a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,10 +37,13 @@ All notable changes to ForgeLM are documented here. ### Changed -- Admit newer majors of four optional or dev dependencies: `pypdf` 6 and `xxhash` 4 (the `ingestion` extra), `datasketch` 2 (the +- Admit newer majors of five dependencies: `huggingface_hub` 2, `pypdf` 6 and `xxhash` 4 (the `ingestion` extra), `datasketch` 2 (the `ingestion-scale` extra) and `pytest-randomly` 4 and 5 (the `dev` extra). The ingestion, data-audit and doctor test modules pass on pypdf 6.21.1, xxhash 4.0.1 and datasketch 2.0.0 with the same results as on the previous majors; CI installs datasketch 2.0.0 and - pytest-randomly 5.0.0 for the full suite. `huggingface_hub` stays below 2: ForgeLM has not been tested with it. + pytest-randomly 5.0.0 for the full suite. With `huggingface_hub>=1.5.0,<3.0.0` CI resolves `huggingface_hub` 2.2.0 (with transformers + 5.20.1) and the full suite passes on Python 3.10 to 3.13; every `huggingface_hub` symbol ForgeLM imports exists in 2.2.0. Hub 2 uses a + different HTTP stack (`httpx2`), so an environment sees new transitive packages. The tests mock the network, so real Hub calls on 2.x are + not covered, and transformers 5.10 to 5.17 still keep pip on `huggingface_hub` 1.x (5.18 and later allow 2.x). - Raise the `transformers` floor from `>=5.3.0` to `>=5.10.1` (and `huggingface_hub` from `>=1.3.0` to `>=1.5.0`, which transformers 5.10 requires). `5.3.0` was described as the CVE fix floor but still admitted CVE-2026-5241 (a LightGlue `config.json` overrides `trust_remote_code: false`, fixed in 5.5.0) and CVE-2026-9856 (a `chat_template` name in diff --git a/pyproject.toml b/pyproject.toml index 95a3092a..2bdd3ddf 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -62,7 +62,7 @@ dependencies = [ "pydantic>=2.0.0,<3.0.0", "trl>=1.0.0,<2.0.0", "tensorboard>=2.15.0,<3.0.0", - "huggingface_hub>=1.5.0,<2.0.0", + "huggingface_hub>=1.5.0,<3.0.0", "requests>=2.32.2,<3.0.0", # requests-toolbelt is required by forgelm._http for the HostHeaderSSLAdapter # used to close the DNS-rebinding TOCTOU window in safe_post / safe_get From e5c0eb1f6de92f1e413864fae1d08c844ee7301d Mon Sep 17 00:00:00 2001 From: hiyouwhatsapp <89997383+hiyouwhatsapp@users.noreply.github.com> Date: Sun, 18 Oct 2026 16:41:55 +0300 Subject: [PATCH 16/22] fix(cli): preserve root flags before subcommands (#1012) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Keep explicitly supplied root flags when a subcommand omits the same option. For example, `forgelm --dry-run purge` now keeps `dry_run=True`. ## Related issues Fixes #414 Resolves: CR-A05-001 ## Changes - Set `argparse.SUPPRESS` on the eight duplicated optional flags. - Cover flags before and after the subcommand, omitted defaults and subcommand precedence using `parse_args` only. The tests never execute purge. - Add the changelog entry and document the parser budget increase from 1372 to 1375 LOC; the new defaults and ruff wrapping exceed the previous budget by three lines. ## Type - [x] Bug fix - [ ] New feature - [ ] Documentation - [ ] Refactoring / code quality - [ ] Test coverage - [ ] CI / infrastructure ## Testing [Linux verification](https://github.com/hiyouwhatsapp/ForgeLM/actions/runs/39119653962) ran the full canonical gauntlet on Ubuntu with Python 3.11 and CPU torch, including site tests, dry-run, typing and every repository guard. Ruff formatting was checked without rewriting files; mypy used a minimal environment, following the existing CI approach. At `b1e2704`: **5,893 passed, 22 skipped**, with the rest of the gauntlet green. On Windows, the final CLI/subcommand/module-size suites passed all 148 tests. Against the old parser, eight new before-subcommand cases fail. An earlier Windows full-suite attempt reported 5,531 passed, 74 failed and 104 skipped, including symlink privilege failures (WinError 1314). The final full-suite verification above is on Linux. - [x] `pytest tests/` passes - [x] `ruff check .` passes - [x] `ruff format --check .` passes - [x] New tests added for changed code - [x] `python -m forgelm --config config_template.yaml --dry-run` works (the `python -m` form is deliberate: a console script's `sys.path[0]` is its own `bin/` directory, so plain `forgelm …` validates whatever is installed in site-packages rather than your working tree) ## Checklist - [x] My code follows the project's style (ruff formatted) and `docs/standards/coding.md` - [ ] I've updated documentation if needed — and its Turkish mirror (`-tr.md`, or the `tr/` tree for user manuals) in the same PR - [x] I've added tests for new functionality - [x] No new dependencies added (or added as optional: `pip install forgelm[...]`) - [ ] Config template (`config_template.yaml`) updated if new config fields added - [x] CHANGELOG entry under `[Unreleased]` if a user or library consumer can observe the change No new CLI flags or config fields. Existing user documentation already describes the intended behavior, so no EN/TR or config-template changes were needed. ## Summary by Sourcery Preserve root-level CLI flag values across subcommands without changing their existing defaults or override behavior. Bug Fixes: - Preserve explicitly supplied root CLI values for shared flags when subcommands omit those options, while retaining subcommand precedence and omitted defaults. Tests: - Add parser coverage for shared flags before and after subcommands, omitted defaults, and subcommand overrides. Chores: - Update the changelog and parser module-size budget for the CLI changes. ## Summary by CodeRabbit * **Bug Fixes** * Top-level `--dry-run`, `--config`, `--offline`, and `--output` options are now preserved when a subcommand omits the corresponding option. Explicit subcommand values still take precedence. Co-authored-by: Cemil ILIK --- CHANGELOG.md | 1 + forgelm/cli/_parser.py | 15 +++++++---- tests/test_cli.py | 55 ++++++++++++++++++++++++++++++++++++++ tools/check_module_size.py | 7 ++++- 4 files changed, 72 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bafdf23a..69df425d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -94,6 +94,7 @@ All notable changes to ForgeLM are documented here. ### Fixed +- Preserve top-level `--dry-run`, `--config`, `--offline` and `--output` values when a subcommand omits the same flag (CR-A05-001, #414). - Stop describing the human-approval gate as enforcing segregation of duties (English and Turkish): it records the trainer and approver identities and does not compare them, and both are self-asserted. The ISO 27001 and SOC 2 mappings, the Statement of Applicability, the change-management SOP, the deployer guide and the user manual now diff --git a/forgelm/cli/_parser.py b/forgelm/cli/_parser.py index b76d985b..4bd75f23 100644 --- a/forgelm/cli/_parser.py +++ b/forgelm/cli/_parser.py @@ -118,7 +118,9 @@ def _add_deploy_subcommand(subparsers) -> None: choices=["ollama", "vllm", "tgi", "hf-endpoints"], help="Target serving runtime.", ) - p.add_argument("--output", type=str, default=None, metavar="FILE", help="Output file path (default: auto).") + p.add_argument( + "--output", type=str, default=argparse.SUPPRESS, metavar="FILE", help="Output file path (default: auto)." + ) p.add_argument("--system", type=str, default=None, metavar="PROMPT", help="System prompt (Ollama only).") p.add_argument("--max-length", type=_positive_int, default=4096, help="Context window length (default: 4096).") p.add_argument( @@ -174,13 +176,14 @@ def _add_quickstart_subcommand(subparsers) -> None: p.add_argument( "--output", type=str, - default=None, + default=argparse.SUPPRESS, metavar="FILE", help="Where to write the generated YAML (default: ./configs/