diff --git a/README.md b/README.md index 3616255..3caca40 100644 --- a/README.md +++ b/README.md @@ -52,10 +52,11 @@ Three ways in, one version number: - **Targets as you name them** — a repository, an organisation or a group by path or by web URL, narrowed by topics and path globs (`acme/svc-*`), minus `exclude` globs (`corp:platform/legacy/**`) -- **Six platforms, one hub** — GitHub (github.com, GHE.com, Enterprise Server), GitLab - (gitlab.com and self-managed), Gitea, Forgejo, Bitbucket Cloud and Azure DevOps Services - (the last two built from their documentation, not yet run on a live account); one hub can - deliver to several at once, and its own CI can be any of them +- **Seven platforms, one hub** — GitHub (github.com, GHE.com, Enterprise Server), GitLab + (gitlab.com and self-managed), Gitea, Forgejo, Bitbucket Cloud, Azure DevOps Services + and Bitbucket Data Center (the last three built from their documentation, not yet run on + a live account); one hub can deliver to several at once, and its own CI can be any of + them but Bitbucket Data Center, which has none - **Declines remembered** — content in a pull request closed without merging is not proposed again; reopen it, or tick a box in its description, to change your mind - **A plan before delivery** — `plan` in a hub pull request reports what every target would diff --git a/docs/agents.md b/docs/agents.md index bd4a538..755047d 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -9,7 +9,7 @@ | Program | `touchmark`, one static Go binary; the same binary in the image `ghcr.io/bedrock-python/touchmark` and the GitHub Action `bedrock-python/touchmark` | | Requires | git 2.31+ for `check`, `status`, `apply`, `manifest`; git 2.45+ for `plan` and `distribute`, which read the targets (the image brings its own) | | Install | archives on [GitHub releases](https://github.com/bedrock-python/touchmark/releases) · `go install github.com/bedrock-python/touchmark/cmd/touchmark@latest` (Go 1.26+) · `docker run ghcr.io/bedrock-python/touchmark:` | -| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+; as providers a hub delivers to: Bitbucket Cloud, Azure DevOps Services | +| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+; as providers a hub delivers to: Bitbucket Cloud, Azure DevOps Services, Bitbucket Data Center 8.19+ | | Hub template | | | Source | | @@ -49,7 +49,7 @@ nothing to a repository without the opt-in file, unless `targets.yml` subscribes (`opt_in: assumed`). There is no pull mode (targets do not fetch from the hub), no remote pack sources, and no `setup` for Bitbucket or Azure DevOps (a hub on Bitbucket Cloud runs in Bitbucket Pipelines, one in Azure Repos in Azure Pipelines, both -set up by hand). +set up by hand; Bitbucket Data Center has no CI, so it is only ever a target). ## Mental model @@ -254,7 +254,7 @@ ten `::error` and ten `::warning` annotations. | `version` | `1` | format version | | `id` | — | slug, 3–40 characters; names the branch `touchmark/`; `change-me` fails `check` | | `writer`, `sign` | — , `auto` | single-provider shorthand: the writer account, commit signing (`auto`, `always`) | -| `providers[]` | from CI | `id`, `type` (`github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`), `url` (required for gitea, forgejo and azure-devops: `https://dev.azure.com/`), `api_url`, `ca_file`, `writer`, `known_authors`, `automation_accounts`, `sign`, `limits`; replaces the shorthand | +| `providers[]` | from CI | `id`, `type` (`github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, `bitbucket-datacenter`), `url` (required for gitea, forgejo, azure-devops: `https://dev.azure.com/`, and bitbucket-datacenter: the base URL with its context path), `api_url`, `ca_file`, `writer`, `known_authors`, `automation_accounts`, `sign`, `limits`; replaces the shorthand | | `branch`, `branch_aliases` | `touchmark/`, none | the sync branch, and former names whose pull requests stay the hub's | | `previous_fingerprints` | none | the hub's fingerprints before it moved | | `opt_in_file` | `.engineering-assets.yml` | the opt-in file's path in targets | @@ -290,7 +290,7 @@ covers everything under it). At most 64 KiB. `.touchmark/operations.yml` (in the hub; `touchmark schema operations`): `recreate[]` (`target`, `head`: acts while the branch head is that commit), -`forget_declines[]` (`target`, `pr`: acts once; on Bitbucket Cloud and Azure DevOps while present), `allow_mass_close` (`max`, `until`), +`forget_declines[]` (`target`, `pr`: acts once; on Bitbucket and Azure DevOps while present), `allow_mass_close` (`max`, `until`), `adopt_unmarked` (`until`). Dates are `YYYY-MM-DD`, UTC, inclusive. Environment, one set per provider; `` is the provider id upper-cased with `-` as @@ -314,7 +314,9 @@ from its environment once read: clone (`--filter`). In CI: `fetch-depth: 0` (GitHub, Gitea), `GIT_DEPTH: "0"` (GitLab). 2. **Reader and writer are two different accounts** on every platform: two GitHub Apps, two GitLab service accounts (or group access tokens), two Gitea or Forgejo bot users, - two Azure DevOps users with personal access tokens (Code read; Code read & write). + two Azure DevOps users with personal access tokens (Code read; Code read & write), two + Bitbucket Data Center users with HTTP access tokens (repository read; repository write; + list the reader in `automation_accounts`, since it can decline pull requests there). The read key reaches any branch of the hub; the write key must not. 3. **The write key is visible to the default branch only.** GitHub: secrets in the environment `touchmark-distribute` with deployment branches *Selected* = the default diff --git a/docs/concepts/memory.md b/docs/concepts/memory.md index 40bda51..a4c729d 100644 --- a/docs/concepts/memory.md +++ b/docs/concepts/memory.md @@ -120,6 +120,17 @@ that property; descriptions do carry the tick box *Rebuild this branch* while a paused; and touchmark never reactivates an abandoned pull request, though Azure DevOps would let it. +## On Bitbucket Data Center + +Everything [above](#on-bitbucket-cloud) applies, with three differences. A person may +reopen a declined pull request there: it is then touchmark's open pull request again, +and no longer a decline; touchmark itself never reopens one. Bitbucket declines a pull +request without activity for four weeks by default, as its system user, which touchmark +counts as a bot: such a close waits for the cooldown, as a stale bot's does. And a user +who may only read a repository may decline its pull requests, the reader included: list +the reader in `automation_accounts`, so that a decline made with its token counts as a +bot's, not as the team's (`plan` warns until it is there). + ## Limits - Only the last 50 closed pull requests on the sync branches count. diff --git a/docs/guide/operations.md b/docs/guide/operations.md index a0ea0a8..1763a30 100644 --- a/docs/guide/operations.md +++ b/docs/guide/operations.md @@ -37,7 +37,7 @@ cleanup to stay safe: | Entry | Acts | Then | |---|---|---| | `recreate` | while the sync branch's head is exactly `head` | a new push to the branch makes it inert | -| `forget_declines` | once: touchmark marks pull request `pr` revoked; on Bitbucket Cloud and Azure DevOps, while the entry is present (see below) | inert | +| `forget_declines` | once: touchmark marks pull request `pr` revoked; on Bitbucket Cloud, Bitbucket Data Center and Azure DevOps, while the entry is present (see below) | inert | | `allow_mass_close` | up to `max` closes in a run, until the end of `until` (UTC) | inert after the date | | `adopt_unmarked` | until the end of `until` (UTC) | inert after the date | diff --git a/docs/guide/providers.md b/docs/guide/providers.md index c80472f..e222810 100644 --- a/docs/guide/providers.md +++ b/docs/guide/providers.md @@ -132,7 +132,8 @@ A provider of type `bitbucket` is Bitbucket Cloud at `https://bitbucket.org` (le out). `plan`, `distribute` and `doctor` work on it from a hub on GitHub or GitLab, from a hub on Bitbucket itself, whose CI is Bitbucket Pipelines (see [A hub on Bitbucket Cloud](../getting-started/bitbucket.md)), or wherever you run -touchmark; `setup` has nothing for it yet. Bitbucket Data Center is not supported. +touchmark; `setup` has nothing for it yet. Bitbucket Data Center has a provider type of +its own (see [Bitbucket Data Center](#bitbucket-data-center)). ```yaml # hub.yml @@ -301,6 +302,112 @@ renamed repositories answer; who can write pull request properties (whether proj Readers, through "Contribute to pull requests", can set or change `touchmark.marker`); and whether `Retry-After` comes on answers that went through. +## Bitbucket Data Center + +A provider of type `bitbucket-datacenter` is one Bitbucket Data Center instance, 8.19 or +later, and its `url` is the instance's base URL, context path included +(`https://git.example.com/bitbucket`); the REST API is under it, at +`/rest/api/latest`. `plan`, `distribute` and `doctor` work on it from a hub on GitHub +or GitLab, or wherever you run touchmark. Bitbucket Data Center has no CI of its own, so +a hub cannot live there, and `setup` has nothing for it. + +```yaml +# hub.yml +providers: + - id: bbdc + type: bitbucket-datacenter + url: https://git.example.com/bitbucket + writer: touchmark-writer # the writer user's slug + automation_accounts: [touchmark-reader] # the reader: see "The read key can decline" +``` + +**Accounts.** Use two users that only touchmark uses, each with a personal HTTP access +token. touchmark sends every token as `Authorization: Bearer`, to the API and to git, on +every version from 8.19. Project and repository access tokens work too, but they act as a +service user Bitbucket makes for the token: it has no email address, and its name shows +only in answers (`doctor` and the report name it). + +| | Reader | Writer | +|---|---|---| +| Variable | `TOUCHMARK_BBDC_READ_TOKEN` | `TOUCHMARK_BBDC_WRITE_TOKEN` | +| Token permissions | repository read | repository write; no admin | +| Repositories | read access to the targets | write access to the targets only (directly, through a group or the project); not admin, and no access to the hub | + +Bitbucket's API does not tell which token a request came with: `doctor` reports the +token's expiry and permissions as unknown, so keep a reminder to rotate them. + +- **Accounts are user slugs.** `writer`, `known_authors` and `automation_accounts` name + users by their slug, as their profile URL shows it (`/users/`); touchmark keeps + their numeric ids. +- **Targets.** `org` is a project, by its key (`org: ACME`, or its URL + `/projects/ACME`); `repo` is `/`, and web and clone + URLs such as `/projects/ACME/repos/billing` and `/scm/acme/billing.git` work. + `topics` are repository labels: an entry with `topics: [python]` takes the project's + repositories that carry the label `python`. Archived repositories are skipped, and so + are those whose default branch does not exist (as empty). A project key that lists no + repository is looked up: an unknown or changed key fails the provider's resolve, so no + stale pull request is swept on its account. Personal + repositories (`~user` projects) are not targets. A renamed repository changes its slug: + update `targets.yml`, or select it through `org`. +- **No labels.** Bitbucket Data Center has labels on repositories only, not on pull + requests: `pr.labels` sets nothing there, and touchmark records no label as set. +- **Drafts** are Bitbucket's own draft flag (`pr.draft`); on an instance that turned drafts + off, the pull request opens ready. +- **No default reviewers.** Bitbucket adds a repository's default reviewers to pull + requests made in its web interface, not to those made through the API: add reviewers + with a merge check or by hand. +- **Closing.** touchmark closes its own pull request by declining it, after writing the + close into its description. It never reopens or changes a declined pull request, though + Bitbucket would let it: a pull request touchmark closed stays closed, and the next + proposal is a new one. A person may reopen one; it is touchmark's open pull request + again. +- **Declines** are remembered as on Bitbucket Cloud, without writing to the declined pull + request: while a pull request is open, its marker records the state of the opt-in file, + and a person's decline holds while that file is parsed the same. `forget_declines` acts + while it is present (see + [Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)). +- **Who declined.** Bitbucket declines a pull request without activity for four weeks by + default, as its own system user: touchmark reads who declined a pull request from its + activities, and takes a decline by the system user, or by a service user, for an + automatic close, which holds the content back for `memory.auto_close_cooldown` only. Only + a person's decline holds until the opt-in file changes. +- **The read key can decline.** On Bitbucket Data Center a user who may only read a + repository may still decline its pull requests, and the reader's token is in every + branch of the hub: list the reader in `automation_accounts`, so that a decline made + with it is taken for an automatic close, not the team's decision. `plan` warns until + it is there. +- **No tick boxes, no HTML.** Bitbucket shows HTML in descriptions as text: the marker is + a Markdown link reference definition, descriptions carry no tick boxes, a paused pull + request asks for a `recreate` entry in `.touchmark/operations.yml`, and + `pr.intro_file` should hold no HTML, as on Bitbucket Cloud. A description holds 32 768 + characters; touchmark keeps its own under 30 000. +- **Branch permissions** are readable with repository admin rights only, which the writer + should not have: `doctor` shows `rules` as unknown. A push a branch permission refuses + is `blocked:rules:pull-request-only` when the branch takes changes through pull + requests only, `blocked:rules:branch-permissions` for another permission, and + `blocked:rules:pre-receive-hook` for a hook (Reject Force Push, Verify Committer, the + Jira commit checker). +- **Commit author.** touchmark authors the writer's commits with the user's email address, + which links them to the user, and with `@touchmark.invalid` for a service user, + which has none. +- **Pace.** Bitbucket's rate limit gives each user a bucket of 60 requests, refilled at 5 + a second, on each node, and answers 429 past it: touchmark reads 120 times a minute with + four targets at once and writes at most twice a second by default; `limits` overrides + it. + +What the driver assumes from the REST reference and Atlassian's documentation, without a +live instance: that `X-AUSERNAME` names the user of every authenticated answer (how +touchmark learns who a token acts as), and the names of project and repository token +users, and whether `GET /users?filter=` finds those users (without it, such a token +cannot tell who it is, and the provider stops); that an HTML comment in a description shows as text and a link reference +definition does not; whether a declined pull request can be edited; what +`at` with `direction=OUTGOING` selects in pull request listings; whether the dashboard +lists another user's pull requests to the reader (the sweep of stale pull requests; +without it the sweep is skipped as incomplete); who the activities name for an automatic +decline; the texts of push refusals other than pull-request-only; the headers of 429 +answers; and what the file APIs answer for a symlink (one may read as a file whose +content is its target) and for a missing path. + ## Moving - **A target moves to another provider on the same host** (a new App, a new group): add diff --git a/docs/index.md b/docs/index.md index bdb9164..f03124c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -67,9 +67,9 @@ of what each one would receive on every pull request to the hub. fails the run when a branch could see the write key. See the [security model](concepts/security.md). - **One hub, several platforms.** GitHub (github.com, GHE.com, GitHub Enterprise Server), - GitLab (gitlab.com and self-managed), Gitea, Forgejo, Bitbucket Cloud and Azure DevOps - Services, at once from one hub. Bitbucket and Azure DevOps are built from their - documentation and not yet run on a live account. + GitLab (gitlab.com and self-managed), Gitea, Forgejo, Bitbucket Cloud, Azure DevOps + Services and Bitbucket Data Center, at once from one hub. Bitbucket and Azure DevOps are + built from their documentation and not yet run on a live account. ## Where to start @@ -117,7 +117,7 @@ of what each one would receive on every pull request to the hub. | | | |---|---| -| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+, Bitbucket Cloud, Azure DevOps Services | +| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+, Bitbucket Cloud, Azure DevOps Services, Bitbucket Data Center 8.19+ | | Runs as | a GitHub Action, a container image, a single static binary | | Needs | git 2.31+ for `check`, `status`, `apply`; git 2.45+ for `plan` and `distribute` (the image brings its own) | | Configuration | `hub.yml`, `targets.yml`, `.touchmark/operations.yml` in the hub; `.engineering-assets.yml` in a target | diff --git a/docs/project/threat-model.md b/docs/project/threat-model.md index 5ad1f12..f7ce658 100644 --- a/docs/project/threat-model.md +++ b/docs/project/threat-model.md @@ -74,6 +74,7 @@ How to read the tests column: | Opt-in file: at most 64 KiB, a bounded number of YAML anchors and aliases, a strict schema | `config` (`decodeStrict`, `ParseOptIn`), the drivers' `ReadFile` with a size limit | `TestDecodeLimits`, `TestParseHubAnchors`, `TestParseOptIn`, `TestOptInIgnoreLimit`, `FuzzParse`, `TestPlanAssumeOptInTooLarge` | — | | Marker: at most 16 KiB, at most 64 KiB inflated, strict JSON | `marker` | `TestParseGzipBomb`, `TestParseRejectsHeader`, `TestParseRejectsData`, `TestParseRejectsPayload`, `TestParseMaxChanges`, `TestFindLargeBody`, `FuzzParse`, `FuzzFind` | — | | Azure DevOps: the marker lives in a pull request property (`touchmark.marker`), apart from the description; the author, the head, the head's repository (a fork) and the closer come from the API, never from the property; a new pull request whose property cannot be stored is abandoned, or marked through an edit when the abandon fails | `azuredevops` (`markerOf`, `toPR`, `storeNewMarker`), `decide.Identity`, `distribute` (`markCreated`) | `TestPRs`, `TestCreatePRMarkerAbandons`, `TestRunAzureDevOpsMarkerLost` *(git 2.45)* | The property is not protected. Anyone who may contribute to pull requests in the project may be able to write it through the API: project Readers have "Contribute to pull requests" by default, and the REST reference names only the scope `vso.code_write` (to confirm live, [providers.md](../guide/providers.md#azure-devops)). Such a person can block a target (`blocked:marker-invalid`, by breaking the marker of touchmark's open pull request, until an `adopt` entry takes it over) or, before the team abandons a pull request, write `closed.by=touchmark` into its marker so that the decline does not count and the content is proposed again. They cannot make a pull request of their own touchmark's: its author, head and fork come from the API. | +| Bitbucket Data Center: declining a pull request needs read access only, so the reader's token, which every branch of the hub can use, can decline touchmark's pull requests. touchmark reads who declined from the pull request's activities, never from its description; `plan` warns until the reader is in `automation_accounts`, whose declines count as automatic closes, as Bitbucket's system user's (the decline of inactive pull requests) and service users' do | `bitbucketdc` (`closer`, `toAccount`), `distribute` (`connect`), `decide.ClassifyClose` | `TestPRs`, `TestPRsCloser` (bitbucketdc), `TestPlanReaderCloses`, `TestRunBitbucketDataCenterDecline` *(git 2.45)* | With the reader in `automation_accounts`, whoever holds the read key can still hold content back for `memory.auto_close_cooldown`, and three such declines in a row count as a decline, which a `forget_declines` entry lifts. Without it, a decline made with the read key counts as the team's. Who the activities name for an automatic decline is not documented (to confirm live, [providers.md](../guide/providers.md#bitbucket-data-center)): a decline without a user counts as the system's. | | Azure Pipelines runs any logging command (`##vso[…]`) a step prints; under `TF_BUILD=True` touchmark breaks every `##vso[` in its output, also across writes, so no text of a platform or a target becomes a command | `cli.commandGuard` | `TestCommandGuard`, `TestMainBreaksLoggingCommands` | The template's own step prints `##vso[task.uploadsummary]` for the report files, which touchmark wrote. | | API answers: at most 32 MiB | `httpx` (`DefaultMaxBody`) | `TestMaxBody`, `TestOnlyHTTPXSpeaksHTTP` | — | | Objects of a target: a commit at most 1 MiB, a blob at most 8 MiB; more is `failed:git` | `gitx` (`MaxCommitObject`, `MaxReadBlob`), `distribute.failCommit` | `TestObjectLimits` | — | diff --git a/docs/reference/compatibility.md b/docs/reference/compatibility.md index 84b63c3..b4bdff6 100644 --- a/docs/reference/compatibility.md +++ b/docs/reference/compatibility.md @@ -32,7 +32,7 @@ changelog before you merge it. and reject fields they do not know. - **What touchmark leaves in repositories:** the marker in pull request bodies (`touchmark:v1`, an HTML comment, or a Markdown reference definition on Bitbucket - Cloud; on Azure DevOps the comment lives in the pull request property + Cloud and Bitbucket Data Center; on Azure DevOps the comment lives in the pull request property `touchmark.marker`), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`, `Touchmark-Content`, `Touchmark-Hub-Commit`), the control lines of a pull request body, the marker of the plan comment, and the sync branch's name. Every release reads what an diff --git a/docs/reference/hub.md b/docs/reference/hub.md index c96b72b..1c90da7 100644 --- a/docs/reference/hub.md +++ b/docs/reference/hub.md @@ -108,11 +108,11 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `security.writer_on_hub` | one of `refuse`, `guard` | What a writer that reaches the hub means. refuse: doctor fails it, and setup gitlab refuses a hub inside the group of targets. guard (GitLab): the writer may be a Developer of the hub; doctor verifies that it cannot get content onto the default branch, and plan refuses a hub merge request the writer opened or pushed to. Default: refuse. | | `providers` | list of objects | Platforms this hub delivers to. Targets refer to them by id. | | `providers[].id` | string, required | Name targets.yml uses for this provider, and the <ID> of its TOUCHMARK_<ID>_\* variables. | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud; azure-devops: one organization of Azure DevOps Services (bitbucket and azure-devops: plan, distribute and doctor; setup does not set up a hub on them yet). | -| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo, and for azure-devops: https://dev.azure.com/<organization>, one provider per organization. A target targets.yml names by web URL belongs to the provider whose url it lies under. | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, `bitbucket-datacenter`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud; azure-devops: one organization of Azure DevOps Services; bitbucket-datacenter: a Bitbucket Data Center instance (bitbucket, azure-devops and bitbucket-datacenter: plan, distribute and doctor; setup does not set up a hub on them yet). | +| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo, for bitbucket-datacenter (the base URL, with its context path, like https://bitbucket.example.com/bitbucket), and for azure-devops: https://dev.azure.com/<organization>, one provider per organization. A target targets.yml names by web URL belongs to the provider whose url it lies under. | | `providers[].api_url` | string | API URL, when it cannot be derived from url. | | `providers[].ca_file` | string | Hub file with extra CA certificates for this instance. | -| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}; on Azure DevOps the user's id, a GUID. | +| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}; on Azure DevOps the user's id, a GUID; on Bitbucket Data Center the user's slug. | | `providers[].known_authors` | list of strings, each up to 255 characters | Former writer accounts whose pull requests still count as this hub's own. | | `providers[].automation_accounts` | list of strings, each up to 255 characters | Accounts whose closing of a pull request is automatic, not a team's decision. | | `providers[].sign` | one of `auto`, `always` | auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto. | @@ -131,7 +131,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `packs..formerly` | list of strings, each up to 64 characters | Former names of this pack. Their history counts as this pack's own. | | `writer` | string, up to 255 characters | Single-provider shorthand: the writer account, as the platform names it (on GitHub the App's bot, <slug>[bot]). Without platform, the provider's type and URL come from the CI environment, or locally from the hub's origin remote. Not allowed with providers. | | `sign` | one of `auto`, `always` | Single-provider shorthand: commit signing. Not allowed with providers. | -| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | +| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, `bitbucket-datacenter` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | | `base_url` | string | Single-provider shorthand: the provider URL, for self-managed instances. Needs platform; not allowed with providers. | diff --git a/docs/reference/operations.md b/docs/reference/operations.md index 7a5b84d..adad83c 100644 --- a/docs/reference/operations.md +++ b/docs/reference/operations.md @@ -22,7 +22,7 @@ writes the path alone. Every entry limits itself, so a forgotten one does nothing: a branch head that moved, a decline already revoked, a date that passed. The exception is a `forget_declines` entry -for a Bitbucket Cloud or Azure DevOps target: touchmark never edits a declined pull +for a Bitbucket Cloud, Bitbucket Data Center or Azure DevOps target: touchmark never edits a declined pull request there, so it is never marked revoked, and the entry acts while it is present (see [Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)). `check` warns about entries past their date. @@ -40,7 +40,7 @@ warns about entries past their date. | `recreate` | list of objects | Rebuild a paused sync branch from scratch. Commits others added to it are dropped. An entry acts only while the branch head is the given commit. | | `recreate[].target` | string, up to 512 characters, required | One target repository: <provider>:<path> or <path>, as touchmark prints it. | | `recreate[].head` | string, required | Full commit id of the sync branch head to replace: 40 or 64 lowercase hex digits. | -| `forget_declines` | list of objects | Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud and Azure DevOps, where touchmark never edits a declined pull request, the entry acts while it is present. | +| `forget_declines` | list of objects | Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud, Bitbucket Data Center and Azure DevOps, where touchmark never edits a declined pull request, the entry acts while it is present. | | `forget_declines[].target` | string, up to 512 characters, required | One target repository: <provider>:<path> or <path>, as touchmark prints it. | | `forget_declines[].pr` | integer ≥ 1, required | Number of the declined pull request. | | `allow_mass_close` | object | Lift the guard against closing many pull requests in one run, up to max closes, until the date. | diff --git a/docs/reference/output.md b/docs/reference/output.md index 1ec5c5e..54d1bda 100644 --- a/docs/reference/output.md +++ b/docs/reference/output.md @@ -87,7 +87,7 @@ and the branch head before and after. | `assume_opt_in` | boolean | plan --assume-opt-in: for this report only, every target without an opt-in file, or whose opt-in file is not a regular file or is too large, counts as opted in; one whose opt-in file says enabled: false stays opted out. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, `bitbucket-datacenter`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].reader` | string | Login of the read account. | | `providers[].writer` | string | Login of the write account. | @@ -199,7 +199,7 @@ hub, for each provider's write identity, and for each target. See | `hub_checks[].detail` | string | What the check found; absent for a target the report does not name. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`, `bitbucket-datacenter`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].writer` | string | Login of the writer hub.yml names. | | `providers[].self` | string | Login the write credential acts as. | diff --git a/docs/reference/targets.md b/docs/reference/targets.md index ab25421..70809db 100644 --- a/docs/reference/targets.md +++ b/docs/reference/targets.md @@ -190,8 +190,8 @@ either way. | `targets[].provider` | string | Id of a provider in hub.yml. | | `targets[].packs` | list of strings, each up to 64 characters | — | | `targets[].opt_in` | one of `required`, `assumed` | required (the default): a repository the entry selects gets nothing until it adds the opt-in file. assumed: it counts as opted in without one, with the packs targets.yml gives it, until its opt-in file says enabled: false. One entry with assumed is enough to subscribe a repository. | -| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace, a Bitbucket workspace or an Azure DevOps organization (the provider's url; a project is selected with match) (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | -| `targets[].topics` | list of strings | Only repositories with all of these topics. Bitbucket and Azure DevOps have no topics: check refuses them on their entries; use match. Only with `org` or `group`. | +| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace, a Bitbucket workspace, a Bitbucket Data Center project (by its key) or an Azure DevOps organization (the provider's url; a project is selected with match) (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | +| `targets[].topics` | list of strings | Only repositories with all of these topics (repository labels on Bitbucket Data Center). Bitbucket Cloud and Azure DevOps have no topics: check refuses them on their entries; use match. Only with `org` or `group`. | | `targets[].subgroups` | boolean | Include repositories in nested groups. Default: true. Only with `org` or `group`. | | `targets[].forks` | boolean | Include forks. Default: false. Only with `org` or `group`. | | `targets[].match` | list of strings, each up to 512 characters | Only repositories whose full path, without the provider, matches at least one of these patterns: \* within one segment, \*\* any number of segments, ? one character, compared ignoring case, such as acme/svc-\* or platform/\*\*/api. Quote a pattern that starts with \*, which YAML reads as an alias. Only with `org` or `group`. | diff --git a/internal/cli/drivers.go b/internal/cli/drivers.go index 3242a4d..1ca4bc6 100644 --- a/internal/cli/drivers.go +++ b/internal/cli/drivers.go @@ -3,6 +3,7 @@ package cli import ( "github.com/bedrock-python/touchmark/internal/platform/azuredevops" "github.com/bedrock-python/touchmark/internal/platform/bitbucket" + "github.com/bedrock-python/touchmark/internal/platform/bitbucketdc" "github.com/bedrock-python/touchmark/internal/platform/gitea" "github.com/bedrock-python/touchmark/internal/platform/github" "github.com/bedrock-python/touchmark/internal/platform/gitlab" @@ -26,4 +27,6 @@ func init() { distributeDrivers["bitbucket"] = bitbucket.NewWriter planDrivers["azure-devops"] = azuredevops.NewReader distributeDrivers["azure-devops"] = azuredevops.NewWriter + planDrivers["bitbucket-datacenter"] = bitbucketdc.NewReader + distributeDrivers["bitbucket-datacenter"] = bitbucketdc.NewWriter } diff --git a/internal/cli/drivers_test.go b/internal/cli/drivers_test.go index 4816de7..56c62fc 100644 --- a/internal/cli/drivers_test.go +++ b/internal/cli/drivers_test.go @@ -4,9 +4,9 @@ import "testing" // TestDriversRegistered: this build plans and distributes on every // platform it supports: Gitea and Forgejo, one driver for both, GitLab, -// GitHub, Bitbucket Cloud and Azure DevOps Services. +// GitHub, Bitbucket Cloud, Azure DevOps Services and Bitbucket Data Center. func TestDriversRegistered(t *testing.T) { - for _, typ := range []string{"gitea", "forgejo", "gitlab", "github", "bitbucket", "azure-devops"} { + for _, typ := range []string{"gitea", "forgejo", "gitlab", "github", "bitbucket", "azure-devops", "bitbucket-datacenter"} { if planDrivers[typ] == nil || distributeDrivers[typ] == nil { t.Errorf("no %s driver: plan %v, distribute %v", typ, planDrivers[typ] != nil, distributeDrivers[typ] != nil) } diff --git a/internal/cli/setup.go b/internal/cli/setup.go index bc5491e..9b28376 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -105,6 +105,9 @@ func runSetup(ctx context.Context, e *env, o *options, s *setupOptions) (err err return configErrorf("setup does not set up azure-devops yet: set up a hub on Azure Pipelines by hand (the bot users and their "+ "personal access tokens, the pipeline and its build validation policy, the variable group and the environment), as %s shows; "+ "a hub on GitHub or GitLab delivers to Azure DevOps as one of its providers", docsurl.GettingStarted("azure-devops")) + case "bitbucket-datacenter": + return configErrorf("setup does not set up bitbucket-datacenter: Bitbucket Data Center has no CI of its own to run a hub; " + + "a hub on GitHub or GitLab delivers to Bitbucket Data Center as one of its providers") case "github", "gitlab": default: return usageErrorf("unknown platform %q: setup sets up github or gitlab", platform) diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go index 95ba042..85abe54 100644 --- a/internal/cli/setup_test.go +++ b/internal/cli/setup_test.go @@ -48,6 +48,7 @@ func TestSetupRefusals(t *testing.T) { {"unknown", nil, []string{"setup", "sourcehut", "--hub", h.dir}, `unknown platform "sourcehut"`}, {"azure-devops", nil, []string{"setup", "azure-devops", "--hub", h.dir}, "setup does not set up azure-devops yet"}, {"bitbucket", nil, []string{"setup", "bitbucket", "--hub", h.dir}, "setup does not set up bitbucket yet"}, + {"bitbucket-datacenter", nil, []string{"setup", "bitbucket-datacenter", "--hub", h.dir}, "Bitbucket Data Center has no CI of its own"}, {"gitea", nil, []string{"setup", "gitea", "--hub", h.dir}, "every branch the secrets"}, {"forgejo", nil, []string{"setup", "forgejo"}, "setup does not set up forgejo"}, {"flags-first", nil, []string{"setup", "--hub", h.dir, "gitea", "--dry-run"}, "setup does not set up gitea"}, diff --git a/internal/config/check.go b/internal/config/check.go index a5f3477..d9287f3 100644 --- a/internal/config/check.go +++ b/internal/config/check.go @@ -193,8 +193,11 @@ func (c *checker) exclude() { segs := strings.Split(pat.Path, "/") if typ := c.flatType(res.provider(pat.Provider)); typ != "" && len(segs) > 2 && !slices.Contains(segs, "**") { shape := "a " + typ + " repository path is owner/name" - if typ == "azure-devops" { + switch typ { + case "azure-devops": shape = "an Azure DevOps repository path is project/repository" + case "bitbucket-datacenter": + shape = "a Bitbucket Data Center repository path is PROJECT/repository" } c.warnf(TargetsFile, "exclude[%d]: %s has %d path segments, and %s, so it excludes nothing", i, ex, len(segs), shape) } @@ -266,13 +269,15 @@ func (c *checker) match() { } // flatType returns the type of the provider with id when its platform has no -// nested namespaces (github, gitea, forgejo, bitbucket, azure-devops): a -// repository path there is owner/name (project/repository on Azure DevOps). +// nested namespaces (github, gitea, forgejo, bitbucket, azure-devops, +// bitbucket-datacenter): a repository path there is owner/name +// (project/repository on Azure DevOps, the project's key and the +// repository's slug on Bitbucket Data Center). // It returns "" for any other type, and for a provider hub.yml does not // list (the implicit one, whose type the CI tells). func (c *checker) flatType(id string) string { switch typ := c.providerType(id); typ { - case "github", "gitea", "forgejo", "bitbucket", "azure-devops": + case "github", "gitea", "forgejo", "bitbucket", "azure-devops", "bitbucket-datacenter": return typ } return "" diff --git a/internal/config/check_test.go b/internal/config/check_test.go index 4c9b7f2..06a014a 100644 --- a/internal/config/check_test.go +++ b/internal/config/check_test.go @@ -209,6 +209,24 @@ func TestCheck(t *testing.T) { "targets.yml: targets[1].match[2]: Billing/api/x has 3 path segments, and an Azure DevOps repository path is project/repository, so it selects nothing", }, }, + { + // Topics are repository labels on Bitbucket Data Center. + name: "topics and paths on bitbucket-datacenter", + hub: &Hub{ID: "acme-eng", Providers: []Provider{ + {ID: "bbdc", Type: "bitbucket-datacenter", URL: "https://git.example.com/bitbucket"}, + }}, + targets: &Targets{ + Targets: []Entry{ + {Org: "ACME", Topics: []string{"python"}}, + {Org: "ACME", Match: []string{"ACME/svc-*", "ACME/api/x"}}, + }, + Exclude: []string{"LEGACY/old/x"}, + }, + warnings: []string{ + "targets.yml: exclude[0]: LEGACY/old/x has 3 path segments, and a Bitbucket Data Center repository path is PROJECT/repository, so it excludes nothing", + "targets.yml: targets[1].match[1]: ACME/api/x matches no repository directly under ACME (a bitbucket-datacenter namespace has no subgroups), so it selects nothing", + }, + }, { name: "listed and excluded", hub: selectHub(), diff --git a/internal/config/config.go b/internal/config/config.go index 6d5bc49..2f3c6c5 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -103,7 +103,7 @@ type Security struct { type Provider struct { ID string `yaml:"id"` - Type string `yaml:"type"` // github | gitlab | gitea | forgejo | bitbucket | azure-devops + Type string `yaml:"type"` // github | gitlab | gitea | forgejo | bitbucket | azure-devops | bitbucket-datacenter URL string `yaml:"url"` APIURL string `yaml:"api_url"` CAFile string `yaml:"ca_file"` diff --git a/internal/config/hub.go b/internal/config/hub.go index bd1da1d..cb74d4b 100644 --- a/internal/config/hub.go +++ b/internal/config/hub.go @@ -27,7 +27,7 @@ const ( // Enumerations of hub.yml. var ( - providerTypes = []string{"github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops"} + providerTypes = []string{"github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops", "bitbucket-datacenter"} linkHubModes = []string{"auto", "always", "never"} isolationModes = []string{"platform", "external", "none"} privateTargets = []string{"skip", "deliver"} @@ -254,7 +254,7 @@ func checkProvider(pr *Provider, f providerFields, p *problems) { if err := checkURL(pr.URL); err != nil { p.errorf(f.name("url"), "%v", err) } - } else if pr.Type == "gitea" || pr.Type == "forgejo" { + } else if pr.Type == "gitea" || pr.Type == "forgejo" || pr.Type == "bitbucket-datacenter" { p.errorf(f.name("url"), "required for %s", pr.Type) } else if pr.Type == "azure-devops" { p.errorf(f.name("url"), "required for azure-devops: https://dev.azure.com/") diff --git a/internal/config/providers.go b/internal/config/providers.go index 64278ae..534c113 100644 --- a/internal/config/providers.go +++ b/internal/config/providers.go @@ -469,6 +469,10 @@ func derivedAPIURL(typ, webURL, hostname, host string) string { // The REST API of an organization is under its web URL, // https://dev.azure.com//_apis. return webURL + case "bitbucket-datacenter": + // The REST API of Bitbucket Data Center is under its base URL, + // context path included: /rest/api/latest. + return webURL + "/rest/api/latest" } return webURL + "/api/v1" } diff --git a/internal/config/providers_test.go b/internal/config/providers_test.go index 20b80f3..de2d5b3 100644 --- a/internal/config/providers_test.go +++ b/internal/config/providers_test.go @@ -42,6 +42,8 @@ func TestResolveProviders(t *testing.T) { {ID: "bb-test", Type: "bitbucket", URL: "http://localhost:8080", APIURL: "http://localhost:8080/2.0"}, {ID: "ado", Type: "azure-devops", URL: "https://dev.azure.com/acme/"}, {ID: "ado-test", Type: "azure-devops", URL: "http://localhost:8081/acme", APIURL: "http://localhost:8081/acme"}, + {ID: "bbdc", Type: "bitbucket-datacenter", URL: "https://git.example.com/bitbucket/"}, + {ID: "bbdc-root", Type: "bitbucket-datacenter", URL: "https://bitbucket.example.com:7990"}, }} want := map[string]resolvedURLs{ "gh": {"https://github.com", "github.com", "https://api.github.com", "https://api.github.com/graphql", "TOUCHMARK_GH_"}, @@ -60,6 +62,10 @@ func TestResolveProviders(t *testing.T) { "bb-test": {"http://localhost:8080", "localhost:8080", "http://localhost:8080/2.0", "", "TOUCHMARK_BB_TEST_"}, "ado": {"https://dev.azure.com/acme", "dev.azure.com", "https://dev.azure.com/acme", "", "TOUCHMARK_ADO_"}, "ado-test": {"http://localhost:8081/acme", "localhost:8081", "http://localhost:8081/acme", "", "TOUCHMARK_ADO_TEST_"}, + // The REST API of Bitbucket Data Center is under its base URL, context + // path included. + "bbdc": {"https://git.example.com/bitbucket", "git.example.com", "https://git.example.com/bitbucket/rest/api/latest", "", "TOUCHMARK_BBDC_"}, + "bbdc-root": {"https://bitbucket.example.com:7990", "bitbucket.example.com:7990", "https://bitbucket.example.com:7990/rest/api/latest", "", "TOUCHMARK_BBDC_ROOT_"}, } // The CI environment does not matter for declared providers, except // GITHUB_API_URL on the hub's own host (below). @@ -333,6 +339,11 @@ func TestResolveProvidersErrors(t *testing.T) { hub: &Hub{Providers: []Provider{{ID: "srht", Type: "sourcehut"}}}, want: `hub.yml: providers[0]: type: "sourcehut" must be one of github, gitlab, gitea, forgejo, bitbucket, azure-devops`, }, + { + name: "bitbucket-datacenter without url", + hub: &Hub{Providers: []Provider{{ID: "bbdc", Type: "bitbucket-datacenter"}}}, + want: "hub.yml: providers[0]: url: required for bitbucket-datacenter", + }, { name: "azure-devops without url", hub: &Hub{Providers: []Provider{{ID: "ado", Type: "azure-devops"}}}, diff --git a/internal/config/urls.go b/internal/config/urls.go index 65ea3ac..e5899cf 100644 --- a/internal/config/urls.go +++ b/internal/config/urls.go @@ -276,11 +276,17 @@ func (r urlResolver) resolve(s string, k urlKind, want, fallback string) (string p := r.providers[match[0]] base := r.sites[match[0]].segs segs := slices.Clone(u.segs[len(base):]) - if p.Type == "azure-devops" { + switch p.Type { + case "azure-devops": var err error if segs, err = azureSegments(segs, base, k); err != nil { return "", fmt.Errorf("%s: %w", s, err) } + case "bitbucket-datacenter": + var err error + if segs, err = dataCenterSegments(segs, k); err != nil { + return "", fmt.Errorf("%s: %w", s, err) + } } if last := len(segs) - 1; k != urlNamespace && len(segs[last]) > len(".git") && strings.HasSuffix(strings.ToLower(segs[last]), ".git") { // A repository URL may end in .git; dropping it from a pattern @@ -349,6 +355,32 @@ func azureSegments(segs, base []string, k urlKind) ([]string, error) { return nil, errors.New(repoForm) } +// dataCenterSegments returns the path a Bitbucket Data Center URL of kind k +// names, from segs, its path segments after the provider's url (which +// keeps any context path): / for a repository's page, +// /projects//repos/[/browse…], or its clone URL, +// /scm//.git; for a project's page, /projects/; and +// for an exclude entry the same, or a pattern of a project's repositories +// (/projects//repos/*). Personal repositories (/users//repos/…, +// project key ~) are not targets: their paths would not pass +// targets.yml's path rules. +func dataCenterSegments(segs []string, k urlKind) ([]string, error) { + const repoForm = "a Bitbucket Data Center repository URL is /projects//repos/ (or its clone URL, /scm//.git)" + switch { + case k == urlNamespace && len(segs) == 2 && segs[0] == "projects": + return []string{segs[1]}, nil + case k == urlNamespace: + return nil, errors.New("a Bitbucket Data Center org entry names a project, /projects/") + case len(segs) >= 4 && segs[0] == "projects" && segs[2] == "repos" && (len(segs) == 4 || segs[4] == "browse" && k != urlExclude): + return []string{segs[1], segs[3]}, nil + case len(segs) == 3 && segs[0] == "scm": + return []string{segs[1], segs[2]}, nil + case k == urlExclude && len(segs) >= 4 && segs[0] == "projects" && segs[2] == "repos": + return append([]string{segs[1]}, segs[3:]...), nil + } + return nil, errors.New(repoForm) +} + // checkURLShape checks the path segments a URL of kind k names on a // platform of type typ: GitHub, Gitea, Forgejo and Bitbucket have no nested // namespaces, so an exclude URL of more than owner/name, without a "**" @@ -357,6 +389,12 @@ func azureSegments(segs, base []string, k urlKind) ([]string, error) { // project (a file, a merge request), not at it. func checkURLShape(typ string, segs []string, k urlKind) error { switch typ { + case "bitbucket-datacenter": + // dataCenterSegments has reduced the URL to / or . + if k == urlExclude && len(segs) > 2 && !slices.Contains(segs, "**") { + return errors.New("a Bitbucket Data Center repository URL is /projects//repos/; " + + "this one points inside a repository and would exclude nothing") + } case "github", "gitea", "forgejo", "bitbucket": switch { case k == urlRepo && len(segs) != 2: diff --git a/internal/config/urls_test.go b/internal/config/urls_test.go index 620e8d3..c2bc739 100644 --- a/internal/config/urls_test.go +++ b/internal/config/urls_test.go @@ -443,3 +443,72 @@ providers: } } } + +// TestResolveURLsBitbucketDataCenter: a Bitbucket Data Center provider is +// one instance, often under a context path; its repository URLs are +// /projects//repos/ (with /browse… after them in a browser) or +// clone URLs, /scm//.git, and a project's URL is the namespace +// of an org entry. +func TestResolveURLsBitbucketDataCenter(t *testing.T) { + provs := resolvedProviders(t, `version: 1 +id: acme-eng +providers: + - id: bbdc + type: bitbucket-datacenter + url: https://git.example.com/bitbucket +`) + for _, tc := range []struct { + entry Entry + want string // the resolved value, or a substring of the error after "!" + }{ + {Entry{Repo: "https://git.example.com/bitbucket/projects/ACME/repos/api"}, "bbdc:ACME/api"}, + {Entry{Repo: "https://git.example.com/bitbucket/projects/ACME/repos/api/browse"}, "bbdc:ACME/api"}, + {Entry{Repo: "https://git.example.com/bitbucket/projects/ACME/repos/api/browse/src/main.go"}, "bbdc:ACME/api"}, + {Entry{Repo: "https://git.example.com/bitbucket/scm/acme/api.git"}, "bbdc:acme/api"}, + {Entry{Org: "https://git.example.com/bitbucket/projects/ACME"}, "bbdc:ACME"}, + {Entry{Org: "https://git.example.com/bitbucket/projects/ACME/"}, "bbdc:ACME"}, + {Entry{Org: "https://git.example.com/bitbucket/ACME"}, "!a Bitbucket Data Center org entry names a project, /projects/"}, + {Entry{Repo: "https://git.example.com/bitbucket/ACME/api"}, "!a Bitbucket Data Center repository URL is /projects//repos/"}, + {Entry{Repo: "https://git.example.com/bitbucket/projects/ACME/repos/api/pull-requests/3"}, "!a Bitbucket Data Center repository URL is"}, + {Entry{Repo: "https://git.example.com/bitbucket/users/jane/repos/notes"}, "!a Bitbucket Data Center repository URL is"}, + {Entry{Repo: "https://git.example.com/projects/ACME/repos/api"}, "!is not under the url of any provider"}, + } { + got, err := ResolveURLs(&Targets{Targets: []Entry{tc.entry}}, provs) + if want, ok := strings.CutPrefix(tc.want, "!"); ok { + if err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("%+v: error %v, want one with %q", tc.entry, err, want) + } + continue + } + if err != nil { + t.Errorf("%+v: %v", tc.entry, err) + continue + } + if v := selectorValue(&got.Targets[0]); v != tc.want { + t.Errorf("%+v: %q, want %q", tc.entry, v, tc.want) + } + } + for _, tc := range []struct{ ex, want string }{ + {"https://git.example.com/bitbucket/projects/LEGACY/repos/old", "bbdc:LEGACY/old"}, + {"https://git.example.com/bitbucket/projects/LEGACY/repos/old-*", "bbdc:LEGACY/old-*"}, + {"https://git.example.com/bitbucket/projects/LEGACY/repos/*", "bbdc:LEGACY/*"}, + {"https://git.example.com/bitbucket/scm/legacy/old.git", "bbdc:legacy/old"}, + {"https://git.example.com/bitbucket/projects/LEGACY/repos/old/browse", "!points inside a repository and would exclude nothing"}, + {"https://git.example.com/bitbucket/projects/LEGACY", "!a Bitbucket Data Center repository URL is"}, + } { + got, err := ResolveURLs(&Targets{Exclude: []string{tc.ex}}, provs) + if want, ok := strings.CutPrefix(tc.want, "!"); ok { + if err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("exclude %s: error %v, want one with %q", tc.ex, err, want) + } + continue + } + if err != nil { + t.Errorf("exclude %s: %v", tc.ex, err) + continue + } + if got.Exclude[0] != tc.want { + t.Errorf("exclude %s: %q, want %q", tc.ex, got.Exclude[0], tc.want) + } + } +} diff --git a/internal/distribute/commit.go b/internal/distribute/commit.go index 2031098..65fe4d8 100644 --- a/internal/distribute/commit.go +++ b/internal/distribute/commit.go @@ -426,13 +426,16 @@ func (r *run) commitPerson(p *provider) gitx.Person { // the commit then names the bot by its UUID, links to no account, and // can never be taken for someone else's. Azure DevOps has none either: its // driver gives the user's sign-in address when connectionData shows one, -// else the author is "@touchmark.invalid". +// else the author is "@touchmark.invalid". Nor has Bitbucket +// Data Center: its driver gives the user's address, and the service user +// of a project or repository token, which has none, is +// "@touchmark.invalid". func noReplyEmail(typ, host, id, login string) string { if h, _, err := net.SplitHostPort(host); err == nil { host = h } switch typ { - case "bitbucket", "azure-devops": + case "bitbucket", "azure-devops", "bitbucket-datacenter": return strings.Trim(cmp.Or(id, login), "{}") + "@touchmark.invalid" case "gitlab": if id != "" { diff --git a/internal/distribute/keptbody_test.go b/internal/distribute/keptbody_test.go index 10003aa..202e4d4 100644 --- a/internal/distribute/keptbody_test.go +++ b/internal/distribute/keptbody_test.go @@ -28,7 +28,7 @@ func TestKeptBody(t *testing.T) { if unticked == read { t.Fatal("fixture: Untick changed nothing") } - for _, f := range []fake.Flavor{fake.GitHub, fake.GitLab, fake.Gitea, fake.Forgejo, fake.Bitbucket, fake.AzureDevOps} { + for _, f := range []fake.Flavor{fake.GitHub, fake.GitLab, fake.Gitea, fake.Forgejo, fake.Bitbucket, fake.AzureDevOps, fake.BitbucketDataCenter} { x := &targetExec{t: &target{prov: &provider{caps: fake.CapsFor(f)}}} for _, body := range []string{read, unticked} { got, kept := x.keptBody(read, body, newLine) diff --git a/internal/distribute/resolve.go b/internal/distribute/resolve.go index 41f489d..62929eb 100644 --- a/internal/distribute/resolve.go +++ b/internal/distribute/resolve.go @@ -199,6 +199,10 @@ func (r *run) connect(ctx context.Context, p *provider) bool { if !r.lookupAutomation(ctx, p) { return false } + if r.mode == ModePlan && caps.ReaderCloses && self.ID != "" && !p.automation[self.ID] { + r.warnf("provider %s: the read credential's account %s can decline pull requests on %s; list it in automation_accounts, "+ + "so that a pull request declined with the read key is not taken for the team's decision", p.cfg.ID, self.Login, caps.Flavor) + } switch { case len(p.ids) == 0: r.warnf("provider %s: touchmark cannot recognize its own pull requests, so an open pull request on a sync branch counts as someone else's", p.cfg.ID) diff --git a/internal/distribute/run_bitbucketdc_test.go b/internal/distribute/run_bitbucketdc_test.go new file mode 100644 index 0000000..9b7e883 --- /dev/null +++ b/internal/distribute/run_bitbucketdc_test.go @@ -0,0 +1,110 @@ +package distribute + +import ( + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/platform/fake" + "github.com/bedrock-python/touchmark/internal/report" +) + +// The delivery scenarios on Bitbucket Data Center's flavor of the fake: +// Bitbucket Cloud's (no labels, the marker as a Markdown reference +// definition, no tick boxes, touchmark writes no declined pull request) +// with a shorter description, and declined pull requests that a person may +// reopen. Every run goes through Run's full phase C in git mode. + +// dcWorld is a git world of the Bitbucket Data Center flavor, whose hub +// lists the reader among automation_accounts, as the guide asks (else a +// plan warns, and a dry run, under the writer, does not). +func dcWorld(t *testing.T) *gitWorld { + t.Helper() + g := newGitWorld(t, fake.WithFlavor(fake.BitbucketDataCenter)) + if c := g.p.Caps(); !c.ClosedImmutable || !c.NoLabels || c.BodyControls() || c.MaxBody != 30000 || !c.ReaderCloses { + t.Fatalf("fixture: caps %+v", c) + } + g.hubYML = dcHubYML + return g +} + +// dcHubYML is the default hub.yml with the reader among +// automation_accounts. +var dcHubYML = strings.Replace(defaultHubYML, " known_authors:", " automation_accounts: [\"acme-read[bot]\"]\n known_authors:", 1) + +// TestRunBitbucketDataCenterDecline: a person declines touchmark's pull +// request: declined, with no write to it, run after run. A person reopens +// it: it is touchmark's open pull request again, and nothing needs +// writing. A bot declines it: an automatic close, which holds the content +// back for the cooldown only. +func TestRunBitbucketDataCenterDecline(t *testing.T) { + t.Parallel() + g := dcWorld(t) + api := g.optedIn("acme/api", nil) + t0 := time.Date(2026, 9, 2, 12, 0, 0, 0, time.UTC) + + g.bbStep("the first run", nil, report.OutcomeOpened, "", 1, 2) + pr := g.bbCheckBody(g.p.PR(api.ID, 1)) + if pr.Data.OptIn == "" { + t.Error("#1 records no opt-in state") + } + + g.p.SetPRState(api.ID, 1, platform.Closed, &g.person, t0) + g.ok() + g.bbStep("declined", nil, report.OutcomeDeclined, "", 1, 0) + + // A person reopens it. + g.p.SetPRState(api.ID, 1, platform.Open, nil, time.Time{}) + g.ok() + g.bbStep("reopened", nil, report.OutcomeUnchanged, "", 1, 0) + if got := g.p.PR(api.ID, 1); got.State != platform.Open { + t.Errorf("#1 is %s after the reopening", got.State) + } + + // A bot declines it (as Bitbucket's system user does after weeks + // without activity): not the team's decision. + g.p.SetPRState(api.ID, 1, platform.Closed, &g.bot, t0.Add(time.Hour)) + g.ok() + plan := g.both(nil) + if tg := targetOf(t, plan, "gh:acme/api"); tg.Outcome == report.OutcomeDeclined { + t.Errorf("a bot's decline is the team's: %s %s", tg.Outcome, tg.Reason) + } +} + +// TestPlanReaderCloses: on a platform where the read credential can close +// pull requests, a plan warns until its account is in automation_accounts. +func TestPlanReaderCloses(t *testing.T) { + const warning = "the read credential's account acme-read[bot] can decline pull requests on bitbucket-datacenter" + w := newWorld(t, fake.WithFlavor(fake.BitbucketDataCenter)) + w.optedIn("acme/api", nil) + if rep := w.plan(w.deps()); !hasWarning(rep.Warnings, warning) { + t.Errorf("warnings %q", rep.Warnings) + } + w.hubYML = dcHubYML + if rep := w.plan(w.deps()); hasWarning(rep.Warnings, "can decline pull requests") { + t.Errorf("warnings %q with the reader among automation_accounts", rep.Warnings) + } + gh := newWorld(t) + gh.optedIn("acme/api", nil) + if rep := gh.plan(gh.deps()); hasWarning(rep.Warnings, "can decline pull requests") { + t.Errorf("warnings %q on GitHub", rep.Warnings) + } +} + +// TestRunBitbucketDataCenterSelfClose: touchmark closes its own pull +// request in one edit that writes the closed marker, then declines it. +func TestRunBitbucketDataCenterSelfClose(t *testing.T) { + t.Parallel() + g := dcWorld(t) + api := g.optedIn("acme/api", nil) + g.bbStep("the first run", nil, report.OutcomeOpened, "", 1, 2) + g.push(api, "main", g.person, baseFiles...) + g.bbStep("no-diff", nil, report.OutcomeClosed, "no-diff", 1, 3) + pr := g.p.PR(api.ID, 1) + m := g.bbCheckBody(pr) + if pr.State != platform.Closed || m.Data.Closed == nil || m.Data.Closed.By != "touchmark" { + t.Errorf("#1: %s, closed %+v", pr.State, m.Data.Closed) + } + g.bbStep("after the close", nil, report.OutcomeUnchanged, "", 0, 0) +} diff --git a/internal/distribute/sim_crash_test.go b/internal/distribute/sim_crash_test.go index 581f34b..96e917e 100644 --- a/internal/distribute/sim_crash_test.go +++ b/internal/distribute/sim_crash_test.go @@ -70,6 +70,7 @@ var simCrashCases = []simCrashCase{ {name: "open-gitea", flavor: fake.Gitea, setup: func(*simWorld) {}}, {name: "open-bitbucket", flavor: fake.Bitbucket, setup: func(*simWorld) {}}, {name: "open-azure-devops", flavor: fake.AzureDevOps, setup: func(*simWorld) {}}, + {name: "open-bitbucket-datacenter", flavor: fake.BitbucketDataCenter, setup: func(*simWorld) {}}, {name: "update", flavor: fake.GitHub, setup: func(w *simWorld) { w.run(ModeDistribute, nil) w.ship("AGENTS.md", 2) @@ -126,6 +127,10 @@ var simCrashCases = []simCrashCase{ w.run(ModeDistribute, nil) w.excluded["api"] = true }}, + {name: "close-no-diff-bitbucket-datacenter", flavor: fake.BitbucketDataCenter, setup: func(w *simWorld) { + w.run(ModeDistribute, nil) + w.mustDo(w.push(w.target("api"), "main", maps.Clone(w.files[simBase]))) + }}, // On Azure DevOps the close stores the closed marker in the property, // then abandons the pull request. {name: "close-no-diff-azure-devops", flavor: fake.AzureDevOps, setup: func(w *simWorld) { diff --git a/internal/distribute/sim_property_test.go b/internal/distribute/sim_property_test.go index 135be00..8f4cfce 100644 --- a/internal/distribute/sim_property_test.go +++ b/internal/distribute/sim_property_test.go @@ -125,6 +125,26 @@ func TestPropertyDistributeAzureDevOps(t *testing.T) { } } +// TestPropertyDistributeBitbucketDataCenter runs the scenarios of the +// property test on Bitbucket Data Center's flavor of the fake: Bitbucket +// Cloud's capabilities with a description of 30 000 bytes. By default 10 +// scenarios run; TOUCHMARK_PROPERTY_BBDC_SCENARIOS sets how many, the +// rounds and the first seed are the property test's. +func TestPropertyDistributeBitbucketDataCenter(t *testing.T) { + needDeliveryGit(t) + simHeavy(t) + n := simEnvInt(t, "TOUCHMARK_PROPERTY_BBDC_SCENARIOS", 10) + rounds := max(1, simEnvInt(t, "TOUCHMARK_PROPERTY_ROUNDS", 2)) + first := simEnvInt(t, "TOUCHMARK_PROPERTY_SEED", 1) + for i := range n { + seed := uint64(first + i) + t.Run(fmt.Sprintf("seed=%d", seed), func(t *testing.T) { + t.Parallel() + runSimScenarioOf(t, seed, rounds, nil, fake.BitbucketDataCenter) + }) + } +} + // simEnvInt reads a positive number from the environment, def when unset. func simEnvInt(t *testing.T, name string, def int) int { t.Helper() diff --git a/internal/distribute/sim_world_test.go b/internal/distribute/sim_world_test.go index 846a6e2..4b4d332 100644 --- a/internal/distribute/sim_world_test.go +++ b/internal/distribute/sim_world_test.go @@ -52,9 +52,9 @@ const ( var ( simHosts = map[fake.Flavor]string{fake.GitHub: "github.com", fake.GitLab: "gitlab.example.com", fake.Gitea: "gitea.example.com", fake.Forgejo: "forgejo.example.com", fake.Bitbucket: "bitbucket.org", - fake.AzureDevOps: "dev.azure.com"} + fake.AzureDevOps: "dev.azure.com", fake.BitbucketDataCenter: "bitbucket.example.com"} simTypes = map[fake.Flavor]string{fake.GitHub: "github", fake.GitLab: "gitlab", fake.Gitea: "gitea", fake.Forgejo: "forgejo", - fake.Bitbucket: "bitbucket", fake.AzureDevOps: "azure-devops"} + fake.Bitbucket: "bitbucket", fake.AzureDevOps: "azure-devops", fake.BitbucketDataCenter: "bitbucket-datacenter"} ) // simTokens are the git tokens of the accounts of a world. diff --git a/internal/distribute/units_test.go b/internal/distribute/units_test.go index afb6f3d..d0992fa 100644 --- a/internal/distribute/units_test.go +++ b/internal/distribute/units_test.go @@ -37,6 +37,7 @@ func TestNoReplyEmail(t *testing.T) { {"bitbucket", "bitbucket.org", "{583f7ec5-ed93-49a9-b449-cfc4556cd7f8}", "{583f7ec5-ed93-49a9-b449-cfc4556cd7f8}", "583f7ec5-ed93-49a9-b449-cfc4556cd7f8@touchmark.invalid"}, {"bitbucket", "bitbucket.org", "", "{583f7ec5-ed93-49a9-b449-cfc4556cd7f8}", "583f7ec5-ed93-49a9-b449-cfc4556cd7f8@touchmark.invalid"}, {"azure-devops", "dev.azure.com", "56b0f042-b05e-86d9-b0f7-c5ddb1e76385", "56b0f042-b05e-86d9-b0f7-c5ddb1e76385", "56b0f042-b05e-86d9-b0f7-c5ddb1e76385@touchmark.invalid"}, + {"bitbucket-datacenter", "git.example.com", "14", "project_1_bot", "14@touchmark.invalid"}, } { if got := noReplyEmail(tc.typ, tc.host, tc.id, tc.login); got != tc.want { t.Errorf("noReplyEmail(%q, %q, %q, %q) = %q, want %q", tc.typ, tc.host, tc.id, tc.login, got, tc.want) @@ -272,6 +273,32 @@ func TestPushRuleBitbucketBranchRestriction(t *testing.T) { } } +// TestPushRuleBitbucketDataCenter: Bitbucket Data Center refuses a push +// that a branch permission forbids with "pre-receive hook declined" and the +// permission's own message (Atlassian's tracker, BSERV-7759): a branch +// whose changes need a pull request is rules:pull-request-only; another +// permission, whose message only points at the branch permissions, +// rules:branch-permissions; a hook of its own, rules:pre-receive-hook. +func TestPushRuleBitbucketDataCenter(t *testing.T) { + const oid = "3f786850e387550fdab836ed7e6dc881de23001b" + res := gitx.ParsePush("To https://git.example.com/bitbucket/scm/acme/api.git\n!\t"+oid+":refs/heads/main\t[remote rejected] (pre-receive hook declined)\nDone\n", + "remote: Branch refs/heads/main can only be modified through pull requests.\n"+ + "remote: Check your branch permissions configuration with the project administrator.\n"+ + "To https://git.example.com/bitbucket/scm/acme/api.git\n ! [remote rejected] "+oid+" -> main (pre-receive hook declined)\n"+ + "error: failed to push some refs to 'https://git.example.com/bitbucket/scm/acme/api.git'\n") + if outcome, reason := pushOutcome(res); outcome != report.OutcomeBlocked || reason != "rules:pull-request-only" { + t.Errorf("pull-request-only: %s:%s (%q)", outcome, reason, res.Message) + } + for msg, want := range map[string]string{ + "remote: You can't push to refs/heads/touchmark/acme-eng.\nremote: Check your branch permissions configuration with the project administrator.": "branch-permissions", + "remote: Rejected by the Jira commit checker.\n[remote rejected] (pre-receive hook declined)": "pre-receive-hook", + } { + if got := pushRule(msg); got != want { + t.Errorf("pushRule(%q) = %q, want %q", msg, got, want) + } + } +} + // TestPushAzureRefusals: Azure Repos refuses a push to a branch a policy // protects with TF402455 (blocked:rules:policy) and a push the identity // lacks a Git permission for with TF401027 (blocked:permission:push). diff --git a/internal/distribute/write.go b/internal/distribute/write.go index 3a5fda2..f14810c 100644 --- a/internal/distribute/write.go +++ b/internal/distribute/write.go @@ -298,8 +298,12 @@ func pushOutcome(res gitx.PushResult) (report.Outcome, string) { // which comes with "pre-receive hook declined"), Azure Repos' branch // policies (TF402455: "Pushes to this branch are not permitted; you must // use a pull request to update this branch": rule "policy", the name Azure -// DevOps gives such rules), a pre-receive hook, a deletion rule; "push" when -// the message names none. +// DevOps gives such rules), Bitbucket Data Center's branch permissions +// ("Branch refs/heads/… can only be modified through pull requests": +// "pull-request-only", the permission's type; another of them, which +// Atlassian's documentation does not quote, "Check your branch permissions +// configuration…": "branch-permissions"), a pre-receive hook, a deletion +// rule; "push" when the message names none. func pushRule(msg string) string { m := strings.ToLower(msg) switch { @@ -312,6 +316,10 @@ func pushRule(msg string) string { case strings.Contains(m, "gh006"), strings.Contains(m, "protected branch"), strings.Contains(m, "permission denied to update branch"): return "protected-branch" + case strings.Contains(m, "can only be modified through pull requests"): + return "pull-request-only" + case strings.Contains(m, "branch permissions"): + return "branch-permissions" case strings.Contains(m, "delet"): return "deletion" case strings.Contains(m, "pre-receive"): diff --git a/internal/platform/bitbucketdc/bitbucketdc.go b/internal/platform/bitbucketdc/bitbucketdc.go new file mode 100644 index 0000000..10ce5ec --- /dev/null +++ b/internal/platform/bitbucketdc/bitbucketdc.go @@ -0,0 +1,107 @@ +// Package bitbucketdc is the platform driver for Bitbucket Data Center +// (self-managed Bitbucket, formerly Bitbucket Server; REST API 1.0, served +// as /rest/api/latest), 8.19 and later. Bitbucket Cloud has another API +// (package bitbucket). +// +// NewReader serves plan; NewWriter serves distribute and doctor: it opens, +// edits, declines and comments on pull requests and pushes over git, and +// reports the writer's access for doctor (platform.Checker). +// +// Authentication is an HTTP access token, personal (of a user) or of a +// project or repository (of a service user Bitbucket makes for it). The +// API and git over HTTPS both get "Authorization: Bearer ", which +// every kind of token takes on 8.19 to 10.x (git through +// http.extraHeader); Basic with x-token-auth works for project and +// repository tokens from 9.4 only. +// +// What the driver relies on, from the OpenAPI descriptions of Bitbucket +// Data Center 8.0 to 10.5 (developer.atlassian.com/server/bitbucket/rest/) +// and Atlassian's documentation (confluence.atlassian.com/bitbucketserver), +// none of it seen on a live instance: +// - a repository is /, its numeric id stays across +// renames (its slug follows its name); a personal repository's project +// key is ~; a project is the namespace, with no nesting; +// repository labels are the topics' counterpart +// (GET /labels/{name}/labeled); +// - accounts are users, known by their slug (the login in hub.yml) and +// numeric id; type NORMAL is a person, SERVICE a service user (the +// user of a project or repository token); there is no "who am I" +// endpoint: every answer carries X-AUSERNAME, the user's name, which +// GET /users?filter= turns into the user (an Atlassian engineer's +// answer, not in the reference); +// - collections are pages of values with start, limit, isLastPage and +// nextPageStart, which a client must use as the next start; pages hold +// at most 1 000 (500 for a pull request's activities); +// - a pull request has states OPEN, DECLINED and MERGED, a version that +// every change must name (409 when it is stale), one open pull request +// per pair of branches, a native draft flag (8.18 and later), no +// labels and no writable properties; a declined one can be reopened, +// and whether it can be edited is not documented, so the driver edits +// none (Caps.ClosedImmutable); who declined it is in its activities; +// inactive pull requests are declined by Bitbucket's system user after +// four weeks by default; +// - HTML in Markdown is escaped (an HTML comment would show), and +// link reference definitions render as nothing (CommonMark): the +// marker is one (MarkerInRefDef); a description holds at most 32 768 +// characters (Atlassian's tracker, BSERV-14135); +// - creating, declining and reopening pull requests and commenting need +// read access only; pushes need write access; branch permissions are +// readable with repository admin rights only, so a push meets them +// ("Branch … can only be modified through pull requests"); +// - limits are a token bucket per user and node (60, refilled at 5 a +// second, by default), answered with 429. +// +// The driver never retries or paces (internal/throttle does), never knows +// about packs or markers, and masks secrets in its errors. +package bitbucketdc + +import ( + "errors" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// NewReader returns the read driver for provider p (type +// "bitbucket-datacenter") with credential c (an auth.Token, an HTTP access +// token; a zero credential reads anonymously, which an instance answers +// only where it allows anonymous access) over client, which sends the +// token only to p's API host. +func NewReader(p config.ResolvedProvider, c auth.Credential, client *httpx.Client) (platform.Reader, error) { + cl, err := newClient(p, c, client) + if err != nil { + return nil, err + } + return &reader{c: cl}, nil +} + +// NewWriter returns the write driver for provider p with credential c, an +// HTTP access token of the user that writes (a write identity is never +// anonymous). Target checks the user's write access to the repository and +// returns a TargetWriter bound to it; Bitbucket cannot mint narrower +// tokens, so Close only retires the TargetWriter: later calls through it +// fail with ClassAuth. +func NewWriter(p config.ResolvedProvider, c auth.Credential, client *httpx.Client) (platform.Writer, error) { + if c.Kind == 0 { + return nil, errors.New("bitbucket-datacenter: the write driver needs a token") + } + cl, err := newClient(p, c, client) + if err != nil { + return nil, err + } + return &writer{reader: reader{c: cl}}, nil +} + +// reader is platform.Reader over one identity. +type reader struct{ c *client } + +// writer is platform.Writer: a reader that can mint TargetWriters. +type writer struct{ reader } + +var ( + _ platform.Reader = (*reader)(nil) + _ platform.Writer = (*writer)(nil) + _ platform.Checker = (*writer)(nil) +) diff --git a/internal/platform/bitbucketdc/check.go b/internal/platform/bitbucketdc/check.go new file mode 100644 index 0000000..2706177 --- /dev/null +++ b/internal/platform/bitbucketdc/check.go @@ -0,0 +1,90 @@ +package bitbucketdc + +import ( + "context" + "fmt" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Check reports the writer's checks for doctor (platform.Checker). +// +// With no repository, the identity's: "token-expiry" and "scopes" are +// unknown, since the API does not tell which of a user's HTTP access +// tokens a request came with (the user's Manage account page shows their +// expiry and permissions), and so is "2fa"; the user itself is read first +// (Self), so a refused token fails the call. +// +// With repositories, per repository: "access" from the writer's permission +// on it (REPO_WRITE ok; REPO_ADMIN warns: more than it needs; less fails), +// fail when it does not see the repository; "rules" unknown: branch +// permissions are readable with admin rights only, so a push meets them. +func (w *writer) Check(ctx context.Context, repos []platform.Repo, _ []string) ([]platform.Finding, error) { + if len(repos) == 0 { + return w.identityChecks(ctx) + } + var out []platform.Finding + for _, r := range repos { + fs, err := w.repoChecks(ctx, r) + if err != nil { + return nil, err + } + out = append(out, fs...) + } + return out, nil +} + +// identityChecks are the checks of the writer's token and user. +func (w *writer) identityChecks(ctx context.Context) ([]platform.Finding, error) { + if _, err := w.c.selfAccount(ctx); err != nil { + return nil, err + } + return []platform.Finding{ + {Check: "token-expiry", Status: platform.FindingUnknown, + Detail: "Bitbucket Data Center's API does not tell which HTTP access token a request came with; the writer's Manage account > HTTP access tokens page shows when each expires"}, + {Check: "scopes", Status: platform.FindingUnknown, + Detail: "Bitbucket Data Center's API does not show the token's permissions; the writer's token needs repository write, and no admin permission"}, + {Check: "2fa", Status: platform.FindingUnknown, + Detail: "Bitbucket Data Center's API does not show whether a user signs in with two-step verification"}, + }, nil +} + +// repoChecks are the checks of one repository. +func (w *writer) repoChecks(ctx context.Context, r platform.Repo) ([]platform.Finding, error) { + const op = "check" + find := func(check string, status platform.FindingStatus, format string, args ...any) platform.Finding { + return platform.Finding{Repo: r.Path, Check: check, Status: status, Detail: fmt.Sprintf(format, args...)} + } + rules := find("rules", platform.FindingUnknown, "branch permissions need repository admin to read; a push meets them") + repo, err := w.c.targetRepo(ctx, op, r) + switch { + case err == nil: + case stops(err): + return nil, err + case platform.ClassOf(err) == platform.ClassNotFound, platform.ClassOf(err) == platform.ClassPermission: + return []platform.Finding{find("access", platform.FindingFail, "the writer does not see this repository")}, nil + default: + return []platform.Finding{find("access", platform.FindingUnknown, "%v", err), rules}, nil + } + write, err := w.c.hasPermission(ctx, op, repo, permWrite) + admin := false + if err == nil && write { + admin, err = w.c.hasPermission(ctx, op, repo, permAdmin) + } + var access platform.Finding + switch { + case err != nil && stops(err): + return nil, err + case err != nil: + access = find("access", platform.FindingUnknown, "%v", err) + case admin: + access = find("access", platform.FindingWarn, "the writer administers the repository: write is enough, "+ + "and a leaked token could change its settings and branch permissions") + case write: + access = find("access", platform.FindingOK, "the writer may push and write pull requests") + default: + access = find("access", platform.FindingFail, "the writer may read the repository, not write to it: it may not push; "+ + "give it write permission (directly, or through a group or the project)") + } + return []platform.Finding{access, rules}, nil +} diff --git a/internal/platform/bitbucketdc/client.go b/internal/platform/bitbucketdc/client.go new file mode 100644 index 0000000..191a215 --- /dev/null +++ b/internal/platform/bitbucketdc/client.go @@ -0,0 +1,390 @@ +package bitbucketdc + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/url" + "strconv" + "strings" + "sync" + "unicode" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/redact" +) + +// providerType is the type of a Bitbucket Data Center provider in hub.yml. +const providerType = "bitbucket-datacenter" + +// pageLimit is the limit a listing asks for: the default cap of most +// collections (page.max.repositories, page.max.pullrequests, +// page.max.users and page.max.branches are 1 000; an instance may lower +// them, and the answer's own limit and nextPageStart are what count). +const pageLimit = 1000 + +// activityLimit is the limit of a pull request's activities +// (page.max.pullrequest.activities is 500). +const activityLimit = 500 + +// maxRedirects bounds the redirects one GET follows. +const maxRedirects = 3 + +// snippetLimit bounds the body kept from a failed raw request, in bytes, as +// httpx keeps for JSON requests. +const snippetLimit = 1 << 10 + +// client is the HTTP side of one driver: one provider under one identity. +// It is safe for concurrent use. +type client struct { + http *httpx.Client + // api is the REST base without a trailing slash + // (https://bitbucket.example.com/rest/api/latest); apiURL is api + // parsed: redirects and the like are followed only below it. + api string + apiURL *url.URL + // web is the provider's url without a trailing slash (the instance's + // base URL, with its context path); clone URLs are under it + // (/scm//.git). host is the provider host (Repo.Host). + web, host string + token string // "" when anonymous + auth *httpx.Auth + masks *redact.Registry + + mu sync.Mutex + self *platform.Account // cached by selfAccount + // version is the instance's version, read once by Probe. + version *[3]int + // names maps the id of a user that Self or Lookup resolved to its + // name, which the dashboard API takes (OpenPRsBy). + names map[string]string +} + +// newClient checks the provider and the credential and builds the client. +func newClient(p config.ResolvedProvider, cred auth.Credential, hc *httpx.Client) (*client, error) { + if p.Type != providerType { + return nil, fmt.Errorf("bitbucket-datacenter: provider %s has type %q, not %s", p.ID, p.Type, providerType) + } + if hc == nil { + return nil, errors.New("bitbucket-datacenter: no HTTP client") + } + api, apiURL, err := baseURL("api_url", p.APIURL) + if err != nil { + return nil, fmt.Errorf("bitbucket-datacenter: provider %s: %w", p.ID, err) + } + web, webURL, err := baseURL("url", p.URL) + if err != nil { + return nil, fmt.Errorf("bitbucket-datacenter: provider %s: %w", p.ID, err) + } + host := strings.ToLower(p.Host) + if host == "" { + host = strings.ToLower(webURL.Host) + } + c := &client{http: hc, api: api, apiURL: apiURL, web: web, host: host, masks: redact.New(), names: map[string]string{}} + switch cred.Kind { + case 0: + if cred.Token != "" || cred.AppID != "" || len(cred.AppKey) > 0 { + return nil, errors.New("bitbucket-datacenter: a credential without a kind") + } + case auth.Token: + if cred.Token == "" { + return nil, errors.New("bitbucket-datacenter: an empty token") + } + c.token = cred.Token + c.masks.Add(cred.Token) + // The token goes to the API's host and to the web URL's (git), which + // are one host unless api_url says otherwise. + hosts := []string{strings.ToLower(apiURL.Host)} + if h := strings.ToLower(webURL.Host); h != hosts[0] { + hosts = append(hosts, h) + } + c.auth = &httpx.Auth{Hosts: hosts, Header: c.header} + case auth.App: + return nil, fmt.Errorf("bitbucket-datacenter: provider %s: a GitHub App cannot sign in to Bitbucket; use an HTTP access token", p.ID) + default: + return nil, fmt.Errorf("bitbucket-datacenter: provider %s: unknown credential kind %d", p.ID, cred.Kind) + } + return c, nil +} + +// baseURL checks an http(s) base URL without credentials, query or +// fragment, and returns it without a trailing slash. +func baseURL(what, raw string) (string, *url.URL, error) { + s := strings.TrimRight(raw, "/") + u, err := url.Parse(s) + switch { + case s == "": + return "", nil, fmt.Errorf("%s is empty", what) + case err != nil: + return "", nil, fmt.Errorf("%s does not parse", what) + case u.Scheme != "https" && u.Scheme != "http": + return "", nil, fmt.Errorf("%s is not an http or https URL", what) + case u.Host == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "": + return "", nil, fmt.Errorf("%s must be a URL with a host and without credentials, query or fragment", what) + } + return s, u, nil +} + +// header is the Authorization header of API requests and of git over +// HTTPS: every kind of HTTP access token is a bearer token on 8.19 and +// later. +func (c *client) header(ctx context.Context) (string, error) { + if err := ctx.Err(); err != nil { + return "", err + } + return "Bearer " + c.token, nil +} + +// mask replaces the credential's forms in s. +func (c *client) mask(s string) string { return c.masks.Replace(s) } + +// endpoint joins the API base and path segments, each escaped. +func (c *client) endpoint(segments ...string) string { + var b strings.Builder + b.WriteString(c.api) + for _, s := range segments { + b.WriteByte('/') + b.WriteString(url.PathEscape(s)) + } + return b.String() +} + +// repoURL is the endpoint of repository key/slug and more segments. +func (c *client) repoURL(key, slug string, more ...string) string { + return c.endpoint(append([]string{"projects", key, "repos", slug}, more...)...) +} + +// pathSegments splits a slash-separated file path into its segments, which +// endpoint escapes one by one. +func pathSegments(p string) []string { return strings.Split(p, "/") } + +// call sends one API request of op and decodes a 2xx JSON response into +// out (nil skips decoding). Every error is a masked *platform.Error, but a +// canceled context's. A GET follows a redirect to the same origin below +// the API base, at most maxRedirects times, with the same credential; +// httpx follows no redirect of an authenticated request itself. +func (c *client) call(ctx context.Context, op, method, u string, query url.Values, in, out any) (*httpx.Response, error) { + if len(query) > 0 { + u += "?" + query.Encode() + } + for hops := 0; ; hops++ { + resp, err := c.http.JSON(ctx, method, u, c.auth, in, out) + if err == nil { + return resp, nil + } + if method == http.MethodGet && hops < maxRedirects { + if next, ok := c.redirect(u, err); ok { + u = next + continue + } + } + return resp, c.apiError(op, err) + } +} + +// get is call with GET. +func (c *client) get(ctx context.Context, op, u string, query url.Values, out any) (*httpx.Response, error) { + return c.call(ctx, op, http.MethodGet, u, query, nil, out) +} + +// redirect returns where the redirect err answered to a request for from +// leads, when it is one to follow. +func (c *client) redirect(from string, err error) (string, bool) { + var se *httpx.StatusError + if !errors.As(err, &se) { + return "", false + } + return c.follow(from, se.Status, se.Header) +} + +// follow returns where a response of status with header to a request for +// from leads, when it is a redirect to follow: a 301, 302, 303, 307 or 308 +// whose Location, resolved against from, is on the API's scheme, host and +// port, without credentials, and below the API base. +func (c *client) follow(from string, status int, header http.Header) (string, bool) { + switch status { + case http.StatusMovedPermanently, http.StatusFound, http.StatusSeeOther, + http.StatusTemporaryRedirect, http.StatusPermanentRedirect: + default: + return "", false + } + loc := strings.TrimSpace(header.Get("Location")) + base, perr := url.Parse(from) + if loc == "" || perr != nil { + return "", false + } + to, perr := base.Parse(loc) + if perr != nil || !c.belowAPI(to) { + return "", false + } + to.Fragment, to.RawFragment = "", "" + return to.String(), true +} + +// belowAPI reports whether u is on the API's scheme, host and port, without +// credentials, and below the API base. +func (c *client) belowAPI(u *url.URL) bool { + if u.User != nil || !sameOrigin(u, c.apiURL) { + return false + } + root := strings.TrimRight(c.apiURL.Path, "/") + "/" + return strings.HasPrefix(u.Path, root) && !strings.Contains(u.Path, "/../") && !strings.HasSuffix(u.Path, "/..") +} + +// sameOrigin reports whether a and b have the same scheme, host and port +// (the scheme's default when absent), ignoring case. +func sameOrigin(a, b *url.URL) bool { + port := func(u *url.URL) string { + if p := u.Port(); p != "" { + return p + } + if strings.EqualFold(u.Scheme, "https") { + return "443" + } + return "80" + } + return strings.EqualFold(a.Scheme, b.Scheme) && strings.EqualFold(a.Hostname(), b.Hostname()) && port(a) == port(b) +} + +// raw reads the body of u with query, a GET outside JSON (a file's raw +// content). A redirect below the API base is followed as call follows it; +// a non-2xx answer is classified as call classifies it. +func (c *client) raw(ctx context.Context, op, u string, query url.Values) ([]byte, error) { + if len(query) > 0 { + u += "?" + query.Encode() + } + for hops := 0; ; hops++ { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil) + if err != nil { + return nil, &platform.Error{Op: op, Class: platform.ClassInvalid, Err: c.masked(err)} + } + req.Header.Set("Accept", "*/*") + resp, err := c.http.Do(req, c.auth) + if err != nil { + return nil, c.apiError(op, err) + } + if resp.Status >= 200 && resp.Status <= 299 { + return resp.Body, nil + } + if next, ok := c.follow(u, resp.Status, resp.Header); ok && hops < maxRedirects { + u = next + continue + } + return nil, c.statusError(op, &httpx.StatusError{ + Method: http.MethodGet, URL: withoutQuery(u), Status: resp.Status, + Header: resp.Header, Snippet: snippet(resp.Body), + }) + } +} + +// withoutQuery returns u without its query and fragment, as httpx shows +// URLs in errors. +func withoutQuery(u string) string { + if i := strings.IndexAny(u, "?#"); i >= 0 { + return u[:i] + } + return u +} + +// snippet returns at most snippetLimit bytes of body on one line. +func snippet(body []byte) string { + if len(body) > snippetLimit { + body = body[:snippetLimit] + } + return strings.Join(strings.FieldsFunc(strings.ToValidUTF8(string(body), "?"), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// page is one page of a paged collection (REST intro, "Paged APIs"): +// values, isLastPage, and nextPageStart, the start of the next page, which +// a client must use as it is. +type page[T any] struct { + Values []T `json:"values"` + IsLastPage *bool `json:"isLastPage"` + NextPageStart *int64 `json:"nextPageStart"` +} + +// listAll reads a paged collection of T at u with query and calls each +// for every item, in order, asking for limit items a page and following +// nextPageStart. complete is false when the listing was capped at +// maxPages. An error of a page after the first comes with the page in err +// (see pageError), so a caller can tell a failed listing from a failed +// first request. A page that says it is not the last but names no next +// start, or one that does not move forward, is an unexpected answer. +func listAll[T any](ctx context.Context, c *client, op, u string, query url.Values, limit, maxPages int, each func(T) error) (complete bool, err error) { + start := int64(0) + for n := 1; n <= maxPages; n++ { + q := url.Values{} + for k, v := range query { + q[k] = v + } + q.Set("limit", strconv.Itoa(limit)) + if start > 0 { + q.Set("start", strconv.FormatInt(start, 10)) + } + var p page[T] + if _, err := c.get(ctx, op, u, q, &p); err != nil { + if n > 1 { + return false, &pageError{page: n, err: err} + } + return false, err + } + for _, it := range p.Values { + if err := each(it); err != nil { + return false, err + } + } + switch { + case p.IsLastPage == nil: + return false, shapeError(op, "a page without isLastPage") + case *p.IsLastPage: + return true, nil + case p.NextPageStart == nil || *p.NextPageStart <= start: + return false, shapeError(op, "a page that is not the last names no later start") + } + start = *p.NextPageStart + } + return false, nil +} + +// pageError is the failure of a page after the first of a listing. +type pageError struct { + page int + err error +} + +func (e *pageError) Error() string { return fmt.Sprintf("page %d: %v", e.page, e.err) } +func (e *pageError) Unwrap() error { return e.err } + +// laterPage reports whether err is the failure of a page after the first. +func laterPage(err error) bool { + var pe *pageError + return errors.As(err, &pe) +} + +// splitRepoPath splits "/". Bitbucket Data Center has +// no nested projects: a path of another shape names no repository (ok +// false). +func splitRepoPath(p string) (key, slug string, ok bool) { + key, slug, found := strings.Cut(p, "/") + if !found || key == "" || slug == "" || strings.Contains(slug, "/") || + strings.ContainsAny(p, "\x00?#") || key == "." || key == ".." || slug == "." || slug == ".." { + return "", "", false + } + return key, slug, true +} + +// repoPath returns the project key and slug of r, or a ClassInvalid error +// for op. +func repoPath(op string, r platform.Repo) (key, slug string, err error) { + key, slug, ok := splitRepoPath(r.Path) + if !ok { + return "", "", invalid(op, "%q is not a / path", r.Path) + } + return key, slug, nil +} diff --git a/internal/platform/bitbucketdc/errors.go b/internal/platform/bitbucketdc/errors.go new file mode 100644 index 0000000..940b460 --- /dev/null +++ b/internal/platform/bitbucketdc/errors.go @@ -0,0 +1,371 @@ +package bitbucketdc + +import ( + "context" + "crypto/tls" + "crypto/x509" + "encoding/json" + "errors" + "fmt" + "math" + "net/http" + "slices" + "strconv" + "strings" + "time" + "unicode" + + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxRetryAfter bounds a wait read from response headers: a broken or +// hostile header must not park the provider for days. +const maxRetryAfter = time.Hour + +// now is the clock of Retry-After dates and reset times (tests replace it). +var now = time.Now + +// maskedError carries a masked message over the original error, which +// errors.Is and errors.As still see. +type maskedError struct { + msg string + err error +} + +func (e *maskedError) Error() string { return e.msg } +func (e *maskedError) Unwrap() error { return e.err } + +// masked returns err with its message masked. +func (c *client) masked(err error) error { + return &maskedError{msg: c.mask(err.Error()), err: err} +} + +// apiError classifies err, the failure of an API request of op, into a +// platform.Error: a non-2xx status by its code, a timeout as transient, a +// request httpx refused as invalid, a certificate the client does not +// trust, a response over the size bound or of an unexpected shape as +// unknown (none of them gets better by retrying), any other transport +// failure as transient. A canceled context stays unclassified. Messages +// never hold the credential nor headers. +func (c *client) apiError(op string, err error) error { + var se *httpx.StatusError + if errors.As(err, &se) { + return c.statusError(op, se) + } + class := platform.ClassTransient + switch { + case errors.Is(err, context.Canceled): + return fmt.Errorf("%s: %w", op, c.masked(err)) + case errors.Is(err, context.DeadlineExceeded): + case errors.Is(err, httpx.ErrRefused): + class = platform.ClassInvalid + case errors.Is(err, httpx.ErrTooLarge), certificateError(err), decodeError(err): + class = platform.ClassUnknown + } + return &platform.Error{Op: op, Class: class, Err: c.masked(err)} +} + +// certificateError reports whether err is a TLS certificate failure. +func certificateError(err error) bool { + var ( + verify *tls.CertificateVerificationError + unknown x509.UnknownAuthorityError + invalid x509.CertificateInvalidError + hostError x509.HostnameError + ) + return errors.As(err, &verify) || errors.As(err, &unknown) || errors.As(err, &invalid) || errors.As(err, &hostError) +} + +// decodeError reports whether err is a JSON body that does not decode. +func decodeError(err error) bool { + var ( + syntax *json.SyntaxError + typ *json.UnmarshalTypeError + ) + return errors.As(err, &syntax) || errors.As(err, &typ) || errors.Is(err, errShape) +} + +// errShape is wrapped by errors about responses that decode but lack what +// the driver depends on. +var errShape = errors.New("unexpected response") + +// shapeError is an error of op about a response that lacks what the driver +// depends on: the API changed shape. +func shapeError(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%w: "+format, append([]any{errShape}, args...)...)} +} + +// statusError classifies a non-2xx response of op. +// +// Bitbucket Data Center's error body is {"errors": [{"context": …, +// "message": …, "exceptionName": …}]} for 401, 403, 404 and 409 (REST +// intro, "Errors"); the exception names are Java class names the reference +// does not list (com.atlassian.bitbucket.repository.NoSuchRepositoryException +// and the like). A 401 of an authenticated user is a missing permission, +// not a refused credential (see authenticated). A rate limit is 429; +// whether it carries Retry-After is not documented, and one is honoured +// when it does. +func (c *client) statusError(op string, se *httpx.StatusError) error { + m := parseMessage(se.Snippet) + detail := m.text() + if detail == "" { + detail = se.Snippet + } + msg := c.mask(describe(se, detail)) + e := &platform.Error{Op: op, Class: statusClass(se.Status), Status: se.Status} + if se.Status == http.StatusUnauthorized && authenticated(se.Header, m) { + e.Class = platform.ClassPermission + } + switch se.Status { + case http.StatusNotFound, http.StatusGone: + e.Err = ¬FoundError{exceptions: m.exceptions(), err: fmt.Errorf("%s: %w", msg, platform.ErrNotFound)} + return e + case http.StatusTooManyRequests, http.StatusServiceUnavailable: + e.RetryAfter = retryAfter(se.Header, now()) + } + e.Err = errors.New(msg) + return e +} + +// authenticated reports whether a 401 answered a user Bitbucket knows: it +// answers 401 both to a credential it refuses and to a user without the +// permission a resource needs ("The currently authenticated user has +// insufficient permissions", throughout the reference). The second carries +// the user's name in X-AUSERNAME, as every authenticated answer does, or +// names an AuthorisationException; a refused or missing credential does +// neither. +func authenticated(h http.Header, m apiMessage) bool { + if strings.TrimSpace(h.Get("X-AUSERNAME")) != "" { + return true + } + for _, e := range m.exceptions() { + simple := e[strings.LastIndexByte(e, '.')+1:] + if simple == "AuthorisationException" || simple == "AuthorizationException" { + return true + } + } + return false +} + +// notFoundError is the error of a 404 (or 410): the exception names of +// Bitbucket's body (none when the body is no Bitbucket error), over the +// ErrNotFound it wraps. +type notFoundError struct { + exceptions []string + err error +} + +func (e *notFoundError) Error() string { return e.err.Error() } +func (e *notFoundError) Unwrap() error { return e.err } + +// missingException reports whether err is a 404 of the API whose body names +// an exception whose simple name is one of names (NoSuchPathException); +// ok is false when err is no 404 of the API. +func missingException(err error, names ...string) (named, ok bool) { + var nf *notFoundError + if !errors.As(err, &nf) { + return false, false + } + for _, e := range nf.exceptions { + simple := e[strings.LastIndexByte(e, '.')+1:] + if slices.Contains(names, simple) { + return true, true + } + } + return false, true +} + +// statusClass maps an HTTP status to the class the core reacts to. +func statusClass(status int) platform.Class { + switch { + case status == http.StatusUnauthorized: + return platform.ClassAuth + case status == http.StatusForbidden: + return platform.ClassPermission + case status == http.StatusNotFound, status == http.StatusGone: + return platform.ClassNotFound + case status == http.StatusConflict: + return platform.ClassConflict + case status == http.StatusTooManyRequests: + return platform.ClassRateLimited + case status == http.StatusRequestTimeout, status == http.StatusTooEarly: + return platform.ClassTransient + case status == http.StatusNotImplemented: + return platform.ClassUnsupported + case status >= 500: + return platform.ClassTransient + case status >= 400: + return platform.ClassInvalid + } + // An unexpected redirect or informational status. + return platform.ClassUnknown +} + +// apiMessage is Bitbucket Data Center's error body. +type apiMessage struct { + Errors []struct { + Context *string `json:"context"` + Message string `json:"message"` + ExceptionName *string `json:"exceptionName"` + } `json:"errors"` +} + +// text returns the messages on one line, each with its context (a field) +// when it names one. +func (m apiMessage) text() string { + parts := make([]string, 0, len(m.Errors)) + for _, e := range m.Errors { + t := e.Message + if e.Context != nil && *e.Context != "" { + t = *e.Context + ": " + t + } + if t != "" { + parts = append(parts, t) + } + } + return strings.Join(strings.FieldsFunc(strings.Join(parts, "; "), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// exceptions returns the exception names of the body. +func (m apiMessage) exceptions() []string { + var out []string + for _, e := range m.Errors { + if e.ExceptionName != nil && *e.ExceptionName != "" { + out = append(out, *e.ExceptionName) + } + } + return out +} + +// parseMessage decodes an error body; a body of another shape gives the +// zero apiMessage. +func parseMessage(snippet string) apiMessage { + var m apiMessage + if json.Unmarshal([]byte(snippet), &m) != nil { + return apiMessage{} + } + return m +} + +// describe returns "METHOD URL: status text: detail". +func describe(se *httpx.StatusError, detail string) string { + var b strings.Builder + fmt.Fprintf(&b, "%s %s: %d", se.Method, se.URL, se.Status) + if text := http.StatusText(se.Status); text != "" { + b.WriteString(" " + text) + } + if detail != "" { + b.WriteString(": " + detail) + } + return b.String() +} + +// retryAfter reads how long to wait from a response's headers, in order: +// Retry-After (seconds or an HTTP date), then X-RateLimit-Reset (seconds +// until the window resets, or a Unix time when it is large), should a +// proxy or a later version send one: Bitbucket Data Center documents +// neither. It is 0 when none is usable, and at most maxRetryAfter. +func retryAfter(h http.Header, at time.Time) time.Duration { + d, ok := time.Duration(0), false + if v := strings.TrimSpace(h.Get("Retry-After")); v != "" { + if secs, err := strconv.ParseInt(v, 10, 64); err == nil { + d, ok = seconds(secs), true + } else if t, err := http.ParseTime(v); err == nil { + d, ok = t.Sub(at), true + } + } + if !ok { + if n, err := strconv.ParseInt(strings.TrimSpace(h.Get("X-RateLimit-Reset")), 10, 64); err == nil { + // A Unix time is far above any sensible number of seconds. + if n > 1_000_000_000 { + d, ok = time.Unix(n, 0).Sub(at), true + } else { + d, ok = seconds(n), true + } + } + } + switch { + case !ok || d < 0: + return 0 + case d > maxRetryAfter: + return maxRetryAfter + } + return d +} + +// seconds converts a count of seconds, saturating instead of overflowing. +func seconds(n int64) time.Duration { + if n < 0 { + return -1 + } + if n > math.MaxInt64/int64(time.Second) { + return maxRetryAfter + } + return time.Duration(n) * time.Second +} + +// closedError is the error of a TargetWriter used after Close. +func closedError(op string) error { + return &platform.Error{Op: op, Class: platform.ClassAuth, Err: errors.New("the target writer is closed")} +} + +// invalid is a ClassInvalid error of op: the request was wrong. +func invalid(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassInvalid, Err: fmt.Errorf(format, args...)} +} + +// notFound is a ClassNotFound error of op that wraps ErrNotFound. +func notFound(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassNotFound, Err: fmt.Errorf(format+": %w", append(args, platform.ErrNotFound)...)} +} + +// unknown is a ClassUnknown error of op that says what happened (format +// and args) and, when err is not nil, what err says, with err's status but +// without the ErrNotFound err may wrap: a 404 that must not count as a +// missing file. +func unknown(op string, err error, format string, args ...any) error { + msg := fmt.Sprintf(format, args...) + if err != nil { + msg += ": " + detail(err) + } + return &platform.Error{Op: op, Class: platform.ClassUnknown, Status: statusOf(err), Err: errors.New(msg)} +} + +// detail returns what err says without its class: the message of the +// *platform.Error in it, without the ErrNotFound suffix. +func detail(err error) string { + var pe *platform.Error + if errors.As(err, &pe) && pe.Err != nil { + return strings.TrimSuffix(pe.Err.Error(), ": "+platform.ErrNotFound.Error()) + } + return err.Error() +} + +// statusOf returns the HTTP status of a classified error, 0 for none. +func statusOf(err error) int { + var pe *platform.Error + if errors.As(err, &pe) { + return pe.Status + } + return 0 +} + +// stops reports whether err ends a call at once, whatever it already +// found: the context ended, the platform limits the rate or refuses the +// credential. +func stops(err error) bool { + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + return true + } + c := platform.ClassOf(err) + return c == platform.ClassRateLimited || c == platform.ClassAuth +} + +// fatal reports whether err must fail a listing instead of making it +// incomplete: it stops the call, or the platform failed for now (the caller +// retries the whole listing). +func fatal(err error) bool { return stops(err) || platform.ClassOf(err) == platform.ClassTransient } diff --git a/internal/platform/bitbucketdc/files.go b/internal/platform/bitbucketdc/files.go new file mode 100644 index 0000000..1db37ba --- /dev/null +++ b/internal/platform/bitbucketdc/files.go @@ -0,0 +1,218 @@ +package bitbucketdc + +import ( + "context" + "crypto/sha1" + "encoding/hex" + "errors" + "fmt" + "net/url" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// modeFile is the mode of every file the driver reads: the REST API tells +// no mode (an executable file reads as a plain one). +const modeFile = "100644" + +// Node types of GET …/browse/{path}?type=true. +const ( + nodeFile = "FILE" + nodeDirectory = "DIRECTORY" + nodeSubmodule = "SUBMODULE" +) + +// Exceptions of a 404 that say what is missing. The reference names none; +// these are the Java exceptions of Bitbucket's API (its Javadoc), whose +// class names the error body carries. +var ( + // missingPathExceptions say the path is missing at a commit that + // exists. + missingPathExceptions = []string{"NoSuchPathException"} + // missingOtherExceptions say the repository or the commit is missing. + missingOtherExceptions = []string{"NoSuchRepositoryException", "NoSuchCommitException", "NoSuchProjectException"} +) + +// ReadFile returns a regular file of r at ref ("" is the default branch +// head): its type and size from GET …/browse/{path}?type=true and ?size= +// true, then its raw bytes from GET …/raw/{path}, all at the commit ref +// resolves to, so that the answers describe the same tree. The blob id is +// computed here and the mode is always 100644: the API gives neither. +// Directories and submodules are ErrNotRegular, a file over max bytes +// ErrTooLarge. Symlinks are not documented: one may read as a file whose +// content is its target. +// +// Only a path that an existing commit lacks is ErrNotFound: the core takes a +// missing opt-in file for an opt-out and closes the target's pull requests. +// A missing repository, ref or commit, a 404 that does not say the path is +// missing while the commit is gone, and a file that the type shows and the +// raw read does not find are ClassUnknown. An empty repository has no file: +// ErrNotFound. +func (d *reader) ReadFile(ctx context.Context, r platform.Repo, ref, filePath string, max int64) (platform.File, error) { + const op = "read file" + key, slug, err := repoPath(op, r) + if err != nil { + return platform.File{}, err + } + if err := checkTreePath(filePath); err != nil { + return platform.File{}, invalid(op, "%v", err) + } + if max < 0 { + return platform.File{}, invalid(op, "negative size limit %d", max) + } + commit, err := d.c.commitOf(ctx, op, r, key, slug, ref) + if err != nil { + return platform.File{}, err + } + var node struct { + Type string `json:"type"` + } + _, err = d.c.get(ctx, op, d.c.browseURL(key, slug, filePath), url.Values{"at": {commit}, "type": {"true"}}, &node) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return platform.File{}, d.c.missingPath(ctx, op, r, key, slug, commit, filePath, err) + case err != nil: + return platform.File{}, err + } + switch node.Type { + case nodeFile: + case nodeDirectory: + return platform.File{}, fmt.Errorf("%s: %s: %s is a directory: %w", op, r.Path, filePath, platform.ErrNotRegular) + case nodeSubmodule: + return platform.File{}, fmt.Errorf("%s: %s: %s is a submodule: %w", op, r.Path, filePath, platform.ErrNotRegular) + default: + return platform.File{}, shapeError(op, "%s: node type %q", filePath, node.Type) + } + var size struct { + Size *int64 `json:"size"` + } + _, err = d.c.get(ctx, op, d.c.browseURL(key, slug, filePath), url.Values{"at": {commit}, "size": {"true"}}, &size) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return platform.File{}, unknown(op, err, "%s: %s at %s: the type shows it, its size is not found", r.Path, filePath, commit) + case err != nil: + return platform.File{}, err + case size.Size == nil || *size.Size < 0: + return platform.File{}, shapeError(op, "%s: no size", filePath) + case *size.Size > max: + return platform.File{}, fmt.Errorf("%s: %s: %s has %d bytes, more than %d: %w", op, r.Path, filePath, *size.Size, max, platform.ErrTooLarge) + } + content, err := d.c.raw(ctx, op, d.c.repoURL(key, slug, append([]string{"raw"}, pathSegments(filePath)...)...), url.Values{"at": {commit}}) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + // Never ErrNotFound: the type of the same commit has just shown it. + return platform.File{}, unknown(op, err, "%s: %s at %s: the type shows it, its content is not found", r.Path, filePath, commit) + case err != nil: + return platform.File{}, err + case int64(len(content)) != *size.Size: + return platform.File{}, shapeError(op, "%s: %d bytes of content, the size is %d", filePath, len(content), *size.Size) + } + return platform.File{Path: filePath, Mode: modeFile, OID: blobID(content), Content: content}, nil +} + +// browseURL is the browse API's URL of filePath in key/slug. +func (c *client) browseURL(key, slug, filePath string) string { + return c.repoURL(key, slug, append([]string{"browse"}, pathSegments(filePath)...)...) +} + +// commitOf resolves ref of r to a full commit id: a full id is taken as it +// is (missingPath checks that it exists when a path is missing), a branch +// through the branch API, and "" through the default branch (GET +// …/branches/default). A ref the API does not find, or a repository it +// does not find, is ClassUnknown: never a missing file. An empty +// repository is ErrNotFound. +func (c *client) commitOf(ctx context.Context, op string, r platform.Repo, key, slug, ref string) (string, error) { + if isHexOID(ref) { + return strings.ToLower(ref), nil + } + if ref != "" { + b, ok, err := c.branch(ctx, op, key, slug, strings.TrimPrefix(ref, "refs/heads/")) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return "", unknown(op, err, "%s: the repository is not found", r.Path) + case err != nil: + return "", err + case !ok: + return "", unknown(op, nil, "%s: ref %s is not found", r.Path, ref) + } + return strings.ToLower(b.LatestCommit), nil + } + def, err := c.defaultBranchOf(ctx, op, key, slug) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return "", unknown(op, err, "%s: the repository is not found", r.Path) + case err != nil: + return "", err + case def.empty: + return "", notFound(op, "%s: the repository is empty", r.Path) + case def.missing || def.head == "": + return "", unknown(op, nil, "%s: the default branch %s does not exist", r.Path, def.name) + } + return def.head, nil +} + +// missingPath judges err, a 404 of the type of filePath at commit. A path +// is missing only at a commit that exists: the 404 names +// NoSuchPathException, or names no exception of a missing repository or +// commit and the commit API finds the commit (GET …/commits/{id}); else +// the repository or the commit is gone, which is ClassUnknown. +func (c *client) missingPath(ctx context.Context, op string, r platform.Repo, key, slug, commit, filePath string, err error) error { + if named, _ := missingException(err, missingPathExceptions...); named { + return notFound(op, "%s: %s at %s", r.Path, filePath, commit) + } + if named, _ := missingException(err, missingOtherExceptions...); named { + return unknown(op, err, "%s: %s at %s: the repository or the commit is not found", r.Path, filePath, commit) + } + _, cerr := c.get(ctx, op, c.repoURL(key, slug, "commits", commit), nil, nil) + switch { + case cerr != nil && platform.ClassOf(cerr) == platform.ClassNotFound: + return unknown(op, err, "%s: %s at %s: neither the file nor the commit is found", r.Path, filePath, commit) + case cerr != nil: + return cerr + } + return notFound(op, "%s: %s at %s", r.Path, filePath, commit) +} + +// checkTreePath accepts a repository-relative path: no leading slash, no +// empty, "." or ".." segment, no NUL. +func checkTreePath(p string) error { + if p == "" { + return errors.New("empty path") + } + if strings.IndexByte(p, 0) >= 0 { + return fmt.Errorf("path %q contains NUL", p) + } + for seg := range strings.SplitSeq(p, "/") { + if seg == "" || seg == "." || seg == ".." { + return fmt.Errorf("path %q has an empty, \".\" or \"..\" segment", p) + } + } + return nil +} + +// blobID is git's SHA-1 id of a blob with content: the driver reads SHA-1 +// repositories only (Repo.ObjectFormat). +func blobID(content []byte) string { + h := sha1.New() + fmt.Fprintf(h, "blob %d\x00", len(content)) + h.Write(content) + return hex.EncodeToString(h.Sum(nil)) +} + +// isHexOID reports whether s is a full SHA-1 object id. +func isHexOID(s string) bool { return len(s) == 40 && isHex(s) } + +// isHex reports whether s is a non-empty string of hexadecimal digits. +func isHex(s string) bool { + if s == "" { + return false + } + for i := 0; i < len(s); i++ { + c := s[i] + if (c < '0' || c > '9') && (c < 'a' || c > 'f') && (c < 'A' || c > 'F') { + return false + } + } + return true +} diff --git a/internal/platform/bitbucketdc/files_test.go b/internal/platform/bitbucketdc/files_test.go new file mode 100644 index 0000000..6edb581 --- /dev/null +++ b/internal/platform/bitbucketdc/files_test.go @@ -0,0 +1,181 @@ +package bitbucketdc + +import ( + "errors" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// apiRepoOf is the platform.Repo of ACME/api. +func (f *fixture) apiRepoOf() platform.Repo { + return platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api", DefaultBranch: "main"} +} + +// file declares path at any commit: its type, its size and its raw content. +func (f *fixture) file(path, typ string, content []byte) { + route := repoRoute("ACME", "api") + f.handle(http.MethodGet, route+"/browse/"+path, func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Query().Get("type") == "true": + writeJSON(w, http.StatusOK, map[string]any{"type": typ}) + case r.URL.Query().Get("size") == "true": + writeJSON(w, http.StatusOK, map[string]any{"size": len(content)}) + default: + t := f.t + t.Errorf("browse without type or size: %s", r.URL.RawQuery) + } + }) + f.handle(http.MethodGet, route+"/raw/"+path, func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "text/plain;charset=UTF-8") + _, _ = w.Write(content) + }) +} + +func TestReadFile(t *testing.T) { + f := newFixture(t) + content := []byte("version: 1\nenabled: true\n") + f.defBranch("ACME", "api", "main", mainSHA) + f.file(".engineering-assets.yml", "FILE", content) + got, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1<<20) + if err != nil { + t.Fatal(err) + } + // git hash-object of the content. + if got.OID != blobID(content) || got.Mode != "100644" || string(got.Content) != string(content) || got.Path != ".engineering-assets.yml" { + t.Errorf("ReadFile = %+v", got) + } + if blobID([]byte("hello\n")) != "ce013625030ba8dba906f756967f9e9ca394464a" { + t.Error("blobID is not git's") + } + for _, c := range f.requests(http.MethodGet, "") { + if strings.Contains(c.Path, "/browse/") || strings.Contains(c.Path, "/raw/") { + if c.Query.Get("at") != mainSHA { + t.Errorf("%s at %q, want the default branch's head", c.Path, c.Query.Get("at")) + } + } + } + + // A branch and a commit. + f.reset() + f.pages(repoRoute("ACME", "api")+"/branches", 1000, []any{ + branch(syncBranch, headSHA, false), branch(syncBranch+"-old", mainSHA, false), + }) + if _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), syncBranch, ".engineering-assets.yml", 1<<20); err != nil { + t.Fatal(err) + } + if c := f.requests(http.MethodGet, repoRoute("ACME", "api")+"/raw/.engineering-assets.yml"); len(c) != 1 || c[0].Query.Get("at") != headSHA { + t.Errorf("the branch's file was read at %v", c) + } + if q := f.requests(http.MethodGet, repoRoute("ACME", "api")+"/branches")[0].Query; q.Get("filterText") != syncBranch || q.Get("boostMatches") != "true" { + t.Errorf("the branch query %v", q) + } + f.reset() + if _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), strings.ToUpper(headSHA), ".engineering-assets.yml", 1<<20); err != nil { + t.Fatal(err) + } + if n := len(f.requests(http.MethodGet, repoRoute("ACME", "api")+"/branches")); n != 0 { + t.Error("a commit id was looked up as a branch") + } +} + +func TestReadFileKinds(t *testing.T) { + f := newFixture(t) + f.defBranch("ACME", "api", "main", mainSHA) + f.file("docs", "DIRECTORY", nil) + f.file("vendor/lib", "SUBMODULE", nil) + f.file("big.bin", "FILE", make([]byte, 2048)) + f.file("odd", "LINK", nil) + for _, tc := range []struct { + path string + sentinel error + }{{"docs", platform.ErrNotRegular}, {"vendor/lib", platform.ErrNotRegular}, {"big.bin", platform.ErrTooLarge}} { + _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", tc.path, 1024) + if err == nil || !errors.Is(err, tc.sentinel) { + t.Errorf("%s: %v, want %v", tc.path, err, tc.sentinel) + } + } + if n := len(f.requests(http.MethodGet, repoRoute("ACME", "api")+"/raw/big.bin")); n != 0 { + t.Error("a file over the limit was downloaded") + } + _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", "odd", 1024) + wantClass(t, "an unknown node type", err, platform.ClassUnknown, nil) + for _, p := range []string{"", "/abs", "a//b", "a/../b", "a\x00b"} { + _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", p, 1024) + wantClass(t, "path "+p, err, platform.ClassInvalid, nil) + } +} + +func TestReadFileMissing(t *testing.T) { + route := repoRoute("ACME", "api") + missing := func(f *fixture, exception string) { + f.json(route+"/browse/.engineering-assets.yml", http.StatusNotFound, errorBody(exception, "The path \".engineering-assets.yml\" does not exist at revision \""+mainSHA+"\"")) + } + + // The path is missing at a commit that exists. + f := newFixture(t) + f.defBranch("ACME", "api", "main", mainSHA) + missing(f, noPathException) + _, err := f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a missing path", err, platform.ClassNotFound, platform.ErrNotFound) + if n := len(f.requests(http.MethodGet, route+"/commits/"+mainSHA)); n != 0 { + t.Error("NoSuchPathException did not suffice") + } + + // A 404 that names nothing: the commit tells. + missing(f, "") + f.json(route+"/commits/"+mainSHA, http.StatusOK, map[string]any{"id": mainSHA, "displayId": mainSHA[:11]}) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a 404 without an exception, the commit found", err, platform.ClassNotFound, platform.ErrNotFound) + f.json(route+"/commits/"+mainSHA, http.StatusNotFound, errorBody("", "gone")) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "neither the file nor the commit", err, platform.ClassUnknown, nil) + if err != nil && errors.Is(err, platform.ErrNotFound) { + t.Error("an unknown failure wraps ErrNotFound") + } + + // The repository is gone between two requests. + missing(f, noRepoException) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a missing repository", err, platform.ClassUnknown, nil) + + // An empty repository has no file. + f = newFixture(t) + f.json(route+"/branches/default", http.StatusNoContent, nil) + f.json(route+"/default-branch", http.StatusOK, map[string]any{"id": "refs/heads/main", "displayId": "main"}) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "an empty repository", err, platform.ClassNotFound, platform.ErrNotFound) + + // A missing repository, or a missing ref, is no missing file. + f = newFixture(t) + f.json(route+"/branches/default", http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + f.json(route+"/default-branch", http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + f.json(route, http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a missing repository", err, platform.ClassUnknown, nil) + // The repository is there, its default branch is not: no missing file. + f.json(route, http.StatusOK, repo(repoID, "ACME", "api")) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a missing default branch", err, platform.ClassUnknown, nil) + f.pages(route+"/branches", 1000, []any{branch(syncBranch+"-old", mainSHA, false)}) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), syncBranch, ".engineering-assets.yml", 1024) + wantClass(t, "a missing branch", err, platform.ClassUnknown, nil) + + // The type shows the file, the raw read does not find it. + f = newFixture(t) + f.defBranch("ACME", "api", "main", mainSHA) + f.file(".engineering-assets.yml", "FILE", []byte("x")) + f.json(route+"/raw/.engineering-assets.yml", http.StatusNotFound, errorBody(noPathException, "gone")) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", ".engineering-assets.yml", 1024) + wantClass(t, "a file the raw read misses", err, platform.ClassUnknown, nil) + + // The raw content is not the size the browse API gave. + f = newFixture(t) + f.defBranch("ACME", "api", "main", mainSHA) + f.file("a.txt", "FILE", []byte("abc")) + f.handle(http.MethodGet, route+"/raw/a.txt", func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte("abcd")) }) + _, err = f.reader.ReadFile(t.Context(), f.apiRepoOf(), "", "a.txt", 1024) + wantClass(t, "a size mismatch", err, platform.ClassUnknown, nil) +} diff --git a/internal/platform/bitbucketdc/helpers_test.go b/internal/platform/bitbucketdc/helpers_test.go new file mode 100644 index 0000000..4b50ab4 --- /dev/null +++ b/internal/platform/bitbucketdc/helpers_test.go @@ -0,0 +1,377 @@ +package bitbucketdc + +import ( + "bytes" + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "strings" + "sync" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// The fixtures below follow the OpenAPI descriptions of Bitbucket Data +// Center 8.19 to 10.5 (developer.atlassian.com/server/bitbucket/rest/) and +// the examples of its REST documentation: every field the driver reads, +// and a few it ignores, as the reference shapes them. None was seen on a +// live instance. + +// contextPath is the instance's context path: Bitbucket is often served +// under one (https://example.com/bitbucket). +const contextPath = "/bitbucket" + +// apiPrefix is the REST base under the server's root. +const apiPrefix = contextPath + "/rest/api/latest" + +// apiServer is an httptest server that answers the routes a test declares +// and records every request. An undeclared route fails the test, and so +// does any write unless the test is the writer's (allowWrites), whose +// writes are declared routes too. Routes are paths under the REST base as +// sent, escaped. An answer to the server's token carries X-AUSERNAME, as +// Bitbucket's answers to an authenticated user do. +type apiServer struct { + t *testing.T + srv *httptest.Server + mu sync.Mutex + routes map[string]http.HandlerFunc + calls []apiCall + // token is the credential the server knows, username its user's name. + token, username string + // allowWrites lets the writer's tests write (their declared routes). + allowWrites bool +} + +// apiCall is one request the server received. +type apiCall struct { + Method, Path string // Path escaped, under the REST base + Query url.Values + Auth string // Authorization + Body []byte +} + +func newAPIServer(t *testing.T) *apiServer { + t.Helper() + s := &apiServer{t: t, routes: map[string]http.HandlerFunc{}, token: testToken(t), username: "touchmark.reader"} + s.srv = httptest.NewServer(http.HandlerFunc(s.serve)) + t.Cleanup(s.srv.Close) + // The read driver never writes. + t.Cleanup(func() { + if s.allowWrites { + return + } + for _, c := range s.writes() { + t.Errorf("the reader wrote: %s %s", c.Method, c.Path) + } + }) + return s +} + +// base is the instance's base URL, context path included. +func (s *apiServer) base() string { return s.srv.URL + contextPath } + +func (s *apiServer) serve(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + r.Body = io.NopCloser(bytes.NewReader(body)) + p, ok := strings.CutPrefix(r.URL.EscapedPath(), apiPrefix) + authz := r.Header.Get("Authorization") + s.mu.Lock() + s.calls = append(s.calls, apiCall{Method: r.Method, Path: p, Query: r.URL.Query(), Auth: authz, Body: body}) + h := s.routes[r.Method+" "+p] + s.mu.Unlock() + if authz == "Bearer "+s.token && s.username != "" { + w.Header().Set("X-AUSERNAME", s.username) + } + if !ok || h == nil { + s.t.Errorf("unexpected request %s %s?%s", r.Method, r.URL.EscapedPath(), r.URL.RawQuery) + writeJSON(w, http.StatusNotFound, errorBody("com.atlassian.bitbucket.NoSuchResourceException", "Resource not found")) + return + } + h(w, r) +} + +// handle declares a route: method and an escaped path under the REST base. +func (s *apiServer) handle(method, path string, h http.HandlerFunc) { + s.mu.Lock() + defer s.mu.Unlock() + s.routes[method+" "+path] = h +} + +// json declares a GET route that answers status with v as JSON (no body +// when v is nil). +func (s *apiServer) json(path string, status int, v any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { writeJSON(w, status, v) }) +} + +// pages declares a GET route that answers items, at most per a page +// whatever the limit asked (as page.max.* caps it), with start, limit, +// isLastPage and nextPageStart as the REST intro shows them. +func (s *apiServer) pages(path string, per int, items []any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, r *http.Request) { + servePage(w, r, per, items) + }) +} + +// servePage writes the page of items r asks for. +func servePage(w http.ResponseWriter, r *http.Request, per int, items []any) { + q := r.URL.Query() + start, _ := strconv.Atoi(q.Get("start")) + limit, _ := strconv.Atoi(q.Get("limit")) + if limit <= 0 { + limit = 25 + } + limit = min(limit, per) + from := min(start, len(items)) + to := min(from+limit, len(items)) + body := map[string]any{"values": items[from:to], "size": to - from, "limit": limit, "start": start, + "isLastPage": to >= len(items), "filter": nil} + if to < len(items) { + body["nextPageStart"] = to + } + writeJSON(w, http.StatusOK, body) +} + +// requests returns the recorded calls of method to path under the REST +// base ("" matches all). +func (s *apiServer) requests(method, path string) []apiCall { + s.mu.Lock() + defer s.mu.Unlock() + var out []apiCall + for _, c := range s.calls { + if (method == "" || c.Method == method) && (path == "" || c.Path == path) { + out = append(out, c) + } + } + return out +} + +// writes returns the recorded requests that are not GETs. +func (s *apiServer) writes() []apiCall { + var out []apiCall + for _, c := range s.requests("", "") { + if c.Method != http.MethodGet { + out = append(out, c) + } + } + return out +} + +// reset forgets the recorded calls. +func (s *apiServer) reset() { + s.mu.Lock() + s.calls = nil + s.mu.Unlock() +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + if v == nil { + w.WriteHeader(status) + return + } + w.Header().Set("Content-Type", "application/json;charset=UTF-8") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} + +// errorBody is Bitbucket's error body (REST intro, "Errors"). +func errorBody(exception, msg string) map[string]any { + return map[string]any{"errors": []any{map[string]any{"context": nil, "message": msg, "exceptionName": exception}}} +} + +// Exceptions and messages of the fixtures' errors. +const ( + noRepoException = "com.atlassian.bitbucket.repository.NoSuchRepositoryException" + noRepoMessage = "Repository API/missing does not exist." + noPathException = "com.atlassian.bitbucket.content.NoSuchPathException" + authorisation = "com.atlassian.bitbucket.AuthorisationException" +) + +// testToken returns a token made at run time, as long as a real one. +func testToken(t *testing.T) string { + t.Helper() + b := make([]byte, 22) + if _, err := rand.Read(b); err != nil { + t.Fatal(err) + } + return "BBDC-" + hex.EncodeToString(b) +} + +// provider returns the resolved Bitbucket Data Center provider at the +// server. +func (s *apiServer) provider() config.ResolvedProvider { + u, _ := url.Parse(s.base()) + return config.ResolvedProvider{ + Provider: config.Provider{ID: "bbdc", Type: "bitbucket-datacenter", URL: s.base()}, + Host: strings.ToLower(u.Host), + APIURL: s.base() + "/rest/api/latest", + EnvPrefix: "TOUCHMARK_BBDC_", + } +} + +// fixture is a reader with the server's token. +type fixture struct { + *apiServer + reader *reader +} + +func newFixture(t *testing.T) *fixture { + t.Helper() + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{Kind: auth.Token, Token: s.token}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + return &fixture{apiServer: s, reader: r.(*reader)} +} + +// Fixed ids and names. +const ( + repoID = 101 + forkID = 102 + readerID = 11 + writerID = 12 + personID = 13 + serviceID = 14 + syncBranch = "touchmark/acme-eng" + headSHA = "8d51122def5632836d1cb1026e879069e10a1e13" + mainSHA = "d2391895ff0b38a0c28c4f1b3e0f1a4b5c6d7e8f" +) + +// user is a user as GET /users/{slug} and pull requests send it. +func user(id int64, name, slug, typ string) map[string]any { + return map[string]any{"name": name, "emailAddress": name + "@example.com", "active": true, + "displayName": strings.ToUpper(name[:1]) + name[1:], "id": id, "slug": slug, "type": typ, + "links": map[string]any{"self": []any{map[string]any{"href": "https://bitbucket.example.com/users/" + slug}}}} +} + +// project is a repository's project. +func project(key string) map[string]any { + return map[string]any{"key": key, "id": 1, "name": "Acme " + key, "public": false, "type": "NORMAL", + "links": map[string]any{"self": []any{map[string]any{"href": "https://bitbucket.example.com/projects/" + key}}}} +} + +// repoOpt changes a repository fixture. +type repoOpt func(map[string]any) + +func with(k string, v any) repoOpt { return func(m map[string]any) { m[k] = v } } + +// repo is a repository as GET /projects/{key}/repos/{slug} sends it. +func repo(id int64, key, slug string, opts ...repoOpt) map[string]any { + m := map[string]any{ + "slug": slug, "id": id, "name": slug, "hierarchyId": "e3c939f9ef4a7fae272e", "scmId": "git", + "state": "AVAILABLE", "statusMessage": "Available", "forkable": true, "public": false, "archived": false, + "project": project(key), + "links": map[string]any{ + "clone": []any{ + map[string]any{"href": "https://bitbucket.example.com/scm/" + strings.ToLower(key) + "/" + slug + ".git", "name": "http"}, + map[string]any{"href": "ssh://git@bitbucket.example.com:7999/" + strings.ToLower(key) + "/" + slug + ".git", "name": "ssh"}, + }, + "self": []any{map[string]any{"href": "https://bitbucket.example.com/projects/" + key + "/repos/" + slug + "/browse"}}, + }, + } + for _, o := range opts { + o(m) + } + return m +} + +// repoRef is the repository of a pull request's ref. +func repoRef(id int64, key, slug string) map[string]any { + return map[string]any{"id": id, "slug": slug, "name": slug, "project": project(key), "public": false, "scmId": "git", "state": "AVAILABLE"} +} + +// branch is a branch as GET …/branches sends it. +func branch(name, commit string, def bool) map[string]any { + return map[string]any{"id": "refs/heads/" + name, "displayId": name, "type": "BRANCH", + "latestCommit": commit, "latestChangeset": commit, "isDefault": def} +} + +// defBranch declares the default branch of key/slug: GET +// …/branches/default answering the branch. +func (s *apiServer) defBranch(key, slug, name, commit string) { + s.json(repoRoute(key, slug)+"/branches/default", http.StatusOK, branch(name, commit, true)) +} + +// repoRoute is the route of a repository. +func repoRoute(key, slug string) string { return "/projects/" + key + "/repos/" + slug } + +// prSpec describes a pull request fixture. +type prSpec struct { + id int64 + state string // OPEN (default), MERGED, DECLINED + author map[string]any + source string + sourceRepo map[string]any // default the target repository + target string // default "main" + targetRepo map[string]any // default acme/api + commit string // the source commit, headSHA by default + title string + body string // "" leaves the description out, as the API does + draft bool + version int64 +} + +// pr is a pull request as the pull request APIs send it. +func pr(p prSpec) map[string]any { + state := cmpStr(p.state, "OPEN") + dest := p.targetRepo + if dest == nil { + dest = repoRef(repoID, "ACME", "api") + } + src := p.sourceRepo + if src == nil { + src = dest + } + target := cmpStr(p.target, "main") + m := map[string]any{ + "id": p.id, "version": p.version, "title": cmpStr(p.title, "sync"), "state": state, + "open": state == "OPEN", "closed": state != "OPEN", "draft": p.draft, "locked": false, + "createdDate": int64(1788163200000) + p.id, "updatedDate": int64(1788336000000) + p.id, + "fromRef": map[string]any{"id": "refs/heads/" + p.source, "displayId": p.source, "type": "BRANCH", + "latestCommit": cmpStr(p.commit, headSHA), "repository": src}, + "toRef": map[string]any{"id": "refs/heads/" + target, "displayId": target, "type": "BRANCH", + "latestCommit": mainSHA, "repository": dest}, + "author": map[string]any{"user": p.author, "role": "AUTHOR", "approved": false, "status": "UNAPPROVED"}, + "reviewers": []any{}, + "participants": []any{}, + "links": map[string]any{"self": []any{map[string]any{"href": fmt.Sprintf("https://bitbucket.example.com/projects/ACME/repos/api/pull-requests/%d", p.id)}}}, + } + if state != "OPEN" { + m["closedDate"] = int64(1788422400000) + p.id + } + if p.body != "" { + m["description"] = p.body + } + return m +} + +func cmpStr(a, b string) string { + if a != "" { + return a + } + return b +} + +// wantClass checks the class of err (and, with sentinel, that it wraps it). +func wantClass(t *testing.T, what string, err error, class platform.Class, sentinel error) { + t.Helper() + switch { + case err == nil: + t.Errorf("%s: no error, want class %v", what, class) + case platform.ClassOf(err) != class: + t.Errorf("%s: class %v of %v, want %v", what, platform.ClassOf(err), err, class) + case sentinel != nil && !errors.Is(err, sentinel): + t.Errorf("%s: %v does not wrap %v", what, err, sentinel) + } +} diff --git a/internal/platform/bitbucketdc/probe.go b/internal/platform/bitbucketdc/probe.go new file mode 100644 index 0000000..b991af4 --- /dev/null +++ b/internal/platform/bitbucketdc/probe.go @@ -0,0 +1,236 @@ +package bitbucketdc + +import ( + "context" + "errors" + "net/url" + "regexp" + "strconv" + "strings" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Capabilities of Bitbucket Data Center. +const ( + // maxBody is the body budget: a description holds at most 32 768 + // characters (Atlassian's tracker, BSERV-14135), and touchmark's body + // stays below with room to spare. + maxBody = 30000 + // minVersion is the oldest version touchmark supports: 8.19, the + // oldest long-term support release it was built against (drafts need + // 8.18). + minMajor, minMinor = 8, 19 +) + +// limits are the default pacing. Bitbucket meters each user with a token +// bucket (by default 60 requests, refilled at 5 a second, per node of a +// cluster): 120 reads a minute, four targets and two git fetches at once, +// and half a second between two writes stay well within it. +var limits = platform.Limits{Reads: 4, GitReads: 2, ReadsPerMinute: 120, MinInterval: 500 * time.Millisecond} + +// branchPermissions is the check the writer cannot make upfront: branch +// permissions are readable with repository admin rights only, so a push +// meets them ("Branch … can only be modified through pull requests"). +const branchPermissions = "branch-permissions" + +// versionRe matches the version of application-properties: 8.19.3, 10.2.0. +var versionRe = regexp.MustCompile(`^(\d+)\.(\d+)(?:\.(\d+))?`) + +// apiUser is a user as the API reports it (RestApplicationUser): slug is +// the login, name the user name (which X-AUSERNAME carries), type NORMAL +// for a person and SERVICE for a service user, such as the user of a +// project or repository access token. +type apiUser struct { + ID int64 `json:"id"` + Name string `json:"name"` + Slug string `json:"slug"` + DisplayName string `json:"displayName"` + EmailAddress string `json:"emailAddress"` + Active *bool `json:"active"` + Type string `json:"type"` +} + +// Probe reads the instance's version (GET /application-properties, which +// needs no credential) and reports the capabilities of Bitbucket Data +// Center. A version older than 8.19 is ClassUnsupported. +func (d *reader) Probe(ctx context.Context) (platform.Caps, error) { + const op = "probe" + v, raw, err := d.c.instanceVersion(ctx, op) + if err != nil { + return platform.Caps{}, err + } + if v[0] < minMajor || v[0] == minMajor && v[1] < minMinor { + return platform.Caps{}, &platform.Error{Op: op, Class: platform.ClassUnsupported, + Err: errors.New("Bitbucket Data Center " + raw + " is older than 8.19, the oldest version touchmark supports")} + } + return platform.Caps{ + Flavor: "bitbucket-datacenter", + Version: raw, + MaxBody: maxBody, + Draft: platform.DraftNative, + // Pull requests have no labels. + NoLabels: true, + // The activities of a declined pull request name who declined it. + CloserKnown: true, + // Whether a declined pull request can be edited is not documented: + // touchmark writes to none. + ClosedImmutable: true, + // Declining a pull request needs read access only. + ReaderCloses: true, + // Markdown escapes HTML: an HTML comment would show. + Marker: platform.MarkerInRefDef, + RuntimeOnly: []string{branchPermissions}, + Limits: limits, + }, nil +} + +// instanceVersion reads and caches the instance's version. +func (c *client) instanceVersion(ctx context.Context, op string) ([3]int, string, error) { + var props struct { + Version string `json:"version"` + } + if _, err := c.get(ctx, op, c.endpoint("application-properties"), nil, &props); err != nil { + return [3]int{}, "", err + } + m := versionRe.FindStringSubmatch(strings.TrimSpace(props.Version)) + if m == nil { + return [3]int{}, "", shapeError(op, "application-properties: version %q", props.Version) + } + var v [3]int + for i := range 3 { + v[i], _ = strconv.Atoi(m[i+1]) + } + c.mu.Lock() + c.version = &v + c.mu.Unlock() + return v, strings.TrimSpace(props.Version), nil +} + +// Self is the user the credential acts as. +func (d *reader) Self(ctx context.Context) (platform.Account, error) { + return d.c.selfAccount(ctx) +} + +// selfAccount returns the credential's user, looked up once. Bitbucket has +// no "who am I" endpoint: every authenticated answer carries X-AUSERNAME, +// the user's name, so a cheap read (GET /application-properties) gives +// the name, and GET /users?filter= the user whose name it is (an +// Atlassian engineer's advice, not in the reference). An answer without +// the header, or a name no user has, is ClassUnknown. +func (c *client) selfAccount(ctx context.Context) (platform.Account, error) { + const op = "get account" + c.mu.Lock() + if c.self != nil { + a := *c.self + c.mu.Unlock() + return a, nil + } + c.mu.Unlock() + if c.token == "" { + return platform.Account{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errors.New("no credential: the reader is anonymous")} + } + resp, err := c.get(ctx, op, c.endpoint("application-properties"), nil, nil) + if err != nil { + return platform.Account{}, err + } + name := strings.TrimSpace(resp.Header.Get("X-AUSERNAME")) + if name == "" { + return platform.Account{}, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: errors.New("the answer names no user (no X-AUSERNAME header): the token may be refused, or a proxy drops the header")} + } + // Jira percent-encodes the header's value; Bitbucket is not known to, + // and a name with an escape is read both ways. + if unescaped, uerr := url.PathUnescape(name); uerr == nil && unescaped != name { + name = unescaped + } + var found *apiUser + q := url.Values{"filter": {name}} + _, err = listAll(ctx, c, op, c.endpoint("users"), q, pageLimit, 10, func(u apiUser) error { + if found == nil && u.Name == name { + found = &u + } + return nil + }) + switch { + case err != nil: + return platform.Account{}, err + case found == nil: + return platform.Account{}, shapeError(op, "no user is named %q, the name X-AUSERNAME gives", name) + } + a := toAccount(found) + if a.ID == "" || a.Login == "" { + return platform.Account{}, shapeError(op, "user %q has no id or slug", name) + } + if validEmail(found.EmailAddress) { + a.Email = found.EmailAddress + } + c.remember(found) + c.mu.Lock() + c.self = &a + c.mu.Unlock() + return a, nil +} + +// validEmail reports whether s can be a commit author's address: one "@" +// between non-empty parts, and nothing git would refuse or misread. +func validEmail(s string) bool { + local, domain, ok := strings.Cut(s, "@") + return ok && local != "" && domain != "" && !strings.Contains(domain, "@") && + !strings.ContainsFunc(s, func(r rune) bool { return r <= ' ' || r == '<' || r == '>' || r == 0x7f }) +} + +// Lookup resolves a login, a user's slug, to its user: GET /users/{slug}. +// A slug the instance does not know is ErrNotFound. +func (d *reader) Lookup(ctx context.Context, login string) (platform.Account, error) { + const op = "look up account" + if login == "" || strings.ContainsAny(login, "/\x00?#") || login == "." || login == ".." { + return platform.Account{}, invalid(op, "%q is no user slug", login) + } + var u apiUser + if _, err := d.c.get(ctx, op, d.c.endpoint("users", login), nil, &u); err != nil { + return platform.Account{}, err + } + a := toAccount(&u) + if a.ID == "" || a.Login == "" { + return platform.Account{}, shapeError(op, "GET /users/%s: no id or slug", login) + } + d.c.remember(&u) + return a, nil +} + +// remember keeps the name of u by its id. +func (c *client) remember(u *apiUser) { + if u == nil || u.ID <= 0 || u.Name == "" { + return + } + c.mu.Lock() + c.names[strconv.FormatInt(u.ID, 10)] = u.Name + c.mu.Unlock() +} + +// nameOf returns the name of the user with id, "" when Self or Lookup has +// not resolved it. +func (c *client) nameOf(id string) string { + c.mu.Lock() + defer c.mu.Unlock() + return c.names[id] +} + +// toAccount converts a user: its numeric id is the account's id, its slug +// its login. A service user is a bot, a normal user a person; a user of +// another type is unknown. +func toAccount(u *apiUser) platform.Account { + if u == nil || u.ID <= 0 || u.Slug == "" { + return platform.Account{} + } + a := platform.Account{ID: strconv.FormatInt(u.ID, 10), Login: u.Slug, Kind: platform.KindUnknown} + switch u.Type { + case "NORMAL": + a.Kind = platform.KindUser + case "SERVICE": + a.Kind = platform.KindBot + } + return a +} diff --git a/internal/platform/bitbucketdc/probe_test.go b/internal/platform/bitbucketdc/probe_test.go new file mode 100644 index 0000000..615310b --- /dev/null +++ b/internal/platform/bitbucketdc/probe_test.go @@ -0,0 +1,200 @@ +package bitbucketdc + +import ( + "net/http" + "slices" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// properties is GET /application-properties. +func properties(version string) map[string]any { + return map[string]any{"version": version, "buildNumber": "8019003", "buildDate": "1724900000000", "displayName": "Bitbucket"} +} + +func TestProbe(t *testing.T) { + f := newFixture(t) + f.json("/application-properties", http.StatusOK, properties("8.19.3")) + caps, err := f.reader.Probe(t.Context()) + if err != nil { + t.Fatal(err) + } + switch { + case caps.Flavor != "bitbucket-datacenter" || caps.Version != "8.19.3": + t.Errorf("flavor %q, version %q", caps.Flavor, caps.Version) + case caps.Marker != platform.MarkerInRefDef || caps.MaxBody != maxBody || caps.MaxBody >= 32768: + t.Errorf("marker %v, max body %d", caps.Marker, caps.MaxBody) + case !caps.NoLabels || !caps.CloserKnown || !caps.ClosedImmutable || caps.Draft != platform.DraftNative: + t.Errorf("caps = %+v", caps) + case !slices.Contains(caps.RuntimeOnly, branchPermissions): + t.Errorf("runtime-only checks %v lack %s", caps.RuntimeOnly, branchPermissions) + case caps.Limits.ReadsPerMinute != 120 || caps.Commit.API: + t.Errorf("limits %+v, API commits %v", caps.Limits, caps.Commit.API) + case caps.BodyControls(): + t.Error("descriptions offer tick boxes on a platform that escapes HTML") + } + for _, v := range []string{"10.2.1", "9.4.0", "8.20.0", "11.0"} { + f.json("/application-properties", http.StatusOK, properties(v)) + if _, err := f.reader.Probe(t.Context()); err != nil { + t.Errorf("Probe on %s: %v", v, err) + } + } + for _, v := range []string{"8.18.2", "7.21.20", "8.5"} { + f.json("/application-properties", http.StatusOK, properties(v)) + _, err := f.reader.Probe(t.Context()) + wantClass(t, "Probe on "+v, err, platform.ClassUnsupported, nil) + } + for _, v := range []string{"", "latest", "v8.19"} { + f.json("/application-properties", http.StatusOK, properties(v)) + _, err := f.reader.Probe(t.Context()) + wantClass(t, "Probe on version "+v, err, platform.ClassUnknown, nil) + } + if calls := f.requests(http.MethodGet, "/application-properties"); len(calls) == 0 || calls[0].Auth != "Bearer "+f.token { + t.Errorf("the probe sent %q", calls[0].Auth) + } +} + +func TestSelf(t *testing.T) { + f := newFixture(t) + f.json("/application-properties", http.StatusOK, properties("9.4.0")) + f.pages("/users", 1000, []any{ + user(99, "touchmark.reader2", "touchmark.reader2", "NORMAL"), + user(readerID, "touchmark.reader", "touchmark.reader", "NORMAL"), + }) + a, err := f.reader.Self(t.Context()) + if err != nil { + t.Fatal(err) + } + want := platform.Account{ID: "11", Login: "touchmark.reader", Email: "touchmark.reader@example.com", Kind: platform.KindUser} + if a != want { + t.Errorf("Self = %+v, want %+v", a, want) + } + if q := f.requests(http.MethodGet, "/users")[0].Query; q.Get("filter") != "touchmark.reader" { + t.Errorf("the users query %v", q) + } + if _, err := f.reader.Self(t.Context()); err != nil || len(f.requests(http.MethodGet, "/users")) != 1 { + t.Errorf("Self is not cached: %v", err) + } + if f.reader.c.nameOf("11") != "touchmark.reader" { + t.Error("Self does not remember the user's name") + } +} + +func TestSelfFailures(t *testing.T) { + // No X-AUSERNAME: the token is refused, or a proxy drops the header. + f := newFixture(t) + f.username = "" + f.json("/application-properties", http.StatusOK, properties("9.4.0")) + _, err := f.reader.Self(t.Context()) + wantClass(t, "an answer without X-AUSERNAME", err, platform.ClassUnknown, nil) + if err != nil && !strings.Contains(err.Error(), "X-AUSERNAME") { + t.Errorf("the error %q does not say what is missing", err) + } + + // A name sent percent-encoded is read unescaped. + f = newFixture(t) + f.username = "jane%40example.com" + f.json("/application-properties", http.StatusOK, properties("9.4.0")) + f.pages("/users", 1000, []any{user(personID, "jane@example.com", "jane_example.com", "NORMAL")}) + if a, err := f.reader.Self(t.Context()); err != nil || a.ID != "13" { + t.Errorf("Self with an encoded name = %+v, %v", a, err) + } + + // A name no user has. + f = newFixture(t) + f.json("/application-properties", http.StatusOK, properties("9.4.0")) + f.pages("/users", 1000, []any{user(99, "touchmark.reader2", "touchmark.reader2", "NORMAL")}) + _, err = f.reader.Self(t.Context()) + wantClass(t, "a name no user has", err, platform.ClassUnknown, nil) + + // A service user without an address: a project token's. + f = newFixture(t) + f.username = "project_1_bot" + f.json("/application-properties", http.StatusOK, properties("9.4.0")) + bot := user(serviceID, "project_1_bot", "project_1_bot", "SERVICE") + delete(bot, "emailAddress") + f.pages("/users", 1000, []any{bot}) + a, err := f.reader.Self(t.Context()) + if err != nil || a.Kind != platform.KindBot || a.Email != "" || a.ID != "14" { + t.Errorf("Self of a project token = %+v, %v", a, err) + } + + // Anonymous. + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + _, err = r.Self(t.Context()) + wantClass(t, "an anonymous Self", err, platform.ClassAuth, nil) + if len(s.requests("", "")) != 0 { + t.Error("an anonymous Self sent requests") + } +} + +func TestLookup(t *testing.T) { + f := newFixture(t) + f.json("/users/jane.doe", http.StatusOK, user(personID, "Jane.Doe", "jane.doe", "NORMAL")) + f.json("/users/gone", http.StatusNotFound, errorBody("com.atlassian.bitbucket.user.NoSuchUserException", "User gone does not exist.")) + a, err := f.reader.Lookup(t.Context(), "jane.doe") + if err != nil || a.ID != "13" || a.Login != "jane.doe" || a.Kind != platform.KindUser || a.Email != "" { + t.Errorf("Lookup = %+v, %v", a, err) + } + if f.reader.c.nameOf("13") != "Jane.Doe" { + t.Error("Lookup does not remember the user's name") + } + _, err = f.reader.Lookup(t.Context(), "gone") + wantClass(t, "a missing user", err, platform.ClassNotFound, platform.ErrNotFound) + f.reset() + for _, login := range []string{"", "a/b", "..", "x?y"} { + _, err := f.reader.Lookup(t.Context(), login) + wantClass(t, "Lookup("+login+")", err, platform.ClassInvalid, nil) + } + if n := len(f.requests("", "")); n != 0 { + t.Errorf("invalid logins sent %d requests", n) + } +} + +func TestToAccount(t *testing.T) { + for _, tc := range []struct { + typ string + kind platform.AccountKind + }{{"NORMAL", platform.KindUser}, {"SERVICE", platform.KindBot}, {"", platform.KindUnknown}, {"ROBOT", platform.KindUnknown}} { + a := toAccount(&apiUser{ID: 5, Slug: "x", Type: tc.typ}) + if a.Kind != tc.kind || a.ID != "5" || a.Login != "x" { + t.Errorf("type %q: %+v", tc.typ, a) + } + } + if a := toAccount(&apiUser{Slug: "x"}); a.ID != "" { + t.Errorf("a user without an id = %+v", a) + } + if a := toAccount(nil); a.ID != "" { + t.Errorf("no user = %+v", a) + } +} + +func TestNewClient(t *testing.T) { + s := newAPIServer(t) + p := s.provider() + hc := httpx.New(httpx.Options{}) + if _, err := NewWriter(p, auth.Credential{}, hc); err == nil { + t.Error("an anonymous writer") + } + if _, err := NewReader(p, auth.Credential{Kind: auth.App, AppID: "1"}, hc); err == nil { + t.Error("an app credential") + } + other := p + other.Type = "bitbucket" + if _, err := NewReader(other, auth.Credential{}, hc); err == nil { + t.Error("a provider of another type") + } + bad := p + bad.APIURL = "ftp://example.com" + if _, err := NewReader(bad, auth.Credential{}, hc); err == nil { + t.Error("an api_url that is not http") + } +} diff --git a/internal/platform/bitbucketdc/pulls.go b/internal/platform/bitbucketdc/pulls.go new file mode 100644 index 0000000..c6f4bd0 --- /dev/null +++ b/internal/platform/bitbucketdc/pulls.go @@ -0,0 +1,474 @@ +package bitbucketdc + +import ( + "cmp" + "context" + "errors" + "fmt" + "net/url" + "slices" + "strconv" + "strings" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Bounds of listings. +const ( + // maxPRPages bounds the pull requests of one branch of one repository. + // PRs has no way to say it saw only part: past the bound it fails. + maxPRPages = 10 + // maxActivityPages bounds the activities read to find who declined a + // pull request: 2 000 at 500 a page. Past the bound the closer stays + // unknown. + maxActivityPages = 4 + // maxSweepPages bounds the open pull requests of one author. + maxSweepPages = 20 +) + +// Pull request states (RestPullRequest.state). A declined pull request can +// be reopened; a merged one cannot. +const ( + stateOpen = "OPEN" + stateDeclined = "DECLINED" + stateMerged = "MERGED" +) + +// systemID is the id ClosedBy carries for a decline whose activity names +// no user: Bitbucket's auto-decline of inactive pull requests is by "the +// Bitbucket system user", whom the reference does not describe. A user's +// id is a number, so it never meets one. +const systemID = "system" + +// apiPR is a pull request as the API reports it (RestPullRequest), in +// lists and alone. A description that is empty is absent. +type apiPR struct { + ID int64 `json:"id"` + Version *int64 `json:"version"` + Title string `json:"title"` + Description *string `json:"description"` + State string `json:"state"` + Draft bool `json:"draft"` + FromRef apiRef `json:"fromRef"` + ToRef apiRef `json:"toRef"` + Author *apiParticipant `json:"author"` + // Reviewers are sent back by EditPR: whether an update without them + // keeps them is not documented. + Reviewers []apiParticipant `json:"reviewers"` + CreatedDate *int64 `json:"createdDate"` + UpdatedDate *int64 `json:"updatedDate"` + ClosedDate *int64 `json:"closedDate"` +} + +// apiRef is the source or target of a pull request (RestPullRequestRef): +// its full ref, its head commit and its repository. +type apiRef struct { + ID string `json:"id"` + DisplayID string `json:"displayId"` + LatestCommit string `json:"latestCommit"` + Repository *apiRepoRef `json:"repository"` +} + +// apiParticipant is a pull request's author or reviewer. +type apiParticipant struct { + User *apiUser `json:"user"` +} + +// apiActivity is one activity of a pull request (RestPullRequestActivity). +type apiActivity struct { + ID int64 `json:"id"` + CreatedDate int64 `json:"createdDate"` + Action string `json:"action"` + User *apiUser `json:"user"` +} + +// check reports what the driver depends on that a pull request lacks. +func (p *apiPR) check(op string) error { + switch { + case p.ID <= 0: + return shapeError(op, "a pull request without an id") + case p.State != stateOpen && p.State != stateDeclined && p.State != stateMerged: + return shapeError(op, "pull request #%d has state %q", p.ID, p.State) + case p.ToRef.Repository == nil || p.ToRef.Repository.ID <= 0 || p.ToRef.Repository.path() == "": + return shapeError(op, "pull request #%d lacks its target repository", p.ID) + case p.base() == "": + return shapeError(op, "pull request #%d targets %q, no branch", p.ID, p.ToRef.ID) + case p.CreatedDate == nil: + return shapeError(op, "pull request #%d has no createdDate", p.ID) + case p.Version == nil: + return shapeError(op, "pull request #%d has no version", p.ID) + } + return nil +} + +// branchOf returns the branch a full ref names, "" for another ref. +func branchOf(ref string) string { + b, ok := strings.CutPrefix(ref, "refs/heads/") + if !ok { + return "" + } + return b +} + +// head returns the source branch of p, "" when it is no branch (a tag). +func (p *apiPR) head() string { return branchOf(p.FromRef.ID) } + +// base returns the target branch of p. +func (p *apiPR) base() string { return branchOf(p.ToRef.ID) } + +// authorID returns the id of p's author, "" when there is none. +func (p *apiPR) authorID() string { + if p.Author == nil || p.Author.User == nil || p.Author.User.ID <= 0 { + return "" + } + return strconv.FormatInt(p.Author.User.ID, 10) +} + +// repoID returns the id of the repository of ref, "" when it names none. +func (r apiRef) repoID() string { + if r.Repository == nil || r.Repository.ID <= 0 { + return "" + } + return strconv.FormatInt(r.Repository.ID, 10) +} + +// millis converts epoch milliseconds to a time in UTC. +func millis(ms int64) time.Time { return time.UnixMilli(ms).UTC() } + +// toPR converts an API pull request; BaseExists, HeadSHA and ClosedBy are +// left to the caller. Declined pull requests are closed, merged ones +// merged; ClosedAt is closedDate (updatedDate when absent). +func (c *client) toPR(p *apiPR) platform.PR { + body := "" + if p.Description != nil { + body = *p.Description + } + var author platform.Account + if p.Author != nil { + author = toAccount(p.Author.User) + } + pr := platform.PR{ + Number: p.ID, + URL: c.repoWeb(p.ToRef.Repository.Project.Key, p.ToRef.Repository.Slug) + "/pull-requests/" + strconv.FormatInt(p.ID, 10), + State: platform.Open, + Draft: p.Draft, + Head: p.head(), + Base: p.base(), + RepoID: p.ToRef.repoID(), + Title: p.Title, + Body: body, + Labels: []string{}, + Author: author, + CreatedAt: millis(*p.CreatedDate), + } + if pr.Head != "" { + pr.HeadRepoID = p.FromRef.repoID() + } + switch p.State { + case stateMerged: + pr.State = platform.Merged + case stateDeclined: + pr.State = platform.Closed + } + if pr.State != platform.Open { + switch { + case p.ClosedDate != nil: + pr.ClosedAt = millis(*p.ClosedDate) + case p.UpdatedDate != nil: + pr.ClosedAt = millis(*p.UpdatedDate) + } + } + return pr +} + +// PRs returns, newest first, the pull requests into r from heads by authors +// in every state and the open ones by anyone: for each head, GET +// …/pull-requests?state=ALL&direction=OUTGOING&at=refs/heads/, the +// pull requests from that branch of r, filtered again here (the reference +// does not spell out what at means with direction). Pull requests from a +// branch of the same name in a fork are not listed: the core never takes +// them for its own. The listing is complete or the call fails. +// +// BaseExists reads the branch API for a base other than r's default branch +// (once per base). HeadSHA is the source commit of an open pull request. +// ClosedBy of a declined pull request is who declined it, from its +// activities (see closer). +func (d *reader) PRs(ctx context.Context, r platform.Repo, heads []string, authors []platform.Account) ([]platform.PR, error) { + const op = "list pull requests" + key, slug, err := repoPath(op, r) + if err != nil { + return nil, err + } + wanted := setOf(heads) + if len(wanted) == 0 { + return []platform.PR{}, nil + } + ids := accountIDs(authors) + var found []apiPR + seen := map[int64]bool{} + for _, head := range sortedKeys(wanted) { + q := url.Values{ + "state": {"ALL"}, + "direction": {"OUTGOING"}, + "at": {"refs/heads/" + head}, + "order": {"NEWEST"}, + "withAttributes": {"false"}, + "withProperties": {"false"}, + } + complete, err := listAll(ctx, d.c, op, d.c.repoURL(key, slug, "pull-requests"), q, pageLimit, maxPRPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + into := r.ID == "" || p.ToRef.repoID() == r.ID + from := r.ID == "" || p.FromRef.repoID() == r.ID + if !seen[p.ID] && into && from && p.head() == head && (p.State == stateOpen || ids[p.authorID()]) { + seen[p.ID] = true + found = append(found, p) + } + return nil + }) + if err != nil { + return nil, err + } + if !complete { + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%s has more pull requests from %s than %d pages; touchmark does not act on a partial list", r.Path, head, maxPRPages)} + } + } + out := make([]platform.PR, 0, len(found)) + bases := map[string]bool{} + for i := range found { + p := &found[i] + pr := d.c.toPR(p) + if pr.BaseExists, err = d.c.baseExists(ctx, op, r, key, slug, pr.Base, bases); err != nil { + return nil, err + } + if pr.State == platform.Open && isHexOID(p.FromRef.LatestCommit) { + pr.HeadSHA = strings.ToLower(p.FromRef.LatestCommit) + } + if pr.State == platform.Closed { + if pr.ClosedBy, err = d.c.closer(ctx, op, key, slug, p.ID); err != nil { + return nil, err + } + } + out = append(out, pr) + } + slices.SortFunc(out, func(a, b platform.PR) int { return cmp.Compare(b.Number, a.Number) }) + return out, nil +} + +// closer returns who declined pull request id of key/slug: the user of its +// latest DECLINED activity (GET …/pull-requests/{id}/activities, read whole +// up to maxActivityPages: the reference gives no order), nil when none is +// found or the activities run past the bound. A person is a NORMAL user, a +// service user a bot (toAccount); an activity without a user is taken for +// Bitbucket's system user, a bot (systemID). A pull request gone since the +// listing has no closer. +func (c *client) closer(ctx context.Context, op, key, slug string, id int64) (*platform.Account, error) { + var last *apiActivity + complete, err := listAll(ctx, c, op, c.repoURL(key, slug, "pull-requests", strconv.FormatInt(id, 10), "activities"), nil, activityLimit, maxActivityPages, func(a apiActivity) error { + if a.Action == stateDeclined && (last == nil || a.CreatedDate > last.CreatedDate || a.CreatedDate == last.CreatedDate && a.ID > last.ID) { + last = &a + } + return nil + }) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound && !laterPage(err): + return nil, nil + case err != nil: + return nil, err + case !complete || last == nil: + return nil, nil + case last.User == nil: + return &platform.Account{ID: systemID, Login: systemID, Kind: platform.KindBot}, nil + } + a := toAccount(last.User) + if a.ID == "" { + return nil, nil + } + return &a, nil +} + +// baseExists reports whether branch, a pull request's base in r, exists: +// r's default branch does; another is looked up once per call (known holds +// the answers). A pull request whose base was deleted keeps its ref. +func (c *client) baseExists(ctx context.Context, op string, r platform.Repo, key, slug, branch string, known map[string]bool) (bool, error) { + if branch == r.DefaultBranch { + return true, nil + } + if v, ok := known[branch]; ok { + return v, nil + } + _, ok, err := c.branch(ctx, op, key, slug, branch) + if err != nil { + return false, err + } + known[branch] = ok + return ok, nil +} + +// OpenPRsBy lists the open pull requests by authors from heads in every +// repository this identity sees, for the stale sweep: for each author, GET +// /dashboard/pull-requests?state=OPEN&role=AUTHOR&user=, the open +// pull requests the author made, filtered by source branch here. The +// dashboard takes a user's name, which Self and Lookup learn: an author +// neither resolved makes the result incomplete. Each repository is read +// once, for the report and the default branch. HeadSHA stays "": the sweep +// closes pull requests and reads no heads. +// +// A listing that fails, on any page, with a rate limit, a refused +// credential, a transient failure or the end of ctx fails the call, so the +// core can pause or retry; any other failure of a listing (a dashboard +// that refuses another user's name) makes the result incomplete, as does a +// listing capped at its bound. A repository gone since the listing is left +// out; one that cannot be read otherwise makes the result incomplete. +func (d *reader) OpenPRsBy(ctx context.Context, authors []platform.Account, heads []string) (platform.Swept, error) { + const op = "list open pull requests" + ids := accountIDs(authors) + wanted := setOf(heads) + out := platform.Swept{PRs: []platform.RepoPR{}, Complete: true} + if len(ids) == 0 || len(wanted) == 0 { + return out, nil + } + if d.c.token == "" { + return platform.Swept{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errAnonymousSweep} + } + hits := map[string]apiPR{} + for _, id := range sortedKeys(ids) { + name := d.c.nameOf(id) + if name == "" { + out.Complete = false + continue + } + q := url.Values{"state": {stateOpen}, "role": {"AUTHOR"}, "user": {name}, "order": {"NEWEST"}} + complete, err := listAll(ctx, d.c, op, d.c.endpoint("dashboard", "pull-requests"), q, pageLimit, maxSweepPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if p.State == stateOpen && wanted[p.head()] && p.authorID() == id { + hits[p.ToRef.repoID()+"#"+strconv.FormatInt(p.ID, 10)] = p + } + return nil + }) + switch { + case err == nil: + out.Complete = out.Complete && complete + case fatal(err): + return platform.Swept{}, err + default: + out.Complete = false + } + } + keys := make([]string, 0, len(hits)) + for k := range hits { + keys = append(keys, k) + } + slices.Sort(keys) + type known struct { + repo *platform.Repo + bases map[string]bool + } + repos := map[string]*known{} + for _, k := range keys { + p := hits[k] + ref := p.ToRef.Repository + rid := p.ToRef.repoID() + kr, ok := repos[rid] + if !ok { + kr = &known{bases: map[string]bool{}} + r, err := d.repoByRef(ctx, op, ref) + switch { + case err == nil: + kr.repo = &r + case platform.ClassOf(err) == platform.ClassNotFound: + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + } + repos[rid] = kr + } + if kr.repo == nil { + continue + } + key, slug, _ := splitRepoPath(kr.repo.Path) + pr := d.c.toPR(&p) + var err error + pr.BaseExists, err = d.c.baseExists(ctx, op, *kr.repo, key, slug, pr.Base, kr.bases) + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + continue // gone since the listing + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + continue + } + out.PRs = append(out.PRs, platform.RepoPR{Repo: *kr.repo, PR: pr}) + } + slices.SortFunc(out.PRs, func(a, b platform.RepoPR) int { + if c := strings.Compare(strings.ToLower(a.Repo.Path), strings.ToLower(b.Repo.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Repo.ID, b.Repo.ID); c != 0 { + return c + } + return cmp.Compare(b.PR.Number, a.PR.Number) + }) + return out, nil +} + +// repoByRef reads the repository a pull request names, with its default +// branch; one that answers with another id is ClassUnknown. +func (d *reader) repoByRef(ctx context.Context, op string, ref *apiRepoRef) (platform.Repo, error) { + r, err := d.c.getRepo(ctx, op, ref.Project.Key, ref.Slug) + if err != nil { + return platform.Repo{}, err + } + if r.ID != ref.ID { + return platform.Repo{}, shapeError(op, "repository %s answers with id %d, not %d", ref.path(), r.ID, ref.ID) + } + def, err := d.c.defaultBranchOf(ctx, op, r.Project.Key, r.Slug) + if err != nil { + return platform.Repo{}, err + } + return d.c.toRepo(r, def), nil +} + +// errAnonymousSweep is why an anonymous reader cannot sweep: only a user +// has a dashboard. +var errAnonymousSweep = errors.New("no credential: the reader is anonymous, and only a user has pull requests to sweep") + +// accountIDs returns the set of the accounts' ids. +func accountIDs(accounts []platform.Account) map[string]bool { + ids := map[string]bool{} + for _, a := range accounts { + if a.ID != "" { + ids[a.ID] = true + } + } + return ids +} + +// setOf returns the set of the non-empty strings of list. +func setOf(list []string) map[string]bool { + out := map[string]bool{} + for _, s := range list { + if s != "" { + out[s] = true + } + } + return out +} + +// sortedKeys returns the keys of m in order. +func sortedKeys(m map[string]bool) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + slices.Sort(out) + return out +} diff --git a/internal/platform/bitbucketdc/pulls_test.go b/internal/platform/bitbucketdc/pulls_test.go new file mode 100644 index 0000000..6604a91 --- /dev/null +++ b/internal/platform/bitbucketdc/pulls_test.go @@ -0,0 +1,221 @@ +package bitbucketdc + +import ( + "fmt" + "net/http" + "strconv" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Accounts of the pull request fixtures. +var ( + writerUser = user(writerID, "touchmark.writer", "touchmark.writer", "NORMAL") + personUser = user(personID, "Jane.Doe", "jane.doe", "NORMAL") + serviceUser = user(serviceID, "project_1_bot", "project_1_bot", "SERVICE") + writerAcct = platform.Account{ID: strconv.Itoa(writerID), Login: "touchmark.writer", Kind: platform.KindUser} +) + +// activity is an activity of a pull request. +func activity(id int64, action string, at int64, u map[string]any) map[string]any { + m := map[string]any{"id": id, "createdDate": at, "action": action} + if u != nil { + m["user"] = u + } + return m +} + +// prRoute is the route of pull request n of ACME/api, and more. +func prRoute(n int64, more string) string { + return repoRoute("ACME", "api") + "/pull-requests/" + strconv.FormatInt(n, 10) + more +} + +func TestPRs(t *testing.T) { + f := newFixture(t) + fork := repoRef(forkID, "ACME", "fork") + f.pages(repoRoute("ACME", "api")+"/pull-requests", 2, []any{ + pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "body\n\n[touchmark]: # \"touchmark:v1 x\"", version: 3}), + pr(prSpec{id: 8, source: syncBranch, author: personUser, state: "DECLINED"}), // not ours, closed + pr(prSpec{id: 7, source: syncBranch, author: personUser, draft: true}), // anyone's, open + pr(prSpec{id: 6, source: syncBranch, author: writerUser, state: "DECLINED", target: "release"}), + pr(prSpec{id: 5, source: syncBranch, author: writerUser, state: "MERGED"}), + pr(prSpec{id: 4, source: "feature", author: writerUser}), // another branch + pr(prSpec{id: 3, source: syncBranch, author: writerUser, targetRepo: repoRef(201, "OTHER", "api")}), // into another repository + pr(prSpec{id: 2, source: syncBranch, author: writerUser, state: "DECLINED", target: "gone"}), + }) + f.pages(prRoute(6, "/activities"), 500, []any{ + activity(60, "COMMENTED", 1788500000000, personUser), + activity(61, "DECLINED", 1788600000000, personUser), + activity(62, "REOPENED", 1788400000000, writerUser), + activity(63, "DECLINED", 1788300000000, writerUser), + activity(64, "OPENED", 1788200000000, writerUser), + }) + f.pages(prRoute(2, "/activities"), 500, []any{activity(20, "DECLINED", 1788600000000, nil)}) + f.pages(repoRoute("ACME", "api")+"/branches", 1000, []any{branch("release", mainSHA, false), branch("release-2", mainSHA, false)}) + _ = fork + + got, err := f.reader.PRs(t.Context(), f.apiRepoOf(), []string{syncBranch}, []platform.Account{writerAcct}) + if err != nil { + t.Fatal(err) + } + var nums []int64 + for _, p := range got { + nums = append(nums, p.Number) + } + if fmt.Sprint(nums) != "[9 7 6 5 2]" { + t.Fatalf("PRs = %v, want [9 7 6 5 2]", nums) + } + open := got[0] + switch { + case open.State != platform.Open || open.HeadSHA != headSHA || open.Head != syncBranch || open.Base != "main" || !open.BaseExists: + t.Errorf("open PR = %+v", open) + case open.RepoID != "101" || open.HeadRepoID != "101" || open.Author.ID != "12" || open.Author.Kind != platform.KindUser: + t.Errorf("open PR ids = %+v", open) + case open.URL != f.base()+"/projects/ACME/repos/api/pull-requests/9" || open.Body != "body\n\n[touchmark]: # \"touchmark:v1 x\"": + t.Errorf("open PR url %q, body %q", open.URL, open.Body) + case !open.CreatedAt.Equal(time.UnixMilli(1788163200009)) || !open.ClosedAt.IsZero() || open.ClosedBy != nil: + t.Errorf("open PR times %v %v", open.CreatedAt, open.ClosedAt) + case open.Labels == nil: + t.Error("labels are nil") + } + if !got[1].Draft || got[1].Body != "" { + t.Errorf("a draft without a description = %+v", got[1]) + } + declined := got[2] + switch { + case declined.State != platform.Closed || declined.HeadSHA != "" || declined.Base != "release" || !declined.BaseExists: + t.Errorf("declined PR = %+v", declined) + case declined.ClosedBy == nil || declined.ClosedBy.ID != "13" || declined.ClosedBy.Kind != platform.KindUser: + t.Errorf("the latest decline is Jane's, got %+v", declined.ClosedBy) + case !declined.ClosedAt.Equal(time.UnixMilli(1788422400006)): + t.Errorf("closed at %v", declined.ClosedAt) + } + if got[3].State != platform.Merged || got[3].ClosedBy != nil { + t.Errorf("merged PR = %+v", got[3]) + } + system := got[4] + if system.BaseExists || system.ClosedBy == nil || system.ClosedBy.ID != systemID || system.ClosedBy.Kind != platform.KindBot { + t.Errorf("a decline without a user = %+v, closed by %+v", system, system.ClosedBy) + } + q := f.requests(http.MethodGet, repoRoute("ACME", "api")+"/pull-requests")[0].Query + if q.Get("state") != "ALL" || q.Get("direction") != "OUTGOING" || q.Get("at") != "refs/heads/"+syncBranch || q.Get("order") != "NEWEST" { + t.Errorf("the listing asked %v", q) + } + if n := len(f.requests(http.MethodGet, prRoute(8, "/activities"))); n != 0 { + t.Error("the closer of someone else's pull request was read") + } + if n := len(f.requests(http.MethodGet, repoRoute("ACME", "api")+"/branches")); n != 2 { + t.Errorf("%d branch lookups, want one per base (release, gone)", n) + } +} + +func TestPRsCloser(t *testing.T) { + f := newFixture(t) + f.pages(repoRoute("ACME", "api")+"/pull-requests", 1000, []any{ + pr(prSpec{id: 3, source: syncBranch, author: writerUser, state: "DECLINED"}), + }) + for _, tc := range []struct { + name string + items []any + want *platform.Account + }{ + {"a service user", []any{activity(1, "DECLINED", 5, serviceUser)}, &platform.Account{ID: "14", Login: "project_1_bot", Kind: platform.KindBot}}, + {"no decline", []any{activity(1, "OPENED", 5, writerUser)}, nil}, + {"many pages", func() []any { + var items []any + for i := range 2001 { + items = append(items, activity(int64(i), "COMMENTED", int64(i), personUser)) + } + return append(items, activity(5000, "DECLINED", 9999, personUser)) + }(), nil}, + } { + f.pages(prRoute(3, "/activities"), 500, tc.items) + got, err := f.reader.PRs(t.Context(), f.apiRepoOf(), []string{syncBranch}, []platform.Account{writerAcct}) + if err != nil || len(got) != 1 { + t.Fatalf("%s: %v, %v", tc.name, got, err) + } + if fmt.Sprint(got[0].ClosedBy) != fmt.Sprint(tc.want) { + t.Errorf("%s: closed by %+v, want %+v", tc.name, got[0].ClosedBy, tc.want) + } + } + f.json(prRoute(3, "/activities"), http.StatusServiceUnavailable, errorBody("", "busy")) + _, err := f.reader.PRs(t.Context(), f.apiRepoOf(), []string{syncBranch}, []platform.Account{writerAcct}) + wantClass(t, "activities that fail", err, platform.ClassTransient, nil) +} + +func TestPRsFailures(t *testing.T) { + f := newFixture(t) + var many []any + for i := range maxPRPages*1000 + 1 { + many = append(many, pr(prSpec{id: int64(i + 1), source: syncBranch, author: personUser, state: "MERGED"})) + } + f.pages(repoRoute("ACME", "api")+"/pull-requests", 1000, many) + _, err := f.reader.PRs(t.Context(), f.apiRepoOf(), []string{syncBranch}, nil) + wantClass(t, "a listing past its bound", err, platform.ClassUnknown, nil) + + f.pages(repoRoute("ACME", "api")+"/pull-requests", 1000, []any{map[string]any{"id": 1, "state": "OPEN"}}) + _, err = f.reader.PRs(t.Context(), f.apiRepoOf(), []string{syncBranch}, nil) + wantClass(t, "a pull request without refs", err, platform.ClassUnknown, nil) + + f.reset() + if got, err := f.reader.PRs(t.Context(), f.apiRepoOf(), nil, nil); err != nil || len(got) != 0 { + t.Errorf("no heads = %v, %v", got, err) + } + if n := len(f.requests("", "")); n != 0 { + t.Errorf("no heads sent %d requests", n) + } +} + +func TestOpenPRsBy(t *testing.T) { + f := newFixture(t) + f.json("/users/touchmark.writer", http.StatusOK, writerUser) + writer, err := f.reader.Lookup(t.Context(), "touchmark.writer") + if err != nil { + t.Fatal(err) + } + f.pages("/dashboard/pull-requests", 1, []any{ + pr(prSpec{id: 9, source: syncBranch, author: writerUser}), + pr(prSpec{id: 4, source: "feature", author: writerUser}), + pr(prSpec{id: 3, source: syncBranch, author: writerUser, targetRepo: repoRef(201, "OTHER", "tool")}), + pr(prSpec{id: 2, source: syncBranch, author: writerUser, targetRepo: repoRef(301, "GONE", "x")}), + pr(prSpec{id: 1, source: syncBranch, author: personUser}), + }) + f.json(repoRoute("ACME", "api"), http.StatusOK, repo(repoID, "ACME", "api")) + f.defBranch("ACME", "api", "main", mainSHA) + f.json(repoRoute("OTHER", "tool"), http.StatusOK, repo(201, "OTHER", "tool")) + f.defBranch("OTHER", "tool", "main", mainSHA) + f.json(repoRoute("GONE", "x"), http.StatusNotFound, errorBody(noRepoException, "gone")) + + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{writer}, []string{syncBranch}) + if err != nil { + t.Fatal(err) + } + if !got.Complete || len(got.PRs) != 2 { + t.Fatalf("OpenPRsBy = %+v", got) + } + if got.PRs[0].Repo.Path != "ACME/api" || got.PRs[0].PR.Number != 9 || got.PRs[1].Repo.Path != "OTHER/tool" || !got.PRs[0].PR.BaseExists { + t.Errorf("swept %+v", got.PRs) + } + q := f.requests(http.MethodGet, "/dashboard/pull-requests")[0].Query + if q.Get("user") != "touchmark.writer" || q.Get("role") != "AUTHOR" || q.Get("state") != "OPEN" { + t.Errorf("the dashboard query %v", q) + } + + // An author whose name the reader never learnt. + got, err = f.reader.OpenPRsBy(t.Context(), []platform.Account{writer, {ID: "77", Login: "ghost"}}, []string{syncBranch}) + if err != nil || got.Complete { + t.Errorf("an unknown author = %+v, %v", got, err) + } + + // A dashboard that refuses another user's name. + f.json("/dashboard/pull-requests", http.StatusUnauthorized, errorBody(authorisation, "You are not permitted to access this resource")) + got, err = f.reader.OpenPRsBy(t.Context(), []platform.Account{writer}, []string{syncBranch}) + if err != nil || got.Complete { + t.Errorf("a refused dashboard = %+v, %v", got, err) + } + f.json("/dashboard/pull-requests", http.StatusTooManyRequests, nil) + _, err = f.reader.OpenPRsBy(t.Context(), []platform.Account{writer}, []string{syncBranch}) + wantClass(t, "a rate limit", err, platform.ClassRateLimited, nil) +} diff --git a/internal/platform/bitbucketdc/repos.go b/internal/platform/bitbucketdc/repos.go new file mode 100644 index 0000000..83123ce --- /dev/null +++ b/internal/platform/bitbucketdc/repos.go @@ -0,0 +1,427 @@ +package bitbucketdc + +import ( + "context" + "errors" + "net/http" + "net/url" + "slices" + "strconv" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Bounds of listings. +const ( + // maxRepoPages bounds a project's or a label's repositories: 20 000 at + // 1 000 a page. A larger listing resolves incompletely. + maxRepoPages = 20 + // maxBranchPages bounds the branches read looking for one by name. + maxBranchPages = 5 +) + +// apiRepo is a repository as the API reports it (RestRepository, and +// RestLabelable, its form in a label's listing). +type apiRepo struct { + ID int64 `json:"id"` + Slug string `json:"slug"` + Name string `json:"name"` + State string `json:"state"` + Public bool `json:"public"` + Archived bool `json:"archived"` + Project *apiProject `json:"project"` + Origin *apiRepoRef `json:"origin"` + // LabelableType is REPOSITORY in a label's listing, absent elsewhere. + LabelableType string `json:"labelableType"` +} + +// apiProject is a repository's project: its key, and type NORMAL or +// PERSONAL (a user's own, whose key is ~). +type apiProject struct { + Key string `json:"key"` + Public bool `json:"public"` + Type string `json:"type"` +} + +// apiRepoRef is the short repository of a fork's origin or of a pull +// request's ref. +type apiRepoRef struct { + ID int64 `json:"id"` + Slug string `json:"slug"` + Project *apiProject `json:"project"` +} + +// path returns the key/slug of a repository reference, "" when it lacks +// either. +func (r *apiRepoRef) path() string { + if r == nil || r.Project == nil || r.Project.Key == "" || r.Slug == "" { + return "" + } + return r.Project.Key + "/" + r.Slug +} + +// apiBranch is a branch (RestBranch): its full ref id, its name, its head +// commit and whether it is the default branch. +type apiBranch struct { + ID string `json:"id"` + DisplayID string `json:"displayId"` + LatestCommit string `json:"latestCommit"` + IsDefault bool `json:"isDefault"` +} + +// check reports what the driver depends on that a repository lacks. +func (r *apiRepo) check(op string) error { + switch { + case r.ID <= 0 || r.Slug == "": + return shapeError(op, "a repository without id or slug") + case r.Project == nil || r.Project.Key == "": + return shapeError(op, "repository %s has no project key", r.Slug) + } + return nil +} + +// path returns the repository's /. +func (r *apiRepo) path() string { return r.Project.Key + "/" + r.Slug } + +// toRepo converts an API repository; the default branch and whether the +// repository is empty come from defaultBranch: one whose default branch +// does not exist is skipped as empty, though other branches may. A +// repository that is not AVAILABLE (still initialising, failed or offline) +// is disabled; an archived one is read-only. The web URL is under the +// provider's url. +func (c *client) toRepo(r *apiRepo, def defaultBranch) platform.Repo { + visibility := "private" + if r.Public || r.Project.Public { + visibility = "public" + } + return platform.Repo{ + Host: c.host, + ID: strconv.FormatInt(r.ID, 10), + Path: r.path(), + DefaultBranch: def.name, + WebURL: c.repoWeb(r.Project.Key, r.Slug), + Visibility: visibility, + ObjectFormat: "sha1", + Archived: r.Archived, + Disabled: r.State != "" && r.State != "AVAILABLE", + Empty: def.empty || def.missing, + Fork: r.Origin != nil, + } +} + +// repoWeb is the web page of repository key/slug. +func (c *client) repoWeb(key, slug string) string { + return c.web + "/projects/" + url.PathEscape(key) + "/repos/" + url.PathEscape(slug) +} + +// defaultBranch is what the driver knows of a repository's default branch: +// its name ("" when Bitbucket names none), its head commit, whether the +// repository has no branch at all (empty), and whether the default branch +// does not exist while other branches do (missing; head is ""). +type defaultBranch struct { + name, head string + empty, missing bool +} + +// defaultBranchOf reads the default branch of key/slug: GET +// …/branches/default (deprecated, still in the reference), which answers +// the branch with its head, 204 for an empty repository, and 404 when the +// repository or its configured default branch does not exist; then GET +// …/default-branch for the configured name, whose 404 says the same. When +// both answer 404, the repository itself is read: gone, it is +// ClassNotFound; there, its default branch is missing. +func (c *client) defaultBranchOf(ctx context.Context, op, key, slug string) (defaultBranch, error) { + var b apiBranch + resp, err := c.get(ctx, op, c.repoURL(key, slug, "branches", "default"), nil, &b) + switch { + case err == nil && resp.Status == http.StatusNoContent: + def, derr := c.configuredDefault(ctx, op, key, slug) + if derr != nil && platform.ClassOf(derr) == platform.ClassNotFound { + derr = nil + } + def.empty = true + return def, derr + case err == nil: + name := strings.TrimPrefix(b.ID, "refs/heads/") + if name == "" || name == b.ID || !isHexOID(b.LatestCommit) { + return defaultBranch{}, shapeError(op, "the default branch of %s/%s: id %q, commit %q", key, slug, b.ID, b.LatestCommit) + } + return defaultBranch{name: name, head: strings.ToLower(b.LatestCommit)}, nil + case platform.ClassOf(err) != platform.ClassNotFound: + return defaultBranch{}, err + } + def, err := c.configuredDefault(ctx, op, key, slug) + if err != nil && platform.ClassOf(err) == platform.ClassNotFound { + if _, rerr := c.getRepo(ctx, op, key, slug); rerr != nil { + return defaultBranch{}, rerr + } + err = nil + } + def.missing = true + return def, err +} + +// configuredDefault reads the configured default branch's name: GET +// …/default-branch. +func (c *client) configuredDefault(ctx context.Context, op, key, slug string) (defaultBranch, error) { + var ref struct { + ID string `json:"id"` + } + if _, err := c.get(ctx, op, c.repoURL(key, slug, "default-branch"), nil, &ref); err != nil { + return defaultBranch{}, err + } + name := strings.TrimPrefix(ref.ID, "refs/heads/") + if name == ref.ID { + return defaultBranch{}, shapeError(op, "the default branch of %s/%s: id %q", key, slug, ref.ID) + } + return defaultBranch{name: name}, nil +} + +// branch returns the branch name of key/slug, with ok false when it does +// not exist: GET …/branches?filterText=, which matches parts of names, with +// exact and prefix matches first (boostMatches); the exact ref is kept. +func (c *client) branch(ctx context.Context, op, key, slug, name string) (apiBranch, bool, error) { + var found *apiBranch + q := url.Values{"filterText": {name}, "boostMatches": {"true"}, "details": {"false"}} + complete, err := listAll(ctx, c, op, c.repoURL(key, slug, "branches"), q, pageLimit, maxBranchPages, func(b apiBranch) error { + if found == nil && b.ID == "refs/heads/"+name { + found = &b + } + return nil + }) + switch { + case err != nil: + return apiBranch{}, false, err + case found != nil: + if !isHexOID(found.LatestCommit) { + return apiBranch{}, false, shapeError(op, "branch %s has no head commit", name) + } + return *found, true, nil + case !complete: + return apiBranch{}, false, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: errors.New("more branches match " + name + " than touchmark reads, and the branch is not among them")} + } + return apiBranch{}, false, nil +} + +// Repo returns one repository by its / path, with its +// default branch. The API takes keys and slugs ignoring case; the result +// carries the canonical path. +func (d *reader) Repo(ctx context.Context, path string) (platform.Repo, error) { + const op = "get repository" + key, slug, ok := splitRepoPath(path) + if !ok { + return platform.Repo{}, notFound(op, "repository %q: Bitbucket Data Center paths are /", path) + } + r, err := d.c.getRepo(ctx, op, key, slug) + if err != nil { + return platform.Repo{}, err + } + def, err := d.c.defaultBranchOf(ctx, op, r.Project.Key, r.Slug) + if err != nil { + return platform.Repo{}, err + } + return d.c.toRepo(r, def), nil +} + +// getRepo reads GET /projects/{key}/repos/{slug}. +func (c *client) getRepo(ctx context.Context, op, key, slug string) (*apiRepo, error) { + var r apiRepo + if _, err := c.get(ctx, op, c.repoURL(key, slug), nil, &r); err != nil { + return nil, err + } + if err := r.check(op); err != nil { + return nil, err + } + return &r, nil +} + +// Resolve lists the repositories sel selects, sorted by path ignoring case. +// A namespace is a project, by its key: GET /repos?projectkey= +// &archived=ALL, every repository of it the identity reads, archived ones +// included (GET /projects/{key}/repos would need read access to the +// project itself). Projects do not nest, so Subgroups changes nothing. +// Topics are repository labels, every one of them on a repository: GET +// /labels/{name}/labeled?type=REPOSITORY lists a label's repositories the +// identity sees, in every project; the first label's are kept in the +// project, the others' intersect them. Forks only with sel.Forks. Each +// repository's default branch is read (one request each); archived and +// unavailable repositories are included for the core to skip. +// +// The listing is incomplete when it is capped or a later page fails; a +// failure of the first page, a rate limit, an auth error or a transient +// failure is returned instead, so the caller can retry or pause. +func (d *reader) Resolve(ctx context.Context, sel platform.Selector) (platform.Resolved, error) { + const op = "resolve" + if sel.Repo != "" { + r, err := d.Repo(ctx, sel.Repo) + if err != nil { + return platform.Resolved{}, err + } + return platform.Resolved{Repos: []platform.Repo{r}, Complete: true}, nil + } + switch { + case sel.Namespace == "": + return platform.Resolved{}, invalid(op, "the selector names no repository and no namespace") + case strings.ContainsAny(sel.Namespace, "/\x00?#") || sel.Namespace == "." || sel.Namespace == "..": + return platform.Resolved{}, notFound(op, "project %q: Bitbucket Data Center has no nested projects", sel.Namespace) + } + inProject := func(r apiRepo) bool { return strings.EqualFold(r.Project.Key, sel.Namespace) } + var ( + found []apiRepo + complete bool + err error + ) + if len(sel.Topics) == 0 { + q := url.Values{"projectkey": {sel.Namespace}, "archived": {"ALL"}} + found, complete, err = d.c.repos(ctx, op, d.c.endpoint("repos"), q, inProject) + } else { + found, complete, err = d.c.labeled(ctx, op, sel.Topics[0], inProject) + for _, label := range sel.Topics[1:] { + if err != nil || len(found) == 0 { + break + } + var ( + more []apiRepo + c bool + ) + more, c, err = d.c.labeled(ctx, op, label, inProject) + complete = complete && c + ids := map[int64]bool{} + for _, r := range more { + ids[r.ID] = true + } + found = slices.DeleteFunc(found, func(r apiRepo) bool { return !ids[r.ID] }) + } + } + if err != nil { + return platform.Resolved{}, err + } + if len(found) == 0 { + // The listings name no missing project: a key that is wrong, or + // that the project changed, lists nothing. + if err := d.c.checkProject(ctx, op, sel.Namespace); err != nil { + return platform.Resolved{}, err + } + } + out := platform.Resolved{Repos: []platform.Repo{}, Complete: complete} + seen := map[int64]bool{} + for i := range found { + r := &found[i] + if seen[r.ID] || r.Origin != nil && !sel.Forks { + continue + } + seen[r.ID] = true + def, err := d.c.defaultBranchOf(ctx, op, r.Project.Key, r.Slug) + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + continue // gone since the listing + case fatal(err): + return platform.Resolved{}, err + default: + out.Complete = false + continue + } + out.Repos = append(out.Repos, d.c.toRepo(r, def)) + } + sortRepos(out.Repos) + return out, nil +} + +// checkProject tells whether project key exists as the identity sees it: +// GET /projects/{key}. A project Bitbucket does not find is ClassNotFound, +// and so is one that answers with another key (its key changed, and the +// old one leads to it); a project the identity may not read (401 with a +// user, 403) is no error: its repositories may still be readable. +func (c *client) checkProject(ctx context.Context, op, key string) error { + var p apiProject + _, err := c.get(ctx, op, c.endpoint("projects", key), nil, &p) + switch { + case err == nil && !strings.EqualFold(p.Key, key): + return notFound(op, "project %s is %s now", key, p.Key) + case err == nil, platform.ClassOf(err) == platform.ClassPermission: + return nil + } + return err +} + +// repos lists the repositories at u with query that keep accepts. complete +// is false when the listing was capped or a later page failed; a failure +// of the first page, or one that stops the call or is transient, is +// returned. +func (c *client) repos(ctx context.Context, op, u string, query url.Values, keep func(apiRepo) bool) ([]apiRepo, bool, error) { + var out []apiRepo + complete, err := listAll(ctx, c, op, u, query, pageLimit, maxRepoPages, func(r apiRepo) error { + if err := r.check(op); err != nil { + return err + } + if keep(r) { + out = append(out, r) + } + return nil + }) + switch { + case err != nil && (!laterPage(err) || fatal(err)): + return nil, false, err + case err != nil: + complete = false + } + return out, complete, nil +} + +// labeled lists the repositories with label that keep accepts: GET +// /labels/{name}/labeled?type=REPOSITORY. A label no repository has is 404 +// (none). +func (c *client) labeled(ctx context.Context, op, label string, keep func(apiRepo) bool) ([]apiRepo, bool, error) { + if label == "" || strings.ContainsAny(label, "/\x00?#") { + return nil, true, nil + } + q := url.Values{"type": {"REPOSITORY"}} + found, complete, err := c.repos(ctx, op, c.endpoint("labels", label, "labeled"), q, func(r apiRepo) bool { + return (r.LabelableType == "" || r.LabelableType == "REPOSITORY") && keep(r) + }) + if err != nil && platform.ClassOf(err) == platform.ClassNotFound && !laterPage(err) { + return nil, true, nil + } + return found, complete, err +} + +// sortRepos orders repositories by path ignoring case, then path, then id. +func sortRepos(repos []platform.Repo) { + slices.SortFunc(repos, func(a, b platform.Repo) int { + if c := strings.Compare(strings.ToLower(a.Path), strings.ToLower(b.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Path, b.Path); c != 0 { + return c + } + return strings.Compare(a.ID, b.ID) + }) +} + +// Remote is the repository's git URL under the provider's url +// (/scm//.git), never a host the API names, with the +// credential's Bearer header (none when anonymous). +func (d *reader) Remote(_ context.Context, r platform.Repo) (platform.Remote, error) { + const op = "get remote" + if r.Host != "" && !strings.EqualFold(r.Host, d.c.host) { + return platform.Remote{}, invalid(op, "repository %s is on %s, not %s", r.Path, r.Host, d.c.host) + } + key, slug, err := repoPath(op, r) + if err != nil { + return platform.Remote{}, err + } + rem := platform.Remote{URL: d.c.cloneURL(key, slug)} + if d.c.token != "" { + rem.Header = d.c.header + } + return rem, nil +} + +// cloneURL is the HTTP clone URL of key/slug: /scm//.git, +// the key lowercased as Bitbucket writes it. +func (c *client) cloneURL(key, slug string) string { + return c.web + "/scm/" + url.PathEscape(strings.ToLower(key)) + "/" + url.PathEscape(slug) + ".git" +} diff --git a/internal/platform/bitbucketdc/repos_test.go b/internal/platform/bitbucketdc/repos_test.go new file mode 100644 index 0000000..13edf5f --- /dev/null +++ b/internal/platform/bitbucketdc/repos_test.go @@ -0,0 +1,347 @@ +package bitbucketdc + +import ( + "errors" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestRepo(t *testing.T) { + f := newFixture(t) + f.json(repoRoute("ACME", "api"), http.StatusOK, repo(repoID, "ACME", "api")) + f.defBranch("ACME", "api", "main", mainSHA) + f.json(repoRoute("ACME", "fork"), http.StatusOK, repo(forkID, "ACME", "fork", with("public", true), + with("origin", repoRef(repoID, "ACME", "api")))) + f.defBranch("ACME", "fork", "develop", mainSHA) + // An empty repository: 204 from …/branches/default, its configured + // default branch from …/default-branch. + f.json(repoRoute("ACME", "empty"), http.StatusOK, repo(103, "ACME", "empty")) + f.json(repoRoute("ACME", "empty")+"/branches/default", http.StatusNoContent, nil) + f.json(repoRoute("ACME", "empty")+"/default-branch", http.StatusOK, map[string]any{"id": "refs/heads/master", "displayId": "master", "type": "BRANCH"}) + // Branches exist, the configured default branch does not: skipped as + // empty. + f.json(repoRoute("ACME", "odd"), http.StatusOK, repo(104, "ACME", "odd", with("archived", true), with("state", "OFFLINE"))) + f.json(repoRoute("ACME", "odd")+"/branches/default", http.StatusNotFound, errorBody("com.atlassian.bitbucket.repository.NoSuchBranchException", "The default branch does not exist.")) + f.json(repoRoute("ACME", "odd")+"/default-branch", http.StatusOK, map[string]any{"id": "refs/heads/trunk", "displayId": "trunk", "type": "BRANCH"}) + f.json(repoRoute("ACME", "gone"), http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + + got, err := f.reader.Repo(t.Context(), "ACME/api") + if err != nil { + t.Fatal(err) + } + want := platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api", DefaultBranch: "main", + WebURL: f.base() + "/projects/ACME/repos/api", Visibility: "private", ObjectFormat: "sha1"} + if fmt.Sprint(got) != fmt.Sprint(want) { + t.Errorf("Repo = %+v\nwant %+v", got, want) + } + fork, err := f.reader.Repo(t.Context(), "ACME/fork") + if err != nil || !fork.Fork || fork.Visibility != "public" || fork.DefaultBranch != "develop" || fork.Path != "ACME/fork" { + t.Errorf("fork = %+v, %v", fork, err) + } + empty, err := f.reader.Repo(t.Context(), "ACME/empty") + if err != nil || !empty.Empty || empty.DefaultBranch != "master" { + t.Errorf("an empty repository = %+v, %v", empty, err) + } + odd, err := f.reader.Repo(t.Context(), "ACME/odd") + if err != nil || !odd.Empty || odd.DefaultBranch != "trunk" || !odd.Archived || !odd.Disabled { + t.Errorf("a repository whose default branch is missing = %+v, %v", odd, err) + } + _, err = f.reader.Repo(t.Context(), "ACME/gone") + wantClass(t, "a missing repository", err, platform.ClassNotFound, platform.ErrNotFound) + if err != nil && !strings.Contains(err.Error(), noRepoMessage) { + t.Errorf("the error %q lacks the API's message", err) + } + f.reset() + for _, p := range []string{"ACME", "ACME/a/b", "/api", "ACME/..", ""} { + _, err := f.reader.Repo(t.Context(), p) + wantClass(t, "Repo("+p+")", err, platform.ClassNotFound, platform.ErrNotFound) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("paths that name no repository sent %d requests", len(calls)) + } +} + +func TestRepoShape(t *testing.T) { + f := newFixture(t) + f.json(repoRoute("ACME", "api"), http.StatusOK, repo(0, "ACME", "api")) + _, err := f.reader.Repo(t.Context(), "ACME/api") + wantClass(t, "a repository without id", err, platform.ClassUnknown, nil) + + f.json(repoRoute("ACME", "api"), http.StatusOK, repo(repoID, "ACME", "api")) + f.json(repoRoute("ACME", "api")+"/branches/default", http.StatusOK, branch("main", "", true)) + _, err = f.reader.Repo(t.Context(), "ACME/api") + wantClass(t, "a default branch without a commit", err, platform.ClassUnknown, nil) +} + +// projectRepos are the repositories of project ACME the identity reads, +// out of order, with a fork and an archived one, and one of another +// project that a loose match lets through. +func projectRepos() []any { + return []any{ + repo(3, "ACME", "web"), + repo(1, "ACME", "API"), + repo(forkID, "ACME", "api-fork", with("origin", repoRef(201, "OTHER", "api"))), + repo(2, "ACME", "billing", with("archived", true)), + repo(4, "ACME", "zeta"), + repo(5, "ACMEX", "stray"), + } +} + +func TestResolve(t *testing.T) { + f := newFixture(t) + f.pages("/repos", 2, projectRepos()) + for _, slug := range []string{"web", "API", "api-fork", "billing", "zeta"} { + f.defBranch("ACME", slug, "main", mainSHA) + } + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Subgroups: true}) + if err != nil { + t.Fatal(err) + } + var paths []string + for _, r := range got.Repos { + paths = append(paths, r.Path) + } + if strings.Join(paths, " ") != "ACME/API ACME/billing ACME/web ACME/zeta" || !got.Complete { + t.Errorf("Resolve = %v, complete %v", paths, got.Complete) + } + if !got.Repos[1].Archived || got.Repos[0].DefaultBranch != "main" { + t.Errorf("repositories %+v", got.Repos) + } + calls := f.requests(http.MethodGet, "/repos") + if len(calls) != 3 { + t.Fatalf("%d pages read, want 3", len(calls)) + } + if q := calls[0].Query; q.Get("projectkey") != "acme" || q.Get("archived") != "ALL" || q.Get("limit") != "1000" || q.Has("start") { + t.Errorf("the first page asked %v", q) + } + if q := calls[1].Query; q.Get("start") != "2" { + t.Errorf("the second page asked %v", q) + } + if n := len(f.requests(http.MethodGet, repoRoute("ACME", "api-fork")+"/branches/default")); n != 0 { + t.Error("the default branch of a skipped fork was read") + } + + forks, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME", Forks: true}) + if err != nil || len(forks.Repos) != 5 { + t.Errorf("Resolve with forks = %+v, %v", forks.Repos, err) + } +} + +func TestResolveLabels(t *testing.T) { + f := newFixture(t) + // Labels list repositories of every project (RestLabelable). + labeled := func(items ...any) []any { + for _, it := range items { + it.(map[string]any)["labelableType"] = "REPOSITORY" + } + return items + } + f.pages("/labels/python/labeled", 1000, labeled(repo(3, "ACME", "web"), repo(1, "ACME", "api"), repo(9, "OTHER", "tool"))) + f.pages("/labels/service/labeled", 1000, labeled(repo(1, "ACME", "api"), repo(9, "OTHER", "tool"))) + f.json("/labels/none/labeled", http.StatusNotFound, errorBody("com.atlassian.bitbucket.label.NoSuchLabelException", "Label none does not exist.")) + f.json("/projects/ACME", http.StatusOK, project("ACME")) + f.defBranch("ACME", "api", "main", mainSHA) + f.defBranch("ACME", "web", "main", mainSHA) + + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME", Topics: []string{"python", "service"}}) + if err != nil || len(got.Repos) != 1 || got.Repos[0].Path != "ACME/api" || !got.Complete { + t.Errorf("Resolve with two labels = %+v, %v", got, err) + } + if q := f.requests(http.MethodGet, "/labels/python/labeled")[0].Query; q.Get("type") != "REPOSITORY" { + t.Errorf("the label query %v", q) + } + one, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME", Topics: []string{"python"}}) + if err != nil || len(one.Repos) != 2 { + t.Errorf("Resolve with one label = %+v, %v", one, err) + } + none, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME", Topics: []string{"python", "none"}}) + if err != nil || len(none.Repos) != 0 || !none.Complete { + t.Errorf("a label no repository has = %+v, %v", none, err) + } + if n := len(f.requests(http.MethodGet, "/repos")); n != 0 { + t.Errorf("a selector with labels listed the project (%d requests)", n) + } +} + +func TestResolveFailures(t *testing.T) { + // A later page fails: incomplete. + f := newFixture(t) + f.handle(http.MethodGet, "/repos", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("start") != "" { + writeJSON(w, http.StatusBadRequest, errorBody("", "bad page")) + return + } + servePage(w, r, 1, projectRepos()) + }) + f.defBranch("ACME", "web", "main", mainSHA) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + if err != nil || got.Complete || len(got.Repos) != 1 { + t.Errorf("a failed later page = %+v, %v", got, err) + } + + // The first page fails: an error. + f = newFixture(t) + f.json("/repos", http.StatusUnauthorized, errorBody("com.atlassian.bitbucket.auth.AuthenticationException", "Authentication failed. Please check your credentials and try again.")) + f.username = "" + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a refused credential", err, platform.ClassAuth, nil) + + // A default branch that fails for now fails the call; one that is + // refused makes the listing incomplete; a repository gone is left out. + f = newFixture(t) + f.pages("/repos", 1000, []any{repo(1, "ACME", "a"), repo(2, "ACME", "b"), repo(3, "ACME", "c")}) + f.defBranch("ACME", "a", "main", mainSHA) + f.json(repoRoute("ACME", "b")+"/branches/default", http.StatusUnauthorized, errorBody(authorisation, "You are not permitted to access this resource")) + f.json(repoRoute("ACME", "c")+"/branches/default", http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + f.json(repoRoute("ACME", "c")+"/default-branch", http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + f.json(repoRoute("ACME", "c"), http.StatusNotFound, errorBody(noRepoException, noRepoMessage)) + got, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + if err != nil || got.Complete || len(got.Repos) != 1 || got.Repos[0].Path != "ACME/a" { + t.Errorf("Resolve = %+v, %v", got, err) + } + f.json(repoRoute("ACME", "b")+"/branches/default", http.StatusServiceUnavailable, errorBody("", "Bitbucket is starting")) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a transient failure", err, platform.ClassTransient, nil) + + f.reset() + for _, sel := range []platform.Selector{{}, {Namespace: "ACME/sub"}, {Namespace: ".."}} { + _, err := f.reader.Resolve(t.Context(), sel) + if err == nil { + t.Errorf("Resolve(%+v) passed", sel) + } + } + if n := len(f.requests("", "")); n != 0 { + t.Errorf("invalid selectors sent %d requests", n) + } +} + +// TestResolveProject: a project key that lists no repository is checked: +// a project Bitbucket does not find, or one whose key changed, is +// ClassNotFound (the core then sweeps nothing); one the identity may not +// read lists nothing, completely. +func TestResolveProject(t *testing.T) { + f := newFixture(t) + f.pages("/repos", 1000, nil) + f.json("/projects/ACME", http.StatusNotFound, errorBody("com.atlassian.bitbucket.project.NoSuchProjectException", "Project ACME does not exist.")) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a missing project", err, platform.ClassNotFound, platform.ErrNotFound) + f.json("/projects/ACME", http.StatusOK, project("ACME2")) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a project whose key changed", err, platform.ClassNotFound, platform.ErrNotFound) + f.json("/projects/ACME", http.StatusUnauthorized, errorBody(authorisation, "You are not permitted to access this resource")) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + if err != nil || !got.Complete || len(got.Repos) != 0 { + t.Errorf("a project the identity may not read = %+v, %v", got, err) + } + f.json("/projects/ACME", http.StatusOK, project("acme")) + if _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}); err != nil { + t.Errorf("an empty project: %v", err) + } + // A project with repositories is not read. + f.reset() + f.pages("/repos", 1000, []any{repo(1, "ACME", "a")}) + f.defBranch("ACME", "a", "main", mainSHA) + if _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}); err != nil || len(f.requests(http.MethodGet, "/projects/ACME")) != 0 { + t.Errorf("a project with repositories: %v, %d project reads", err, len(f.requests(http.MethodGet, "/projects/ACME"))) + } +} + +func TestRemote(t *testing.T) { + f := newFixture(t) + r := platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api"} + rem, err := f.reader.Remote(t.Context(), r) + if err != nil { + t.Fatal(err) + } + if rem.URL != f.base()+"/scm/acme/api.git" || rem.Header == nil { + t.Errorf("Remote = %+v", rem) + } + if h, err := rem.Header(t.Context()); err != nil || h != "Bearer "+f.token { + t.Errorf("the git header is wrong: %v", err) + } + _, err = f.reader.Remote(t.Context(), platform.Repo{Host: "evil.example.com", Path: "ACME/api"}) + wantClass(t, "a repository of another host", err, platform.ClassInvalid, nil) + + anon, err := NewReader(f.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + if rem, err := anon.Remote(t.Context(), r); err != nil || rem.Header != nil { + t.Errorf("an anonymous remote = %+v, %v", rem, err) + } +} + +func TestErrors(t *testing.T) { + f := newFixture(t) + path := repoRoute("ACME", "api") + for _, tc := range []struct { + name string + status int + body any + username string + class platform.Class + }{ + {"a refused credential", http.StatusUnauthorized, errorBody("com.atlassian.bitbucket.auth.IncorrectPasswordAuthenticationException", "Authentication failed."), "", platform.ClassAuth}, + {"a user without the permission", http.StatusUnauthorized, errorBody("", "The currently authenticated user has insufficient permissions"), "touchmark.reader", platform.ClassPermission}, + {"an AuthorisationException", http.StatusUnauthorized, errorBody(authorisation, "You are not permitted to access this resource"), "", platform.ClassPermission}, + {"a forbidden request", http.StatusForbidden, errorBody("", "Forbidden"), "touchmark.reader", platform.ClassPermission}, + {"a conflict", http.StatusConflict, errorBody("", "The specified version is out of date."), "touchmark.reader", platform.ClassConflict}, + {"a rate limit", http.StatusTooManyRequests, nil, "touchmark.reader", platform.ClassRateLimited}, + {"a server error", http.StatusInternalServerError, errorBody("", "boom"), "touchmark.reader", platform.ClassTransient}, + } { + f.username = tc.username + f.json(path, tc.status, tc.body) + _, err := f.reader.c.getRepo(t.Context(), "get repository", "ACME", "api") + wantClass(t, tc.name, err, tc.class, nil) + if err != nil && strings.Contains(err.Error(), f.token) { + t.Errorf("%s: the error shows the token", tc.name) + } + } + f.username = "touchmark.reader" + f.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "30") + writeJSON(w, http.StatusTooManyRequests, nil) + }) + _, err := f.reader.c.getRepo(t.Context(), "get repository", "ACME", "api") + var pe *platform.Error + if !errors.As(err, &pe) || pe.RetryAfter.Seconds() != 30 { + t.Errorf("a rate limit with Retry-After = %v", err) + } + + // A page that is not the last but names no next start. + f.json("/repos", http.StatusOK, map[string]any{"values": []any{}, "isLastPage": false, "start": 0, "limit": 25, "size": 0}) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a page without nextPageStart", err, platform.ClassUnknown, nil) + f.json("/repos", http.StatusOK, map[string]any{"values": []any{}, "start": 0}) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "ACME"}) + wantClass(t, "a page without isLastPage", err, platform.ClassUnknown, nil) +} + +func TestRedirects(t *testing.T) { + f := newFixture(t) + // A repository renamed: its old path redirects below the API. + f.handle(http.MethodGet, repoRoute("ACME", "old"), func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Location", apiPrefix+repoRoute("ACME", "api")) + w.WriteHeader(http.StatusMovedPermanently) + }) + f.json(repoRoute("ACME", "api"), http.StatusOK, repo(repoID, "ACME", "api")) + f.defBranch("ACME", "api", "main", mainSHA) + if r, err := f.reader.Repo(t.Context(), "ACME/old"); err != nil || r.Path != "ACME/api" { + t.Errorf("a redirected repository = %+v, %v", r, err) + } + // A redirect elsewhere is not followed. + f.handle(http.MethodGet, repoRoute("ACME", "away"), func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Location", "https://evil.example.com/rest/api/latest/projects/ACME/repos/api") + w.WriteHeader(http.StatusFound) + }) + _, err := f.reader.Repo(t.Context(), "ACME/away") + if err == nil { + t.Error("a redirect to another host was followed") + } +} diff --git a/internal/platform/bitbucketdc/target.go b/internal/platform/bitbucketdc/target.go new file mode 100644 index 0000000..556996f --- /dev/null +++ b/internal/platform/bitbucketdc/target.go @@ -0,0 +1,520 @@ +package bitbucketdc + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/url" + "strconv" + "strings" + "sync" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxPermPages bounds the listing that tells whether the writer may write +// to a repository: the repositories of its project with its name that the +// writer may write to. +const maxPermPages = 5 + +// Repository permissions (the permission filter of GET /repos). +const ( + permWrite = "REPO_WRITE" + permAdmin = "REPO_ADMIN" +) + +// hasPermission reports whether the identity has perm on repo: GET +// /repos?projectkey=&name=&permission=&archived=ALL lists +// the repositories of the project with that name on which the identity has +// perm, matched here by id. Bitbucket shows a user's permissions on a +// repository to its admins only; this listing is the identity's own view. +func (c *client) hasPermission(ctx context.Context, op string, repo *apiRepo, perm string) (bool, error) { + q := url.Values{"projectkey": {repo.Project.Key}, "name": {repo.Name}, "permission": {perm}, "archived": {"ALL"}} + found := false + complete, err := listAll(ctx, c, op, c.endpoint("repos"), q, pageLimit, maxPermPages, func(r apiRepo) error { + if r.ID == repo.ID { + found = true + } + return nil + }) + switch { + case err != nil: + return false, err + case !found && !complete: + return false, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("more repositories named %q than touchmark reads, and %s is not among them", repo.Name, repo.path())} + } + return found, nil +} + +// Target checks that the writer may write to r and returns a TargetWriter +// bound to it, before the target's writes. The repository is read as the +// writer: one the writer cannot see is ClassNotFound, and so is one that +// answers with another id (r is gone, and another took its path). Pushes +// need write permission (REPO_WRITE); pull requests need only read +// permission, which seeing the repository implies, and CI files need +// nothing of their own. A writer without write permission gets +// ClassPermission with Rule "contents". +func (w *writer) Target(ctx context.Context, r platform.Repo, need platform.Perms) (platform.TargetWriter, error) { + const op = "target" + if _, err := w.c.selfAccount(ctx); err != nil { + return nil, err + } + if _, err := w.Remote(ctx, r); err != nil { + return nil, err + } + repo, err := w.c.targetRepo(ctx, op, r) + if err != nil { + return nil, err + } + if need.Contents { + ok, err := w.c.hasPermission(ctx, op, repo, permWrite) + switch { + case err != nil: + return nil, err + case !ok: + return nil, &platform.Error{Op: op, Class: platform.ClassPermission, Status: http.StatusForbidden, Rule: "contents", + Err: fmt.Errorf("the writer may not write to %s (no REPO_WRITE): it may not push", repo.path())} + } + } + def, err := w.c.defaultBranchOf(ctx, op, repo.Project.Key, repo.Slug) + if err != nil { + return nil, err + } + pr := w.c.toRepo(repo, def) + rem, err := w.Remote(ctx, pr) + if err != nil { + return nil, err + } + return &target{c: w.c, key: repo.Project.Key, slug: repo.Slug, repo: pr, remote: rem.URL}, nil +} + +// targetRepo reads r as the identity, by its path. A repository that now +// answers with another id than r's is ClassNotFound: r is gone. +func (c *client) targetRepo(ctx context.Context, op string, r platform.Repo) (*apiRepo, error) { + key, slug, err := repoPath(op, r) + if err != nil { + return nil, err + } + repo, err := c.getRepo(ctx, op, key, slug) + if err != nil { + return nil, err + } + if r.ID != "" && strconv.FormatInt(repo.ID, 10) != r.ID { + return nil, notFound(op, "%s is another repository (%d) now, not %s", r.Path, repo.ID, r.ID) + } + return repo, nil +} + +// target is a platform.TargetWriter. The writer's token is not narrowed: +// Close retires the TargetWriter, and later calls through it fail with +// ClassAuth, as after a revoked per-target token elsewhere. +type target struct { + c *client + key, slug string + repo platform.Repo + remote string + + mu sync.Mutex + closed bool +} + +// live returns closedError(op) after Close. +func (t *target) live(op string) error { + t.mu.Lock() + defer t.mu.Unlock() + if t.closed { + return closedError(op) + } + return nil +} + +// Remote is the repository's URL with the writer's Bearer header, which +// fails with ClassAuth after Close. +func (t *target) Remote() platform.Remote { + return platform.Remote{URL: t.remote, Header: func(ctx context.Context) (string, error) { + if err := t.live("git credentials"); err != nil { + return "", err + } + return t.c.header(ctx) + }} +} + +// Close retires the TargetWriter; closing twice is fine. +func (t *target) Close() error { + t.mu.Lock() + t.closed = true + t.mu.Unlock() + return nil +} + +// refBody names a branch of the target repository in a pull request. +type refBody struct { + ID string `json:"id"` + Repository repoBody `json:"repository"` +} + +// repoBody names a repository by its project's key and its slug. +type repoBody struct { + Slug string `json:"slug"` + Project struct { + Key string `json:"key"` + } `json:"project"` +} + +// ref returns the body of branch name of the target repository. +func (t *target) ref(name string) *refBody { + r := &refBody{ID: "refs/heads/" + name} + r.Repository.Slug = t.slug + r.Repository.Project.Key = t.key + return r +} + +// reviewerBody names a reviewer by user name. +type reviewerBody struct { + User struct { + Name string `json:"name"` + } `json:"user"` +} + +// createPR is the body of POST …/pull-requests: from a branch of the +// repository itself, without reviewers (Bitbucket adds default reviewers +// to pull requests made in its web interface, not through REST). +type createPR struct { + Title string `json:"title"` + Description string `json:"description"` + Draft bool `json:"draft"` + FromRef *refBody `json:"fromRef"` + ToRef *refBody `json:"toRef"` +} + +// editPR is the body of PUT …/pull-requests/{id}: the version read, the +// title, description and draft flag as they are or change, the reviewers +// sent back as they are (an update without them is not documented), the +// target branch only to change it. +type editPR struct { + Version int64 `json:"version"` + Title string `json:"title"` + Description string `json:"description"` + Draft bool `json:"draft"` + Reviewers []reviewerBody `json:"reviewers"` + ToRef *refBody `json:"toRef,omitempty"` +} + +// CreatePR opens a pull request from np.Head in the repository itself to +// np.Base, a draft when np.Draft (Bitbucket's own flag; when the instance +// refuses drafts, it is opened ready). Bitbucket Data Center has no labels +// on pull requests: np.Labels are ignored (Caps.NoLabels). +// +// An open pull request from the same head in the repository itself, to +// whatever base, is returned with ErrExists before anything is written. +// When the POST is refused with 409 (Bitbucket keeps one open pull request +// per pair of branches) and such a pull request is open then, it is +// returned with ErrExists too; otherwise the refusal stands. A POST whose +// answer was lost is the core's to reconcile: it lists the pull requests +// of the head before it tries again. +func (t *target) CreatePR(ctx context.Context, np platform.NewPR) (platform.PR, error) { + const op = "create pull request" + if err := t.live(op); err != nil { + return platform.PR{}, err + } + switch { + case np.Head == "" || np.Base == "": + return platform.PR{}, invalid(op, "head and base are required") + case np.Head == np.Base: + return platform.PR{}, invalid(op, "head and base are both %q", np.Head) + case strings.TrimSpace(np.Title) == "": + return platform.PR{}, invalid(op, "the title is empty") + } + open, err := t.openFrom(ctx, op, np) + if err != nil { + return platform.PR{}, err + } + if open != nil { + return t.exists(ctx, op, open, np.Head, 0) + } + body := createPR{Title: np.Title, Description: np.Body, Draft: np.Draft, FromRef: t.ref(np.Head), ToRef: t.ref(np.Base)} + var p apiPR + _, err = t.c.call(ctx, op, http.MethodPost, t.prsURL(), nil, body, &p) + if err != nil && np.Draft && draftRefused(err) { + body.Draft = false + _, err = t.c.call(ctx, op, http.MethodPost, t.prsURL(), nil, body, &p) + } + if err != nil { + if platform.ClassOf(err) == platform.ClassConflict { + return t.existing(ctx, op, np, err) + } + return platform.PR{}, err + } + if err := p.check(op); err != nil { + return platform.PR{}, err + } + return t.finish(ctx, op, &p) +} + +// draftRefused reports whether err, a refused POST, is about the draft +// flag: an instance with drafts turned off (feature.pull.request.drafts). +func draftRefused(err error) bool { + c := platform.ClassOf(err) + return (c == platform.ClassInvalid || c == platform.ClassConflict || c == platform.ClassUnsupported) && + strings.Contains(strings.ToLower(err.Error()), "draft") +} + +// existing returns the open pull request from np.Head in the repository +// itself with ErrExists, after refused, a 409 of CreatePR's POST; refused +// alone when none is open. +func (t *target) existing(ctx context.Context, op string, np platform.NewPR, refused error) (platform.PR, error) { + match, err := t.openFrom(ctx, op, np) + if err != nil { + return platform.PR{}, errors.Join(refused, err) + } + if match == nil { + return platform.PR{}, refused + } + return t.exists(ctx, op, match, np.Head, statusOf(refused)) +} + +// openFrom returns the open pull request from np.Head in the repository +// itself (source and target), the one to np.Base first; nil when none is +// open. +func (t *target) openFrom(ctx context.Context, op string, np platform.NewPR) (*apiPR, error) { + var match *apiPR + q := url.Values{"state": {stateOpen}, "direction": {"OUTGOING"}, "at": {"refs/heads/" + np.Head}, + "withAttributes": {"false"}, "withProperties": {"false"}} + complete, err := listAll(ctx, t.c, op, t.prsURL(), q, pageLimit, maxPRPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if !t.ownOpen(&p, np.Head) { + return nil + } + if match == nil || p.base() == np.Base && match.base() != np.Base { + match = &p + } + return nil + }) + switch { + case err != nil: + return nil, err + case !complete && match == nil: + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%s has more open pull requests from %s than %d pages", t.repo.Path, np.Head, maxPRPages)} + } + return match, nil +} + +// ownOpen reports whether p is an open pull request from head in the +// repository itself. +func (t *target) ownOpen(p *apiPR, head string) bool { + return p.State == stateOpen && p.head() == head && p.FromRef.repoID() == t.repo.ID && p.ToRef.repoID() == t.repo.ID +} + +// exists returns p, the open pull request from head, with the ErrExists of +// CreatePR; status is the HTTP status that told it (0 when a listing did). +func (t *target) exists(ctx context.Context, op string, p *apiPR, head string, status int) (platform.PR, error) { + pr, err := t.finish(ctx, op, p) + if err != nil { + return platform.PR{}, err + } + return pr, &platform.Error{Op: op, Class: platform.ClassConflict, Status: status, + Err: fmt.Errorf("#%d is open from %s: %w", p.ID, head, platform.ErrExists)} +} + +// finish converts a pull request the API answered with, as PRs reports +// it: BaseExists, and the head of an open pull request. A closed one +// carries no ClosedBy: the core reads closers through PRs. +func (t *target) finish(ctx context.Context, op string, p *apiPR) (platform.PR, error) { + pr := t.c.toPR(p) + var err error + if pr.BaseExists, err = t.c.baseExists(ctx, op, t.repo, t.key, t.slug, pr.Base, map[string]bool{}); err != nil { + return platform.PR{}, err + } + if pr.State == platform.Open && isHexOID(p.FromRef.LatestCommit) { + pr.HeadSHA = strings.ToLower(p.FromRef.LatestCommit) + } + return pr, nil +} + +// EditPR changes a pull request. It reads the pull request first, for its +// version, which every change must name. +// +// - Open: title, body and base go in one PUT (with the version, the +// current title, body, draft flag and reviewers sent back, so that +// nothing else changes), skipped when it would change nothing; then, +// for State Closed, the pull request is declined (POST …/decline with +// the version the PUT answered). The PUT comes first: touchmark writes +// no declined pull request, and the body it closes its own pull +// requests with carries the marker's "closed". A failure between the +// two leaves an open pull request with the new body, which the next +// run decides on again. A 409 (the pull request changed since it was +// read) reads it again and tries once more. +// - Declined: State Closed alone changes nothing; State Open, and any +// change of title, body or base, are ClassUnsupported: touchmark +// writes no declined pull request (Caps.ClosedImmutable), though +// Bitbucket can reopen one. +// - Merged: any change is ClassConflict, as a pull request merged since +// the core read it. +// +// State Merged is ClassUnsupported: touchmark merges nothing. Labels are +// ignored (Caps.NoLabels); the draft flag never changes. +func (t *target) EditPR(ctx context.Context, number int64, e platform.PREdit) (platform.PR, error) { + const op = "edit pull request" + if err := t.live(op); err != nil { + return platform.PR{}, err + } + switch { + case number <= 0: + return platform.PR{}, notFound(op, "pull request #%d", number) + case e.Title != nil && strings.TrimSpace(*e.Title) == "": + return platform.PR{}, invalid(op, "the title is empty") + case e.State != nil && *e.State == platform.Merged: + return platform.PR{}, &platform.Error{Op: op, Class: platform.ClassUnsupported, Err: errors.New("touchmark merges no pull request")} + case e.State != nil && *e.State != platform.Open && *e.State != platform.Closed: + return platform.PR{}, invalid(op, "state %q: only open and closed can be set", *e.State) + case e.Base != nil && *e.Base == "": + return platform.PR{}, invalid(op, "the base is empty") + } + var cur *apiPR + for attempt := 0; ; attempt++ { + var err error + if cur, err = t.getPR(ctx, op, number); err != nil { + return platform.PR{}, err + } + done, err := t.edit(ctx, op, cur, e) + switch { + case err == nil: + cur = done + case attempt == 0 && platform.ClassOf(err) == platform.ClassConflict && statusOf(err) == http.StatusConflict: + continue // changed since it was read + default: + return platform.PR{}, err + } + break + } + return t.finish(ctx, op, cur) +} + +// edit applies e to cur, the pull request as just read (see EditPR), and +// returns the pull request as it then is. +func (t *target) edit(ctx context.Context, op string, cur *apiPR, e platform.PREdit) (*apiPR, error) { + body := "" + if cur.Description != nil { + body = *cur.Description + } + changes := e.Title != nil && *e.Title != cur.Title || e.Body != nil && *e.Body != body || + e.Base != nil && *e.Base != cur.base() + switch cur.State { + case stateMerged: + if changes || e.State != nil { + return nil, conflict(op, "#%d is merged: it can no longer be changed", cur.ID) + } + return cur, nil + case stateDeclined: + switch { + case e.State != nil && *e.State == platform.Open: + return nil, &platform.Error{Op: op, Class: platform.ClassUnsupported, + Err: fmt.Errorf("#%d is declined: touchmark reopens no declined pull request on Bitbucket Data Center", cur.ID)} + case changes: + return nil, &platform.Error{Op: op, Class: platform.ClassUnsupported, + Err: fmt.Errorf("#%d is declined: touchmark changes no declined pull request on Bitbucket Data Center", cur.ID)} + } + return cur, nil + } + if changes { + put := editPR{Version: *cur.Version, Title: cur.Title, Description: body, Draft: cur.Draft, Reviewers: []reviewerBody{}} + if e.Title != nil { + put.Title = *e.Title + } + if e.Body != nil { + put.Description = *e.Body + } + if e.Base != nil && *e.Base != cur.base() { + put.ToRef = t.ref(*e.Base) + } + for _, r := range cur.Reviewers { + if r.User != nil && r.User.Name != "" { + var rb reviewerBody + rb.User.Name = r.User.Name + put.Reviewers = append(put.Reviewers, rb) + } + } + var got apiPR + if _, err := t.c.call(ctx, op, http.MethodPut, t.prURL(cur.ID), nil, put, &got); err != nil { + return nil, err + } + if err := got.check(op); err != nil { + return nil, err + } + cur = &got + } + if e.State != nil && *e.State == platform.Closed { + version := strconv.FormatInt(*cur.Version, 10) + var got apiPR + if _, err := t.c.call(ctx, op, http.MethodPost, t.prURL(cur.ID, "decline"), url.Values{"version": {version}}, + map[string]any{"version": *cur.Version}, &got); err != nil { + return nil, err + } + if err := got.check(op); err != nil { + return nil, err + } + if got.State == stateOpen { + return nil, conflict(op, "Bitbucket did not decline #%d (it is %s)", cur.ID, got.State) + } + cur = &got + } + return cur, nil +} + +// getPR reads pull request number alone. +func (t *target) getPR(ctx context.Context, op string, number int64) (*apiPR, error) { + var p apiPR + if _, err := t.c.get(ctx, op, t.prURL(number), nil, &p); err != nil { + return nil, err + } + if err := p.check(op); err != nil { + return nil, err + } + if p.ID != number { + return nil, shapeError(op, "GET pull request #%d answers #%d", number, p.ID) + } + return &p, nil +} + +// Comment adds a comment to pull request number (POST …/comments with the +// text as Markdown). Bitbucket takes comments on declined and merged pull +// requests too, but not in an archived repository. +func (t *target) Comment(ctx context.Context, number int64, body string) error { + const op = "comment" + if err := t.live(op); err != nil { + return err + } + switch { + case number <= 0: + return notFound(op, "pull request #%d", number) + case strings.TrimSpace(body) == "": + return invalid(op, "the comment is empty") + } + _, err := t.c.call(ctx, op, http.MethodPost, t.prURL(number, "comments"), nil, map[string]string{"text": body}, nil) + return err +} + +// EnsureLabels does nothing: Bitbucket Data Center pull requests have no +// labels (Caps.NoLabels), and the core asks for none. +func (t *target) EnsureLabels(context.Context, []string) ([]string, error) { + if err := t.live("ensure labels"); err != nil { + return nil, err + } + return nil, nil +} + +func (t *target) prsURL() string { return t.c.repoURL(t.key, t.slug, "pull-requests") } + +func (t *target) prURL(number int64, more ...string) string { + return t.c.repoURL(t.key, t.slug, append([]string{"pull-requests", strconv.FormatInt(number, 10)}, more...)...) +} + +// conflict is a ClassConflict error of op. +func conflict(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassConflict, Err: fmt.Errorf(format, args...)} +} diff --git a/internal/platform/bitbucketdc/target_test.go b/internal/platform/bitbucketdc/target_test.go new file mode 100644 index 0000000..9e96bcc --- /dev/null +++ b/internal/platform/bitbucketdc/target_test.go @@ -0,0 +1,407 @@ +package bitbucketdc + +import ( + "encoding/json" + "errors" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// writerFixture is a writer with the server's token over one test server +// that takes the writes its routes declare. The token is the writer's. +type writerFixture struct { + *apiServer + writer *writer +} + +// prsRoute is the route of ACME/api's pull requests. +var prsRoute = repoRoute("ACME", "api") + "/pull-requests" + +func newWriterFixture(t *testing.T) *writerFixture { + t.Helper() + s := newAPIServer(t) + s.allowWrites = true + s.username = "touchmark.writer" + w, err := NewWriter(s.provider(), auth.Credential{Kind: auth.Token, Token: s.token}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + s.json("/application-properties", http.StatusOK, properties("9.4.0")) + s.pages("/users", 1000, []any{writerUser}) + s.json(repoRoute("ACME", "api"), http.StatusOK, repo(repoID, "ACME", "api")) + s.defBranch("ACME", "api", "main", mainSHA) + return &writerFixture{apiServer: s, writer: w.(*writer)} +} + +// perms declares the repositories named api the writer has each permission +// on: ACME/api for the permissions given, another project's otherwise. +func (f *writerFixture) perms(on ...string) { + f.handle(http.MethodGet, "/repos", func(w http.ResponseWriter, r *http.Request) { + items := []any{repo(901, "ACMEX", "api")} + for _, p := range on { + if r.URL.Query().Get("permission") == p { + items = append(items, repo(repoID, "ACME", "api")) + } + } + servePage(w, r, 1000, items) + }) +} + +// target returns the TargetWriter of ACME/api, the writer having write +// permission, with the recorded calls forgotten. +func (f *writerFixture) target(t *testing.T) *target { + t.Helper() + f.perms(permWrite) + tw, err := f.writer.Target(t.Context(), platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api"}, platform.Perms{Contents: true, PRs: true}) + if err != nil { + t.Fatal(err) + } + f.reset() + return tw.(*target) +} + +// body decodes the JSON body of a recorded request. +func body(t *testing.T, c apiCall) map[string]any { + t.Helper() + var m map[string]any + if err := json.Unmarshal(c.Body, &m); err != nil { + t.Fatalf("%s %s: body %q: %v", c.Method, c.Path, c.Body, err) + } + return m +} + +// writeLog returns the writes recorded, as "METHOD path". +func (s *apiServer) writeLog() []string { + var out []string + for _, c := range s.writes() { + out = append(out, c.Method+" "+c.Path) + } + return out +} + +func TestTarget(t *testing.T) { + f := newWriterFixture(t) + r := platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api"} + f.perms(permWrite) + tw, err := f.writer.Target(t.Context(), r, platform.Perms{Contents: true, PRs: true, Workflows: true}) + if err != nil { + t.Fatal(err) + } + q := f.requests(http.MethodGet, "/repos")[0].Query + if q.Get("projectkey") != "ACME" || q.Get("name") != "api" || q.Get("permission") != "REPO_WRITE" || q.Get("archived") != "ALL" { + t.Errorf("the permission query %v", q) + } + rem := tw.Remote() + if rem.URL != f.base()+"/scm/acme/api.git" { + t.Errorf("remote %q", rem.URL) + } + if h, err := rem.Header(t.Context()); err != nil || h != "Bearer "+f.token { + t.Errorf("the git header is wrong: %v", err) + } + if err := tw.Close(); err != nil { + t.Fatal(err) + } + _, err = rem.Header(t.Context()) + wantClass(t, "git credentials after Close", err, platform.ClassAuth, nil) + _, err = tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "x"}) + wantClass(t, "CreatePR after Close", err, platform.ClassAuth, nil) + + // Read permission only. + f.perms() + _, err = f.writer.Target(t.Context(), r, platform.Perms{Contents: true, PRs: true}) + wantClass(t, "a reader", err, platform.ClassPermission, nil) + var pe *platform.Error + if errors.As(err, &pe) && pe.Rule != "contents" { + t.Errorf("rule %q, want contents", pe.Rule) + } + // Pull requests alone need read permission only. + if _, err := f.writer.Target(t.Context(), r, platform.Perms{PRs: true}); err != nil { + t.Errorf("pull requests alone: %v", err) + } + + // Another repository took the path. + _, err = f.writer.Target(t.Context(), platform.Repo{Host: r.Host, ID: "55", Path: "ACME/api"}, platform.Perms{Contents: true}) + wantClass(t, "a replaced repository", err, platform.ClassNotFound, platform.ErrNotFound) + // A repository the writer does not see. + f.json(repoRoute("ACME", "secret"), http.StatusNotFound, errorBody(noRepoException, "Repository ACME/secret does not exist.")) + _, err = f.writer.Target(t.Context(), platform.Repo{Host: r.Host, ID: "7", Path: "ACME/secret"}, platform.Perms{Contents: true}) + wantClass(t, "a hidden repository", err, platform.ClassNotFound, nil) +} + +func TestCreatePR(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + f.pages(prsRoute, 1000, nil) + f.handle(http.MethodPost, prsRoute, func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusCreated, pr(prSpec{id: 12, source: syncBranch, author: writerUser, body: "b", draft: true, version: 0})) + }) + got, err := tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "Sync", Body: "b", Labels: []string{"x"}, Draft: true}) + if err != nil { + t.Fatal(err) + } + if got.Number != 12 || !got.Draft || got.HeadSHA != headSHA || !got.BaseExists || got.Body != "b" { + t.Errorf("CreatePR = %+v", got) + } + posts := f.requests(http.MethodPost, prsRoute) + if len(posts) != 1 { + t.Fatalf("%d POSTs", len(posts)) + } + b := body(t, posts[0]) + want := `map[description:b draft:true fromRef:map[id:refs/heads/touchmark/acme-eng repository:map[project:map[key:ACME] slug:api]] title:Sync toRef:map[id:refs/heads/main repository:map[project:map[key:ACME] slug:api]]]` + if fmt.Sprint(b) != want { + t.Errorf("POST body %v\nwant %s", b, want) + } + if _, ok := b["reviewers"]; ok { + t.Error("reviewers were sent") + } +} + +func TestCreatePRExisting(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + // Open already, before anything is written: the one to the base first. + f.pages(prsRoute, 1000, []any{ + pr(prSpec{id: 7, source: syncBranch, author: writerUser, target: "develop"}), + pr(prSpec{id: 6, source: syncBranch, author: writerUser}), + pr(prSpec{id: 5, source: syncBranch, author: writerUser, sourceRepo: repoRef(forkID, "ACME", "fork")}), + }) + got, err := tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "Sync"}) + if !errors.Is(err, platform.ErrExists) || got.Number != 6 { + t.Errorf("CreatePR = #%d, %v; want #6 with ErrExists", got.Number, err) + } + if n := len(f.writes()); n != 0 { + t.Errorf("%d writes", n) + } + + // Opened meanwhile: the POST is refused with 409. + f = newWriterFixture(t) + tw = f.target(t) + listed := 0 + f.handle(http.MethodGet, prsRoute, func(w http.ResponseWriter, r *http.Request) { + listed++ + var items []any + if listed > 1 { + items = append(items, pr(prSpec{id: 8, source: syncBranch, author: writerUser})) + } + servePage(w, r, 1000, items) + }) + f.handle(http.MethodPost, prsRoute, func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusConflict, errorBody("com.atlassian.bitbucket.pull.DuplicatePullRequestException", "Only one pull request may be open for a given source and target branch")) + }) + got, err = tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "Sync"}) + if !errors.Is(err, platform.ErrExists) || got.Number != 8 { + t.Errorf("a 409 = #%d, %v; want #8 with ErrExists", got.Number, err) + } + + // Refused otherwise: up to date, so nothing to propose. + f = newWriterFixture(t) + tw = f.target(t) + f.pages(prsRoute, 1000, nil) + f.handle(http.MethodPost, prsRoute, func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusConflict, errorBody("com.atlassian.bitbucket.pull.EmptyPullRequestException", "The pull request has no changes")) + }) + _, err = tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "Sync"}) + wantClass(t, "a refused POST", err, platform.ClassConflict, nil) + if errors.Is(err, platform.ErrExists) { + t.Error("a refusal without an open pull request is ErrExists") + } +} + +func TestCreatePRDraftsOff(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + f.pages(prsRoute, 1000, nil) + f.handle(http.MethodPost, prsRoute, func(w http.ResponseWriter, r *http.Request) { + var b map[string]any + _ = json.NewDecoder(r.Body).Decode(&b) + if b["draft"] == true { + writeJSON(w, http.StatusBadRequest, errorBody("", "Draft pull requests are disabled")) + return + } + writeJSON(w, http.StatusCreated, pr(prSpec{id: 3, source: syncBranch, author: writerUser})) + }) + got, err := tw.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "Sync", Draft: true}) + if err != nil || got.Draft || len(f.requests(http.MethodPost, prsRoute)) != 2 { + t.Errorf("drafts off = %+v, %v", got, err) + } + for _, np := range []platform.NewPR{{Base: "main", Title: "x"}, {Head: "a", Base: "a", Title: "x"}, {Head: "a", Base: "main", Title: " "}} { + _, err := tw.CreatePR(t.Context(), np) + wantClass(t, fmt.Sprintf("CreatePR(%+v)", np), err, platform.ClassInvalid, nil) + } +} + +func TestEditPR(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + reviewer := user(personID, "Jane.Doe", "jane.doe", "NORMAL") + cur := pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "old", version: 4}) + cur["reviewers"] = []any{map[string]any{"user": reviewer, "role": "REVIEWER", "approved": false, "status": "UNAPPROVED"}} + f.json(prsRoute+"/9", http.StatusOK, cur) + f.handle(http.MethodPut, prsRoute+"/9", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusOK, pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "new", version: 5})) + }) + f.handle(http.MethodPost, prsRoute+"/9/decline", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusOK, pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "new", version: 6, state: "DECLINED"})) + }) + newBody, closed := "new", platform.Closed + got, err := tw.EditPR(t.Context(), 9, platform.PREdit{Body: &newBody, State: &closed, AddLabels: []string{"x"}}) + if err != nil { + t.Fatal(err) + } + if got.State != platform.Closed || got.Body != "new" || got.HeadSHA != "" { + t.Errorf("EditPR = %+v", got) + } + if fmt.Sprint(f.writeLog()) != fmt.Sprintf("[PUT %s/9 POST %s/9/decline]", prsRoute, prsRoute) { + t.Errorf("writes %v", f.writeLog()) + } + put := body(t, f.requests(http.MethodPut, prsRoute+"/9")[0]) + if put["version"] != float64(4) || put["description"] != "new" || put["title"] != "sync" || put["draft"] != false || put["toRef"] != nil { + t.Errorf("PUT body %v", put) + } + if fmt.Sprint(put["reviewers"]) != "[map[user:map[name:Jane.Doe]]]" { + t.Errorf("reviewers sent back %v", put["reviewers"]) + } + decline := f.requests(http.MethodPost, prsRoute+"/9/decline")[0] + if decline.Query.Get("version") != "5" || body(t, decline)["version"] != float64(5) { + t.Errorf("decline with %v %s, want the PUT's version 5", decline.Query, decline.Body) + } + + // Nothing changes: no write. + f.reset() + f.json(prsRoute+"/9", http.StatusOK, pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "new", version: 5})) + if _, err := tw.EditPR(t.Context(), 9, platform.PREdit{Body: &newBody}); err != nil || len(f.writes()) != 0 { + t.Errorf("an edit that changes nothing: %v, writes %v", err, f.writeLog()) + } + + // A new base. + base := "develop" + f.pages(repoRoute("ACME", "api")+"/branches", 1000, []any{branch("develop", mainSHA, false)}) + if _, err := tw.EditPR(t.Context(), 9, platform.PREdit{Base: &base}); err != nil { + t.Fatal(err) + } + put = body(t, f.requests(http.MethodPut, prsRoute+"/9")[0]) + if fmt.Sprint(put["toRef"]) != "map[id:refs/heads/develop repository:map[project:map[key:ACME] slug:api]]" { + t.Errorf("toRef %v", put["toRef"]) + } +} + +func TestEditPRStale(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + reads := 0 + f.handle(http.MethodGet, prsRoute+"/9", func(w http.ResponseWriter, _ *http.Request) { + reads++ + writeJSON(w, http.StatusOK, pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "old", version: int64(reads)})) + }) + puts := 0 + f.handle(http.MethodPut, prsRoute+"/9", func(w http.ResponseWriter, _ *http.Request) { + puts++ + if puts == 1 { + writeJSON(w, http.StatusConflict, errorBody("com.atlassian.bitbucket.pull.PullRequestOutOfDateException", "You are attempting to modify a pull request based on out-of-date information.")) + return + } + writeJSON(w, http.StatusOK, pr(prSpec{id: 9, source: syncBranch, author: writerUser, body: "new", version: 9})) + }) + newBody := "new" + if _, err := tw.EditPR(t.Context(), 9, platform.PREdit{Body: &newBody}); err != nil { + t.Fatal(err) + } + if v := body(t, f.requests(http.MethodPut, prsRoute+"/9")[1])["version"]; v != float64(2) { + t.Errorf("the second PUT names version %v, want the one read again", v) + } + // Stale twice: the conflict stands. + puts = 0 + f.handle(http.MethodPut, prsRoute+"/9", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusConflict, errorBody("com.atlassian.bitbucket.pull.PullRequestOutOfDateException", "out of date")) + }) + _, err := tw.EditPR(t.Context(), 9, platform.PREdit{Body: &newBody}) + wantClass(t, "stale twice", err, platform.ClassConflict, nil) +} + +func TestEditPRClosed(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + newBody, open, closed, merged := "new", platform.Open, platform.Closed, platform.Merged + f.json(prsRoute+"/4", http.StatusOK, pr(prSpec{id: 4, source: syncBranch, author: writerUser, state: "DECLINED", body: "old"})) + if got, err := tw.EditPR(t.Context(), 4, platform.PREdit{State: &closed}); err != nil || got.State != platform.Closed { + t.Errorf("closing a declined pull request = %+v, %v", got, err) + } + _, err := tw.EditPR(t.Context(), 4, platform.PREdit{State: &open}) + wantClass(t, "reopening", err, platform.ClassUnsupported, nil) + _, err = tw.EditPR(t.Context(), 4, platform.PREdit{Body: &newBody}) + wantClass(t, "editing a declined pull request", err, platform.ClassUnsupported, nil) + f.json(prsRoute+"/3", http.StatusOK, pr(prSpec{id: 3, source: syncBranch, author: writerUser, state: "MERGED", body: "old"})) + _, err = tw.EditPR(t.Context(), 3, platform.PREdit{Body: &newBody}) + wantClass(t, "editing a merged pull request", err, platform.ClassConflict, nil) + _, err = tw.EditPR(t.Context(), 3, platform.PREdit{State: &merged}) + wantClass(t, "merging", err, platform.ClassUnsupported, nil) + if n := len(f.writes()); n != 0 { + t.Errorf("%d writes to closed pull requests", n) + } +} + +func TestComment(t *testing.T) { + f := newWriterFixture(t) + tw := f.target(t) + f.handle(http.MethodPost, prsRoute+"/9/comments", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusCreated, map[string]any{"id": 1, "version": 0, "text": "hello"}) + }) + if err := tw.Comment(t.Context(), 9, "hello"); err != nil { + t.Fatal(err) + } + if b := body(t, f.requests(http.MethodPost, prsRoute+"/9/comments")[0]); fmt.Sprint(b) != "map[text:hello]" { + t.Errorf("comment body %v", b) + } + wantClass(t, "an empty comment", tw.Comment(t.Context(), 9, " "), platform.ClassInvalid, nil) + if ids, err := tw.EnsureLabels(t.Context(), []string{"x"}); err != nil || ids != nil { + t.Errorf("EnsureLabels = %v, %v", ids, err) + } +} + +func TestCheck(t *testing.T) { + f := newWriterFixture(t) + r := platform.Repo{Host: f.provider().Host, ID: "101", Path: "ACME/api"} + ids, err := f.writer.Check(t.Context(), nil, nil) + if err != nil || len(ids) != 3 { + t.Fatalf("identity checks = %+v, %v", ids, err) + } + for _, c := range ids { + if c.Status != platform.FindingUnknown { + t.Errorf("%s is %s, want unknown", c.Check, c.Status) + } + } + for _, tc := range []struct { + perms []string + want platform.FindingStatus + }{ + {[]string{permWrite}, platform.FindingOK}, + {[]string{permWrite, permAdmin}, platform.FindingWarn}, + {nil, platform.FindingFail}, + } { + f.perms(tc.perms...) + got, err := f.writer.Check(t.Context(), []platform.Repo{r}, nil) + if err != nil || len(got) != 2 || got[0].Check != "access" || got[0].Status != tc.want || got[1].Check != "rules" || got[1].Status != platform.FindingUnknown { + t.Errorf("perms %v: %+v, %v", tc.perms, got, err) + } + } + f.json(repoRoute("ACME", "gone"), http.StatusNotFound, errorBody(noRepoException, "gone")) + got, err := f.writer.Check(t.Context(), []platform.Repo{{Host: r.Host, Path: "ACME/gone"}}, nil) + if err != nil || len(got) != 1 || got[0].Status != platform.FindingFail { + t.Errorf("a repository the writer does not see = %+v, %v", got, err) + } + if !strings.Contains(got[0].Detail, "does not see") { + t.Errorf("detail %q", got[0].Detail) + } + // A repository hidden behind 401 (with the user's name): fail too. + f.json(repoRoute("ACME", "hidden"), http.StatusUnauthorized, errorBody(authorisation, "You are not permitted to access this resource")) + got, err = f.writer.Check(t.Context(), []platform.Repo{{Host: r.Host, Path: "ACME/hidden"}}, nil) + if err != nil || len(got) != 1 || got[0].Status != platform.FindingFail { + t.Errorf("a repository refused with 401 = %+v, %v", got, err) + } +} diff --git a/internal/platform/fake/fake.go b/internal/platform/fake/fake.go index 00fffd8..cddf08a 100644 --- a/internal/platform/fake/fake.go +++ b/internal/platform/fake/fake.go @@ -14,9 +14,10 @@ // capabilities of its flavor: title-prefix drafts on // GitLab, Gitea and Forgejo; bodies and comments that would run GitLab // quick actions are refused (git mode runs them and records a violation); -// closers are not reported on Gitea and Forgejo; on Bitbucket Cloud labels -// are refused and a pull request closed without merging can never be -// edited or reopened (Caps.ClosedImmutable), and so on Azure DevOps, whose +// closers are not reported on Gitea and Forgejo; on Bitbucket Cloud and +// Bitbucket Data Center labels are refused and a pull request closed +// without merging can never be edited or reopened (Caps.ClosedImmutable), +// and so on Azure DevOps, whose // marker lives in a pull request property (Caps.MarkerInProperties: the // description alone is held to Caps.MaxBody, a marker line in it never // counts, and a person editing the description leaves the stored marker @@ -73,6 +74,8 @@ const ( Bitbucket Flavor = "bitbucket" // AzureDevOps is Azure DevOps Services. AzureDevOps Flavor = "azure-devops" + // BitbucketDataCenter is Bitbucket Data Center. + BitbucketDataCenter Flavor = "bitbucket-datacenter" ) // Tree entry modes. @@ -94,8 +97,10 @@ var Epoch = time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) // requests immutable, no permission of its own for CI files. Azure // DevOps' are its driver's too: a 4 000-byte description with the marker // in a property (MarkerInProperties), native drafts, labels by name, a -// known closer, closed pull requests immutable. A flavor other than the -// six known ones gets GitHub's capabilities under its own name. +// known closer, closed pull requests immutable. Bitbucket Data Center's are +// Bitbucket Cloud's with a description of 32 768 characters (30 000 for +// touchmark). A flavor other than the seven known ones gets GitHub's +// capabilities under its own name. func CapsFor(f Flavor) platform.Caps { c := platform.Caps{ Flavor: string(f), @@ -126,6 +131,14 @@ func CapsFor(f Flavor) platform.Caps { c.Marker = platform.MarkerInRefDef c.RuntimeOnly = []string{"branch-restrictions"} c.Limits = platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second} + case BitbucketDataCenter: + c.MaxBody = 30000 + c.NoLabels = true + c.ClosedImmutable = true + c.ReaderCloses = true + c.Marker = platform.MarkerInRefDef + c.RuntimeOnly = []string{"branch-permissions"} + c.Limits = platform.Limits{Reads: 4, GitReads: 2, ReadsPerMinute: 120, MinInterval: 500 * time.Millisecond} case AzureDevOps: c.MaxBody = 4000 c.ClosedImmutable = true @@ -750,7 +763,7 @@ func (p *Platform) SetPRState(repoID string, number int64, state platform.PRStat } switch state { case platform.Open: - if p.caps.ClosedImmutable && ps.pr.State == platform.Closed { + if p.caps.ClosedImmutable && ps.pr.State == platform.Closed && Flavor(p.caps.Flavor) != BitbucketDataCenter { p.setupf("SetPRState(%s, #%d): a declined pull request cannot be reopened", repoID, number) return } @@ -1262,6 +1275,9 @@ func (p *Platform) prURL(s *repoState, number int64) string { return base + "/pulls/" + n case Bitbucket: return base + "/pull-requests/" + n + case BitbucketDataCenter: + project, name, _ := strings.Cut(s.repo.Path, "/") + return "https://" + p.host + "/projects/" + project + "/repos/" + name + "/pull-requests/" + n case AzureDevOps: project, name, _ := strings.Cut(s.repo.Path, "/") return "https://" + p.host + "/" + project + "/_git/" + name + "/pullrequest/" + n @@ -1355,7 +1371,7 @@ func notFound(op, format string, args ...any) error { } // unsupported is a write the platform refuses whatever the identity may -// do (a declined pull request on Bitbucket Cloud). +// do (a declined pull request on Bitbucket). func unsupported(op, format string, args ...any) error { return &platform.Error{Op: op, Class: platform.ClassUnsupported, Status: http.StatusBadRequest, Err: fmt.Errorf(format, args...)} diff --git a/internal/platform/fake/gitserve.go b/internal/platform/fake/gitserve.go index 4b12a11..6ebc37c 100644 --- a/internal/platform/fake/gitserve.go +++ b/internal/platform/fake/gitserve.go @@ -84,11 +84,12 @@ func (p *Platform) gitHeader(id string) func(context.Context) (string, error) { // appTokens reports whether the flavor is GitHub-like: identities are Apps // whose tokens carry permissions (every flavor but GitLab, Gitea, Forgejo, -// Bitbucket and Azure DevOps, whose identities are accounts with tokens of +// Bitbucket, Bitbucket Data Center and Azure DevOps, whose identities are +// accounts with tokens of // their own). Called with mu held. func (p *Platform) appTokens() bool { switch Flavor(p.caps.Flavor) { - case GitLab, Gitea, Forgejo, Bitbucket, AzureDevOps: + case GitLab, Gitea, Forgejo, Bitbucket, BitbucketDataCenter, AzureDevOps: return false } return true diff --git a/internal/platform/fake/meter.go b/internal/platform/fake/meter.go index ff9551b..ce82beb 100644 --- a/internal/platform/fake/meter.go +++ b/internal/platform/fake/meter.go @@ -183,7 +183,8 @@ func (t *target) createWrites(np platform.NewPR) (first, then int) { // editWrites is how many writes the flavor's driver sends for e, before // the edit is known to apply (first) and after (then): the edit, on Gitea // and Forgejo after a state or base change in a request of its own, on -// Bitbucket followed by the decline of a close, on Azure DevOps after the +// Bitbucket and Bitbucket Data Center followed by the decline of a close, on +// Azure DevOps after the // property of a new body's marker and one call per label added; then // labels added in a call of their own but on GitLab and Azure DevOps, after // each missing one is created. @@ -199,7 +200,7 @@ func (t *target) editWrites(e platform.PREdit) (first, then int) { if (flavor == Gitea || flavor == Forgejo) && (e.State != nil || e.Base != nil) && (e.Title != nil || e.Body != nil) { first++ } - if flavor == Bitbucket && e.State != nil && (e.Title != nil || e.Body != nil || e.Base != nil) { + if (flavor == Bitbucket || flavor == BitbucketDataCenter) && e.State != nil && (e.Title != nil || e.Body != nil || e.Base != nil) { first++ } if len(e.AddLabels) > 0 && flavor != GitLab { diff --git a/internal/platform/fake/writer.go b/internal/platform/fake/writer.go index 63aefaa..f3fc631 100644 --- a/internal/platform/fake/writer.go +++ b/internal/platform/fake/writer.go @@ -279,8 +279,8 @@ func (t *target) createPR(ctx context.Context, np platform.NewPR) (platform.PR, // writer as the closer), a new base is taken to exist, labels are only // added. Draft never changes. // -// With Caps.ClosedImmutable (Bitbucket Cloud) a PR closed without merging -// is final, as Bitbucket's driver reports it: reopening it, or changing its +// With Caps.ClosedImmutable (Bitbucket) a PR closed without merging is +// final, as Bitbucket's drivers report it: reopening it, or changing its // title, body or base, is ClassUnsupported and changes nothing; Closed // alone changes nothing. An open PR's new body is written before it closes, // in the same call. diff --git a/internal/platform/fake/writer_test.go b/internal/platform/fake/writer_test.go index 58187a4..546674a 100644 --- a/internal/platform/fake/writer_test.go +++ b/internal/platform/fake/writer_test.go @@ -378,6 +378,37 @@ func TestBitbucketFlavor(t *testing.T) { wantSetupErr(t, e.p, "cannot be reopened") } +// TestBitbucketDataCenterFlavor: the Bitbucket Data Center flavor reports +// the capabilities of its driver and treats a declined pull request as +// Bitbucket Cloud's flavor does, but that a person may reopen it. +func TestBitbucketDataCenterFlavor(t *testing.T) { + t.Parallel() + ctx := t.Context() + c := fake.CapsFor(fake.BitbucketDataCenter) + if c.Flavor != "bitbucket-datacenter" || !c.NoLabels || !c.ClosedImmutable || c.Marker != platform.MarkerInRefDef || c.BodyControls() || + c.Draft != platform.DraftNative || !c.CloserKnown || c.WorkflowPerm || c.MaxBody != 30000 { + t.Errorf("CapsFor(BitbucketDataCenter) = %+v", c) + } + e := newEnv(t, fake.WithFlavor(fake.BitbucketDataCenter)) + r := e.repo("ACME/api", "README.md", "x") + tw := e.target(r) + pr, err := tw.CreatePR(ctx, platform.NewPR{Head: "touchmark/hub", Base: "main", Title: "sync", Body: "first"}) + if err != nil || pr.URL != "https://github.com/projects/ACME/repos/api/pull-requests/1" { + t.Fatalf("CreatePR = %+v, %v", pr, err) + } + closed, open := platform.Closed, platform.Open + if _, err := tw.EditPR(ctx, pr.Number, platform.PREdit{Body: ptr("closed"), State: &closed}); err != nil { + t.Fatal(err) + } + _, err = tw.EditPR(ctx, pr.Number, platform.PREdit{State: &open}) + wantRule(t, "the writer reopens", err, platform.ClassUnsupported, http.StatusBadRequest, "") + // A person may. + e.p.SetPRState(r.ID, pr.Number, platform.Open, nil, time.Time{}) + if err := e.p.Err(); err != nil || e.p.PR(r.ID, pr.Number).State != platform.Open { + t.Errorf("a person's reopening: %v", err) + } +} + // TestAzureDevOpsFlavor: the Azure DevOps flavor reports the capabilities // of its driver, keeps the marker apart from the description (the // description alone fits Caps.MaxBody, a marker pasted into it never diff --git a/internal/platform/platform.go b/internal/platform/platform.go index e9316f0..b0099c2 100644 --- a/internal/platform/platform.go +++ b/internal/platform/platform.go @@ -1,5 +1,6 @@ // Package platform defines what touchmark needs from a code hosting platform -// (GitHub, GitLab, Gitea, Forgejo, Bitbucket Cloud, Azure DevOps): the +// (GitHub, GitLab, Gitea, Forgejo, Bitbucket Cloud, Azure DevOps, Bitbucket +// Data Center): the // Reader and Writer interfaces drivers implement, their capabilities, and one // error model. // @@ -300,8 +301,8 @@ type Limits struct { // Caps describes a platform instance, found once per run by Reader.Probe. type Caps struct { // Flavor is "github", "ghe.com", "ghes", "gitlab", "gitea", "forgejo", - // "bitbucket" (Bitbucket Cloud) or "azure-devops" (Azure DevOps - // Services). + // "bitbucket" (Bitbucket Cloud), "azure-devops" (Azure DevOps + // Services) or "bitbucket-datacenter". Flavor string Version string // MaxBody is the body budget in bytes, marker included, but where the @@ -333,8 +334,15 @@ type Caps struct { // entry acts while it is present instead of once (see // decide.MemoryConfig.ClosedImmutable). ClosedImmutable bool - Marker MarkerStore - Commit struct{ API, SignedByPlatform, CAS bool } + // ReaderCloses is set when an identity that may only read a repository + // can still close its pull requests (Bitbucket Data Center, where + // declining needs read access): a plan then warns unless the read + // credential's account is among automation_accounts, so that a pull + // request declined with the read key, which every branch of the hub + // can use, is not taken for the team's decision. + ReaderCloses bool + Marker MarkerStore + Commit struct{ API, SignedByPlatform, CAS bool } // RuntimeOnly lists checks the writer cannot read upfront, e.g. // "push_rules" on GitLab. RuntimeOnly []string diff --git a/schemas/doctor.schema.json b/schemas/doctor.schema.json index ed2acb0..b09871c 100644 --- a/schemas/doctor.schema.json +++ b/schemas/doctor.schema.json @@ -113,7 +113,7 @@ "required": ["id", "type", "host", "resolve_complete", "checks"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops", "bitbucket-datacenter"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/hub.schema.json b/schemas/hub.schema.json index 5381f43..cd6175d 100644 --- a/schemas/hub.schema.json +++ b/schemas/hub.schema.json @@ -210,7 +210,7 @@ { "if": { "required": ["platform"], - "properties": { "platform": { "enum": ["gitea", "forgejo"] } } + "properties": { "platform": { "enum": ["gitea", "forgejo", "bitbucket-datacenter"] } } }, "then": { "required": ["base_url"] } }, @@ -260,8 +260,8 @@ "pattern": "^[a-z][a-z0-9-]{0,31}$" }, "providerType": { - "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud; azure-devops: one organization of Azure DevOps Services (bitbucket and azure-devops: plan, distribute and doctor; setup does not set up a hub on them yet).", - "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops"] + "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud; azure-devops: one organization of Azure DevOps Services; bitbucket-datacenter: a Bitbucket Data Center instance (bitbucket, azure-devops and bitbucket-datacenter: plan, distribute and doctor; setup does not set up a hub on them yet).", + "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops", "bitbucket-datacenter"] }, "sign": { "description": "auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto.", @@ -301,7 +301,7 @@ }, "type": { "$ref": "#/$defs/providerType" }, "url": { - "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo, and for azure-devops: https://dev.azure.com/, one provider per organization. A target targets.yml names by web URL belongs to the provider whose url it lies under.", + "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo, for bitbucket-datacenter (the base URL, with its context path, like https://bitbucket.example.com/bitbucket), and for azure-devops: https://dev.azure.com/, one provider per organization. A target targets.yml names by web URL belongs to the provider whose url it lies under.", "$ref": "#/$defs/url" }, "api_url": { @@ -313,7 +313,7 @@ "$ref": "#/$defs/relativePath" }, "writer": { - "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}; on Azure DevOps the user's id, a GUID.", + "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}; on Azure DevOps the user's id, a GUID; on Bitbucket Data Center the user's slug.", "$ref": "#/$defs/account" }, "known_authors": { @@ -349,7 +349,7 @@ "allOf": [ { "if": { - "properties": { "type": { "enum": ["gitea", "forgejo", "azure-devops"] } } + "properties": { "type": { "enum": ["gitea", "forgejo", "azure-devops", "bitbucket-datacenter"] } } }, "then": { "required": ["url"] } }, diff --git a/schemas/operations.schema.json b/schemas/operations.schema.json index ba88dd4..2d58327 100644 --- a/schemas/operations.schema.json +++ b/schemas/operations.schema.json @@ -28,7 +28,7 @@ } }, "forget_declines": { - "description": "Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud and Azure DevOps, where touchmark never edits a declined pull request, the entry acts while it is present.", + "description": "Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud, Bitbucket Data Center and Azure DevOps, where touchmark never edits a declined pull request, the entry acts while it is present.", "type": ["array", "null"], "items": { "type": "object", diff --git a/schemas/report.schema.json b/schemas/report.schema.json index bdcb19a..6d28b40 100644 --- a/schemas/report.schema.json +++ b/schemas/report.schema.json @@ -142,7 +142,7 @@ "required": ["id", "type", "host", "write_check", "resolve_complete"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket", "azure-devops", "bitbucket-datacenter"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/targets.schema.json b/schemas/targets.schema.json index 4088d8a..925108f 100644 --- a/schemas/targets.schema.json +++ b/schemas/targets.schema.json @@ -89,7 +89,7 @@ "required": ["org"], "properties": { "org": { - "description": "Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace, a Bitbucket workspace or an Azure DevOps organization (the provider's url; a project is selected with match) (a synonym of group), named by its path or its web URL.", + "description": "Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace, a Bitbucket workspace, a Bitbucket Data Center project (by its key) or an Azure DevOps organization (the provider's url; a project is selected with match) (a synonym of group), named by its path or its web URL.", "$ref": "#/$defs/namespace" }, "provider": { "$ref": "#/$defs/providerId" }, @@ -134,7 +134,7 @@ } }, "topics": { - "description": "Only repositories with all of these topics. Bitbucket and Azure DevOps have no topics: check refuses them on their entries; use match.", + "description": "Only repositories with all of these topics (repository labels on Bitbucket Data Center). Bitbucket Cloud and Azure DevOps have no topics: check refuses them on their entries; use match.", "type": ["array", "null"], "items": { "type": "string", "pattern": "[^\t\n\u000b\f\r \u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000\ufeff]" } },