Repository navigation
feat: add Gitea and Forgejo providers, and integration tests for the forge CLIs - #88
Merged
Merged
Conversation
The new `integration` shell has the packages of the default shell, and also curl, gh, glab, tea, fj (forgejo-cli), and the gitea and forgejo servers, at the versions of the locked nixpkgs. Forgejo is broken on aarch64-darwin, so the shell has it only where it builds. The default, nvim-0_12_0, and nightly shells do not change.
`make test-integration` runs the files in `tests/integration/`, and `make test` now leaves them out. `make test-file` runs a file of either kind. The new blocking `Integration` CI job runs the target in the `integration` shell with `CMP_GIT_INTEGRATION_REQUIRE=1`, so that a missing CLI fails its cases and does not skip them. `tests/integration/helpers.lua` has a fake HTTP/1.1 server on `vim.uv`, over TCP or a Unix socket, with a route table, a request log, and a proxy mode that removes a path prefix. It also gives gh and glab config directories that send their requests to the server. `helpers.new_set` takes `pre_once` and `post_once` hooks, and `helpers.defer` runs a function at the end of a case.
The cases send all requests of gh 2.102.0 to the fake server over a Unix socket. They cover the GraphQL queries of `gh issue list --json` and `gh pr list --json` with the state and the limit of the config, the mentions preflight and two pages of collaborators with `gh api`, and the contributors after a failed preflight. `gh repo set-default --view` reads the default repository from the git config and sends no request. The commands run in a directory outside a repository, and a curl that fails at once keeps the curl fallback off the network.
The cases send the `glab api` requests for the issues, the merge requests, and the users of a project in a subgroup to the fake server. They check the items, the requests, and the token of glab. An HTTP error gives no items and is not cached.
`remote_url.parse` now also returns `scheme = "http"` or `scheme = "https"` for an `http://` or `https://` remote. Other remotes have no scheme. Discovery passes the field through the SSH aliases and the host prefixes, so the Gitea and Forgejo providers can use the scheme of the remote for their API URLs. The field is optional, and the GitHub and GitLab providers do not read it.
`change_requests.with_issues` starts the jobs of the issues and the change requests at once and gives one list, as the GitHub source did in `_issues_and_change_requests`. `utils.empty_nulls` replaces the JSON null text fields, as the GitLab adapter did. The Gitea and Forgejo providers use both. The test helper `helpers.fake_commands` replaces the copies in the GitHub and GitLab source tests.
The new `gitea` provider completes the issues, the pull requests, and the mentions of gitea.com and of the hosts in `gitea.hosts`. The new `forgejo` provider does the same for codeberg.org and `forgejo.hosts`. Both use `/api/v1` of the server, which Forgejo kept after its fork, through one shared adapter and one shared source. - `#` completes the issues and the pull requests, `!` the pull requests, and `@` the assignees of the repository. A pull request shows as `#N` under `#` and as `!N` under `!`. - The issues and the pull requests come from the issues API, with `type=issues` and `type=pulls`, one page of at most `limit` items. The default limit is 50, the most that the servers give. - curl sends `GITEA_TOKEN` or `FORGEJO_TOKEN`, then asks again without a token. A token of another host fails also on a public repository, so the last request gets the public items. The assignees API needs a token, so the mentions have no request without one. - An `http://` remote uses plain HTTP for the API. All other remotes use HTTPS. A host can have a port and a path prefix. The items go to the disk cache in `gitea-*` and `forgejo-*` files. Core gets the `gitea_*` and `forgejo_*` actions and passes the hosts of both providers to discovery. The default trigger actions run them after the GitHub and GitLab actions.
A list command that exits with 0 and prints an error object, or other output that is not a list, now fails as a command with an exit code that is not 0. The fallback then runs the next command, and the callback gets one list. Before, the error object ended the fallback with an empty list. `tea api` exits with 0 on an HTTP error, so the Gitea provider needs this to fall back to curl. gh and glab exit with an error code on an HTTP error, so their requests do not change.
The Gitea provider now asks `tea login list -o json` for the logins of tea once in the session, without blocking. A login matches when its URL without the scheme and the trailing `/` is the host of the remote. Of more than one match, the default login wins, and else the first one. With a matching login, each request runs `tea api -l <login>` first, then the curl commands. Without `-l`, tea can use the login of another server. Without a matching login, or without tea, no `tea api` process starts. Requests that wait for the lookup share it, and a cancelled request starts no command. `:CmpGitClearCache` drops the logins, so the next request that is not a cache hit reads them again.
The Forgejo provider now reads the token that fj keeps for the host from fj's `keys.json`, and sends it with curl first. Then curl sends `FORGEJO_TOKEN`, and then asks without a token. - The file is `keys.json` in the data directory of fj: `$XDG_DATA_HOME/forgejo-cli` or `~/.local/share/forgejo-cli` on Linux, `~/Library/Application Support/forgejo-cli.forgejo-cli` on macOS, and `%APPDATA%\forgejo-cli\forgejo-cli\data` on Windows. - The host key has the port and the path prefix, as fj writes it. - The token of an `Application` or an `OAuth` entry is used. fj refreshes an OAuth token only when fj runs, so an expired token fails, and the next command runs. - The file is read for each request, so a new `fj auth` login applies to the next request that is not a cache hit. A missing file, invalid JSON, or another shape gives no token. The test cases clear `XDG_DATA_HOME` and `APPDATA`, so that the keys of the developer do not apply.
`integration.forge("gitea")` and `integration.forge("forgejo")` start
a real server from the integration shell once for each test file, in
the first case that needs it, and `post_once` stops it. The server
runs on a free port of 127.0.0.1 in its own temporary directory, with
SQLite, no SSH, no mail, and no network. It gets the admin `root` with
an access token, and the seed data: `root/public` and the private
`root/private`, each with an issue, a pull request from a branch, and
the collaborator `ada`.
- The start time goes to the notes of the case, for CI. A server
starts and seeds in about 3 s.
- The log of the server goes to the notes of a case that fails.
- A server with a path prefix in ROOT_URL runs behind the fake server
as a proxy, which removes the prefix.
- The Forgejo cases always skip on aarch64-darwin, where nixpkgs marks
forgejo as broken.
The Gitea cases run the real tea with a login, curl with
`GITEA_TOKEN`, an invalid token on a public repository, `@` without
credentials, a server with a path prefix, and an `http://` remote
through discovery. The Forgejo cases run curl with the token that
`fj auth add-token` writes, with `FORGEJO_TOKEN`, and without a token.
The README now lists the Gitea and Forgejo features, tea and fj as optional requirements, the setup for private repositories with the token scopes, and the config defaults with the new trigger actions. `docs/configuration.md` describes: - The new actions, the new sources of callback actions, and the `scheme` field of the git info. - The item fields and the sort of the new capabilities. - The command chains of Gitea and Forgejo, the tea login match, the key file of fj on each OS, the refresh of an expired fj OAuth token, the 50-item cap, and the mentions that need a token. - The Gitea and Forgejo hosts, their path prefixes, and the plain HTTP of an `http://` remote, with a warning. - The cache of the new items, and the tea logins that `:CmpGitClearCache` drops. The sections "GitHub CLI, GitLab CLI, and curl" and "Hosted GitHub and GitLab instances" are now "CLIs, tokens, and curl" and "Hosted instances". CONTRIBUTING describes the `integration` shell, `make test-integration`, the integration helpers, the `Integration` CI job, and the new isolation of each case.
The types of Neovim 0.12 give `vim.env.XDG_DATA_HOME` the type `string`, so lua-language-server rejected the fallback to nil when `HOME` is not known. The variable is now `string?`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the
giteaandforgejoproviders, which complete the issues, the pull requests, and the mentions of Gitea and Forgejo servers. It also adds integration tests that run the realgh,glab,tea,fj, and curl, and a blockingIntegrationCI job for them.Closes #35
Behavior
giteahandles gitea.com and the hosts ingitea.hosts.forgejohandles codeberg.org and the hosts inforgejo.hosts. Both use/api/v1of the server through one shared adapter and one shared source.#completes the issues and the pull requests,!the pull requests, and@the assignees of the repository. A pull request shows as#Nunder#and as!Nunder!.tea api -l <login>if tea has a login for the host, then curl withGITEA_TOKEN, then curl without a token.fjkeeps for the host in itskeys.json, then curl withFORGEJO_TOKEN, then curl without a token.@shows no items and starts no request.limititems (default 50, the most that the servers give). The servers sort by creation time, so the 50 newest items show.http://remote uses plain HTTP for the Gitea and Forgejo requests. All other remotes use HTTPS. GitHub and GitLab always use HTTPS. Discovery now keeps theschemeof anhttp://orhttps://remote.:CmpGitClearCachealso drops the tea logins.Other changes
tea apiexits with 0 on an HTTP error, so the Gitea chain needs this. gh and glab exit with an error code on an HTTP error, so their requests do not change.change_requests.with_issuesandutils.empty_nullsare shared by the GitHub, GitLab, Gitea, and Forgejo code.docs/configuration.md, and CONTRIBUTING describe the providers, the chains, the token scopes (read:issueandread:repository), the limits, the fj OAuth refresh, the hosted instances, and the integration tests.Integration tests
nix develop .#integrationhasgh,glab,tea,fj, and thegiteaandforgejoservers. nixpkgs marksforgejoas broken onaarch64-darwin, so the shell has noforgejothere, and the Forgejo cases skip.make test-integrationrunstests/integration/.make testleaves this directory out.ghandglabtalk to a fake HTTP server in the Neovim of the tests. tea, the fj token, and curl talk to real Gitea and Forgejo servers on 127.0.0.1, with seeded public and private repositories. A server starts and seeds in about 3 s. No case uses the network.CMP_GIT_INTEGRATION_REQUIRE=1, as in the newIntegrationCI job, a missing binary fails the case. Otherwise the case skips.Validation
make cipasses.make testpasses on Neovim 0.12.0 (288 cases).nix develop .#integration -c make test-integrationpasses on x86_64-linux withCMP_GIT_INTEGRATION_REQUIRE=1(24 cases).gitea/tea) and codeberg.org (forgejo-contrib/forgejo-cli) without credentials:#gives the issues and the pull requests,!the pull requests, and@no items.make bench-comparereports 5 regressions, all instartup/. For example,startup/require0.73 → 0.88 ms (165 → 189 KB), andstartup/new_source32 → 43 µs (16 → 23 KB). The cause is the code and the default config of the two new providers:config.normalizenow copies five provider sections instead of three. The keystroke, request, and e2e cases are within the limits.