Since forgejo/forgejo!10397 and forgejo/forgejo!12335 have landed, there shouldn't be a need for the `hooks` directory in each repository.
This PR cleans up the `tests/gitea-repositories-meta/*` testdata.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12824
Reviewed-by: limiting-factor <limiting-factor@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Extracted from #12824 on suggestion of @limiting-factor; refactored after @Gusted pointed out forgejo/forgejo!12335.
Behavior change: previously a missing `hooks` folder in a repository tree (should not happen before forgejo/forgejo!12335) would return a 500 on `/api/v1/repos/%s/hooks/git`. It now returns a 200, with the same reply as an empty `hooks` folder.
Test has been added to ensure correct handling of missing `hooks` folder and of its creation if necessary.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12833
Reviewed-by: limiting-factor <limiting-factor@noreply.codeberg.org>
Prevent examples hooks, description file, and other files from the default template to be created during git init.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12335
Reviewed-by: Otto <otto@codeberg.org>
A handful of routes, described in this PR as "mixed routes", are currently accessible by both web-based sessions and authenticated API users. The goal of this PR is to allow access to these routes for Authorized Integrations as well, bringing them to full API compatibility (to my knowledge) with other authentication methods. These routes are impacted:
- `/{username}/{repo}/raw/*`
- `/{username}/{repo}/archive/*`
- `/{username}/{repo}/releases/download/{vTag}/{fileName}`
- `/{username}/{repo}/attachments/{uuid}`
- `/attachments/{uuid}`
The major work in this PR was to refactoring the existing authentication methods so that "path based matching" that they were currently doing was no longer required, as I didn't want to introduce that into Authorized Integrations. All the path based matching is removed in this PR, and authentication methods are enabled entirely by the middleware applied to their endpoints.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12776
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Since the final size is already known, no need to `ReadAll` a `LimitedReader`: directly `ReadFull` a properly sized buffer.
Tests are already present in `blob_test.go` (a failure can be triggered by creating a smaller `buf`).
`go test -run=TestBlob_Data -bench=Blob_Data -benchmem` before:
```
Benchmark_Blob_Data-18 43964 28727 ns/op 1373 B/op 11 allocs/op
```
After:
```
Benchmark_Blob_Data-18 41308 27679 ns/op 846 B/op 10 allocs/op
```
🎉 one allocation spared!
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12795
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
The goal is to enable access to Git LFS resources with Authorized Integrations JWTs.
Blocker that needed to be resolved is that adding the `AuthorizedIntegration` auth method would conflict with the LFS tokens, which are handed out during git ssh clones to allow access to LFS resources -- `AuthorizedIntegration` would mark these as `AuthenticationAttemptedIncorrectCredential`, and therefore the requests would 401 before they got to the LFS-specific token validation routines. The fix is to move LFS token authentication into an authentication group so that it could be resolved at the same time as the authorized integration, rather than doing it inside the LFS server routines.
Refactors for LFS tokens are covered by refreshed test automation. Authorized integrations LFS Access has been manually tested, and will be further covered in an end-to-end integration test (https://code.forgejo.org/forgejo/end-to-end/pulls/1954).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [ ] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12725
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
This PR adds zoekt as a code search engine for forgejo. This Pull Request is a continuation of the discussion #8302.
The meilisearch search engine was not suitable, as it is not designed for searching by code. The zoekt project was proposed instead. Zoekt copes well with code indexing, but its operating principle differs from such search engines as elasticsearch.
While elasticsearch can return a result in a ready-made form (with pagination, ready-made snippets, etc.) and forgejo only needs to show this result in the interface with a little work with the data, zoekt works completely differently.
Zoekt finds matches in the repository index and returns a response. The response contains a line with the search word, its number from the file, and also a context, if specified in the request. This response is not suitable for Forgejo, so you need to assemble it yourself. To assemble the response from Zoekt into a form acceptable for Forgejo, I had to write some code and create a new function `searchZoektResult`, since the existing `searchResult` function is completely unsuitable for this search engine. I also had to write logic for pagination, highlighting, and correct display of lines in found snippets with a match, but this is a feature of Zoekt.
At the moment, Zoekt does not support deleting a repository index by repo_id, it only supports complete deletion of all repositories. But I still implemented the Delete function, which deletes a specific repository by its ID.
Co-authored-by: Aleksandr Gamzin <gamzin@altlinux.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/8827
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Load the primary language of the repository when it's converted to a API struct. This is simpler than adding `LoadAttributes` to a lot of places.
Resolvesforgejo/forgejo#12729
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12737
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
These are by the far the longest time spent on during a migration.
Indicate the progress of how many issues and PRs were migrated so far.
Don't overwhelm the messenger, so they are only updated once a batch is
migrated. Which is "slow" enough to see it's not stuck and still doing work.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12738
Reviewed-by: Ellen Εμίλια Άννα Zscheile <fogti@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Forgejo would not trigger Actions workflows `on: pull_request:` with `paths:` or `paths-ignore:` filters when the pull request was merged. The reason was that the triggers were evaluated after the PR was merged, but Forgejo still looked for changed files between the base branch and the PR's HEAD, which by then was already in the base branch.
Resolves https://codeberg.org/forgejo/forgejo/issues/12585.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12739
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Allow authentication to git HTTP & git LFS via an authorized integration.
This is the first step in getting rid of OAuth, basic auth, etc.'s usage of [`isGitRawOrAttachPath(req)`](26f18a94ee/services/auth/method/basic.go (L38-L40)). I don't want to follow that pattern of HTTP route matching in the authentication method, so I've broken the HTTP routes related to git functionality out to using a separate authentication middleware in the top-level `web.Routes` handler. As this approach is expanded to the other endpoints in order to add support to them for authorized integrations, eventually it will be possible to remove this URL matching completely and just rely on middleware installation.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12715
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Weirdly, git doesn't verify the consistency of objects when receiving
new objects. Enable that git verifies this, so we don't allow a
repository to get in a weird or even corrupt state.
We've already dealt with a few cases of inconsistent objects, the most
notable one being mode of objects (forgejo/forgejo!9161). This can be
risky, as such ignore 3 consistency checks that are not harmful to
ignore and is battle tested by Gitlab.
bad timezone:
692a0d3476
missing space:
2da0b39399
non-zero padded filemode:
db8f2e8da5
Typically we set these settings in `modules/git/git.go`, but that means
a instance administrator wouldn't be able to override it. Given we don't
strictly require these settings to be set. A instance admin could
choose to disable the consistency checks or override our set of ignores
this would allow them to do so via the `[git.config]` section.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12695
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Reviewed-by: elle <0xllx0@noreply.codeberg.org>
* remove two unused strings I identified while doing other things
* update two strings per request of @mahlzahn while avoiding a whole separate PR for this
* move 126 strings to JSON, some are remapped with a better structure
* previous migration: forgejo/forgejo!12280
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12718
Reviewed-by: Robert Wolff <mahlzahn@posteo.de>
`ENABLE_AUTO_GIT_WIRE_PROTOCOL`:
Its sole usage is to set `-c protocol.version=2` on each git command
execution. The default value is already 2 since at least the minimum
version of Git that Forgejo requires. When this setting was added, this
was not the case.
Thus, automatically defaulting to protocol v2 is already the case due to
git themselves making it the default. And instances that want to use a
older protocol already have to override the value like:
```ini
[git.config]
protocol.version=1
```
---
`git.reflog` was deprecated in v1.21 warnings have been emitted. Remove it finally.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12681
Reviewed-by: limiting-factor <limiting-factor@noreply.codeberg.org>
Currently, Forgejo supports configuring static group team mappings for
an OIDC authentication source that map OIDC groups to Forgejo
organizations and teams. For example, the following mapping
```json
{"Developer": {"MyForgejoOrganization": ["MyForgejoTeam1", "MyForgejoTeam2"]}}
```
automatically adds a user in the OIDC group `Developer` to the teams
`MyForgejoTeam1` and `MyForgejoTeam2` in organization
`MyForgejoOrganization`.
In order to support more dynamic mappings and to avoid having to update
the mappings for new organizations and teams, add an additional
configuration option that supports mappings with placeholders like in
the following example:
```json
["group-{org}-{team}", "other:{org}/{team}"]
```
In this example, the mappings add a user in OIDC groups
`group-org1-team1`, `group-org2-team2`, and `other:org3/team3` to team
`team1` in organization `org1`, team `team2` in organization `org2`, and
to team `team3` in organization `org3`.
Additionally, this adds a configuration option to dynamically remove
users from organization teams. If enabled, a user is removed from all
teams that are not added via a static or dynamic mapping. Thus, users
are only in teams that are added via such a mapping and no other teams.
Docs: forgejo/docs!1950
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11656
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
## Checklist
Following the previous contribution that added admin-level management of user access tokens (particularly useful for bot/service accounts), this change exposes the created_at field in the API response when listing or retrieving access tokens.
This information is needed to implement token rotation policies for these users — knowing when a token was created allows administrators to identify and revoke stale tokens.
### Tests for Go changes
- I added test coverage for Go changes...
- [X] in their respective `*_test.go` for unit tests.
- [X] `make pr-go` before pushing
### Documentation
- [X] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [X] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/12620): <!--number 12620 --><!--line 0 --><!--description ZXhwb3NlIGFjY2VzcyB0b2tlbiBjcmVhdGlvbiBkYXRlIGluIEFQSSByZXNwb25zZXM=-->expose access token creation date in API responses<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12620
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
The original issue only mentions 'Verified', but 'Updated' was also
missing and so is also included.
The integration test only covers the initial `false` state. Attempting
to cover the flip to true seemed to introduce more problems than
benefits (as outlined in `tests/integration/api_keys_test.go`)
Manual testing was performed to check that verifying the key in the web
ui caused the return value to change from false to true in the API
response (using `curl`).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12625
Reviewed-by: Cyborus <cyborus@disroot.org>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Extends the UI introduced in #12558 to have edit capabilities. (not in scope: "Add" for a new Authorized Integration will be the next update to this UI; `create-authorized-integration` CLI is still the only way to create a new record)
This PR includes a few refactoring steps. The goal of these steps is to have `services/auth` be a single entrypoint for validating, inserting, or updating an authorized integration. Some logic is moved out of `services/authz` because it is not authorization related, and some is moved out of `services/auth/method` to allow it to be reused during validation without creating a cyclical module dependency.
This PR also adds comprehensive validation to the more complex fields in the authorized integration, such as the issuer and claim rules. This validation applies to the `forgejo admin user create-authorized-integration` CLI as well.
The visible UI is the same as #12558, but with a "Save" button, and the ability to display errors:
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [x] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
- Documentation is on my TODO list and will be completed before release.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12601
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
A separate commit status is introduced for skipped checks. That enables marking them as such in the UI instead of successful, which could be misleading.
Resolves https://codeberg.org/forgejo/forgejo/issues/10138.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12606
Reviewed-by: Cyborus <cyborus@disroot.org>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Fixes#2325.
This introduces a way to download downsized versions of the user and repository avatars:
* `/avatars/123abcd` still serves the full-size avatar
* `/avatars/123abcd?size=64` serves it at size 64x64 px
Those downsized versions are computed on demand when requested for the first time and cached. The caching is done in a storage location configurable in the instance settings, just like the storage locations for the full-sized avatars are. The sizes of the downsized images are restricted to a fixed set of sizes, so that the cache doesn't grow too big. The caching and resizing logic is exposed in a way that could potentially be reused for other types of images (such as user uploads in issue discussions).
Luckily, the Go templates already specify in many places which size those avatars should be rendered, even if this information was only used for external avatar providers (such as Gravatar) until now.
The range of sizes requested by the HTML templates is rather wide: the table below lists all the sizes I could find, and the corresponding size served by the backend with the logic I implemented. The scaling factor of 2 was already used for requesting resized external avatars, and likely exists to make sure that users with display scaling enabled get a sharper picture.
| Size requested in the template | After scaling (x2) | Size of the image served |
|---------|---------|---------|
| 256 px | 512 px | original (512 px) |
| 140 px | 280 px | original (512 px) |
| 48 px | 96 px | 128 px |
| 40 px | 80 px | 128 px |
| 32 px | 64 px | 64 px |
| 28 px | 56 px | 64 px |
| 24 px | 48 px | 64 px |
| 20 px | 40 px | 64 px |
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11242
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
The bleve indexer included a fast path to consider a single token to be of MUST rather than should.
However, the condition missed an additional check and would erroneosly include a NOT as a MUST.
This was not spotted by the tests as such exclude queries were usually made along with another term to avoid noise.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12589
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
This change renders file links in org-mode like `./module.el::20` as a link to the 20th
line, for example. It also strips off other search types that are not currently supported
in forgejo like regex search to avoid generating invalid URLs.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12496
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Include `run_id` in the responses emitted by all `...actions/runners/jobs` endpoints. Helps with correlating pending jobs with other jobs and the runs they belong to.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12480
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
This pr PR is fixing a type introduced in #10397. In case of an error during the creation of the centralised hooks `git.InitFull` would have returned early, missing some of the configuration steps
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(can be removed for Go changes)
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12427
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Follow-up to https://code.forgejo.org/forgejo/runner/pulls/1509 -- improves the UX in Forgejo when a reusable workflow is skipped, marking the workflow as skipped rather than succeeded.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12412
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Allows us to make use of Go features introduced in v1.26.
I require a feature from v1.26 for a PR I want to make later.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12369
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
This PR extends Forĝejo's PyPI package index to support [the simple JSON repository API](https://packaging.python.org/en/latest/specifications/simple-repository-api/#json-serialization). Since the existing implementation was for the HTML serialization of the same simple API, no new endpoint has been added. Instead, Forĝejo chooses between serialization schemes based on the "Accept" header in the request. This, together with CORS, will make Forĝejo compatible with [micropip](https://github.com/pyodide/micropip).
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
(can be removed for JavaScript changes)
- I added test coverage for Go changes...
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [x] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
*The decision if the pull request will be shown in the release notes is up to the mergers / release team.*
The content of the `release-notes/<pull request number>.md` file will serve as the basis for the release notes. If the file does not exist, the title of the pull request will be used instead.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/12095): <!--number 12095 --><!--line 0 --><!--description SG9zdGVkIFB5UEkgcGFja2FnZXMgbWF5IGJlIGFjY2Vzc2VkIHZpYSB0aGUgW3NpbXBsZSBKU09OIEFQSV0oaHR0cHM6Ly9wYWNrYWdpbmcucHl0aG9uLm9yZy9lbi9sYXRlc3Qvc3BlY2lmaWNhdGlvbnMvc2ltcGxlLXJlcG9zaXRvcnktYXBpLyNqc29uLXNlcmlhbGl6YXRpb24pIGluIGFkZGl0aW9uIHRvIHRoZSBzaW1wbGUgSFRNTCBBUEkgYWxyZWFkeSBhdmFpbGFibGUu-->Hosted PyPI packages may be accessed via the [simple JSON API](https://packaging.python.org/en/latest/specifications/simple-repository-api/#json-serialization) in addition to the simple HTML API already available.<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12095
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
When attempting to render a markdown code block that does not have a language set in it, Forgejo will fail to render and log an error:
```
2026/04/29 08:47:47 ...markdown/markdown.go:162:func1() [W] Unable to render markdown due to panic in goldmark: runtime error: invalid memory address or nil pointer dereference
```
This is a regression introduced by #12056.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
- pre-release regression
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12325
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Enhances authorized integrations (#12261) with a cache of the remote OpenID Connect descriptor file and JSON Web Key Set (JWKS), improving runtime performance and reducing intermittent reliability risks. By default a 10 minute cache is used, configurable through `[authorized_integration].CACHE_TTL`.
To mock the cache for testing, mockery code generation is added, and a previous manually generated mock for `AuthorizationReducer` was replaced with the code generation.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
- Authorized integrations are not yet exposed to end-users.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12275
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
This change introduces a Path method on the TreeEntry struct, that
collects the path by moving upwards in the tree.
The existing FollowSymlink(s) methods interface has been changed, the
previously returned string has been removed, as after the fix it wasn't
used anywhere.
Fixes: #9931
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12246
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
This PR is replacing repository based hooks hooks with centralised files, this way the files don't need to be copied into every repository, only one line of config need to be added in the repository.
Closes: #3523
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10397
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Authorized Integrations is a new feature to allow users to define external systems which can generate JSON Web Tokens (JWTs) that Forgejo will trust in order to perform API access on behalf of that user. This is an authentication mechanism that requires zero preconfiguration of shared secrets, and instead establishes trust through short-lived secrets (JWTs) that are signed by the issuer, signatures are validated by comparison with published public keys, and a public-keys retrieved through well-known HTTP endpoints secured with TLS verification.
The primary goal of Authorized Integrations is to support a mechanism for Forgejo Actions to receive elevated, but controlled, additional access to Forgejo. More details as to what the end result will look like are available in the [design proposal](https://codeberg.org/forgejo/forgejo/issues/3571#issuecomment-13268004) on #3571.
This PR adds the core database storage and authentication verification for Authorized Integrations, with these capabilities:
- An Authorized Integration is resolved by a unique key of an "issuer" and an "audience". The value of "issuer" is defined by the remote integration, and the value of "audience" will incorporate a unique identifier generated by Forgejo.
- Example issuer: `https://token.actions.githubusercontent.com/` is the issuer for GitHub JWTs
- Example audience: `https://forgejo.example.org/-/mfenniak/authorized-integration/6cc55ba0` is the expected format for a random audience field that Forgejo will generate.
- JWTs can contain any number of claims, which are represented as a JSON object; Forgejo can validate these with a flexible policy.
- eg. a claim may be `{"sub": "repo:coolguy/forgejo-runner-testrepo:pull_request"}` indicating that an OIDC token was received from an Actions execution in a specific repo on a specific event.
- Authorized Integrations support a `ClaimRules` system which allows claim equal, glob, and nested object inspection.
- `{"claim":"sub","comparison":"eq","value":"repo:mfenniak/forgejo-runner-testrepo:pull_request"}` -- would validate that `sub` exactly equals the specific value
- `{"claim":"sub","comparison":"glob","value":"repo:mfenniak/forgejo-runner-testrepo:*"}` -- would validate that `sub` matches the given string prefix but allow any event
- When a JWT is received on an incoming API call, Forgejo retrieves the Authorized Integration from the DB (if present), validates the token signature against a remote JWKS, validates the claims, and grants API access as the user with a permission scope defined on the Authorized Integration.
In addition to the unit testing provided here, this PR has been manually integration tested against three JWT issuing systems: Forgejo Actions, GitHub Actions, and AWS STS GetWebIdentityToken.
Careful consideration has been made of these security concerns:
- SSRF attacks against Forgejo are prevented by:
- having a blocklist on remote HTTP validation requests which prevent access to internal network resources,
- ensuring that authorized integrations are created by users with matching issuers, before attempting to validate tokens
- Resource utilization attacks against Forgejo are reduced by limiting the possible size of external metadata requests; when fetching `/.well-known/openid-configuration` and `jkws_uri`'s from remote, untrusted servers, a maximum response size of 16 kB is enforced
- Only well-known secure assymmetric JWT signing algorithms are supported -- in particular, the sketchy `none` JWT algorithm isn't supported.
- JWT validation is covered by extensive unit tests, covering validation of all JWT timestamps, validation of the issuers, validation of the issuer's documented supported signing algorithms.
This PR serves as a core, and many enhancements are required for this to be a usable system for users.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [ ] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- Documentation updates for new config entries will be authored.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [x] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
- Marking not visible as there's no mechanism to interact with this backend yet.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12261
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
## Context
the three commits in this series are the first step towards the goal of removing the special casing around `JWT_SECRET`, which is used for various modules via `GetGeneralTokenSigningSecret()`. Ultimately, I want to work towards enabling seamless migration away from general use of the common secret. To enable this, we need proper secret/key rotation support, that is, we need to allow for configuration of additional secrets/keys which are accepted for token validation, but not used to issue tokens.
I have this _Verifier_ support basically implemented, but this PR is not it.
This PR contains cleanup refactoring which I worked on before writing the _Verifier_ support, because I noticed that the existing secret/key handling across modules was inconsistent and required duplicated code.
I am submitting this part now to allow for incremental review of not too large a diff, and because these commits remained unchanged during two weeks since I moved on the the next task.
## The problem being addressed
Configuration of JWT signing secrets/keys was inconsistent:
Under `[oauth2]` the full configuration set was supported:
- `JWT_SIGNING_ALGORITHM` configured the algorithm
- `JWT_SECRET` configured a literal secret for symmetric algorithms
- `JWT_SECRET_URI` configured a `file:` uri of a secret for symmetric algorithms
- `JWT_SIGNING_PRIVATE_KEY_FILE` configured a file for asymmetric algorithms
For `[server]`, the LFS module only supported `LFS_JWT_SECRET`, and the signing method was hardcoded to `HS256`
For `[actions]`, only asymmetric signing methods were supported via `ID_TOKEN_SIGNING_ALGORITHM` and `ID_TOKEN_SIGNING_PRIVATE_KEY_FILE`.
## ini unification
The proposed code centralizes ini parsing to always support the following ini keys:
- `[pfx]SIGNING_ALGORITHM` determines the algorithm
- `[pfx]SECRET` is a literal secret for symmetric algorithms
- `[pfx]SECRET_URI` is the uri of a secret for symmetric algorithms
- `[pfx]SIGNING_PRIVATE_KEY_FILE` is a file with a private key for asymmetric algorithms
`[pfx]` is specific to the module and chosen to support the existing ini keys
Centralizing this code and unifying the ini keys will come handy for at least the following reasons:
- consistent behavior across modules is easier to understand
- less duplicated code
- easier to expand later, which is my main motivation
## implementation notes
as might be apparent by the _take3_ branch name, this is the third iteration of this patch series. The main reason why I abandoned the other two is that I first tried to move all the key initialization into the code called from settings.go when the ini file is parsed. But that lead to a lot of friction with test cases, because private key files which are configured, but do not exist will get created and hence require a writable `AppDataPath` and additional clean up.
To avoid a lot of noise and complications in test cases, I kept the existing two stage process, where
- the settings component creates missing symmetric signing keys and writes them to the .ini
- the settings component creates a simple configuration struct
- which is then used from the module init to create the actual key, which also includes creating a private key file if asymmetric crypto is configured and the key file does not exist.
I would have wished this patch was a net negative in terms of LOCs, but I hope it contributes to clarity and many added lines are in test cases.
## Commits
Because sometimes PRs are merged as squashes with the PR text remaining, I am repeating here the individual messages of the individual commits for future reference:
### Refactor signing key initalization and oauth2 use of it
This commit is the first in a series towards the goal of addressing the
FIXME comment in modules/setting/oauth2.go to remove
GeneralTokenSigningSecret
To do it properly, the task also requires addition of signing secret/key
rotation: We ultimately want to be able to change a signing key, but
continue to accept the previous one. This is particularly relevant to
offer a path from GeneralTokenSigningSecret aka JWT_SECRET to new,
specific component key configuration, where it should be possible to add
the former JWT_SECRET as a key accepted for verification to enable a
seamless transition.
This perspective, in turn, calls for refactoring of the existing secret
initialization code to centralize the common functions of parsing
signing key related configuration directives: The oauth2 module
currently is the only component accepting symmetric and asymmetric keys,
with the limitation of the symmetric key being also the
GeneralTokenSigningSecret. Other components either enforce HS256 or
public key algorithms.
We should really give the choice of algorithm selection and avoid code
duplication in other places, so this commit
- generalizes setting parsing into a configuration struct: A prefix can
be provided, with which the common configuration directives are
processed:
- [pfx]SIGNING_ALGORITHM determines the algorithm
- [pfx]SECRET is a literal secret for symmetric algorithms
- [pfx]SECRET_URI is the uri of a secret for symmetric algorithms
- [pfx]SIGNING_PRIVATE_KEY_FILE is a file with a private key for asymmetric algorithms
- which is then accepted by jwtx.InitSigningKey() to create an actual
signing key
The reasons for the two stage process are explained in a long-ish
comment in modules/setting/security.go. In short, other options would
either violate sensible module boundaries or cause too much friction.
These other options have actually been tried, this is take 3 of the
proposed changes.
### Refactor services/lfs: Change token code to use SigningKey
This now also enables use of token algorithms other than HS256.
In this case, signing key initialization also happens during settings
initialization, because LFS is also used in CLI commands.
### Refactor api/actions to use new signingkey API
This now also enables use of symmetric token algorithms.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11194
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Nils Goroll <nils.goroll@uplex.de>
Co-committed-by: Nils Goroll <nils.goroll@uplex.de>
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Tests for JavaScript changes
(not applicable — Go-only change)
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
## Summary
Add public REST API endpoints under `/api/v1/` for listing, inspecting, downloading, and deleting Actions artifacts. Previously, artifacts could only be accessed through the web UI or the internal runner API.
### New endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/repos/{owner}/{repo}/actions/artifacts` | List all artifacts for a repository |
| `GET` | `/repos/{owner}/{repo}/actions/runs/{run_id}/artifacts` | List artifacts for a workflow run |
| `GET` | `/repos/{owner}/{repo}/actions/artifacts/{artifact_id}` | Get artifact metadata |
| `GET` | `/repos/{owner}/{repo}/actions/artifacts/{artifact_id}/zip` | Download artifact as zip |
| `DELETE` | `/repos/{owner}/{repo}/actions/artifacts/{artifact_id}` | Delete an artifact |
- List endpoints support `page`, `limit`, and `name` query parameters
- Both v1-v3 (multi-file, zip on-the-fly) and v4 (single zip) artifact backends are supported
- Expired artifacts are listed with `expired: true` but cannot be downloaded
- Delete requires write permission; all other endpoints require read permission
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12140
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: ShellWen <me@shellwen.com>
Co-committed-by: ShellWen <me@shellwen.com>
Followup to https://codeberg.org/forgejo/forgejo/pulls/6203
Currently it is logging an error wherever a template is rendered in language that doesn't have all plural strings covered. For example, Esperanto isn't well maintained.
Since more plural strings were migrated in v15 to new format, these errors became much more common. However, for all languages but the base one (English) they are completely harmless and just indicate an incomplete translation.
However, for base (English) they indicate a bug in either template or en-US.json, which should be still logged as an error.
The error is being logged by `LookupPluralByForm`, which is called by `TrPluralStringAllForms` and (`TrPluralString` through `LookupPluralByCount`). I originally intended to just pass log func directly to `LookupPluralByForm` from both, but since `TrPluralString` isn't calling `LookupPluralByForm` directly, it didn't look clean, so I went with passing a flag around instead and implemented logging logic in `LookupPluralByForm` itself.
I little concern is with that the so-called "default lang" is configurable, and if it is configured to something with less than 100% completion, it will cause fallback bugs, as well as a lot of logging of this as an error. But this is why changing "default lang" is a bad idea in the first place, and broken fallbacks should be greater concern than junk in the logs.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12183
Reviewed-by: Beowulf <beowulf@beocode.eu>
Co-authored-by: 0ko <0ko@noreply.codeberg.org>
Co-committed-by: 0ko <0ko@noreply.codeberg.org>
- The documentation has the correct behavior about `linguist-detectable`: In cases where a file should be considered for language statistics, regardless of its category, the linguist-detectable attribute can be used.
- This patch follows that behavior by not skipping the file even if some heuristic would've said to skip the file.
- Document the conditions in more natural language.
- Resolvesforgejo/forgejo#11248
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11685
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
- It's quite hard to determine when and why this was added here, my best
guess is that this being the "oldest" subcommand at some point loading
the configuration was not unified. Now it is unified in
`prepareWorkPathAndCustomConf` which is run before any subcommand is
run. It determines the work path, custom path and (custom) config and
then loads the settings by calling `LoadCommonSettings`.
- Between `prepareWorkPathAndCustomConf` being called and
`serveInstalled` being called the `setting.CustomConf` is not changed.
There was a possibility this being necessary for install page ->
installed, but the install code already ensures that the new config is
loaded and used.
- Thus calling to load the settings again here is not necessary. There's
a small possibility some settings loading code was written to only work
after being loaded the second time. That's a bug that needs to be fixed,
because all other subcommands does not load the settings twice and would
see a different view of the settings in that case. I don't fear such
code being present here.
- Resolvesforgejo/forgejo#11024
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12111
Reviewed-by: Mathieu Fenniak <mfenniak@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
First round of patches to re-enable some lints from my side.
This PR also refactors the general key fetching code quite a bit due to the way it currently worked
with relying on some values being nil sometimes.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11253
Reviewed-by: elle <0xllx0@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: famfo <famfo@famfo.xyz>
Co-committed-by: famfo <famfo@famfo.xyz>
With the completion of #12015, when a comment is left on a changed line in a pull request, we track the comment against the line of code with `git blame` and then identify where it currently is in any diff with `git blame --reverse`. However, this strategy only works for the *modified* lines of code -- eg. the `+...` in diffs, and not the `-...` in diffs. The reason is that `git blame --reverse` can't track a line of code's location past the commit that it was removed in.
To permit comments that are left on lines of code that are removed to appear correctly in the UI, a separate approach is required for those comments. This PR performs two major changes, which have been complex to figure out, but are reasonably easy to understand:
- When a comment is placed on a removed line in a PR, perform a `git blame --reverse` from the PR's base to the currently viewed commit, and use this information to record in the comment:
- the **last commit that the line of code existed in** (stored in the `commit_sha` field)
- the **line of code as of that commit** (stored in the `line` field, negative, to indicate that the comment is on a removal).
- the **patch** where the comment was placed (stored in the field `patch`); existing functionality unchanged in this PR
- When viewing any diff in the PR, for each comment on a removal, perform a diff from the `commit_sha` (last commit that the line of code existed in) to the current commit being viewed, and verify that within that diff the left-hand-side line removal still exists at the same line of code in the diff, by comparing the current diff with the stored patch.
- If present, place the commit in the UI at the line number.
- If the line of code no longer exists in the diff at that point (for example, it was removed, commented upon, and then re-added in a later commit), then the comment is considered outdated and isn't displayed.
The algorithm used for marking a comment as "outdated" is also updated to use this approach.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [x] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12092
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Currently, forgejo does not support syntax highlighting code-blocks that have comma separated attributes after the language. This is a pattern sometimes seen in Rust code blocks, with tests like this:
\`\`\`rust
#[test]
fn run_this_test() { /* ... */ }
\`\`\`
\`\`\`rust,ignore
#[test]
fn skip_this_test() { /* ... */ }
\`\`\`
Currently, forgejo only does syntax highlighting in the first case:
```rust
#[test]
fn run_this_test() { /* ... */ }
```
```rust,ignore
#[test]
fn skip_this_test() { /* ... */ }
```
An example of this causing problems can be seen in this commit (5be9c5b7d2) causing the following issue (https://codeberg.org/zesterer/ariadne/issues/188).
This PR fixes fixes the second case not getting proper syntax highlighting.
Co-authored-by: TurtleArmy <44322335+TurtleArmyMc@users.noreply.github.com>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12056
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Ellen Εμίλια Άννα Zscheile <fogti@noreply.codeberg.org>
Co-authored-by: TurtleArmy <turtlearmy@noreply.codeberg.org>
Co-committed-by: TurtleArmy <turtlearmy@noreply.codeberg.org>
This is hopefully the final part of PR #4767, rebased and squashed.
More thorough federation tests are at https://code.forgejo.org/forgejo/end-to-end/pulls/1276 but the mock has been extended to hopefully cover a good chunk as well.
Co-authored-by: Gergely Nagy <forgejo@gergo.csillger.hu>
Co-authored-by: Michael Jerger <michael.jerger@meissa-gmbh.de>
Co-authored-by: zam <mirco.zachmann@meissa.de>
Co-authored-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/10380
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: famfo <famfo@famfo.xyz>
Co-committed-by: famfo <famfo@famfo.xyz>
When a review comment is placed on a PR in Forgejo, Forgejo performs a `git blame` to identify which commit originated the line, and records that commit and line number in the comment's database record. Later when the review is viewed, Forgejo currently makes no effort to place that comment in the correct *current* location, which may vary -- for example, if a PR had two commits and the comment was made on a line in the first commit, but the second commit changes line numbers in that file, the comment will appear in the incorrect location.
This PR adds the usage of `git blame --reverse` to calculate the correct location to display the comment in the current view (whether reviewing the PR commit-by-commit, or "Files changed"). It certainly does not fix all problems with comment placement (see comments).
Another major addition in this PR is a test harness for making relatively complex PRs and reviewing the diffs on the per-commit view and PR-diff views.
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. All work and communication must conform to Forgejo's [AI Agreement](https://codeberg.org/forgejo/governance/src/branch/main/AIAgreement.md). There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests for Go changes
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I ran...
- [ ] `make pr-go` before pushing
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [x] This change will be noticed by a Forgejo user or admin (feature, bug fix, performance, etc.). I suggest to include a release note for this change.
- [ ] This change is not visible to a Forgejo user or admin (refactor, dependency upgrade, etc.). I think there is no need to add a release note for this change.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/12015): <!--number 12015 --><!--line 0 --><!--description cmVsb2NhdGUgUFIgcmV2aWV3IGNvbW1lbnRzIHVzaW5nIGBnaXQgYmxhbWUgLS1yZXZlcnNlYCwgaW1wcm92aW5nIGNvbW1lbnQgcGxhY2VtZW50-->relocate PR review comments using `git blame --reverse`, improving comment placement<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/12015
Reviewed-by: Andreas Ahlenstorf <aahlenst@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
Fixes#9282
Adds a new admin panel category for federation related administration.
Includes views for:
- Instance Federation Configuration
- List of Federation Hosts
- (Per-Instance) List of Federated Users
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/11115
Reviewed-by: elle <0xllx0@noreply.codeberg.org>
Reviewed-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Florian Pallas <mail@fpallas.com>
Co-committed-by: Florian Pallas <mail@fpallas.com>
