#4719 Integration with builds.sr.ht for CI
Opened 4 years ago by ddevault. Modified a year ago

Requirements:

  • An API documentation page for Pagure with a table of contents, dear god
  • Some kind of OAuth or OAuth substitute, which would allow us to redirect users to Pagure to consent to API access, then return back to us with an exchange token we can use to obtain an API key OOB. Ideally this would allow us to scope our API access to only what we need.
  • API endpoints for configuring webhooks
  • API endpoints for examining git data, such as trees and blobs, for pulling out build manifests
  • API endpoints for setting CI status on commits and pull requests

Some issues we've run into with other integrations:

  • GitLab shipped without support for MR builds because the GitLab API doesn't allow us to update the commit status of foreign commits in a MR. So we're stuck with just building commits for them. Gitlab bug
  • GitLab and GitHub both don't have very fine-grained access controls, so we're forced to ask for more access than we actually need.
  • For GitLab, we have to have SourceHut administration register an account with the GitLab instance and set up an OAuth client manually for every supported GitLab instance. Open to ideas for avoiding this for Pagure.

An API documentation page for Pagure with a table of contents, dear god

We used to have this, I swear, it somehow got lost in a UI change and at this point the entire page needs a refresh.

@fbo Are any of these things stuff you're working on at the moment for the Zuul CI integration?

@ngompa, yes partially:

  • Some kind of OAuth or OAuth substitute ... it looks like something like a Github app where a third party application can be attached to user's repositories. And it makes me think of the issue I've opened: https://pagure.io/pagure/issue/4680 some time ago.
  • API endpoints for configuring webhooks: this is possible via the project config endpoint. We use it here: https://pagure.io/fedora-project-config/blob/master/f/tools/project-settings-helper/helper.py#_86
  • API endpoints for examining git data, such as trees ...: Zuul keeps a copy of the repo locally so it does not rely any specific endpoints to examine the repo.
  • API endpoints for setting CI status on commits and pull requests: Some endpoints exist for that. The Zuul Pagure driver sets and reads the PR CI status via the API of Pagure.

So https://pagure.io/pagure/pull-request/4698 will help third party CI integration when merged. I've nothing else in progress atm.

An API documentation page for Pagure with a table of contents, dear god

done: https://pagure.io/pagure/c/1f2b600f5bbe3eeefe421c6730efd1b94704898e?branch=master

@ddevault You can check out the updated API documentation on the staging instance here: https://stg.pagure.io/api/

Nice! That page is much improved.

@ddevault With the APIs available as described in the documentation in staging, do you see any remaining gaps? The OAuth(ish) flow thing is the only thing that stands out to me.

Thoughts:

  • Yeah, an OAuth(ish) thing is still needed
  • Cloning the repo to inspect its git data isn't an approach I'm interested in. It's orders of magnitude more expensive than just asking what the contents of a specific file are at a specific commit or ref.
  • I'm still not sure I understand how I can configure webhooks programmatically.

Cloning the repo to inspect its git data isn't an approach I'm interested in. It's orders of magnitude more expensive than just asking what the contents of a specific file are at a specific commit or ref.

You should be able to fetch a file by using https://<server>/<path/to/project>/raw/<commitish>/f/<path/to/file>

For example: https://pagure.io/pagure/raw/master/f/.cico.pipeline

How do I list the contents of a directory? This is a hack, it should be part of the API.

Metadata Update from @ngompa:
- Issue set to the milestone: Coming 3 months

3 years ago

@ddevault We now have an API for listing directory contents without requiring a git clone.

You can get a JSON response with a list of contents of a directory by using https://<server>/api/0/<path/to/project>/tree/<commitish>[/f/<path/to/folder>/]

For example: https://pagure.io/api/0/pagure/tree/master

Nice. Should definitely be possible to use that to write a
dispatch.sr.ht patch. I'm on #sr.ht on freenode if anyone wants some
pointers on where to get started with such a patch.

Metadata Update from @wombelix:
- Issue set to the milestone: 5.13 (was: Coming 3 months)

a year ago

Login to comment on this ticket.

Metadata