fedpkg

Clone

#439 [RFE] Add support for `fedpkg remote`
Closed: Fixed 2 years ago by onosek. Opened 2 years ago by vondruch.

Closed: Fixed
Reopen Issue

I'd like to propose new command fedpkg remote (or set-remote, add-remote or something along the lines). The purpose of this would be just to provide additional remote for the dist git repo. Let me explain.

Given that I am working on CentOS and I did centpkg co somepackage, I'd like to be able to cherry pick some commit from Fedora. Therefore I often need to:

$ git remote add fedora git://whatevertheur/to/somepacakge
$ git fetch fedora
$ git cherry-pick fedora/rawhide

To make this easier, the call to fedpkg remote would setup such remote, deriving the somepackage name from the directory name (or some other suitable source).

Not sure if this is the right place of if this should go to rpkg, because this would be certainly useful for the whole *pkg family, but I'll try here.

Of cousre there could be aslo variant fedpkg remote somepackage, which would not mind to setup the remote in whatever git repository.

Hi, @vondruch

We looked at your suggestion. What could the solution look like?
Currently, you are executing 3 commands:

$ git remote add fedora git://whatevertheur/to/somepacakge
$ git fetch fedora
$ git cherry-pick fedora/rawhide

in requested scenario we can replace first two commands with

$ fedpkg add-remote <some_package_specification>

The remote needs to be named somehow. Two possibilities are:

$ fedpkg add-remote --remote-name <my_remote_name> <some_package_specification>
$ fedpkg add-remote <some_package_specification> <my_remote_name>

The way is not yet specified. The default value (when argument is missing) might be determined by <some_package_specification>. For example:

fedpkg add-remote ssh://git@pagure.io/rpkg.git

This would name the remote as "rpkg".

<some_package_specification> can be URL or package_name. But there are some limitations:
You can use package_name only within one environment. For example, when executing centpkg and you want to add a package from Fedora as a remote, you have to use a URL.

We are going to implement the feature in rpkg, so it could be used in derived tools like fedpkg, centpkg, ...
Edited 2 years ago by onosek

Minor note: rhpkg already has remote subcommand for pretty similar use case. Maybe some potential for sharing?

Although, that works from the other side (rhpkg is called to add fedora remote, and so rhpkg needs some info about other commands). This proposal is much better in that fedpkg would install fedora remote into another repo.

I don't think that the remote would need any additional parameters by default. Lets have this scenario:

$ centpkg co rubygem-mysql2
Cloning into 'rubygem-mysql2'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 19 (delta 6), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (19/19), 5.42 KiB | 5.42 MiB/s, done.
Resolving deltas: 100% (6/6), done.

$ cd rubygem-mysql2

Now I want to add fedora remote:

$ fedpkg remote

I don't know how fedpkg/centpk works on background, but I assume that at minimum centpkg should be able to know that it works with rubygem-mysql2 package.

And the since the remote command is executed via fedpkg, some sensible default such as fedora would be used (but if it was named fedpkg it would be probably a bit strange, but it would suffice. So similarly, if centpkg remote was executed, it would created cs remote (or centpkg). rhpkg remote would create rhel remote (or rhpkg).

Additionally there could have been the --remote-name <my_remote_name> option to override the default name.

I'd like to point out on this place that the whole idea is to save people from figuring out the git URLs. fedpkg as well as centpkg already know the git URLs.
Edited 2 years ago by vondruch

Some further thoughts.

From my my sql example, I would not mind if this was the initial scenario:

$ git remote -v
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)

i.e. if there was origin as well as cs remotes pointing at the same place right after the initial clone.

And the following would be also interesting scenario:

$ git remote -v
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)

$ cd ..

$ fedpkg co rubygem-mysql2
fatal: destination path 'rubygem-mysql2' already exists and is not an empty directory.
Could not execute clone: Failed to execute command.

# Actually the command above would be successful and the result would be the state below:

$ cd rubygem-mysql2

$ git remote -v
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
cs  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)
fedora  ssh://vondruch@pkgs.fedoraproject.org/rpms/rubygem-mysql2 (fetch)
fedora  ssh://vondruch@pkgs.fedoraproject.org/rpms/rubygem-mysql2 (push)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (fetch)
origin  git+ssh://git@gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git (push)

I.e. the fedpkg co rubygem-mysql2 would actually add remotes to the existing dist-git clone.

Hi, @vondruch thank you a lot for the detailed specification. It is obvious, I understood it differently. Your proposal makes sense.
It was important to note "... idea is to save people from figuring out the git URLs." too.
We will try to implement it in rpkg. We will start with the initial scenario and will see what's next.

@lsedlar thank you for pointing out the rhpkg presence of the "remote" subcommand. I forgot it.

Hey @vondruch, I'm currently and working on this feature. I wanted to just test whether the current approach matches your idea for the feature.

As of this moment, the following workflow occurs (anonymous cloning due to local issue): 

[jkunstle@jkunstle test]$ centpkg clone -a rubygem-mysql2
+++++++++++ Inside local rpkg +++++++++++
+++++++++++ Inside local centpkg +++++++++++
Cloning into 'rubygem-mysql2'...
warning: redirecting to https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git/
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 19 (delta 6), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (19/19), 5.40 KiB | 691.00 KiB/s, done.

[jkunstle@jkunstle test]$ cd rubygem-mysql2/

[jkunstle@jkunstle rubygem-mysql2]$ fedpkg remote add fedora
+++++++++++ Inside local rpkg +++++++++++
Module name: rpms/rubygem-mysql2
warning: no common commits
remote: Enumerating objects: 143, done.
remote: Counting objects: 100% (143/143), done.
remote: Compressing objects: 100% (140/140), done.
remote: Total 143 (delta 55), reused 13 (delta 0), pack-reused 0
Receiving objects: 100% (143/143), 23.48 KiB | 23.48 MiB/s, done.
Resolving deltas: 100% (55/55), done.
From https://src.fedoraproject.org/rpms/rubygem-mysql2
 * [new branch]      f19                         -> fedora/f19
 * [new branch]      f20                         -> fedora/f20
 * [new branch]      f21                         -> fedora/f21
 * [new branch]      f22                         -> fedora/f22
 * [new branch]      f23                         -> fedora/f23
 * [new branch]      f24                         -> fedora/f24
 * [new branch]      f25                         -> fedora/f25
 * [new branch]      f26                         -> fedora/f26
 * [new branch]      f27                         -> fedora/f27
 * [new branch]      f28                         -> fedora/f28
 * [new branch]      f29                         -> fedora/f29
 * [new branch]      f30                         -> fedora/f30
 * [new branch]      f31                         -> fedora/f31
 * [new branch]      f32                         -> fedora/f32
 * [new branch]      f33                         -> fedora/f33
 * [new branch]      f34                         -> fedora/f34
 * [new branch]      main                        -> fedora/main
 * [new branch]      rawhide                     -> fedora/rawhide
 * [new tag]         rubygem-mysql2-0.4.4-1.fc25 -> rubygem-mysql2-0.4.4-1.fc25
 * [new tag]         rubygem-mysql2-0.3.16-1.fc21 -> rubygem-mysql2-0.3.16-1.fc21
 * [new tag]         rubygem-mysql2-0.4.0-1.fc24  -> rubygem-mysql2-0.4.0-1.fc24

[jkunstle@jkunstle rubygem-mysql2]$ git remote -v
fedora  https://src.fedoraproject.org/rpms/rubygem-mysql2.git (fetch)
fedora  https://src.fedoraproject.org/rpms/rubygem-mysql2.git (push)
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (fetch)
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (push)

I would appreciate knowing if this fits your expectation for a feature that would be useful to you.
Edited 2 years ago by jkunstle

Yes, that is certainly good direction.

I wonder what would be the result of $ fedpkg remote, $ fedpkg remote add or $ fedpkg remote add foo

And since you have used anonymous clone, I wonder now, what would be the result of non-anonymous clones. Will there be also something such as -a flag and what would be the default behavior.

Speaking to all of those possibilities, they would fail because (1) "fedpkg remote" and "fedpkg remote add" would require a target, e.g. centos or fedora and (2) "fedpkg remote add foo" would fail because foo is not 'fedora' or 'centos'. However, we can incorporate the functionality that you're implying.

Right now, both "centpkg remote ..." and "fedpkg remote ..." are invoking the same shared function to perform their task. However, I can understand that it would be preferable to have package-specific uses, such as using "fedpkg remote" in a cloned folder if one desired a remote of the fedora upstream of that package. I'll keep you in the loop.

The following is the workflow that makes sense to me at this moment given the feedback that you've given so far. What do you think? I made the decision that "x-pkg remote" ought to not overload the way that one would use "git remote" or "git remote -v", and would still display the extant remotes.

However, I did add the ability to simply add the upstream remote from the desired package site without specifying the name.

[jkunstle@jkunstle test]$ centpkg clone -a rubygem-mysql2
+++++++++++ Inside local rpkg +++++++++++
+++++++++++ Inside local centpkg +++++++++++
Cloning into 'rubygem-mysql2'...
warning: redirecting to https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2.git/
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 19 (delta 6), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (19/19), 5.40 KiB | 789.00 KiB/s, done.

[jkunstle@jkunstle test]$ cd rubygem-mysql2/

[jkunstle@jkunstle rubygem-mysql2]$ centpkg remote
+++++++++++ Inside local rpkg +++++++++++
+++++++++++ Inside local centpkg +++++++++++
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (fetch)
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (push)

[jkunstle@jkunstle rubygem-mysql2]$ fedpkg remote add
+++++++++++ Inside local rpkg +++++++++++
warning: no common commits
remote: Enumerating objects: 143, done.
remote: Counting objects: 100% (143/143), done.
remote: Compressing objects: 100% (140/140), done.
remote: Total 143 (delta 55), reused 13 (delta 0), pack-reused 0
Receiving objects: 100% (143/143), 23.83 KiB | 23.83 MiB/s, done.
Resolving deltas: 100% (55/55), done.
From https://src.fedoraproject.org/rpms/rubygem-mysql2
 * [new branch]      f19                         -> fedpkg_remote_upstream/f19
 * [new branch]      f20                         -> fedpkg_remote_upstream/f20
 * [new branch]      f21                         -> fedpkg_remote_upstream/f21
 * [new branch]      f22                         -> fedpkg_remote_upstream/f22
 * [new branch]      f23                         -> fedpkg_remote_upstream/f23
 * [new branch]      f24                         -> fedpkg_remote_upstream/f24
 * [new branch]      f25                         -> fedpkg_remote_upstream/f25
 * [new branch]      f26                         -> fedpkg_remote_upstream/f26
 * [new branch]      f27                         -> fedpkg_remote_upstream/f27
 * [new branch]      f28                         -> fedpkg_remote_upstream/f28
 * [new branch]      f29                         -> fedpkg_remote_upstream/f29
 * [new branch]      f30                         -> fedpkg_remote_upstream/f30
 * [new branch]      f31                         -> fedpkg_remote_upstream/f31
 * [new branch]      f32                         -> fedpkg_remote_upstream/f32
 * [new branch]      f33                         -> fedpkg_remote_upstream/f33
 * [new branch]      f34                         -> fedpkg_remote_upstream/f34
 * [new branch]      main                        -> fedpkg_remote_upstream/main
 * [new branch]      rawhide                     -> fedpkg_remote_upstream/rawhide
 * [new tag]         rubygem-mysql2-0.4.4-1.fc25 -> rubygem-mysql2-0.4.4-1.fc25
 * [new tag]         rubygem-mysql2-0.3.16-1.fc21 -> rubygem-mysql2-0.3.16-1.fc21
 * [new tag]         rubygem-mysql2-0.4.0-1.fc24  -> rubygem-mysql2-0.4.0-1.fc24

[jkunstle@jkunstle rubygem-mysql2]$ centpkg remote
+++++++++++ Inside local rpkg +++++++++++
+++++++++++ Inside local centpkg +++++++++++
fedpkg_remote_upstream  https://src.fedoraproject.org/rpms/rubygem-mysql2.git (fetch)
fedpkg_remote_upstream  https://src.fedoraproject.org/rpms/rubygem-mysql2.git (push)
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (fetch)
origin  https://gitlab.com/redhat/centos-stream/rpms/rubygem-mysql2 (push)

If one were to use the command as "fedpkg remote add <name>" then the remote takes on that name, but for the sake of this example, and in general, an unnamed remote will have the name "x-pkg_remote_upstream".

Thx a lot. I think the proposal looks reasonable. Two remarks.

  1. What happens in you scenario, if one would use in the centpkg cloned repository centpkg remote add? Will it add the centpkg_remote_upstream remote or not? I assume the former.

  2. x-pkg_remote_upstream could you elaborate a bit about this remote name? It is really wordy to my taste. Why not just x-pkg?

And actually speaking about the branch name, now I realized, that I had been working recently on RHEL9, where my starting point was the rhpkg co repository, where I was adding cs and cs-fork remotes. This is very likely for separate ticket, but it is probably worth considering, how x-pkg fork will be compatible with this functionality and remote naming scheme.

On each account:

  1. If one were to clone a repository using "centpkg co <repo>" then using "centpkg remote add" would add the same repo as remote.

  2. The choice of using a longer upstream name is to be in line with the nomenclature of the action. In a git context, one might say "git remote add upstream <url>". One can always name their remote something more to their taste.

  3. Yes please make a new ticket for the "fork" case if that would be useful to you. It would simply be helpful from an organizational standpoint to have the two issues separate

Please see the following:

https://pagure.io/rpkg/pull-request/560#

Commit 7dc5e1a8 fixes this issue

Metadata Update from @onosek:
- Issue set to the milestone: 1.41
2 years ago

Login to comment on this ticket.

Metadata
None
None
None
None
Related Pull Requests
  • #560 Merged 2 years ago
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.