#102 RFE: separate scripts for import providers (tito, mockscm, rubygems,...)
Closed: MIGRATED by nikromen. Opened by praiskup.

** reproducibility, testability, reusability **

I feel there are repeated questions/bugreports for the Copr's mock scm feature; the problem is that there's overly too much code implemented in copr base code while it should be separate script or maybe project; what do you think?

Looking at the web-ui page, there are fields 'SCM Type', 'SCM url', 'Spec file', 'Git branch'. What if we had script 'foo' taking those four arguments doing the (a) initial git-clone; (b) downloading and/or (c) generating the sources?

Having this you could reproduce the issues locally; the final "import" into distgit would be just a matter of rpkg/pyrpkg addition (I'll create new ticket for this).


Having this you could reproduce the issues locally; the final "import" into distgit would be just a matter of rpkg/pyrpkg addition (I'll create new ticket for this).

Here is the pyrpkg ticket:
https://pagure.io/rpkg/issue/229
(if that was refused or not implemented, we already have rpkg package that could implement this as a thin layer on top of pyrpkg).

This is all just quick braindump; but if we had this implemented elsewhere then in copr -- it would be much easier for the users to debug what's wrong with their spec files without bothering us.

Cloning should be a separate thing really, it should not be a part of a script. It would make its interface too complicated in my opinion.

But after cloning,rpkg import <repo-path> --commit <id> would be probably nice.

Clone & download from lookaside is already implemented in rpkg.

What we need in rpkg now IMO is automated import from valid directory (where spec + patches + sources are available).

Good point is that we need to be able to specify the . I'll update the pyrpkg issue.

Cloning should be a separate thing really, it should not be a part of a
script. It would make its interface too complicated in my opinion.

Reading again, I understand your point now -- but I disagree. This work-flow
would totally make the debugging easier:

$ cd whatever-git-repository
$ copr-import-scm --scm-type ... --repo ... --refspec ... --specfile
... do all the work, or fail deterministic way ...

It is design issue to require users to install whole copr stack to debug such
trivial processes. The other thing is that such copr-import-* scripts
could be reside in completely separate package at least; I'm sure users
would love using it also for other cases than through copr build system..
(I would at least).

Comming from #175: I really miss this feature (it is huge bottleneck that I can not run the copr-rpmbuild without referencing copr build-id, when there's none...).

Comming from #175: I really miss this feature (it is huge bottleneck that I can not run the copr-rpmbuild without referencing copr build-id, when there's none...).

 copr-rpmbuild from-scm --clone-url https://something.org/repo.git

Does this look alright to you?


Also e.g.

copr-rpmbuild from-copr --build-id 123456
copr-rpmbuild from-spec --url https://something.org/x.spec
copr-rpmbuild from-gem --name A_123
copr-rpmbuild from-pypi --name motionpaint

We are basically using (the only) positional argument as selector for the build type (and build command-line options respectively). It's a bit unusual but should be usable and I don't have a better idea. I would like each respective subparser to inherit the global options (like e.g. --srpm) so that we can then write:

copr-rpmbuild from-pypi --name motionpaint --srpm

to build just srpm.

And from-copr should be a default if there is no positional argument.

Are you up to implement it?

P.S.: I guess we can drop the "from-" prefix from all the subcommands (i.e. copr-rpmbuild scm, copr-rpmbuild pypi etc.) and copr-rpmbuild copr may be just copr-rpmbuild by itself as it is now. Implementation of this will be welcome.

@clime @frostyx Commenting on this with regard to allowing copr-rpmbuild to talk directly to the builder without going through frontend in a Docker based instance of COPR. Relevant IRC logs are attached.

COPR-discussion-tigeros.weechatlog

@abkahrs, copr-rpmbuild is tool on "builder", where by "builder" we usually mean dedicated virtual machine for building the package (if you don't do builder tasks on backend directly, e.g.).

The copr-rpmbuild talks to frontend, so it knows where to clone package from (copr-dist-git machine) and download dependency packages from (backend).

https://docs.pagure.org/copr.copr/developer_documentation.html#srpm-url-upload-build-schema

There are two major design problems in actual implementation -- first is that
- copr-rpmbuild reads only build-id which obtains all the info from frontend (copr-rpmbuild should rather take everything on commandline for easier reproducibility) and/or
- that copr-rpmbuild does too much logic, it could just "download the build id" and construct commandline for other tool you would be able to use (that's mostly what I wanted to do in this bug report).

Went quickly through the irc log, and if you need local build for CI purposes -- yes, one of those two points (or both) need to be fixed. Just wanted to point out some architecture overview, hth.
From what I see in the log, @clime plans to support both running copr-rpmbuild both with --build-id and without (with all the options on commandline). Looks good, even though having two tools with two different expected inputs/APIs would sound slightly nicer to me (one tool which is totally out of question for local builds which requires copr stack running, and one which works fine locally and is wrapped by the other one).

Fix for SCM: https://pagure.io/copr/copr/c/ddbc3644e7852484483a6cd18e7bf7381228a645?branch=master.

Metadata Update from @frostyx:
- Issue tagged with: RFE

This issue has been migrated to GitHub: https://github.com/fedora-copr/copr/issues/102

Metadata Update from @nikromen:
- Issue close_status updated to: MIGRATED
- Issue status updated to: Closed (was: Open)

Metadata