Here's the problem: I am working on a package spec file. I have local uncommitted changes, and I want to test locally, with the container tag, so that my system is insulated possible problems.
I can pass the test subject to ansible, yes. But I have to prepare the container first. To prepare the container, I have to manually build my specfile (with rpmbuild and mock), then have some Dockerfile handy that I use to hard-code in the location of the binary rpm produced in the previous step. Once that container is built, only then can I give it to ansible to run the containerized test for my rpm package.
That's a lot of work! Many people will give us. Others will invent their own bash script and we'll end up with 500 different ways to do this.
If the standard test roles could do this automatically for us, that would be best. @bkabrda seems to have an idea of how it could be done.
I have a poc to solve this by using the TEST_SUBJECTS env variable. For example to test with docker and systemd packages, I'd do that:
export TEST_SUBJECTS="docker:docker.io/library/fedora:27 /home/fred/RPM/RPMS/x86_64/systemd-236-0.1.20171207151953.e97b7b5.fc27.x86_64.rpm /home/fred/RPM/RPMS/x86_64/systemd-libs-236-0.1.20171207151953.e97b7b5.fc27.x86_64.rpm /home/fred/RPM/RPMS/x86_64/systemd-pam-236-0.1.20171207151953.e97b7b5.fc27.x86_64.rpm"
WDYT?
@flepied this sounds reasonable, but I think it solves one of two issues, which is "I want to pass arbitrary RPM file to be tested". The other issue is "I have changes in this local dist-git repo and want to test them". We can solve the second issue with your approach as well, but packagers would still need to do mock build manually, so we should be able to go one step further - do a mock build from a dist-git repo with local changes and pass it automatically. IOW, we should make this as convenient as possible.
@bkabrda my view on the second issue is that is should be solved by fedpkg having a test sub-command which will use the capability to pass rpm names in the TEST_SUBJECTS.
@flepied that's a very good point, it sounds reasonable to me. Thanks!
Here is the corresponding PR https://pagure.io/standard-test-roles/pull-request/109
Please also see "How do I run the standard test interface tests with docker?" thread on ci mailing list:
I wrote a script that runs some commands (pdflatex in particular). If that script exits with 0, I consider it good. If it exits with >0, I consider it bad. I can successfully run the script on my machine to test if my pdflatex works as expected. Now I want to put this into CI, so when a new version of texlive is built, this script runs on a system with the newly built latex. If it fails, somebody needs to be notified. In order to do this, I went trough [1] and I created a bunch of boilerplate in yaml to run the script (which is tedious, but acceptable, I guess). Now I want to verify that my yaml boilerplate works. I want to say: run-this-standard-test-in-docker --image fedora:rawhide --nvr texlive-2017-3.fc29 Yet I struggle to find a way how. Note that the following works for me as well: run-this-standard-test-in-mock --mock fedora-rawhide-x86_64 --nvr texlive-2017-3.fc29 [1] https://fedoraproject.org/wiki/CI/Tests#Wrapping
I wrote a script that runs some commands (pdflatex in particular). If that script exits with 0, I consider it good. If it exits with >0, I consider it bad. I can successfully run the script on my machine to test if my pdflatex works as expected.
Now I want to put this into CI, so when a new version of texlive is built, this script runs on a system with the newly built latex. If it fails, somebody needs to be notified.
In order to do this, I went trough [1] and I created a bunch of boilerplate in yaml to run the script (which is tedious, but acceptable, I guess).
Now I want to verify that my yaml boilerplate works. I want to say:
run-this-standard-test-in-docker --image fedora:rawhide --nvr texlive-2017-3.fc29
Yet I struggle to find a way how.
Note that the following works for me as well:
run-this-standard-test-in-mock --mock fedora-rawhide-x86_64 --nvr texlive-2017-3.fc29
[1] https://fedoraproject.org/wiki/CI/Tests#Wrapping
(Note that instead of Koji nvr, I can provide a built RPM, that works as well.)
Log in to comment on this ticket.