PR#962: Support the modulemd buildopts.rpms.whitelist option - fm-orchestrator

fm-orchestrator

#962 Support the modulemd buildopts.rpms.whitelist option

Merged 5 years ago by mprahl. Opened 5 years ago by mprahl.

custom-whitelist into master

Fix the Docker volume mount for running the tests using Jenkins

mprahl • 5 years ago

ddf248f

Support the modulemd buildopts.rpms.whitelist option

mprahl • 5 years ago

74e4c99

Jenkinsfile

file modified

+1 -1

		`@@ -38,7 +38,7 @@`

		`stage('Run Test Suite') {`
		`timeout(20) {`
		`- onmyduffynode 'cd fm-orchestrator && docker run -v $PWD:/src:Z mbs/test'`
		`+ onmyduffynode 'docker run -v ~/fm-orchestrator:/src:Z mbs/test'`
		`}`
		`}`

module_build_service/builder/KojiModuleBuilder.py

file modified

+2 -1

		`@@ -400,7 +400,8 @@`
		`self.module_build_tag = self._koji_create_tag(`
		`self.tag_name + "-build", self.arches, perm="admin")`

		`- self._koji_whitelist_packages(self.components)`
		`+ self._koji_whitelist_packages(`
		`+ self.mmd.props.buildopts.props.rpm_whitelist or self.components)`

		`# If we have just created the build tag in this buildroot_connect call, block all`
		# the components in `blocked_packages` list. We want to do that just once, because

tests/test_builder/test_koji.py

file modified

+21 -3

		`@@ -413,7 +413,8 @@`
		`assert weights == {"httpd": 1.5, "apr": 1.5}`

		`@pytest.mark.parametrize('blocklist', [False, True])`
		`- def test_buildroot_connect(self, blocklist):`
		`+ @pytest.mark.parametrize('custom_whitelist', [False, True])`
		`+ def test_buildroot_connect(self, custom_whitelist, blocklist):`
		`if blocklist:`
		`mmd = self.module.mmd()`
		`xmd = glib.from_variant_dict(mmd.get_xmd())`
		`@@ -421,6 +422,13 @@`
		`mmd.set_xmd(glib.dict_values(xmd))`
		`self.module.modulemd = mmd.dumps()`

		`+ if custom_whitelist:`
		`+ mmd = self.module.mmd()`
		`+ opts = mmd.get_buildopts()`
		`+ opts.set_rpm_whitelist(['custom1', 'custom2'])`
		`+ mmd.set_buildopts(opts)`
		`+ self.module.modulemd = mmd.dumps()`
		`+`
		`builder = FakeKojiModuleBuilder(`
		`owner=self.module.owner, module=self.module, config=conf, tag_name='module-foo',`
		`components=["nginx"])`
		`@@ -431,8 +439,18 @@`
		`groups['srpm-build'] = set(["fedora-release"])`
		`builder.buildroot_connect(groups)`

		`- expected_calls = [mock.call('module-foo', 'nginx', u'Moe Szyslak'),`
		`- mock.call('module-foo-build', 'nginx', u'Moe Szyslak')]`
		`+ if custom_whitelist:`
		`+ expected_calls = [`
		`+ mock.call('module-foo', 'custom1', 'Moe Szyslak'),`
		`+ mock.call('module-foo', 'custom2', 'Moe Szyslak'),`
		`+ mock.call('module-foo-build', 'custom1', 'Moe Szyslak'),`
		`+ mock.call('module-foo-build', 'custom2', 'Moe Szyslak')`
		`+ ]`
		`+ else:`
		`+ expected_calls = [`
		`+ mock.call('module-foo', 'nginx', 'Moe Szyslak'),`
		`+ mock.call('module-foo-build', 'nginx', 'Moe Szyslak')`
		`+ ]`
		`assert session.packageListAdd.mock_calls == expected_calls`

		`expected_calls = [mock.call('module-foo-build', 'build'),`

mprahl commented 5 years ago

Addresses FACTORY-2611

mprahl commented 5 years ago

Please note we can't deploy this until this is stable:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-87e0c7fb16

ralph commented on line 8 of docker/Dockerfile-tests 5 years ago

Nice. Remove this line once https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-f5640a1613 goes stable?

ralph commented 5 years ago

Nice! Straightforward patch.

ralph commented 5 years ago

:+1: to merge in whatever order you think you can Matt.

You could wait for the libmodulemd update to go stable so you can revert the Dockerfile change, but then we have to wait for the libmodulemd update before even starting the mbs update process.
Is there any way to get a jump on the mbs update process before libmodulemd goes stable so we don't have to serialize those two wait times?

@jwboyer is eager to see this one land to unblock some other folks. :)

ralph commented 5 years ago

FWIW - if we hardcode the rpm NVR url in the dockerfile, our test suite here could count towards a karma +1 on their latest libmodulemd bodhi update. ;)

rebased onto cc568003463f8d47d7506df4f2b053ed72907b27

5 years ago

1 new commit added

Fix the Docker volume mount for running the tests using Jenkins

5 years ago

mprahl commented 5 years ago

@ralph thanks for the review. I updated the libmodulemd version and the tests still pass at:
https://ci.centos.org/job/factory2-fm-orchestrator/job/custom-whitelist/3

I +1'd the Bodhi update. If we don't want to wait for Bodhi to push libmodulemd to stable, we could tag the libmodulemd build so that it goes into the epel7Server-infra repo. If we do that, we can also remove the hard-coded value in the Dockerfile in this PR since we use that repo as part of the tests.

ralph commented 5 years ago

Nice, thank you. :)

ralph commented 5 years ago

@mprahl, time to merge?

ralph commented 5 years ago

(Once merged, no need to wait for any other PRs to land first. Let's get the release machine rolling for this one.)

mprahl commented 5 years ago

It just got submitted to stable earlier today, so we'll have to wait until tomorrow for it to be synced to the mirrors.

rebased onto 74e4c99

5 years ago

2 new commits added