PR#226: Add 'expand_basearch' compose flag. - odcs

odcs

#226 Add 'expand_basearch' compose flag.

Closed 5 years ago by jkaluza. Opened 5 years ago by jkaluza.

jkaluza/odcs expand-basearch into master

Jan Kaluza • 5 years ago

55b0ee7

README.md

file modified

		`@@ -133,6 +133,7 @@`
		- `flags` - List of flags to further modify the compose output:
		- `no_deps` - For `tag` `source_type`, do not resolve dependencies between packages and include only packages listed in the `packages` in the compose. For `module` `source_type`, do not resolve dependencies between modules and include only the requested module in the compose.
		- `include_unpublished_pulp_repos` - For `pulp` `source_type`, include also unpublished repositories for input content-sets.
		+ - `expand_basearch` - For "tag" and "module" source_type, expand the $basearch in .repo file to all requested architectures. The expanded repositories will contain "-$arch" suffix in its name. For example, without this flag, ODCS generates single record in .repo file called for example "odcs-123" with "$basearch" in baseurl. With this flag, ODCS generates multiple records in .repo file called, based on the requested architectures, odcs-123-i686, odcs-123-x86_64, ... with $basearch in baseurl expanded to particular architecture name.
		- `sigkeys` - List of signature keys IDs. Only packages signed by one of these keys will be included in a compose. If there is no signed version of a package, compose will fail. It is also possible to pass an empty-string in a list meaning unsigned packages are allowed. For example if you want to prefer packages signed by key with ID `123` and also allow unsigned packages to appear in a compose, you can do it by setting sigkeys to `["123", ""]`.
		- `results` - List of additional results which will be generated as part of a compose. Valid keys are:
		- `iso` - Generates non-installable ISO files with RPMs from a compose.

common/odcs/common/types.py

file modified

		`@@ -77,6 +77,10 @@`
		`"no_inheritance": 2,`
		`# For "pulp" source_type, include unpublished Pulp repos.`
		`"include_unpublished_pulp_repos": 4,`
		`+ # For "tag" and "module" source_type, expand the $basearch in .repo file`
		`+ # so it can be used to install packages for different architectures`
		`+ # than the $basearch.`
		`+ "expand_basearch": 8,`
		`}`

		`INVERSE_COMPOSE_FLAGS = {v: k for k, v in COMPOSE_FLAGS.items()}`

server/odcs/server/backend.py

file modified

+24 -4

		`@@ -438,9 +438,28 @@`
		`will be generated.`
		`"""`
		`if not data:`
		`- baseurl = os.path.join(`
		`- compose.result_repo_url, "$basearch", "os")`
		`- data = """[%s]`
		`+ # If "expand_basearch" compose flag is used, we need to generate`
		`+ # one repo record in repo file per each compose architecture.`
		`+ # This is needed for example when the compose requester wants to`
		`+ # install both i686 and x86_64 packages no matter what is his`
		`+ # $basearch.`
		`+ if compose.flags & COMPOSE_FLAGS["expand_basearch"]:`
		`+ arches = compose.arches.split(" ")`
		`+ else:`
		`+ arches = ["$basearch"]`
		`+`
		`+ data = ""`
		`+ for arch in arches:`
		`+ # If arches is $basearch, use the real compose.name, otherwise`
		`+ # append the "-$arch" suffix so the repo name is still unique.`
		`+ if arch[0] == "$":`
		`+ name = compose.name`
		`+ else:`
		`+ name = compose.name + "-" + arch`
		`+`
		`+ baseurl = os.path.join(`
		`+ compose.result_repo_url, arch, "os")`
		`+ data += """[%s]`
		`name=ODCS repository for compose %s`
		`baseurl=%s`
		`type=rpm-md`
		`@@ -449,7 +468,8 @@`
		`repo_gpgcheck=0`
		`enabled=1`
		`enabled_metadata=1`
		`- """ % (compose.name, compose.name, baseurl)`
		`+`
		`+ """ % (name, name, baseurl)`

		`# Ensure the directory exists`
		`dirname = os.path.dirname(compose.result_repofile_path)`

server/tests/test_backend.py

file modified

+63 -1

		`@@ -33,7 +33,8 @@`
		`from odcs.server.pungi import PungiSourceType`
		`from odcs.server.backend import (resolve_compose, get_reusable_compose,`
		`generate_compose, generate_pulp_compose,`
		`- generate_pungi_compose, validate_pungi_compose)`
		`+ generate_pungi_compose, validate_pungi_compose,`
		`+ _write_repo_file)`
		`from odcs.server.utils import makedirs`
		`import odcs.server.backend`
		`from .utils import ModelsBaseTest`
		`@@ -730,3 +731,64 @@`
		`db.session.commit()`

		`validate_pungi_compose(self.c)`
		`+`
		`+`
		`+ class TestWriteRepoFile(ModelsBaseTest):`
		`+`
		`+ def setUp(self):`
		`+ super(TestWriteRepoFile, self).setUp()`
		`+`
		`+ self.c = Compose.create(`
		`+ db.session, "me", PungiSourceType.KOJI_TAG, "f26",`
		`+ COMPOSE_RESULTS["repository"], 60, packages='pkg1 pkg2 pkg3',`
		`+ arches="x86_64 i686")`
		`+ db.session.commit()`
		`+`
		`+ def tearDown(self):`
		`+ super(TestWriteRepoFile, self).tearDown()`
		`+ shutil.rmtree(self.c.toplevel_dir)`
		`+`
		`+ def test_write_repo_file_basearch(self):`
		`+ _write_repo_file(self.c)`
		`+ with open(self.c.result_repofile_path, "r") as f:`
		`+ data = f.read()`
		`+`
		`+ self.assertEqual(data, """[odcs-1]`
		`+ name=ODCS repository for compose odcs-1`
		`+ baseurl=http://localhost/odcs/latest-odcs-1-1/compose/Temporary/$basearch/os`
		`+ type=rpm-md`
		`+ skip_if_unavailable=False`
		`+ gpgcheck=0`
		`+ repo_gpgcheck=0`
		`+ enabled=1`
		`+ enabled_metadata=1`
		`+`
		`+ """)`
		`+`
		`+ def test_write_repo_file_expand_basearch(self):`
		`+ self.c.flags = COMPOSE_FLAGS["expand_basearch"]`
		`+ _write_repo_file(self.c)`
		`+ with open(self.c.result_repofile_path, "r") as f:`
		`+ data = f.read()`
		`+`
		`+ self.assertEqual(data, """[odcs-1-x86_64]`
		`+ name=ODCS repository for compose odcs-1-x86_64`
		`+ baseurl=http://localhost/odcs/latest-odcs-1-1/compose/Temporary/x86_64/os`
		`+ type=rpm-md`
		`+ skip_if_unavailable=False`
		`+ gpgcheck=0`
		`+ repo_gpgcheck=0`
		`+ enabled=1`
		`+ enabled_metadata=1`
		`+`
		`+ [odcs-1-i686]`
		`+ name=ODCS repository for compose odcs-1-i686`
		`+ baseurl=http://localhost/odcs/latest-odcs-1-1/compose/Temporary/i686/os`
		`+ type=rpm-md`
		`+ skip_if_unavailable=False`
		`+ gpgcheck=0`
		`+ repo_gpgcheck=0`
		`+ enabled=1`
		`+ enabled_metadata=1`
		`+`
		`+ """)`

jkaluza commented 5 years ago

See the README.md part of this change for the description of expand_basearch.

We need this new flag to generate .repo file which can be used to install packages for both i686 and x86_64 even when you build on x86_64. This is not possible with $basearch, because it would be replaced by x86_64 by DNF and i686 packages would be unavailable.

rebased onto 55b0ee7

5 years ago

cqi commented 5 years ago

jkaluza commented 5 years ago

I'm closing this without merging, we have discussed that futher with OSBS team and this PR wouldn't help him. Instead, we have found out another way to implement this feature. I will submit new PR later.

Pull-Request has been closed by jkaluza

5 years ago