#835 pkgset: Only add missing packages from global tag
Merged 6 years ago by lsedlar. Opened 6 years ago by lsedlar.

@@ -134,7 +134,13 @@ 

  

          return self.rpms_by_arch

  

-     def merge(self, other, primary_arch, arch_list):

+     def merge(self, other, primary_arch, arch_list, unique_name=False):

+         """

+         Merge ``other`` package set into this instance.

+ 

+         With ``unique_name=True`` a package will be added only if there is not

+         a package with the same name already.

+         """

          msg = "Merging package sets for %s: %s" % (primary_arch, arch_list)

          self.log_debug("[BEGIN] %s" % msg)

  
@@ -157,11 +163,15 @@ 

          else:

              exclusivearch_list = None

          for arch in arch_list:

+             known_packages = set(pkg.name for pkg in self.rpms_by_arch.get(arch, []))

              self.rpms_by_arch.setdefault(arch, [])

              for i in other.rpms_by_arch.get(arch, []):

                  if i.file_path in self.file_cache:

                      # TODO: test if it really works

                      continue

+                 if unique_name and i.name in known_packages:

+                     self.log_debug('Not merging in %r' % i)

+                     continue

                  if exclusivearch_list and arch == "noarch":

                      if is_excluded(i, exclusivearch_list, logger=self._logger):

                          continue

@@ -309,7 +309,8 @@ 

              if len(compose_tags) == 1:

                  global_pkgset = pkgset

              else:

-                 global_pkgset.merge(pkgset, None, list(all_arches))

+                 global_pkgset.merge(pkgset, None, list(all_arches),

+                                     unique_name=compose_tag in compose.conf['pkgset_koji_tag'])

          with open(global_pkgset_path, 'wb') as f:

              data = pickle.dumps(global_pkgset)

              f.write(data)

Background story: if a compose is combining modular and traditional compose, the configuration will contain multiple Koji tags to build package set from (one tag for each module, plus at least one tag for the traditional content). However some packages might be present in multiple tags, and if the package set contains both, there's no way to control which one will end up in the compose.

The solution for this is to give preference to the modular compose. If a package with the same name exists in multiple tags, we only take the first one we find. This relies on ordering of collected tags: modular ones are always first, and traditional tags are at the end of the list.

If there are multiple modules that contain the same package, only one of them will be used, which is not correct. Allegedly this should not happen. In any case such use case does not work without this patch either, so we're not losing anything.


The patch does not change anything in a fully traditional compose. Fully modular composes are also not affected.

This is yet another part of #830.

known_packages = set([pkg.name for pkg in self.rpms_by_arch.get(arch, [])])
in this simple situation I would use one liner, but this is OK too.

rebased onto 121ffb4

6 years ago

Pull-Request has been merged by lsedlar

6 years ago