#1542 Fix local build error due to missing yum repodata
Closed 4 years ago by qwan. Opened 4 years ago by qwan.
qwan/fm-orchestrator fix-local-build-error  into  v3

@@ -102,6 +102,9 @@ 

          self.resultsdir = os.path.join(self.tag_dir, "results")

          if not os.path.exists(self.resultsdir):

              os.makedirs(self.resultsdir)

+             # generate repodata even it's an empty repo, otherwise it can

+             # cause yum error for our first build due to missing repodata

+             execute_cmd(["/usr/bin/createrepo_c", self.resultsdir])

  

          # Create "config" sub-directory.

          self.configdir = os.path.join(self.tag_dir, "config")
@@ -115,13 +118,18 @@ 

          # before the first build is done, otherwise we would remove files

          # which we already build in this module build.

          if MockModuleBuilder._build_id == 1:

+             removed_rpms = 0

              # Remove all RPMs from the results directory, but keep old logs.

              for name in os.listdir(self.resultsdir):

                  if name.endswith(".rpm"):

                      os.remove(os.path.join(self.resultsdir, name))

+                     removed_rpms = removed_rpms + 1

  

-             # Remove the old RPM repository from the results directory.

-             if os.path.exists(os.path.join(self.resultsdir, "repodata/repomd.xml")):

+             # Remove the old RPM repository from the results directory. However

+             # if there was no rpms removed, assume we're preparing for the first

+             # build and the resultsdir is empty at this moment.

+             repomd_path = os.path.join(self.resultsdir, "repodata/repomd.xml")

+             if os.path.exists(repomd_path) and removed_rpms > 0:

                  os.remove(os.path.join(self.resultsdir, "repodata/repomd.xml"))

  

              # Remove old config files from config directory.

@@ -73,6 +73,19 @@ 

      'noarch']) in Koji tag `tag` to `repo_dir` and creates repository in that

      directory. Needs config.koji_profile and config.koji_config to be set.

      """

+     # Create the output directory

+     try:

+         os.makedirs(repo_dir)

+     except OSError as exception:

+         if exception.errno != errno.EEXIST:

+             raise

+ 

+     repodata_path = os.path.join(repo_dir, "repodata")

+     # if repodata doesn't exist, run createrepo_c first to generate the repodata dir,

+     # this can prevent yum error if we have no package to download

+     if not os.path.exists(repodata_path):

+         log.info("Creating local repository in %s from empty" % repo_dir)

+         execute_cmd(["/usr/bin/createrepo_c", repo_dir])

  

      # Placed here to avoid py2/py3 conflicts...

      import koji
@@ -124,13 +137,6 @@ 

  

      log.info("Downloading %d packages from Koji tag %s to %s" % (len(download_args), tag, repo_dir))

  

-     # Create the output directory

-     try:

-         os.makedirs(repo_dir)

-     except OSError as exception:

-         if exception.errno != errno.EEXIST:

-             raise

- 

      def _download_file(url_and_dest):

          """

          Download a file in a memory efficient manner
@@ -156,7 +162,6 @@ 

  

      # If we downloaded something, run the createrepo_c.

      if repo_changed:

-         repodata_path = os.path.join(repo_dir, "repodata")

          if os.path.exists(repodata_path):

              shutil.rmtree(repodata_path)

  

  1. When preparing for the first build, the resultsdir is empty, mock
    command will fail due to missing repodata in 'localrepo'.

  2. Some koji tags don't have any package or build at all, this can
    result in an empty local repo created from the tags, and it can cause
    mock failure.

So we create repodata for the empty repo just after it is created.

This doesn't cause build failures in master branch because the mock failures are ignored which I believe is not intentional.

Build #689 failed (commit: a6c7d60).
Rebase or make new commits to rebuild.

@qwan it looks like the v3 branch is not rebased on the following commits:
https://pagure.io/fm-orchestrator/c/be641dee57d189c12c87ad22b5316164fa1db4bc?branch=master
https://pagure.io/fm-orchestrator/c/1160d47e7a965c962dda278a4de81f5ec8026c6d?branch=master

These were fixes added to resolve the local build issues. Could you try rebasing and seeing if those resolve the issue?

@mprahl Thanks for pointing that out, you're right, it requires some effort for rebasing, but I've cherry-picked the two commits you mentioned, and local build works now. I'm going to abandon this.

Pull-Request has been closed by qwan

4 years ago