#815 ostree/utils: Drop timestamps from generated repo names
Closed 6 years ago by lsedlar. Opened 6 years ago by walters.
walters/pungi ostree-no-datestamp  into  master

file modified
+20 -22
@@ -66,19 +66,6 @@ 

      return commitid

  

  

- def _write_repofile(path, name, repo):

-     """Write a .repo file with given data."""

-     with open(path, 'w') as f:

-         f.write("[%s]\n" % name)

-         f.write("name=%s\n" % name)

-         f.write("baseurl=%s\n" % repo['baseurl'])

-         exclude = repo.get('exclude', None)

-         if exclude:

-             f.write("exclude=%s\n" % exclude)

-         gpgcheck = '1' if repo.get('gpgcheck', False) else '0'

-         f.write("gpgcheck=%s\n" % gpgcheck)

- 

- 

  def tweak_treeconf(treeconf, source_repos=None, keep_original_sources=False, update_dict=None):

      """

      Update tree config file by adding new repos, and remove existing repos
@@ -86,23 +73,34 @@ 

      Additionally, other values can be passed to method by 'update_dict' parameter to

      update treefile content.

      """

-     # add this timestamp to repo name to get unique repo filename and repo name

-     # should be safe enough

-     time = datetime.datetime.now().strftime("%Y%m%d%H%M%S")

- 

      treeconf_dir = os.path.dirname(treeconf)

      with open(treeconf, 'r') as f:

          treeconf_content = json.load(f)

  

      # backup the old tree config

-     os.rename(treeconf, '%s.%s.bak' % (treeconf, time))

+     os.rename(treeconf, '{}.bak'.format(treeconf))

  

      repos = []

      if source_repos:

-         for repo in source_repos:

-             name = "%s-%s" % (repo['name'], time)

-             _write_repofile("%s/%s.repo" % (treeconf_dir, name), name, repo)

-             repos.append(name)

+         # Sort to ensure reliable ordering

+         source_repos = sorted(source_repos, key=lambda x: x['name'])

+         # Now, since pungi includes timestamps in the repo names which

+         # currently defeats rpm-ostree's change detection, let's just

+         # use repos named 'repo-<number>'.

+         # https://pagure.io/pungi/issue/811

+         with open("{}/pungi.repo".format(treeconf_dir), 'w') as f:

+             for i,repo in enumerate(source_repos):

+                 name = 'repo-{}'.format(i)

+                 f.write("[%s]\n" % name)

+                 f.write("name=%s\n" % name)

+                 f.write("baseurl=%s\n" % repo['baseurl'])

+                 exclude = repo.get('exclude', None)

+                 if exclude:

+                     f.write("exclude=%s\n" % exclude)

+                 gpgcheck = '1' if repo.get('gpgcheck', False) else '0'

+                 f.write("gpgcheck=%s\n" % gpgcheck)

+ 

+                 repos.append(name)

  

      original_repos = treeconf_content.get('repos', [])

      if keep_original_sources:

Since we drop these files in a separate workdir each time,
there's no need to datestamp them. Doing so is part of the
cause for invalidating's rpm-ostree input change hashing.

Issue: https://pagure.io/pungi/issue/811
Signed-off-by: Colin Walters walters@verbum.org

(Like all my pungi patches, not tested locally)

1 new commit added

  • fixup! ostree/utils: Generate a single pungi.repo file, use repo-<num> IDs
6 years ago

rebased onto 5e0aef3e6b18cdfec954c2629fc1719a9666fe00

6 years ago

rebased onto 5d5c1bb

6 years ago

I have plans to change rpm-ostree to render (possibly by default) information about the source repositories. Today in:

$ rpm-ostree status 
State: idle
Deployments:
  fedora-27:fedora/27/x86_64/workstation
                   Version: 27.43 (2018-01-08 16:52:25)
                BaseCommit: ace7915b6d0bdb594ad8a8f13aecf9f6f4a3fbcbfa35527006b5ce42de2a4389
              GPGSignature: Valid signature by 860E19B0AFA800A1751881A6F55E7430F5282EE4
$ ostree show --raw ace7915b6d0bdb594ad8a8f13aecf9f6f4a3fbcbfa35527006b5ce42de2a4389
commit ace7915b6d0bdb594ad8a8f13aecf9f6f4a3fbcbfa35527006b5ce42de2a4389
({'rpmostree.inputhash': <'8e8e9562d4a9006a008a77dcb8c581602c027ec32a4a6e86eccf0fc41c0d456e'>, 'rpmostree.rpmmd-repos': <[{'id': <'Fedora-27-updates-20180108.2-Everything-20180108161618'>, 'timestamp': <uint64 1515427981>}, {'id': <'https:__kojipkgs.fedoraproject.org_compose_27_latest-Fedora-27_compose_Everything__basearch_os_-20180108161618'>, 'timestamp': <uint64 1509861107>}]>, 'version': <'27.43'>}, [byte 0xf8, 0x4e, 0xba, 0x35, 0x35, 0x48, 0xc6, 0x31, 0xaa, 0xfb, 0xba, 0x04, 0x67, 0xfa, 0x8c, 0xc4, 0xb3, 0x97, 0x93, 0x5d, 0x1e, 0x2e, 0x06, 0x47, 0x03, 0x74, 0x11, 0x4f, 0x3f, 0x05, 0x34, 0xfc], @a(say) [], '', '', uint64 1515430345, [byte 0xac, 0x11, 0x09, 0x81, 0xab, 0x9c, 0xcf, 0x41, 0x4c, 0x00, 0xb3, 0x53, 0xd4, 0xdb, 0x9f, 0x45, 0x51, 0xb2, 0x3f, 0x2a, 0x40, 0x8a, 0x2b, 0xf1, 0x55, 0xd3, 0xa4, 0xff, 0xd8, 0xbe, 0x6f, 0x55], [byte 0x50, 0x77, 0x38, 0x17, 0xe4, 0x51, 0x96, 0x29, 0xfb, 0x06, 0x1c, 0xb3, 0xcf, 0xe4, 0xdd, 0xae, 0x0a, 0x99, 0x6c, 0x12, 0x33, 0x6d, 0x08, 0x70, 0x42, 0x48, 0x1f, 0xbe, 0xab, 0x1a, 0x38, 0x0c])
Found 1 signature:

  Signature made Mon 08 Jan 2018 11:52:34 AM EST using RSA key ID F55E7430F5282EE4
  Good signature from "Fedora 27 <fedora-27@fedoraproject.org>"

In that dump you can see that the id of the repos pungi provides are...let's say "verbose":

'id': <'https:__kojipkgs.fedoraproject.org_compose_27_latest-Fedora-27_compose_Everything__basearch_os_-20180108161618' and 'id': <'Fedora-27-updates-20180108.2-Everything-20180108161618'.

Obviously my PR here makes things shorter but also more obscure. I don't know offhand if there is more useful metadata in pungi today to inject; something a bit closer to what users expect on the client side today (fedora and fedora-updates). See also https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-348540604 and https://github.com/projectatomic/rpm-ostree/issues/1081#issuecomment-349178929

This PR could be possibly closed. I moved code into
https://pagure.io/pungi/pull-request/828
and did some test modifications.

Pull-Request has been closed by lsedlar

6 years ago
Metadata