#43 Add special case for latest target w/ Atomic ISO, it's not in fedmsg data
Merged 3 years ago by maxamillion. Opened 3 years ago by maxamillion.
maxamillion/fedora-websites add-atomic-latest-target  into  master

@@ -16,6 +16,7 @@ 

  import json

  import logging

  import os

+ import sys

  import socket


  from datetime import datetime, timedelta

@@ -27,6 +28,14 @@ 


  log = logging.getLogger("atomic_vars")


+ try:

+     sys.path.append('../build.d')


+     import globalvar

+ except ImportError:

+     log.error("Unable to import globalvar")

+     sys.exit(1)


  base_url = 'https://apps.fedoraproject.org/datagrepper/raw'

  topic = "org.fedoraproject.prod.releng.atomic.twoweek.complete"


@@ -185,6 +194,20 @@ 

              iso_size_key = iso_size_prefix + mapping[key]

              results['iso_size'][iso_size_key] = str(length)


+     # Special case for Atomic ISO latest redirect rule mapping because it's

+     # not included in fedmsg data

+     atomic_iso_filename = "Fedora-Atomic-dvd-x86_64-{}-{}.iso".format(

+         globalvar.release['curr_id'],

+         results['release'][composedate_prefix + 'atomic_composedate']

+     )

+     results['release']['redir_map']['atomic_iso'] = {}

+     results['release']['redir_map']['atomic_iso']['redirect'] = \

+         globalvar.path['download_atomic'] + "/stable/Fedora-Atomic-" + \

+         globalvar.release['curr_id'] + '-' + \

+         results['release'][composedate_prefix + 'atomic_composedate'] + \

+         "/Atomic/x86_64/iso/{}".format(atomic_iso_filename)

+     results['release']['redir_map']['atomic_iso']['filename'] = atomic_iso_filename


      return results



1 new commit added

  • don't run atomic iso latest special case in make_templates loop, not necessary
3 years ago

This adds another request to datagrepper, looking for atomic images. There should be only two requests, now there are 4. Not sure why, but we should investigate before merging this. The rest looks good to me.

The third call to datagrepper is introduced because of https://pagure.io/fedora-websites/blob/master/f/getfedora.org/build/atomic_vars.py#_180 which I believe is a side effect of adding a new object to results which causes the dogpile cache to become invalid and this pull request adds the fourth which I also believe is because of the initialization of a new object into the results dict. Are you aware of any way to work around this?


3 years ago


3 years ago

Pull-Request has been merged by maxamillion

3 years ago

Oh man, this is kind of awful. I haven't added support for the 'stable' two-week Atomic composes to fedfind yet, or I'd suggest using that, but at least it would be better to parse images.json, not look for magic file name matches.

That file will always exist for a compose that came from Pungi 4 (and which releng hasn't stripped it from, which they don't do for two-week Atomic composes). It gives you nice metadata on the compose's images. The 'same' image should always have the same subvariant, type and format. The type for OSTree installer images is always supposed to be dvd-ostree, but it seems that right now the composes are being built with an old Pungi, or something, because it's currently boot.

anyway, yeah, this kind of thing is why the compose metadata exists and why I wrote fedfind. It gets very fragile very fast if all kinds of things are slapping together filename regexes on the fly and stuff.

Oh, the fix for the image type only landed in upstream pungi a few days ago. Once we get a new build of pungi for Rawhide and F25, the type should be correct.

fedfind actually corrects the type for these images from boot to dvd-ostree, but as I said, it doesn't handle the stable two-week Atomic composes yet. I should add that.