#5 allow to (or automatically create) wiki matrices before a compose is complete
Closed 3 years ago Opened 3 years ago by kparal.

Because RC compose takes extremely long, we often start testing it before it is finished (once some iso images are available). The problem is that we have nowhere to write the results to, the validation matrices still don't exist. I tried to create the matrices today for a compose that haven't completed yet, but I received an error:

$ relval compose -r 29 -m Beta -c 1.4 -i Fedora-29-20180919.0 
Enter your FAS password: 
Logging in with user name: kparal
ERROR:fedfind.release:Pungi4Release: failed to download metadata!
ERROR:fedfind.release:Pungi4Release: failed to download metadata!
Fedora-29-20180919.0 looks like a production compose, but found no label! Cannot determine event.

It would be great if you could make this work somehow. It would be very helpful to allow us to create the matrices asap.

An alternative idea is to make all of this automatic - adjust the fedmsg listener to create the matrices as soon as an RC compose is started (perhaps just avoid adjusting the Current redirects, in case the compose fails), and finalize the matrices after the compose is finished (add the download links, update the Current redirects, etc). I'm not sure whether this can be done without extending the current fedmsg content, though.


I think this is actually more or less impossible at present, because of exactly the thing the error message says: "found no label". pungi fedmsgs give you the location of the compose and the compose ID. they don't give you the label. fedfind gets the label from the compose's compose/metadata/composeinfo.json, but that is one of the last things written out by the compose, right at the end of the process: until that happens, we really just cannot know what the label of a candidate compose is. (OK, so we could write a stupid thing to find the label of the last compose and try to figure out where we are in the release cycle and figure out what the label is going to be based on that, but...let's not?)

To fix this either composeinfo.json would need to be written out at the start of the compose (not sure if that's possible, or if it includes info that is only available at the end), or the pungi fedmsgs would need to include the label.

There's also a later problem that occurs if we do this. Some stuff we do (I will need more coffee to remember what) requires the opposite operation: given that you know the event is for the compose with the label 'Beta-1.4' or whatever, what is the compose ID? Right now that is done by looking the compose up by label in PDC, but only completed composes are imported to PDC. Failed ones aren't. So this would be impossible for any event we create for a compose that winds up failing. I'll have to remember what actually uses this to know how much of a problem that would be.

oh, I wrote the above while thinking of the automated processes. I didn't think about the manual case you tried. I'll have to look at the code to remember why it tries to find the label via fedfind even though you told it the label via the CLI.

Ah, OK. I remember now. For your CLI case, this should actually have worked:

relval compose -r 29 -m Beta -c 1.4

i.e. DON'T include the compose ID. relval basically throws the release, milestone, compose and compose ID you provide at wikitcms and says 'identify the event that matches these'. wikitcms treats "release/milestone/compose" and "compose ID" as two alternative ways you can ask it to find an event; if you give it both, it will ignore the release/milestone/compose, and look up the event by compose ID. Which fails as explained above, because fedfind can't find the label. If you just leave the compose ID out, you've provided sufficient info to identify a (currently non-existing) event, and wikitcms will just go ahead and return it.

You should've got the event created, with current pages redirected (pass --no-current to avoid that), and a warning that it could not create the download page (as it would not yet have been able to find the images).

The relval compose -h output sort of explains this, although it's a bit outdated:

"You must specify either --milestone and --compose or --url; if --url is specified, any --release, --milestone and --compose setting will be ignored."

Obviously I wrote that before changing the code from accepting a URL to accepting a compose ID :)

OK, thanks for info, I'll try to remember to omit -i in the future. Could you please make this more obvious when executed as I did? Either fail with an error ("You've specified both --cid and --release/--milestone/--compose, which is not supported"), or at least print a warning ("You've specified --cid and some of --release/--milestone/--compose, the latter values will be ignored"). Thanks.

This should be improved a lot in 2.4.0, which I just released and sent out to updates-testing. Let me know if you think it still needs further improvement. Thanks for the report.

Metadata Update from @adamwill:
- Issue status updated to: Closed (was: Open)

3 years ago

I think you broke it:

$ relval compose -i Fedora-29-20181024.1 -w
Guessing release: 29
You specified both --cid and at least one of --release, --milestone and --compose. This is ambiguous and not allowed. Please specify EITHER --cid OR --release, --milestone and --compose.


Metadata Update from @kparal:
- Issue status updated to: Open (was: Closed)

3 years ago

Hah, there's actually a bit that's meant to deal with that, only it's the one bit that I still didn't switch from 'url' to 'cid'...

OK, 2.4.1 should fix that. yell if I broke anything else!

Metadata Update from @adamwill:
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.