PR created ..
loop @kevin @humaton

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/b3aa3487bd914382878130b3a3cc94bc

• fi-ansible-lint-diff : SUCCESS in 5m 19s
• fi-yamllint-diff : SUCCESS in 3m 03s

Ok, this might not just work like this. I think you will have to change the receiving logic a little bit. The API endpoint of PDC and Bodhi are not 1:1 compatible. What you want to do is check the original query output and modify the receiving part of the Bodhi API so it works the same as with the PDC.

Sorry for the confusion, this is part of a bigger initiative to remove PDC and was originally meant to be picked by some of the initiative members.

But since you picked it I am happy to help you get this done.

@humaton / @seddik whats the status here? Need to adjust where? here? bodhi?

As @humaton said we can try like that.
@kevin i will focus on this week, but not sure if i have right privileges to run the script ... so what do you think ?

Yeah, I am not sure we have any good way to test this. ;( Perhaps @humaton could try in staging?

@humaton i will be available if we need to synchronize ;)

@humaton you have a chance to help move this one forward? :)

Yeah, we are just discussing it in our PDC removal sync. This is one of the last removals we are missing. @seddik are you able to sync with me or @lenkaseg in the upcoming days? Lenka showed interest in pushing this forward.

There is more work to be done here than just replacing the URLs. Please let me or Lenka know how can we assist or if you are no longer interested Lenka could take over this work.

Not sure what has changed here.

Active releases are not only those with state=current, but also state=pending and state=frozen

As humaton mentions earlier, the output of pdc API is different than what you get from bodhi.
See what kind of output you get with curl -X GET -s -H 'Accept: application/json' 'https://bodhi.fedoraproject.org/releases/?state=current' | jq. The script needs to be adjusted to that.

Here's an example on retrieving all active branches from bodhi in a loop and parsing the output: https://pagure.io/releng/blob/main/f/scripts/get_retired_packages.sh#_12

Active releases are not only those with state=current, but also state=pending and state=frozen

FYI, to get all active releases you can use https://bodhi.fedoraproject.org/releases/?state=current&state=pending&state=frozen.

Also, https://bodhi.fedoraproject.org/releases/?exclude_archived=true achieves the same result, but just because we never used the disabled state on any release...

@lenkaseg I've started to look about migrating roles/bodhi2/backend/templates/owner-sync-pagure.j2, but it's not just a matter of changing URL and adjust output parsing...
The script downloads the full list of active projects for each namespace from src.fp.o, then queries PDC to retrieve active branches on each project. However, Bodhi doesn't know what branch is active per project, all it knows is what release is active and its branch name.

So, I think the workflow should be modifed to:

• download the full list of active projects from src.fp.o for each namespace
• query again src.fp.o to list all branches for each active project
• query bodhi for active releases/branch name
• merge results to get the mapping between project and its respective active branches

Is there any other better solution that I don't see?

EDIT: at a later thought, the workflow I described is also wrong. A package can be retired in a branch, but the branch still exists in src.fp.o...

@mattia Could we get this information from mdapi instead?

@mattia Could we get this information from mdapi instead?

From a quick look mdapi doesn't seem to do what we need.

But maybe we can roll back to the steps I wrote in the comment above adding a check directly in Koji to avoid touching package ownership if the package is blocked in that tag (which would mean it was retired in that branch). It's just an idea, though.

Perhaps the 'extras' dumps we have pagure do could help here?

https://src.fedoraproject.org/extras/

but... do we really need that script? ;-)
Just checked and roles/bodhi2/backend/templates/owner-sync-pagure.j2 seems only referenced by roles/bodhi2/backend/files/koji_sync_listener.py which is "a glue script to run /usr/local/bin/owner-sync-pagure on a given package anytime a ticket gets closed at https://pagure.io/releng/fedora-scm-requests".

So, a mass synch of all active packages is never performed, at least in an automated way. If this is true, we could move the relevant code in the toddler that process fedora-scm-requests and get all the data from the ticket.

Maybe the other files requiring PDC queries can be dropped or moved to other workflows as well.

So, this is somewhat historical, and we could change it, but... back in the fedmsg days the pattern was to process things on seeing messages and then run peroidically a 'full sync' type thing in case messages were lost or something happened.

With fedora-messaging, we could decide that things are known to be delivered or not, and just not run the 'full sync' part. But that means if messages fail to process for whatever reason, we would have to go manually clean up things, we couldn't count on it just getting fixed the next time the full sync runs.

The reason this job runs on bodhi-backend01 is that it uses the koji profile/auth there to do this work. We could move it to a toddler, but then we have to give that toddler (or all of them) koji admin ability (which we may already do for other ones?)

So, I'm not opposed, but wanted to note the history here.

As I understand, we need the script to set the tag ownership in Koji in two cases:

• a new repository or branch is set up for a package
• an existing package in src.fp.o change "ownership"

I think the first case is simple: we can have the scm_request_processor toddler to do that at the same time it creates the repository or branch. It should be safe to think nothing will be lost: either the toddlers completes its task and close the scm ticket or not.

The second case is a bit trickier... I suppose pagure emits a message when a package get their owner changed, so we can hook on that (the bugzilla assignee gets changed when package maintainer changes, but I'm not sure if it is done by Pagure itself). fedora-messaging should be quite reliable, but even if we loose a message I think volumes are quite low and can be manually processed.

Finally, a side note: I had a quick look at roles/distgit/pagure/templates/pagure-sync-bugzilla.py.j2 and I think we don't need that script anymore as I dob't see it referenced anywhere and the distgit_bugzilla_sync toddler does the same work.

When a main admin changes on a package yeah, there should be a message... bugzilla is updated by a toddler, so in theory this or another toddler could do that too.

Yep. Thats the old script from before there was a toddler.

So, where do we run the changing ownership? In another toddler?

I removed the unused pagure-sync-bugzilla.py.j2 script => PR#1929
Looking at the owner-sync-pagure.j2. Making a toddler sounds good.

So, shall we close this PR now and work on a new toddler?

There is a PR#1957 open removing PDC calls from owner-sync-pagure.