#8213 fedmsg -> fedora-messaging migration tracker
Opened a month ago by kevin. Modified 9 hours ago

We should track and finish moving everything from fedmsg to fedora-messaging.

We have at least:

releng/scripts/fedmsg-functions.sh (used by branched/rawhide composes)
roles/nagios_server/files/nagios/commands/notify.cfg (used by nagios)

and all the following playbooks still calling fedmsg/base:

playbooks/groups/autocloud-backend.yml:  - fedmsg/base
playbooks/groups/autocloud-web.yml:  - fedmsg/base
playbooks/groups/badges-backend.yml:  - fedmsg/base
playbooks/groups/badges-web.yml:  - fedmsg/base
playbooks/groups/bugyou.yml:  - fedmsg/base
playbooks/groups/bugzilla2fedmsg.yml:  - fedmsg/base
playbooks/groups/busgateway.yml:  - fedmsg/base
playbooks/groups/datagrepper.yml:  - fedmsg/base
playbooks/groups/elections.yml:  - fedmsg/base
playbooks/groups/fas.yml:  - fedmsg/base
playbooks/groups/fedimg.yml:  - fedmsg/base
playbooks/groups/fedocal.yml:  - fedmsg/base
playbooks/groups/freshmaker.yml:  - fedmsg/base
playbooks/groups/github2fedmsg.yml:  - fedmsg/base
playbooks/groups/hubs.yml:  - fedmsg/base
playbooks/groups/kerneltest.yml:   - fedmsg/base
playbooks/groups/koji-hub.yml:  - fedmsg/base
playbooks/groups/koschei-backend.yml:  - fedmsg/base
playbooks/groups/loopabull.yml:    - fedmsg/base
playbooks/groups/mailman.yml:  - fedmsg/base
playbooks/groups/mbs.yml:  - fedmsg/base
playbooks/groups/mdapi.yml:  - fedmsg/base
playbooks/groups/mirrormanager.yml:  - fedmsg/base
playbooks/groups/modernpaste.yml:  - fedmsg/base
playbooks/groups/noc.yml:  - fedmsg/base
playbooks/groups/notifs-backend.yml:  - fedmsg/base
playbooks/groups/notifs-web.yml:  - fedmsg/base
playbooks/groups/nuancier.yml:  - fedmsg/base
playbooks/groups/odcs.yml:  - fedmsg/base
playbooks/groups/odcs.yml:  - role: fedmsg/base
playbooks/groups/openqa.yml:   - { role: fedmsg/base, tags: ['fedmsg_base', 'fedmsg'] }
playbooks/groups/packages.yml:  - fedmsg/base
playbooks/groups/pdc.yml:  - fedmsg/base
playbooks/groups/pdc.yml:  - fedmsg/base
playbooks/groups/people.yml:  - fedmsg/base
playbooks/groups/pkgs.yml:  - fedmsg/base
playbooks/groups/releng-compose.yml:  - fedmsg/base
playbooks/groups/resultsdb.yml:   - { role: fedmsg/base,
playbooks/groups/retrace.yml:# - fedmsg/base
playbooks/groups/sundries.yml:  - role: fedmsg/base
playbooks/groups/taskotron.yml:   - { role: fedmsg/base }
playbooks/groups/value.yml:  - fedmsg/base
playbooks/groups/wiki.yml:  - fedmsg/base
playbooks/groups/zanata2fedmsg.yml:  - fedmsg/base
playbooks/hosts/happinesspackets-stg.fedorainfracloud.org.yml:  - fedmsg/base
playbooks/hosts/happinesspackets.fedorainfracloud.org.yml:  - fedmsg/base

Koschei migration from fedmsg to fedora-messaging in production is planned on 2019-09-26. Staging instance already uses fedora-messaging, eg. this message ("username": "amqp-bridge")

@karsten you were going to look at the infrastructure scripts moving right? Could you perhaps focus on/start with planet? It seems to somehow be broken right now, and I can't seem to figure out why. Moving it to fedora-messaging would hopefully fix it and get us a bit further along too at the same time. :)

It runs on people02.fedoraproject.org... the venus-planet package will need adjusting I guess: https://koji.fedoraproject.org/koji/buildinfo?buildID=942792

Or if thats not your cup o tea, we can look for someone else. :)

Also, we need a plan for notifs.

playbooks/groups/resultsdb.yml: - { role: fedmsg/base,

I thought that we already ported this to fedora-messaging.

playbooks/groups/taskotron.yml: - { role: fedmsg/base }

As taskotron is in maintenance mode, I'm not really keen on putting time into porting the one sub-component which still uses fedmsg to fedora-messaging unless there's no other option

playbooks/groups/resultsdb.yml: - { role: fedmsg/base,

I thought that we already ported this to fedora-messaging.

We did, this is likely just a clean up that we didn't do :(

Same for elections, koji-hub and pkgs I think.

Also Ansible callback plugins need to be ported:

callback_plugins/fedmsg_callback.py
callback_plugins/fedmsg_callback2.py

Metadata Update from @cverna:
- Issue tagged with: backlog

18 days ago

Metadata Update from @kevin:
- Issue assigned to karsten

4 days ago

I'm dropping the following from the list:

autocloud retires with f29:

playbooks/groups/autocloud-backend.yml: - fedmsg/base
playbooks/groups/autocloud-web.yml: - fedmsg/base

we are giving away badges:

playbooks/groups/badges-backend.yml: - fedmsg/base
playbooks/groups/badges-web.yml: - fedmsg/base

bugyou never took off/should be retired now.

playbooks/groups/bugyou.yml: - fedmsg/base

elections we are giving away:

playbooks/groups/elections.yml: - fedmsg/base

fedocal we are giving away:

playbooks/groups/fedocal.yml: - fedmsg/base

hubs is gone.

playbooks/groups/hubs.yml: - fedmsg/base

going away:

playbooks/groups/modernpaste.yml: - fedmsg/base
playbooks/groups/nuancier.yml: - fedmsg/base

done:
playbooks/groups/koji-hub.yml: - fedmsg/base
playbooks/groups/pkgs.yml: - fedmsg/base
playbooks/groups/koschei-backend.yml: - fedmsg/base
playbooks/groups/openqa.yml: - { role: fedmsg/base, tags: ['fedmsg_base', 'fedmsg'] }
playbooks/groups/resultsdb.yml: - { role: fedmsg/base,

Unknown????????????

playbooks/groups/fedimg.yml: - fedmsg/base
playbooks/groups/freshmaker.yml: - fedmsg/base

That leaves:

playbooks/groups/bugzilla2fedmsg.yml: - fedmsg/base
playbooks/groups/busgateway.yml: - fedmsg/base
playbooks/groups/datagrepper.yml: - fedmsg/base
playbooks/groups/fas.yml: - fedmsg/base
playbooks/groups/github2fedmsg.yml: - fedmsg/base
playbooks/groups/kerneltest.yml: - fedmsg/base
playbooks/groups/loopabull.yml: - fedmsg/base
playbooks/groups/mailman.yml: - fedmsg/base
playbooks/groups/mbs.yml: - fedmsg/base
playbooks/groups/mdapi.yml: - fedmsg/base
playbooks/groups/mirrormanager.yml: - fedmsg/base
playbooks/groups/noc.yml: - fedmsg/base
playbooks/groups/notifs-backend.yml: - fedmsg/base
playbooks/groups/notifs-web.yml: - fedmsg/base
playbooks/groups/odcs.yml: - fedmsg/base
playbooks/groups/odcs.yml: - role: fedmsg/base
playbooks/groups/packages.yml: - fedmsg/base
playbooks/groups/pdc.yml: - fedmsg/base
playbooks/groups/pdc.yml: - fedmsg/base
playbooks/groups/people.yml: - fedmsg/base
playbooks/groups/releng-compose.yml: - fedmsg/base
playbooks/groups/retrace.yml:# - fedmsg/base
playbooks/groups/sundries.yml: - role: fedmsg/base
playbooks/groups/taskotron.yml: - { role: fedmsg/base }
playbooks/groups/value.yml: - fedmsg/base
playbooks/groups/wiki.yml: - fedmsg/base
playbooks/groups/zanata2fedmsg.yml: - fedmsg/base
ansible callbacks

playbooks/groups/bugzilla2fedmsg.yml: - fedmsg/base

Being ported by @abompard

playbooks/groups/fas.yml: - fedmsg/base

going away (sooner or later :()

playbooks/groups/kerneltest.yml: - fedmsg/base

Being handed over to Jeremy

playbooks/groups/loopabull.yml: - fedmsg/base

Ported to fedora-messaging upstream, we can migrate when we want

playbooks/groups/packages.yml

I think @cverna was saying yesterday that this no longer works, and thus we should be able to remove it.

Is there any sense of what should be prioritized higher in this list? Just curious for some expert opinion @kevin @pingou

There are a few angles and I don't think any good answer to this question:

  • We want to retire fedmsg so all of these should be ported to fedora-messaging
  • We will need to migrate datagrepper, busgateway and FMN from fedmsg to fedora-messaging but both of them require the features that today fedmsg_meta_fedora_infrastructure provides and these features are now handled as part of the message schemas that each application should provide
  • To make the migration from fedmsg faster, we have not added support to message schemas to the application we've migrated, so we'll need to go back to a number of our applications to add these schemas.

Some that we could look at porting today would be:

  • wiki
  • sundries (not quite sure what uses fedmsg there though)
  • retrace
  • odcs
  • mbs
  • people (here as well, not sure what uses fedmsg there)
  • releng-compose
  • noc

Once these are done we'll likely ran into the more complex ones.

Login to comment on this ticket.

Metadata