I noticed that in the if statement, it uses pagure-staging as a tested value for env variable https://pagure.io/fork/amedvede/fedora-infra/ansible/blob/main/f/roles/pagure/templates/fedora-messaging.toml#_12. In other templates only {% if env == "staging" %} is used. Pagure template was in that state even before the last updates. So I not sure now what it can be connected with.

I noticed that in the if statement, it uses pagure-staging as a tested value for env variable https://pagure.io/fork/amedvede/fedora-infra/ansible/blob/main/f/roles/pagure/templates/fedora-messaging.toml#_12. In other templates only {% if env == "staging" %} is used. Pagure template was in that state even before the last updates. So I not sure now what it can be connected with.

We found out that isn't the case. My suspicion is that the fedora-messaging cert for staging pagure just expired.

I thought that could be too, but... it doesn't seem to be.

Ah. I think I see...

@abompard make a new pki.new for staging in ansible-private. I guess this was changes to support the new rabbitmq version.
However we are still installing the ones from the pki dir. ;)

So, @abompard can we rm the old staging/pki dir and replace it with the staging/pki.new dir and re-run playbooks?
I think that might fix this...but I am unsure how any other staging thing is working. ;)

This ticket is fixed, huge thanks to Aurelien for help. I got sick so when I will feel better I will update this ticket with solution for future reference and close it

I'm curious what it was. ;)

The problem was that rabbitmq didn't have pagure.stg user. Because the playbook contained a staging env variable that was missing env_suffix. It's Ansible commit that fixes it: https://pagure.io/fedora-infra/ansible/c/cfedbcd2ae39f742e8422874ca21c2796439f90a?branch=main