Since Nov 23, I have not received any notification from FMN. It probably needs similar medicine as in #9932
Metadata Update from @mohanboddu: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-gain, low-trouble, ops
@vondruch are you missing email or IRC notifications?
According to the logs, an email notification was last successfully sent an hour or so ago:
Nov 24 19:12:14 notifs-backend01.iad2.fedoraproject.org twistd[7832]: [2021-11-24 19:12:14][fmn.delivery.backends.mail INFO] Email successfully delivered to vondruch@███████████
I restarted it this morning... it seems to get 'stuck' from time to time. ;(
So, probibly we can close this and hope the python3 port is better?
I restarted it this morning... it seems to get 'stuck' from time to time. ;( So, probibly we can close this and hope the python3 port is better?
Just out of intterst, do you just restart the fedmsg-hub service on notifs-backend? or one of the other services? or the whole notifs-backend machine?
Last email I have received was Tue, 23 Nov 2021 12:47:23 +0000 (GMT) (and interestingly, it says "Notification time stamped 2021-11-22 21:39:04 UTC" so it is more the 12h delayed?) and after reporting this thicket, I have received first email on Wed, 24 Nov 2021 18:35:30 +0000 (GMT) (and again, it says "Notification time stamped 2021-11-23 03:59:27 UTC"). So the requests may get queued somewhere and delayed.
Anyway, since I have reported the ticket, I have received ~28 email notification, so it is probably unstuck.
The last notification I have received is from Wed, 24 Nov 2021 22:30:33 +0000 (GMT) with the body "Notification time stamped 2021-11-24 22:30:22 UTC". And looking at the notifications, they corresponds with the actions I did during a day, so it seem the system catch up.
I restarted it this morning... it seems to get 'stuck' from time to time. ;( So, probibly we can close this and hope the python3 port is better? Just out of intterst, do you just restart the fedmsg-hub service on notifs-backend? or one of the other services? or the whole notifs-backend machine?
I do:
systemctl restart fmn-celerybeat.service fmn-worker@1 fmn-worker@2 fmn-worker@3 fmn-worker@4 fmn-worker@5 fmn-backend@1 fmn-worker@6 fedmsg-hub
Perhaps we could be more targeted if we figured out what part was stuck, but that is the big hammer.
Yeah, it takes a while to catch up. ;(
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.