#10414 FMN notifications are one day late
Closed: Fixed 2 years ago by kevin. Opened 2 years ago by vondruch.

FMN notifications are very delayed. I am now receiving notifications for actions I did yesterday. I see no gap in the notifications, otherwise it might be related to #10370


This is due to the opening of epel9 branches. maintainers are adding epel9 branches and merging in rawhide or whatever and our hook is too simple to squash that into one email and instead sends 1 email per commit.
Those are then all sent to the scm-commits list which multiplies them by the number of susbscribers.

Then, google has decided to throttle emails from us due to this, making it hard to get our of the delay we are in. ;(

In the past (during mass branches), google would only throttle particular users who would get SCM messages or other 'firehose' emails. Google has instead decided to throttle all email for all people from fedoraproject.org because of this.

Metadata Update from @mohanboddu:
- Issue tagged with: medium-gain, medium-trouble, ops

2 years ago

Metadata Update from @mohanboddu:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

2 years ago

well, I think it's the scm-commits people causing the queue to be so large that all the rest of the users get lost in the delay. ;(

Metadata Update from @kevin:
- Issue priority set to: Waiting on External (was: Waiting on Assignee)

2 years ago

Google has instead decided to throttle all email for all people from fedoraproject.org because of this.

LOL, what else I can say?

BTW I assume this was also discussed with RH IT folks. Or should I fire my own ticket?

If you can file an internal ticket and they can do anything to get google to raise our limits that would be great. I don't think there's much chance of that sadly. ;(

BTW I have not received any notification since Dec 9th

It worked for a bit, but there is gap again since Dec 15, 8:34 UTC

I think it's pretty close to caught up now... until the next time something causes too much traffic. ;(

Hopefully we can at least get our python3 re-write in soon so we can debug it when it stops processing.

Thanks for the ticket!

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

This is desperate. It is 18:24 UTC, last notification I have received is 8:58 UTC :/ So is this GMail or something else?

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog