fedora-badges

#728 Use a much less memory hungry lambda
Merged by misc. Opened by misc.
misc/fedora-badges fix_rules  into  master

Download 728.patch

Since [msg for msg in query.all() if msg.msg['new'] is 1] is
constructing a gigantic list in memory with all decoded json
message, we are hitting memory limit on a regular basis. I suspect
something changed on fedmsg side recently that did start
to make the msg take a lot more memory since fedbadge crash
every 10 to 15 minutes each time a build is made.

Increasing memory is not enough, since that consume more than 20G
(I did test that). So the simplest solution is to not
build a list to count the items, but just count them with sum.

The same pattern should be applied to others rules.

For people who want to see the effect on memory, I did a hotfix on the server, so you can take a look at the graph:

https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=badges-backend01.phx2.fedoraproject.org&plugin=memory&timespan=3600&action=show_selection&ok_button=OK

Look around 9h on the 31 of march 2020.

Ok so I still see some memory error, but much less. I do wonder if that's caused by others queries or something, as they are also reusing the same pattern.

Also related: https://github.com/fedora-infra/fedbadges/issues/71 and https://pagure.io/fedora-badges/pull-request/729

So, those have been in use since 1 week, I am going to merge

Pull-Request has been merged by misc
Metadata
None
No Tags
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.