#1679 git.centos.org under AI scrapers load
Closed: Fixed a month ago by arrfab. Opened a month ago by arrfab.

While partially mitigated during weekend with some iptables/ipset rules, there is still a negative impact on https://git.centos.org performance
Applied basic mod_qos setting but it needs tuning


Metadata Update from @arrfab:
- Issue assigned to arrfab

a month ago

Metadata Update from @arrfab:
- Issue tagged with: centos-build-pipeline, centos-common-infra, high-gain, high-trouble, investigation

a month ago

mitigated for now but still under some kind of "manageable" load.
Actually a script verifies log and based on some pattern would add the ip in a specific ipset that is used in iptables rules to just drop matching source ip from that ipset list

Keeping ticket open for now as we still need to find a better way to deal with this (so ongoing effort)

closing this one .. let's see how that goes but quite a few IP addresses added in ipset :

ipset list ai-blocked |head -n 7
Name: ai-blocked
Type: hash:ip
Revision: 4
Header: family inet hashsize 262144 maxelem 2000000
Size in memory: 11712272
References: 1
Number of entries: 539859

Metadata Update from @arrfab:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

a month ago

Log in to comment on this ticket.

Metadata