Not possible to tag multiple people as 'assignee', so mentioning @siddharthvipul1 here so that you both can have a look ?

@arrfab is this something thats just started happening or has it always been happening? Is there any evidence of this happening from the staging IPs? I wonder if we see anything 8.43.84.237 from this ip in the logs also?

We did do an upgrade on Monday 25th.

Nothing for .237, so only for 8.43.84.248
I got alert through zabbix that filesystem was getting full, so something changed recently (as node works fine with provisioned vdisk for years now) and between last sunday (when httpd logs are rotated) and today :

grep 8.43.84.248 ssl-id.centos.org_access.log|wc -l
806083

So clearly someting wrong there :-)

@arrfab how often is that log rotated? If its not rotated, can you share the first instance of when this log message began appearing?

id.centos.org that's Ipsilon (https://pagure.io/ipsilon), and its using OIDC.

I spoke with @arrfab yesterday evening, this issue is still ongoing, its only on one of the clusters (production) but it appears its started since the upgrade between 4.6.9 -> 4.6.12 see [0].

Doesn't seem to be anything related as far as I can see in the upgrade, I've asked a question on the CoreOS chat to see if anyone knows or can point me in the right direction [1].

• [0] Release notes 4.6.12: https://docs.openshift.com/container-platform/4.6/release_notes/ocp-4-6-release-notes.html#ocp-4-6-12
• [1] https://coreos.slack.com/archives/CB48XQ4KZ/p1612885177081600

Any news on that issue ? it's still happening and also due to other ipsilon small issue on 8-stream, the log files and files left behind are eating inodes on filesystem, so if this particular issue about number of requests to ipsilon can be solved, that would be great

bug triaging : still no news for that issue ? nothing is broken because of this, so either we can get confirmation that it's now normal behaviour to ask the oauth provider every second about configuration or we just consider that we can close this ticket and not care about it (as long as we have enough temporary disk for logging and rotation in place)

[backlog refinement] : bugs triaging and while initial behavior is still present, there is no performance impact on the oauth idp server itself (ipsilon) and as we rotate the logs and we have a bigger FS (already expanded previously) it's not considered a bug/issue anymore