I list this as possible, as the problems with http throttling may be expected or my proposed fix is not allowed for some reason. I did look through several old tickets to see if either of these had been pointed out before but my search may not have been thorough.
I would like to request the addition of https:// to the mirror.stream.centos.org website. In the last month I have tried to debug systems unable to get packages from http://mirror.stream.centos.org/9-stream/ in order to build images. What happens is that an initial connection is allowed and then it is throttled down to very low connections. In reviewing where this happens, it is happening inside of corporate firewalls with strong security stances. Changing the connection to https and to a different website allows for the connection to work quickly. Doing the http:// from outside of their network works at full speed.
My initial guess is that the network has a system which locks down unknown http connections and throttles down the connection to a slow crawl. This is bad for the client and the CentOS servers because now they are dealing with multiple very slow connections. However that guess may not be correct and it could be something that mirror.stream is doing to slow down usage. [If this is the case, I need to update documentation to not use this in Image Builder and similar tools.]
Going from other complaints from other corporate users in IRC, this seems to be a new 'security standard' that is being put in place that unencrypted http is not to be tolerated. If that is the case, then my proposed fix is that https is added to the mirror.stream.centos.org using letsencrypt or similar tooling.
Metadata Update from @zlopez: - Issue tagged with: need-more-info
What is needed for me to help?
So I think the problem is that the projects I have been dealing with have been treating http://mirror.stream.centos.org/9-stream/ like it is something like download.fedoraproject.org/pub/fedora where it will redirect to a mirror and downloads of packages can be done. I believe this is done via mirrormanager which 9-stream is also using so I wonder if a download.stream.centos.org setup can be also used in this case.
There are also other possible answers too.
(follow-up based on internal discussion) : tracked now in proper Jira tracker under ticket CS-1226 so closing it in this one (per policy)
Metadata Update from @arrfab: - Issue close_status updated to: Duplicate - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.