The openQA boxes in infra - openqa01, openqa-stg01, qa05, qa06, qa07 - don't get a local mirror from MirrorManager. That is, if I hit e.g. https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-23&arch=x86_64 from any of them, I get some random public mirror at the top of the list - a different one each time.
It would be much better if MirrorManager was configured to give these boxes a mirror within infra. It will make openQA tests run much faster and not generate unnecessary external traffic. Every night when tests are run against boot.iso most of them (all except the ones which explicitly specify a repo in some way, for testing repo configuration) will use a repo provided by MirrorManager, so that's 50-100+ installs running every day, obviously we don't want to be sending all that traffic externally unnecessarily.
Not sure how to handle this. ;(
Right now, those machines go to proxy01/10 for mirrors.fedoraproject.org, so mirrorlist servers would see their 10.5.x.x ip's as the source.
mirrormanager doesn't let you set non routable ips/nets as preferred, since normally no one would be able to reach our mirrorlists with a non routable ip.
So, perhaps we could somehow hard code those non routable nets to return just dl.phx2.fedoraproject.org always (and if someone managed to get a query to one of our mirrorlists it wouldn't work, but then perhaps thats ok).
Adding pingou and adrianr for comment from the mirrormanager side of things here.
Just saw this ticket right now (CC was to my IRC nick which is different from the FAS name).
Concerning mirrormanager doesn't let you set non routable ips/nets as preferred: I just tried it and the web interface at least accepts it. I added 10.120.130.140/32 to my mirror and I will check it in a few hours if that works.
Having a quick look at the code I do not see where we might filter certain netblocks. We probably should add some filters as the old check to prevent 0.0.0.0/0 doesn't seem to exist any more. (I will open a ticket against MM2 for this).
curl -s 'https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-23&arch=x86_64&ip=10.120.130.140' | head
So, this actually works. We can add the 10.5.x.x network to the main mirror.
https://github.com/fedora-infra/mirrormanager2/issues/148
I created a 'dl.phx2.fedoraproject.org' site and added the 10.5.x.x networks in phx2 to it. I marked it always up to date.
https://admin.fedoraproject.org/mirrormanager/host/2086
However, it still doesn't seem to be working. Does it need to be crawled first or something before it appears?
I am not sure if this a newly with MM2 introduced problem or if this has always been this way, but it seems private mirror and "always up to date" cannot be mixed. The crawler also runs for mirrors which are marked as "always up to date". Those mirrors are not crawled, but the crawler marks the mirror correctly. For a private mirror which is "always up to date" a report_mirror run is probably necessary. I will try this now.
So even after running report_mirror it still does not work. I have been looking at one private mirror which also does not work for the last few weeks (with extremely low priority) and it seems this could be the same error. Something with private mirrors seems to be broken currently. Not all private mirrors but maybe only newly created private mirrors. I switched 'dl.phx2.fedoraproject.org' to be a public mirror but that did not help yet. Still investigating.
Now it seems to work:
{{{ curl -s 'https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-23&arch=x86_64&ip=10.5.131.140' | head -2
http://dl.phx2.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ }}}
I removed the private flag from the URLs which, unfortunately, has a different meaning than what people seem to expect from it. The description what it means got lost during the migration from MM1 to MM2 and needs to be re-added.
Excellent. ;)
Thanks for tracking this down...
https://github.com/fedora-infra/mirrormanager2/issues/149
Indeed, confirmed! Thanks very much. Now we just need the stale metadata issue solved, or the internal mirror will be rejected for being 'out of date' :)
Login to comment on this ticket.