#12859 No crawling for mirror site since June
Closed: Fixed a month ago by kevin. Opened 2 months ago by cicku.

Hi team,

My mirror was pulled off from the Fedora mirrormanager few months ago without notice, I reviewed and found that it actually happened across all projects using mirrormanager v2.

There is an issue with mirrormanager that it considers "HTTP/3 alt-svc upgrade header returned via HTTP" invalid. Returning that header via HTTP basically instructs all clients to initiate UDP connection on a specific port if possible, which isn't wrong since clients can still choose to not follow. In my case, the HTTP/3 is deployed on port 443 (because QUIC is encrypted by default), which essentially makes the HTTP request (via HTTP/3) HTTPS always.

I have no idea why mirrormanager insists on following the upgrade header, but because I will not turn off HTTP/3, HTTP mirror URLs will be removed.

After I removed all HTTP links from the site, Fedora mirrormanager still has not crawled my site again since June.

Can you check what is still preventing the crawling there? I would avoid deleting the entire mirror site and create a new mirror.

Thanks!


Is there a reason this needs to be private?

CC: @abompard

Metadata Update from @cicku:
- Issue private status set to: False (was: True)

2 months ago

Metadata Update from @phsmoura:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: low-gain, low-trouble, ops

2 months ago

Hello! I looked at the site in mirrormanager, it's been disabled by an admin with the comment:
"Reaching out to the mirror admin for rsync whitelist".
Does this ring a bell?

Hello! I looked at the site in mirrormanager, it's been disabled by an admin with the comment:
"Reaching out to the mirror admin for rsync whitelist".
Does this ring a bell?

No, that is my own comment since the inception of the mirror site, never changed since 2024.

Yeah it's an admin flag that only admins see. I've re-enabled the site now.

Metadata Update from @kevin:
- Issue assigned to kevin

a month ago

I see crawl logs now, so I assume this is all done? If anything feel free to reopen and let us know.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

a month ago

Log in to comment on this ticket.

Metadata
Attachments 1