#8529 Fedora mirror stats broken in the new year
Closed: Fixed 4 years ago by smooge. Opened 4 years ago by mattdm.

Describe what you would like us to do:

The stats at https://data-analysis.fedoraproject.org/csv-reports/mirrors/ seem to be broken. I hope we can update that with DNF Better Counting sometime soon, but in the meantime I rely on this. There's a big drop on Dec 31st and then it bounces back, and then drop off again on 2020-01-06 and even more on 2020-01-07 — and it seems it's not updating at all past that.

When do you need this to be done by? (YYYY/MM/DD)

Before DevConf, please! (2020-01-23, but before that would be ideal.)

Thank you!


Metadata Update from @smooge:
- Issue assigned to smooge

4 years ago

Metadata Update from @smooge:
- Assignee reset
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: downloads

4 years ago

All the proxies were updated to Fedora 31 on Jan 08. It turns out that Fedora 31 no longer users cron to with logrotate. Instead it is using systemd and those are disabled by default. I am trying to get them enabled so we hopefully rotate tomorrow.

The other statistics generated by mirrormanager are also not working correctly: https://admin.fedoraproject.org/mirrormanager/statistics

This collects the data from all proxies on mm-backend01.prod and the download speed for the log files from proxy09 to mm-backend01.prod seems to be under 100KB/s (according to scp), which means that the logfile of 500MB takes some time to be transferred. Not sure if this has changed or if has always been this slow. This is also not related to the different logrotate, but also something I wanted to mention.

proxy09 has been slow for some time. I have not been able to find a 'reason' for it, but think it will be time to move it to another datacenter soon.

I do not know how mirrormanager stores or deals with stats.. so not sure how to fix that. Is the data also historically available in a csv form? [@mattdm would probably love that.]

The fixed logrotate unbroke it for me.

Is the data also historically available in a csv form?

It is available in generated html files. Probably parsable, but not easily. On mm-frontend01 (and 02) it is in /var/www/mirrormanager-statistics/data/. Interestingly these files would be gone after re-install.

We seem to have logrotates and should get stats soon. CLosing

Metadata Update from @smooge:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.

Metadata