tl;dr: the /mnt/fedora_stats/combined-http logs either aren't being updated or have moved. either log-merging needs to get fixed, or I need to know where to find mirror logs going forward.
/mnt/fedora_stats/combined-http
The current countme-mirrors code (for https://fedoraproject.org/wiki/Changes/DNF_Better_Counting) uses the combined logs in /mnt/fedora_stats/combined-http/$YYYY/$MM/$DD/mirrors.fedoraproject.org-access.log*, but no new logs have been added there since June 8:
/mnt/fedora_stats/combined-http/$YYYY/$MM/$DD/mirrors.fedoraproject.org-access.log*
[wwoods@log01 mirrors-countme][PROD-IAD2]$ ./countme-update-rawdb.sh -n -v --rawdb $HOME/countme-raw.db --start 2020-05-31 Date range: 2020-05-31 -- today Logdir: /mnt/fedora_stats/combined-http no match for '/mnt/fedora_stats/combined-http/2020/06/09/mirrors.fedoraproject.org-access.log' [...] no match for '/mnt/fedora_stats/combined-http/2020/06/23/mirrors.fedoraproject.org-access.log'
I think those are generated by the awstats role; see roles/awstats/files/combineHttpLogs.sh.
awstats
mirrors-countme could be updated to pull data from the un-merged logs in /var/log/hosts on log01, but the other web-data-analysis tools also use the combined logs, so I suspect something got broken (or temporarily disabled?) during the colo move.
mirrors-countme
/var/log/hosts
log01
web-data-analysis
Let's say 2020/06/25? It's blocking countme data updates, but nothing depends on those yet, so it's not an emergency.
countme
Metadata Update from @mohanboddu: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: groomed, medium-gain, medium-trouble
Whats the status here? I want to say we fixed this, but I am not sure?
I have not had time to make sure it is working and it needs to be part of other tickets related to data-analysis
Yeah, I think we're good now. I don't remember the exact problem & fix - I think @kevin had to flip some permission bits so that host could write to NFS? - but the combined logs are appearing as expected by mirrors-countme and we can once again generate data. Closing.
Metadata Update from @wwoods: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.