Hi infrastructure team,
due to rework of the way how RedHat is internally processing schedules (Product Pages), there is a push to reduce the number of files Fedora schedule consist of. Currently a Fedora schedule contains several HTML, CSV, ICS and TJX files ( https://fedorapeople.org/groups/schedule/f-${RELEASE}/ }
Before I will continue on a rework of the schedule, I would like to request statistics from the fedorapeople.org server (where we publish these schedules) for approx. last 3 months, showing aggregated numbers how many times was each file in https://fedorapeople.org/groups/schedule/f-27/ and https://fedorapeople.org/groups/schedule/f-26/ accessed. It helps me to make a decision what we can reduce and what we can not.
Thanks, Jan
Metadata Update from @smooge: - Issue assigned to smooge
So the fedorapeople logs do not record user agents so I am having to calculate what hosts were crawlers by reverse DNS. Most are good about this but a couple are a pain. There is also a Red Hat host which hits the link f-26/f-26-key-milestones.tjx 168 times a day which pushed that one to being the most looked for file.
Removing the obvious spiders and that outlier, the top 30 files in the last 3 months in descending order:
1515 f-27/external.png 1348 f-26/f-26-all-milestones.ics 828 f-27/f-27-key.ics 518 f-27/f-27-design-tasks.html 339 f-27/f-27-ambassadors-tasks.html 303 f-27/f-27-devel-tasks.html 282 f-26/external.png 262 f-27/f-27-docs-tasks.html 255 f-27/f-27-marketing-tasks.html 248 f-26/f-26-key.ics 231 f-27/f-27-all-tasks.html 230 f-27/f-27-quality-tasks.html 206 f-27/f-27-releng-tasks.html 168 f-27/f-27-web-tasks.html 118 f-27/f-27-trans-tasks.html 114 f-26/f-26-key-tasks.html 111 f-26/f-26-design-tasks.html 106 f-27/f-27-elections.html 106 f-26/f-26-key-milestones.tjx 77 f-26/f-26-docs-tasks.html 73 f-27/f-27-design.ics 73 f-26/f-26-elections.html 63 f-26/f-26-quality-tasks.html 63 f-26/f-26-marketing-tasks.html 62 f-27/f-27-pm-tasks.html 58 f-26/f-26-releng-tasks.html 58 f-26/f-26-devel-tasks.html 57 f-27/f-27-spins-tasks.html 57 f-27/f-27-key-tasks.html 57 f-26/f-26-all-tasks.html
The ics files are hit multiple times by single ips per day which leads me to believe evolution or thunderbird 'updating' themselves.
If we sort the data down to 1 ip address per day the list of files being accessed in order:
1445 f-27/external.png 482 f-27/f-27-design-tasks.html 316 f-27/f-27-ambassadors-tasks.html 289 f-27/f-27-devel-tasks.html 276 f-26/external.png 254 f-27/f-27-docs-tasks.html 245 f-27/f-27-key.ics 235 f-26/f-26-key.ics 223 f-27/f-27-all-tasks.html 219 f-27/f-27-marketing-tasks.html 215 f-27/f-27-quality-tasks.html 196 f-27/f-27-releng-tasks.html 192 f-26/f-26-all-milestones.ics 165 f-27/f-27-web-tasks.html 113 f-26/f-26-key-tasks.html 112 f-27/f-27-trans-tasks.html 105 f-26/f-26-design-tasks.html 104 f-27/f-27-elections.html 102 f-26/f-26-key-milestones.tjx 76 f-26/f-26-docs-tasks.html 71 f-26/f-26-elections.html 63 f-26/f-26-marketing-tasks.html 62 f-26/f-26-quality-tasks.html 58 f-26/f-26-releng-tasks.html 58 f-26/f-26-devel-tasks.html 56 f-26/f-26-all-tasks.html 55 f-27/f-27-spins-tasks.html 55 f-26/f-26-ambassadors-tasks.html 54 f-26/f-26-web-tasks.html 53 f-27/f-27-key-tasks.html
Does this help any?
Thanks a lot @smooge . That is exactly what I needed to know.
Metadata Update from @jkurik: - Issue status updated to: Closed (was: Open)