Issue #12833: /etc/cron.daily/sync-http-logs-and-merge.sh on log01 errors - fedora-infrastructure

fedora-infrastructure

#12833 /etc/cron.daily/sync-http-logs-and-merge.sh on log01 errors

Closed: Migrated to Fedora Forge Feb 2, 2026 by ryanlerch. Opened Oct 6, 2025 by kevin.

This daily cron job is outputting errors:

/usr/local/bin/sync-http-logs.py:244: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
  config = yaml.load(config_file)
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
/usr/local/bin/combineHttpLogs.sh: line 27: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
...
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 91: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 100: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 102: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 113: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 115: simple_message_to_bus: command not found
/usr/local/bin/combineHttpLogs.sh: line 132: simple_message_to_bus: command not found

The kojipkgs errors are likely because they don't have the rsyncd role setup to allow collecting from them.
No idea on the other one. ;)

Metadata Update from @james:
- Issue assigned to james

Oct 6, 2025

james commented Oct 6, 2025

simple_message_to_bus is a path problem.

Metadata Update from @james:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: low-gain, low-trouble

Oct 6, 2025

Issue tagged with: sprint-0

Oct 22, 2025

kevin commented Oct 22, 2025

FYI, the current logs have:

/usr/local/bin/condense-mirrorlogs.sh: line 31: simple_message_to_bus: command not found
/usr/local/bin/condense-mirrorlogs.sh: line 78: simple_message_to_bus: command not found
/usr/local/bin/condense-mirrorlogs.sh: line 80: simple_message_to_bus: command not found
/usr/local/bin/condense-mirrorlogs.sh: line 89: simple_message_to_bus: command not found
/usr/local/bin/condense-mirrorlogs.sh: line 91: simple_message_to_bus: command not found
plot ["2007-05-17":"2024-12-31"]      '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:2  title 'epel4' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv'
using 1:3  title 'epel5' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:4  title 'epel6' with lines lw 3,
'/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:5  title 'epel7' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:72  title 'epel8' with lines
lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:73  title 'epel9' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:6  title 'fed03'
with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:7  title 'fed04' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:8  title
'fed05' with lines lw 3,     '/var/www/html/csv-reports/mirrors/mirro
... (bunch more of these).
^
"/usr/local/share/web-data-analysis/mirrors-data.gp" line 107: all points y value undefined!
/usr/local/bin/condense-mirrorlogs.sh: line 119: simple_message_to_bus: command not found

zlopez commented Jan 16, 2026

I wonder how the cron job itself is executed. As I tried run-parts /etc/cron.daily and didn't saw the error. The PATH is correctly configured in /etc/crontab

[root@log01 ~][PROD-RDU3]# run-parts /etc/cron.daily/                                                                                                                                                                                       
/etc/cron.daily/sync-http-logs-and-merge.sh:                                                                                                                                                                                                
/usr/local/bin/sync-http-logs.py:244: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.                                     
  config = yaml.load(config_file)                                                                                     
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                              
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)                                                                                                                             
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)                                                                                                                        
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)       
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]    
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)                                                                                                                        
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]     
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)                                                                                                                              
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                             
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                    
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)                                                                                                                        
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)                                                                                                                             
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)      
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)  
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)       
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                  
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)      
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)                                                                                                                        
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)        
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)      
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)  
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                             
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)                                                                                                                              
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                               
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                   
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110) 
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                              
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)                                                                                                                            
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)                                                                                                                        
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)      
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)  
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy05.vpn.fedoraproject.org (192.168.1.25): Connection timed out (110)                                                                                                                             
rsync: [Receiver] failed to connect to proxy03.vpn.fedoraproject.org (192.168.1.7): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                  
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                     
rsync: [Receiver] failed to connect to proxy14.vpn.fedoraproject.org (192.168.1.159): Connection timed out (110)      
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]                                                                                                                                                           
rsync: [Receiver] failed to connect to kojipkgs01.rdu3.fedoraproject.org (10.16.169.106): Connection timed out (110)  
rsync error: error in socket IO (code 10) at clientserver.c(138) [Receiver=3.2.5]

kevin commented Jan 19, 2026

proxy03/14 were because the vpn ip wasn't getting staticly assigned. I fixed that.

proxy05 is down.

kojipkgs01 looks like a rhit firewall issue. We will need to file a internal ticket to get that allowed.
Not sure why kojipkgs02 isn't listed, it should be (and also doesn't work)

kevin commented Jan 20, 2026

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/infra/tickets/issues/12833

Please continue any further discussion there.

Metadata Update from @ryanlerch:
- Issue close_status updated to: Migrated to Fedora Forge
- Issue status updated to: Closed (was: Open)

Feb 2, 2026

Metadata

Assignee

james

Tags

Blocking

None

Depending on

None

Priority

Normal

fedora-infrastructure

#12833 /etc/cron.daily/sync-http-logs-and-merge.sh on log01 errors Closed: Migrated to Fedora Forge Feb 2, 2026 by ryanlerch. Opened Oct 6, 2025 by kevin.

Metadata

low-gain low-trouble sprint-0

#12833 /etc/cron.daily/sync-http-logs-and-merge.sh on log01 errors

Closed: Migrated to Fedora Forge Feb 2, 2026 by ryanlerch. Opened Oct 6, 2025 by kevin.