Not sure what is going on but vpn connections for various tools are very slow with 20 to 40 second to start
local:
[root@proxy02 xinetd.d][PROD]# time rsync localhost:: log Server Logs real 0m0.012s user 0m0.002s sys 0m0.005s
from log01 to proxy02
[smooge@log01 fedora_stats][PROD-IAD2]$ time rsync proxy02.vpn.fedoraproject.org:: log Server Logs real 2m0.382s user 0m0.000s sys 0m0.003s
Tried other connections and got fast time
time curl --connect-to fedoraproject.org:443:proxy02.vpn.fedoraproject.org:443 "https://fedoraproject.org/" <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="https://getfedora.org/">here</a>.</p> <hr> <address>Apache Server at fedoraproject.org Port 443</address> </body></html> real 0m0.346s user 0m0.018s sys 0m0.009s
This seems to be caused by SSH and our default ifcfg using DNS from IAD2 for all systems.
Metadata Update from @smooge: - Issue assigned to smooge - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: dns, medium-gain, medium-trouble, ops
This was caused by duh duh duhhhh DNS. systemd-resolve was firing up before every connection to xinetd and the DNS configuration for system-resolve was using 10.3.163.33 versus /etc/resolv.conf. From looking at verbose logs connections were doing: try to look up host on 10.3.163.33 UDP and timeout try to look up host on 10.3.163.33 TCP and timeout try to look up host on 10.3.164.34 UDP and timeout try to look up host on 10.3.164.34 TCP and timeout. go to /etc/resolv.conf and get the hostname repeat.
Changing all the networkmanager DNS to use the same as /etc/resolv.conf fixed the issue.
Metadata Update from @smooge: - Issue untagged with: dns, medium-gain, medium-trouble, ops - Issue close_status updated to: Fixed - Issue priority set to: Needs Review (was: Waiting on Assignee) - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.