We did a migration of softwarefactory-project.io yesterday to a new cloud provider. IP addresses of services changed, domain name is still the same. The DNS changed was done yesterday (> 12 hours ago).
Problem is that we no longer receive web hook calls from src.fedoraproject.org. A first investigation from @pingou shown that src.fedoraproject.org resolves the previous ip address of softwarefactory-project.io. From pagure.io it resolves the right IP address.
from src.fedoraproject.org: softwarefactory-project.io has address 38.145.34.47 from pagure.io softwarefactory-project.io has address 38.102.83.76
from my laptop: dig +short @8.8.8.8 softwarefactory-project.io 38.102.83.76 ^ This is the new service address.
Could you check what is happening ?
ASAP as the Pull Request Distgit CI provided by Zuul is unable to process incoming PRs.
Thanks in advance
Could this be related to the unspecified DNS fix done for https://pagure.io/fedora-infrastructure/issue/8320 ?
The problem is with the Red Hat internal DNS servers which are 'authoritative' for softwarefactory.io. They still point to the old ip address and that is what is going to be used by anything inside of a Red Hat network. A dig +trace internal shows that if you are external to PHX2 (aka where src.fedoraproject.org is) then the authoritative answer is 38.102.83.76 but internal it is something else.
Please open a ticket with Red Hat IT to get this fixed. I can't do anything myself as I don't own the zone affected.
OK there does seem to be another problem which I will be following up with internal Red Hat. We are not able to get to the ns1/ns2/ns3 nameservers at this time when we could yesterday or so. I am working on figuring out what is broken here.
I have opened an internal red hat ticket to determine what changed and how to fix
Metadata Update from @smooge: - Issue priority set to: None (was: Needs Review) - Issue tagged with: dns, high-gain, high-trouble, outage
Thanks for the followup and investigation.
Changes were made to allow for us to use different internal nameservers. The problem seems to be fixed.
Metadata Update from @smooge: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
I can confirm in https://src.fedoraproject.org/rpms/python-setuptools/pull-request/34 that the Zuul job started. Thanks for the fix @smooge!
Yes thank @smooge ! the Zuul CI is fixed now !
Login to comment on this ticket.