changed to outage as this issue prevents us from using ppc-comm01.phx2.fedoraproject.org as a koji builder

Some more information
This works:
[root@ppc-comm02]# traceroute -T -n pkgs.fedoraproject.org
traceroute to pkgs.fedoraproject.org (209.132.181.4), 30 hops max, 60 byte packets
1 10.5.124.252 0.349 ms 0.407 ms 0.474 ms
2 10.4.2.34 1.514 ms 1.354 ms 1.567 ms
3 209.132.181.98 2.740 ms 3.005 ms 3.094 ms
4 10.5.124.2 3.125 ms 3.503 ms 3.424 ms
5 10.4.42.5 3.270 ms 3.437 ms 2.779 ms
6 209.132.181.4 2.325 ms 2.264 ms 2.801 ms

This doesn't:
[root@ppc-comm01]# traceroute -T -n pkgs.fedoraproject.org
traceroute to pkgs.fedoraproject.org (209.132.181.4), 30 hops max, 60 byte packets
1 10.5.124.252 1.449 ms 1.578 ms 1.652 ms
2 10.4.2.34 1.999 ms 2.071 ms 1.941 ms
3 209.132.181.98 3.509 ms 3.447 ms 3.270 ms
4 *
5 *
6 *
7 *
8 *
9 * * *

Network folks are still trying to figure this one out. ;(

Is there anything different in the config of the two? Does one have a public ip or anything odd like that?

Hopefully they will get to the bottom of it soon... sorry...

More information, hopefully better formatted:
{{{
[karsten@ppc-comm02 ~]$ ifconfig
eth0 Link encap:Ethernet Hardware Adresse 00:21:5E:B6:B9:30
inet Adresse:10.5.124.182 Bcast:10.5.124.255 Maske:255.255.255.128
inet6 Adresse: fe80::221:5eff:feb6:b930/64 Gültigkeitsbereich:Verbindung
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6511003 errors:0 dropped:0 overruns:0 frame:0
TX packets:1904704 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:1225741648 (1.1 GiB) TX bytes:14720702866 (13.7 GiB)

lo Link encap:Lokale Schleife
inet Adresse:127.0.0.1 Maske:255.0.0.0
inet6 Adresse: ::1/128 Gültigkeitsbereich:Maschine
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:31406439 errors:0 dropped:0 overruns:0 frame:0
TX packets:31406439 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:0
RX bytes:46947674005 (43.7 GiB) TX bytes:46947674005 (43.7 GiB)

[karsten@ppc-comm02 ~]$ route -n
Kernel IP Routentabelle
Ziel Router Genmask Flags Metric Ref Use Iface
10.5.124.128 0.0.0.0 255.255.255.128 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
0.0.0.0 10.5.124.254 0.0.0.0 UG 0 0 0 eth0

[root@ppc-comm02 karsten]# traceroute -T -n pkgs.fedoraproject.org
traceroute to pkgs.fedoraproject.org (209.132.181.4), 30 hops max, 60 byte packets
1 10.5.124.252 0.535 ms 0.574 ms 0.640 ms
2 10.4.2.34 1.730 ms 1.780 ms 1.370 ms
3 209.132.181.98 2.626 ms 2.737 ms 2.784 ms
4 10.5.124.2 3.187 ms 2.451 ms 2.793 ms
5 10.4.42.5 3.788 ms 3.487 ms 3.842 ms
6 209.132.181.4 2.855 ms 2.939 ms 2.438 ms

[root@ppc-comm02 karsten]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
}}}

{{{
[karsten@ppc-comm01 ~]$ ifconfig
eth0 Link encap:Ethernet Hardware Adresse 00:21:5E:B6:93:80
inet Adresse:10.5.124.181 Bcast:10.5.124.255 Maske:255.255.255.128
inet6 Adresse: fe80::221:5eff:feb6:9380/64 Gültigkeitsbereich:Verbindung
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:20180 errors:0 dropped:0 overruns:0 frame:0
TX packets:29137 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX bytes:2240576 (2.1 MiB) TX bytes:2940544 (2.8 MiB)

lo Link encap:Lokale Schleife
inet Adresse:127.0.0.1 Maske:255.0.0.0
inet6 Adresse: ::1/128 Gültigkeitsbereich:Maschine
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:3 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
Kollisionen:0 Sendewarteschlangenlänge:0
RX bytes:264 (264.0 b) TX bytes:264 (264.0 b)

[karsten@ppc-comm01 ~]$ route -n
Kernel IP Routentabelle
Ziel Router Genmask Flags Metric Ref Use Iface
10.5.124.128 0.0.0.0 255.255.255.128 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
0.0.0.0 10.5.124.254 0.0.0.0 UG 0 0 0 eth0

[root@ppc-comm01 karsten]# traceroute -T -n pkgs.fedoraproject.org
traceroute to pkgs.fedoraproject.org (209.132.181.4), 30 hops max, 60 byte packets
1 10.5.124.252 0.396 ms 0.450 ms 0.595 ms
2 10.4.2.34 1.450 ms 1.403 ms 1.402 ms
3 209.132.181.98 3.758 ms 3.599 ms 4.020 ms
4 *
5 *
6 *
7 * * *

[root@ppc-comm01 karsten]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
}}}

Can you ping/clone from 10.5.125.44 ?
(Thats the internal pkgs ip).

ping works:

{{{
[karsten@ppc-comm01 ~]$ ping 10.5.125.44
PING 10.5.125.44 (10.5.125.44) 56(84) bytes of data.
64 bytes from 10.5.125.44: icmp_seq=1 ttl=62 time=0.701 ms
64 bytes from 10.5.125.44: icmp_seq=2 ttl=62 time=0.487 ms
64 bytes from 10.5.125.44: icmp_seq=3 ttl=62 time=0.553 ms
}}}

clone doesn't:

{{{
[karsten@ppc-comm01 ~]$ git clone -n git://10.5.125.44/nfs-utils tmpnfs-utils
Cloning into tmpkhnfs-utils...
10.5.125.44[0: 10.5.125.44]: errno=Connection timed out
fatal: unable to connect a socket (Connection timed out)
}}}

git clone works as expected on ppc-comm02, btw.

Turned out to be a firewall issue:
{{{
Jul 18 08:47:44 10.4.2.36 Jul 18 2011 05:47:44: %FWSM-4-106023: Deny tcp src fedora-qa:10.5.124.181/53479 dst fedora-build:10.5.125.44/9418 by access-group "fedora-qa" [0x453fb874, 0x0]
}}}

This needs do be escalated to Red Hat by the Fedora Admin team so that they can fix this.

ok, we got this sorted out.

The one machine worked because it has an external IP mapping. So, it can talk to pkgs via that external ip and it works. The other machine didn't, so it couldn't get any replies back due to hairpinning.

Setting the other machine to use the internal 10.5.125.x ip for pkgs and allowing access to/from that solves the issue on that machine.