#7333 in Uruguay for mirror manager: Try first with mirrors closer to Miami EEUU or Argentina but not with Colombia or Ecuador
Closed: Will Not/Can Not fix 5 years ago by kevin. Opened 5 years ago by pablodav.

Got better information now:

https://pagure.io/fedora-infrastructure/issue/7333#comment-550607

mtr traceroute:

mtr --report mirror.upb.edu.co
Start: 2018-10-28T15:46:35-0300
HOST: force1                      Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 2800:a4:23d5:1c01:ce7b:35  0.0%    10    0.5   0.5   0.5   0.6   0.0
  2.|-- 2800:a0:a11::1             0.0%    10   10.0   9.6   7.8  10.2   0.7
  3.|-- 2800:a0:a11::2             0.0%    10    9.6  11.0   9.6  20.3   3.3
  4.|-- 2800:a0:801:c828:400b:200 20.0%    10   17.8  18.1  17.8  19.5   0.6
  5.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0
  6.|-- 2001:1498:1::100:252       0.0%    10  137.1 137.5 136.8 138.6   0.6
  7.|-- 2001:1498:1::100:227       0.0%    10  136.9 137.5 136.6 142.5   1.8
  8.|-- 2001:1498:1:3cd::2         0.0%    10  151.6 151.1 150.6 151.6   0.4
  9.|-- lo-0-v6.ear2.Miami1.Level  0.0%    10  138.2 138.2 137.7 139.3   0.5
 10.|-- 2001:1900:25::19           0.0%    10  141.5 141.2 140.7 141.5   0.3
 11.|-- 2001:450:2001:1000:0:670:  0.0%    10  149.7 150.0 148.5 150.5   0.6
 12.|-- 2001:13b0:a000:1::2        0.0%    10  159.3 158.8 156.8 159.6   0.8
 13.|-- fd00:190::1                0.0%    10  153.1 152.7 151.6 153.8   0.6
 14.|-- mirror.upb.edu.co          0.0%    10  152.7 152.9 152.2 154.3   0.6
  • When do you need this? (YYYY/MM/DD)
    N/A

  • When is this no longer needed or useful? (YYYY/MM/DD)

  • If we cannot complete your request, what is the impact?
    dnf will have timout errors in Uruguay


I can try with more sites, my IP is dynamic (it changes on every connection and every 12h)

I can try from some enterprise environment too. (I work on infraestructure and I see this on many internet connections not just mine in my home).

See this:

Miami, ping 159ms:

http://www.speedtest.net/result/7754643822

Bogota, ping 199ms:
http://www.speedtest.net/result/7754647206

Argentina, ping 29ms:
http://www.speedtest.net/result/7754649698

Ecuador, ping 227ms:
http://www.speedtest.net/result/7754654111

Brasil 45ms:
http://www.speedtest.net/result/7754655950

It depends on IP versions (4 or 6), I think speedtest does test with ipv4.

Historically we had better connection with Miami other than other countries in south america, today looks like with have some things better with Argentina and a little slower with Brasil. But any other country it is worse.

Hi. Let me propose another soluction. I set up a mirror for Uruguay. Help me getting it on-line. It is already serving content and up-to-date. The announce was made last week to the mirror list. I am sure everybody is working on release 29 final release.

Anyway http://espejito.fder.edu.uy/fedora/ might help.

Your ping may drop to 5ms, although, no IPv6 service yet.

Thank you for the mirror. Have you added a mirror manager account?

I am trying to register it.

I created the site, the host, my account an run the report-mirror script:

espejito:~ # /root/bin/report_mirror
checked in successful

In everything works fine, we will get some traffic soon :-)

Let me know if something else is required.

regards

ariel

El 30/10/18 a las 9:37, Stephen J Smoogen escribió:

smooge added a new comment to an issue you are following:
Thank you for the mirror. Have you added a mirror manager account?

To reply, visit the link below or just reply to this email
https://pagure.io/fedora-infrastructure/issue/7333

Gracias Ariel!

It is working better today (probably it is taking some closer mirrors), but with local mirror it will be much better!

Working now.

We already served about 9000 request so far (just warming up!)

Let us know if you find any problem

bienvenidos!

ariel

El 30/10/18 a las 10:04, Pablo Estigarribia escribió:

pablodav added a new comment to an issue you are following:
``
Gracias Ariel!

It is working better today (probably it is taking some closer mirrors), but with local mirror it will be much better!

``

To reply, visit the link below or just reply to this email
https://pagure.io/fedora-infrastructure/issue/7333

Metadata Update from @pablodav:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 years ago

Metadata Update from @pablodav:
- Issue status updated to: Open (was: Closed)

5 years ago

Unfortunately I'm still getting too much timeouts and low speed, example:

Error: Error al descargar los paquetes:
  Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=updates-testing-f29&arch=x86_64 [Operation timed out after 30000 milliseconds with 0 out of 0 bytes received]

But it is not just this, for a simple install or update I have to try 5 or 6 times until it starts working fine (looks like it is trying with slow mirrors first).

Thats a timeout getting the mirror list itself from our proxy network.

Is this still happening or was it transitory?

If you are still seeing it can you attach your /var/log/dnf.librepo.log file so we can see what proxy it's hitting?

looks to be transitory now, excelent!

have done dnf update and some install with no issues today.

Metadata Update from @pablodav:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 years ago

looks like the timeout is not only in uruguay:

    fatal: [ansible_test-04]: FAILED! => {"changed": false, "msg": "Failed to download packages: Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f29&arch=x86_64 [Connection timed out after 30002 milliseconds]", "results": []}

got during automatic tests in travis

https://travis-ci.org/CoffeeITWorks/ansible_burp2_server/builds/471254209

test failed due to this error

Metadata Update from @pablodav:
- Issue status updated to: Open (was: Closed)

5 years ago

Today I think I got something useful:

when using command with default ipv6:

sudo dnf update --refresh

I got:

2019-01-23T00:14:00Z DEBUG No se puede descargar 'https://codecs.fedoraproject.org/openh264/29/x86_64/': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried.
2019-01-23T00:14:04Z DEBUG reviving: 'fedora-modular' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:14:04Z DEBUG not found other for: Fedora Modular 29 - x86_64
2019-01-23T00:14:04Z DEBUG not found deltainfo for: Fedora Modular 29 - x86_64
2019-01-23T00:14:04Z DEBUG not found updateinfo for: Fedora Modular 29 - x86_64
2019-01-23T00:14:04Z DEBUG fedora-modular: usando metadatos de mié 24 oct 2018 19:22:05 -03.
2019-01-23T00:14:34Z DEBUG error: Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f29&arch=x86_64 [Operation timed out after 30001 milliseconds with 0 out of 0 bytes received] (https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f29&arch=x86_64).
2019-01-23T00:14:34Z DEBUG No se puede descargar 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f29&arch=x86_64': Cannot prepare internal mirrorlist: Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f29&arch=x86_64 [Operation timed out after 30001 milliseconds with 0 out of 0 bytes received].
2019-01-23T00:14:34Z DDEBUG Cleaning up.
2019-01-23T00:14:34Z SUBDEBUG 
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/dnf/repo.py", line 566, in load
    ret = self._repo.load()
  File "/usr/lib64/python3.7/site-packages/libdnf/repo.py", line 503, in load
    return _repo.Repo_load(self)
RuntimeError: No se pudo sincronizar la caché para el repositorio 'updates-modular'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 64, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 99, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.7/site-packages/dnf/cli/main.py", line 115, in cli_run
    cli.run()
  File "/usr/lib/python3.7/site-packages/dnf/cli/cli.py", line 1099, in run
    self._process_demands()
  File "/usr/lib/python3.7/site-packages/dnf/cli/cli.py", line 824, in _process_demands
    load_available_repos=self.demands.available_repos)
  File "/usr/lib/python3.7/site-packages/dnf/base.py", line 400, in fill_sack
    self._add_repo_to_sack(r)
  File "/usr/lib/python3.7/site-packages/dnf/base.py", line 135, in _add_repo_to_sack
    repo.load()
  File "/usr/lib/python3.7/site-packages/dnf/repo.py", line 568, in load
    raise dnf.exceptions.RepoError(str(e))
dnf.exceptions.RepoError: No se pudo sincronizar la caché para el repositorio 'updates-modular'
2019-01-23T00:14:34Z CRITICAL Error: No se pudo sincronizar la caché para el repositorio 'updates-modular'

But when using ipv4:

sudo dnf update --refresh -4

Everything works super fast and without any error.

2019-01-23T00:18:45Z DEBUG DNF version: 4.0.9
2019-01-23T00:18:45Z DDEBUG Command: dnf update --refresh -4 
2019-01-23T00:18:45Z DDEBUG Installroot: /
2019-01-23T00:18:45Z DDEBUG Releasever: 29
2019-01-23T00:18:45Z DEBUG cachedir: /var/cache/dnf
2019-01-23T00:18:45Z DDEBUG Base command: update
2019-01-23T00:18:45Z DDEBUG Extra commands: ['update', '--refresh', '-4']
2019-01-23T00:18:45Z DEBUG Valor de configuración desconocido: failovermethod=priority en /etc/yum.repos.d/fedora-cisco-openh264.repo; Configuration: OptionBinding with id "failovermethod" does not exist
2019-01-23T00:18:50Z DEBUG reviving: 'fedora-cisco-openh264' se puede reutilizar, los metadatos coinciden.
2019-01-23T00:18:50Z DEBUG not found other for: Fedora 29 openh264 (From Cisco) - x86_64
2019-01-23T00:18:50Z DEBUG not found modules for: Fedora 29 openh264 (From Cisco) - x86_64
2019-01-23T00:18:50Z DEBUG not found updateinfo for: Fedora 29 openh264 (From Cisco) - x86_64
2019-01-23T00:18:50Z DEBUG fedora-cisco-openh264: usando metadatos de mié 12 set 2018 11:23:26 -03.
2019-01-23T00:18:52Z DEBUG reviving: 'fedora-modular' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:18:52Z DEBUG not found other for: Fedora Modular 29 - x86_64
2019-01-23T00:18:52Z DEBUG not found deltainfo for: Fedora Modular 29 - x86_64
2019-01-23T00:18:52Z DEBUG not found updateinfo for: Fedora Modular 29 - x86_64
2019-01-23T00:18:52Z DEBUG fedora-modular: usando metadatos de mié 24 oct 2018 19:22:05 -03.
2019-01-23T00:18:53Z DEBUG reviving: 'updates-modular' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:18:53Z DEBUG not found other for: Fedora Modular 29 - x86_64 - Updates
2019-01-23T00:18:53Z DEBUG not found deltainfo for: Fedora Modular 29 - x86_64 - Updates
2019-01-23T00:18:53Z DEBUG updates-modular: usando metadatos de vie 18 ene 2019 23:04:27 -03.
2019-01-23T00:18:55Z DEBUG reviving: 'updates-testing' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:18:55Z DEBUG not found other for: Fedora 29 - x86_64 - Test Updates
2019-01-23T00:18:55Z DEBUG not found modules for: Fedora 29 - x86_64 - Test Updates
2019-01-23T00:18:56Z DEBUG updates-testing: usando metadatos de lun 21 ene 2019 23:55:53 -03.
2019-01-23T00:18:59Z DEBUG reviving: 'updates' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:18:59Z DEBUG not found other for: Fedora 29 - x86_64 - Updates
2019-01-23T00:18:59Z DEBUG not found modules for: Fedora 29 - x86_64 - Updates
2019-01-23T00:18:59Z DEBUG updates: usando metadatos de mar 22 ene 2019 14:37:29 -03.
2019-01-23T00:19:01Z DEBUG reviving: 'fedora' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:19:01Z DEBUG not found other for: Fedora 29 - x86_64
2019-01-23T00:19:01Z DEBUG not found modules for: Fedora 29 - x86_64
2019-01-23T00:19:01Z DEBUG not found deltainfo for: Fedora 29 - x86_64
2019-01-23T00:19:01Z DEBUG not found updateinfo for: Fedora 29 - x86_64
2019-01-23T00:19:01Z DEBUG fedora: usando metadatos de mié 24 oct 2018 19:20:15 -03.
2019-01-23T00:19:02Z DEBUG reviving: 'google-chrome' se puede reutilizar, los metadatos coinciden.
2019-01-23T00:19:02Z DEBUG not found other for: google-chrome
2019-01-23T00:19:02Z DEBUG not found modules for: google-chrome
2019-01-23T00:19:02Z DEBUG not found deltainfo for: google-chrome
2019-01-23T00:19:02Z DEBUG not found updateinfo for: google-chrome
2019-01-23T00:19:02Z DEBUG google-chrome: usando metadatos de mar 22 ene 2019 18:58:09 -03.
2019-01-23T00:19:05Z DEBUG reviving: 'rpmfusion-free-updates' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:19:05Z DEBUG not found other for: RPM Fusion for Fedora 29 - Free - Updates
2019-01-23T00:19:05Z DEBUG not found modules for: RPM Fusion for Fedora 29 - Free - Updates
2019-01-23T00:19:05Z DEBUG not found deltainfo for: RPM Fusion for Fedora 29 - Free - Updates
2019-01-23T00:19:05Z DEBUG not found updateinfo for: RPM Fusion for Fedora 29 - Free - Updates
2019-01-23T00:19:05Z DEBUG rpmfusion-free-updates: usando metadatos de jue 17 ene 2019 20:56:18 -03.
2019-01-23T00:19:07Z DEBUG reviving: 'rpmfusion-free' se puede reutilizar, las comprobaciones del metalink cuadran.
2019-01-23T00:19:07Z DEBUG not found other for: RPM Fusion for Fedora 29 - Free
2019-01-23T00:19:07Z DEBUG not found modules for: RPM Fusion for Fedora 29 - Free
2019-01-23T00:19:07Z DEBUG not found deltainfo for: RPM Fusion for Fedora 29 - Free
2019-01-23T00:19:07Z DEBUG not found updateinfo for: RPM Fusion for Fedora 29 - Free
2019-01-23T00:19:07Z DEBUG rpmfusion-free: usando metadatos de mar 23 oct 2018 08:05:19 -03.
2019-01-23T00:19:12Z DEBUG reviving: 'code' se puede reutilizar, los metadatos coinciden.
2019-01-23T00:19:12Z DEBUG not found other for: Visual Studio Code
2019-01-23T00:19:12Z DEBUG not found modules for: Visual Studio Code
2019-01-23T00:19:12Z DEBUG not found deltainfo for: Visual Studio Code
2019-01-23T00:19:12Z DEBUG not found updateinfo for: Visual Studio Code
2019-01-23T00:19:12Z DEBUG code: usando metadatos de mar 08 ene 2019 13:12:06 -03.
2019-01-23T00:19:13Z DDEBUG timer: sack setup: 27840 ms
2019-01-23T00:19:13Z DEBUG Completion plugin: Generating completion cache...
2019-01-23T00:19:14Z DEBUG --> Comenzando resolución de dependencias
2019-01-23T00:19:15Z DEBUG --> Resolución de dependencias finalizada
2019-01-23T00:19:15Z DDEBUG timer: depsolve: 990 ms
2019-01-23T00:19:15Z INFO Dependencias resueltas.
2019-01-23T00:19:15Z INFO Nada por hacer.
2019-01-23T00:19:15Z INFO ¡Listo!
2019-01-23T00:19:15Z DDEBUG Cleaning up.

So is ipv6 actually working otherwise? Can you ping6 mirrors.fedoraproject.org ok? Can you curl manually that fedora-modular repo?

Note that lots of places seem to have broken ipv6, where they give you an address, but it doesn't route or work properly.

Yes, seems ipv6 issues here.

ping6 mirrors.fedoraproject.org
PING mirrors.fedoraproject.org(proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9)) 56 data bytes
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=1 ttl=46 time=221 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=2 ttl=46 time=224 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=3 ttl=46 time=220 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=4 ttl=46 time=222 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=5 ttl=46 time=221 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=6 ttl=46 time=221 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=7 ttl=46 time=220 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=8 ttl=46 time=223 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=9 ttl=46 time=222 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=10 ttl=46 time=223 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=11 ttl=46 time=223 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=12 ttl=46 time=223 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=13 ttl=46 time=223 ms
64 bytes from proxy06.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:fed9): icmp_seq=14 ttl=46 time=222 ms
^C
--- mirrors.fedoraproject.org ping statistics ---
14 packets transmitted, 14 received, 0% packet loss, time 28ms
rtt min/avg/max/mdev = 220.301/222.063/223.741/1.160 ms
curl -o test.html --ipv6 --verbose 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-modular-f29&arch=x86_64'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2610:28:3090:3001:dead:beef:cafe:fed3...
* TCP_NODELAY set
* Connected to mirrors.fedoraproject.org (2610:28:3090:3001:dead:beef:cafe:fed3) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [74 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2862 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [556 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=North Carolina; L=Raleigh; O=Red Hat Inc.; CN=*.fedoraproject.org
*  start date: Feb  1 00:00:00 2017 GMT
*  expire date: May  1 12:00:00 2020 GMT
*  subjectAltName: host "mirrors.fedoraproject.org" matched cert's "*.fedoraproject.org"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*  SSL certificate verify ok.
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x5627afafb570)
} [5 bytes data]
> GET /metalink?repo=updates-released-modular-f29&arch=x86_64 HTTP/2
> Host: mirrors.fedoraproject.org
> User-Agent: curl/7.61.1
> Accept: */*
> 
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 200 
< date: Tue, 05 Feb 2019 22:10:58 GMT
< server: Apache/2.4.37 (Fedora) mod_wsgi/4.6.4 Python/2.7
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
< referrer-policy: same-origin
< content-length: 3194
< vary: Accept-Encoding
< content-type: application/metalink+xml
< apptime: D=3957
< appserver: proxy04.fedoraproject.org
< 
{ [1045 bytes data]
100  3194  100  3194    0     0   3213      0 --:--:-- --:--:-- --:--:--  3210
* Connection #0 to host mirrors.fedoraproject.org left intact

Metadata Update from @bowlofeggs:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

5 years ago

So where are we here? Is this still happening? Is there anything we can do?

The last comment looks like it worked? Or did it fail there?

Well actually it works fine if I add:

grep '4' /etc/dnf/dnf.conf 
ip_resolve=4

If not, it still is happening.

Odd. You can duplicate it with curl? Can you file a curl bug in bugzilla.redhat.com with the above output and we can move debugging there as they will know much more about curl than we.

Or if you prefer I can file it... just let me know

I'm going to sleep now, and tomorrow a hard day waits to me.

I don't know when I'm will be able to file the bug requested, maybe in some days.

ok. Let us know if there's anything more we can do here.

Metadata Update from @kevin:
- Issue close_status updated to: Will Not/Can Not fix
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata