#9435 Containers builds fail without a known reason quite often
Closed: Fixed 3 years ago by kevin. Opened 3 years ago by hhorak.

  • Describe the issue
    I've submitted mysql container build task several times and it failed on the "RPM install" step. Without changing anything, it worked on 7th attempt (the exact same SCM commit, same target). It looks like koji (OSBS has some troubles with fetching RPMs during the container builds).

Successful task:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1503449

Faild tasks:
https://koji.fedoraproject.org/koji/taskinfo?taskID=44115864
https://koji.fedoraproject.org/koji/taskinfo?taskID=44115520
https://koji.fedoraproject.org/koji/taskinfo?taskID=44115022
https://koji.fedoraproject.org/koji/taskinfo?taskID=44114957
https://koji.fedoraproject.org/koji/taskinfo?taskID=44114858
https://koji.fedoraproject.org/koji/taskinfo?taskID=44114410

  • When do you need this? (YYYY/MM/DD)
    2020/05/05

  • When is this no longer needed or useful? (YYYY/MM/DD)
    N/A

  • If we cannot complete your request, what is the impact?
    Container builds fail repeatedly, maintainers need to re-submit tasks over and over.


@cverna Can you take a look at this?

Metadata Update from @mohanboddu:
- Issue assigned to cverna

3 years ago

I think this is related to that https://pagure.io/fedora-infrastructure/issue/6741

Not sure we can do much and that does not seems to happen very often.

I wouldn't say it does not happen very often -- in the case above, it happened 6 times in 7 attempts. I'm trying again with a different container image and so far have seen 3 failures in a row:

https://koji.fedoraproject.org/koji/taskinfo?taskID=44221862
https://koji.fedoraproject.org/koji/taskinfo?taskID=44222461
https://koji.fedoraproject.org/koji/taskinfo?taskID=44350654

Trying to overcome this with a cycle now:
while true ; do fedpkg container-build && break ; sleep 10m ; done

Am I just unlucky because I run the container builds at wrong time time of the day? Any idea when the master mirrors are updated, so I can try different time of the day, whether it is better?

Metadata Update from @cverna:
- Assignee reset

3 years ago

So openQA used to run into this problem a lot. My general approach to work around it is to tweak the repository configuration to use mirrorlist instead of metalink. That way the infra repo always comes back at the top of the list and we always use it - all the clever-clever metadata timestamping stuff is skipped.

Would it be plausible to do that in this case?

So openQA used to run into this problem a lot. My general approach to work around it is to tweak the repository configuration to use mirrorlist instead of metalink. That way the infra repo always comes back at the top of the list and we always use it - all the clever-clever metadata timestamping stuff is skipped.

Would it be plausible to do that in this case?

Is this still happening?

I'm going to close this now... if it's still happening, please re-open. Thanks!

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata