#4146 Kojira: check_external_repos only works for repos over HTTP, not for local files
Opened 10 months ago by taliaferro-llnl. Modified 4 months ago

The check_external_repos option for Koji checks the timestamp on external repos by using Requests to fetch the repo's repomd.xml file. This works fine if the repo is an http:// or https:// URL, but fails for local RPM repositories with a file:// URL. This happens because the Requests library in its base configuration only handles HTTP urls. As I see it this could be solved one of two ways.

One solution would be to use a transport adapter for Requests, which would allow it to handle other kinds of URLs. A Requests transport adapter for file:// urls is already available in Fedora and downstream distributions, so one solution to this problem could be to add that package as a dependency and mount it to the Requests session for Kojira.

The other solution would be to remove Requests from Kojira entirely, and just use the stdlib's urllib.request.urlopen, which does correctly handle file:// URLs. Fetching the repo metadata is the only use of requests within kojira, so we could actually just remove that import entirely if we went about it this way.

I'd like to contribute this fix if that's alright?
If anybody has thoughts about which of these approaches to take I'm open to suggestions.


To be honest, we have never considered use of file:// here. Such use presumes the path is local to:

  • the host handling the createrepo jobs
  • the host where kojira runs
  • the build host consuming the merged repo
  • any other host that wants to consume the merged repo

This seems like a very narrow case to support. Why not just use https?

Metadata Update from @mikem:
- Custom field Size adjusted to None

10 months ago

In our Koji installation we keep a local copy of upstream repositories on an NFS share that's shared with all our builders -- it's a little faster that way.

Metadata Update from @tkopecek:
- Issue set to the milestone: 1.36

10 months ago

@mikem do we want to implement this? I'm not 100% sure.

Did this ever work for you? We've been using python requests to read repomd.xml for external repos since koji-1.24.0.

I'm not in a rush to make a change for this. The options suggested might seem simple, but both carry some risk. My concern with urllib is that some subtle variation with the libraries (e.g. cert management) could break a different case.

The transport adapter is an interesting option, but it should not be a strict dependency. I.e. catch the import error, only set up the adapter if it is present, and don't include the explicit dep.

Log in to comment on this ticket.

Metadata