#10745 Proposal: Drop .sqlite metadata from repositories provided for Fedora / CentOS / EPEL
Opened 3 months ago by dalley. Modified a month ago

  • Describe the issue

To the best of my understanding, .sqlite metadata files were used only by legacy "yum" and the "repoview" tool. DNF no longer utilizes them, and the "repoview" tool is abandoned and no longer used very much - it is not packaged for Fedora 28+ or EL8+ because it depends on Python 2 and some ancient (also unmaintained) Python 2 libraries.

Therefore it seems that you could simplify the compose process by cutting out the step to generate this sqlite metadata, and reduce the size of a repository mirror by a few hundred megabytes.

Any users of either "yum" or the "repoview" tool will still have workarounds available. Legacy "yum" should merely be a bit slower since it will create the metadata for itself if it isn't available. "repoview" would just require an intermediate step, to use a tool such as "sqliterepo_c" to generate the metadata.

I've looked through a ton of repos, and the only user of "repoview" I found is RPMFusion. They seem to use a Fedora 26 package to do it though...

Some additional discussion here, in a project I help maintain: https://discourse.pulpproject.org/t/planning-to-remove-a-feature-from-the-rpm-plugin-sqlite-metadata/418/2

  • When do you need this? (YYYY/MM/DD)

N/A

  • When is this no longer needed or useful? (YYYY/MM/DD)

N/A

  • If we cannot complete your request, what is the impact?

Status Quo / None


This is referring to the files declared as "primary_db", "filelists_db", "other_db" in repomd.xml

e.g: https://dl.fedoraproject.org/pub/fedora/linux/updates/35/Everything/x86_64/repodata/repomd.xml

This makes sense to me. I think the only thing that uses it still is mdapi. Once that's ported away from the SQLite stuff to XML rpmmd, then you should be set.

@lsedlar Does this require any changes to createrepo_c or pungi?

Metadata Update from @mohanboddu:
- Issue tagged with: medium-gain, medium-trouble, ops

3 months ago

It wouldn't require any changes to createrepo_c.

I think the only change required to Pungi would be to flip this boolean to "False", either here (the default value) or at the callsite(s). Granted that is based on a very cursory browsing of the source code, but unless I'm missing something it would be just that simple.

https://pagure.io/pungi/blob/master/f/pungi/wrappers/createrepo.py#_47

I think there is a little bit more to it, but still not much. This file and in particular this method [0] has some repoview-related code which doesn't need to be removed simultaneously but probably should be removed. If it were to be kept, you'd just want to make sure it doesn't just assume that the sqlite metadata is always generated.

[0] https://pagure.io/pungi/blob/master/f/pungi/gather.py#_1638

I'd be willing to make a patch if this change is considered acceptable.

@mohanboddu Is this issue as currently written and the suggestions I've written in the comments something that would be accepted?

We definitely cannot make this change if mdapi is still using this... (I don't know for sure if it is).

https://pagure.io/mdapi/

It looks like it is. But what I meant was "is the general idea sound" - and if it is then we can proceed with having a discussion about what to do about mdapi.

Login to comment on this ticket.

Metadata
Boards 1
Ops Status: Backlog