#7731 Duplicated copies of rawhide packages on the mirrors
Closed: Fixed 5 years ago Opened 5 years ago by tibbs.

Yesterday about 700GB of additional content was pushed out to the mirrors, consisting of non-hardlinked duplicates of rawhide package content. The various arch-specific "Packages" directories are present twice: once under the arch-specific directory and again under an "os" directory:

This happened in fedora-secondary as well. The content is not hardlinked on the mirrors.

It would be good to figure out which content is supposed to stay. If all of it is supposed to be there, can we make sure it gets hardlinked properly? Otherwise we're wasting a lot of time transferring everything twice and taking up a few hundred extra GB on all the mirrors.


is there a good reason for the extra os in the directory ? Probably would be easier over time if the targets matched the contents Pungi outputs, which doesn't have an os in there.

Well, the "os" directory has been there since Fedora Core 1. I guess revisiting it after 30 releases is certainly something that could be done, but it seems reasonable to discuss it and have a transition period where the two layouts are hardlinked for a bit. We have changed the repository layout in the past; Fedora 7 introduced the extra layer with the "Everything" directory, for example.

is there a good reason for the extra os in the directory ? Probably would be easier over time if the targets matched the contents Pungi outputs, which doesn't have an os in there.

looks like maybe I was mistaken because of the bad first FBR mohan sent earlier today.. looks like the os directory is there in the pungi output: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20180828.n.1/compose/Everything/x86_64/os/

Looks like the duplicated content dropped off my mirrors on Saturday. The two "Xtra" counts below show the files/directories which are on the local mirror but not on the master; those are deleted.

Sep 01 12:51:29 pubmirror1.math.uh.edu quick-fedora-mirror[20758]: Counts for fedora-enchilada: Svr:1572680/5366 Loc:1804678/5578 Diff:15905 New:0/0 Xtra:244841/216 Miss:12846/4 Size:61 Csum:26 Dtim:765
Sep 01 12:52:16 pubmirror1.math.uh.edu quick-fedora-mirror[20758]: Counts for fedora-secondary: Svr:1813954/6610 Loc:2024658/6826 Diff:12548 New:1/0 Xtra:221661/216 Miss:10960/0 Size:468 Csum:22 Dtim:674

I think this can be closed now. I might even be able to do that myself.

Metadata Update from @tibbs:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata