@iwienand Huh, that was interesting. Anyway, I removed the setgid, could you please try it again and let us know.

Please reopen the ticket if its not fixed.

Thanks.

Just chatting with tibbs on IRC (as we use a mirror maintained by him at ut.edu) and it seems that the permissions persist on the master copy; and are therefore replicated on the mirror

There is some question if the quick-fedora-mirror will pull the new permissions, but that's probably secondary while the master permissions are incorrect.

I received a ping about this since my local mirrors still show some setgid bits, and so I just checked what I can see from batcave01:

[tibbs@batcave01 linux][PROD]$ ls -ld /srv/web/pub/fedora/linux/updates
drwxrwsr-x. 25 263 263 4096 Aug 14 22:04 /srv/web/pub/fedora/linux/updates

And within there, a few setgid directories still exist:

drwxrwsr-x.  2 263 263 4096 Mar 19  2013 13
drwxrwsr-x.  2 263 263 4096 Feb 24  2012 14
drwxr-sr-x.  2 263 263 4096 Mar 19  2009 8

Not sure what happened, or if that's related to the problem on updates/29 mentioned above (since 29 doesn't have the setgid bit set).

Just to confirm, however, our 2018-11-19T20:06:51 mirror run did NOT see

rsync: failed to set permissions on "/afs/.openstack.org/mirror/fedora/updates/29/.": Permission denied (13)

which suggests that the updated permissions on the parent directory were applied and mirrored correctly. I guess the question is the random-ish selection of remaining setgid bit directories

I looked on my mirror and there are nearly 2500 directories with the setgid bit set. I'm going to assume it's intentional since it's far more than just a few, but it's also oddly inconsistent.

I removed setgid bit on the parent directory that is updates/29 but I didn't think that they were set recursively. I will talk to @kevin about this and if its okay I will remove it recursively which should solve the issue.

@iwienand Could you please give it another try. It should be fixed now.

There is still a large number of setgid directories on the mirrors. So many, in fact, that I believe it may actually have been the expected and normal state of the mirrors at one point in time. There are plenty in alt and a load in archive, not to mention pub/fedora/linux/updates which may still cause problems for the reporter.

Rather than trying to fix these piecemeal (and worrying about whether they may come back after being fixed), it would be nice to have some decision about whether these things are allowed to be there or not, and if so then the rare affected clients will simply need to handle it. I think it could be handled relatively easily using the --chmod option to rsync with something like `--chmod=Dg-s', but I don't have an AFS setup to test.

We have indeed worked around it [1]; it just seemed like a weird thing to pop up so I wondered if it pointed to some sort of upstream mis-configuration.

[1] https://review.openstack.org/#/c/618504/

That seems like an odd workaround (just disabling -p entirely). Does the functionality rsync provides for this (--chmod) not work for your use case? It sure seems from the documentation like it would just let you just mask out the setgid bit on the permissions provided by the source mirror.

I know of no reason we would have set this... so as far as I am concerned we can just remove it from all directories/files.

It's set even on new content so I assume someone or something is doing it intentionally.

I was running find /srv/web/pub -type d -perm /2000 -ls but aborted it after a hundred or so hits because I didn't want to hammer the file server.

If not needed, we can just remove them recursively from top directories and that should fix it.

AFAIK, there are three things that put stuff on mirrors

1. RelEng when they manual sync the releases and stuff (include some scripts and manual work)
2. Nightly scripts
3. Bodhi syncs

If we can look at them and fix them, that should probably fix this issue.

I could also just have quick-fedora-mirror add the magic --chmod=Dg-s option, since I can't really understand what use that could possibly have on a downstream mirror. But without confirmation that it actually helps the AFS case, it's tough to say whether it's worth the effort. I really don't have the resources to set up AFS locally.

@mohanboddu believes this is fixed. If it happens again, please reopen this ticket.