If you're a pre-bitflip mirror, it's possible to miss the bitflip if the file lists aren't regenerated immediately when the flip happens. And it's possible to miss updates to existing non-checksummed files because of file list update delays.

Here's the bitflip case:

• Bit flip happens at time T but file lists aren't immediately updated
• Mirror polls at time U; sees no changes.
• File lists are regenerated at V > U > T. File list now contains change time T for the relevant directory.
• Mirror polls again at W, gets new file list, but since the last mirror time U > T, it doesn't see any change there and so doesn't include the directory in the transfer list.

Plus, even if it does get into the transfer list, our directory times aren't necessary kept in sync so rsync might not transfer the changes anyway.

Mirrors which just get unrestricted content will simply see the files appear and, because it doesn't have them locally, transfer them regardless of timestamp.

In fact, this is a more general problem if the content of a file changes and the file lists aren't updated before a client next polls. If it takes X minutes for the host to generate a file list after the change, the client must look back for the next X minutes for updated files, regardless of how often it polls.

A possible solution is to somehow sort/diff the file list and unconditionally transferring any changes. In most cases we will have the previous file list handy so this shouldn't actually be too difficult.

So it comes down to have an extra step:

• Extract timestamp/path pair from both old and new file lists (or timestamp/size/path tuple), sorting both by the path and diffing. In fact, it should be possible to use the existing awk syntax to ust pull the [files] section and use that directly. This should already be done for the new file lists.
• Using sort -t$'\t' -k2 (or -k3 depending) to sort by the path.
• Using diff --changed-group-format='%>' --unchanged-group-format='' old new makes it possible to avoid postprocessing the diff output too much, though you still need to extract just the path from the set of changes.

This should catch permission changes whenever they happen, as well. It will in fact basically duplicate some of the existing change detection. But that's OK; we want to be thorough and we will deduplicate the transfer list at the end of the process.