#4942 Unpacked git snapshot on fedorapeople
Closed: Fixed None Opened 8 years ago by tibbs.

This is more of a thing which might be nice to have but isn't remotely essential. If this is too crazy or useless, please consider just closing this ticket.

It would be nice to have a copy of the current pkgs git accessible from fedorapeople for grepping and overall inspection. We have nice web interfaces, but you can't run scripts on them. Downloading a git checkout seed just to grep things is kind of wasteful since it's nearly 6GB, and keeping a complete checkout up to date is a lot of bandwidth. Having these accessible from a command line somewhere without having to download everything could be quite useful.

It doesn't have to be a complete clone with full history, though obviously just unpacking the checkout seed or (if the network actually allows it) just having a readonly NFS mount of pkgs:/srv/git_seed/git-checkout would provide all of that.


Sadly people01 and pkgs02 are in different datacenters, so there's no easy way to use NFS or the like. ;(

We would have to copy over the seed and unpack it every day or something. ;(

Will see if I can come up with a more clever way to do this.

Maybe we could do somehing like:

  1. cd /srv/git-checkout
  2. for repo in /srv/$path_to_distgit/*; do git clone $repo || git pull $repo; done # this should use hardlinks for .git if /srv/git-checkout and distgit are on the same filesystem
  3. rsync everything from /srv/git-checkout without the .git dirs to fedorapeople

Sadly, I am not seeing any easy way to provide this.

I agree it might be handy, but fedorapeople is in another datacenter from pkgs, so it would incurr all the issues with bandwith and disk that any other remote copy would have. ;(

If someone can come up with a way to do this easily, please re-open.

What till suggested would work, though it perhaps could be optimized by doing a shallow clone. I'm not really sure if it's worth it, though. I don't actually think it would need to be done so often, but then again anyone who really needs to grep the entire thing could just pull the checkout seed and unpack it locally.

This leads me to wonder if we couldn't make another tarball that doesn't have the .git directories. It should be trivial and would almost certainly be much smaller. I guess I should file a separate ticket for that, though I also should just do the work myself and file the ticket when I'm done.

Maybe if that's done then implementing the stuff in this ticket would be much easier.

Login to comment on this ticket.

Metadata