#2241 Archive TCs/RCs
Closed: Fixed None Opened 10 years ago by robatino.

At present, the TCs/RCs are not archived (not even Alpha/Beta). There are bugs such as https://bugzilla.redhat.com/show_bug.cgi?id=597283 which only affect a few people with specific hardware and are therefore unlikely to get caught during testing. If the TCs/RCs (including Alpha/Beta) were available, it would be possible to narrow down their first appearance. Delta compression (such as deltaisos) could be used to reduce the space requirements. For example, going from F12 to F13, the size of the deltaisos for x86_64 install DVDs (listed at http://thepiratebay.org/user/andre14965/ ) were

  • F12->F13 Alpha TC1: 1.31 GiB
  • F13 Alpha TC1->TC2: 129.7 MiB
  • F13 Alpha TC2->RC1: 131.94 MiB
  • F13 Alpha RC1->RC2: 447.95 MiB
  • F13 Alpha RC2->RC3: 105.19 MiB
  • F13 Alpha RC3->RC4 (Alpha Final): 67.06 MiB
  • F13 Alpha -> Beta TC0: 251.8 MiB
  • F13 Beta TC0->TC1: 98.69 MiB
  • F13 Beta TC1->RC1: 172.81 MiB
  • F13 Beta RC1->RC2: 71.26 MiB
  • F13 Beta RC2->RC3: 61.13 MiB
  • F13 Beta RC3->RC4: 68.48 MiB
  • F13 Beta RC4->RC5 (Beta Final): 72.92 MiB
  • F13 Beta -> Final TC1: 368.25 MiB
  • F13 Final TC1->RC1: 225.94 MiB
  • F13 Final RC1->RC2: 62.52 MiB
  • F13 Final RC2->RC3 (13 Final): 65.62 MiB

Total size = 3.655 GiB which is approximately the same as one full ISO. The i386 sizes are similar. Deltaisos could also be produced from each DVD ISO to each CD ISO for the same version. Since the RPMs are identical, the deltaisos for these would be very small (basically just duplicating the non-RPM content). So archiving the DVD and CD install images for all the TCs/RCs would require about the space of two full ISOs.

The netinst images would have to be archived in full (or use a different form of delta compression) since they contain no RPMs, so deltaisos won't help. Same for the Live images.

There is already a ticket https://fedorahosted.org/rel-eng/ticket/3575 for providing deltaisos for testers. If the archive deltaisos were available quickly, the testers could use the archive and that ticket could be closed.

Of course, there is nothing special about deltaisos - any available efficient means of delta compression could be used.

At the very least, the checksum files for all the TCs/RCs should be archived since they are of negligible size, and if someone out there has a copy of the full ISO, they could upload it and it could be verified. There is the issue that the checksum files are presently unsigned (except for Alpha/Beta/Final) so some measures would have to be taken to prevent them from being faked.

I was mistaken about the netinst images - deltaisos appear to use delta compresion on the non-RPM content as well. This means that all the non-DVD install images can be basically archived for free. For example, here are the sizes of the deltaisos from the F13 x86_64 DVD ISO to each of the other F13 x86_64 install ISOs:

  • 11452784 Fedora-13-x86_64-DVD_disc1.diso
  • 46062 Fedora-13-x86_64-DVD_disc2.diso
  • 40093 Fedora-13-x86_64-DVD_disc3.diso
  • 34145 Fedora-13-x86_64-DVD_disc4.diso
  • 27370 Fedora-13-x86_64-DVD_disc5.diso
  • 20312 Fedora-13-x86_64-DVD_netinst.diso

So deltaisos can be used for archiving all the install images. Some other form of delta compression would be needed for the Live images, or they would have to be archived as full images, or not at all.

I've archived all deltaisos going from F13 to F14. For each primary arch (i386 and x86_64), the corresponding disos fit easily on a single DVD. From these I can rebuild any DVD, CD, or netinst ISO for any TC or RC between 13 and 14 Final. Since split media is gone for F15 and later, the DVD->CD disos are no longer needed.

The recent change in xz compression means that to use these disos, a system using F15 or later will have to temporarily downgrade its xz-* packages to the F14 versions before running applydeltaiso. This is a minor nuisance since these packages can be up/downgraded by themselves (no other dependencies).

Unfortunately, at the moment it appears that Rawhide/F15 consists of a mix of RPMs compressed using the old and new xz (for example, F14's BackupPC-3.1.0-16.fc14.x86_64.rpm and Rawhide's BackupPC-3.1.0-16.fc15.x86_64.rpm were both built on August 2 using the old xz compression). Unless this changes, it means that it will not be possible to build a functional diso from F14 to F15 Alpha TC1 (since rebuilding will fail using either old or new xz) or any later release such as F15 Alpha, Beta, or Final. Hopefully, at least disos where the old ISO is later than F14 should work (using the new xz), so I can archive F15 Alpha TC1 in full, then archive just the disos from that all the way to F15 Final.

I am now archiving at http://alt.fedoraproject.org/pub/alt/stage/deltaisos/ . All deltaisos between successive TCs/RCs for 14 and 15 development are available. The sizes of the DVD->netinst disos are negligible. The DVD->DVD disos for 14 -> 15 Alpha TC1 were nonfunctional due to the xz compression change, so the full 15 Alpha TC1 DVD ISOs are included. The sizes of the x86_64 DVD->DVD disos are as follows:

  • F13 -> F14 Alpha TC1: 1.57 GiB
  • F14 Alpha TC1->TC2: 80.497 MiB
  • F14 Alpha TC2->RC1: 220.97 MiB
  • F14 Alpha RC1->RC2: 96.28 MiB
  • F14 Alpha RC2->RC3: 69.03 MiB
  • F14 Alpha RC3->RC4 (Alpha Final): 64.65 MiB
  • F14 Alpha -> Beta TC1: 499.9 MiB
  • F14 Beta TC1->RC1: 196.4 MiB
  • F14 Beta RC1->RC2: 97.21 MiB
  • F14 Beta RC2->RC3 (Beta Final): 93.25 MiB
  • F14 Beta -> Final TC1: 483.2 MiB
  • F14 Final TC1->RC1 (14 Final): 185.86 MiB

  • F14 -> F15 Alpha TC1: 3.396 GiB (full ISO, diso nonfunctional due to xz change)

  • F15 Alpha TC1->TC2: 590.86 MiB
  • F15 Alpha TC2->RC1: 331.38 MiB
  • F15 Alpha RC1->RC2 (Alpha Final): 271.26 MiB
  • F15 Alpha -> Beta TC1: 750.38 MiB
  • F15 Beta TC1->RC1: 530.59 MiB
  • F15 Beta RC1->RC2 (Beta Final): 258.04 MiB
  • F15 Beta -> Final TC1: 569.08 MiB
  • F15 Final TC1->RC1: 505.1 MiB
  • F15 Final RC1->RC2: 195.69 MiB
  • F15 Final RC2->RC3 (15 Final): 195.4 MiB

The sizes for the i386 disos are similar. Notice the average size increase in the F15 disos (even when only a few RPMs have changed). The new minimum size seems to be around 200 MiB instead of 70 MiB which is about the same as the size of the netinst ISOs. I suspect that the non-RPM content of the DVD ISOs is no longer delta-compressing very well, possibly due to the changes in the format of these images.

Closing since I'm currently doing this, and it's easy enough that I have no problem with it. May still add comments occasionally concerning the increased size issue - reported as https://bugzilla.redhat.com/show_bug.cgi?id=722987 , BTW - and as a place to point people to as a description of how the archiving is done.

Login to comment on this ticket.