#7215 older drpms not synced
Opened 3 years ago by kevin. Modified 7 months ago

As noted in this post:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/7RQL6FUZNZPYGDBVWDMBPUD3E2HYY7YB/

it seems we are only syncing out the drpms made in that specific compose. We should keep a (ideally configurable) amount of older ones around. I think for rawhide we were keeping 2 weeks, we could start with that.

as it is now unless someone updates on the day an update is pushed they won't get the advantage of the drpm.


I could be wrong on this, but I think we used to keep all deltarpms used to build to the current rpm. For releases, that was GA→n and n-1→n, while for rawhide this was n-1→n.

Metadata Update from @syeghiay:
- Issue assigned to mohanboddu

3 years ago

@mohanboddu will take a look at this to see if it's still an issue.

Metadata Update from @kevin:
- Issue tagged with: backlog

2 years ago

Closing this ticket now as it may no longer be an issue, however please feel free to reopen if it is.

Metadata Update from @amoloney:
- Issue close_status updated to: taiga
- Issue status updated to: Closed (was: Open)

2 years ago

This is still an issue, but I'm not sure that it's worth the effort to re-open. I half wonder if we should just stop generating drpms entirely.

People, please, let's fix this and save the world 85% of the petabytes of bandwidth that are associated with all the world's Fedora installations doing their regular updates.

jdieter says he half wonders we should just stop generating drpms entirely. Why? Aren't they entirely a good thing? They work, we already have the technology, so it's an easy savings. Just because the Internet has gotten so fast that the average user hardly notices the time/bandwidth used by dnf upgrades anymore doesn't mean there is no cost involved. It's just spread around so thinly that everybody just shrugs it off... but it's there, it's a non-trivial cost to the world, in terms of bandwidth and energy and thus money and GHG emissions.

People, please, let's fix this and save the world 85% of the petabytes of bandwidth that are associated with all the world's Fedora installations doing their regular updates.

It's not that simple. :)

Sure, you save BW, but you spend more in CPU and disk I/O to reassemble the rpms on every client machine.
You also make updates pushes slower and use more resources.

Anyhow, I'm open to making more drpms, but we will have to try and figure out how to do that, it's not just a switch to flip, it's deep changes in the updates compose pipeline.

Metadata Update from @kevin:
- Issue status updated to: Open (was: Closed)

a year ago

but you spend more in CPU and disk I/O to reassemble the rpms on every client

OK, that's a good point; I had just assumed that the bandwidth costs were greater, probably because it wasn't so long ago that I was stuck with a lousy Internet connection for a few years and drpms were really saving me a lot of pain then. But I haven't done the math. That isn't so easy to do for the bandwidth related costs... the drpm reassembly costs are easier to ballpark estimate, and my gut says they're significantly smaller, but I have to admit that I don't really know. In any case, there are still people with not-so-great Internet connections in the world, so for their sake alone I think this is worth giving some attention to. What I do know is that when it mattered to me, the existence of delta rpms alone was a major argument for using Fedora over other distributions.

Believe me, I'm with you. I wrote the original yum plugin that allowed you to use deltarpms in Fedora. My concern with our current situation is that we're throwing away all old deltarpms at the end of every compose. In other words, if LibreOffice is updated on Monday, deltarpms are generated, but on Tuesday those deltarpms will be thrown away. That means that the only way to take full advantage of deltarpms in Fedora is to update every single day.

My preference would be to figure out how to keep old deltarpms, but, if we're not going to do that, I'm not sure that it's worth the extra compose time to build them.

Metadata Update from @mohanboddu:
- Issue tagged with: meeting

a year ago

@lsedlar What would you think of adding the following to pungi?

  • Look for old composes for this current compose and gather up all the drpms from them.
  • discards drpms older than X days
  • put those drpms along with the ones createrepo_c creates in our compose in drpms dir

This wouldn't add any overhead to updates pushes, the drpms are already created.

I guess the tricky part is finding them and making sure they are put in the right place.

Ugh, so I guess that would also mean that createrepo_c would need to somehow know about those drpms and update the repodata. ;( Or pungi would have to...

Metadata Update from @humaton:
- Issue tagged with: mini-initiative

8 months ago

Login to comment on this ticket.

Metadata
Boards 2
Mini Initiative Status: Backlog
mini-initative Status: Backlog