#112 Discussion: Module lifecycles
Closed: Fixed 23 days ago by asamalik. Opened 5 months ago by asamalik.

Discussion for the #3 Module life cycles epic.

This is very packager focused. I want to design the packager experience first before diving into implementation details.

Traditional branching and lifecycles:

There is a branch for every release. To ship a specific version of a package for a specific Fedora release, you simply push the right source into the right branch in dist-git.

Dist-git branches:

  • nodejs 8 source -> "f27" branch
  • nodejs 8 source -> "f28" branch
  • nodejs 10 source -> "f29" branch
  • nodejs 10 source -> "master" branch

What's available:

  • Fedora 27: nodejs 8
  • Fedora 28: nodejs 8
  • Fedora 29: nodejs 10
  • Fedora rawhide: nodejs 10

Lifecycle expectation: Package is being branched for new releases and built forever until retired in rawhide.

Very simple, yet limited: you can't have multiple versions in one release. And you might be pushing the same source into multiple branches.

Modular branching and lifecycles:

There are stream branches — structured in any way the packager desires — but typically based on major versions. You push each version only once to its respective branch. But what goes where?

Modules are a way of defining what goes where. One version can go to multiple Fedora releases, and one Fedora release can receive multiple versions. And defaults are a way of deciding which of the multiple version is available as a default — so traditional workflows keep functioning.

Dist-git branches:

  • nodejs source 8 -> "8" branch
  • nodejs source 10 -> "10" branch

NEW: Module definitions:

  • nodejs 8 -> Fedora 27, Fedora 28, Fedora 29, Fedora 30 which is rawhide now
  • nodejs 10 -> Fedora 28, Fedora 29, Fedora 30 which is rawhide now

NEW: Default definitions:

  • Fedora 27: nodejs 8
  • Fedora 28: nodejs 8
  • Fedora 29: nodejs 10
  • Fedora 30 (rawhide now): nodejs 10

What's available:

  • Fedora 27: nodejs 8 by default
  • Fedora 28: nodejs 8 by default, but also nodejs 10
  • Fedora 29: nodejs 10 by default, but also nodejs 8
  • Fedora rawhide: nodejs 10 by default, but also nodejs 8

However, we don't want you to manually list releases in your module definition and requiring you to update it every time there is a new release.

That's why you can declare:

  • nodejs 8 -> all Fedora releases
  • nodejs 10 -> all Fedora releases

But what to do when nodejs 8 reaches its EOL (end of life)? We still don't want to ask you to suddenly list "all Fedora releases up until now" in the module definition so it stops building for the next one. We need a way for you to define an EOL so the build system can figure this out automatically.

We also want to keep the Lifecycle expectation: Package is being built forever until retired.

Proposal

Edit 2019-02-20: A complete proposal: https://pagure.io/modularity/working-documents/blob/master/f/lifecycles-upgrades-ownership/lifecycles-general.md


Metadata Update from @asamalik:
- Issue tagged with: Meeting

5 months ago

Metadata Update from @asamalik:
- Issue assigned to asamalik

5 months ago

IMO EOL information can be where all SCM data is kept, in git.

I think that's generally reasonable, but:

  • We don't really have a place to store the EOL at the moment. It used to be PDC but...
  • Both GA and release EOL dates can slip

I think the easiest option would be just relying on the platform dependencies here but we'd like some input from other groups, such as FESCo and releng. I'll file a FESCo ticket for this.

we'll discuss this one with FESCo

Note we are building a PDC replacement called fpdc: https://fpdc.fedoraproject.org/ https://github.com/fedora-infra/fpdc It could well contain this info.

I believe that module EOL has to correspond to a particular Fedora release. We can't leave users hanging with no security updates should a vulnerability be newly discovered after module EOL.

There was some suggestion at todays FESCo meeting that people wanted to store EOL data and the like in dist-git directly.

What PDC/FPDC data do we currently use and would it make sense to store some or all in dist-git?

CC @cverna @abompard

There was some suggestion at todays FESCo meeting that people wanted to store EOL data and the like in dist-git directly.
What PDC/FPDC data do we currently use and would it make sense to store some or all in dist-git?
CC @cverna @abompard

So currently in FPDC we store the following information about releases see --> https://fpdc.stg.fedoraproject.org/api/v1/release

I did not read the full proposal so I am not sure if that would be useful or not :smile:

I believe that module EOL has to correspond to a particular Fedora release. We can't leave users hanging with no security updates should a vulnerability be newly discovered after module EOL.

I'm with @jforbes. More onto the same topic:

Let's say a EOL of module stream is scheduled to 2020-01-01. When this day comes, what actually happens? If we just stop building, users are left with unmaintained (possibly defective or dangerous) software. How do we inform them? Do we remove the software (that's wicked)? Do we leave them in false sense of security and stability (that's evil)? In theory, if we could display the information in a very easily discoverable way (aka the users get constant notifications about the EOL when it approaches), then maybe. But that's not the case.

Also, IMHO defining the EOL as a date but making it effective aligned with Fedora release EOL is confusing (both for maintainers and for "readers" of such information). Hence:

I propose there is a way to declare an EOL for module streams in only one possible format: A specific Fedora release (as in: the last Fedora release that will have this module).

Side note: What happens with actuall installations after the stream EOLs? Traditionally, when we retire packages, we either let them rot (I hate that) or we obsolete them from fedora-obsolete-packages (and remove them from users installations, other people hate that). IIRC the current guideline is: obsolete them if they can cause broken dependencies or similar problems.
Dependency problems can happen with modules as well: Suppose I have a nodejs-8 module-stream that builds against system version of libuv (artificial example). As long as nodejs-8 is supported (before EOL), we rebuild it for any Fedoa release. If libuv is bumped, we rebuild nodejs-8. However, once nodejs-8 goes EOL in Fedora N and we bump libuv in Fedora N+k, we no longer rebuild nodejs-8 and the users who have it installed/enabled will have trouble upgrading to Fedora N+k.

Packages that are not in a module are not being built, so they don't really need an EOL declaration, because they don't exist from the user's perspective.

This assumes that everything is in modules. That's not really the case yet.

I propose there is a way to declare an EOL for module streams in only one possible format: A specific Fedora release

Yeah, I think that's reasonable.

We could provide an alternative format where a date is specified, with the assumption that if the EOL of the Fedora release slips, the EOL date of the module slips automatically too. But that might be too much magic. So if people are strongly on the side of "a specific fedora release" I can agree with that.

Side note: What happens with actuall installations after the stream EOLs?

I think we should follow the same policy as with "ursine" packages, but with the preference to add Obsoletes for EOL streams more aggressively than for ursine packages. I.e. unless there's a good reason to allow people to keep the EOL stream, add the stream to fedora-obsolete-package.

Can nonmodular package even obsolete modular packages?

To move this forward, I’m updating the proposal with a specific format of the EOL definition. This is only about the format, I’ll follow up with where to store it and how to set / update the information, including a policy around all of that.

Proposal of the lifecycle format

I propose that a module lifeycle is defined as an EOL (end of life) information mapping to a specific Fedora release and/or RHEL minor release for EPEL.

A single module stream will have two lifecycle definitions associated with it, one for Fedora and one for EPEL. Separation of these two is necessary as their EOL are not aligned, and I want to prevent module streams reaching their EOL mid-release.

Besides a specific EOL, I’m also allowing two additional values: “build forever” and “don’t build”.

The format could look similar to the following pseudo-syntax:

EOL:
    - Fedora: <VAL>
    - EPEL: <VAL>

… where <VAL> is either True to indicate “build forever”, a specific release such as 35 or 8.5 to indicate an EOL, or False to indicate “don’t build”. Not setting an EOL would implicitly default to True for both Fedora and EPEL.

Because we might not want to store a boolean value and a string in a single field, the format could be extended to separate them. However, the exact syntax is not necessarily in scope here.

Scenarios

I include some example scenarios to demonstrate why I propose such format.

Scenario 1: Fedora forever

“I want to build this module stream only for Fedora (not EPEL) forever.”

EOL:
    - Fedora: true
    - EPEL: false

... or:

EOL:
    - EPEL: false

Scenario 2: Fedora until f35

“I want to build this module stream only for Fedora (not EPEL) but only until Fedora 35.”

EOL:
    - Fedora: “f35”
    - EPEL: false

Scenario 3: Only EPEL until 8.5

“I want to build this module stream only for EPEL (not Fedora).”

EOL:
    - Fedora: false
    - EPEL: “8.5”

Scenario 4: Fedora until f35, EPEL until RHEL 8.5

“I want to build this module until Fedora 35 and RHEL 8.5.”

EOL:
    - Fedora: “f35”
    - EPEL: “8.5”

Scenario 5: Everywhere forever

“I want to build this module stream forever in both Fedora and EPEL. I might set a specific EOL value later.”

EOL:
- Fedora: True
- EPEL: True

Providing no EOL information would have the same effect.

EOL: False

I read as: EOL did not happen yet.

EOL: True

I read as: already EOL

Yeah, the reverse meaning with EOL is too confusing. I tried to figure out a mechanism where different terms are used, keeping the "EOL" section, but this is hard. The problem is that "EOL" is a negative term, i.e. something like all those NoFeature= options. It's usually much better to switch the option name to positive, i.e. Feature=. In this particular case, the opposite of EOL is "Support" or "Build-until" or something like this. Your five scenarios then become:

# 1
Build-until:
     - Fedora: "always"
# 2
Build-until:
     - Fedora: "f35"
# 3 
Build-until:
    - EPEL: “8.5”
# 4
Build-until:
    - Fedora: “f35”
    - EPEL: “8.5”
# 5
Build-until:
    - Fedora: “always”
    - EPEL: “always”

What about not setting it would mean "build forever" and setting it to 0 would mean "don't build"?

# 1
EOL:
     - EPEL: "0"
# 2
EOL:
     - Fedora: "f35"
     - EPEL: "0"
# 3 
EOL:
    - Fedora: "0"
    - EPEL: “8.5”
# 4
EOL:
    - Fedora: “f35”
    - EPEL: “8.5”
# 5
(nothing) 

So, to comment on the intent (rather than the syntax), the proposal is essentially:
Allow maintainers to specify that a module stream will live until the EOL date of a particular Fedora release or EPEL minor release, with special cases for "just keep building until I say otherwise"?

If so, I think that makes good sense and I'm +1 on it. The exact format we store this data in can be worked out once we agree on that.

+1 to @sgallagh

What about not setting it would mean "build forever" and setting it to 0 would mean "don't build"?

This still has the problem that what is not shown is more important than what is. E.g. imagine that we want to add a new release output, e.g. EPEL-for-opensuse. Now all modules are implicitly declared to support it. This is bound to be confusing to newcomers and even to advanced packagers who do a quick glance.

Thanks for the feedback.

This still has the problem that what is not shown is more important than what is. E.g. imagine that we want to add a new release output, e.g. EPEL-for-opensuse. Now all modules are implicitly declared to support it. This is bound to be confusing to newcomers and even to advanced packagers who do a quick glance.

@zbyszek Good point, however, look at it from the other side. We're not very likely to introduce things like EPEL-for-opensuse and such that often, but we do expect people to create new modules or module streams. The way I propose to handle the EOL means that their new module just builds, without the extra step of marking it as "not EOL" that you propose. Not introducing new steps (or even removing or simplifying the existing) for newcomers is quite important to me.

Once set, these values may be extended (enabling support on later releases), but may not be shortened.

sgallagh: mhroncok: Can you explain what you found creepy about the lifecycle proposal?
mhroncok: sgallagh: that once you say the EOL is in Fedora 38, you cannot reconsider and move to Fedora 35
sgallagh: mhroncok: that’s to your consumers’ benefit. You are giving them a promise that this stream will be around for N months
sgallagh: Allowing you to shorten that severely devalues this concept as a promise
mhroncok: there should be a way to change it
mhroncok: a documented, loud process, but still
sgallagh: mhroncok: Put that in the FESCo ticket for discussion then.
mhroncok: sgallagh: if I change the EOF to never, is that a promise?
sgallagh: No
mhroncok: especially since never is the default
sgallagh: “Never” is a special case. It’s a *lack* of a promise
sgallagh: I can clarify that in the proposal. Please provide feedback on the ticket and I’ll get to it
mhroncok: so as a maintainer, I won't set the eol even if I'm 80 % sure of it, because I would not be bale to change it later
sgallagh: You would be able to make it later.
sgallagh: So a best bet would be to set it sooner and push it out if you decided to
sgallagh: That’s better for your users; they will see it as a bonus
Pharaoh_Atem: sgallagh: I don't know, I think I'm with mhroncok here
Pharaoh_Atem: I'd err on the side of never specifying it
Pharaoh_Atem: and only specify it when it's about to get terminated
sgallagh: That’s useful feedback. Please write it down on the ticket and we will discuss it :-)
mhroncok: sgallagh: I will
sgallagh: Thanks

(I don't want to forget this, but I don't have time to rewrite it now, hence the IRC dump, sorry about that.)

So, the main takeaway I got from this conversation is that packagers might tend to largely ignore this configuration and leave the EOL entirely unset until such time as they decide to kill off a stream. We might need to make it clear that an unset EOL carries with it no implicit or explicit lifecycle guarantee; realistically the user can only assume that it will be available on the current release they are running.

I don't see this as a problem, honestly. It allows packagers to opt-in to asserting how long a stream is going to be maintained. If they want to indicate that it will be supported for multiple releases, they have that option. If not, they're indicating that it may go away at any time, which is also useful information to a user.

I think that we provide a process to lower the EOL somehow, at least for exceptional cases. I would even be OK to say that in exceptional cases, this can be lowered with and approved exception from the Modularity WG.

I think that we provide a process to lower the EOL somehow, at least for exceptional cases. I would even be OK to say that in exceptional cases, this can be lowered with and approved exception from the Modularity WG.

I'm fine with that. I usually just work under the assumption that any rule is changeable. But yes, we should ensure that whatever mechanism can update these values has an override for exceptional cases.

Stupid question: Why is pizza module tasty not pictured in epel7 when it doesn't specify eol at all? Because epel7 doesn't have modules?

Stupid question: Why is pizza module tasty not pictured in epel7 when it doesn't specify eol at all? Because epel7 doesn't have modules?

That's correct.

AGREED: The proposal is approved with the additional requirement that the maintainer can update the information at any time, automatically, without releng or any other human involvement, assuming the update is valid based on policy (+6, 0, 0)

Metadata Update from @asamalik:
- Issue untagged with: Meeting

23 days ago

I have added the additional requirement to the proposal, making it final. Closing this issue. I might open new ones to discuss the next steps.

Metadata Update from @asamalik:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

23 days ago

Login to comment on this ticket.

Metadata