#39 faq update + stream expansion page
Merged 8 months ago by asamalik. Opened 8 months ago by asamalik.
fedora-docs/ asamalik/modularity faq-and-stream-expansion  into  master

empty or binary file added
file modified
+3 -2

@@ -1,10 +1,11 @@ 

  * xref:architecture.adoc[Architecture]

  ** xref:architecture/building.adoc[Building Software]

- *** xref:architecture/building/name-and-id.adoc[Module name and ID]

  ** xref:architecture/consuming.adoc[Consuming Software]

  *** xref:architecture/consuming/dnf-behavior.adoc[DNF Behavior]

- *** xref:architecture/consuming/naming-policy.adoc[Naming Policy]

  *** xref:architecture/consuming/upgrade-paths.adoc[Upgrade Paths]

+ ** xref:architecture/building/name-and-id.adoc[Module name and ID]

+ *** xref:architecture/consuming/naming-policy.adoc[Naming Policy]

+ ** xref:architecture/stream-expansion.adoc[Stream Expansion]

  * xref:making-modules.adoc[Making Modules]

  ** xref:making-modules/adding-new-modules.adoc[Adding New Modules]

  ** xref:making-modules/updating-modules.adoc[Updating Existing Modules]

@@ -0,0 +1,17 @@ 

+ = Module Stream Expansion


+ Stream Expansion allows for automatic submission of multiple builds of the same component based on multiple versions of its build dependencies.


+ == Build


+ image::stream-expansion-build.png[,100%,]


+ One module can be built against multipe Fedora releases, or against multiple streams of other modules such as language runtimes, producing a matrix of binaries for all defined combinations.


+ These are the same modules of the same stream, but of a different context.


+ == User view


+ image::stream-expansion-user-view.png[,100%,]


+ User only sees the modules and their streams, but the context is hidden. The client tooling chooses the right streams of dependencies based on the version of system the user is running, what dependencies have been already installed or enabled by the user, or based on defaults. 

\ No newline at end of file

file modified
+23 -16

@@ -2,27 +2,32 @@ 



  Exactly what problem are you trying to solve?::

-     The “too fast vs. too slow” problem, outdated containers, and complex packaging workflow.

-     See The Problem to be Solved page for more detailed information.

+     Deploying software has many solutions, but what gets deployed often plays out as a fight between developers and operators. Developers want the latest (or at least later) features. Operators want software in packages, certified, with a known period of support. Fedora Modularity provides multiple versions of packages in a Linux distribution with the qualities expected from a Linux distribution: transparently built and delivered, actively maintained, and easy to install — making both happy.

  How is this different from containers?::

-     This is a build step that happens before containers.

-     Modularity provides parallel availability of multiple versions that can be consumed by containers to enable parallel installation.

+     Containers are a solution to a very different problem. They allow you to bundle your application and all of its dependencies in a container image that you can run in isolation on basically any Linux system with a container runtime. +

+      +

+     Modularity provides multiple versions of packages in a Linux distribution with the qualities expected from a Linux distribution: transparently built and delivered, actively maintained, and easy to install. +

+      +

+     Modularity and containers work nicely together — since you need to get the software (such as your app's dependencies) from somewhere. Getting them from a Linux distribution (and now with the choice of version thanks to Modularity) makes building and updating containers (with security updates for example) easier.

  How does this differ from Software Collections (SCLs)?::

-     Modularity is about building multiple versions of packages, and making them all available to users.

-     SCLs are about installing software in parallel using RPM.

-     These two don’t overlap, they complement each other. You can build a modular SCL.

+     SCLs use a different method of packaging allowing for multiple versions of the same piece of software

+     to be installed on one system, by putting them into separate namespaced paths. Modularity on the other hand

+     uses standard RPM packaging — so things are where you expect them to be — but you can only install one version at a time. +

+      +

+     SCLs have proven to be hard to maintain and hard to use (Special macros in spec files, package name mangling, running 'scl enable' in order to make them visible).

+     And the ability to install multiple versions in parallel has turned out not to be a common use case.

+     The real benefit of SCLs was the ability to choose a specific version of software — and that's exactly what Modularity offers.

  Couldn’t we solve this with having compat packages, or putting versions into package names for different version streams?::

-     That works technically, but the packaging workload doesn’t scale, and dependency interactions become complex. This will let packagers make more content available to users without an explosion of effort.

-     Modules also help to solve the discoverability problem.

-     For example, is httpd v2.4 in a package called httpd-2-4 or http-2?

+     That could work for some scenarios, however, it would introduce an unnecessary complexity to the whole system by adding even more packages to choose from. You would also need to encode information such as the major version (stream) in the package name, in a similar way how SCLs do that. How would you distinguish packages built against different versions of runtimes or other dependencies? +

+      +

+     Modularity groups packages into modules — a representation of an application, a language runtime, or any logical group. That makes the content more granular and easier to navigate. Modules can be available in multiple streams — usually representing a major version of the software they ship. And technologies such as xref::architecture/stream-expansion.adoc[Stream Expansion] allow packagers to automatically build their module against multiple streams of runtimes, or entire Fedora releases — significantly lowering the manual effort. And for users, the client tooling is smart enough to choose the right combinations without exposing the underlying complexity by default — making it easier to use.

  Can I install more versions at once?::

-     Modularity does not invent a new technology for parallel installation.

-     Existing solution such as containers or Software Collections can be used to achieve this.

+     As modularity uses standard RPM packaging in order to install software in the usual places, the answer is no. However, there are other existing technologies — such as containers — allowing exactly that and work well with Modularity.

  How is this different from RPM? Why not just different repos?::

-     Technical: DNF performance with many repositories. We envision having tens or even hundreds of modules available to a single system. +

-     User experience: A repository is a source of software provided by a vendor, not necessarily representing a single application, language stack, or any other logical unit of software.

-     A repository typically contains multiple applications, stacks, etc.

-     We do not want to change this concept.

+     Modularity uses RPM as the core building block. From the user perspective, replacing modules with repositories could be technically possible, however: +

+     A) Modules as a core concept in the client tooling significantly simplifies usage. Users can easily query for the modules available and install them with a simple command, without the need of manually enabling repositories. +

+     B) From our perspective, a repository is a source of software provided by a vendor, not necessarily representing a single application. We want to keep this principal and allow multiple applications to be present in a repository. +

+     C) Modularity also helps packagers to automatically submit multiple module builds at once based on their build dependencies, producing multiple combinations of binaries. And the client tooling then chooses the right binary when installing. See xref::architecture/stream-expansion.adoc[Stream Expansion] for more info.

  Are you going to produce all versions? What life cycles and versions are envisioned?::

      We are developing the technology such as the build pipeline and client tooling to enable contributors to build multiple versions.

      We envision longer life cycles of stacks mainly for server, and devel/rolling releases of some stacks for developers.

@@ -32,5 +37,7 @@ 

      Modules, on the other hand, represent a major version that is still being actively maintained by the upstream and therefore should receive updates.

  Is this different from single app VMs with custom versions?::

      This helps with a single app VM with custom versions.

+     But instead of getting them from somewhere and maintaining them yourself,

+     there is a good chance a module with the version you need already exists and is maintained — so you can just install it.

      Same as with containers, it helps users to get the right version for their system.


I have updated the FAQ with answers that better match the current state of things and are also more descriptive.

As I had to mention Stream Expansion and I couldn't find a page about it, I've created it as well. So that's why it's weirdly in a single PR.

I've built previews:

"or the whole Fedora releases" -> "or entire Fedora releases"

"depencies" -> "dependencies"

1 new commit added

  • fix minor typos
8 months ago

Thanks for the quick review, @sgallagh!

Pull-Request has been merged by asamalik

8 months ago