fedora-docs / modularity

Clone
Source Code
GIT

#3 more content cleanup & adding placeholders for user & packager docs; restructuring a bit
Merged 6 years ago by asamalik. Opened 6 years ago by langdon.
fedora-docs/ langdon/modularity master  into  master

file modified
+5 -11 
@@ -29,6 +29,10 @@ 
 

  #    Alias: en-US/brian
 

    - Name: What is Modularity
 

      File: index
 

+   - Name: Making Modules
 

+     File: Making_Modules
 

+   - Name: Using Modules in Fedora
 

+     File: Using_Modules
 

    - Name: Architecture
 

      Dir: architecture
 

      Topics:
 
@@ -38,21 +42,11 @@ 
 

          File: Building_Software
 

        - Name: Consuming Software
 

          File: Consuming_Software
 

-   - Name: For Contributors
 

+   - Name: For Modularity Contributors
 

      Dir: contributors
 

      Topics:
 

        - Name: Communication
 

          File: Communication
 

-   - Name: For Packagers
 

-     Dir: packagers
 

-     Topics:
 

-       - Name: Defining a Module
 

-         File: test
 

-   - Name: For Users
 

-     Dir: users
 

-     Topics:
 

-       - Name: Managing Streams using DNF
 

-         File: test
 

    - Name: References
 

      File: References
 

    - Name: FAQ

file modified
+7 -7 
@@ -2,7 +2,7 @@ 
 

  

  [qanda]
 

  Exactly what problem are you trying to solve?::
 

-     The “too fast vs. too slow” problem, outdated containers, and complex packaging workflow. 

+     The “too fast vs. too slow” problem, outdated containers, and complex packaging workflow.
 

      See The Problem to be Solved page for more detailed information.
 

  How is this different from containers?::
 

      This is a build step that happens before containers.
 
@@ -12,16 +12,16 @@ 
 

      SCLs are about installing software in parallel using RPM.
 

      These two don’t overlap, they complement each other. You can build a Modular SCL.
 

  Couldn’t we solve this with having compat- packages, or putting versions into package names for different version streams?::
 

-     That works technically, but the packaging workload doesn’t scale, and dependency interactions become complex. This will let packagers make more content available to users without an explosion of effort. 

-     Modules also help to solve the discoverability problem. 

+     That works technically, but the packaging workload doesn’t scale, and dependency interactions become complex. This will let packagers make more content available to users without an explosion of effort.
 

+     Modules also help to solve the discoverability problem.
 

      For example, is httpd v2.4 in a package called httpd-2-4 or http-2?
 

  Can I install more versions at once?::
 

-     Modularity does not invent a new technology for parallel installation. 

+     Modularity does not invent a new technology for parallel installation.
 

      Existing solution such as containers or Software Collections can be used to achieve this.
 

  How is this different from RPM? Why not just different repos?::
 

      Technical: DNF performance with many repositories. We envision having tens or even hundreds of modules available to a single system. +
 

-     User experience: A repository is a source of software provided by a vendor, not necessarily representing a single application, language stack, or any other logical unit of software. 

-     A repository typically contains multiple applications, stacks, etc. 

+     User experience: A repository is a source of software provided by a vendor, not necessarily representing a single application, language stack, or any other logical unit of software.
 

+     A repository typically contains multiple applications, stacks, etc.
 

      We do not want to change this concept.
 

  Are you going to produce all versions? What lifecycles and versions are envisioned?::
 

      We are developing the technology such as the build pipeline and client tooling to enable contributors to build multiple versions.
 
@@ -31,6 +31,6 @@ 
 

      Using an older version that is no longer maintained can be dangerous, mostly because of the lack of security updates.
 

      Modules, on the other hand, represent a major version that is still being actively maintained by the upstream and therefore should receive updates.
 

  Is this different from single app VMs with custom versions?::
 

-     This helps with a single app VM with custom versions. 

+     This helps with a single app VM with custom versions.
 

      Same as with containers, it helps users to get the right version for their system.

file added
+29 
@@ -0,0 +1,29 @@ 
 

+ = Making Modules
 

+ 

+ == Overview of module files
 

+ 

+ explain module.yaml, what it should be named, what the branch should be called, where to find the spec, some example
 

+ 

+ == How to generate a module
 

+ 

+ using fedmod
 

+ 

+ == Requesting a new module
 

+ 

+ mostly a link to the process in the wiki
 

+ 

+ == Requesting a new module stream
 

+ 

+ mostly a link to the process in the wiki
 

+ 

+ == Testing your module
 

+ 

+ how to do a local build, why there is no scratch build, using odcs with your new module
 

+ 

+ == Publishing your module
 

+ 

+ how to do a mbs build
 

+ 

+ 

+ 

+

file added
+31 
@@ -0,0 +1,31 @@ 
 

+ = Using Modules in Fedora
 

+ 

+ == Enable Modularity on Fedora 28
 

+ 

+ how do you turn on the modular repo? is it there by default?
 

+ 

+ == Find a Module
 

+ 

+ list & search for modules with dnf
 

+ 

+ == Discover & use installation profiles
 

+ 

+ dnf module info ?? are we gonna get anything better?
 

+ 

+ == Enable a Module
 

+ 

+ enable (& install?) a module
 

+ 

+ == Your System Profile
 

+ 

+ how to view the system profile, what you can change about install
 

+ 

+ == Using a module from a container
 

+ 

+ inject a system profile with the correct versions; dnf install blah
 

+ 

+ == How to ask for a new module or stream
 

+ 

+ enable (& install?) a module
 

+ 

+

file modified
+8 -8 
@@ -6,10 +6,10 @@ 
 

  

  image::mod-doc-trad-builds.png[,80%,]
 

  

- In the existing Fedora world, packages are maintained in branches that map to specific Fedora releases. 

+ In the existing Fedora world, packages are maintained in branches that map to specific Fedora releases.
 

  Package repositories have branches e.g. “f27” for Fedora 27, “f28” for Fedora 28, etc.
 

  

- There are no changes to the existing packager workflow. 

+ There are no changes to the existing packager workflow.
 

  Maintaining release-specific branches and using the existing commands to build them keep working.
 

  

  == Modular package builds
 
@@ -20,14 +20,14 @@ 
 

  

  Branching packages according to their major version as opposed to the distribution release brings more clarity — because we can see what lives in a branch, and removes source duplication — because there is one branch per version.
 

  

- While this approach makes more sense from the software perspective, there is no longer a clear mapping of a package branch to a particular release. 

+ While this approach makes more sense from the software perspective, there is no longer a clear mapping of a package branch to a particular release.
 

  Modules are a way of defining this.
 

  

  == Building one source for multiple releases
 

  

  image::mod-doc-build-more-releases.png[,40%,]
 

  

- Modules define what package branches are built for which Fedora releases. 

+ Modules define what package branches are built for which Fedora releases.
 

  One module can be built for multiple releases.
 

  

  The above example produces three different binaries, one for each release. In case there are multiple architectures, three binaries will be produced (one each).
 
@@ -36,10 +36,10 @@ 
 

  

  image::mod-doc-build-more-versions.png[,40%,]
 

  

- Multiple versions of packages can be built for one release by defining multiple modules. 

+ Multiple versions of packages can be built for one release by defining multiple modules.
 

  Users are able to choose which version they want to use on their system.
 

  

- The above example produces two different binaries, each with a different Python version. 

+ The above example produces two different binaries, each with a different Python version.
 

  Again, in case there are multiple architectures, two binaries will be produced for each one.
 

  

  Modularity also leverages existing technologies such as containers to enable users to run multiple versions at the same time on one "machine."
 
@@ -48,8 +48,8 @@ 
 

  

  image::mod-doc-build-matrix.png[,60%,]
 

  

- Modules can be also be built against other modules to achieve more complex results. 

+ Modules can be also be built against other modules to achieve more complex results.
 

  Client tooling always makes sure the right version is installed for a given context.
 

  

- The above example produces two different binaries, each bound to a different Python version. 

+ The above example produces two different binaries, each bound to a different Python version.
 

  Again, in case there are multiple architectures, two binaries will be produced for each one.

file modified
+10 -10 
@@ -16,29 +16,29 @@ 
 

  

  == Consuming the traditional packages
 

  

- There are no changes to the traditional user experience. 

- Packages from the traditional repository will be installed and updated using the same methods as before. 

+ There are no changes to the traditional user experience.
 

+ Packages from the traditional repository will be installed and updated using the same methods as before.
 

  Everything keeps working as it used to.
 

  

  == Consuming the modular packages
 

  

- If a user desires to use the optioanl Modular repositories in order to consume non-default versions of software, there will be some new concepts introduced in the client tooling to manage them. 

+ If a user desires to use the optioanl Modular repositories in order to consume non-default versions of software, there will be some new concepts introduced in the client tooling to manage them.
 

  We outline these below.
 

  

  === Enabling a module
 

  

- Enabling a module makes its packages available on the system. 

- Packages delivered as part of a module always have a priority over the ones from the traditional base, regardless of their actual version. 

+ Enabling a module makes its packages available on the system.
 

+ Packages delivered as part of a module always have a priority over the ones from the traditional base, regardless of their actual version.
 

  Packages in modules are often replacements of the ones in the traditional base.
 

  

- Modularity brings parallel availability, not parallel installability. 

- Only one stream of a given module can be enabled on a system — so it is always clear which version gets installed. 

+ Modularity brings parallel availability, not parallel installability.
 

+ Only one stream of a given module can be enabled on a system — so it is always clear which version gets installed.
 

  Installing and running multiple versions of software can be achieved by using existing technologies, such as containers.
 

  

  === Installing a module
 

  

- To make installation easy, some modules can be also installed as a unit, without the need of enabling them first and then installing individual packages. 

- Installing a module doesn’t necessarily mean installing all of its packages. 

+ To make installation easy, some modules can be also installed as a unit, without the need of enabling them first and then installing individual packages.
 

+ Installing a module doesn’t necessarily mean installing all of its packages.
 

  Modules can define something called an "installation profile" to help users with the installation.
 

  

  ==== Installation profile
 
@@ -49,7 +49,7 @@ 
 

  

  Updating the system always respects user’s module choices (or lack of module) even when there are multiple (and possibly higher) versions available.
 

  

- If the user doesn’t enable any modules, all packages on their system get updated to the latest versions provided by the traditional base repository. 

+ If the user doesn’t enable any modules, all packages on their system get updated to the latest versions provided by the traditional base repository.
 

  However, if the user enables a module stream, packages get updated to the newest version provided by the module.
 

  

  Thanks to this mechanism, the user has better control over the versions of packages on their system while receiving updates (such as security patches) for the whole system.

file modified
+17 -17 
@@ -8,20 +8,20 @@ 
 

  

  We are reusing packages as they are.
 

  

- Packages are the smallest unit of software delivery. 

- Right now, Fedora uses RPM packages to build and ship all of its software. 

+ Packages are the smallest unit of software delivery.
 

+ Right now, Fedora uses RPM packages to build and ship all of its software.
 

  There is a distinction between source RPM (SRPM) packages and binary RPM packages.
 

  

- As the name suggests, *source packages* contain the source code, but also the build process definition including a list of build dependencies. 

+ As the name suggests, *source packages* contain the source code, but also the build process definition including a list of build dependencies.
 

  Building a source package results in one or multiple binary packages.
 

  

- *Binary packages* are produced by building source packages in a build environment called a buildroot. 

- Different buildroots produce different binary packages. 

+ *Binary packages* are produced by building source packages in a build environment called a buildroot.
 

+ Different buildroots produce different binary packages.
 

  This means that a source package without a context is not a complete definition of a resulting binary package.
 

  

- Every Fedora release has its own buildroot. 

+ Every Fedora release has its own buildroot.
 

  Even when multiple releases contain the same package using the same source, they are built multiple times, once in each buildroot, producing different (and often incompatible) binary packages.
 

- Also, over time, as the distribution gets updated, so does its buildroot. 

+ Also, over time, as the distribution gets updated, so does its buildroot.
 

  As a result, a package built today may not result in the same binary as a package built tomorrow.
 

  

  *Packages are either a part of a release, or a part a module.*
 
@@ -32,31 +32,31 @@ 
 

  

  Releases are the generations of Fedora e.g. “Fedora 28”, “Fedora 29”, or “epel 7”.
 

  

- Each release can only contain one major version of every package (by policy). 

- In general, Fedora tends to ship the latest stable versions of software available. 

+ Each release can only contain one major version of every package (by policy).
 

+ In general, Fedora tends to ship the latest stable versions of software available.
 

  At the same time, packages need to maintain a certain API/ABI stability throughout the lifecycle of the release they are part of.
 

  

- The problem with this approach is finding the right balance of being too fast vs. too slow. 

- For example, developers in general tend to prefer the newest versions of software, while server administrators want API/ABI stability for a longer period of time. 

+ The problem with this approach is finding the right balance of being too fast vs. too slow.
 

+ For example, developers in general tend to prefer the newest versions of software, while server administrators want API/ABI stability for a longer period of time.
 

  This problem is solved by Modules.
 

  

  == Modules and Streams
 

  

  image:mod-doc-module.png[,80%,]
 

  

- *Modules* are collections of packages representing a logical unit e.g. an application, a language stack, a set of tools. 

+ *Modules* are collections of packages representing a logical unit e.g. an application, a language stack, a set of tools.
 

  These packages are built, tested, and released together.
 

  

  Each module defines its own lifecycle which is closer to the upstream rather than the Fedora release.
 

  

- Modules are the mechanism of delivering multiple streams (versions) of software for the same release. 

+ Modules are the mechanism of delivering multiple streams (versions) of software for the same release.
 

  This also works the other way round, the same stream for multiple releases.
 

  

  image::mod-doc-stream.png[,80%,]
 

  

- A *stream* is a series of versions having a goal such as backwards API/ABI compatibility or delivering the latest stable version. 

- A “nodejs” interpreter can have streams such as “https://src.fedoraproject.org/modules/nodejs/branch/6[6]” or “https://src.fedoraproject.org/modules/nodejs/branch/8[8]” (two current versions of nodejs), but also “devel” or “latest”. 

- Some packages don’t have a major version. 

+ A *stream* is a series of versions having a goal such as backwards API/ABI compatibility or delivering the latest stable version.
 

+ A “nodejs” interpreter can have streams such as “https://src.fedoraproject.org/modules/nodejs/branch/6[6]” or “https://src.fedoraproject.org/modules/nodejs/branch/8[8]” (two current versions of nodejs), but also “devel” or “latest”.
 

+ Some packages don’t have a major version.
 

  An example could be the “calc” package that is being developed in two branches “stable” and “unstable”.
 

  

  Simply put, streams are the mechanism of delivering and updating software at different speeds, allowing developers to have bleeding-edge software, and server admins longer stability.
 
@@ -67,5 +67,5 @@ 
 

  

  Artifacts are consumable outputs of the build and release processes e.g. an iso image, an RPM repository, a container base image, or an ostree tree.
 

  

- Modules will be delivered as an RPM repository that will work on top of existing releases. 

+ Modules will be delivered as an RPM repository that will work on top of existing releases.
 

  This repository will also be available in the Fedora container base image, enabling users to build containers with multiple versions of software.

file modified
+10 -5 
@@ -1,6 +1,6 @@ 
 

  = Communication
 

  

- This page lists the communication channels and meetings of members of the Modularity Working Group (Modularity WG) communicate.
 

+ This page lists the communication channels and meetings of members of the Modularity Working Group (Modularity WG).
 

  

  

  == Channels
 
@@ -13,14 +13,19 @@ 
 

  Most of the discussions happen on this IRC channel. 

  

  https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/[devel@lists.fedoraproject.org]::
 

- The Modularity WG donesn't have a dedicated mailng list. Instead, the Fedora Devel mailing list is used time to time for more complex decisions or announcements.
 

+ The Modularity WG doesn't have a dedicated mailng list. 

+ Instead, the Fedora Devel mailing list is used time to time for more complex decisions or announcements.
 

+ 

+ https://lists.fedoraproject.org/archives/list/server@lists.fedoraproject.org/[server@lists.fedoraproject.org]::
 

+ The https://fedoraproject.org/wiki/Server[Server SIG] also discusses Modularity at times as the Server Edition is a primary stakeholder in the goals of the https://fedoraproject.org/wiki/Objectives#Current[Modularity Objective].
 

  

  == Meetings
 

  

- Meetings are listed on the https://apps.fedoraproject.org/calendar/modularity/[Modularity calendar]. Currently, there are two alternating meetings on Tuesday:
 

+ Meetings are listed on the https://apps.fedoraproject.org/calendar/modularity/[Modularity calendar]. 

+ Currently, there are two alternating meetings on Tuesday:  

  

  Modularity WG::
 

  A formal meeting of the Modularity WG to discuss, post updates, and make dedisions.
 

  

- Working Hours::
 

- No formal agenda, members of the Modularity WG promise to be present at this time for questions or discission.
 

+ Office Hours::
 

+ No formal agenda, members of the Modularity WG promise to be present at this time for questions or discussion.

file modified
+15 -15 
@@ -1,6 +1,6 @@ 
 

  = What is Modularity
 

  

- Modularity introduces a new optional repository to Fedora called Modular (often referred to as the "Application Stream" or AppStream for short) that ships additional versions of software on independent lifecycles. 

+ Modularity introduces a new optional repository to Fedora called Modular (often referred to as the "Application Stream" or AppStream for short) that ships additional versions of software on independent lifecycles.
 

  This way users can keep their operating system up-to-date while having the right version of an application for their use case, even when the default version in the distribution changes.
 

  

  image::modularity-appstream-overview.png[,100%,]
 
@@ -10,44 +10,44 @@ 
 

  

  === Too fast vs. too slow
 

  

- Different users have different needs. 

- Developers want the latest versions possible, system administrators want stability for longer period of time. 

- There are many Linux distributions out there, each targetting a different audience, for example Fedora vs CentOS. 

+ Different users have different needs.
 

+ Developers want the latest versions possible, system administrators want stability for longer period of time.
 

+ There are many Linux distributions out there, each targetting a different audience, for example Fedora vs CentOS.
 

  

  Fedora generally ships the latest stable versions of its component packages when it is released twice per year.
 

- That is convenient for desktop users and developers. 

+ That is convenient for desktop users and developers.
 

  Even though many people use Fedora on a server, it is sometimes necessary to have a stable version of certain packages for a longer time, mostly because of third-party applications.
 

  

- At the same time, some people consider Fedora to be moving too slowly for them and want even newer runtimes on their system. 

+ At the same time, some people consider Fedora to be moving too slowly for them and want even newer runtimes on their system.
 

  Some upstreams release their software faster than twice a year.
 

  

- On the other hand, CentOS targets long-term stability and releases a new version once every few years. 

- This is convenient for server admins as there are fewer changes over longer periods of time. 

+ On the other hand, CentOS targets long-term stability and releases a new version once every few years.
 

+ This is convenient for server admins as there are fewer changes over longer periods of time.
 

  But some of the software gets old for modern applications and newer versions of runtimes might be needed.
 

  

- In other words, it would be convenient to be able to choose some parts of the system to be moving slowly, and other parts to be moving faster. 

+ In other words, it would be convenient to be able to choose some parts of the system to be moving slowly, and other parts to be moving faster.
 

  Can we do that?
 

  

  https://example.com[YES!]
 

  

  === Outdated containers
 

  

- There are many containers out there. 

- The majority of them are built manually, not actively maintained, not patched with security fixes, and still used by many people. 

+ There are many containers out there.
 

+ The majority of them are built manually, not actively maintained, not patched with security fixes, and still used by many people.
 

  This is especially true for the ones using software of a different version than the distribution their are based on provides.
 

  

- If Fedora had multiple versions of software that is actively maintained and built, could we use it to produce containers? 

+ If Fedora had multiple versions of software that is actively maintained and built, could we use it to produce containers?
 

  Could these containers get automatically rebuilt every time the packages get updated?
 

  

  https://example.com[YES!]
 

  

  === Complex packager workflows
 

  

- Fedora contributors maintain their packages in multiple branches — one for each release. 

- Even when the packages are the same. 

+ Fedora contributors maintain their packages in multiple branches — one for each release.
 

+ Even when the packages are the same.
 

  Leading to a series of manual steps associated with the build process.
 

  

- Could we enable packagers to maintain packages in branches that would follow the package version instead of an arbitrary distribution release version? 

+ Could we enable packagers to maintain packages in branches that would follow the package version instead of an arbitrary distribution release version?
 

  Having a single branch that builds across multiple releases would save some work for packagers.
 

  

  https://example.com[YES!]

file removed
-1 
@@ -1,1 +0,0 @@ 
 

- = Test 

\ No newline at end of file

file removed
-1 
@@ -1,1 +0,0 @@ 
 

- = Test 

\ No newline at end of file

Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.