on one hand it makes sense, on the other hand it is saying what will be produced out of this component..

Yes. This has come up multiple times during the past year. I understand how it can cause confusion, especially initially, but I still prefer it over splitting the RPM content into two types.

Source RPMs are RPMs as well. The main distinction should come from the section rather than whether this is source-y artifact and this other thing is the output. If we follow that path, we'll need to distinguish source/non-source for all other content, too, which is just meh. Source PyPI/binary PyPI? Source CPAN/binary CPAN? Source rubygem/binary rubygem? Etc. This is still in scope.

Once the user learns components is where they define the input, this should become clear.

This is a fairly significant usability problem with the current format, as in practice, "srpm names" and "rpm names" are distinct namespaces in Fedora and derivatives, with the latter typically, but not necessarily, being a superset of the former (as there is no technical requirement for an SRPM to actually produce an RPM with the same name).

As somebody attempting to both parse and generate the specification (fedmod needs to read existing module specifications when determining module level dependencies), it's quite confusing that the same term ("rpms") is used to mean four different things in the same file:

• "rpms" = RPM names (resolved via yum/dnf metadata)
  • profiles section
  • api section
  • filters section
• "rpms" = SRPM names (resolved as a VCS reference, typically in dist-git)
  • components section
• "rpms" = settings when building SRPMs
  • buildopts section
• "rpms" = full RPM NEVRA
  • artifacts section

This makes reading and processing modulemd files highly context dependent, and really doesn't make it clear that you can't usefully compare the so-called "rpms" from different sections of the spec in the general case. It also means we can't resolve https://pagure.io/modulemd/issue/54 just by adding an "srpms" section under filters (with the meaning "filter out all RPMs built from the named SRPM")

Of them all, it's mainly buildopts and components that are problematic, since they don't refer to RPM names, and they don't get resolved using dnf/yum metadata, but they still use the rpms category heading.

Instead, these should be called srpms as @karsten suggested, since their input is a VCS repository with a spec file, and their output is an SRPM (+ built binary RPMs). By contrast, the other places that refer to rpms actually are talking about RPM names.

Given such a change, an srpms section could also be added to filters, indicating that all RPMs built from the named source RPM should be filtered out (once the module build process is updated to track which RPMs came from which SRPMs).

In the near term, components.rpms and buildopts.rpms would need to remain as compatibility aliases for components.srpms and buildopts.srpms.

This is a fairly significant usability problem with the current format, as in practice, "srpm names" and "rpm names" are distinct namespaces in Fedora and derivatives, with the latter typically, but not necessarily, being a superset of the former (as there is no technical requirement for an SRPM to actually produce an RPM with the same name).

They are not all that distinct, really. More on that in my previous comment as well as below.

Yes, source RPMs can build RPMs with names completely unrelated to the name of the input source RPM. This is something every packager is aware of.

As somebody attempting to both parse and generate the specification (fedmod needs to read existing module specifications when determining module level dependencies), it's quite confusing that the same term ("rpms") is used to mean four different things in the same file:

"rpms" = RPM names (resolved via yum/dnf metadata)
profiles section
api section
filters section

These are simply binary RPM names. Whether you find them in repodata, on a filesystem or elsewhere isn't relevant.

"rpms" = SRPM names (resolved as a VCS reference, typically in dist-git)
components section

And these are simply source RPM names. Again, whether you find them in repodata (source RPM repositories exist and are frequently used), on a filesystem or elsewhere isn't, again, relevant.

As it happens, in Fedora the names of dist-git repositories follow the names of source RPMs they host. This, however, isn't a requirement for modulemd. You may supply your own repository URL using the repository property. In this section it's really source RPM names, not VCS references.

I may be reading your comment wrong, though.

"rpms" = settings when building SRPMs
buildopts section

This comes from the concept that source and binary RPMs aren't different. They are the same format, they both have dependencies, they both are included in their own repositories. They're not truly distinct.

It's obvious you're looking at this from a different angle but perhaps explaining mine could clear things up a bit. Obviously improving (or even creating, eh) our documentation wouldn't be such a bad idea :)

"rpms" = full RPM NEVRA
artifacts section

The same thing, just more specific. Thinking about it, we should allow full NEVRAs under profiles, api and filter, even though their use in these places would be highly impractical and borderline impossible.

This makes reading and processing modulemd files highly context dependent, and really doesn't make it clear that you can't usefully compare the so-called "rpms" from different sections of the spec in the general case. It also means we can't resolve https://pagure.io/modulemd/issue/54 just by adding an "srpms" section under filters (with the meaning "filter out all RPMs built from the named SRPM")

Glad you added the explanation because having srpms under filter would, to me, read as "filter these RPMs from the source RPM repository". Not the best choice to achieve your goal either.

I will comment on issue 54 in the ticket.

Of them all, it's mainly buildopts and components that are problematic, since they don't refer to RPM names, and they don't get resolved using dnf/yum metadata, but they still use the rpms category heading.
Instead, these should be called srpms as @karsten suggested, since their input is a VCS repository with a spec file, and their output is an SRPM (+ built binary RPMs). By contrast, the other places that refer to rpms actually are talking about RPM names.
Given such a change, an srpms section could also be added to filters, indicating that all RPMs built from the named source RPM should be filtered out (once the module build process is updated to track which RPMs came from which SRPMs).
In the near term, components.rpms and buildopts.rpms would need to remain as compatibility aliases for components.srpms and buildopts.srpms.

See my comments above.

I'm still not convinced this change would actually help with anything. I can see how it could confuse or bother people with my way of thinking, however. In either case, we need good docs.

Source RPMs and binary RPMs are different. They have different provenance, they have different purposes, they have different roles to play. The fact that they use the same file format on disk is no more relevant to their semantic role than the fact that Python scripts and bash scripts are technically both just text files with a shebang line - it's an implementation detail of the storage formats, not something that should be required in order to comprehend the UX.

Most other tools for working with RPMs keep the distinction clear - you use rpmbuild -bs to go from spec->SRPM for example, and rpmbuild -bb to go from SRPM->RPM, or rpmbuild -ba to do both steps in one command, and we keep the source repositories clearly distinct from the binary repositories when managing systems.

Having these be the same doesn't make anything simpler, since you haven't actually reduced the number of conceptual entities that users need to understand - you've just reduced the number of names they have available to refer to them.

As far as the filtering goes, having filter.srpms filter the SRPM out of the source repository in addition to filtering all the related binary artifacts out the binary repository would be excellent, and exactly the behaviour I would expect from such a field.

https://pagure.io/modularity/fedmod/c/cc06839dbcf27db3814161643304986f50a7adb0?branch=master is an example of the kind of bug the current blurring of the lines between RPMs and SRPMs in the file format helps encourage: I had forgotten to convert the RPM names found during repoclosure generation back to the corresponding SRPM names before populating the generated modulemd.

While it wouldn't have been a guarantee, the bug would have been easier to spot if I'd been feeding RPM names to an API call that looked like self.mmd.components.add_srpm(.... As it is, it's the correct code that looks wrong, since it involves feeding SRPM names to an API that looks like self.mmd.components.add_rpm(....

Working on https://pagure.io/modularity/fedmod/issue/12, it turns out another highly relevant semantic difference between RPMs and SRPMs relates to the meaning of their dependency metadata:

• if you ask libsolv to install an RPM, it will resolve the runtime dependencies of that component
• if you ask libsolv to install an SRPM, it will resolve the build dependencies of that component

Source RPMs and binary RPMs are different. They have different provenance, they have different purposes, they have different roles to play. The fact that they use the same file format on disk is no more relevant to their semantic role than the fact that Python scripts and bash scripts are technically both just text files with a shebang line - it's an implementation detail of the storage formats, not something that should be required in order to comprehend the UX.

It's not indeed relevant. I was merely replying to you implying source RPMs are mere VCS references and binary RPMs are the only thing you work with when dealing with repodata in a previous comment. Neither is true.

Most other tools for working with RPMs keep the distinction clear - you use rpmbuild -bs to go from spec->SRPM for example, and rpmbuild -bb to go from SRPM->RPM, or rpmbuild -ba to do both steps in one command, and we keep the source repositories clearly distinct from the binary repositories when managing systems.

The sole purpose of rpmbuild is packing sources and transforming one RPM into another. For these tasks it uses a number of commands and options, indeed. The context of modulemd, where we declaratively describe handling of a particular artifact type (and I still consider source and binary RPMs being one artifact type) might be similar in some ways but is still different.

Having these be the same doesn't make anything simpler, since you haven't actually reduced the number of conceptual entities that users need to understand - you've just reduced the number of names they have available to refer to them.

Hey, at least I've reduced something! ;)

As far as the filtering goes, having filter.srpms filter the SRPM out of the source repository in addition to filtering all the related binary artifacts out the binary repository would be excellent, and exactly the behaviour I would expect from such a field.

This would be really great (even though not really applicable in Fedora or any other free distrbution for legal reasons) but is not what you were previously requesting, and that's filtering binary RPMs based on source RPM names.

https://pagure.io/modularity/fedmod/c/cc06839dbcf27db3814161643304986f50a7adb0?branch=master is an example of the kind of bug the current blurring of the lines between RPMs and SRPMs in the file format helps encourage: I had forgotten to convert the RPM names found during repoclosure generation back to the corresponding SRPM names before populating the generated modulemd.

While it wouldn't have been a guarantee, the bug would have been easier to spot if I'd been feeding RPM names to an API call that looked like self.mmd.components.add_srpm(.... As it is, it's the correct code that looks wrong, since it involves feeding SRPM names to an API that looks like self.mmd.components.add_rpm(....

This is really subjective. To me it looks fine (for the same reasons I keep repeating). I would be fine with your idea and shifting my view in the context of modulemd content if we could make a meaningful distinction between components.rpms and components.srpms, meaning that we could have both and it would still make sense -- kinda like the filter idea. Now there are half-baked unwritten proposals (one is mine, even) where this could be used (such as binary component reuse across modules without rebuilds) that could push us in this direction but it's not where we are now and I'm not sure we will be.

Working on https://pagure.io/modularity/fedmod/issue/12, it turns out another highly relevant semantic difference between RPMs and SRPMs relates to the meaning of their dependency metadata:
if you ask libsolv to install an RPM, it will resolve the runtime dependencies of that component
if you ask libsolv to install an SRPM, it will resolve the build dependencies of that component

While this might look so from the SPEC file developers' point of view, this is not a semantic difference on RPM level, this is how rpmbuild translates SPEC files. Source RPMs' runtime dependencies come from the BuildRequires lines.

if you ask libsolv to install an RPM, it will resolve the runtime dependencies of that component

Actually it is same as normal RPM, but architecture is src instead of noarch, x86_64 or whatsoever

In truth, the best name for this section would probably be something like components.specs, since we're providing a reference to a spec file and the related source tarballs, rather than a reference to an already built SRPM.

components.srpms would then be a way to request that an existing SRPM be pulled from a source repo somewhere (with fields to specify the source repo), while components.rpms would become a way to bundle prebuilt binary RPMs into a module without rebuilding them (with fields to specify the binary repo).

Whether Fedora's MBS allowed use of the later two options would be an open question, but they'd be reasonable features for module building in COPR to offer.

As a backwards compatibility measure, components.rpms would still support the same keys as components.specs, in which case it would treat that entry as a spec to be built, rather than a binary RPM to bundle.