Background:
This interaction hasn't been problematic before, because generally using nodocs is an endpoint choice with no distribution after that. But now, we are looking at building some official cloud and container images with nodocs, so it suddenly becomes important.
As a bonus, it's my understanding that this tag can automatically handle hardlinking identical license files.
Specifically, I propose:
I filed the original rpm feature ticket on this: http://www.rpm.org/ticket/116
I don't believe that %license does automatic hardlinking, although, I asked for that.
That said, we should definitely start using %license in new packages and I'm happy to help with the proven packager changes for cloud in rawhide (a source package list would be nice).
Cool. I'll get you a preliminary list of src packages RSN. List may change by F21 release but I don't think significantly.
I'm also +1 to a guideline to use %license for license files.
Probably should file a FESCo change request for the updating of packages to include that tag -- the reason being that some endusers might be using --nodocs already to save space and %license would mean that they are likely to want to add an rm -rf /usr/share/doc to their scripts to build their images.
Replying to [comment:3 toshio]:
Probably should file a FESCo change request for the updating of packages to include that tag
Yes, I will file the change -- see point 3 above :)
I wanted to put the general plan out here first just to make sure it generally seemed good from this perspective. Spot, do you want to be a change co-owner or should I just note that you'll help?
List of changes that are needed to implement this:
https://fedoraproject.org/wiki/Packaging:Guidelines#Documentation
Add a paragraph:
"""Although license files are documentation, they are treated specially (including using a different tag. Please see [[Packaging:LicensingGuidelines]] for how to handle them"""
Change every instance of %doc on this page to %license:
https://fedoraproject.org/wiki/Packaging:LicensingGuidelines
Change item #4 in the ReviewChecklist to %license: https://fedoraproject.org/wiki/Packaging:ReviewGuidelines
%doc to %license in spec templates: http://fedoraproject.org/wiki/Packaging:Haskell http://fedoraproject.org/wiki/Packaging:D https://fedoraproject.org/wiki/Packaging:Python (Probably other spec templates as well... This was just a quick google search)
Google didn't find any place under wiki/Legal: that mentioned the %doc tag but someone might want to review those pages (When we set it up, we tried to keep "how to package" in Packaging: and "what is legal" under Legal: so it makes sense that there wouldn't be anything to change there but it doesn't hurt to double check).
Finally a question that should come up in fesco -- is it '''critical''' that everything on the cloud product has their license tags changed from %doc to %license (because the use of --nodocs means that some packages could be shipped without their accompanying license if they are not marked) or is "best effort" and fixing any packages we've missed if the missing text is reported going to be okay?
Yes, this is definitely something we need to do, and I'll toss in my +1 now.
However, which RPM versions actually support this? We should be careful adding "MUST" guidelines in a way that breaks the "one spec across Fedora and EPEL" thing that some people insist on.
Hey, Spot, I took a look at https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-cloud-base.ks to figure out what packages are installed in the Fedora Base Cloud image. I don’t know if this is the complete list, maybe someone can add more, but this is what I found so far. This is for the F20 image, currently.
I separated the list into packages available in just Fedora or packages available in Fedora and EPEL, in case there are any problems with what tibbs was saying of not a recent enough RPM version for RHEL.
Cloud images packages
Core
Fedora only
http://pkgs.fedoraproject.org/cgit/grubby.git/
http://pkgs.fedoraproject.org/cgit/tar.git/
http://pkgs.fedoraproject.org/cgit/rsync.git/
EPEL
http://pkgs.fedoraproject.org/cgit/cloud-init.git/
http://pkgs.fedoraproject.org/cgit/heat-cfntools.git/
Subpackages:
https://apps.fedoraproject.org/packages/cloud-utils-growpart
https://apps.fedoraproject.org/packages/syslinux-extlinux
To clarify some aspects from rpm POV:
The "special" %license that behaves similarly to special %doc is supported in rpm >= 4.11. In other words, its fairly recent but all current Fedora versions support it. Current EPEL does not support it, but 7 will.
Files packaged as %license do not go into the same directory as %doc, but a similar per-package directory in /usr/share/licenses. The reason for that is mostly technical, %doc is a hack that does not like other hacks around so this was painful enough as it is :)
No automatic hardlinking is done currently. Considering how small license files generally are, the space savings would be neglible... except perhaps for (cloud etc) images where every byte might count, but image builders can just do 'hardlink /usr/share/licenses'.
Wrt EPEL compatibility concerns in the FPC meeting - specs can be kept compatible by "mapping" license to %doc on rhel < 7. Something like: {{{ %files %if 0%{?rhel} && 0%{?rhel} < 7 %global license %%doc %endif
%defattr(-,root,root,-) %license COPYING }}}
Just note this "mapping" cannot go to spec preamble because License-tags in the spec will override it with the tag content. It doesn't have to be in %files, but it needs to be after the last License: tag.
I was hoping for a single line "if old, %doc, if else, %license", but I can't see how to do that. Guess this is the best option.
Ask and thou shalt receive :P
Just realized that inside a spec it's possible to test support for the special %license directive by testing whether %_licensedir is defined. So this should do the right thing everywhere, regardless of the distro and all:
{{{ %files %{!?_licensedir:%global license %%doc} %license COPYING }}}
My hat is off to you sir. This is exactly the sort of thing I wanted.
What if the license text does not have a dedicated file, but it's a part of another file. E.g. ''README''. I guess that file should be marked as ''%license'' and thus it would disappear from ''/usr/share/doc''. Is this what users expect?
Since I'm not around for meetings for a few weeks, I am +1 to this change.
We'll want to be sure to say that %{!?_licensedir:%global license %%doc} needs to go after the preamble in both the licensing guidelines and review checklist
-1
I consider this proposal to be absurd.
Why? - This all (cloud images) addresses a small group of user who are using a crippled distribution. Their demands are such kind of specialized to justify the churn these people are causing.
The sizes of %doc'ed files are almost negligent in comparison to the overall size of the rest of the distro. The effort to implement this change is not justified.
%license is redundant to Licence: => I do not see any need to add %license but am expecting it to cause confusions, bugs and conflicts.
The effort to implement this HUGE. If I was hostile, I'd insist on letting the proponents of this proposal to implement act of bureaucracy themselves.
Replying to [comment:15 corsepiu]:
This all (cloud images) addresses a small group of user who are using a crippled distribution. Their demands are such kind of specialized to justify the churn these people are causing.
"Demands" sounds so harsh, and I don't know what you mean by "crippled". But in any case, this is a strategically important area for Fedora.
The first part of this isn't true; it's a full 5% of the base cloud image (on disk), and a significantly higher fraction for some Docker containers. It's true that it'd be nice to get more significant reduction than that, but there are few other single changes that will have a bigger impact (and we're working on those, too). Together, they add up.
The second part is irrelevant, since Tom has volunteered to help with the effort, and I'm sure others are willing to pitch in as needed, too.
A different name would be fine. I'm not picky. This seems fine to me -- and it is how the RPM maintainer decided to do it so I don't see much need to bikeshed.
See above.
Still waiting for a working rawhide cloud image in order to give the list of cloud packages to Spot. I'll update as soon as available.
Update: we have a working cloud image now, but due to dependency bloat in cloud-init (of all things), there are about twice as many packages as there should be. So, I'm hoping to get that fixed first before providing the full list.
Here's the list of source RPMs which land on the F21 cloud base image. I would like to get this number down, but we get into dependency tangles, so here we are.
{{{ acl-2.2.52-5.fc21.src.rpm attr-2.4.47-7.fc21.src.rpm audit-2.3.7-2.fc21.src.rpm basesystem-10.0-10.fc21.src.rpm bash-4.3.18-2.fc21.src.rpm bind-9.9.5-6.P1.fc21.src.rpm bzip2-1.0.6-12.fc21.src.rpm ca-certificates-2013.1.97-3.fc21.src.rpm checkpolicy-2.3-2.fc21.src.rpm chkconfig-1.3.61-2.fc21.src.rpm cloud-init-0.7.5-6.fc21.src.rpm cloud-utils-0.27-11.fc21.src.rpm coreutils-8.22-16.fc21.src.rpm cpio-2.11-28.fc21.src.rpm cracklib-2.9.1-3.fc21.src.rpm cronie-1.4.11-7.fc21.src.rpm crontabs-1.11-8.20130830git.fc21.src.rpm crypto-policies-20140708-1.git3a7ae3f.fc21.src.rpm cryptsetup-1.6.5-1.fc21.src.rpm curl-7.37.0-3.fc21.src.rpm cyrus-sasl-2.1.26-17.fc21.src.rpm dbus-1.8.6-1.fc21.src.rpm device-mapper-multipath-0.4.9-65.fc21.src.rpm dhcp-4.3.0-15.fc21.src.rpm diffutils-3.3-7.fc21.src.rpm dmidecode-2.12-5.fc21.src.rpm dracut-038-1.fc21.src.rpm e2fsprogs-1.42.10-5.fc21.src.rpm elfutils-0.159-5.fc21.src.rpm expat-2.1.0-8.fc21.src.rpm fedora-release-22-0.1.src.rpm fedora-repos-22-0.1.src.rpm file-5.19-1.fc21.src.rpm filesystem-3.2-26.fc21.src.rpm findutils-4.5.12-5.fc21.src.rpm fipscheck-1.4.1-5.fc21.src.rpm gawk-4.1.1-3.fc21.src.rpm gcc-4.9.0-14.fc21.src.rpm gdbm-1.11-2.fc21.src.rpm glib2-2.41.1-1.fc21.src.rpm glibc-2.19.90-26.fc21.src.rpm gmp-6.0.0-5.fc22.src.rpm gnupg2-2.0.24-1.fc21.src.rpm gpgme-1.4.3-2.fc21.src.rpm grep-2.20-2.fc21.src.rpm groff-1.22.2-9.fc21.src.rpm grubby-8.35-2.fc21.src.rpm gzip-1.6-3.fc21.src.rpm hardlink-1.0-19.fc21.src.rpm heat-cfntools-1.2.7-1.fc21.src.rpm hostname-3.15-2.fc21.src.rpm initscripts-9.54-4.fc21.src.rpm iproute-3.15.0-1.fc21.src.rpm iptables-1.4.21-9.fc21.src.rpm iputils-20140519-2.fc21.src.rpm json-c-0.11-7.fc21.src.rpm kernel-3.16.0-0.rc4.git2.1.fc22.src.rpm keyutils-1.5.9-2.fc21.src.rpm kmod-18-1.fc21.src.rpm krb5-1.12.1-10.fc21.src.rpm less-458-11.fc21.src.rpm libarchive-3.1.2-8.fc21.src.rpm libassuan-2.1.0-3.fc21.src.rpm libcap-2.24-5.fc21.src.rpm libcap-ng-0.7.4-5.fc21.src.rpm libcgroup-0.41-3.fc21.src.rpm libdb-5.3.28-6.fc21.src.rpm libedit-3.1-6.20140213cvs.fc21.src.rpm libestr-0.1.9-2.fc21.src.rpm libffi-3.1-4.fc21.src.rpm libgcrypt-1.6.1-5.fc21.src.rpm libgpg-error-1.13-1.fc21.src.rpm libidn-1.28-3.fc21.src.rpm liblogging-1.0.4-2.fc21.src.rpm libmetalink-0.1.2-5.fc21.src.rpm libmnl-1.0.3-7.fc21.src.rpm libnetfilter_conntrack-1.0.4-2.fc21.src.rpm libnfnetlink-1.0.1-4.fc21.src.rpm libpipeline-1.3.0-2.fc21.src.rpm libpwquality-1.2.3-3.fc21.src.rpm libseccomp-2.1.1-2.fc21.src.rpm libselinux-2.3-3.fc21.src.rpm libsemanage-2.3-3.fc21.src.rpm libsepol-2.3-2.fc21.src.rpm libssh2-1.4.3-13.fc21.src.rpm libtasn1-4.0-1.fc21.src.rpm libuser-0.60-4.fc21.src.rpm libutempter-1.1.6-4.fc21.src.rpm libverto-0.2.6-2.fc21.src.rpm libxml2-2.9.1-3.fc21.src.rpm libyaml-0.1.6-3.fc21.src.rpm linux-atm-2.5.1-9.fc21.src.rpm logrotate-3.8.7-2.fc21.src.rpm lua-5.2.2-6.fc21.src.rpm lvm2-2.02.107-1.fc21.src.rpm lzo-2.06-6.fc21.src.rpm man-db-2.6.7.1-4.fc21.src.rpm mtools-4.0.18-5.fc21.src.rpm ncurses-5.9-14.20140323.fc21.src.rpm net-tools-2.0-0.25.20140707git.fc21.src.rpm nspr-4.10.6-1.fc21.src.rpm nss-3.16.2-1.fc21.src.rpm nss-softokn-3.16.2-1.fc21.src.rpm nss-util-3.16.2-1.fc21.src.rpm openldap-2.4.39-8.fc21.src.rpm openssh-6.6.1p1-1.fc21.1.src.rpm openssl-1.0.1h-5.fc21.src.rpm p11-kit-0.20.3-1.fc21.src.rpm pam-1.1.8-12.fc21.src.rpm parted-3.1-28.fc21.src.rpm passwd-0.79-3.fc21.src.rpm pcre-8.35-3.fc21.src.rpm pinentry-0.8.3-2.fc21.src.rpm pkgconfig-0.28-4.fc21.src.rpm policycoreutils-2.3-6.fc21.src.rpm popt-1.16-4.fc21.src.rpm procps-ng-3.3.9-10.fc21.src.rpm pth-2.0.7-23.fc21.src.rpm pygpgme-0.3-10.fc21.src.rpm pyliblzma-0.5.3-11.fc21.src.rpm python-2.7.7-2.fc21.src.rpm python-backports-1.0-4.fc21.src.rpm python-backports-ssl_match_hostname-3.4.0.2-2.fc21.src.rpm python-boto-2.27.0-2.fc21.src.rpm python-chardet-2.2.1-1.fc21.src.rpm python-cheetah-2.4.4-7.fc21.src.rpm python-configobj-5.0.5-1.fc21.src.rpm python-iniparse-0.4-14.fc21.src.rpm python-IPy-0.81-11.fc21.src.rpm python-jsonpatch-1.2-3.fc21.src.rpm python-jsonpointer-1.0-5.fc21.src.rpm python-prettytable-0.7.2-3.fc21.src.rpm python-psutil-1.2.1-3.fc21.src.rpm python-pycurl-7.19.3.1-3.fc21.src.rpm python-requests-2.3.0-2.fc21.src.rpm python-six-1.6.1-3.fc21.src.rpm python-urlgrabber-3.10.1-1.fc21.src.rpm python-urllib3-1.8.2-3.fc21.src.rpm pyxattr-0.5.3-1.fc21.src.rpm PyYAML-3.11-3.fc21.src.rpm qrencode-3.4.2-2.fc21.src.rpm readline-6.3-2.fc21.src.rpm rootfiles-8.1-17.fc21.src.rpm rpm-4.11.90-0.git12844.5.fc21.src.rpm rsync-3.1.1-1.fc21.src.rpm rsyslog-7.4.10-2.fc21.src.rpm sed-4.2.2-7.fc21.src.rpm selinux-policy-3.13.1-63.fc21.src.rpm setools-3.3.8-3.fc21.src.rpm setup-2.9.0-2.fc21.src.rpm shadow-utils-4.1.5.1-14.fc22.src.rpm shared-mime-info-1.3-9.fc21.src.rpm sqlite-3.8.5-2.fc21.src.rpm sudo-1.8.8-5.fc21.src.rpm syslinux-6.02-5.fc21.src.rpm systemd-215-3.fc21.src.rpm tar-1.27.1-5.fc21.src.rpm tcp_wrappers-7.6-77.fc21.src.rpm texinfo-5.2-3.fc21.src.rpm tzdata-2014e-4.fc21.src.rpm ustr-1.0.4-16.fc21.src.rpm util-linux-2.25-0.3.fc21.src.rpm vim-7.4.307-2.fc21.src.rpm xz-5.1.2-12alpha.fc21.src.rpm yum-3.4.3-152.fc21.src.rpm yum-metadata-parser-1.1.4-12.fc21.src.rpm zlib-1.2.8-5.fc21.src.rpm }}}
Thanks, Tom!
The %license change doesn't seem to be reflected in the packaging guidelines yet
I want to second what Rahul said above. This should be reflected in packaging guidelines and using %license instead of %doc for licenses should be recommended (probably not required?) for all packages.
Packaging Guidelines still say:
License Text
If the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc
And fedora-review show this notes:
[ ]: Package requires other packages for directories it uses. Note: No known owner of /usr/share/licenses [ ]: Package must own all directories that it creates. Note: Directories without known owners: /usr/share/licenses
Can I ignore these messages in the revision of the package where it NOT will be by default in Fedora Cloud? Or it is better do not use the %license macro yet?
I'm not sure why this is still set to meeting. It looks like it's ready to be written up, so I'll set that now.
I wrote this up, but I wasn't entirely sure about the %{!?_licensedir:%global license %%doc} bit. It seems this is only needed for EPEL 5 and 6 at this point, so I'm guessing that should just go in the EPEL guidelines.
Announcement text (important, should be at the top of the announcement):
%license must be used in place of %doc to designate any file containing the license information for a package. See https://fedoraproject.org/wiki/Packaging:Guidelines#Documentation and https://fedoraproject.org/wiki/Packaging:LicensingGuidelines
Replying to [comment:26 tibbs]:
Correct, only EPEL < 7 needs the compat hack.
Am Wed, 28 Jan 2015 12:02:27 +0000 schrieb devel-announce-request@lists.fedoraproject.org:
Date: Tue, 27 Jan 2015 21:03:05 -0600 From: "Jason L Tibbitts III" tibbs@math.uh.edu To: devel-announce@lists.fedoraproject.org Subject: [Guielines Change] Changes to the packaging guidelines Message-ID: ufaoapjzjx2.fsf@epithumia.math.uh.edu Content-Type: text/plain; charset=utf-8 %license must be used in place of %doc to designate any file containing the license information for a package. See https://fedoraproject.org/wiki/Packaging:Guidelines#Documentation and https://fedoraproject.org/wiki/Packaging:LicensingGuidelines
Date: Tue, 27 Jan 2015 21:03:05 -0600 From: "Jason L Tibbitts III" tibbs@math.uh.edu To: devel-announce@lists.fedoraproject.org Subject: [Guielines Change] Changes to the packaging guidelines Message-ID: ufaoapjzjx2.fsf@epithumia.math.uh.edu Content-Type: text/plain; charset=utf-8
IMHO this is a very good improvement. But is fedora-review (or especially rpmlint) able to check for the existance of %license ? And what happens to all the existing packages, not only cloud ones? When do they have to confirm the new MUST guidelines and who/what is responsible to check that?
Well, I could do a script over all Fedora packages and in SCM, but it would produce a lot of bugzilla content, so well …
None of the issues you raise are FPC-related. We do not control the fedora-review or rpmlint scripts. We do not retroactively enforce packaging guidelines (or, indeed, enforce them at all.) If you would like to request clarification of the packaging guidelines, or to submit a draft, please open a new ticket with your issue.
note for completeness -- patch for fedora-review to support %license submitted:
https://lists.fedorahosted.org/pipermail/fedorareview/2015-February/000211.html
Note that:
{{{ License: BSD
%{!?_licensedir:%global license %%doc} %doc README PATENTS NEWS AUTHORS %license COPYING }}}
does not appear to work on EPEL6. I get:
{{{ + /bin/mkdir -p /builddir/build/BUILDROOT/libwebp-0.4.3-1.el6.x86_64/usr/share/doc/libwebp-0.4.3 + cp -pr README PATENTS NEWS AUTHORS /builddir/build/BUILDROOT/libwebp-0.4.3-1.el6.x86_64/usr/share/doc/libwebp-0.4.3 + exit 0 RPM build errors: File must begin with "/": BSD File must begin with "/": COPYING
}}}
In case anyone is stuck on this, here's a solution which worked for me:
{{{ %if 0%{?rhel} || 0%{?fedora} < 21 %doc LICENSE %else %license LICENSE %endif }}}
JFTR: https://fedoraproject.org/wiki/EPEL:Packaging#The_.25license_tag have the correct form :
%{!?_licensedir:%global license %doc}
Works with %doc not %%doc.
Metadata Update from @orion: - Issue assigned to tibbs
Login to comment on this ticket.