#593 use RPM tags for langpacks description

Created 2 years ago by jsilhan
Modified 8 months ago

This proposal is to make langpacks packaging more transparent - to be declarativelly described by RPM tags (combination of weak and rich dependencies) rather than by procedural logic of dnf langpack plugin ([https://bugzilla.redhat.com/show_bug.cgi?id=1297520 which doesn't work as expected now] anyway)

We would like to update [https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Addon_Packages_.28langpacks.29 Langpacks part of packaging guide line] with [https://fedoraproject.org/wiki/PackagingDrafts/Langpack new version]. The changes are:

1) demand one line of langpack relation (the Suggest part) in the package

2) example is more comprehensive

3) added part why are langpacks useful

4) changed naming of langpacks from <pkgname>-<langcode> to <pkgname>-langpack-<langcode> (as was [https://fedoraproject.org/wiki/Features/YumLangpackPlugin#Detailed_Description suggested originally]). The later format is better aproach for awareness of all available langpacks. Langpacks that are already in fedora does not need to be renamed.

[https://bugzilla.redhat.com/show_bug.cgi?id=1114422#c22 Here is shown demo] how new design of langpacks work in DNF from user POV. By applying this guide line, Anaconda with a [https://github.com/rhinstaller/anaconda/pull/484 little patch] would finally support langpacks installation with DNF.

A proper diff:

https://fedoraproject.org/w/index.php?title=PackagingDrafts%2FLangpack&diff=current&oldid=432651

I've made a pass over the draft to fix some grammar and do a bit of formatting. Also, the name of langpack in the example doesn't appear to be correct according to item 4 so I'm not sure what's intended.

Also, in keeping with my recent macro frenzy, I wonder if we couldn't simplify this with some nice macros. The tesseract package has a load of language subpackages which it sets up with a %lang_subpkg() macro. We could try to do the same in a general fashion, though getting the files list correct could be "fun". Is there any uniformity in the location of these files? Is it enough to do what %find_lang does (except to split that by language, of course.)

I specifically changed from translations to language contents but looks like tibbs again reversed it. Langpacks are not just providing translations but dictionaries, hyphenations rules, OCR data files, sound files etc. I don't think this fits into "translations" word.

We are trying to propose a new naming guidelines to existing that "-langpack-" be added for all future coming langpacks packages so that there can be consistency in "dnf search langpack" query. But then this is just a suggestion and if FPC thinks it will not be useful as we are not proposing to rename existing packages then we can drop it from name which I see tibbs already did. But, I need some advice from FPC members on this point 4.

I'm not sure why you think I changed something regarding translations. Perhaps you should take a look at the edit history of the draft. The only changes I made are at https://fedoraproject.org/w/index.php?title=PackagingDrafts%2FLangpack&diff=432794&oldid=432790

Nothing in that diff involves translations. Although it does look like there was some sort of overlapping edit in there, because I definitely didn't change the package names in the example to remove the "langpacks-".

I might be wrong in putting the correct words in my only edit previously but I see this change from above commit

From

''Langpacks is a concept of dividing language contents (translations) into subpackages'' in the case the size of translation is enormous and/or the package is part of core image that should be minimal.

To

''The idea behind "langpacks" is to divide translations into subpackages'' in the case that the size of translation is enormous or the package is part of a core image that should be minimal.

I am asking about this change. I see that words "language contents" are removed.

The other change is following where

{{{
%package langpack-bs
changed to
%package bs
}}}

and

{{{
%files langpack-bs
to
%files bs
}}}

for this change I am asking advice to FPC should we recommend langpack-bs instead just bs?

Thanks for drafting this, Jan.
Generally it looks pretty good to me.

I brought back <pkgname>-langpack-<langcode> to the text
and mentioned language content in addition to translations.

I think the suggest about a langpack macro is good.
(<rant>We should do this kind of thing a lot more in Fedora - I don't
know why we make everyone roll their own macros since RHL.</rant>:)
Yes, there are already various such macros floating around in various packages. Maybe we could have %find_langpacks say for generating
filelists for common cases.

"Specifically, the langcode value used in the package name must match the langcode identifier used in the directory path by upstream for the language translation files."

Maybe some examples would help to clarify this.
I think we should just use 2-3 letter lang codes for the subpackages unless it is ambiguous (like for pt or zh), but maybe that is already the intention?

eg I think it is okay for package-langpack-ja to contain ja/ or ja_JP/ but not jav/ of course. The current wording reads to me like if the dir is ja_JP/ then the subpackage must be named langpack-ja_JP which I don't think we want (similarly we should have langpack-de not langpack-de_DE, etc, but we do need langpack-pt_BR and langpack-zh_CN).

For now I simply changed "match" to "agree", I feel that captures the spirit of it well enough.

Note we're (mfabian et al) also working on glibc locales subpackaging for F24. :) There currently glibc-langpack-ja contains "ja_JP.utf8/", etc.

As I wrote previously, there appear to have been overlapping edits. I have no idea what mediawiki does when multiple people try to edit the same page. I had it open for a while and it looks like it didn't handle that well.

WRT macro, I don't think we can make any general rpm macro although I can add another snipet how the marco would look like for current example. I was looking how tesseract, glibc ([https://copr-be.cloud.fedoraproject.org/results/mfabian/glibc/fedora-rawhide-x86_64/00154808-glibc/ from mfabian's COPR]) and hungspell packages define langpacks. Their langpacks have additional rpm tags in subpackages (provides, requires); the files, as Parag mentioned, are not only translations so they are in different directories and with various extensions and could be arch specific (glibc).

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-01-28/fpc.2016-01-28-17.00.txt):

  • 593 use RPM tags for langpacks description (geppetto, 17:09:55)

  • Test repo. is available at: copr enable jsilhan/langpacks
    (geppetto, 17:52:26)
  • Use RPM tags for langpacks (+1:4, 0:0, -1:0) … more time needed for
    people to see what happens/why/etc. (geppetto, 18:20:27)

people who +1'd at the meeting: tibbs, geppetto, tomspur, Rathann

I have added few packages with Supplements: tag in my Copr repository. Just enable it as "sudo dnf copr enable pnemade/weakdeps"

I have included packages like hunspell-, hyphen-, mythes-, man-pages-, stardict-dic-*, kde-l10n to test Supplements: tag. Mostly the copr repository have packages for languages like de, es, fr, it, ja, pt, ru, sv.

You can test now Supplements: tag just by installing any of these languages. E.g. if you do "dnf install langpacks-fr" and if you have any of the base packages installed e.g. man-pages, kde-l10n and hunspell then you will also get man-pages-fr, kde-l10n-fr, hunspell-fr packages installed on you system.

{{{
$ sudo dnf install langpacks-fr
Last metadata expiration check performed 0:08:00 ago on Fri Jan 29 20:13:36 2016.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
gnome-getting-started-docs-fr noarch 3.18.2-2.fc23 pnemade-weakdeps 10 M
hunspell-fr noarch 4.6-7.fc23 pnemade-weakdeps 349 k
hyphen-fr noarch 2.0-11.fc23 pnemade-weakdeps 15 k
kde-l10n-fr noarch 15.08.3-3.fc23 pnemade-weakdeps 27 M
langpacks-fr noarch 1.0-3.fc23 pnemade-weakdeps 6.5 k
man-pages-fr noarch 3.70-3.fc23 pnemade-weakdeps 5.9 M
mythes-fr noarch 2.3-7.fc23 pnemade-weakdeps 1.3 M

Transaction Summary

Install 7 Packages

Total download size: 45 M
Installed size: 60 M
Is this ok [y/N]: n
}}}

In the next example I have langpacks-hi already installed. Now installing base package stardict will install stardict-dic-hi langpacks package.
{{{
$ sudo dnf install stardict
Last metadata expiration check performed 0:02:35 ago on Fri Jan 29 20:26:12 2016.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
stardict x86_64 3.0.6-3.fc23 fedora 2.6 M
stardict-dic-hi noarch 3.0.1-14.fc23 pnemade-weakdeps 1.0 M

Transaction Summary

Install 2 Packages

Total download size: 3.6 M
Installed size: 16 M
Is this ok [y/N]: y
}}}

To remove the language support from the system,
{{{
$ sudo dnf remove langpacks-hi
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Removing:
kde-l10n-hi noarch 15.08.3-3.fc23 @pnemade-weakdeps 3.1 M
langpacks-hi noarch 1.0-3.fc23 @pnemade-weakdeps 0
stardict-dic-hi noarch 3.0.1-14.fc23 @pnemade-weakdeps 3.1 M

Transaction Summary

Remove 3 Packages

Installed size: 6.2 M
Is this ok [y/N]: n
}}}

Note that you must be having "clean_requirements_on_remove=true" in /etc/dnf/dnf.conf file for above command.

you can also try how packaging of glibc langpacks from Mike Fabian branch looks like.

Have the Parag's copr still enabled:

{{{# sudo dnf copr enable pnemade/weakdeps}}}

Enable Mike's copr too:

{{{# sudo dnf copr enable mfabian/glibc}}}

Try to install newest glibc + some langpacks. I.e.:

{{{

dnf install glibc-2.22.90-52.fc23 langpacks-de --assumeno

Last metadata expiration check performed 0:00:54 ago on Wed Feb 3 14:11:30 2016.
Dependencies resolved.
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
glibc-langpack-de x86_64 2.22.90-52.fc23 mfabian-glibc 276 k
hunspell-de noarch 0.20151222-2.fc23 pnemade-weakdeps 354 k
hyphen-de noarch 0.20060120-14.fc23 pnemade-weakdeps 33 k
langpacks-de noarch 1.0-3.fc23 pnemade-weakdeps 6.5 k
man-pages-de noarch 1.9-3.fc23 pnemade-weakdeps 1.3 M
mythes-de noarch 0.20151216-2.fc23 pnemade-weakdeps 8.2 M
Upgrading:
glibc x86_64 2.22.90-52.fc23 mfabian-glibc 3.7 M
glibc-common x86_64 2.22.90-52.fc23 mfabian-glibc 3.3 M
glibc-devel x86_64 2.22.90-52.fc23 mfabian-glibc 916 k
glibc-headers x86_64 2.22.90-52.fc23 mfabian-glibc 496 k

Transaction Summary

Install 6 Packages
Upgrade 4 Packages
}}}

Dear FPC members,
Is there any more input needed here? The Supplements: tag is just working fine and langpacks are getting installed when respective language meta-package is already installed on the system.

update: Anaconda has the support of [https://github.com/rhinstaller/anaconda/pull/484 new langpacks] too.

Just so you know, a big chunk of the committee is still away and we probably won't meet again until the 18th. If someone who didn't vote at the previous meeting wishes to toss in a +1 then this is ready to go; otherwise it will need to wait for more discussion.

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-02-18/fpc.2016-02-18-17.00.txt):

I've written this up as best I can. I just realized the draft stuck all of that in the the naming guidelines, and... it really doesn't belong there. I placed it in its own page and added references from the main guidelines.

I added a bit of explanation to the macro snippet, and corrected the directory ownership (which should be double-checked).

Also, the section on %find_lang actually conflicts with the langpack guideline, because use of %find_lang is mandatory. I tried to add a bit of language to indicate that this only applies if the locale files are all going to be stored in the main package, and to point to the langpack guidelines if not.

Announcement text:

A page describing the implementation of Langpacks has been added to the guidelines.

Also, it appears that these don't work on F22, so I've added a note to that effect.

I hope FPC is aware that the approved langpacks guideline conflicts with [https://fedoraproject.org/wiki/Packaging:Guidelines#Rich.2FBoolean_dependencies]:

Packages in Fedora must not use the new rich (or Boolean) dependency feature [...]

AFAIR https://fedorahosted.org/fpc/ticket/559 was targeted for F23 because there was no use case for rich deps yet and the feature was new and not tested. My testing package went through koji builder process and the rich deps works from F23+ in Fedora infrastructure.

Oops, yeah, the intent was to remove that after things had been tested, but nobody told us that had happened. I rewrite the section to indicate that they're OK for F23+. Probably worth an announcement:

Announcement text:

The use of rich (or Boolean) dependencies is now OK for F23+.

The agreed-upon draft seems to say use of rich deps is unconditionally ok for f23+.

Perhaps fpc folks missed my recent related post to -devel list:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WGMAZSE76GW2WH3OGXYQM5RLRNBV5ZWV/

short version of my findings:
using rich deps with Supplements is ok
using rich deps with Requires/Recommends is not ok

This is mostly due to fedora tools that are still yum-based. yum can grok Requires/Recommends, but not rich deps. Supplements are ok, since yum currently does not support it (and it is ignored).

Was this considered?

It wasn't considered because the change was made five weeks ago and I've only now gone ahead an announced it.

To expand on comment:28:

The main problem tool here is mash. It's still using yum, and it only expects some 'flags' in Requires, so it bombs out seeing booleans there. So, we currently simply cannot push updates that have rich bool Requires/Recommends in them.

It's unclear to me if pungi is also affected. It seems like it's not, but it is still using yum in places so perhaps it could be in some cases.

That probably explains bug in reporting "broken dependencies" <https://fedorahosted.org/rel-eng/ticket/6365>.

8 months ago

Metadata Update from @james:
- Issue assigned to tibbs

Login to comment on this ticket.

cancel