#30 treecompose-post.sh: keep all supported LANGs
Closed 7 years ago by jlebon. Opened 7 years ago by jlebon.
jlebon/fedora-atomic pr/add-locales  into  f25

@@ -12,7 +12,49 @@ 

  

      "initramfs-args": ["--no-hostonly", "--add", "iscsi"],

  

-     "install-langs": ["en_US"],

+     "install-langs-comment": "Keep this in sync with treecompose-post.sh",

+     "install-langs": [

+       "pt_BR",

+       "fr",

+       "fr_FR",

+       "de",

+       "de_DE",

+       "it",

+       "it_IT",

+       "ru",

+       "ru_RU",

+       "es",

+       "es_ES",

+       "en_US",

+       "zh_CN",

+       "ja",

+       "ja_JP",

+       "ko",

+       "ko_KR",

+       "zh_TW",

+       "as",

+       "as_IN",

+       "bn",

+       "bn_IN",

+       "gu",

+       "gu_IN",

+       "hi",

+       "hi_IN",

+       "kn",

+       "kn_IN",

+       "ml",

+       "ml_IN",

+       "mr",

+       "mr_IN",

+       "or",

+       "or_IN",

+       "pa",

+       "pa_IN",

+       "ta",

+       "ta_IN",

+       "te",

+       "te_IN"

+     ],

  

      "postprocess-script": "treecompose-post.sh",

  

file modified
+42 -3
@@ -8,9 +8,48 @@ 

  # Work around https://github.com/systemd/systemd/issues/4082

  find /usr/lib/systemd/system/ -type f -exec sed -i -e '/^PrivateTmp=/d' -e '/^Protect\(Home\|System\)=/d' {} \;

  

+ # The loop below is too spammy otherwise...

+ set +x

+ 

  # See: https://bugzilla.redhat.com/show_bug.cgi?id=1051816

- KEEPLANG=en_US

- find /usr/share/locale -mindepth  1 -maxdepth 1 -type d -not -name "${KEEPLANG}" -exec rm -rf {} +

- localedef --list-archive | grep -a -v ^"${KEEPLANG}" | xargs localedef --delete-from-archive

+ # and: https://bugzilla.redhat.com/show_bug.cgi?id=1186757

+ # Keep this in sync with the `install-langs` in the treefile JSON

+ KEEPLANGS="

+ pt_BR

+ fr_FR

+ de_DE

+ it_IT

+ ru_RU

+ es_ES

+ en_US

+ zh_CN

+ ja_JP

+ ko_KR

+ zh_TW

+ as_IN

+ bn_IN

+ gu_IN

+ hi_IN

+ kn_IN

+ ml_IN

+ mr_IN

+ or_IN

+ pa_IN

+ ta_IN

+ te_IN

+ "

+ 

+ # Filter out locales from glibc which aren't UTF-8 and in the above set.

+ # TODO: https://github.com/projectatomic/rpm-ostree/issues/526

+ localedef --list-archive | while read locale; do

+     lang=${locale%%.*}

+     lang=${lang%%@*}

+     if [[ $locale != *.utf8 ]] || ! grep -q "$lang" <<< "$KEEPLANGS"; then

+         localedef --delete-from-archive "$locale"

+     fi

+ done

+ 

+ set -x

+ 

  cp -f /usr/lib/locale/locale-archive /usr/lib/locale/locale-archive.tmpl

  build-locale-archive

In https://bugzilla.redhat.com/show_bug.cgi?id=1186757, it was
acknowledged that a just middle between deleting all the locales and
keeping them all was to only keep "supported" locales. There doesn't
seem to be such a list for Fedora, though RHEL does have one. So for
now, we piggyback on the RHEL definition as to what "supported" means.

As of right now, this increases the on-disk size of an F25 tree by 17M.
Its compressed size should be even more insignificant.

I really don't like increasing the size of the image but I can see from the BZ this has been causing pain. I would vote that we wait on making this change until after we get the first release for F25 out the door.

I was testing this by porting it to CentOS, and encountered a few bugs.

Basically, while your test:

env LANG=fr_FR.UTF-8 date

is fine, my test is:

env LANG=fr_FR.UTF-8 touch /bacon

which wasn't translated, because the coreutils translations are stored in /usr/share/locale/fr, which didn't match your fr_FR pattern.

Looking at this a bit more, I realized what we really wanted was to use the install-langs as exposed by rpm-ostree -> rpm, except that was broken: https://github.com/projectatomic/rpm-ostree/pull/525

Anyways, I took this and adapted it for CentOS:

https://github.com/CentOS/sig-atomic-buildscripts/pull/179

Let's treat that PR as canonical and side-port to Fedora if you agree.

rebased

7 years ago

Thanks, I updated the PR with the CentOS version. Though I suppose we have to wait until a newer rpm-ostree before merging this.

Ok, I pushed: https://pagure.io/fork/walters/fedora-atomic/c/3d95646f41f238c7400fc9380d53a0e0dd72b793?branch=f25-locales
which drops the bits that glibc is already doing from treecompose-post.sh - basically we now only use RPM's _install_langs.

This also semantically changes things to retain translations, so the effect is larger. Basically we add ~58MB, from 741MB to 799MB.

(For reference, keeping all language support would bring us to ~935 MB)

@walters Could you open a PR with that commit? (And definitely reset authorship, there are no lines in common anymore with this original PR :)).

I'll close this one.

Pull-Request has been closed by jlebon

7 years ago

Oh I see, the F25 rpm-ostree doesn't have the instlang backport yet.