99f960b po: minimize & canonicalize translations stored in git

Authored and Committed by berrange 5 years ago
    po: minimize & canonicalize translations stored in git
    
    Similar to the virt-viewer.pot, .po files contain line numbers and file
    names identifying where in the source a translatable string comes from.
    The source locations in the .po files are thrown away and replaced with
    content from the virt-viewer.pot whenever msgmerge is run, so this is not
    precious information that needs to be stored in git.
    
    When msgmerge processes a .po file, it will add in any msgids from the
    virt-viewer.pot that were not already present. Thus, if a particular msgid
    currently has no translation, it can be considered redundant and again
    does not need storing in git.
    
    When msgmerge processes a .po file and can't find an exact existing
    translation match, it will try todo fuzzy matching instead, marking such
    entries with a "# fuzzy" comment to alert the translator to take a
    look and either discard, edit or accept the match. Looking at the
    existing fuzzy matches in .po files shows that the quality is awful,
    with many having a completely different set of printf format specifiers
    between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
    generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
    entries could be useful to translators if they were working on the .po
    files directly from git, but Virt-Viewer outsourced translation to the
    Fedora Zanata system, so keeping fuzzy matches in git is not much help.
    
    Finally, by default msgids are sorted based on source location. Thus, if
    a bit of code with translatable text is moved from one file to another,
    it may shift around in the .po file, despite the msgid not itself changing.
    If the msgids were sorted alphabetically, the .po files would have
    stable ordering when code is refactored.
    
    This patch takes advantage of the above observations to canonicalize
    and minimize the content stored for .po files in git. Instead of storing
    the real .po files, we now store .mini.po files.
    
    The .mini.po files are the same file format as .po files, but have no
    source location comments, are sorted alphabetically, and all fuzzy
    msgstrs and msgids with no translation are discarded. This cuts the size
    of content in the po directory.
    
    Users working from a virt-viewer git checkout who need the full .po files
    can run "make update-po", which merges the virt-viewer.pot and .mini.po
    file to create a .po file containing all the content previously stored
    in git.
    
    Conversely if a full .po file has been modified, for example, by
    downloading new content from Zanata, the .mini.po files can be updated
    by running "make update-mini-po". The resulting diffs of the .mini.po
    file will clearly show the changed translations without any of the noise
    that previously obscured content. Being able to see content changes
    clearly actually identified a bug in the zanata python client where it
    was adding bogus "fuzzy" annotations to many messages:
    
      https://bugzilla.redhat.com/show_bug.cgi?id=1564497
    
    Users working from virt-viewer releases should not see any difference in
    behaviour, since the tarballs only contain the full .po files, not the
    .mini.po files.
    
    As an added benefit, generating tarballs with "make dist", will no
    longer cause creation of dirty files in git, since it won't touch the
    .mini.po files, only the .po files which are no longer kept in git.
    
    The languages are minimized in the following commit since it is a
    large mechanical process.
    
    Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
    
        
file modified
+16 -14
file modified
+48 -10