#9 FlatpakBuilder: Add xa.metadata as ostree commit metadata for runtimes
Merged 3 years ago by otaylor. Opened 3 years ago by kalev.
kalev/flatpak-module-tools runtime-add-xa-metadata  into  master

@@ -678,6 +678,9 @@ 

          # Run flatpak-build-finish to add extra metadata, based on finish-args

          self._build_finish(builddir)

  

+         with open(os.path.join(builddir, 'metadata'), 'r') as f:

+             metadata = f.read()

I checked the full set of metadata for all Flatpaks and runtimes on my system and they are all ascii-only (and almost everything in the metadata looks inherently ascii only) - so this is probably fine - for Python2, it will give a byte string, and for Python3, it will interpret the data according to the locale's encoding.

+ 

          runtime_ref = 'runtime/{id}/{arch}/{branch}'.format(**args)

  

          subprocess.check_call(['ostree', 'commit',
@@ -685,6 +688,7 @@ 

                                 '--owner-gid=0', '--no-xattrs',

                                 '--canonical-permissions',

                                 '--branch', runtime_ref,

+                                '--add-metadata-string', 'xa.metadata=' + metadata,

And in Python3, this is going to reencode according to the locale encoding - which is what the GLib command line processing expects. So, as long as metadata is UTF-8 and the locale encoding is UTF-8., it will even handle non-ASCII fine, though we only need to do with ASCII.

                                 '-s', 'build of ' + runtime_ref,

                                 '--tree=tar=' + tarred_filesystem,

                                 '--tree=dir=' + builddir])

Newer versions of flatpak (after https://github.com/flatpak/flatpak/commit/116a8b848788c582fb2c8577f1250438b100da25)
verify that the metadata matches ostree commit xa.metadata key. As we
use ostree directly for creating runtimes, we need to add the key
manually here.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1822753

Note that I haven't tested this. Pretty sure there's a syntax error or something hiding as I don't usually write python :) Let me figure out how to test this before merging.

The argument construction look ok to me, assuming nothing in python unescapes anything.
However, why read the file when we just wrote it?

Oh, i see, build-finish appends to it.

Yes, exactly, we need to call out to build-finish first to get the final metadata file.

rebased onto 1197302

3 years ago

OK, the patch did have a syntax error which I've now fixed. Seems to be working in my testing; at least I got the metadata back with:

` ``
$ ostree show --repo . --print-metadata-key=xa.metadata runtime/org.fedoraproject.Platform/x86_64/f32
'[Runtime]\nname=org.fedoraproject.Platform\nruntime=org.fedoraproject.Platform/x86_64/f32\nsdk=org.fedoraproject.Sdk/x86_64/f32\n\n[Environment]\nGI_TYPELIB_PATH=/app/lib64/girepository-1.0\nGST_PLUGIN_SYSTEM_PATH=/app/lib64/gstreamer-1.0:/usr/lib64/gstreamer-1.0\nLD_LIBRARY_PATH=/app/lib64\nXDG_DATA_DIRS=/app/share:/usr/share:/usr/share/runtime/share:/run/host/share\n'
```

... which seems right to me.

@jkonecny Can you test installing http://kalev.fedorapeople.org/flatpak-runtime-f32-3220200414150125.oci.tar.gz with rawhide flatpak and see if it fixes https://bugzilla.redhat.com/show_bug.cgi?id=1822753 ?

I've tested this on installed Rawhide machine and in a Rawhide mock.

Both I'm getting this result:

[anaconda@dhcp54 ~]$ flatpak-module install http://kalev.fedorapeople.org/flatpak-runtime-f32-3220200414150125.oci.tar.gz
2020-04-22 12:16:42,623 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2020-04-22 12:16:42,807 - MainThread - MBS.utils.greenwave - WARNING - Greenwave is not configured or configured improperly
INSTALLING
==========
running: ostree init --mode=archive-z2 --repo /home/anaconda/.local/share/flatpak-module-tools/repo
running: flatpak build-update-repo /home/anaconda/.local/share/flatpak-module-tools/repo
Updating appstream branch
Updating summary
running: flatpak remote-add --user --no-gpg-verify flatpak-module-tools /home/anaconda/.local/share/flatpak-module-tools/repo

Note that the directories

'/var/lib/flatpak/exports/share'
'/home/anaconda/.local/share/flatpak/exports/share'

are not in the search path set by the XDG_DATA_DIRS environment variable, so
applications installed by Flatpak may not appear on your desktop until the
session is restarted.

Downloading image  [####################################]  100%
running: tar xfa /tmp/tmpn8nobjv3/flatpak-runtime-f32-3220200414150125.oci.tar.gz
running: flatpak build-import-bundle --update-appstream --oci /home/anaconda/.local/share/flatpak-module-tools/repo /tmp/tmpn8nobjv3/oci
error: Wrong type for string member release
error: flatpak failed (exit status=1)

Based on discussion with @kalev it seems to be problem with how the runtime is build not with the fix.

We've decided to test this in Rawhide directly because the runtime is already broken and it will be much, much easier to test it there.

I checked the full set of metadata for all Flatpaks and runtimes on my system and they are all ascii-only (and almost everything in the metadata looks inherently ascii only) - so this is probably fine - for Python2, it will give a byte string, and for Python3, it will interpret the data according to the locale's encoding.

And in Python3, this is going to reencode according to the locale encoding - which is what the GLib command line processing expects. So, as long as metadata is UTF-8 and the locale encoding is UTF-8., it will even handle non-ASCII fine, though we only need to do with ASCII.

Pull-Request has been merged by otaylor

3 years ago
Metadata